openvibers/openvibe.bench Commits

live.clip: a submitter may only submit for a recording it is granted (#111)

OP OpenVibers committed 10 days ago · 2facca34a7 · 6 files, +191 −4

* live.clip: a submitter may only submit for a recording it is granted

A clip names the recording it was cut from, so a service that may submit clips for one recording must not be able
to submit a clip for another: the payload would carry a recording the submitter never held, and the round would
compare a clip of a stranger's stream against one of its own.

The grant is the service token's `ns` claim, the same namespace list OpenVibe.Media reads for its own objects: the
namespace of a recording is `rec:<recording_id>`. A token holding `ns: ["rec:vod_*"]` may submit for the recordings
whose id starts with `vod_`; `rec:*` (or `*`) is every recording. A token that names no namespace is authorised for
none: the check fails closed, because a token without a namespace grant says nothing about which recordings its
holder may use.

server/bench/clip-source.js decides it; server/store.js applies it when a live.clip entry is stored (a clip and an
abstention alike, both of which name their recording); server/http/api.js passes the caller's token claims on.

* Merge branch 'main' into feature/bench-live-clip

Changes

ddiff --git a/docs/api.md b/docs/api.md
iindex ab3ce21..50738ba 100644
---- a/docs/api.md
++++ b/docs/api.md
@@@ -95,6 +95,48 @@ down in `bench_daily_rounds`.
| What the day's rounds may cost | `BENCH_DAILY_BUDGET_USD` | 1.00 |
| What one round is assumed to cost | `BENCH_DAILY_ROUND_ESTIMATE_USD` | 0.05 |
++## live.clip: OpenVibe.Live's AI clip picks
++
++`live.clip` is a **private** kind: Live submits the clips its picker made (or chose not to make) and Bench compares
++them blind, against another policy's pick for the same recording. Bench never makes clips itself, and the kind never
++appears in the public kind list, a board, the gallery, the open dataset or the vote dump.
++
++Only Live's service token submits to it (`service:live`), and only for the recordings that token is granted.
++
++| Route | Who | What |
++|---|---|---|
++| `POST /api/v1/kinds/live.clip/tasks` | Live's service token | A task for one recording. It is never public. |
++| `POST /api/v1/tasks/:id/rounds` | Live's service token | The round. It spends no AI and takes none of the site's daily round budget. |
++| `POST /api/v1/rounds/:id/entries` | Live's service token | One clip, or one abstention. |
++| `GET /api/v1/kinds/live.clip/policies` | Live's service token | What each picker did: entries, abstentions, cost, wins, ties, losses, both-bad votes and the accept rate. |
++
++An entry's payload is typed, and only the typed fields are kept:
++
++```json
++{
++ "source": { "recording_id": "vod_5021", "start_ms": 60000, "end_ms": 90000, "language": "ja", "genre": "irl" },
++ "playback": { "url": "https://openvibe.media/c/1234?exp=…&sig=…", "expires_at": "2026-11-03T12:00:00.000Z" },
++ "policy": { "selector": "auto-clip", "version": "2026-11-02", "models": ["qwen3-vl-4b"] },
++ "title": "The cat walks onto the keyboard",
++ "cost_usd": 0.002
++}
++```
++
++`end_ms` must be after `start_ms`, the clip 5 to 180 seconds long, `playback.url` an `https` address and
++`playback.expires_at` a time still to come. The `policy` is pinned on the entry when it is stored and is never
++rewritten. An **abstention** is a valid entry — the picker saw nothing worth clipping — and carries no `playback`, a
++`reason` instead, and `abstained: true`.
++
++**Which recordings a service may submit for.** The grant is the service token's `ns` claim, the same namespace list
++OpenVibe.Media reads: the namespace of a recording is `rec:<recording_id>`. A token holding `ns: ["rec:vod_*"]` may
++submit for the recordings whose id starts with `vod_`; `rec:*` (or `*`) is every recording. A token that names no
++namespace is authorised for **none**: the check fails closed, so an entry for a recording the submitter was never
++granted is refused with `bench.clip_source_denied` and nothing is stored.
++
++**The playback is for a voter only.** It is never in a round read, only in the pair a voter is given
++(`GET /api/v1/judgements/next?kind=live.clip`) and on the battle page, and never once `expires_at` has passed: an
++expired clip is shown as expired and its link is not served.
++
## The older API
Bench also serves kinds, tasks, rounds, entries, judgements, the boards, the models, the open dataset and the private
ddiff --git a/server/bench/clip-source.js b/server/bench/clip-source.js
nnew file mode 100644
iindex 0000000..1f30a3c
---- /dev/null
++++ b/server/bench/clip-source.js
@@@ -0,0 +1,57 @@
++'use strict';
++
++/**
++ * Which recording a live.clip submitter is allowed to submit for (item 151, C19).
++ *
++ * A clip names the recording it was cut from, so a service that may submit clips for one recording must not be able
++ * to submit a clip for another: the payload would then carry a recording the submitter never held, and the round
++ * would compare a clip of a stranger's stream against one of its own.
++ *
++ * The grant lives in the service token's `ns` claim, the same namespace list OpenVibe.Media reads for its own
++ * objects (`capabilities.namespaceAllowed`): the namespace of a recording is `rec:<recording_id>`. A token for Live's
++ * evaluation run holds `ns: ['rec:vod_*']` and may submit for exactly the recordings that namespace covers; `rec:*`
++ * (or `*`) is every recording.
++ *
++ * A token that names no namespace is authorised for none. The check fails closed on purpose: a token with no
++ * namespace grant says nothing about which recordings its holder may use, and the cost of guessing wrong here is a
++ * clip of somebody else's stream sitting in a round that compares it against Live's own.
++ *
++ * sourceAllowed(claims, recordingId) → { allowed: true } | { allowed: false, code, reason }
++ * namespaceOf(recordingId) → 'rec:<recording_id>'
++ */
++
++/** The namespace a recording is addressed by in a token's `ns` grant. */
++function namespaceOf(recordingId) {
++ const id = String(recordingId == null ? '' : recordingId).trim();
++ return id ? `rec:${id}` : '';
++}
++
++/** The namespaces a token holds; a claim without a usable `ns` list holds none. */
++function grantedNamespaces(claims) {
++ const ns = claims && Array.isArray(claims.ns) ? claims.ns : [];
++ return ns.map((n) => String(n)).filter(Boolean);
++}
++
++/**
++ * May the holder of `claims` submit a clip cut from `recordingId`?
++ *
++ * The rules, in order: a namespace exactly naming the recording allows it; a wildcard namespace covering it
++ * (`rec:*`, `*`, or `rec:vod_*` for the recordings it names) allows it; anything else is refused.
++ */
++function sourceAllowed(claims, recordingId) {
++ const ns = namespaceOf(recordingId);
++ if (!ns) return { allowed: false, code: 'bench.clip_source_missing', reason: 'A clip names the recording it was cut from.' };
++ const granted = grantedNamespaces(claims);
++ for (const g of granted) {
++ if (g === ns || g === '*') return { allowed: true };
++ // A prefix wildcard: 'rec:*' covers every recording, 'rec:vod_*' covers the recordings it names.
++ if (g.endsWith('*') && ns.startsWith(g.slice(0, -1))) return { allowed: true };
++ }
++ return {
++ allowed: false,
++ code: 'bench.clip_source_denied',
++ reason: `This token is granted no recording, so not ${ns}.`,
++ };
++}
++
++module.exports = { sourceAllowed, namespaceOf, grantedNamespaces };
ddiff --git a/server/http/api.js b/server/http/api.js
iindex 1fd6ee1..83d6f53 100644
---- a/server/http/api.js
++++ b/server/http/api.js
@@@ -181,6 +181,8 @@ function createApi(ctx) {
error: b.error,
uiCheck: b.ui_check,
runId: b.run_id,
++ // A live.clip names the recording it was cut from; the submitter's token says which ones it may.
++ claims: (req.principal && req.principal.claims) || null,
});
res.set('Location', `/api/v1/rounds/${String(req.params.id)}`);
return res.status(201).json({ entry });
ddiff --git a/server/store.js b/server/store.js
iindex 4c8ea4b..9381d3a 100644
---- a/server/store.js
++++ b/server/store.js
@@@ -10,6 +10,7 @@ const crypto = require('crypto');
const { ids } = require('openvibe-contracts');
const { KINDS, ALL_KINDS, PRIVATE_KIND_IDS, JUDGES, WINNERS, kindById, isPrivateKind } = require('./kinds');
const clipLib = require('./bench/clip');
++const clipSource = require('./bench/clip-source');
const ratingLib = require('./rating');
const styleLib = require('./rating/style');
@@@ -317,7 +318,7 @@ function createStore(s, { log = console, random = Math.random, config = null } =
}
/** Insert an entry into a round; moves a room-submitted round to ready when it has two usable entries. */
-- async function addEntry({ roundId, provider, model, route, payload, costUsd = 0, ms = 0, tokens = 0, error = null, uiCheck = null, runId = null }) {
++ async function addEntry({ roundId, provider, model, route, payload, costUsd = 0, ms = 0, tokens = 0, error = null, uiCheck = null, runId = null, claims = null }) {
return await s.tx(async () => {
const r = await db.maybe('SELECT * FROM bench_rounds WHERE id = $1 FOR UPDATE', [String(roundId || '')]);
if (!r) throw new BenchError(404, 'bench.round_not_found', 'No such Bench round.');
@@@ -326,6 +327,10 @@ function createStore(s, { log = console, random = Math.random, config = null } =
const v = clipLib.validateEntry(payload, { now: s.now() });
if (!v.ok) throw new BenchError(422, 'bench.invalid_clip', v.message);
payload = v.payload;
++ // The submitter must be authorised for the recording the clip was cut from: a token that may submit
++ // for one recording may not name another. The check fails closed (no grant, no recording).
++ const src = clipSource.sourceAllowed(claims, payload.source.recording_id);
++ if (!src.allowed) throw new BenchError(403, src.code, src.reason);
route = { policy: v.payload.policy };
model = model || v.payload.policy.models[0] || `${v.payload.policy.selector}@${v.payload.policy.version}`;
provider = provider || 'live';
ddiff --git a/test/clip-source.test.js b/test/clip-source.test.js
nnew file mode 100644
iindex 0000000..ade50be
---- /dev/null
++++ b/test/clip-source.test.js
@@@ -0,0 +1,54 @@
++'use strict';
++
++/**
++ * Which recording a live.clip submitter may submit for (item 151, C19). The grant is the service token's `ns`
++ * claim; the namespace of a recording is `rec:<recording_id>`.
++ */
++const test = require('node:test');
++const assert = require('node:assert/strict');
++const { sourceAllowed, namespaceOf, grantedNamespaces } = require('../server/bench/clip-source');
++
++test('a recording is addressed by its own namespace', () => {
++ assert.equal(namespaceOf('vod_5021'), 'rec:vod_5021');
++ assert.equal(namespaceOf(' vod_5021 '), 'rec:vod_5021');
++ assert.equal(namespaceOf(''), '');
++ assert.equal(namespaceOf(null), '');
++});
++
++test('a token holds no namespace when its claim names none', () => {
++ assert.deepEqual(grantedNamespaces({ ns: ['rec:vod_*'] }), ['rec:vod_*']);
++ assert.deepEqual(grantedNamespaces({ ns: [] }), []);
++ assert.deepEqual(grantedNamespaces({}), []);
++ assert.deepEqual(grantedNamespaces(null), []);
++ assert.deepEqual(grantedNamespaces({ ns: 'rec:vod_*' }), []);
++});
++
++test('an exact namespace allows its recording and no other', () => {
++ const claims = { ns: ['rec:vod_5021'] };
++ assert.equal(sourceAllowed(claims, 'vod_5021').allowed, true);
++ const denied = sourceAllowed(claims, 'vod_5022');
++ assert.equal(denied.allowed, false);
++ assert.equal(denied.code, 'bench.clip_source_denied');
++});
++
++test('a wildcard namespace allows the recordings it covers', () => {
++ assert.equal(sourceAllowed({ ns: ['rec:vod_*'] }, 'vod_5021').allowed, true);
++ assert.equal(sourceAllowed({ ns: ['rec:vod_*'] }, 'vod_5022').allowed, true);
++ assert.equal(sourceAllowed({ ns: ['rec:vod_*'] }, 'clip_99').allowed, false);
++ assert.equal(sourceAllowed({ ns: ['rec:*'] }, 'anything_at_all').allowed, true);
++ assert.equal(sourceAllowed({ ns: ['*'] }, 'anything_at_all').allowed, true);
++});
++
++test('a token granted no recording is authorised for none', () => {
++ for (const claims of [{ ns: [] }, {}, null, { ns: ['other:thing'] }]) {
++ const r = sourceAllowed(claims, 'vod_5021');
++ assert.equal(r.allowed, false);
++ assert.equal(r.code, 'bench.clip_source_denied');
++ }
++});
++
++test('a clip that names no recording is refused', () => {
++ const r = sourceAllowed({ ns: ['rec:*'] }, '');
++ assert.equal(r.allowed, false);
++ assert.equal(r.code, 'bench.clip_source_missing');
++});
ddiff --git a/test/live-clip.test.js b/test/live-clip.test.js
iindex 70fcc06..6f37f79 100644
---- a/test/live-clip.test.js
++++ b/test/live-clip.test.js
@@@ -13,8 +13,11 @@ const { boot, check, done } = require('./helpers/boot');
const t = await boot({ env: { BENCH_ROUNDS_PER_DAY: '100', BENCH_ROUNDS_DAILY: '1000' } });
const voter = t.network.addUser('voter');
const voter2 = t.network.addUser('voter2');
-- const live = t.network.serviceToken({ sub: 'svc:live', audience: 'openvibe.bench', cap: ['bench.task.create', 'bench.round.create', 'bench.entry.create'] });
-- const other = t.network.serviceToken({ sub: 'svc:room', audience: 'openvibe.bench', cap: ['bench.task.create', 'bench.round.create', 'bench.entry.create'] });
++ const CAPS = ['bench.task.create', 'bench.round.create', 'bench.entry.create'];
++ // Live's token is granted the recordings it may cut clips from (the `ns` claim): the namespace of a recording
++ // is `rec:<recording_id>`, and a wildcard covers the recordings it names.
++ const live = t.network.serviceToken({ sub: 'svc:live', audience: 'openvibe.bench', cap: CAPS, ns: ['rec:vod_*'] });
++ const other = t.network.serviceToken({ sub: 'svc:room', audience: 'openvibe.bench', cap: CAPS, ns: ['rec:vod_*'] });
const soon = () => new Date(Date.now() + 3600_000).toISOString();
const clip = (version, extra = {}) => ({
source: { recording_id: 'vod_5021', start_ms: 60_000, end_ms: 90_000, language: 'ja', genre: 'irl' },
@@@ -66,6 +69,30 @@ const { boot, check, done } = require('./helpers/boot');
assert.strictEqual(row.model, 'qwen3-vl-4b');
});
++ await check('a submitter may only submit for a recording its token is granted', async () => {
++ // Live's token holds `rec:vod_*`: the recordings whose id starts with `vod_`. A clip of another recording
++ // (one this token was never granted) is refused, and so is a token that names no recording at all.
++ const r = await newRound();
++ const foreign = await t.get(`/api/v1/rounds/${r.id}/entries`, { bearer: live, json: { payload: clip('v1', { source: { recording_id: 'clip_99', start_ms: 60_000, end_ms: 90_000 } }) } });
++ assert.strictEqual(foreign.status, 403, foreign.text);
++ assert.strictEqual(foreign.json().code, 'bench.clip_source_denied');
++ const none = t.network.serviceToken({ sub: 'svc:live', audience: 'openvibe.bench', cap: CAPS, ns: [] });
++ const noNs = await t.get(`/api/v1/rounds/${r.id}/entries`, { bearer: none, json: { payload: clip('v1') } });
++ assert.strictEqual(noNs.status, 403, 'a token granted no recording is authorised for none');
++ assert.strictEqual(noNs.json().code, 'bench.clip_source_denied');
++ const one = t.network.serviceToken({ sub: 'svc:live', audience: 'openvibe.bench', cap: CAPS, ns: ['rec:vod_5021'] });
++ const exact = await t.get(`/api/v1/rounds/${r.id}/entries`, { bearer: one, json: { payload: clip('v1') } });
++ assert.strictEqual(exact.status, 201, exact.text);
++ const otherRec = await t.get(`/api/v1/rounds/${r.id}/entries`, { bearer: one, json: { payload: clip('v1', { source: { recording_id: 'vod_5022', start_ms: 60_000, end_ms: 90_000 } }) } });
++ assert.strictEqual(otherRec.status, 403, 'granted one recording, not another');
++ // An abstention names its recording too, so it is refused the same way.
++ const abstain = await t.get(`/api/v1/rounds/${r.id}/entries`, { bearer: one, json: { payload: { source: { recording_id: 'vod_5022', start_ms: 0, end_ms: 1000 }, policy: { selector: 'moment-picker', version: 'v1', models: [] }, abstained: true, reason: 'Nothing worth clipping here.' } } });
++ assert.strictEqual(abstain.status, 403, 'an abstention of another recording is refused as well');
++ // The refused entries left nothing behind.
++ const rows = await t.ctx.s.db.many('SELECT id FROM bench_entries WHERE round_id = $1', [r.id]);
++ assert.strictEqual(rows.length, 1, 'only the authorised entry was stored');
++ });
++
await check('the playback is for a voter only: not in a round read, not after it expires', async () => {
const anon = await t.get(`/api/v1/rounds/${roundId}`);
assert.strictEqual(anon.status, 404, 'a private round is not readable by just anyone');
@@@ -146,7 +173,7 @@ const { boot, check, done } = require('./helpers/boot');
await check('a live.clip round spends no AI: it skips the daily round limits and counts against none', async () => {
const quota = await boot({ env: { BENCH_ROUNDS_PER_DAY: '1', BENCH_ROUNDS_DAILY: '1' } });
-- const liveSvc = quota.network.serviceToken({ sub: 'svc:live', audience: 'openvibe.bench', cap: ['bench.task.create', 'bench.round.create', 'bench.entry.create'] });
++ const liveSvc = quota.network.serviceToken({ sub: 'svc:live', audience: 'openvibe.bench', cap: CAPS, ns: ['rec:vod_*'] });
for (let i = 0; i < 4; i++) {
const task = await quota.get('/api/v1/kinds/live.clip/tasks', { bearer: liveSvc, json: { prompt: `stretch ${i}` } });
const rr = await quota.get(`/api/v1/tasks/${task.json().task.id}/rounds`, { bearer: liveSvc, json: {} });