openvibers/openvibe.media Commits

Audio description track on Live watch pages (X1) (#335)

OP OpenVibers committed 6 days ago · 872c2e7083 · 6 files, +249 −1

* Audio description track on Live watch pages (X1)

* Classify the descriptions fetch in the SSRF inventory

* A private track needs the signed-in owner, not a playback signature

* Name the shared release that carries the AD control

Changes

ddiff --git a/CHANGELOG.md b/CHANGELOG.md
iindex 9e925a3..cfe3af4 100644
---- a/CHANGELOG.md
++++ b/CHANGELOG.md
@@@ -2,6 +2,14 @@
## Unreleased
++- **Audio description on a watch page (X1, consent plan 2026-10-31, "AI and data")**: the player now carries a
++ `<track kind="descriptions">` for Live's VODs and clips, served same-origin at `/v/:id/descriptions.vtt` (and
++ `/c/:id/descriptions.vtt`) from the WebVTT Live builds out of its scene notes and speech timeline. Public and link
++ items are cacheable; a private item's track answers the same 404 as a missing id unless the viewer's Network
++ session asks for it (that session is forwarded to Live, which checks the owner). Other apps' media is not described: only Live has the scene-note timeline behind it. The **AD** control and
++ its spoken cues are OpenVibe.Shared's player (openvibe-shared 4.126.0, a pin-round follow-up). Tests:
++ `test/audio-description.test.js` (public, private-owner, other app; track markup).
++
- An avatar a person's profile points at cannot be deleted out from under them (incident 2026-11-06: a cleanup of
public screenshot copies deleted eleven avatars). `media_avatar_refs` (migration `0020_avatar_refs.sql`) records
each profile slot → object; `model.softDelete` refuses an avatar with an unreleased reference and `DELETE` answers
ddiff --git a/server/public/descriptions.js b/server/public/descriptions.js
nnew file mode 100644
iindex 0000000..51f705f
---- /dev/null
++++ b/server/public/descriptions.js
@@@ -0,0 +1,108 @@
++/**
++ * OpenVibe.Media — the audio-description track for a watch page (X1, consent plan 2026-10-31, "AI and data").
++ *
++ * Live owns the scene notes and the speech timeline, and serves the WebVTT at
++ * /api/vods/:id/descriptions.vtt and /api/clips/:id/descriptions.vtt. This module puts the track on the same
++ * origin as the page (`/v/:id/descriptions.vtt`) so the shared player can load it without cross-origin track
++ * rules, and so Media applies its own visibility rule before asking:
++ *
++ * public / unlisted anyone may read it (the page and its track are cacheable);
++ * private only when the viewer is signed in to this site: that Network session cookie is forwarded
++ * to Live, the one service that can check this person owns the item. Nothing private is cached.
++ *
++ * Only Live's media is described: other apps have no scene-note timeline behind them. A missing item, another
++ * app's item, a private item the caller cannot prove, Live being down, or no notes to describe all answer the
++ * same 404 as a missing media id.
++ */
++'use strict';
++
++const db = require('../db/database');
++const frames = require('../thumbnails/live-frame-service');
++
++const CACHE_TTL_MS = 60_000;
++const _cache = new Map(); // "vod:100" → { at, vtt }
++const _inFlight = new Map();
++
++function notFound(res) {
++ return res.status(404).json({ error: 'Not found' });
++}
++
++function recordVisibility(record) {
++ return record.visibility || (record.is_public ? 'public' : 'private');
++}
++
++/**
++ * A private item's caller: the site's own Network session, forwarded to Live, which is the service that knows
++ * whether this person owns the item. An app key or a signed playback URL proves playback to Media (serveMediaRecord
++ * uses them), but it does not prove the owner to Live, so it cannot ask Live for the private description.
++ */
++function mayAskPrivate(record, req, kind) {
++ const token = req.cookies && req.cookies.ov_token;
++ return typeof token === 'string' && token.length > 0 && token.length < 4096;
++}
++
++async function fromLive(kind, id, req) {
++ const base = frames.APP_INTERNAL_URLS.live;
++ if (!base) return null;
++ const path = `${String(base).replace(/\/+$/, '')}/api/${kind === 'clip' ? 'clips' : 'vods'}/${encodeURIComponent(id)}/descriptions.vtt`;
++ const headers = {};
++ const token = req.cookies && req.cookies.ov_token;
++ if (typeof token === 'string' && token.length > 0 && token.length < 4096) headers.Authorization = `Bearer ${token}`;
++ try {
++ const res = await fetch(path, { headers, signal: AbortSignal.timeout(5000) });
++ if (!res.ok) return null;
++ const vtt = await res.text();
++ return /^WEBVTT\r?\n/.test(vtt) ? vtt : null;
++ } catch (err) {
++ console.warn('[Descriptions] Live fetch failed:', err.message);
++ return null;
++ }
++}
++
++async function handle(kind, req, res) {
++ const id = Number(req.params.id);
++ if (!Number.isInteger(id) || id <= 0) return notFound(res);
++ const record = kind === 'clip' ? await db.getClipById(id) : await db.getVodById(id);
++ if (!record) return notFound(res);
++ const visibility = recordVisibility(record);
++ const isPrivate = visibility === 'private';
++ if (isPrivate && !mayAskPrivate(record, req, kind)) return notFound(res);
++ if ((record.app_id || 'live') !== 'live') return notFound(res);
++
++ const cacheKey = `${kind}:${id}`;
++ const mayUseCache = !isPrivate && !(req.cookies && req.cookies.ov_token);
++ if (mayUseCache) {
++ const hit = _cache.get(cacheKey);
++ if (hit && Date.now() - hit.at < CACHE_TTL_MS) return send(res, hit.vtt, false);
++ if (_inFlight.has(cacheKey)) {
++ const vtt = await _inFlight.get(cacheKey);
++ return vtt ? send(res, vtt, false) : notFound(res);
++ }
++ }
++ const pending = fromLive(kind, id, req);
++ if (mayUseCache) {
++ _inFlight.set(cacheKey, pending);
++ pending.finally(() => _inFlight.delete(cacheKey)).catch(() => {});
++ }
++ const vtt = await pending;
++ if (!vtt) return notFound(res);
++ if (mayUseCache) {
++ if (_cache.size > 500) {
++ const cutoff = Date.now() - CACHE_TTL_MS;
++ for (const [k, v] of _cache) { if (v.at < cutoff) _cache.delete(k); }
++ }
++ _cache.set(cacheKey, { at: Date.now(), vtt });
++ }
++ return send(res, vtt, isPrivate);
++}
++
++function send(res, vtt, isPrivate) {
++ res.set({
++ 'Content-Type': 'text/vtt; charset=utf-8',
++ 'Cache-Control': isPrivate ? 'private, no-store' : 'public, max-age=60',
++ 'X-Content-Type-Options': 'nosniff',
++ });
++ return res.send(vtt);
++}
++
++module.exports = { handle, mayAskPrivate, CACHE_TTL_MS };
ddiff --git a/server/public/pages.js b/server/public/pages.js
iindex b41c5fc..874e3e1 100644
---- a/server/public/pages.js
++++ b/server/public/pages.js
@@@ -182,13 +182,18 @@ function renderWatchPage(kind, record, readiness = null) {
by ? (ownerUrl ? `by <a href="${esc(ownerUrl)}">${esc(by)}</a>` : `by ${esc(by)}`) : null,
].filter(Boolean);
++ // Live owns the scene notes behind audio description (X1): every Live watch page offers the track, same
++ // origin; a private item's track only loads for someone who can prove they may see it (descriptions.js).
++ const descriptionTrack = ownedByLive
++ ? `\n <track kind="descriptions" srclang="en" label="Audio description" src="${esc(`${self}/descriptions.vtt`)}">`
++ : '';
const [stateTitle, stateText] = playable ? [null, null] : unavailableText(kind, readiness.reason);
// No preload: a long recording is a fragmented MP4 whose moov has no duration, so Chrome keeps range-reading the
// whole file to find one, never reaches HAVE_METADATA, and the window's load event waits for it (v/6170: 30 s+
// timeout). The poster is the first frame, and pressing play still fetches the bytes; the page itself must load.
const player = playable
? `<div class="player">
-- <video data-ov-player controls playsinline preload="none"${thumb ? ` poster="${esc(thumb)}"` : ''} src="${esc(rawUrl)}">
++ <video data-ov-player controls playsinline preload="none"${thumb ? ` poster="${esc(thumb)}"` : ''} src="${esc(rawUrl)}">${descriptionTrack}
Your browser cannot play this video. <a href="${esc(rawUrl)}">Download it</a> instead.
</video>
</div>`
ddiff --git a/server/public/routes.js b/server/public/routes.js
iindex 369750c..a679c8e 100644
---- a/server/public/routes.js
++++ b/server/public/routes.js
@@@ -470,6 +470,21 @@ router.get('/c/:id', optionalIdentity, sharingLib.attachViewer, async (req, res)
}
});
++// ── Audio description track (X1, consent plan 2026-10-31) ───
++// The same-origin WebVTT a watch page's <track kind="descriptions"> loads; Live builds it from its scene
++// notes and speech timeline. Public/unlisted items are cacheable; a private one needs the owning app, a
++// signed playback URL or this site's session (descriptions.js).
++const descriptionsRoute = (kind) => async (req, res) => {
++ try {
++ await require('./descriptions').handle(kind, req, res);
++ } catch (err) {
++ console.error(`[Public] ${kind} descriptions error:`, err.message);
++ if (!res.headersSent) res.status(500).json({ error: 'Failed to load the descriptions' });
++ }
++};
++router.get('/v/:id/descriptions.vtt', optionalIdentity, descriptionsRoute('vod'));
++router.get('/c/:id/descriptions.vtt', optionalIdentity, descriptionsRoute('clip'));
++
// ── Share card (og:image default) ────────────────────────────
router.get('/og-image.png', (req, res) => {
res.set('Cache-Control', 'public, max-age=86400');
ddiff --git a/test/audio-description.test.js b/test/audio-description.test.js
nnew file mode 100644
iindex 0000000..47eeead
---- /dev/null
++++ b/test/audio-description.test.js
@@@ -0,0 +1,111 @@
++'use strict';
++/**
++ * Audio description on the watch page (X1, consent plan 2026-10-31, "AI and data"): Media serves Live's
++ * WebVTT as a same-origin `/v/:id/descriptions.vtt` (and `/c/:id/...`), adds the `<track
++ * kind="descriptions">` to Live's watch pages, and never asks Live to describe a private item for someone
++ * who cannot prove they may see it. Live is stubbed in-process: the real one is tested in OpenVibe.Live
++ * (test/audio-description.test.js).
++ *
++ * node test/audio-description.test.js
++ */
++const assert = require('assert');
++const fs = require('fs');
++const os = require('os');
++const path = require('path');
++const http = require('http');
++
++(async () => {
++ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-media-ad-'));
++ process.env.MEDIA_PUBLIC_URL = 'https://media.test';
++ process.env.VOD_PATH = path.join(tmp, 'vods');
++ process.env.CLIPS_PATH = path.join(tmp, 'clips');
++
++ const VTT = 'WEBVTT\n\n00:00:05.000 --> 00:00:08.000\nA person in a red coat opens a blue door.\n';
++ const privateIds = new Set(); // filled once the rows exist; the stub uses it to refuse non-owners
++ const live = http.createServer((req, res) => {
++ const owner = req.headers.authorization === 'Bearer owner-token';
++ const m = /^\/api\/(?:vods|clips)\/(\d+)\//.exec(req.url || '');
++ if (m && privateIds.has(m[1]) && !owner) return res.writeHead(404).end();
++ res.writeHead(200, { 'Content-Type': 'text/vtt; charset=utf-8' });
++ res.end(VTT);
++ });
++ await new Promise((r) => live.listen(0, '127.0.0.1', r));
++ process.env.LIVE_APP_INTERNAL_URL = `http://127.0.0.1:${live.address().port}`;
++
++ const db = require('../server/db/database');
++ const pages = require('../server/public/pages');
++ await db.initDb();
++ const vod = await db.createVod({ app_id: 'live', stream_id: 500, user_id: 3, title: 'Described VOD', visibility: 'public', duration_seconds: 60 });
++ const vodId = Number(vod.lastInsertRowid);
++ const privateVod = await db.createVod({ app_id: 'live', stream_id: 500, user_id: 3, title: 'Private VOD', visibility: 'private', duration_seconds: 60 });
++ const privateVodId = Number(privateVod.lastInsertRowid);
++ const gamesVod = await db.createVod({ app_id: 'games', user_id: 4, title: 'Games VOD', visibility: 'public', duration_seconds: 60 });
++ const gamesVodId = Number(gamesVod.lastInsertRowid);
++ const clip = await db.createClip({ app_id: 'live', vod_id: vodId, stream_id: 500, user_id: 3, title: 'Described clip', visibility: 'public', start_time: 3, end_time: 15, duration_seconds: 12, file_path: '/x/c.webm', status: 'ready' });
++ const clipId = Number(clip.lastInsertRowid);
++ const privateClip = await db.createClip({ app_id: 'live', vod_id: privateVodId, stream_id: 500, user_id: 3, title: 'Private clip', visibility: 'private', start_time: 3, end_time: 15, duration_seconds: 12, file_path: '/x/c2.webm', status: 'ready' });
++ const privateClipId = Number(privateClip.lastInsertRowid);
++ privateIds.add(String(privateVodId));
++ privateIds.add(String(privateClipId));
++
++ const express = require('express');
++ const app = express();
++ app.use((req, _res, next) => {
++ req.cookies = {};
++ for (const part of String(req.headers.cookie || '').split(';')) {
++ const i = part.indexOf('=');
++ if (i < 0) continue;
++ req.cookies[part.slice(0, i).trim()] = decodeURIComponent(part.slice(i + 1).trim());
++ }
++ next();
++ });
++ app.use('/', require('../server/public/routes'));
++ const server = app.listen(0, '127.0.0.1');
++ await new Promise((r) => (server.listening ? r() : server.once('listening', r)));
++ const get = (p, headers = {}) => new Promise((resolve, reject) => {
++ http.get({ host: '127.0.0.1', port: server.address().port, path: p, headers }, (res) => {
++ let body = '';
++ res.setEncoding('utf8');
++ res.on('data', (c) => { body += c; });
++ res.on('end', () => resolve({ status: res.statusCode, headers: res.headers, body }));
++ }).on('error', reject);
++ });
++
++ // ── The track: same origin, the WebVTT Live built, cacheable for a public item ──
++ const pub = await get(`/v/${vodId}/descriptions.vtt`);
++ assert.strictEqual(pub.status, 200);
++ assert.match(pub.headers['content-type'], /^text\/vtt/);
++ assert.match(pub.headers['cache-control'], /public/);
++ assert.ok(pub.body.startsWith('WEBVTT') && /red coat/.test(pub.body), 'Live\'s cues are served as they are');
++ assert.strictEqual((await get(`/c/${clipId}/descriptions.vtt`)).status, 200, 'a clip has its own track URL');
++
++ // ── Private: the same answer as a missing id until its owner proves who they are ──
++ assert.strictEqual((await get(`/v/${privateVodId}/descriptions.vtt`)).status, 404, 'a visitor never gets a private track');
++ const other = await get(`/v/${privateVodId}/descriptions.vtt`, { cookie: 'ov_token=other-token' });
++ assert.strictEqual(other.status, 404, 'another signed-in person does not either');
++ const owner = await get(`/v/${privateVodId}/descriptions.vtt`, { cookie: 'ov_token=owner-token' });
++ assert.strictEqual(owner.status, 200, 'the owner asks and gets it');
++ assert.match(owner.headers['cache-control'], /private/);
++ assert.strictEqual((await get(`/v/${privateVodId}/descriptions.vtt`, { cookie: 'ov_token=owner-token' })).status, 200);
++ assert.strictEqual((await get(`/c/${privateClipId}/descriptions.vtt`)).status, 404);
++ assert.strictEqual((await get(`/c/${privateClipId}/descriptions.vtt`, { cookie: 'ov_token=owner-token' })).status, 200);
++
++ // Another app's media has no Live scene notes to describe.
++ assert.strictEqual((await get(`/v/${gamesVodId}/descriptions.vtt`)).status, 404);
++
++ // ── The watch page: a descriptions track on Live's page, and none on another app's ──
++ const publicRecord = { id: vodId, app_id: 'live', title: 'Described VOD', file_path: '/x/v.mp4', is_public: 1, visibility: 'public', duration_seconds: 60 };
++ const html = pages.renderWatchPage('vod', publicRecord);
++ assert.ok(html.includes(`<track kind="descriptions" srclang="en" label="Audio description" src="https://media.test/v/${vodId}/descriptions.vtt">`), 'the player is handed the descriptions track');
++ const gamesRecord = { id: gamesVodId, app_id: 'games', title: 'Games VOD', file_path: '/x/g.mp4', is_public: 1, visibility: 'public', duration_seconds: 60 };
++ assert.ok(!pages.renderWatchPage('vod', gamesRecord).includes('kind="descriptions"'), 'not for another app\'s video');
++ const privateRecord = { id: privateVodId, app_id: 'live', title: 'Private VOD', file_path: '/x/p.mp4', is_public: 0, visibility: 'private', duration_seconds: 60 };
++ assert.ok(pages.renderWatchPage('vod', privateRecord).includes('kind="descriptions"'), 'the owner\'s page carries it; the route is what guards it');
++
++ // The page's CSS and player are the shared ones; no duplicate player script is added by the track.
++ assert.ok(html.includes('/shared/player.js'), 'the network player reads the track');
++
++ server.close();
++ live.close();
++ console.log('media audio description: all checks passed');
++})().catch((e) => { console.error(e); process.exit(1); });
ddiff --git a/test/security-ssrf.test.js b/test/security-ssrf.test.js
iindex 9d6a7e3..04da1fb 100644
---- a/test/security-ssrf.test.js
++++ b/test/security-ssrf.test.js
@@@ -32,6 +32,7 @@ const https = require('https');
'server/account-data.js': 'Network internal API (OV_NETWORK_INTERNAL_URL), fixed paths: export parts and deletion confirmations',
'server/thumbnails/live-frame-service.js': 'the app\'s internal URL (APP_INTERNAL_URLS), fixed path',
'server/public/dev-data.js': 'the app\'s internal URL (APP_INTERNAL_URLS), ids and encoded names in the path',
++ 'server/public/descriptions.js': 'the app\'s internal URL (APP_INTERNAL_URLS), a numeric media id in a fixed path',
'server/jobs/pack.js': 'a presigned GET on B2/R2 (operator-configured buckets) for a segment\'s durable copy',
'server/objects/routes.js': 'a ranged GET on a presigned B2/R2 URL (operator-configured buckets) for a packed segment\'s chunk',
'server/s3/routes.js': 'a ranged GET on a presigned B2/R2 URL (operator-configured buckets) for an S3 GetObject whose bytes are not on local disk',