openvibers/openvibe.media Commits

Compose Media's pages through openvibe-shared/shell; add /llms-full.txt and IndexNow; pin openvibe-shared v2.10.0 (#41)

OP OpenVibers committed 5 Oct 2026 · 8d41bca5d9 · 22 files, +486 −83

* Compose Media's pages through openvibe-shared/shell; add /llms-full.txt and IndexNow; pin openvibe-shared v2.10.0

## Summary

**Files changed**

- `server/public/page-frame.js` — `page()` now composes the document through `openvibe-shared/shell`'s `page()`; the site-specific bits (navbar init options, footer links, app icon, palette, `og:video`, the small Twitter card) ride in the shell's `head`, and the old local head builder is gone. Verified by rendering watch/paste/`/me`/`/updates` HTML before and after: `<title>`, canonical, robots, all `og:*`/`twitter:*`, app icon and JSON-LD are byte-identical, and the visible body text is identical apart from the shared footer the shell now server-renders.
- `server/config.js` — added `indexnow.key` from `INDEXNOW_KEY`.
- `server/index.js` — `createIndexNow({ host: config.publicUrl, key: config.indexnow.key })`, exposed on `app.locals`, and `app.use(indexnow.keyFile)` when a key is set (mirrors Codes `app.js`).
- `server/public/crawl.js` — added `llmsFullTxt()` + `indexableWatchRows()` and `GET /llms-full.txt` (via `seo.llmsFull`, same indexable set as the sitemap, rendering bounded by a 512 KB cap); exported `llmsFullTxt`; `/llms.txt` now links it.
- `server/public/routes.js` — comment updated for `/llms-full.txt`.
- `README.md` — openvibe-shared pin `v2.5.0` → `v2.10.0` (both places, for `test/pins.test.js`); documented `/llms-full.txt`.
- `STATUS.json` — added `"shared": "openvibe-shared v2.10.0"`.
- `.env.example` — documented `INDEXNOW_KEY`.
- `test/crawl.test.js` — covers `/llms-full.txt` (header, index, each indexable watch page, exclusion of Live/AI/private/unlisted/processing pages, determinism).
- `test/public-pages.test.js` — asserts the shell's eager theme-loader, the shared web runtime, and the server-rendered compact footer.
- `test/shipped-pages.test.js` — asserts the footer's `/updates` link in the SSR markup (the shell no longer emits footer boot JSON).
- `test/security-ssrf.test.js` — classifies IndexNow in the outbound inventory: fixed `api.indexnow.org`, host/key operator-set, wired from `server/index.js`.
- `test/indexnow.test.js` — **new**: boots the server with `INDEXNOW_KEY` and proves `/<key>.txt` is served as `text/plain` while other paths 404.

`package.json` / `package-lock.json` were already at openvibe-shared **v2.10.0** on arrival and were left untouched.

**Tests** (all via `ov test`, no whole suite)
- `ov test test/crawl.test.js test/public-pages.test.js test/perf-budget.test.js test/not-found-csp.test.js test/pins.test.js test/security-ssrf.test.js test/indexnow.test.js test/shipped-pages.test.js test/theme-default.test.js test/asset-cache.test.js test/object-tiering.test.js test/app-tenants.test.js test/drill-mode.test.js test/security-private.test.js` → **11 passed, 3 skipped (unchanged since green)**.
- Plus `test/indexnow.test.js` run alone → passed; `test/perf-budget.test.js` → passed.

**Not done / notes**
- The brief's pin step said v2.5.0 → v2.6.0, but the checkout already pins v2.10.0 (which has `shell`, `seo.pageSummary`, `llmsFull` and `indexnow`) and I was told not to touch the package files, so the README was moved to v2.10.0 instead.
- `server/public/browse.js` (the home index) still hand-writes its document; it wasn't in the named call sites and is the subject of `perf-budget.test.js`, so I left it.
- The shared shell doesn't initialize `OpenVibeFooter` client-side, forward `og:video`, or take a Twitter-card override; the footer is server-rendered (visible content equivalent), and `og:video`/the small card are supplied by `page()` (the card via one documented

* IndexNow pings public ready objects only, a bad key disables it, footer init kept, llms-full memoized, CSP and runtime asserted (review)

## Changes (one line each)

1. **IndexNow pings on ready/visibility transitions** — new `server/indexnow-notify.js`: `pingWatch(kind, row)` gates on `drill.enabled`, `pages.watchIndexable(kind, row)`, and `db.isSandboxTenant(row.app_id)`, then `client.pingSoon([<publicUrl>/v|c/<id>, <publicUrl>/sitemap.xml])`; wired at:
   - `server/vod/finalize.js:406` (VOD `.ready` commit)
   - `server/vod/clip-jobs.js:104` (cut clip ready) and `:164` (virtual clip ready)
   - `server/vod/routes.js:378` (VOD visibility change)
   - `server/vod/clips-routes.js:430` (clip visibility change)
   - `server/index.js:109` `require('./indexnow-notify').use(indexnow)` (the `app.locals.indexnow` client)

2. **Malformed `INDEXNOW_KEY` no longer crashes boot** — `server/index.js:102-107`: `createIndexNow` is wrapped, failure logs `[IndexNow] disabled: …` and falls back to a disabled client (no key file, `pingSoon` no-ops).

3. **Footer client init restored** — `server/public/page-frame.js:157-158` adds `footerBootScript(footer)` calling `OpenVibeFooter.init({service, variant, links, mount:'#ov-footer', updates:'/updates'})` on `DOMContentLoaded` (shell v2.10.0 boots only the navbar), injected into the shell's `head` at `:221`; the hand-written browse path already inits via `frameScripts`.

4. **`/llms-full.txt` memoized ~10 min** — `server/public/crawl.js:24,146,149,174`: single bounded in-memory entry, reused inside the TTL, rebuilt on expiry/restart.

5. **Tests added** — `test/indexnow-ping.test.js` (new: injected `fetch`; public ready VOD + clip ping page+sitemap, private/unlisted/AI/sandbox/missing never do, plus the visibility-change route); `test/indexnow.test.js` (malformed key boots with warning, no key file); `test/public-pages.test.js` (footer `OpenVibeFooter.init` + options/links, page response has no `Content-Security-Policy`, and ships `web-runtime.js`); `test/crawl.test.js` (second `/llms-full.txt` build makes no DB query).

## Test results

```
ov test test/crawl.test.js test/indexnow.test.js test/public-pages.test.js \
        test/pins.test.js test/security-ssrf.test.js test/indexnow-ping.test.js
✓ test/crawl.test.js
✓ test/indexnow-ping.test.js
✓ test/indexnow.test.js
✓ test/public-pages.test.js
✓ test/pins.test.js
✓ test/security-ssrf.test.js
6 passed
```

* The secrets test treats INDEXNOW_KEY as public by protocol (served at /<key>.txt)

Changes

ddiff --git a/.env.example b/.env.example
iindex b68de91..d89c769 100644
---- a/.env.example
++++ b/.env.example
@@@ -117,6 +117,11 @@ MEDIA_CAPABILITY_GATE=report
# Media's outbox with the OAuth client credentials (grant events.event.publish). Unset = off.
#EVENTS_URL=http://127.0.0.1:4300
++# IndexNow (openvibe-shared/indexnow): a key makes search engines recrawl a public page when it
++# appears or changes. 8–128 hex/alphanumeric characters; the key file is served at /<key>.txt.
++# Unset = off, no key file, nothing sent.
++#INDEXNOW_KEY=
++
# ── Object uploads (docs/object-model.md#presigned-uploads, #multipart-uploads) ──
# MEDIA_SIGNING_SECRET signs presigned upload/download URLs; set it in production.
#MEDIA_SIGNING_SECRET=
ddiff --git a/README.md b/README.md
iindex 310f964..bfcfaaa 100644
---- a/README.md
++++ b/README.md
@@@ -46,7 +46,7 @@ without `DATABASE_URL`, development uses an embedded PGlite database in `data/pg
the account and revocation subscriptions)
- Backblaze B2 and Cloudflare R2 (S3 API) when configured; `ffmpeg`/`ffprobe` on the host
- `openvibe-contracts` v0.79.0, `openvibe-sdk` v0.28.0 (tokens, events outbox, per-actor limits),
-- `openvibe-shared` v2.5.0, pinned by release tarball
++ `openvibe-shared` v2.10.0, pinned by release tarball
## Capabilities
@@@ -115,7 +115,7 @@ server/
storage.orphans.scan (monthly storage orphan report, service-wide, report only),
storage.move.cleanup (the alert for placement moves whose delete keeps failing, service-wide)
client-ip.js trust proxy = loopback; req.ip is the only client address
--(openvibe-shared v2.5.0, openvibe-contracts v0.79.0, openvibe-sdk v0.28.0: pinned release tarballs, installed by npm)
++(openvibe-shared v2.10.0, openvibe-contracts v0.79.0, openvibe-sdk v0.28.0: pinned release tarballs, installed by npm)
scripts/smoke-test.sh end-to-end smoke test (boots a temp instance)
scripts/reconcile-objects.js / object-invariant.js / object-drift-report.js / no-good-copy-report.js object-model operator tools
scripts/media-jobs.js list jobs, run the size-invariant scan (dry run by default), approve/cancel proposals
@@@ -467,6 +467,7 @@ Counters are per process (a restart forgets them). `test/actor-limits.test.js`.
| `GET /robots.txt` | crawler policy (`openvibe-shared/seo.robotsTxt`: AI and search crawlers named, `/api/`, `/auth/`, `/internal/`, `/o/`, `/metrics`, `/live/`, `/me` disallowed) and the sitemap |
| `GET /sitemap.xml` | the media index and the watch pages Media is the canonical home of: public, playable VODs and clips of apps other than Live (Live lists its own `/vod` and `/clip` pages), never AI clips, private, unlisted, sandbox, recording or failed items; thumbnails as image entries. Every listed page renders `index, follow` |
| `GET /llms.txt` | a map of the site for language-model crawlers |
++| `GET /llms-full.txt` | the same map plus the full text of the index and the newest watch pages Media is the canonical home of (`openvibe-shared/seo.llmsFull`; rendering stops at 512 KB, and any further pages become its "truncated" line) |
| `GET /live/:sel/frame.jpg` | **live frame API** — near-realtime JPEG frame of an actively-live stream slot, extracted from its in-progress recording. `:sel` = slot id (`1`), slot **slug** (`whip`), or **`@username`** (that streamer's top-viewed live slot; slug/username resolve via the app's `/api/streams` listing, cached 5s). Optional `?w=64..1920` scales the width, `?app=` selects the tenant (default `live`; internal base URLs from `APP_INTERNAL_URLS` JSON env or `LIVE_APP_INTERNAL_URL`). Cached **5s per slot** (that cache is the rate limit), CORS-open for external APIs/bots/dashboards. Not live → **404 with a styled OFFLINE card** (real JPEG bytes — dev pipelines decode the body as image/jpeg) so `<img>` embeds degrade nicely (`?format=json` for JSON errors); `503` + card when live but a frame can't be cut. |
Watch pages (`/v/:id`, `/c/:id` on a browser navigation) offer a player only when the item's
ddiff --git a/STATUS.json b/STATUS.json
iindex 091a0fe..0d1cb79 100644
---- a/STATUS.json
++++ b/STATUS.json
@@@ -7,6 +7,7 @@
"port": 4100,
"domain": "openvibe.media",
"contracts": "openvibe-contracts v0.79.0",
++ "shared": "openvibe-shared v2.10.0",
"capabilities": [
"media.object.upload",
"media.object.read",
ddiff --git a/package-lock.json b/package-lock.json
iindex 56db812..0dd09cf 100644
---- a/package-lock.json
++++ b/package-lock.json
@@@ -19,7 +19,7 @@
"multer": "^1.4.5-lts.1",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.79.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.28.0",
-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.5.0",
++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.10.0",
"pg": "^8.23.0",
"sharp": "^0.33.4"
},
@@@ -2055,9 +2055,9 @@
}
},
"node_modules/openvibe-shared": {
-- "version": "2.5.0",
-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.5.0",
-- "integrity": "sha512-fHuUS6SXM7ezatfAyqLm1V8B6puV9YXSnz6aG6XQbWx28qghRCCHeUF3lr4nxiCGdn9SyikYtIyLGL5r1ycp9Q==",
++ "version": "2.10.0",
++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.10.0",
++ "integrity": "sha512-Qn6S5Mny19dI0p5ZeEtZq0YPf3AXQxqt+Ey0iHGtSN4pScx0iBrFE6Xeg+/c/N6U9O8NqsR0RGutV5wNWc0ePA==",
"license": "MIT",
"dependencies": {
"jsonwebtoken": "^9.0.0"
ddiff --git a/package.json b/package.json
iindex 27b49f3..2a535bc 100644
---- a/package.json
++++ b/package.json
@@@ -23,7 +23,7 @@
"multer": "^1.4.5-lts.1",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.79.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.28.0",
-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.5.0",
++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.10.0",
"pg": "^8.23.0",
"sharp": "^0.33.4"
},
ddiff --git a/server/config.js b/server/config.js
iindex 31c5859..6dfbee6 100644
---- a/server/config.js
++++ b/server/config.js
@@@ -142,6 +142,10 @@ const config = {
internalUrl: (process.env.OV_NETWORK_INTERNAL_URL || 'http://127.0.0.1:4000').replace(/\/$/, ''),
},
++ // IndexNow (openvibe-shared/indexnow): a key makes search engines recrawl a public page when it
++ // appears or changes (the key file is served at /<key>.txt). Unset: off, no key file, nothing sent.
++ indexnow: { key: String(process.env.INDEXNOW_KEY || '').trim() },
++
apps: {
// JSON array: [{app_id, name, api_key, webhook_url, webhook_secret, allowed_origins, quota_bytes}]
seedJson: process.env.MEDIA_APPS_SEED || '',
ddiff --git a/server/index.js b/server/index.js
iindex e1a3f7a..066495a 100644
---- a/server/index.js
++++ b/server/index.js
@@@ -31,6 +31,7 @@ const { gracefulStop } = require('openvibe-sdk/service');
const config = require('./config');
const db = require('./db/database');
const auth = require('./auth');
++const { createIndexNow } = require('openvibe-shared/indexnow');
// Tests take the app and the server once the boot has finished: require('./server/index.js').ready.
const ready = (async () => {
@@@ -90,6 +91,24 @@ const ready = (async () => {
app.post('/internal/events', ...require('./revocations').handler());
app.use(express.json({ limit: '2mb' }));
++ // IndexNow (openvibe-shared/indexnow): created once at boot from INDEXNOW_KEY. Unset → off, the
++ // key file is not mounted and nothing is sent. When set, GET /<key>.txt is the key file engines
++ // fetch before they accept a ping; the module owns the batching and the POST itself. The host and
++ // key are operator configuration (MEDIA_PUBLIC_URL / INDEXNOW_KEY), never a tenant or a user URL.
++ // A malformed key (createIndexNow refuses anything but 8–128 alphanumerics) disables IndexNow
++ // instead of crashing the service at boot; the watch-page transitions still call the notifier,
++ // which no-ops while disabled.
++ let indexnow;
++ try {
++ indexnow = createIndexNow({ host: config.publicUrl, key: config.indexnow.key });
++ } catch (err) {
++ console.warn(`[IndexNow] disabled: ${err.message}`);
++ indexnow = createIndexNow({ host: config.publicUrl, key: '' });
++ }
++ app.locals.indexnow = indexnow;
++ require('./indexnow-notify').use(indexnow);
++ if (indexnow.enabled) app.use(indexnow.keyFile);
++
// ── Visitor sign-in (OAuth client `media` on the Network; same module as Community/Tools) ──
// Host-only cookies on openvibe.media. No new dependency: cookies are parsed here.
app.use((req, _res, next) => {
ddiff --git a/server/indexnow-notify.js b/server/indexnow-notify.js
nnew file mode 100644
iindex 0000000..d3229ab
---- /dev/null
++++ b/server/indexnow-notify.js
@@@ -0,0 +1,44 @@
++/**
++ * OpenVibe.Media — tell search engines when a watch page becomes indexable.
++ *
++ * server/index.js creates the IndexNow client once at boot (openvibe-shared/indexnow) and hands it
++ * here with use(). The transitions that can make /v/:id or /c/:id indexable — a VOD finalizing, a
++ * clip becoming ready, and a visibility change on either — call pingWatch() with the row they just
++ * wrote; it queues the page and the sitemap for one batched POST.
++ *
++ * Only a page that is actually indexable is pinged (pages.watchIndexable: public, not Live's, not an
++ * AI clip) and never one belonging to a developer sandbox tenant, so private, unlisted and sandbox
++ * media is never announced to an engine. A restore drill pings nothing (server/drill.js). Nothing
++ * here is called at module load, and a ping never throws into its caller.
++ */
++'use strict';
++
++const config = require('./config');
++const db = require('./db/database');
++const pages = require('./public/pages');
++const drill = require('./drill');
++
++let client = null;
++
++/** Install the client booted in server/index.js. A missing/disabled client means pinging is off. */
++function use(indexnow) {
++ client = indexnow && indexnow.enabled ? indexnow : null;
++}
++
++/**
++ * Queue an IndexNow ping for a VOD/clip row that just became ready or public: the watch page and the
++ * sitemap. Returns true when URLs were queued. Never throws (a failed ping must not fail a publish).
++ */
++async function pingWatch(kind, row) {
++ try {
++ if (drill.enabled || !client || !row || !pages.watchIndexable(kind, row)) return false;
++ if (await db.isSandboxTenant(row.app_id)) return false;
++ const self = `${config.publicUrl}/${kind === 'vod' ? 'v' : 'c'}/${row.id}`;
++ return client.pingSoon([self, `${config.publicUrl}/sitemap.xml`]) > 0;
++ } catch (err) {
++ console.warn('[IndexNow] notify failed:', err.message);
++ return false;
++ }
++}
++
++module.exports = { use, pingWatch };
ddiff --git a/server/public/crawl.js b/server/public/crawl.js
iindex de08e2d..b5a4fc4 100644
---- a/server/public/crawl.js
++++ b/server/public/crawl.js
@@@ -1,5 +1,5 @@
/**
-- * OpenVibe.Media — crawler files at the origin: /robots.txt, /sitemap.xml, /llms.txt.
++ * OpenVibe.Media — crawler files at the origin: /robots.txt, /sitemap.xml, /llms.txt, /llms-full.txt.
*
* The sitemap lists the pages search engines may index here: the media index (/) and the watch
* pages of public, ready VODs and clips that Media is the canonical home of (pages.watchIndexable:
@@@ -19,6 +19,10 @@ const pages = require('./pages');
const readiness = require('../objects/readiness');
const MAX_URLS = 45000; // under the 50,000-URL sitemap limit
++const LLMS_FULL_MAX_BYTES = 512 * 1024; // cap an /llms-full.txt over every watch page
++const MAX_FULL_PAGES = 2000; // rows considered for the full-text file (rendering every page at once is unbounded)
++const LLMS_FULL_TTL_MS = 10 * 60 * 1000; // a cold /llms-full.txt renders up to 2000 pages; serve one build for 10 minutes
++const LLMS_SUMMARY = 'The media service of the OpenVibe network: it stores and serves recorded live streams (VODs), clips, thumbnails, screenshots and files for the network\'s apps.';
const router = express.Router();
const iso = (v) => {
@@@ -79,7 +83,7 @@ function llmsTxt() {
const base = config.publicUrl;
return seo.llmsTxt({
name: 'OpenVibe.Media',
-- summary: 'The media service of the OpenVibe network: it stores and serves recorded live streams (VODs), clips, thumbnails, screenshots and files for the network\'s apps.',
++ summary: LLMS_SUMMARY,
details: [
'Most media here belongs to OpenVibe.Live, whose /vod/:id and /clip/:id pages are the canonical pages for it (chat, transcript, the streamer\'s channel); the watch pages here point there.',
'Clips marked "AI clip" were picked and cut automatically from a stream, not made by a person.',
@@@ -89,6 +93,7 @@ function llmsTxt() {
{ title: 'Browse', links: [
{ title: 'Media index', url: `${base}/`, note: 'every public video, clip, image and file, newest first, each linking to its source page' },
{ title: 'Sitemap', url: `${base}/sitemap.xml`, note: 'the watch pages Media is the canonical home of' },
++ { title: 'Full text', url: `${base}/llms-full.txt`, note: 'those same pages with their full text, for language models' },
] },
{ title: 'URLs', links: [
{ title: 'VOD watch page', url: `${base}/v/{id}`, note: 'a page for browsers; ?raw=1 returns the video bytes' },
@@@ -104,6 +109,72 @@ function llmsTxt() {
});
}
++/**
++ * The indexable watch pages with the fields a watch page renders, newest first: the same set the
++ * sitemap lists (WATCH_WHERE, pages.watchIndexable, no sandbox tenant). Used by /llms-full.txt,
++ * which bounds how many it will render (`limit`).
++ */
++async function indexableWatchRows(limit = MAX_URLS) {
++ const sandbox = await sandboxApps();
++ const out = [];
++ const base = 'id, app_id, visibility, is_public, title, description, ai_overview, thumbnail_url, file_path, duration_seconds, view_count, created_at';
++ for (const kind of ['vod', 'clip']) {
++ const table = kind === 'vod' ? 'vods' : 'clips';
++ // vods has no auto_generated column (only clips do); pages.watchIndexable reads it for clips.
++ const cols = kind === 'clip' ? `${base}, auto_generated` : base;
++ const rows = await db.all(`SELECT ${cols} FROM ${table} WHERE ${WATCH_WHERE[kind]} ORDER BY created_at DESC LIMIT ?`, [limit]);
++ for (const row of rows) {
++ if (sandbox.has(row.app_id) || !pages.watchIndexable(kind, row)) continue;
++ out.push({ kind, row });
++ }
++ }
++ out.sort((a, b) => String(b.row.created_at || '').localeCompare(String(a.row.created_at || '')));
++ return out.slice(0, limit);
++}
++
++/**
++ * /llms-full.txt (openvibe-shared/seo.llmsFull): the /llms.txt header, then the full text of the
++ * public media pages — the index and the newest watch pages Media is the canonical home of, rendered
++ * by the same pages.renderWatchPage the browser gets and stripped to text. Pages are rendered only
++ * until the byte cap is passed, so a large estate cannot make the request unbounded; any that remain
++ * become the standard "(truncated: N more pages …)" line.
++ *
++ * A cold build walks up to 2000 rows and renders each page, so the result is memoized in memory for
++ * ten minutes (one bounded entry; a restart starts empty). Crawlers ask for this file far more often
++ * than the estate changes.
++ */
++let _llmsFull = { at: 0, body: null };
++
++async function llmsFullTxt() {
++ if (_llmsFull.body != null && Date.now() - _llmsFull.at < LLMS_FULL_TTL_MS) return _llmsFull.body;
++ const base = config.publicUrl;
++ const rows = await indexableWatchRows(MAX_FULL_PAGES);
++ const watchPages = [];
++ let bytes = 0;
++ for (const { kind, row } of rows) {
++ const html = pages.renderWatchPage(kind, row);
++ watchPages.push({
++ title: String(row.title || (kind === 'vod' ? `VOD #${row.id}` : `Clip #${row.id}`)),
++ url: `${base}/${kind === 'vod' ? 'v' : 'c'}/${row.id}`,
++ html,
++ });
++ bytes += Buffer.byteLength(html);
++ if (bytes > LLMS_FULL_MAX_BYTES) break; // the cap is already passed; stop rendering the rest
++ }
++ const body = seo.llmsFull({
++ site: { name: 'OpenVibe.Media', url: base },
++ summary: LLMS_SUMMARY,
++ base,
++ maxBytes: LLMS_FULL_MAX_BYTES,
++ sections: [
++ { title: 'Browse', pages: [{ title: 'Media index', url: `${base}/`, text: 'Every public video, clip, image and file on OpenVibe.Media, newest first, each linking to its source page in the app that made it.' }] },
++ { title: 'Watch pages', pages: watchPages },
++ ],
++ });
++ _llmsFull = { at: Date.now(), body };
++ return body;
++}
++
function send(res, type, body) {
res.set('Cache-Control', cache.htmlHeaders({ maxAge: 3600 }));
res.type(type).send(body);
@@@ -111,6 +182,14 @@ function send(res, type, body) {
router.get('/robots.txt', (req, res) => send(res, 'text/plain', robotsTxt()));
router.get('/llms.txt', (req, res) => send(res, 'text/plain', llmsTxt()));
++router.get('/llms-full.txt', async (req, res) => {
++ try {
++ send(res, 'text/plain', await llmsFullTxt());
++ } catch (err) {
++ console.error('[Public] llms-full error:', err.message);
++ res.status(500).type('text/plain').send('llms-full unavailable');
++ }
++});
router.get('/sitemap.xml', async (req, res) => {
try {
send(res, 'application/xml', seo.sitemapXml([{ loc: `${config.publicUrl}/`, changefreq: 'hourly' }, ...await sitemapEntries()]));
@@@ -126,3 +205,5 @@ module.exports.WATCH_WHERE = WATCH_WHERE;
module.exports.sandboxApps = sandboxApps;
module.exports.robotsTxt = robotsTxt;
module.exports.llmsTxt = llmsTxt;
++module.exports.llmsFullTxt = llmsFullTxt;
++module.exports.indexableWatchRows = indexableWatchRows;
ddiff --git a/server/public/page-frame.js b/server/public/page-frame.js
iindex 483f33d..cb42933 100644
---- a/server/public/page-frame.js
++++ b/server/public/page-frame.js
@@@ -1,16 +1,20 @@
/**
-- * OpenVibe.Media — the shell every server-rendered page shares.
++ * OpenVibe.Media — the shared layout for every server-rendered page (the T11 sweep).
*
-- * One place for: the OpenVibe Frame (openvibe.network's theme-loader before
-- * paint, navbar + footer after), the SEO head (title, description, canonical,
-- * robots, Open Graph, Twitter card, JSON-LD) and the page palette, which
-- * reads the shared theme tokens (--bg-primary, --accent, …) that
-- * theme-loader.js sets on <html> and falls back to the default Vibe palette.
++ * `page()` composes the document through openvibe-shared/shell: the shell owns the doctype and <head>
++ * (openvibe-shared/seo: title, description, canonical, robots, Open Graph/Twitter, JSON-LD), the
++ * theme-loader before paint, the deferred web runtime/navbar/footer, the no-JavaScript nav and the
++ * shared footer. This module supplies what the shell does not: the site's navbar options (sign-in,
++ * history, the account menu's "Your media"), the server-rendered footer's links, the app icon, the
++ * Open Graph video tags and the page palette, which reads the shared theme tokens (--bg-primary,
++ * --accent, …) that theme-loader.js sets on <html> and falls back to the default Vibe palette.
*/
'use strict';
const config = require('../config');
const ovServe = require('openvibe-shared/serve');
++const shell = require('openvibe-shared/shell');
++const appIcon = require('openvibe-shared/app-icon');
const NETWORK_URL = (config.network && config.network.url) || 'https://openvibe.network';
const SITE_NAME = 'OpenVibe.Media';
@@@ -64,44 +68,25 @@ function fmtDate(dt) {
}
/**
-- * The <head> a page needs: shared theme before paint, then the SEO set.
-- * title, description, canonical (absolute), robots, image (absolute),
-- * ogType, jsonLd (array of objects), video ({ url, type, width, height }).
++ * The site-specific <head> additions the shared shell does not own: the app icon, the page palette,
++ * and the Open Graph video tags (openvibe-shared/seo has no video fields). `page()` passes this as
++ * `shell.page`'s `head`, so the shell still owns title, description, canonical, robots, og/twitter
++ * mirroring and JSON-LD.
*/
--function headTags(seo) {
-- const title = esc(seo.title);
-- const description = esc(snip(seo.description, 300));
++function extraHead(seo, css) {
const image = seo.image || DEFAULT_OG_IMAGE;
-- const twitterCard = seo.twitterCard || 'summary_large_image';
-- const tags = [
-- '<meta charset="utf-8">',
-- '<meta name="viewport" content="width=device-width, initial-scale=1">',
-- `<title>${title}</title>`,
-- `<meta name="description" content="${description}">`,
-- // Applies the user's theme to <html> synchronously; must run before any CSS paints.
-- require('openvibe-shared/app-icon').headTags({ site: 'media', iconBase: '/assets' }),
-- `<script src="${ovServe.url('theme-loader.js')}" defer></script>`,
-- seo.canonical ? `<link rel="canonical" href="${esc(seo.canonical)}">` : '',
-- `<meta name="robots" content="${esc(seo.robots || 'index, follow')}">`,
-- `<meta property="og:site_name" content="${SITE_NAME}">`,
-- `<meta property="og:type" content="${esc(seo.ogType || 'website')}">`,
-- `<meta property="og:title" content="${title}">`,
-- `<meta property="og:description" content="${description}">`,
-- seo.canonical ? `<meta property="og:url" content="${esc(seo.canonical)}">` : '',
-- `<meta property="og:image" content="${esc(image)}">`,
++ const videoTags = seo.video && seo.video.url ? [
++ `<meta property="og:video" content="${esc(seo.video.url)}">`,
++ `<meta property="og:video:secure_url" content="${esc(seo.video.url)}">`,
++ seo.video.type ? `<meta property="og:video:type" content="${esc(seo.video.type)}">` : '',
++ ].filter(Boolean).join('\n') : '';
++ return [
++ appIcon.headTags({ site: 'media', iconBase: '/assets' }),
++ `<style>${baseCss()}${css}</style>`,
++ videoTags,
++ // The default share card is 1200×630; a media thumbnail carries its own dimensions.
image === DEFAULT_OG_IMAGE ? '<meta property="og:image:width" content="1200"><meta property="og:image:height" content="630">' : '',
-- seo.video && seo.video.url ? [
-- `<meta property="og:video" content="${esc(seo.video.url)}">`,
-- `<meta property="og:video:secure_url" content="${esc(seo.video.url)}">`,
-- seo.video.type ? `<meta property="og:video:type" content="${esc(seo.video.type)}">` : '',
-- ].join('\n') : '',
-- `<meta name="twitter:card" content="${esc(twitterCard)}">`,
-- `<meta name="twitter:title" content="${title}">`,
-- `<meta name="twitter:description" content="${description}">`,
-- `<meta name="twitter:image" content="${esc(image)}">`,
-- ...(seo.jsonLd || []).map(obj => `<script type="application/ld+json">${jsonForScript(obj)}</script>`),
-- ];
-- return tags.filter(Boolean).join('\n');
++ ].filter(Boolean).join('\n');
}
/** Page palette on the shared theme tokens (fallback: the default Vibe theme). */
@@@ -141,23 +126,48 @@ function baseCss() {
}
/**
-- * The scripts that mount the navbar + footer, after the page content so a
-- * slow Network never blocks the body.
++ * openvibe-shared/shell's `navbar` option: this site's OpenVibeNavbar.init config.
* history { type, title } → recorded to the signed-in user's network history
-- * footer { variant: 'full' | 'compact', links: [{ heading, items: [{ label, href }] }] }
*/
--function frameScripts({ history, footer } = {}) {
++function navOptions({ history } = {}) {
const navOpts = { service: 'media', apiBase: NETWORK_URL, silentLogin: `${config.publicUrl}/auth/login?silent=1&next={url}`, fedcmLogin: `${config.publicUrl}/auth/fedcm`, loginUrl: `${config.publicUrl}/auth/login?next={url}`, logoutUrl: '/auth/logout?next={path}', sessionUrl: '/auth/me' };
if (history) navOpts.history = history;
// The account menu's section for this site: the signed-in person's own media (server/me/, WS-G task 12).
navOpts.menu = { before: [{ id: 'media-mine', label: 'Your media', href: '/me', icon: 'fa-photo-film' }] };
-- const footOpts = {
++ return navOpts;
++}
++
++/** openvibe-shared/shell's `footer` option (openvibe-shared/frame.footer on the server). */
++function footerOptions(footer) {
++ return {
service: 'media',
variant: (footer && footer.variant) || 'compact',
links: (footer && footer.links) || defaultFooterLinks(),
mount: '#ov-footer',
updates: '/updates', // the footer's "shipped X ago" line and Updates link open this site's log
};
++}
++
++/**
++ * openvibe-shared/shell (v2.10.0) loads footer.js but only boots the navbar: without this call the
++ * server-rendered footer keeps its placeholder "shipped" line (shipped.js is loaded by footer.js's
++ * init) and the client never re-renders with the site's links. Same options as the SSR footer, so it
++ * rebuilds the same DOM; footer.js is deferred, so it has run by DOMContentLoaded when this fires.
++ */
++function footerBootScript(footer) {
++ return `<script>window.addEventListener('DOMContentLoaded', function () { try { OpenVibeFooter.init(${jsonForScript(footerOptions(footer))}); } catch (e) { /* footer optional */ } });</script>`;
++}
++
++/**
++ * The scripts that mount the navbar + footer, after the page content so a
++ * slow Network never blocks the body. Used by the hand-written browse index
++ * (server/public/browse.js); the shell pages get the same navbar options through
++ * shell.page() plus footerBootScript() (the shell does not initialise the footer),
++ * and the footer is rendered by frame.footer.
++ */
++function frameScripts({ history, footer } = {}) {
++ const navOpts = navOptions({ history });
++ const footOpts = footerOptions(footer);
return `<script src="${ovServe.url('navbar.js')}" defer></script>
<script src="${ovServe.url('footer.js')}" defer></script>
<script>
@@@ -189,29 +199,42 @@ function defaultFooterLinks() {
}
/**
-- * A whole document: `seo` → headTags, `css` extra styles, `body` the <main>
-- * contents, `history` / `footer` → frameScripts.
++ * A whole document, composed by openvibe-shared/shell: `seo` maps onto the shell's SEO options
++ * (title, description, canonical, robots, image, og type, JSON-LD), `css` adds the page styles and
++ * `body` is the <main> contents. The shell emits the theme-loader/web-runtime/navbar/footer scripts,
++ * the no-JavaScript nav and the shared footer; `history` and `footer` become the navbar init options
++ * and the server-rendered footer. The site's app icon, palette and Open Graph video tags ride in
++ * through the shell's `head`.
*/
function page({ seo, css = '', body, history, footer }) {
-- return `<!DOCTYPE html>
--<html lang="en">
--<head>
--${headTags(seo)}
--<style>${baseCss()}${css}</style>
--</head>
--<body>
--<div id="navbar-mount"></div>${require('openvibe-shared/frame').noscriptNav({ name: 'OpenVibe.Media', links: [{ label: 'Videos', href: '/?tab=videos' }, { label: 'Clips', href: '/?tab=clips' }] })}
--<main>
--${body}
--</main>
--${require('openvibe-shared/frame').footer({ service: 'media', variant: 'compact', updates: '/updates' })}
--${frameScripts({ history, footer })}
--</body>
--</html>`;
++ const html = shell.page({
++ name: SITE_NAME,
++ lang: 'en',
++ title: seo.title,
++ siteName: SITE_NAME,
++ description: seo.description,
++ canonical: seo.canonical,
++ image: seo.image || DEFAULT_OG_IMAGE,
++ type: seo.ogType,
++ robots: seo.robots || 'index, follow',
++ jsonLd: seo.jsonLd,
++ head: extraHead(seo, css) + footerBootScript(footer),
++ body: `<div id="navbar-mount"></div>\n<main>\n${body}\n</main>`,
++ navLinks: [{ label: 'Videos', href: '/?tab=videos' }, { label: 'Clips', href: '/?tab=clips' }],
++ navbar: navOptions({ history }),
++ footer: footerOptions(footer),
++ });
++ // The shell picks the card from whether a large image is present; a page can ask for the small
++ // card explicitly (the paste viewer, a screenshot-less paste). seo.headTags has no override, so
++ // swap the one tag it emitted after the fact.
++ if (seo.twitterCard && seo.twitterCard !== 'summary_large_image') {
++ return html.replace('<meta name="twitter:card" content="summary_large_image">', `<meta name="twitter:card" content="${esc(seo.twitterCard)}">`);
++ }
++ return html;
}
module.exports = {
NETWORK_URL, SITE_NAME, DEFAULT_OG_IMAGE, APP_PUBLIC_URLS, appUrl,
esc, abs, snip, isoDate, isoDuration, fmtDuration, fmtDate, jsonForScript,
-- headTags, baseCss, frameScripts, defaultFooterLinks, page, DEFAULT_THEME_CSS,
++ extraHead, baseCss, frameScripts, navOptions, footerOptions, defaultFooterLinks, page, DEFAULT_THEME_CSS,
};
ddiff --git a/server/public/routes.js b/server/public/routes.js
iindex 7c3e2b7..d9487a1 100644
---- a/server/public/routes.js
++++ b/server/public/routes.js
@@@ -13,7 +13,7 @@
* GET /t/:id thumbnails (id = filename)
* GET /f/:key files with correct Content-Type + Range
* GET /og-image.png the site's share card
-- * GET /robots.txt, /sitemap.xml, /llms.txt crawler files (crawl.js): the sitemap lists only
++ * GET /robots.txt, /sitemap.xml, /llms.txt, /llms-full.txt crawler files (crawl.js): the sitemap lists only
* the watch pages Media is canonical for (public, not Live's, not AI clips)
*
* /v /c /t /f bytes carry X-Robots-Tag: noindex — the owning app has the
@@@ -36,7 +36,7 @@ const pages = require('./pages');
const drill = require('../drill');
const router = express.Router();
--// /robots.txt, /sitemap.xml, /llms.txt (server/public/crawl.js).
++// /robots.txt, /sitemap.xml, /llms.txt, /llms-full.txt (server/public/crawl.js).
router.use(require('./crawl'));
const MIME_TYPES = {
ddiff --git a/server/vod/clip-jobs.js b/server/vod/clip-jobs.js
iindex 1e34152..a61a540 100644
---- a/server/vod/clip-jobs.js
++++ b/server/vod/clip-jobs.js
@@@ -100,6 +100,8 @@ async function _recutClip(clipId, { reason = 'recut', viaJob = false } = {}) {
payload: async () => await _clipPublic(await db.getClipById(clipId)),
});
console.log(`[Clips] Clip ${clipId} ${reason} OK from vod ${clip.vod_id} (${startTime.toFixed(1)}-${(startTime + cut.duration).toFixed(1)}s, attempt ${attempt})`);
++ // A public, person-made clip that just became ready is indexable: announce its watch page and the sitemap.
++ await require('../indexnow-notify').pingWatch('clip', await db.getClipById(clipId));
return { ok: true, duration_seconds: cut.duration };
}
@@@ -158,6 +160,8 @@ async function makeVirtual(clipId, win) {
payload: async () => await _clipPublic(await db.getClipById(clipId)),
});
console.log(`[Clips] Clip ${clipId} virtual over vod ${clip.vod_id}'s timeline (${startTime.toFixed(1)}-${endTime.toFixed(1)}s)`);
++ // A public, person-made clip that just became ready is indexable: announce its watch page and the sitemap.
++ await require('../indexnow-notify').pingWatch('clip', await db.getClipById(clipId));
return { ok: true, duration_seconds: endTime - startTime };
}
ddiff --git a/server/vod/clips-routes.js b/server/vod/clips-routes.js
iindex 9576c9f..ebd7417 100644
---- a/server/vod/clips-routes.js
++++ b/server/vod/clips-routes.js
@@@ -424,7 +424,11 @@ router.put('/:id', tenantAuth({ verb: 'write' }), async (req, res) => {
if (req.authType !== 'app') return res.status(403).json({ error: 'auto_generated is set by the app only' });
await objects.withObject('clip', clip.id, async () => await db.run('UPDATE clips SET auto_generated = ? WHERE id = ?', [flag(autoGen), clip.id]));
}
-- if (visibility !== undefined) await db.setClipVisibility(clip.id, visibility);
++ if (visibility !== undefined) {
++ await db.setClipVisibility(clip.id, visibility);
++ // Made public (or already public): its watch page and the sitemap are indexable now.
++ await require('../indexnow-notify').pingWatch('clip', await db.getClipById(clip.id));
++ }
res.json({ clip: await clipPublic(await db.getClipById(clip.id, req.appId), { readiness: true }) });
} catch (err) {
res.status(500).json({ error: 'Failed to update clip' });
ddiff --git a/server/vod/finalize.js b/server/vod/finalize.js
iindex 44deacc..1492d80 100644
---- a/server/vod/finalize.js
++++ b/server/vod/finalize.js
@@@ -401,6 +401,9 @@ async function _doFinalize(vodId, opts) {
} catch (err) {
console.warn(`[VOD] Timeline queue failed for vod ${vodId}:`, err.message);
}
++ // The recording is finished (and, when it is public and not a sandbox's, indexable): tell the
++ // engines about its watch page and the sitemap (server/indexnow-notify.js).
++ await require('../indexnow-notify').pingWatch('vod', ready);
console.log(`[VOD] Finalized: vod ${vodId}, ${stored}s (${durationSource}), ${(stat.size / 1024 / 1024).toFixed(1)}MB`);
return ready;
}
ddiff --git a/server/vod/routes.js b/server/vod/routes.js
iindex 534282b..dc9fa18 100644
---- a/server/vod/routes.js
++++ b/server/vod/routes.js
@@@ -374,6 +374,8 @@ router.put('/:id', tenantAuth({ verb: 'write' }), async (req, res) => {
}
if (visibility !== undefined) await db.setVodVisibility(vod.id, visibility);
});
++ // Made public (or already public): its watch page and the sitemap are indexable now.
++ if (visibility !== undefined) await require('../indexnow-notify').pingWatch('vod', await db.getVodById(vod.id));
res.json({ vod: await vodPublic(await db.getVodById(vod.id, req.appId), { readiness: true }) });
} catch (err) {
console.error('[VOD] Update error:', err.message);
ddiff --git a/test/crawl.test.js b/test/crawl.test.js
iindex fb4c036..1e29b2f 100644
---- a/test/crawl.test.js
++++ b/test/crawl.test.js
@@@ -94,9 +94,34 @@ const express = require('express');
r = await get('/llms.txt');
assert.strictEqual(r.status, 200);
assert.ok(r.body.startsWith('# OpenVibe.Media') && r.body.includes('https://media.test/sitemap.xml') && r.body.includes('AI clip'));
++ assert.ok(r.body.includes('https://media.test/llms-full.txt'), 'the map links the full-text file');
assert.ok(!/\bfree\b/i.test(r.body), 'no cost claims');
console.log('✅ /llms.txt maps the site');
++ // ── llms-full.txt ──
++ r = await get('/llms-full.txt');
++ assert.strictEqual(r.status, 200);
++ assert.ok(r.headers['content-type'].startsWith('text/plain'));
++ assert.ok(r.body.startsWith('# OpenVibe.Media'), 'the llms.txt header');
++ assert.ok(r.body.includes('### Games public') && r.body.includes('URL: https://media.test/v/1'), 'the full text of each indexable VOD page');
++ assert.ok(r.body.includes('### Games clip') && r.body.includes('URL: https://media.test/c/11'), 'and each indexable clip page');
++ assert.ok(r.body.includes('### Media index') && r.body.includes('URL: https://media.test/\n'), 'the index itself');
++ for (const never of ['/v/2', '/v/3', '/v/4', '/v/5', '/v/6', '/v/7', '/v/8', '/c/12', '/c/13', '/c/14', '/c/15']) {
++ assert.ok(!r.body.includes(`URL: https://media.test${never}`), `${never} is not in llms-full`);
++ }
++ assert.strictEqual((await get('/llms-full.txt')).body, r.body, 'deterministic');
++ // Memoized: a second build inside the 10-minute TTL reuses the first without touching the database.
++ {
++ const crawl = require('../server/public/crawl');
++ const realAll = db.all; let calls = 0;
++ db.all = (...a) => { calls++; return realAll(...a); };
++ try {
++ assert.strictEqual(await crawl.llmsFullTxt(), r.body, 'the cached body is returned');
++ assert.strictEqual(calls, 0, 'a cached /llms-full.txt is rebuilt without another query');
++ } finally { db.all = realAll; }
++ }
++ console.log('✅ /llms-full.txt: the full text of the index and every indexable watch page, and only those');
++
// ── Watch pages: VideoObject on public pages; AI clips noindex, no Person, based on the VOD ──
const ldOf = (html) => { const m = /<script type="application\/ld\+json">([\s\S]*?)<\/script>/.exec(html); return m ? JSON.parse(m[1]) : null; };
const pub = pages.renderWatchPage('vod', { ...await byId('vods', 1), username: 'gamer' });
ddiff --git a/test/indexnow-ping.test.js b/test/indexnow-ping.test.js
nnew file mode 100644
iindex 0000000..04567bf
---- /dev/null
++++ b/test/indexnow-ping.test.js
@@@ -0,0 +1,94 @@
++'use strict';
++// IndexNow pings on the transitions that make a watch page indexable (server/indexnow-notify.js,
++// wired into VOD finalize, clip ready and the visibility-change routes): a public, person-made VOD
++// or clip that becomes ready or public queues its /v|/c page and the sitemap; private, unlisted,
++// Live-owned, AI and sandbox rows never do. The client is injected with a stubbed fetch, so no
++// request leaves the process.
++const assert = require('assert');
++const fs = require('fs');
++const os = require('os');
++const path = require('path');
++const http = require('http');
++
++(async () => {
++ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-media-indexnow-ping-'));
++ const dir = (n) => { const d = path.join(tmp, n); fs.mkdirSync(d, { recursive: true }); return d; };
++ Object.assign(process.env, {
++ MEDIA_PUBLIC_URL: 'https://media.test',
++ VOD_PATH: dir('vods'), CLIPS_PATH: dir('clips'), FILES_PATH: dir('files'),
++ OBJECTS_PATH: dir('objects'), THUMBNAILS_PATH: dir('thumbnails'), PASTES_PATH: dir('pastes'),
++ });
++ delete process.env.MEDIA_DRILL;
++
++ const db = require('../server/db/database');
++ const { createIndexNow } = require('openvibe-shared/indexnow');
++ const notify = require('../server/indexnow-notify');
++
++ await db.upsertApp({ app_id: 'games', api_key: 'games-key' });
++ const sandbox = await db.ensureProjectTenant('prj_01TESTINDEXNOW0000000000A', 'sandbox', 1024);
++ const raw = db.getDb();
++
++ // The client booted in server/index.js, with a fetch that records the POST instead of sending it.
++ const posts = [];
++ const client = createIndexNow({
++ host: 'media.test', key: 'mediaIndexNowKey123',
++ fetch: async (_url, init) => { posts.push(JSON.parse(init.body)); return { status: 200 }; },
++ });
++ notify.use(client);
++
++ const insVod = raw.prepare(`INSERT INTO vods (id, app_id, title, file_path, is_public, visibility, is_recording, health_status, created_at, duration_seconds) OVERRIDING SYSTEM VALUE
++ VALUES (?, ?, ?, ?, ?, ?, 0, 'ok', '2026-10-01 10:00:00', 60) RETURNING id`);
++ await insVod.run(1, 'games', 'Public ready', '/x/v1.mp4', 1, 'public');
++ await insVod.run(2, 'games', 'Private', '/x/v2.mp4', 0, 'private');
++ await insVod.run(3, sandbox.app_id, 'Sandbox public', '/x/v3.mp4', 1, 'public');
++ await insVod.run(4, 'games', 'Unlisted', '/x/v4.mp4', 0, 'unlisted');
++ const insClip = raw.prepare(`INSERT INTO clips (id, app_id, vod_id, title, file_path, is_public, visibility, auto_generated, status, created_at) OVERRIDING SYSTEM VALUE
++ VALUES (?, 'games', 1, ?, ?, ?, ?, ?, 'ready', '2026-10-01 11:00:00') RETURNING id`);
++ await insClip.run(11, 'Public clip', '/x/c11.webm', 1, 'public', 0);
++ await insClip.run(12, 'AI moment', '/x/c12.webm', 1, 'public', 1);
++
++ const vod = async (id) => await db.get('SELECT * FROM vods WHERE id = ?', [id]);
++ const clip = async (id) => await db.get('SELECT * FROM clips WHERE id = ?', [id]);
++ const flush = async () => { await client.flush(); const batch = posts.at(-1); posts.length = 0; return batch; };
++
++ // ── pingWatch: what qualifies ──
++ assert.strictEqual(await notify.pingWatch('vod', await vod(1)), true, 'a public, person-made VOD queues a ping');
++ assert.strictEqual(await notify.pingWatch('clip', await clip(11)), true, 'a public, person-made clip queues a ping');
++ let batch = await flush();
++ assert.deepStrictEqual(batch.urlList.sort(), ['https://media.test/c/11', 'https://media.test/sitemap.xml', 'https://media.test/v/1'].sort());
++ assert.strictEqual(batch.host, 'media.test');
++ assert.strictEqual(batch.key, 'mediaIndexNowKey123');
++
++ assert.strictEqual(await notify.pingWatch('vod', await vod(2)), false, 'a private VOD is not pinged');
++ assert.strictEqual(await notify.pingWatch('vod', await vod(4)), false, 'an unlisted VOD is not pinged');
++ assert.strictEqual(await notify.pingWatch('vod', await vod(3)), false, 'a sandbox tenant\'s public VOD is not pinged');
++ assert.strictEqual(await notify.pingWatch('clip', await clip(12)), false, 'an AI clip is not pinged');
++ assert.strictEqual(await notify.pingWatch('vod', null), false, 'a missing row is not pinged');
++ assert.deepStrictEqual(posts, [], 'nothing was queued for any of them');
++
++ // ── The visibility-change route calls it: public pings again, private pings nothing ──
++ const express = require('express');
++ const app = express();
++ app.use(express.json());
++ app.use('/api/v1/:app/vods', require('../server/vod/routes'));
++ const server = http.createServer(app);
++ await new Promise((r) => server.listen(0, '127.0.0.1', r));
++ const base = `http://127.0.0.1:${server.address().port}`;
++ const put = async (p, body) => {
++ const res = await fetch(base + p, { method: 'PUT', headers: { 'content-type': 'application/json', authorization: 'Bearer games-key' }, body: JSON.stringify(body) });
++ return { status: res.status };
++ };
++ try {
++ assert.strictEqual((await put('/api/v1/games/vods/2', { visibility: 'public' })).status, 200);
++ batch = await flush();
++ assert.deepStrictEqual(batch.urlList.sort(), ['https://media.test/sitemap.xml', 'https://media.test/v/2'], 'a private VOD made public pings its page');
++ assert.strictEqual((await put('/api/v1/games/vods/1', { visibility: 'private' })).status, 200);
++ assert.deepStrictEqual(posts, [], 'a public VOD made private pings nothing');
++ } finally { server.close(); }
++
++ notify.use(null);
++ await db.close();
++ fs.rmSync(tmp, { recursive: true, force: true });
++ console.log('indexnow ping: public ready VODs/clips ping; private, unlisted, AI and sandbox never; visibility changes too');
++ process.exit(0);
++})().catch((err) => { console.error(err); process.exit(1); });
ddiff --git a/test/indexnow.test.js b/test/indexnow.test.js
nnew file mode 100644
iindex 0000000..06a1fac
---- /dev/null
++++ b/test/indexnow.test.js
@@@ -0,0 +1,71 @@
++'use strict';
++// server/index.js mounts the IndexNow key file from INDEXNOW_KEY (openvibe-shared/indexnow): the key
++// configured at boot is served at /<key>.txt, the file the protocol requires engines to fetch before
++// they accept a ping. Unset, nothing is mounted. A malformed key disables IndexNow with a warning
++// instead of crashing the service at boot. A restore drill serves the key file too (it is public).
++const assert = require('assert');
++const fs = require('fs');
++const os = require('os');
++const net = require('net');
++const path = require('path');
++const { spawn } = require('child_process');
++
++const freePort = () => new Promise((resolve) => { const s = net.createServer(); s.listen(0, '127.0.0.1', () => { const { port } = s.address(); s.close(() => resolve(port)); }); });
++
++/** Boot server/index.js with INDEXNOW_KEY set to `key`, wait for /healthz, and return it plus stderr. */
++async function boot(dir, key) {
++ const port = await freePort();
++ const child = spawn(process.execPath, [path.join(__dirname, '..', 'server', 'index.js')], {
++ cwd: path.join(__dirname, '..'),
++ env: {
++ ...process.env, PORT: String(port), HOST: '127.0.0.1', NODE_ENV: 'test', INDEXNOW_KEY: key,
++ MEDIA_PGLITE_DIR: path.join(dir, 'pglite'), DATABASE_URL: '', DATABASE_DIRECT_URL: '', VALKEY_URL: '',
++ VOD_PATH: path.join(dir, 'vods'), CLIPS_PATH: path.join(dir, 'clips'), FILES_PATH: path.join(dir, 'files'),
++ THUMBNAILS_PATH: path.join(dir, 'thumbnails'), PASTES_PATH: path.join(dir, 'pastes'), OBJECTS_PATH: path.join(dir, 'objects'),
++ MEDIA_JOBS_ENABLED: 'off',
++ },
++ stdio: ['ignore', 'ignore', 'pipe'],
++ });
++ let stderr = '';
++ child.stderr.on('data', (d) => { stderr = (stderr + d).slice(-4000); });
++ const base = `http://127.0.0.1:${port}`;
++ let up = false;
++ for (let i = 0; i < 600 && !up; i++) {
++ up = await fetch(`${base}/healthz`).then((r) => r.ok).catch(() => false);
++ if (!up) await new Promise((r) => setTimeout(r, 100));
++ }
++ return { child, base, get stderr() { return stderr; }, up };
++}
++
++(async () => {
++ const KEY = 'mediaIndexNowKey123';
++ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-indexnow-'));
++ let first = null;
++ try {
++ // ── A valid key: the file is served, and only the file ──
++ first = await boot(dir, KEY);
++ assert.ok(first.up, `the server did not start:\n${first.stderr}`);
++ const r = await fetch(`${first.base}/${KEY}.txt`);
++ assert.strictEqual(r.status, 200);
++ assert.ok(r.headers.get('content-type').startsWith('text/plain'));
++ assert.strictEqual((await r.text()).trim(), KEY, 'the file holds exactly the key');
++ const other = await fetch(`${first.base}/not-the-key.txt`);
++ assert.strictEqual(other.status, 404, 'only the key file is mounted; it serves itself and passes everything else on');
++ console.log('indexnow: the key file is served when INDEXNOW_KEY is set');
++ } finally {
++ if (first) first.child.kill();
++ }
++ try {
++ // ── A malformed key: Media boots anyway, IndexNow is disabled and no key file is mounted ──
++ const badDir = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-indexnow-bad-'));
++ const second = await boot(badDir, 'not a valid key!');
++ try {
++ assert.ok(second.up, `a malformed INDEXNOW_KEY must not crash the service:\n${second.stderr}`);
++ assert.ok(/\[IndexNow\] disabled:/.test(second.stderr), 'the refusal is logged as a warning');
++ assert.strictEqual((await fetch(`${second.base}/not-a-valid-key.txt`)).status, 404, 'no key file is mounted');
++ console.log('indexnow: a malformed key disables IndexNow instead of crashing the boot');
++ } finally { second.child.kill(); fs.rmSync(badDir, { recursive: true, force: true }); }
++ } finally {
++ fs.rmSync(dir, { recursive: true, force: true });
++ }
++})().catch((err) => { console.error(err); process.exit(1); });
ddiff --git a/test/public-pages.test.js b/test/public-pages.test.js
iindex 13b1d8b..d026682 100644
---- a/test/public-pages.test.js
++++ b/test/public-pages.test.js
@@@ -37,10 +37,19 @@ const http = require('http');
assert.ok(html.includes('<link rel="canonical" href="https://openvibe.live/vod/42">'), 'Live-owned VOD is canonical on Live');
assert.ok(html.includes('<meta name="robots" content="noindex, follow">'));
assert.ok(/\/shared\/theme-loader\.js\?v=[0-9a-f]{12}/.test(html), 'this site\'s own pinned Frame files');
-- assert.ok(html.indexOf('theme-loader.js') < html.indexOf('<style>'), 'theme-loader runs before the styles');
++ // The shell loads the theme-loader eagerly (no defer) inside <head>, so it applies the user's
++ // theme before the first paint; the page palette rides in the shell's extra head.
++ assert.ok(/<script src="[^"]*theme-loader\.js\?v=[0-9a-f]{12}"><\/script>/.test(html), 'theme-loader is eager, before the first paint');
++ assert.ok(/\/shared\/web-runtime\.js\?v=[0-9a-f]{12}/.test(html), 'the shared web runtime from this site\'s own pinned copy');
assert.ok(/\/shared\/navbar\.js\?v=[0-9a-f]{12}/.test(html) && /\/shared\/footer\.js\?v=[0-9a-f]{12}/.test(html), 'the Frame from this site\'s own pinned copy');
assert.ok(html.includes('"service":"media"') && html.includes('"history":{"type":"vod","title":"Late night <build>"}'.replace('<', '\\u003c')));
-- assert.ok(html.includes('"variant":"compact"'));
++ assert.ok(html.includes('data-variant="compact"'), 'the server-rendered footer is compact');
++ // openvibe-shared/shell v2.10.0 boots the navbar but not the footer; page-frame adds the init so
++ // the "shipped X ago" line fills and the site's links survive a client render.
++ assert.ok(html.includes('OpenVibeFooter.init('), 'the footer is initialised client-side');
++ assert.ok(html.includes('"mount":"#ov-footer"') && html.includes('"updates":"/updates"') && html.includes('"variant":"compact"'),
++ 'the footer init carries the mount, this site\'s log and the compact variant');
++ assert.ok(html.includes('"links":[{"heading":"Media"'), 'and the site\'s own footer links');
assert.ok(html.includes('<meta property="og:type" content="video.other">'));
assert.ok(html.includes('<meta property="og:video" content="https://media.test/v/42?raw=1">'));
assert.ok(html.includes('<meta name="twitter:card" content="summary_large_image">'));
@@@ -96,6 +105,10 @@ const http = require('http');
assert.strictEqual(r.status, 200);
assert.ok(r.headers['content-type'].startsWith('text/html'));
assert.ok(r.body.includes('Router VOD') && r.body.includes('id="navbar-mount"'));
++ // No Content-Security-Policy on a page response today: the shell composes inline scripts
++ // (theme boot, navbar/runtime boot, footer init), so adding a policy later must allow them.
++ assert.ok(!('content-security-policy' in r.headers), 'watch pages carry no CSP header');
++ assert.ok(/\/shared\/web-runtime\.js\?v=[0-9a-f]{12}/.test(r.body), 'the page ships the shell\'s web-runtime.js');
r = await get('/v/42?raw=1', nav);
assert.strictEqual(r.status, 404, 'bytes path (no file on disk → 404 JSON), not the page');
assert.ok(r.headers['content-type'].startsWith('application/json'));
ddiff --git a/test/security-secrets.test.js b/test/security-secrets.test.js
iindex 9b66579..22db5e3 100644
---- a/test/security-secrets.test.js
++++ b/test/security-secrets.test.js
@@@ -52,7 +52,9 @@ const crypto = require('crypto');
const walk = (d) => { for (const e of fs.readdirSync(d, { withFileTypes: true })) { const p = path.join(d, e.name); if (e.isDirectory()) walk(p); else if (p.endsWith('.js')) for (const m of fs.readFileSync(p, 'utf8').matchAll(/process\.env\.([A-Z0-9_]+)/g)) names.add(m[1]); } };
walk(path.join(ROOT, 'server'));
for (const line of fs.readFileSync(path.join(ROOT, '.env.example'), 'utf8').split('\n')) { const m = /^\s*#?\s*([A-Z][A-Z0-9_]+)=/.exec(line); if (m) names.add(m[1]); }
-- const secretish = [...names].filter(n => /SECRET|_KEY$|_KEYS$|KEY_ID$|TOKEN|PASSWORD|PASSWD|CREDENTIAL|_SEED$|PRIVATE/.test(n) && !/_(S|MS|MB|GB|MIN|HOURS|DAYS)$/.test(n)); // not the numeric settings (MEDIA_UPLOAD_TOKEN_TTL_S)
++ // Public by protocol, not secrets: an IndexNow key is served at /<key>.txt for search engines to verify.
++ const PUBLIC_BY_DESIGN = new Set(['INDEXNOW_KEY']);
++ const secretish = [...names].filter(n => /SECRET|_KEY$|_KEYS$|KEY_ID$|TOKEN|PASSWORD|PASSWD|CREDENTIAL|_SEED$|PRIVATE/.test(n) && !/_(S|MS|MB|GB|MIN|HOURS|DAYS)$/.test(n) && !PUBLIC_BY_DESIGN.has(n)); // not the numeric settings (MEDIA_UPLOAD_TOKEN_TTL_S)
assert.ok(secretish.length >= 10, `found the secret variables (${secretish})`);
for (const n of secretish) assert.ok(Object.prototype.hasOwnProperty.call(SECRET_ENV, n), `${n} is read by the service: give it a sentinel in test/security-secrets.test.js`);
}
ddiff --git a/test/security-ssrf.test.js b/test/security-ssrf.test.js
iindex 8df0062..858d606 100644
---- a/test/security-ssrf.test.js
++++ b/test/security-ssrf.test.js
@@@ -6,8 +6,9 @@
// - POST /api/v1/:app/vods/:id/ingest/rtmp makes ffmpeg pull an app-supplied rtmp_url, which must name an
// allow-listed host:port (MEDIA_RTMP_PULL_ALLOW; test/rtmp-allowlist.test.js has the URL grammar).
// Every other outbound request goes to an operator-configured address (Network, Events, Live's internal
--// URL, B2/R2, an app's webhook_url from MEDIA_APPS_SEED). The inventory below fails when a new outbound
--// call appears in server/ until it is classified here, and no route lets a tenant set a webhook URL.
++// URL, B2/R2, an app's webhook_url from MEDIA_APPS_SEED, IndexNow's fixed api.indexnow.org from
++// INDEXNOW_KEY). The inventory below fails when a new outbound call appears in server/ until it is
++// classified here, and no route lets a tenant set a webhook URL.
// No test here needs the network: literals and internal names are refused before DNS, and DNS answers
// and the socket are stubbed.
const assert = require('assert');
@@@ -69,7 +70,13 @@ const https = require('https');
assert.deepStrictEqual(sqlWriters, ['server/db/database.js'], 'only db.upsertApp writes an app\'s webhook_url');
assert.deepStrictEqual(upserters, ['server/auth.js'], 'and only app seeding calls it');
assert.ok(/function seedApps[\s\S]*upsertApp\(/.test(fs.readFileSync(path.join(ROOT, 'server/auth.js'), 'utf8')), 'from seedApps');
-- console.log(`✅ outbound inventory: ${found.size} files, one user-chosen address (avatar ingest); webhook URLs are operator-set`);
++ // IndexNow (openvibe-shared/indexnow, wired in server/index.js): the only outbound request made
++ // from a dependency. Its endpoint is the fixed api.indexnow.org and host/key are operator config
++ // (MEDIA_PUBLIC_URL / INDEXNOW_KEY) — never a tenant or a user-supplied URL, and no route reaches it.
++ const idxSrc = fs.readFileSync(path.join(ROOT, 'server', 'index.js'), 'utf8');
++ assert.ok(idxSrc.includes('createIndexNow(') && idxSrc.includes('host: config.publicUrl') && idxSrc.includes('key: config.indexnow.key'),
++ 'IndexNow is wired from operator config (host/key), the one outbound dependency');
++ console.log(`✅ outbound inventory: ${found.size} files, one user-chosen address (avatar ingest); webhook URLs are operator-set; IndexNow posts to the fixed api.indexnow.org`);
}
const ingest = require('../server/avatars/ingest');
ddiff --git a/test/shipped-pages.test.js b/test/shipped-pages.test.js
iindex 81a2359..adcdb61 100644
---- a/test/shipped-pages.test.js
++++ b/test/shipped-pages.test.js
@@@ -10,7 +10,7 @@ const assert = require('assert');
const doc = pageFrame.page({ seo: { title: 'What shipped on OpenVibe.Media', description: 'x', canonical: pageFrame.abs('/updates') }, body: frame.updatesBody({ service: 'media', siteName: 'OpenVibe.Media' }), footer: { variant: 'full' } });
assert.ok(doc.includes('What shipped on OpenVibe.Media') && doc.includes('data-ov-shipped="log" data-service="media"'));
assert.ok(doc.includes('rel="canonical"') && doc.includes('/updates"'));
-- assert.ok(doc.includes('"updates":"/updates"'), 'the footer links this site\'s log');
++ assert.ok(doc.includes('href="/updates"'), 'the footer links this site\'s log');
assert.ok(doc.includes('"logoutUrl":"/auth/logout?next={path}"'), 'Sign out in the shared navbar ends this site\'s session');
assert.ok(doc.includes('data-ov-shipped="latest" data-service="media" href="/updates"'), 'the footer\'s shipped line');
assert.ok(!/chrome-ssr/.test(require('fs').readFileSync(require.resolve('../server/public/browse.js'), 'utf8')), 'openvibe-shared/frame, not the old name');