openvibers/openvibe.vip Commits
Give VIP /llms-full.txt and IndexNow (#10)
Changes
@@@ -70,6 +70,12 @@ VIP_EVENTS_SECRET=# How long a creator's member count (from Billing) is reused# VIP_MEMBER_COUNT_TTL_MS=300000++# ── IndexNow (openvibe-shared/indexnow) ──────────────────++# With a key (8-128 hex or alphanumeric characters, what IndexNow's own tools generate), search++# engines are told the moment a published plan appears, changes or is archived (the key file is served++# at /<key>.txt). Unset: off — nothing is mounted and nothing is sent.++#INDEXNOW_KEY=++# ── Per-actor limits (server/api/actor-limits.js) ───────# Reads one caller (a service principal, a person, else an address) may make to /api/v1 per minute# and per hour; past them 429 rate_limited with Retry-After. Entitlement checks, policy evaluation,@@@ -321,11 +321,30 @@ memberships* card gives the creator the badge and widget HTML and the card URL.Billing's price and period from `GET /api/v1/rates`, join form) · `/:username/plans/:slug` public termshistory · `/me` your memberships, the version you joined under, cancel, badge preference ·`/dashboard` plans, versions, perks and bindings, members, members-only resources, the badge and--widget to share and the member-count switch (`?as=network` for staff) · `/embed/:username/…` (above) · `/robots.txt`, `/sitemap.xml` · `/terms`, `/privacy`, `/dmca`++widget to share and the member-count switch (`?as=network` for staff) · `/embed/:username/…` (above) · `/robots.txt`, `/sitemap.xml`, `/llms.txt`, `/llms-full.txt` · `/terms`, `/privacy`, `/dmca`(openvibe-shared legal). Navbar from `https://openvibe.network/shared/navbar.js`, SSR footer and`<noscript>` navigation from openvibe-shared. Forms carry HMAC anti-forgery tokens(`VIP_FORM_SECRET`); cookies are SameSite=Lax. No feed: plans are not a publication stream.++### Crawlers and AI readers++++`/robots.txt` and `/sitemap.xml` come from openvibe-shared/seo (the sitemap lists the home and every++creator with a published plan). `/llms.txt` is the site map for language models; `/llms-full.txt` is++the same header followed by the full plain text of the home, every public creator page and every++published plan's terms (`seo.llmsFull`, built from public data only, cached an hour in memory and with++`Cache-Control: public, max-age=3600`). Every page's head carries an `ai-summary` meta and WebPage++JSON-LD through openvibe-shared/shell.++++### IndexNow (openvibe-shared/indexnow)++++With `INDEXNOW_KEY` set (8–128 hex or alphanumeric characters, what IndexNow's own tools generate),++the key file is served at `/<key>.txt` as `text/plain` (before static, so a file cannot shadow it) and++a plan that changes tells the engines: publishing, editing a published plan and archiving ping++`api.indexnow.org` with the creator's page, the plan's terms page and `/sitemap.xml`; the module++batches and debounces (one POST per 30 s window). A draft, an unchanged edit, a replayed publish and++network plans (no public creator page) never ping, and a failed ping never takes a publish down.++Unset: off — the key route is not mounted and nothing is sent (`test/indexnow.test.js`).++## Before and after the Billing cutoverBilling runs in shadow mode until the Wave 8 cutover (Live stays authoritative for money,@@@ -380,6 +399,8 @@ Reporting a vulnerability: [SECURITY.md](SECURITY.md).| gated-resource evaluation fails closed (no rule, unknown, perk missing, guest, VIP down in the client) | `test/policies.test.js` || VIP works with Live unavailable | no code path calls Live; every test runs without a Live stub || a creator's membership card, badge and widget are public-data only, the same for everyone, embeddable and inert | `test/embeds.test.js` (published plans only, no subjects or binding configs, no cookie read or set, CORS, SVG escaping and sandbox, the widget's frame-ancestors and hashed style, the creator's member-count switch, Billing outage keeps the last count or shows none) |++| `/llms-full.txt` carries the full text of the home, creator and plan pages, and is cached for an hour | `test/llms-full.test.js`, `test/api.test.js` (discovery artifacts) |++| IndexNow is off without `INDEXNOW_KEY` (no key route, nothing sent); with one the key file is served at `/<key>.txt` as `text/plain` and publishing, editing and archiving a plan ping the creator page, the terms page and the sitemap; a draft and an unchanged edit never ping | `test/indexnow.test.js` |Not yet demonstrated (needs the other services): convergence **across consuming products** end to end(Chat, Community, Blog and Wiki call VIP in code; Live does not), and a real Billing/Events round trip@@@ -23,7 +23,8 @@"14 vip.* capabilities, the service manifest and its events registered in openvibe-contracts v0.19.0 (this repository pins v0.92.0)","per-actor limits on /api/v1 (openvibe-sdk/limits)","the Billing client (server/billing-client.js) is a thin wrapper over openvibe-sdk/commerce (openvibe-sdk v0.28.0), shared with Tips: same capabilities, idempotency keys and retry-once-on-401",-- "renewal/refund convergence via Billing events: a per-period charge ledger (vip_membership_periods, unique per Billing transaction; reversals and reversal-before-grant tombstones), a past_due renewal grace that never authorizes (answer.grace), and a period audit that doubts a paid period without a record (migration 0002; docs/cutover-0002-renewal-periods.md)"++ "renewal/refund convergence via Billing events: a per-period charge ledger (vip_membership_periods, unique per Billing transaction; reversals and reversal-before-grant tombstones), a past_due renewal grace that never authorizes (answer.grace), and a period audit that doubts a paid period without a record (migration 0002; docs/cutover-0002-renewal-periods.md)",++ "crawler and AI-reader artifacts: /llms-full.txt serves the full text of the home, public creator and plan-terms pages (openvibe-shared/seo, cached 3600 s), and IndexNow (openvibe-shared/indexnow, INDEXNOW_KEY) serves its key file at /<key>.txt and pings the creator page, terms page and sitemap when a plan is published, edited or archived"],"not_yet": ["not launched: openvibe.vip is still an OpenVibe.Sites placeholder and the service answers on loopback only",@@@ -40,6 +41,8 @@"openvibe-vip/client with the product cache and its convergence bound","creator plan pages, terms history, member status and creator dashboard","public membership card, badge and embeddable widget",++ "/llms.txt and /llms-full.txt for language-model crawlers, cached an hour",++ "IndexNow: a key serves /<key>.txt and pings plan page changes and the sitemap","SDK outbox events vip.plan.published and vip.membership.changed","per-actor limits (openvibe-sdk/limits)"],@@@ -20,6 +20,7 @@ const express = require('express');const cookieParser = require('cookie-parser');const { http } = require('openvibe-contracts');const cache = require('openvibe-shared/cache-policy');++const { createIndexNow } = require('openvibe-shared/indexnow');const { loadConfig } = require('./config');const { openDb } = require('./db');const { createKeyProvider, createUserAuth } = require('./network');@@@ -52,7 +53,12 @@ async function createApp(opts = {}) {const userAuth = createUserAuth(config, keys);const billing = opts.billing || createBillingClient(config, { fetchImpl });const outbox = opts.outbox || createVipOutbox({ db, config, fetchImpl: opts.eventsFetch, now, log });-- const domain = createDomain({ db, config, outbox, billing, now, log });++ // IndexNow (openvibe-shared/indexnow): created once at boot from INDEXNOW_KEY. Unset → off, no key++ // file, nothing sent; a published plan tells the engines its page changed. Tests inject a spy.++ const indexnow = opts.indexnow !== undefined ? opts.indexnow : createIndexNow({++ host: config.baseUrl, key: config.indexnow.key, fetch: fetchImpl, log,++ });++ const domain = createDomain({ db, config, outbox, billing, now, log, indexnow });const apiAuth = createApiAuth({ config, keys, userAuth });const release = require('openvibe-shared/release').createRelease({ service: 'vip', root: path.join(__dirname, '..') });setRelease(release.release);@@@ -109,6 +115,9 @@ async function createApp(opts = {}) {// This site's own pinned copy of the OpenVibe Frame's browser files (openvibe-shared/serve).app.use('/shared', require('openvibe-shared/serve').handler());++ // GET /<key>.txt — the IndexNow key file (mounted only when a key is configured; it serves itself++ // and steps aside for every other path). Before static, so the key can never be shadowed by a file.++ if (indexnow.enabled) app.use(indexnow.keyFile);app.use(express.static(PUBLIC_DIR, {index: false, redirect: false,setHeaders(res, filePath) {@@@ -133,7 +142,7 @@ async function createApp(opts = {}) {return res.status(500).type('html').send(layout.page({ title: 'Error', robots: 'noindex', body: pages.errorPage({ status: 500, title: 'Something went wrong', message: 'This one is on us. Please try again.' }) }));});-- Object.assign(app.locals, { config, db, domain, keys, outbox, billing, consumer, metrics });++ Object.assign(app.locals, { config, db, domain, keys, outbox, billing, consumer, metrics, indexnow });return app;}@@@ -105,6 +105,11 @@ function loadConfig(env = process.env) {minute: Math.max(1, int(env.VIP_LIMITS_MINUTE, 120)),hour: Math.max(1, int(env.VIP_LIMITS_HOUR, 3000)),},++++ // IndexNow (openvibe-shared/indexnow): a key makes search engines recrawl a creator's page the++ // moment a plan is published, edited or archived (the key file is served at /<key>.txt). Unset:++ // off, no key file, nothing sent. Tests and drills never set it.++ indexnow: { key: String(env.INDEXNOW_KEY || '').trim() },};}@@@ -11,10 +11,10 @@ const { createPolicies } = require('./policies');const { createCheckout } = require('./checkout');const { createCards } = require('./cards');--function createDomain({ db, config, outbox, billing, now = () => Date.now(), log = console }) {++function createDomain({ db, config, outbox, billing, now = () => Date.now(), log = console, indexnow = null }) {const creators = createCreators({ db, now });const perks = createPerks({ db, now });-- const plans = createPlans({ db, now, outbox, creators, perks });++ const plans = createPlans({ db, now, outbox, creators, perks, baseUrl: config.baseUrl, indexnow });const memberships = createMemberships({ db, now, creators, plans });const periods = createPeriods({ db, now, log });const entitlements = createEntitlements({ db, now, config, billing, outbox, memberships, plans, log });@@@ -19,13 +19,28 @@ const { fail, prefixedId, iso, text, slug, slugify, json } = require('../util');const BILLING_KINDS = ['channel_subscription'];--function createPlans({ db, now, outbox, creators, perks }) {++function createPlans({ db, now, outbox, creators, perks, baseUrl = '', indexnow = null }) {const byId = async (id) => await db.prepare('SELECT * FROM vip_plans WHERE id = ?').get(id) || null;const version = async (id) => await db.prepare('SELECT * FROM vip_plan_versions WHERE id = ?').get(id) || null;const latestVersion = async (planId) => await db.prepare('SELECT * FROM vip_plan_versions WHERE plan_id = ? ORDER BY version DESC LIMIT 1').get(planId) || null;const versions = async (planId) => await db.prepare('SELECT * FROM vip_plan_versions WHERE plan_id = ? ORDER BY version DESC').all(planId);const versionPerks = async (versionId) => await db.prepare('SELECT * FROM vip_plan_perks WHERE plan_version_id = ? ORDER BY position').all(versionId);++ /**++ * IndexNow (openvibe-shared/indexnow): a published plan's page, its terms history and the sitemap++ * all changed, so tell the engines. Unset key, a draft, an unchanged edit or a replay changes++ * nothing and never pings. Never throws: a failed ping must never take a publish down.++ */++ async function pingPublic(plan) {++ try {++ if (!indexnow || !indexnow.enabled || !baseUrl || !plan) return;++ const creator = await creators.byId(plan.creator_id);++ if (!creator || !creator.username) return; // network plans have no public page++ const page = `${baseUrl}/${encodeURIComponent(creator.username)}`;++ indexnow.pingSoon([page, `${page}/plans/${encodeURIComponent(plan.slug)}`, `${baseUrl}/sitemap.xml`]);++ } catch { /* a ping never fails a publish */ }++ }++function cleanBenefits(v) {if (v == null) return null;const arr = Array.isArray(v) ? v : String(v).split('\n');@@@ -87,7 +102,7 @@ function createPlans({ db, now, outbox, creators, perks }) {const at = iso(now());const id = prefixedId('vpl', now());try {-- return await db.tx(async () => {++ const plan = await db.tx(async () => {await db.prepare(`INSERT INTO vip_plans (id, creator_id, slug, status, billing_kind, current_version_id, latest_version, created_by, created_at, updated_at)VALUES (?, ?, ?, ?, ?, NULL, 0, ?, ?, ?)`).run(id, creatorId, planSlug, publish ? 'published' : 'draft', kind, actor, at, at);const v = await insertVersion(await byId(id), {@@@ -97,6 +112,8 @@ function createPlans({ db, now, outbox, creators, perks }) {if (publish) await emitPublished(await byId(id), v, traceparent);return await byId(id);});++ if (publish) await pingPublic(plan);++ return plan;} catch (e) { return publishConflict(e); }}@@@ -116,11 +133,13 @@ function createPlans({ db, now, outbox, creators, perks }) {&& JSON.stringify(perkRows.map((p) => p.id)) === JSON.stringify((await versionPerks(last.id)).map((pp) => pp.perk_id));if (same) return { plan, version: last, unchanged: true };const publish = plan.status === 'published';-- return await db.tx(async () => {++ const out = await db.tx(async () => {const v = await insertVersion(plan, { ...next, perkRows, changeNote: text(changeNote, 'change_note', 300), actor, publish });if (publish) await emitPublished(await byId(planId), v, traceparent);return { plan: await byId(planId), version: v, unchanged: false };});++ if (publish) await pingPublic(out.plan);++ return out;}async function publish(planId, { traceparent } = {}) {@@@ -129,7 +148,7 @@ function createPlans({ db, now, outbox, creators, perks }) {if (plan.status === 'published') return { plan, replay: true };if (plan.status === 'archived') fail(409, 'vip.plan_archived', 'an archived plan cannot be published again; create a new plan');try {-- return await db.tx(async () => {++ const out = await db.tx(async () => {const v = await latestVersion(planId);const at = iso(now());await db.prepare('UPDATE vip_plan_versions SET published_at = ? WHERE id = ? AND published_at IS NULL').run(at, v.id);@@@ -137,6 +156,8 @@ function createPlans({ db, now, outbox, creators, perks }) {await emitPublished(await byId(planId), await version(v.id), traceparent);return { plan: await byId(planId), replay: false };});++ await pingPublic(out.plan);++ return out;} catch (e) { return publishConflict(e); }}@@@ -147,7 +168,9 @@ function createPlans({ db, now, outbox, creators, perks }) {if (plan.status === 'archived') return plan;const at = iso(now());await db.prepare("UPDATE vip_plans SET status = 'archived', archived_at = ?, updated_at = ? WHERE id = ?").run(at, at, planId);-- return await byId(planId);++ const archived = await byId(planId);++ if (plan.status === 'published') await pingPublic(archived); // the creator page and the sitemap lose the plan++ return archived;}async function list({ creatorId, includeDrafts = false, includeArchived = false }) {@@@ -8,7 +8,7 @@* GET /dashboard[?as=network] creator dashboard: plans, versions, perks, members, gated resources* GET /:username a creator's plan page (join through Billing)* GET /:username/plans/:slug that plan's public terms history-- * GET /robots.txt, /sitemap.xml, /llms.txt++ * GET /robots.txt, /sitemap.xml, /llms.txt, /llms-full.txt** Every POST needs the signed-in viewer and the form's anti-forgery token; it acts through the same* domain calls as the API and redirects back with a notice (?ok= / ?error=).@@@ -23,10 +23,19 @@ const { viewerMiddleware } = require('./session');const { createForms } = require('./forms');const pages = require('./pages');--const RESERVED = new Set(['me', 'dashboard', 'auth', 'api', 'internal', 'css', 'js', 'terms', 'privacy', 'dmca', 'robots.txt', 'sitemap.xml', 'llms.txt', 'release.json', 'metrics', 'favicon.ico', 'embed']);++const RESERVED = new Set(['me', 'dashboard', 'auth', 'api', 'internal', 'css', 'js', 'terms', 'privacy', 'dmca', 'robots.txt', 'sitemap.xml', 'llms.txt', 'llms-full.txt', 'release.json', 'metrics', 'favicon.ico', 'embed']);// The sitemap's lastmod: when this server booted — a real date, computed once, never per request.const BOOT_AT = new Date().toISOString();++// The public description /llms.txt and /llms-full.txt share. llmstxt.org: a plain-markdown map of++// the site for language-model crawlers — public data only, never the viewer.++const LLMS_NAME = 'OpenVibe.VIP';++const LLMS_SUMMARY = 'OpenVibe.VIP: memberships for creators — plans, perks and benefits recognised across every OpenVibe site.';++const LLMS_DETAILS = 'A creator page lists the plans and perks a creator offers and how to join through Billing. Every plan keeps its published terms history, and a membership always points at the version it was bought under. Reading any public page needs no account; joining needs a signed-in person. Memberships, the dashboard, sign-in, embeds and the API are per-person and are never listed here.';++// /llms-full.txt changes only when a creator or a plan does, so it is built from the database at++// most once an hour (the browser keeps it the same 3600 s).++const LLMS_FULL_TTL_MS = 3600 * 1000;++function createWebRoutes({ domain, config, layout, userAuth }) {const router = express.Router();const { creators, plans, perks, memberships, entitlements, policies, checkout, billing } = domain;@@@ -88,9 +97,9 @@ function createWebRoutes({ domain, config, layout, userAuth }) {});// /llms.txt (llmstxt.org): a plain-markdown map of the site for language-model crawlers.router.get('/llms.txt', (req, res) => res.type('text/plain').send(seo.llmsTxt({-- name: 'OpenVibe.VIP',-- summary: 'OpenVibe.VIP: memberships for creators — plans, perks and benefits recognised across every OpenVibe site.',-- details: 'A creator page lists the plans and perks a creator offers and how to join through Billing. Every plan keeps its published terms history, and a membership always points at the version it was bought under. Reading any public page needs no account; joining needs a signed-in person. Memberships, the dashboard, sign-in, embeds and the API are per-person and are never listed here.',++ name: LLMS_NAME,++ summary: LLMS_SUMMARY,++ details: LLMS_DETAILS,sections: [{ title: 'Start here', links: [{ title: 'Creators with published plans', url: `${config.baseUrl}/`, note: 'every creator on OpenVibe.VIP, with their public plans' },@@@ -99,10 +108,64 @@ function createWebRoutes({ domain, config, layout, userAuth }) {{ title: 'Machine-readable', links: [{ title: 'Sitemap', url: `${config.baseUrl}/sitemap.xml`, note: 'the public pages, with lastmod' },{ title: 'robots.txt', url: `${config.baseUrl}/robots.txt` },++ { title: 'llms-full.txt', url: `${config.baseUrl}/llms-full.txt`, note: 'the full text of every public page' },] },],})));++ // /llms-full.txt: /llms.txt with the full plain text of every public page — the home copy, each++ // creator's plan page and every published plan's terms. Built from public data only and cached++ // in memory for an hour; the response says the same, so caches and crawlers may keep it too.++ function termsText(v) {++ const out = [];++ if (v.description) out.push(v.description);++ if (v.benefits.length) out.push(`Benefits: ${v.benefits.join('; ')}.`);++ if (v.perks.length) out.push(`Perks: ${v.perks.map((p) => p.name).join(', ')}.`);++ if (v.change_note) out.push(`Change: ${v.change_note}.`);++ return out.join('\n');++ }++ async function buildLlmsFull() {++ const creatorPages = [];++ const planPages = [];++ for (const row of await creators.listPublic({ limit: 5000 })) {++ const c = creators.present(row);++ const name = c.display_name || c.username;++ const planRows = await Promise.all((await plans.list({ creatorId: c.id })).map(async (p) => await plans.present(p, { withVersions: true })));++ const about = [`${name} (@${c.username}) offers membership plans on OpenVibe.VIP.`];++ if (c.bio) about.push(c.bio);++ for (const p of planRows) {++ const v = p.current_version;++ about.push(`Plan ${v.name} (version ${v.version})${v.description ? `: ${v.description}` : ''}`);++ }++ creatorPages.push({ title: `${name} memberships`, url: `/${c.username}`, text: about.join('\n\n') });++ for (const p of planRows) {++ const body = [`The published terms of ${name}'s ${p.current_version.name} plan on OpenVibe.VIP. Each edit is a new version; a member keeps the version they joined under.`];++ for (const v of (p.versions || []).filter((x) => x.published_at)) {++ const terms = termsText(v);++ body.push(`Version ${v.version}${v.published_at ? ` (published ${String(v.published_at).slice(0, 10)})` : ''}${terms ? `\n${terms}` : ''}`);++ }++ planPages.push({ title: `${name}: ${p.current_version.name} terms`, url: `/${c.username}/plans/${p.slug}`, text: body.join('\n\n') });++ }++ }++ return seo.llmsFull({++ site: { name: LLMS_NAME, url: config.baseUrl },++ summary: `${LLMS_SUMMARY} ${LLMS_DETAILS}`,++ sections: [++ { title: 'Home', pages: [{ title: `${LLMS_NAME} home`, url: '/', text: `${LLMS_SUMMARY}\n\n${LLMS_DETAILS}\n\nThe home page lists every creator with a published plan.` }] },++ { title: 'Creators', pages: creatorPages },++ { title: 'Plans', pages: planPages },++ ],++ });++ }++ let llmsFullCache = { at: 0, value: null };++ router.get('/llms-full.txt', async (req, res, next) => {++ try {++ if (!llmsFullCache.value || domain.now() - llmsFullCache.at >= LLMS_FULL_TTL_MS) llmsFullCache = { at: domain.now(), value: await buildLlmsFull() };++ res.setHeader('Cache-Control', 'public, max-age=3600');++ res.type('text/plain').send(llmsFullCache.value);++ } catch (e) { next(e); }++ });++// What shipped on OpenVibe.VIP: the shared update log every OpenVibe site has.router.get('/updates', (req, res) => send(res, 200, { viewer: req.viewer, canonicalPath: '/updates', title: 'What shipped on OpenVibe.VIP', body: frame.updatesBody({ service: 'vip', siteName: 'OpenVibe.VIP' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>` }));router.get('/', async (req, res) => send(res, 200, {@@@ -226,6 +226,16 @@ const { test, run } = harness('api');assert.match(llms.headers.get('content-type'), /^text\/plain/);assert.match(llms.text, /^# /, 'llms.txt starts with a markdown heading');assert.match(llms.text, /\[Creators with published plans\]\(http:\/\/vip\.test\/\)/, 'llms.txt lists the main public pages');++ const full = await t.call('GET', '/llms-full.txt', { token: null });++ assert.strictEqual(full.status, 200);++ assert.match(full.headers.get('content-type'), /^text\/plain/);++ assert.strictEqual(full.headers.get('cache-control'), 'public, max-age=3600', 'llms-full.txt is cached for an hour');++ assert.match(full.text, /^# OpenVibe\.VIP/, 'llms-full.txt starts with the llms.txt header');++ assert.match(full.text, /URL: http:\/\/vip\.test\/\n/, 'the home page');++ assert.match(full.text, /URL: http:\/\/vip\.test\/lena\n/, 'the public creator page');++ assert.match(full.text, new RegExp(`URL: http:\\/\\/vip\\.test\\/lena\\/plans\\/${plan.slug}\\n`), 'the plan terms page');++ assert.match(full.text, /Front row/, 'the full plan text');++ assert.match(full.text, /Monthly stream/, 'a benefit of the current version');});test('every rendered page carries the boost marker, boost script (data-main) and the {path} sign-in template', async () => {@@@ -47,7 +47,7 @@ async function boot(opts = {}) {const log = { log: (...a) => logs.push(a.join(' ')), warn: (...a) => logs.push(a.join(' ')), error: (...a) => logs.push(a.join(' ')), debug() {} };// One database per boot (PGlite, or VIP_TEST_STORE=pg: the containers), dropped when the boot closes.const testdb = opts.testdb || await require('./db').testDb();-- const app = await createApp({ config, db: testdb.db, now: clock.now, log });++ const app = await createApp({ config, db: testdb.db, now: clock.now, log, indexnow: opts.indexnow });const server = await new Promise((resolve) => { const s = http.createServer(app); s.listen(0, '127.0.0.1', () => resolve(s)); });const base = `http://127.0.0.1:${server.address().port}`;const domain = app.locals.domain;nnew file mode 100644@@@ -0,0 +1,80 @@++'use strict';++/**++ * IndexNow (openvibe-shared/indexnow): INDEXNOW_KEY unset → the feature is off (no key route,++ * nothing sent). With a key, the key file is served at /<key>.txt as text/plain and publishing,++ * editing or archiving a plan pings the engines with the creator's page, the plan's terms page and++ * the sitemap. A draft, an unchanged edit and a replay never ping.++ */++const assert = require('assert');++const { boot, harness } = require('./helpers/app');++++const { test, run } = harness('indexnow');++const KEY = 'k'.repeat(32);++++(async () => {++ const off = await boot();++ const on = await boot({ env: { INDEXNOW_KEY: KEY } });++++ // A spy in place of the module's HTTP send: records every pingSoon batch.++ const pings = [];++ const spy = {++ enabled: true,++ keyFile: (_req, _res, next) => next(),++ pingSoon: (urls) => { const a = Array.isArray(urls) ? urls : [urls]; pings.push(...a); return a.length; },++ ping: async () => ({ sent: 0, status: 0 }),++ flush: async () => ({ sent: 0, status: 0 }),++ };++ const t = await boot({ indexnow: spy });++ const creator = t.network.newUser('alice', { role: 'streamer' });++ let plan;++++ test('without a key IndexNow is off: no key route and nothing sent', async () => {++ assert.strictEqual(off.app.locals.indexnow.enabled, false);++ const res = await off.call('GET', `/${KEY}.txt`, { token: null });++ assert.strictEqual(res.status, 404, res.text);++ });++++ test('with a key the key file answers text/plain with the key, before static', async () => {++ assert.strictEqual(on.app.locals.indexnow.enabled, true);++ const res = await on.call('GET', `/${KEY}.txt`, { token: null });++ assert.strictEqual(res.status, 200, res.text);++ assert.match(res.headers.get('content-type'), /^text\/plain/);++ assert.strictEqual(res.headers.get('cache-control'), 'public, max-age=3600');++ assert.strictEqual(res.text, KEY);++ });++++ test('a draft never pings', async () => {++ const res = await t.call('POST', '/api/v1/plans', { user: creator, body: { name: 'First light' } });++ assert.strictEqual(res.status, 201, res.text);++ plan = res.json.plan;++ assert.deepStrictEqual(pings, []);++ });++++ test('publishing pings the creator page, the terms page and the sitemap', async () => {++ const res = await t.call('POST', `/api/v1/plans/${plan.id}/publish`, { user: creator });++ assert.strictEqual(res.status, 200, res.text);++ assert.ok(pings.includes('http://vip.test/alice'), JSON.stringify(pings));++ assert.ok(pings.includes(`http://vip.test/alice/plans/${plan.slug}`), JSON.stringify(pings));++ assert.ok(pings.includes('http://vip.test/sitemap.xml'), JSON.stringify(pings));++ });++++ test('an unchanged edit does not ping; a real edit of a published plan does', async () => {++ pings.length = 0;++ await t.call('PATCH', `/api/v1/plans/${plan.id}`, { user: creator, body: { name: 'First light' } });++ assert.deepStrictEqual(pings, [], 'an unchanged edit is not a change');++ const res = await t.call('PATCH', `/api/v1/plans/${plan.id}`, { user: creator, body: { description: 'Now with more.' } });++ assert.strictEqual(res.status, 200, res.text);++ assert.ok(pings.includes('http://vip.test/alice'), JSON.stringify(pings));++ assert.ok(pings.includes(`http://vip.test/alice/plans/${plan.slug}`), JSON.stringify(pings));++ });++++ test('archiving pings the creator page and the sitemap', async () => {++ pings.length = 0;++ const res = await t.call('POST', `/api/v1/plans/${plan.id}/archive`, { user: creator });++ assert.strictEqual(res.status, 200, res.text);++ assert.ok(pings.includes('http://vip.test/alice'), JSON.stringify(pings));++ assert.ok(pings.includes('http://vip.test/sitemap.xml'), JSON.stringify(pings));++ });++++ await run().finally(async () => { await t.close(); await on.close(); await off.close(); });++})();nnew file mode 100644@@@ -0,0 +1,48 @@++'use strict';++/**++ * /llms-full.txt: the /llms.txt header plus the full text of every public page — the home copy,++ * each creator's plan page and every published plan's terms. It is public data, cached for an hour.++ */++const assert = require('assert');++const { boot, harness } = require('./helpers/app');++++const { test, run } = harness('llms-full');++++(async () => {++ const t = await boot();++ const creator = t.network.newUser('dana', { role: 'streamer' });++ let plan;++++ test('the full text covers the home, the creator page and the plan terms', async () => {++ const res = await t.call('POST', '/api/v1/plans', { user: creator, body: { name: 'Backstage', description: 'First description', benefits: ['A badge'], publish: true } });++ assert.strictEqual(res.status, 201, res.text);++ plan = res.json.plan;++ const full = await t.call('GET', '/llms-full.txt', { token: null });++ assert.strictEqual(full.status, 200);++ assert.match(full.headers.get('content-type'), /^text\/plain/);++ assert.strictEqual(full.headers.get('cache-control'), 'public, max-age=3600');++ assert.match(full.text, /^# OpenVibe\.VIP/, 'the llms.txt header comes first');++ assert.match(full.text, /## Home/);++ assert.match(full.text, /## Creators/);++ assert.match(full.text, /## Plans/);++ assert.match(full.text, /URL: http:\/\/vip\.test\/\n/, 'the home page');++ assert.match(full.text, /URL: http:\/\/vip\.test\/dana\n/, 'the public creator page');++ assert.match(full.text, new RegExp(`URL: http:\\/\\/vip\\.test\\/dana\\/plans\\/${plan.slug}\\n`), 'the plan terms page');++ assert.match(full.text, /First description/, 'the full plan text, never clipped');++ assert.match(full.text, /@dana/);++ });++++ test('the body is built once and cached for an hour', async () => {++ const before = (await t.call('GET', '/llms-full.txt', { token: null })).text;++ const edit = await t.call('PATCH', `/api/v1/plans/${plan.id}`, { user: creator, body: { description: 'Second description' } });++ assert.strictEqual(edit.status, 200, edit.text);++ const cached = (await t.call('GET', '/llms-full.txt', { token: null })).text;++ assert.strictEqual(cached, before, 'within the hour the cached copy is served unchanged');++ assert.doesNotMatch(cached, /Second description/);++ t.clock.advance(3600 * 1000 + 1);++ const fresh = (await t.call('GET', '/llms-full.txt', { token: null })).text;++ assert.match(fresh, /Second description/, 'after the hour the body is rebuilt from the database');++ });++++ await run().finally(async () => await t.close());++})();
@@ -70,6 +70,12 @@ VIP_EVENTS_SECRET=
@@ -70,6 +70,12 @@ VIP_EVENTS_SECRET=
# How long a creator's member count (from Billing) is reused
# How long a creator's member count (from Billing) is reused
# VIP_MEMBER_COUNT_TTL_MS=300000
# VIP_MEMBER_COUNT_TTL_MS=300000
+# ── IndexNow (openvibe-shared/indexnow) ──────────────────
+# With a key (8-128 hex or alphanumeric characters, what IndexNow's own tools generate), search
+# engines are told the moment a published plan appears, changes or is archived (the key file is served
+# at /<key>.txt). Unset: off — nothing is mounted and nothing is sent.
+#INDEXNOW_KEY=
+
# ── Per-actor limits (server/api/actor-limits.js) ───────
# ── Per-actor limits (server/api/actor-limits.js) ───────
# Reads one caller (a service principal, a person, else an address) may make to /api/v1 per minute
# Reads one caller (a service principal, a person, else an address) may make to /api/v1 per minute
# and per hour; past them 429 rate_limited with Retry-After. Entitlement checks, policy evaluation,
# and per hour; past them 429 rate_limited with Retry-After. Entitlement checks, policy evaluation,
@@ -321,11 +321,30 @@ memberships* card gives the creator the badge and widget HTML and the card URL.
@@ -321,11 +321,30 @@ memberships* card gives the creator the badge and widget HTML and the card URL.
Billing's price and period from `GET /api/v1/rates`, join form) · `/:username/plans/:slug` public terms
Billing's price and period from `GET /api/v1/rates`, join form) · `/:username/plans/:slug` public terms
history · `/me` your memberships, the version you joined under, cancel, badge preference ·
history · `/me` your memberships, the version you joined under, cancel, badge preference ·
`/dashboard` plans, versions, perks and bindings, members, members-only resources, the badge and
`/dashboard` plans, versions, perks and bindings, members, members-only resources, the badge and
-widget to share and the member-count switch (`?as=network` for staff) · `/embed/:username/…` (above) · `/robots.txt`, `/sitemap.xml` · `/terms`, `/privacy`, `/dmca`
+widget to share and the member-count switch (`?as=network` for staff) · `/embed/:username/…` (above) · `/robots.txt`, `/sitemap.xml`, `/llms.txt`, `/llms-full.txt` · `/terms`, `/privacy`, `/dmca`
(openvibe-shared legal). Navbar from `https://openvibe.network/shared/navbar.js`, SSR footer and
(openvibe-shared legal). Navbar from `https://openvibe.network/shared/navbar.js`, SSR footer and
`<noscript>` navigation from openvibe-shared. Forms carry HMAC anti-forgery tokens
`<noscript>` navigation from openvibe-shared. Forms carry HMAC anti-forgery tokens
(`VIP_FORM_SECRET`); cookies are SameSite=Lax. No feed: plans are not a publication stream.
(`VIP_FORM_SECRET`); cookies are SameSite=Lax. No feed: plans are not a publication stream.
+### Crawlers and AI readers
+
+`/robots.txt` and `/sitemap.xml` come from openvibe-shared/seo (the sitemap lists the home and every
+creator with a published plan). `/llms.txt` is the site map for language models; `/llms-full.txt` is
+the same header followed by the full plain text of the home, every public creator page and every
+published plan's terms (`seo.llmsFull`, built from public data only, cached an hour in memory and with
+`Cache-Control: public, max-age=3600`). Every page's head carries an `ai-summary` meta and WebPage
+JSON-LD through openvibe-shared/shell.
+
+### IndexNow (openvibe-shared/indexnow)
+
+With `INDEXNOW_KEY` set (8–128 hex or alphanumeric characters, what IndexNow's own tools generate),
+the key file is served at `/<key>.txt` as `text/plain` (before static, so a file cannot shadow it) and
+a plan that changes tells the engines: publishing, editing a published plan and archiving ping
+`api.indexnow.org` with the creator's page, the plan's terms page and `/sitemap.xml`; the module
+batches and debounces (one POST per 30 s window). A draft, an unchanged edit, a replayed publish and
+network plans (no public creator page) never ping, and a failed ping never takes a publish down.
+Unset: off — the key route is not mounted and nothing is sent (`test/indexnow.test.js`).
+
## Before and after the Billing cutover
## Before and after the Billing cutover
Billing runs in shadow mode until the Wave 8 cutover (Live stays authoritative for money,
Billing runs in shadow mode until the Wave 8 cutover (Live stays authoritative for money,
@@ -380,6 +399,8 @@ Reporting a vulnerability: [SECURITY.md](SECURITY.md).
@@ -380,6 +399,8 @@ Reporting a vulnerability: [SECURITY.md](SECURITY.md).
| gated-resource evaluation fails closed (no rule, unknown, perk missing, guest, VIP down in the client) | `test/policies.test.js` |
| gated-resource evaluation fails closed (no rule, unknown, perk missing, guest, VIP down in the client) | `test/policies.test.js` |
| VIP works with Live unavailable | no code path calls Live; every test runs without a Live stub |
| VIP works with Live unavailable | no code path calls Live; every test runs without a Live stub |
| a creator's membership card, badge and widget are public-data only, the same for everyone, embeddable and inert | `test/embeds.test.js` (published plans only, no subjects or binding configs, no cookie read or set, CORS, SVG escaping and sandbox, the widget's frame-ancestors and hashed style, the creator's member-count switch, Billing outage keeps the last count or shows none) |
| a creator's membership card, badge and widget are public-data only, the same for everyone, embeddable and inert | `test/embeds.test.js` (published plans only, no subjects or binding configs, no cookie read or set, CORS, SVG escaping and sandbox, the widget's frame-ancestors and hashed style, the creator's member-count switch, Billing outage keeps the last count or shows none) |
+| `/llms-full.txt` carries the full text of the home, creator and plan pages, and is cached for an hour | `test/llms-full.test.js`, `test/api.test.js` (discovery artifacts) |
+| IndexNow is off without `INDEXNOW_KEY` (no key route, nothing sent); with one the key file is served at `/<key>.txt` as `text/plain` and publishing, editing and archiving a plan ping the creator page, the terms page and the sitemap; a draft and an unchanged edit never ping | `test/indexnow.test.js` |
Not yet demonstrated (needs the other services): convergence **across consuming products** end to end
Not yet demonstrated (needs the other services): convergence **across consuming products** end to end
(Chat, Community, Blog and Wiki call VIP in code; Live does not), and a real Billing/Events round trip
(Chat, Community, Blog and Wiki call VIP in code; Live does not), and a real Billing/Events round trip
@@ -23,7 +23,8 @@
@@ -23,7 +23,8 @@
"14 vip.* capabilities, the service manifest and its events registered in openvibe-contracts v0.19.0 (this repository pins v0.92.0)",
"14 vip.* capabilities, the service manifest and its events registered in openvibe-contracts v0.19.0 (this repository pins v0.92.0)",
"per-actor limits on /api/v1 (openvibe-sdk/limits)",
"per-actor limits on /api/v1 (openvibe-sdk/limits)",
"the Billing client (server/billing-client.js) is a thin wrapper over openvibe-sdk/commerce (openvibe-sdk v0.28.0), shared with Tips: same capabilities, idempotency keys and retry-once-on-401",
"the Billing client (server/billing-client.js) is a thin wrapper over openvibe-sdk/commerce (openvibe-sdk v0.28.0), shared with Tips: same capabilities, idempotency keys and retry-once-on-401",
- "renewal/refund convergence via Billing events: a per-period charge ledger (vip_membership_periods, unique per Billing transaction; reversals and reversal-before-grant tombstones), a past_due renewal grace that never authorizes (answer.grace), and a period audit that doubts a paid period without a record (migration 0002; docs/cutover-0002-renewal-periods.md)"
+ "renewal/refund convergence via Billing events: a per-period charge ledger (vip_membership_periods, unique per Billing transaction; reversals and reversal-before-grant tombstones), a past_due renewal grace that never authorizes (answer.grace), and a period audit that doubts a paid period without a record (migration 0002; docs/cutover-0002-renewal-periods.md)",
+ "crawler and AI-reader artifacts: /llms-full.txt serves the full text of the home, public creator and plan-terms pages (openvibe-shared/seo, cached 3600 s), and IndexNow (openvibe-shared/indexnow, INDEXNOW_KEY) serves its key file at /<key>.txt and pings the creator page, terms page and sitemap when a plan is published, edited or archived"
],
],
"not_yet": [
"not_yet": [
"not launched: openvibe.vip is still an OpenVibe.Sites placeholder and the service answers on loopback only",
"not launched: openvibe.vip is still an OpenVibe.Sites placeholder and the service answers on loopback only",
@@ -40,6 +41,8 @@
@@ -40,6 +41,8 @@
"openvibe-vip/client with the product cache and its convergence bound",
"openvibe-vip/client with the product cache and its convergence bound",
"creator plan pages, terms history, member status and creator dashboard",
"creator plan pages, terms history, member status and creator dashboard",
"public membership card, badge and embeddable widget",
"public membership card, badge and embeddable widget",
+ "/llms.txt and /llms-full.txt for language-model crawlers, cached an hour",
+ "IndexNow: a key serves /<key>.txt and pings plan page changes and the sitemap",
"SDK outbox events vip.plan.published and vip.membership.changed",
"SDK outbox events vip.plan.published and vip.membership.changed",
"per-actor limits (openvibe-sdk/limits)"
"per-actor limits (openvibe-sdk/limits)"
],
],
@@ -20,6 +20,7 @@ const express = require('express');
@@ -20,6 +20,7 @@ const express = require('express');
const cookieParser = require('cookie-parser');
const cookieParser = require('cookie-parser');
const { http } = require('openvibe-contracts');
const { http } = require('openvibe-contracts');
const cache = require('openvibe-shared/cache-policy');
const cache = require('openvibe-shared/cache-policy');
+const { createIndexNow } = require('openvibe-shared/indexnow');
const { loadConfig } = require('./config');
const { loadConfig } = require('./config');
const { openDb } = require('./db');
const { openDb } = require('./db');
const { createKeyProvider, createUserAuth } = require('./network');
const { createKeyProvider, createUserAuth } = require('./network');
@@ -52,7 +53,12 @@ async function createApp(opts = {}) {
@@ -52,7 +53,12 @@ async function createApp(opts = {}) {
const userAuth = createUserAuth(config, keys);
const userAuth = createUserAuth(config, keys);
const billing = opts.billing || createBillingClient(config, { fetchImpl });
const billing = opts.billing || createBillingClient(config, { fetchImpl });
const outbox = opts.outbox || createVipOutbox({ db, config, fetchImpl: opts.eventsFetch, now, log });
const outbox = opts.outbox || createVipOutbox({ db, config, fetchImpl: opts.eventsFetch, now, log });
- const domain = createDomain({ db, config, outbox, billing, now, log });
+ // IndexNow (openvibe-shared/indexnow): created once at boot from INDEXNOW_KEY. Unset → off, no key
+ // file, nothing sent; a published plan tells the engines its page changed. Tests inject a spy.
+ const indexnow = opts.indexnow !== undefined ? opts.indexnow : createIndexNow({
+ host: config.baseUrl, key: config.indexnow.key, fetch: fetchImpl, log,
+ });
+ const domain = createDomain({ db, config, outbox, billing, now, log, indexnow });
const apiAuth = createApiAuth({ config, keys, userAuth });
const apiAuth = createApiAuth({ config, keys, userAuth });
const release = require('openvibe-shared/release').createRelease({ service: 'vip', root: path.join(__dirname, '..') });
const release = require('openvibe-shared/release').createRelease({ service: 'vip', root: path.join(__dirname, '..') });
setRelease(release.release);
setRelease(release.release);
@@ -109,6 +115,9 @@ async function createApp(opts = {}) {
@@ -109,6 +115,9 @@ async function createApp(opts = {}) {
// This site's own pinned copy of the OpenVibe Frame's browser files (openvibe-shared/serve).
// This site's own pinned copy of the OpenVibe Frame's browser files (openvibe-shared/serve).
app.use('/shared', require('openvibe-shared/serve').handler());
app.use('/shared', require('openvibe-shared/serve').handler());
+ // GET /<key>.txt — the IndexNow key file (mounted only when a key is configured; it serves itself
+ // and steps aside for every other path). Before static, so the key can never be shadowed by a file.
+ if (indexnow.enabled) app.use(indexnow.keyFile);
app.use(express.static(PUBLIC_DIR, {
app.use(express.static(PUBLIC_DIR, {
index: false, redirect: false,
index: false, redirect: false,
setHeaders(res, filePath) {
setHeaders(res, filePath) {
@@ -133,7 +142,7 @@ async function createApp(opts = {}) {
@@ -133,7 +142,7 @@ async function createApp(opts = {}) {
return res.status(500).type('html').send(layout.page({ title: 'Error', robots: 'noindex', body: pages.errorPage({ status: 500, title: 'Something went wrong', message: 'This one is on us. Please try again.' }) }));
return res.status(500).type('html').send(layout.page({ title: 'Error', robots: 'noindex', body: pages.errorPage({ status: 500, title: 'Something went wrong', message: 'This one is on us. Please try again.' }) }));
});
});
- Object.assign(app.locals, { config, db, domain, keys, outbox, billing, consumer, metrics });
+ Object.assign(app.locals, { config, db, domain, keys, outbox, billing, consumer, metrics, indexnow });
return app;
return app;
}
}
@@ -105,6 +105,11 @@ function loadConfig(env = process.env) {
@@ -105,6 +105,11 @@ function loadConfig(env = process.env) {
minute: Math.max(1, int(env.VIP_LIMITS_MINUTE, 120)),
minute: Math.max(1, int(env.VIP_LIMITS_MINUTE, 120)),
hour: Math.max(1, int(env.VIP_LIMITS_HOUR, 3000)),
hour: Math.max(1, int(env.VIP_LIMITS_HOUR, 3000)),
},
},
+
+ // IndexNow (openvibe-shared/indexnow): a key makes search engines recrawl a creator's page the
+ // moment a plan is published, edited or archived (the key file is served at /<key>.txt). Unset:
+ // off, no key file, nothing sent. Tests and drills never set it.
+ indexnow: { key: String(env.INDEXNOW_KEY || '').trim() },
};
};
}
}
@@ -11,10 +11,10 @@ const { createPolicies } = require('./policies');
@@ -11,10 +11,10 @@ const { createPolicies } = require('./policies');
const { createCheckout } = require('./checkout');
const { createCheckout } = require('./checkout');
const { createCards } = require('./cards');
const { createCards } = require('./cards');
-function createDomain({ db, config, outbox, billing, now = () => Date.now(), log = console }) {
+function createDomain({ db, config, outbox, billing, now = () => Date.now(), log = console, indexnow = null }) {
const creators = createCreators({ db, now });
const creators = createCreators({ db, now });
const perks = createPerks({ db, now });
const perks = createPerks({ db, now });
- const plans = createPlans({ db, now, outbox, creators, perks });
+ const plans = createPlans({ db, now, outbox, creators, perks, baseUrl: config.baseUrl, indexnow });
const memberships = createMemberships({ db, now, creators, plans });
const memberships = createMemberships({ db, now, creators, plans });
const periods = createPeriods({ db, now, log });
const periods = createPeriods({ db, now, log });
const entitlements = createEntitlements({ db, now, config, billing, outbox, memberships, plans, log });
const entitlements = createEntitlements({ db, now, config, billing, outbox, memberships, plans, log });
@@ -19,13 +19,28 @@ const { fail, prefixedId, iso, text, slug, slugify, json } = require('../util');
@@ -19,13 +19,28 @@ const { fail, prefixedId, iso, text, slug, slugify, json } = require('../util');
const BILLING_KINDS = ['channel_subscription'];
const BILLING_KINDS = ['channel_subscription'];
-function createPlans({ db, now, outbox, creators, perks }) {
+function createPlans({ db, now, outbox, creators, perks, baseUrl = '', indexnow = null }) {
const byId = async (id) => await db.prepare('SELECT * FROM vip_plans WHERE id = ?').get(id) || null;
const byId = async (id) => await db.prepare('SELECT * FROM vip_plans WHERE id = ?').get(id) || null;
const version = async (id) => await db.prepare('SELECT * FROM vip_plan_versions WHERE id = ?').get(id) || null;
const version = async (id) => await db.prepare('SELECT * FROM vip_plan_versions WHERE id = ?').get(id) || null;
const latestVersion = async (planId) => await db.prepare('SELECT * FROM vip_plan_versions WHERE plan_id = ? ORDER BY version DESC LIMIT 1').get(planId) || null;
const latestVersion = async (planId) => await db.prepare('SELECT * FROM vip_plan_versions WHERE plan_id = ? ORDER BY version DESC LIMIT 1').get(planId) || null;
const versions = async (planId) => await db.prepare('SELECT * FROM vip_plan_versions WHERE plan_id = ? ORDER BY version DESC').all(planId);
const versions = async (planId) => await db.prepare('SELECT * FROM vip_plan_versions WHERE plan_id = ? ORDER BY version DESC').all(planId);
const versionPerks = async (versionId) => await db.prepare('SELECT * FROM vip_plan_perks WHERE plan_version_id = ? ORDER BY position').all(versionId);
const versionPerks = async (versionId) => await db.prepare('SELECT * FROM vip_plan_perks WHERE plan_version_id = ? ORDER BY position').all(versionId);
+ /**
+ * IndexNow (openvibe-shared/indexnow): a published plan's page, its terms history and the sitemap
+ * all changed, so tell the engines. Unset key, a draft, an unchanged edit or a replay changes
+ * nothing and never pings. Never throws: a failed ping must never take a publish down.
+ */
+ async function pingPublic(plan) {
+ try {
+ if (!indexnow || !indexnow.enabled || !baseUrl || !plan) return;
+ const creator = await creators.byId(plan.creator_id);
+ if (!creator || !creator.username) return; // network plans have no public page
+ const page = `${baseUrl}/${encodeURIComponent(creator.username)}`;
+ indexnow.pingSoon([page, `${page}/plans/${encodeURIComponent(plan.slug)}`, `${baseUrl}/sitemap.xml`]);
+ } catch { /* a ping never fails a publish */ }
+ }
+
function cleanBenefits(v) {
function cleanBenefits(v) {
if (v == null) return null;
if (v == null) return null;
const arr = Array.isArray(v) ? v : String(v).split('\n');
const arr = Array.isArray(v) ? v : String(v).split('\n');
@@ -87,7 +102,7 @@ function createPlans({ db, now, outbox, creators, perks }) {
@@ -87,7 +102,7 @@ function createPlans({ db, now, outbox, creators, perks }) {
const at = iso(now());
const at = iso(now());
const id = prefixedId('vpl', now());
const id = prefixedId('vpl', now());
try {
try {
- return await db.tx(async () => {
+ const plan = await db.tx(async () => {
await db.prepare(`INSERT INTO vip_plans (id, creator_id, slug, status, billing_kind, current_version_id, latest_version, created_by, created_at, updated_at)
await db.prepare(`INSERT INTO vip_plans (id, creator_id, slug, status, billing_kind, current_version_id, latest_version, created_by, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, NULL, 0, ?, ?, ?)`).run(id, creatorId, planSlug, publish ? 'published' : 'draft', kind, actor, at, at);
VALUES (?, ?, ?, ?, ?, NULL, 0, ?, ?, ?)`).run(id, creatorId, planSlug, publish ? 'published' : 'draft', kind, actor, at, at);
const v = await insertVersion(await byId(id), {
const v = await insertVersion(await byId(id), {
@@ -97,6 +112,8 @@ function createPlans({ db, now, outbox, creators, perks }) {
@@ -97,6 +112,8 @@ function createPlans({ db, now, outbox, creators, perks }) {
if (publish) await emitPublished(await byId(id), v, traceparent);
if (publish) await emitPublished(await byId(id), v, traceparent);
return await byId(id);
return await byId(id);
});
});
+ if (publish) await pingPublic(plan);
+ return plan;
} catch (e) { return publishConflict(e); }
} catch (e) { return publishConflict(e); }
}
}
@@ -116,11 +133,13 @@ function createPlans({ db, now, outbox, creators, perks }) {
@@ -116,11 +133,13 @@ function createPlans({ db, now, outbox, creators, perks }) {
&& JSON.stringify(perkRows.map((p) => p.id)) === JSON.stringify((await versionPerks(last.id)).map((pp) => pp.perk_id));
&& JSON.stringify(perkRows.map((p) => p.id)) === JSON.stringify((await versionPerks(last.id)).map((pp) => pp.perk_id));
if (same) return { plan, version: last, unchanged: true };
if (same) return { plan, version: last, unchanged: true };
const publish = plan.status === 'published';
const publish = plan.status === 'published';
- return await db.tx(async () => {
+ const out = await db.tx(async () => {
const v = await insertVersion(plan, { ...next, perkRows, changeNote: text(changeNote, 'change_note', 300), actor, publish });
const v = await insertVersion(plan, { ...next, perkRows, changeNote: text(changeNote, 'change_note', 300), actor, publish });
if (publish) await emitPublished(await byId(planId), v, traceparent);
if (publish) await emitPublished(await byId(planId), v, traceparent);
return { plan: await byId(planId), version: v, unchanged: false };
return { plan: await byId(planId), version: v, unchanged: false };
});
});
+ if (publish) await pingPublic(out.plan);
+ return out;
}
}
async function publish(planId, { traceparent } = {}) {
async function publish(planId, { traceparent } = {}) {
@@ -129,7 +148,7 @@ function createPlans({ db, now, outbox, creators, perks }) {
@@ -129,7 +148,7 @@ function createPlans({ db, now, outbox, creators, perks }) {
if (plan.status === 'published') return { plan, replay: true };
if (plan.status === 'published') return { plan, replay: true };
if (plan.status === 'archived') fail(409, 'vip.plan_archived', 'an archived plan cannot be published again; create a new plan');
if (plan.status === 'archived') fail(409, 'vip.plan_archived', 'an archived plan cannot be published again; create a new plan');
try {
try {
- return await db.tx(async () => {
+ const out = await db.tx(async () => {
const v = await latestVersion(planId);
const v = await latestVersion(planId);
const at = iso(now());
const at = iso(now());
await db.prepare('UPDATE vip_plan_versions SET published_at = ? WHERE id = ? AND published_at IS NULL').run(at, v.id);
await db.prepare('UPDATE vip_plan_versions SET published_at = ? WHERE id = ? AND published_at IS NULL').run(at, v.id);
@@ -137,6 +156,8 @@ function createPlans({ db, now, outbox, creators, perks }) {
@@ -137,6 +156,8 @@ function createPlans({ db, now, outbox, creators, perks }) {
await emitPublished(await byId(planId), await version(v.id), traceparent);
await emitPublished(await byId(planId), await version(v.id), traceparent);
return { plan: await byId(planId), replay: false };
return { plan: await byId(planId), replay: false };
});
});
+ await pingPublic(out.plan);
+ return out;
} catch (e) { return publishConflict(e); }
} catch (e) { return publishConflict(e); }
}
}
@@ -147,7 +168,9 @@ function createPlans({ db, now, outbox, creators, perks }) {
@@ -147,7 +168,9 @@ function createPlans({ db, now, outbox, creators, perks }) {
if (plan.status === 'archived') return plan;
if (plan.status === 'archived') return plan;
const at = iso(now());
const at = iso(now());
await db.prepare("UPDATE vip_plans SET status = 'archived', archived_at = ?, updated_at = ? WHERE id = ?").run(at, at, planId);
await db.prepare("UPDATE vip_plans SET status = 'archived', archived_at = ?, updated_at = ? WHERE id = ?").run(at, at, planId);
- return await byId(planId);
+ const archived = await byId(planId);
+ if (plan.status === 'published') await pingPublic(archived); // the creator page and the sitemap lose the plan
+ return archived;
}
}
async function list({ creatorId, includeDrafts = false, includeArchived = false }) {
async function list({ creatorId, includeDrafts = false, includeArchived = false }) {
@@ -8,7 +8,7 @@
@@ -8,7 +8,7 @@
* GET /dashboard[?as=network] creator dashboard: plans, versions, perks, members, gated resources
* GET /dashboard[?as=network] creator dashboard: plans, versions, perks, members, gated resources
* GET /:username a creator's plan page (join through Billing)
* GET /:username a creator's plan page (join through Billing)
* GET /:username/plans/:slug that plan's public terms history
* GET /:username/plans/:slug that plan's public terms history
- * GET /robots.txt, /sitemap.xml, /llms.txt
+ * GET /robots.txt, /sitemap.xml, /llms.txt, /llms-full.txt
*
*
* Every POST needs the signed-in viewer and the form's anti-forgery token; it acts through the same
* Every POST needs the signed-in viewer and the form's anti-forgery token; it acts through the same
* domain calls as the API and redirects back with a notice (?ok= / ?error=).
* domain calls as the API and redirects back with a notice (?ok= / ?error=).
@@ -23,10 +23,19 @@ const { viewerMiddleware } = require('./session');
@@ -23,10 +23,19 @@ const { viewerMiddleware } = require('./session');
const { createForms } = require('./forms');
const { createForms } = require('./forms');
const pages = require('./pages');
const pages = require('./pages');
-const RESERVED = new Set(['me', 'dashboard', 'auth', 'api', 'internal', 'css', 'js', 'terms', 'privacy', 'dmca', 'robots.txt', 'sitemap.xml', 'llms.txt', 'release.json', 'metrics', 'favicon.ico', 'embed']);
+const RESERVED = new Set(['me', 'dashboard', 'auth', 'api', 'internal', 'css', 'js', 'terms', 'privacy', 'dmca', 'robots.txt', 'sitemap.xml', 'llms.txt', 'llms-full.txt', 'release.json', 'metrics', 'favicon.ico', 'embed']);
// The sitemap's lastmod: when this server booted — a real date, computed once, never per request.
// The sitemap's lastmod: when this server booted — a real date, computed once, never per request.
const BOOT_AT = new Date().toISOString();
const BOOT_AT = new Date().toISOString();
+// The public description /llms.txt and /llms-full.txt share. llmstxt.org: a plain-markdown map of
+// the site for language-model crawlers — public data only, never the viewer.
+const LLMS_NAME = 'OpenVibe.VIP';
+const LLMS_SUMMARY = 'OpenVibe.VIP: memberships for creators — plans, perks and benefits recognised across every OpenVibe site.';
+const LLMS_DETAILS = 'A creator page lists the plans and perks a creator offers and how to join through Billing. Every plan keeps its published terms history, and a membership always points at the version it was bought under. Reading any public page needs no account; joining needs a signed-in person. Memberships, the dashboard, sign-in, embeds and the API are per-person and are never listed here.';
+// /llms-full.txt changes only when a creator or a plan does, so it is built from the database at
+// most once an hour (the browser keeps it the same 3600 s).
+const LLMS_FULL_TTL_MS = 3600 * 1000;
+
function createWebRoutes({ domain, config, layout, userAuth }) {
function createWebRoutes({ domain, config, layout, userAuth }) {
const router = express.Router();
const router = express.Router();
const { creators, plans, perks, memberships, entitlements, policies, checkout, billing } = domain;
const { creators, plans, perks, memberships, entitlements, policies, checkout, billing } = domain;
@@ -88,9 +97,9 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -88,9 +97,9 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
});
});
// /llms.txt (llmstxt.org): a plain-markdown map of the site for language-model crawlers.
// /llms.txt (llmstxt.org): a plain-markdown map of the site for language-model crawlers.
router.get('/llms.txt', (req, res) => res.type('text/plain').send(seo.llmsTxt({
router.get('/llms.txt', (req, res) => res.type('text/plain').send(seo.llmsTxt({
- name: 'OpenVibe.VIP',
- summary: 'OpenVibe.VIP: memberships for creators — plans, perks and benefits recognised across every OpenVibe site.',
- details: 'A creator page lists the plans and perks a creator offers and how to join through Billing. Every plan keeps its published terms history, and a membership always points at the version it was bought under. Reading any public page needs no account; joining needs a signed-in person. Memberships, the dashboard, sign-in, embeds and the API are per-person and are never listed here.',
+ name: LLMS_NAME,
+ summary: LLMS_SUMMARY,
+ details: LLMS_DETAILS,
sections: [
sections: [
{ title: 'Start here', links: [
{ title: 'Start here', links: [
{ title: 'Creators with published plans', url: `${config.baseUrl}/`, note: 'every creator on OpenVibe.VIP, with their public plans' },
{ title: 'Creators with published plans', url: `${config.baseUrl}/`, note: 'every creator on OpenVibe.VIP, with their public plans' },
@@ -99,10 +108,64 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
@@ -99,10 +108,64 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
{ title: 'Machine-readable', links: [
{ title: 'Machine-readable', links: [
{ title: 'Sitemap', url: `${config.baseUrl}/sitemap.xml`, note: 'the public pages, with lastmod' },
{ title: 'Sitemap', url: `${config.baseUrl}/sitemap.xml`, note: 'the public pages, with lastmod' },
{ title: 'robots.txt', url: `${config.baseUrl}/robots.txt` },
{ title: 'robots.txt', url: `${config.baseUrl}/robots.txt` },
+ { title: 'llms-full.txt', url: `${config.baseUrl}/llms-full.txt`, note: 'the full text of every public page' },
] },
] },
],
],
})));
})));
+ // /llms-full.txt: /llms.txt with the full plain text of every public page — the home copy, each
+ // creator's plan page and every published plan's terms. Built from public data only and cached
+ // in memory for an hour; the response says the same, so caches and crawlers may keep it too.
+ function termsText(v) {
+ const out = [];
+ if (v.description) out.push(v.description);
+ if (v.benefits.length) out.push(`Benefits: ${v.benefits.join('; ')}.`);
+ if (v.perks.length) out.push(`Perks: ${v.perks.map((p) => p.name).join(', ')}.`);
+ if (v.change_note) out.push(`Change: ${v.change_note}.`);
+ return out.join('\n');
+ }
+ async function buildLlmsFull() {
+ const creatorPages = [];
+ const planPages = [];
+ for (const row of await creators.listPublic({ limit: 5000 })) {
+ const c = creators.present(row);
+ const name = c.display_name || c.username;
+ const planRows = await Promise.all((await plans.list({ creatorId: c.id })).map(async (p) => await plans.present(p, { withVersions: true })));
+ const about = [`${name} (@${c.username}) offers membership plans on OpenVibe.VIP.`];
+ if (c.bio) about.push(c.bio);
+ for (const p of planRows) {
+ const v = p.current_version;
+ about.push(`Plan ${v.name} (version ${v.version})${v.description ? `: ${v.description}` : ''}`);
+ }
+ creatorPages.push({ title: `${name} memberships`, url: `/${c.username}`, text: about.join('\n\n') });
+ for (const p of planRows) {
+ const body = [`The published terms of ${name}'s ${p.current_version.name} plan on OpenVibe.VIP. Each edit is a new version; a member keeps the version they joined under.`];
+ for (const v of (p.versions || []).filter((x) => x.published_at)) {
+ const terms = termsText(v);
+ body.push(`Version ${v.version}${v.published_at ? ` (published ${String(v.published_at).slice(0, 10)})` : ''}${terms ? `\n${terms}` : ''}`);
+ }
+ planPages.push({ title: `${name}: ${p.current_version.name} terms`, url: `/${c.username}/plans/${p.slug}`, text: body.join('\n\n') });
+ }
+ }
+ return seo.llmsFull({
+ site: { name: LLMS_NAME, url: config.baseUrl },
+ summary: `${LLMS_SUMMARY} ${LLMS_DETAILS}`,
+ sections: [
+ { title: 'Home', pages: [{ title: `${LLMS_NAME} home`, url: '/', text: `${LLMS_SUMMARY}\n\n${LLMS_DETAILS}\n\nThe home page lists every creator with a published plan.` }] },
+ { title: 'Creators', pages: creatorPages },
+ { title: 'Plans', pages: planPages },
+ ],
+ });
+ }
+ let llmsFullCache = { at: 0, value: null };
+ router.get('/llms-full.txt', async (req, res, next) => {
+ try {
+ if (!llmsFullCache.value || domain.now() - llmsFullCache.at >= LLMS_FULL_TTL_MS) llmsFullCache = { at: domain.now(), value: await buildLlmsFull() };
+ res.setHeader('Cache-Control', 'public, max-age=3600');
+ res.type('text/plain').send(llmsFullCache.value);
+ } catch (e) { next(e); }
+ });
+
// What shipped on OpenVibe.VIP: the shared update log every OpenVibe site has.
// What shipped on OpenVibe.VIP: the shared update log every OpenVibe site has.
router.get('/updates', (req, res) => send(res, 200, { viewer: req.viewer, canonicalPath: '/updates', title: 'What shipped on OpenVibe.VIP', body: frame.updatesBody({ service: 'vip', siteName: 'OpenVibe.VIP' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>` }));
router.get('/updates', (req, res) => send(res, 200, { viewer: req.viewer, canonicalPath: '/updates', title: 'What shipped on OpenVibe.VIP', body: frame.updatesBody({ service: 'vip', siteName: 'OpenVibe.VIP' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>` }));
router.get('/', async (req, res) => send(res, 200, {
router.get('/', async (req, res) => send(res, 200, {
@@ -226,6 +226,16 @@ const { test, run } = harness('api');
@@ -226,6 +226,16 @@ const { test, run } = harness('api');
assert.match(llms.headers.get('content-type'), /^text\/plain/);
assert.match(llms.headers.get('content-type'), /^text\/plain/);
assert.match(llms.text, /^# /, 'llms.txt starts with a markdown heading');
assert.match(llms.text, /^# /, 'llms.txt starts with a markdown heading');
assert.match(llms.text, /\[Creators with published plans\]\(http:\/\/vip\.test\/\)/, 'llms.txt lists the main public pages');
assert.match(llms.text, /\[Creators with published plans\]\(http:\/\/vip\.test\/\)/, 'llms.txt lists the main public pages');
+ const full = await t.call('GET', '/llms-full.txt', { token: null });
+ assert.strictEqual(full.status, 200);
+ assert.match(full.headers.get('content-type'), /^text\/plain/);
+ assert.strictEqual(full.headers.get('cache-control'), 'public, max-age=3600', 'llms-full.txt is cached for an hour');
+ assert.match(full.text, /^# OpenVibe\.VIP/, 'llms-full.txt starts with the llms.txt header');
+ assert.match(full.text, /URL: http:\/\/vip\.test\/\n/, 'the home page');
+ assert.match(full.text, /URL: http:\/\/vip\.test\/lena\n/, 'the public creator page');
+ assert.match(full.text, new RegExp(`URL: http:\\/\\/vip\\.test\\/lena\\/plans\\/${plan.slug}\\n`), 'the plan terms page');
+ assert.match(full.text, /Front row/, 'the full plan text');
+ assert.match(full.text, /Monthly stream/, 'a benefit of the current version');
});
});
test('every rendered page carries the boost marker, boost script (data-main) and the {path} sign-in template', async () => {
test('every rendered page carries the boost marker, boost script (data-main) and the {path} sign-in template', async () => {
@@ -47,7 +47,7 @@ async function boot(opts = {}) {
@@ -47,7 +47,7 @@ async function boot(opts = {}) {
const log = { log: (...a) => logs.push(a.join(' ')), warn: (...a) => logs.push(a.join(' ')), error: (...a) => logs.push(a.join(' ')), debug() {} };
const log = { log: (...a) => logs.push(a.join(' ')), warn: (...a) => logs.push(a.join(' ')), error: (...a) => logs.push(a.join(' ')), debug() {} };
// One database per boot (PGlite, or VIP_TEST_STORE=pg: the containers), dropped when the boot closes.
// One database per boot (PGlite, or VIP_TEST_STORE=pg: the containers), dropped when the boot closes.
const testdb = opts.testdb || await require('./db').testDb();
const testdb = opts.testdb || await require('./db').testDb();
- const app = await createApp({ config, db: testdb.db, now: clock.now, log });
+ const app = await createApp({ config, db: testdb.db, now: clock.now, log, indexnow: opts.indexnow });
const server = await new Promise((resolve) => { const s = http.createServer(app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const server = await new Promise((resolve) => { const s = http.createServer(app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const base = `http://127.0.0.1:${server.address().port}`;
const base = `http://127.0.0.1:${server.address().port}`;
const domain = app.locals.domain;
const domain = app.locals.domain;
new file mode 100644
new file mode 100644
@@ -0,0 +1,80 @@
@@ -0,0 +1,80 @@
+'use strict';
+/**
+ * IndexNow (openvibe-shared/indexnow): INDEXNOW_KEY unset → the feature is off (no key route,
+ * nothing sent). With a key, the key file is served at /<key>.txt as text/plain and publishing,
+ * editing or archiving a plan pings the engines with the creator's page, the plan's terms page and
+ * the sitemap. A draft, an unchanged edit and a replay never ping.
+ */
+const assert = require('assert');
+const { boot, harness } = require('./helpers/app');
+
+const { test, run } = harness('indexnow');
+const KEY = 'k'.repeat(32);
+
+(async () => {
+ const off = await boot();
+ const on = await boot({ env: { INDEXNOW_KEY: KEY } });
+
+ // A spy in place of the module's HTTP send: records every pingSoon batch.
+ const pings = [];
+ const spy = {
+ enabled: true,
+ keyFile: (_req, _res, next) => next(),
+ pingSoon: (urls) => { const a = Array.isArray(urls) ? urls : [urls]; pings.push(...a); return a.length; },
+ ping: async () => ({ sent: 0, status: 0 }),
+ flush: async () => ({ sent: 0, status: 0 }),
+ };
+ const t = await boot({ indexnow: spy });
+ const creator = t.network.newUser('alice', { role: 'streamer' });
+ let plan;
+
+ test('without a key IndexNow is off: no key route and nothing sent', async () => {
+ assert.strictEqual(off.app.locals.indexnow.enabled, false);
+ const res = await off.call('GET', `/${KEY}.txt`, { token: null });
+ assert.strictEqual(res.status, 404, res.text);
+ });
+
+ test('with a key the key file answers text/plain with the key, before static', async () => {
+ assert.strictEqual(on.app.locals.indexnow.enabled, true);
+ const res = await on.call('GET', `/${KEY}.txt`, { token: null });
+ assert.strictEqual(res.status, 200, res.text);
+ assert.match(res.headers.get('content-type'), /^text\/plain/);
+ assert.strictEqual(res.headers.get('cache-control'), 'public, max-age=3600');
+ assert.strictEqual(res.text, KEY);
+ });
+
+ test('a draft never pings', async () => {
+ const res = await t.call('POST', '/api/v1/plans', { user: creator, body: { name: 'First light' } });
+ assert.strictEqual(res.status, 201, res.text);
+ plan = res.json.plan;
+ assert.deepStrictEqual(pings, []);
+ });
+
+ test('publishing pings the creator page, the terms page and the sitemap', async () => {
+ const res = await t.call('POST', `/api/v1/plans/${plan.id}/publish`, { user: creator });
+ assert.strictEqual(res.status, 200, res.text);
+ assert.ok(pings.includes('http://vip.test/alice'), JSON.stringify(pings));
+ assert.ok(pings.includes(`http://vip.test/alice/plans/${plan.slug}`), JSON.stringify(pings));
+ assert.ok(pings.includes('http://vip.test/sitemap.xml'), JSON.stringify(pings));
+ });
+
+ test('an unchanged edit does not ping; a real edit of a published plan does', async () => {
+ pings.length = 0;
+ await t.call('PATCH', `/api/v1/plans/${plan.id}`, { user: creator, body: { name: 'First light' } });
+ assert.deepStrictEqual(pings, [], 'an unchanged edit is not a change');
+ const res = await t.call('PATCH', `/api/v1/plans/${plan.id}`, { user: creator, body: { description: 'Now with more.' } });
+ assert.strictEqual(res.status, 200, res.text);
+ assert.ok(pings.includes('http://vip.test/alice'), JSON.stringify(pings));
+ assert.ok(pings.includes(`http://vip.test/alice/plans/${plan.slug}`), JSON.stringify(pings));
+ });
+
+ test('archiving pings the creator page and the sitemap', async () => {
+ pings.length = 0;
+ const res = await t.call('POST', `/api/v1/plans/${plan.id}/archive`, { user: creator });
+ assert.strictEqual(res.status, 200, res.text);
+ assert.ok(pings.includes('http://vip.test/alice'), JSON.stringify(pings));
+ assert.ok(pings.includes('http://vip.test/sitemap.xml'), JSON.stringify(pings));
+ });
+
+ await run().finally(async () => { await t.close(); await on.close(); await off.close(); });
+})();
new file mode 100644
new file mode 100644
@@ -0,0 +1,48 @@
@@ -0,0 +1,48 @@
+'use strict';
+/**
+ * /llms-full.txt: the /llms.txt header plus the full text of every public page — the home copy,
+ * each creator's plan page and every published plan's terms. It is public data, cached for an hour.
+ */
+const assert = require('assert');
+const { boot, harness } = require('./helpers/app');
+
+const { test, run } = harness('llms-full');
+
+(async () => {
+ const t = await boot();
+ const creator = t.network.newUser('dana', { role: 'streamer' });
+ let plan;
+
+ test('the full text covers the home, the creator page and the plan terms', async () => {
+ const res = await t.call('POST', '/api/v1/plans', { user: creator, body: { name: 'Backstage', description: 'First description', benefits: ['A badge'], publish: true } });
+ assert.strictEqual(res.status, 201, res.text);
+ plan = res.json.plan;
+ const full = await t.call('GET', '/llms-full.txt', { token: null });
+ assert.strictEqual(full.status, 200);
+ assert.match(full.headers.get('content-type'), /^text\/plain/);
+ assert.strictEqual(full.headers.get('cache-control'), 'public, max-age=3600');
+ assert.match(full.text, /^# OpenVibe\.VIP/, 'the llms.txt header comes first');
+ assert.match(full.text, /## Home/);
+ assert.match(full.text, /## Creators/);
+ assert.match(full.text, /## Plans/);
+ assert.match(full.text, /URL: http:\/\/vip\.test\/\n/, 'the home page');
+ assert.match(full.text, /URL: http:\/\/vip\.test\/dana\n/, 'the public creator page');
+ assert.match(full.text, new RegExp(`URL: http:\\/\\/vip\\.test\\/dana\\/plans\\/${plan.slug}\\n`), 'the plan terms page');
+ assert.match(full.text, /First description/, 'the full plan text, never clipped');
+ assert.match(full.text, /@dana/);
+ });
+
+ test('the body is built once and cached for an hour', async () => {
+ const before = (await t.call('GET', '/llms-full.txt', { token: null })).text;
+ const edit = await t.call('PATCH', `/api/v1/plans/${plan.id}`, { user: creator, body: { description: 'Second description' } });
+ assert.strictEqual(edit.status, 200, edit.text);
+ const cached = (await t.call('GET', '/llms-full.txt', { token: null })).text;
+ assert.strictEqual(cached, before, 'within the hour the cached copy is served unchanged');
+ assert.doesNotMatch(cached, /Second description/);
+ t.clock.advance(3600 * 1000 + 1);
+ const fresh = (await t.call('GET', '/llms-full.txt', { token: null })).text;
+ assert.match(fresh, /Second description/, 'after the hour the body is rebuilt from the database');
+ });
+
+ await run().finally(async () => await t.close());
+})();