openvibers/openvibe.vip Commits

Give VIP /llms-full.txt and IndexNow (#10)

OP OpenVibers committed 5 Oct 2026 · 1ffc90cda8 · 12 files, +285 −17

## Files changed

- `server/config.js` — new `indexnow.key` read from `INDEXNOW_KEY` (trimmed; empty = off).
- `.env.example` — documented `INDEXNOW_KEY` (unset = off, nothing mounted or sent).
- `server/app.js` — import `createIndexNow`; build it once from `config.indexnow.key` (injectable via `opts.indexnow` for tests); pass it into `createDomain`; mount `indexnow.keyFile` only when enabled, immediately before `express.static`; expose `indexnow` on `app.locals`.
- `server/domain/index.js` — accept `indexnow`, forward `config.baseUrl` + `indexnow` to `createPlans`.
- `server/domain/plans.js` — `pingPublic()` pings the creator page, plan-terms page and `/sitemap.xml` on plan publish, on an edit of a published plan, and on archiving a published plan. Drafts, unchanged edits, replayed publishes and network plans (no public page) never ping; the helper swallows errors so a ping can never fail a publish.
- `server/web/routes.js` — added `GET /llms-full.txt` using `seo.llmsFull` over the home copy, every public creator page and every published plan's terms history; built from public data and cached in memory for an hour plus `Cache-Control: public, max-age=3600`. Lifted the shared `llms.txt` copy into `LLMS_*` constants, linked `/llms-full.txt` from the machine-readable list, added it to `RESERVED`, and updated the route-header comment.
- `test/helpers/app.js` — pass `opts.indexnow` through to `createApp`.
- `test/indexnow.test.js` (new) — off without a key (no key route, 404); with a key the key file is `text/plain`, `max-age=3600`, body is the key; a draft never pings; publish/edit/archive ping creator page + terms page + sitemap; an unchanged edit does not.
- `test/llms-full.test.js` (new) — content covers home/creator/plan pages with the full plan text; cached for an hour (advancing the injected clock rebuilds it).
- `test/api.test.js` — extended the discovery-artifacts test with `/llms-full.txt` status, content-type, cache header and pages.
- `README.md` — route list, new "Crawlers and AI readers" + "IndexNow" sections, and two acceptance-table rows.
- `STATUS.json` — added a `works` entry and two `features` entries.

## Tests run

- `ov test --all --no-cache` → **15/15 test files passed in 55 s** (fresh, no green cache).
- Also ran `node test/run.js indexnow llms-full` and `node test/run.js api` during development → all passed.

## Not done

- Did not commit, push, deploy or install packages (left to the launching session, per instructions).
- The repository has no `CHANGELOG.md`, so only README.md and STATUS.json were kept in step.

Changes

ddiff --git a/.env.example b/.env.example
iindex ec76df6..a5855d7 100644
---- a/.env.example
++++ b/.env.example
@@@ -70,6 +70,12 @@ VIP_EVENTS_SECRET=
# How long a creator's member count (from Billing) is reused
# VIP_MEMBER_COUNT_TTL_MS=300000
++# ── IndexNow (openvibe-shared/indexnow) ──────────────────
++# With a key (8-128 hex or alphanumeric characters, what IndexNow's own tools generate), search
++# engines are told the moment a published plan appears, changes or is archived (the key file is served
++# at /<key>.txt). Unset: off — nothing is mounted and nothing is sent.
++#INDEXNOW_KEY=
++
# ── Per-actor limits (server/api/actor-limits.js) ───────
# Reads one caller (a service principal, a person, else an address) may make to /api/v1 per minute
# and per hour; past them 429 rate_limited with Retry-After. Entitlement checks, policy evaluation,
ddiff --git a/README.md b/README.md
iindex 6edf5a7..215c82e 100644
---- a/README.md
++++ b/README.md
@@@ -321,11 +321,30 @@ memberships* card gives the creator the badge and widget HTML and the card URL.
Billing's price and period from `GET /api/v1/rates`, join form) · `/:username/plans/:slug` public terms
history · `/me` your memberships, the version you joined under, cancel, badge preference ·
`/dashboard` plans, versions, perks and bindings, members, members-only resources, the badge and
--widget to share and the member-count switch (`?as=network` for staff) · `/embed/:username/…` (above) · `/robots.txt`, `/sitemap.xml` · `/terms`, `/privacy`, `/dmca`
++widget to share and the member-count switch (`?as=network` for staff) · `/embed/:username/…` (above) · `/robots.txt`, `/sitemap.xml`, `/llms.txt`, `/llms-full.txt` · `/terms`, `/privacy`, `/dmca`
(openvibe-shared legal). Navbar from `https://openvibe.network/shared/navbar.js`, SSR footer and
`<noscript>` navigation from openvibe-shared. Forms carry HMAC anti-forgery tokens
(`VIP_FORM_SECRET`); cookies are SameSite=Lax. No feed: plans are not a publication stream.
++### Crawlers and AI readers
++
++`/robots.txt` and `/sitemap.xml` come from openvibe-shared/seo (the sitemap lists the home and every
++creator with a published plan). `/llms.txt` is the site map for language models; `/llms-full.txt` is
++the same header followed by the full plain text of the home, every public creator page and every
++published plan's terms (`seo.llmsFull`, built from public data only, cached an hour in memory and with
++`Cache-Control: public, max-age=3600`). Every page's head carries an `ai-summary` meta and WebPage
++JSON-LD through openvibe-shared/shell.
++
++### IndexNow (openvibe-shared/indexnow)
++
++With `INDEXNOW_KEY` set (8–128 hex or alphanumeric characters, what IndexNow's own tools generate),
++the key file is served at `/<key>.txt` as `text/plain` (before static, so a file cannot shadow it) and
++a plan that changes tells the engines: publishing, editing a published plan and archiving ping
++`api.indexnow.org` with the creator's page, the plan's terms page and `/sitemap.xml`; the module
++batches and debounces (one POST per 30 s window). A draft, an unchanged edit, a replayed publish and
++network plans (no public creator page) never ping, and a failed ping never takes a publish down.
++Unset: off — the key route is not mounted and nothing is sent (`test/indexnow.test.js`).
++
## Before and after the Billing cutover
Billing runs in shadow mode until the Wave 8 cutover (Live stays authoritative for money,
@@@ -380,6 +399,8 @@ Reporting a vulnerability: [SECURITY.md](SECURITY.md).
| gated-resource evaluation fails closed (no rule, unknown, perk missing, guest, VIP down in the client) | `test/policies.test.js` |
| VIP works with Live unavailable | no code path calls Live; every test runs without a Live stub |
| a creator's membership card, badge and widget are public-data only, the same for everyone, embeddable and inert | `test/embeds.test.js` (published plans only, no subjects or binding configs, no cookie read or set, CORS, SVG escaping and sandbox, the widget's frame-ancestors and hashed style, the creator's member-count switch, Billing outage keeps the last count or shows none) |
++| `/llms-full.txt` carries the full text of the home, creator and plan pages, and is cached for an hour | `test/llms-full.test.js`, `test/api.test.js` (discovery artifacts) |
++| IndexNow is off without `INDEXNOW_KEY` (no key route, nothing sent); with one the key file is served at `/<key>.txt` as `text/plain` and publishing, editing and archiving a plan ping the creator page, the terms page and the sitemap; a draft and an unchanged edit never ping | `test/indexnow.test.js` |
Not yet demonstrated (needs the other services): convergence **across consuming products** end to end
(Chat, Community, Blog and Wiki call VIP in code; Live does not), and a real Billing/Events round trip
ddiff --git a/STATUS.json b/STATUS.json
iindex db250a1..a2f2b86 100644
---- a/STATUS.json
++++ b/STATUS.json
@@@ -23,7 +23,8 @@
"14 vip.* capabilities, the service manifest and its events registered in openvibe-contracts v0.19.0 (this repository pins v0.92.0)",
"per-actor limits on /api/v1 (openvibe-sdk/limits)",
"the Billing client (server/billing-client.js) is a thin wrapper over openvibe-sdk/commerce (openvibe-sdk v0.28.0), shared with Tips: same capabilities, idempotency keys and retry-once-on-401",
-- "renewal/refund convergence via Billing events: a per-period charge ledger (vip_membership_periods, unique per Billing transaction; reversals and reversal-before-grant tombstones), a past_due renewal grace that never authorizes (answer.grace), and a period audit that doubts a paid period without a record (migration 0002; docs/cutover-0002-renewal-periods.md)"
++ "renewal/refund convergence via Billing events: a per-period charge ledger (vip_membership_periods, unique per Billing transaction; reversals and reversal-before-grant tombstones), a past_due renewal grace that never authorizes (answer.grace), and a period audit that doubts a paid period without a record (migration 0002; docs/cutover-0002-renewal-periods.md)",
++ "crawler and AI-reader artifacts: /llms-full.txt serves the full text of the home, public creator and plan-terms pages (openvibe-shared/seo, cached 3600 s), and IndexNow (openvibe-shared/indexnow, INDEXNOW_KEY) serves its key file at /<key>.txt and pings the creator page, terms page and sitemap when a plan is published, edited or archived"
],
"not_yet": [
"not launched: openvibe.vip is still an OpenVibe.Sites placeholder and the service answers on loopback only",
@@@ -40,6 +41,8 @@
"openvibe-vip/client with the product cache and its convergence bound",
"creator plan pages, terms history, member status and creator dashboard",
"public membership card, badge and embeddable widget",
++ "/llms.txt and /llms-full.txt for language-model crawlers, cached an hour",
++ "IndexNow: a key serves /<key>.txt and pings plan page changes and the sitemap",
"SDK outbox events vip.plan.published and vip.membership.changed",
"per-actor limits (openvibe-sdk/limits)"
],
ddiff --git a/server/app.js b/server/app.js
iindex 9955d19..85552d4 100644
---- a/server/app.js
++++ b/server/app.js
@@@ -20,6 +20,7 @@ const express = require('express');
const cookieParser = require('cookie-parser');
const { http } = require('openvibe-contracts');
const cache = require('openvibe-shared/cache-policy');
++const { createIndexNow } = require('openvibe-shared/indexnow');
const { loadConfig } = require('./config');
const { openDb } = require('./db');
const { createKeyProvider, createUserAuth } = require('./network');
@@@ -52,7 +53,12 @@ async function createApp(opts = {}) {
const userAuth = createUserAuth(config, keys);
const billing = opts.billing || createBillingClient(config, { fetchImpl });
const outbox = opts.outbox || createVipOutbox({ db, config, fetchImpl: opts.eventsFetch, now, log });
-- const domain = createDomain({ db, config, outbox, billing, now, log });
++ // IndexNow (openvibe-shared/indexnow): created once at boot from INDEXNOW_KEY. Unset → off, no key
++ // file, nothing sent; a published plan tells the engines its page changed. Tests inject a spy.
++ const indexnow = opts.indexnow !== undefined ? opts.indexnow : createIndexNow({
++ host: config.baseUrl, key: config.indexnow.key, fetch: fetchImpl, log,
++ });
++ const domain = createDomain({ db, config, outbox, billing, now, log, indexnow });
const apiAuth = createApiAuth({ config, keys, userAuth });
const release = require('openvibe-shared/release').createRelease({ service: 'vip', root: path.join(__dirname, '..') });
setRelease(release.release);
@@@ -109,6 +115,9 @@ async function createApp(opts = {}) {
// This site's own pinned copy of the OpenVibe Frame's browser files (openvibe-shared/serve).
app.use('/shared', require('openvibe-shared/serve').handler());
++ // GET /<key>.txt — the IndexNow key file (mounted only when a key is configured; it serves itself
++ // and steps aside for every other path). Before static, so the key can never be shadowed by a file.
++ if (indexnow.enabled) app.use(indexnow.keyFile);
app.use(express.static(PUBLIC_DIR, {
index: false, redirect: false,
setHeaders(res, filePath) {
@@@ -133,7 +142,7 @@ async function createApp(opts = {}) {
return res.status(500).type('html').send(layout.page({ title: 'Error', robots: 'noindex', body: pages.errorPage({ status: 500, title: 'Something went wrong', message: 'This one is on us. Please try again.' }) }));
});
-- Object.assign(app.locals, { config, db, domain, keys, outbox, billing, consumer, metrics });
++ Object.assign(app.locals, { config, db, domain, keys, outbox, billing, consumer, metrics, indexnow });
return app;
}
ddiff --git a/server/config.js b/server/config.js
iindex 879335a..a27c46f 100644
---- a/server/config.js
++++ b/server/config.js
@@@ -105,6 +105,11 @@ function loadConfig(env = process.env) {
minute: Math.max(1, int(env.VIP_LIMITS_MINUTE, 120)),
hour: Math.max(1, int(env.VIP_LIMITS_HOUR, 3000)),
},
++
++ // IndexNow (openvibe-shared/indexnow): a key makes search engines recrawl a creator's page the
++ // moment a plan is published, edited or archived (the key file is served at /<key>.txt). Unset:
++ // off, no key file, nothing sent. Tests and drills never set it.
++ indexnow: { key: String(env.INDEXNOW_KEY || '').trim() },
};
}
ddiff --git a/server/domain/index.js b/server/domain/index.js
iindex db1e560..8fbe770 100644
---- a/server/domain/index.js
++++ b/server/domain/index.js
@@@ -11,10 +11,10 @@ const { createPolicies } = require('./policies');
const { createCheckout } = require('./checkout');
const { createCards } = require('./cards');
--function createDomain({ db, config, outbox, billing, now = () => Date.now(), log = console }) {
++function createDomain({ db, config, outbox, billing, now = () => Date.now(), log = console, indexnow = null }) {
const creators = createCreators({ db, now });
const perks = createPerks({ db, now });
-- const plans = createPlans({ db, now, outbox, creators, perks });
++ const plans = createPlans({ db, now, outbox, creators, perks, baseUrl: config.baseUrl, indexnow });
const memberships = createMemberships({ db, now, creators, plans });
const periods = createPeriods({ db, now, log });
const entitlements = createEntitlements({ db, now, config, billing, outbox, memberships, plans, log });
ddiff --git a/server/domain/plans.js b/server/domain/plans.js
iindex 8486bb1..c3329a1 100644
---- a/server/domain/plans.js
++++ b/server/domain/plans.js
@@@ -19,13 +19,28 @@ const { fail, prefixedId, iso, text, slug, slugify, json } = require('../util');
const BILLING_KINDS = ['channel_subscription'];
--function createPlans({ db, now, outbox, creators, perks }) {
++function createPlans({ db, now, outbox, creators, perks, baseUrl = '', indexnow = null }) {
const byId = async (id) => await db.prepare('SELECT * FROM vip_plans WHERE id = ?').get(id) || null;
const version = async (id) => await db.prepare('SELECT * FROM vip_plan_versions WHERE id = ?').get(id) || null;
const latestVersion = async (planId) => await db.prepare('SELECT * FROM vip_plan_versions WHERE plan_id = ? ORDER BY version DESC LIMIT 1').get(planId) || null;
const versions = async (planId) => await db.prepare('SELECT * FROM vip_plan_versions WHERE plan_id = ? ORDER BY version DESC').all(planId);
const versionPerks = async (versionId) => await db.prepare('SELECT * FROM vip_plan_perks WHERE plan_version_id = ? ORDER BY position').all(versionId);
++ /**
++ * IndexNow (openvibe-shared/indexnow): a published plan's page, its terms history and the sitemap
++ * all changed, so tell the engines. Unset key, a draft, an unchanged edit or a replay changes
++ * nothing and never pings. Never throws: a failed ping must never take a publish down.
++ */
++ async function pingPublic(plan) {
++ try {
++ if (!indexnow || !indexnow.enabled || !baseUrl || !plan) return;
++ const creator = await creators.byId(plan.creator_id);
++ if (!creator || !creator.username) return; // network plans have no public page
++ const page = `${baseUrl}/${encodeURIComponent(creator.username)}`;
++ indexnow.pingSoon([page, `${page}/plans/${encodeURIComponent(plan.slug)}`, `${baseUrl}/sitemap.xml`]);
++ } catch { /* a ping never fails a publish */ }
++ }
++
function cleanBenefits(v) {
if (v == null) return null;
const arr = Array.isArray(v) ? v : String(v).split('\n');
@@@ -87,7 +102,7 @@ function createPlans({ db, now, outbox, creators, perks }) {
const at = iso(now());
const id = prefixedId('vpl', now());
try {
-- return await db.tx(async () => {
++ const plan = await db.tx(async () => {
await db.prepare(`INSERT INTO vip_plans (id, creator_id, slug, status, billing_kind, current_version_id, latest_version, created_by, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, NULL, 0, ?, ?, ?)`).run(id, creatorId, planSlug, publish ? 'published' : 'draft', kind, actor, at, at);
const v = await insertVersion(await byId(id), {
@@@ -97,6 +112,8 @@ function createPlans({ db, now, outbox, creators, perks }) {
if (publish) await emitPublished(await byId(id), v, traceparent);
return await byId(id);
});
++ if (publish) await pingPublic(plan);
++ return plan;
} catch (e) { return publishConflict(e); }
}
@@@ -116,11 +133,13 @@ function createPlans({ db, now, outbox, creators, perks }) {
&& JSON.stringify(perkRows.map((p) => p.id)) === JSON.stringify((await versionPerks(last.id)).map((pp) => pp.perk_id));
if (same) return { plan, version: last, unchanged: true };
const publish = plan.status === 'published';
-- return await db.tx(async () => {
++ const out = await db.tx(async () => {
const v = await insertVersion(plan, { ...next, perkRows, changeNote: text(changeNote, 'change_note', 300), actor, publish });
if (publish) await emitPublished(await byId(planId), v, traceparent);
return { plan: await byId(planId), version: v, unchanged: false };
});
++ if (publish) await pingPublic(out.plan);
++ return out;
}
async function publish(planId, { traceparent } = {}) {
@@@ -129,7 +148,7 @@ function createPlans({ db, now, outbox, creators, perks }) {
if (plan.status === 'published') return { plan, replay: true };
if (plan.status === 'archived') fail(409, 'vip.plan_archived', 'an archived plan cannot be published again; create a new plan');
try {
-- return await db.tx(async () => {
++ const out = await db.tx(async () => {
const v = await latestVersion(planId);
const at = iso(now());
await db.prepare('UPDATE vip_plan_versions SET published_at = ? WHERE id = ? AND published_at IS NULL').run(at, v.id);
@@@ -137,6 +156,8 @@ function createPlans({ db, now, outbox, creators, perks }) {
await emitPublished(await byId(planId), await version(v.id), traceparent);
return { plan: await byId(planId), replay: false };
});
++ await pingPublic(out.plan);
++ return out;
} catch (e) { return publishConflict(e); }
}
@@@ -147,7 +168,9 @@ function createPlans({ db, now, outbox, creators, perks }) {
if (plan.status === 'archived') return plan;
const at = iso(now());
await db.prepare("UPDATE vip_plans SET status = 'archived', archived_at = ?, updated_at = ? WHERE id = ?").run(at, at, planId);
-- return await byId(planId);
++ const archived = await byId(planId);
++ if (plan.status === 'published') await pingPublic(archived); // the creator page and the sitemap lose the plan
++ return archived;
}
async function list({ creatorId, includeDrafts = false, includeArchived = false }) {
ddiff --git a/server/web/routes.js b/server/web/routes.js
iindex 5f9dbea..fe02354 100644
---- a/server/web/routes.js
++++ b/server/web/routes.js
@@@ -8,7 +8,7 @@
* GET /dashboard[?as=network] creator dashboard: plans, versions, perks, members, gated resources
* GET /:username a creator's plan page (join through Billing)
* GET /:username/plans/:slug that plan's public terms history
-- * GET /robots.txt, /sitemap.xml, /llms.txt
++ * GET /robots.txt, /sitemap.xml, /llms.txt, /llms-full.txt
*
* Every POST needs the signed-in viewer and the form's anti-forgery token; it acts through the same
* domain calls as the API and redirects back with a notice (?ok= / ?error=).
@@@ -23,10 +23,19 @@ const { viewerMiddleware } = require('./session');
const { createForms } = require('./forms');
const pages = require('./pages');
--const RESERVED = new Set(['me', 'dashboard', 'auth', 'api', 'internal', 'css', 'js', 'terms', 'privacy', 'dmca', 'robots.txt', 'sitemap.xml', 'llms.txt', 'release.json', 'metrics', 'favicon.ico', 'embed']);
++const RESERVED = new Set(['me', 'dashboard', 'auth', 'api', 'internal', 'css', 'js', 'terms', 'privacy', 'dmca', 'robots.txt', 'sitemap.xml', 'llms.txt', 'llms-full.txt', 'release.json', 'metrics', 'favicon.ico', 'embed']);
// The sitemap's lastmod: when this server booted — a real date, computed once, never per request.
const BOOT_AT = new Date().toISOString();
++// The public description /llms.txt and /llms-full.txt share. llmstxt.org: a plain-markdown map of
++// the site for language-model crawlers — public data only, never the viewer.
++const LLMS_NAME = 'OpenVibe.VIP';
++const LLMS_SUMMARY = 'OpenVibe.VIP: memberships for creators — plans, perks and benefits recognised across every OpenVibe site.';
++const LLMS_DETAILS = 'A creator page lists the plans and perks a creator offers and how to join through Billing. Every plan keeps its published terms history, and a membership always points at the version it was bought under. Reading any public page needs no account; joining needs a signed-in person. Memberships, the dashboard, sign-in, embeds and the API are per-person and are never listed here.';
++// /llms-full.txt changes only when a creator or a plan does, so it is built from the database at
++// most once an hour (the browser keeps it the same 3600 s).
++const LLMS_FULL_TTL_MS = 3600 * 1000;
++
function createWebRoutes({ domain, config, layout, userAuth }) {
const router = express.Router();
const { creators, plans, perks, memberships, entitlements, policies, checkout, billing } = domain;
@@@ -88,9 +97,9 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
});
// /llms.txt (llmstxt.org): a plain-markdown map of the site for language-model crawlers.
router.get('/llms.txt', (req, res) => res.type('text/plain').send(seo.llmsTxt({
-- name: 'OpenVibe.VIP',
-- summary: 'OpenVibe.VIP: memberships for creators — plans, perks and benefits recognised across every OpenVibe site.',
-- details: 'A creator page lists the plans and perks a creator offers and how to join through Billing. Every plan keeps its published terms history, and a membership always points at the version it was bought under. Reading any public page needs no account; joining needs a signed-in person. Memberships, the dashboard, sign-in, embeds and the API are per-person and are never listed here.',
++ name: LLMS_NAME,
++ summary: LLMS_SUMMARY,
++ details: LLMS_DETAILS,
sections: [
{ title: 'Start here', links: [
{ title: 'Creators with published plans', url: `${config.baseUrl}/`, note: 'every creator on OpenVibe.VIP, with their public plans' },
@@@ -99,10 +108,64 @@ function createWebRoutes({ domain, config, layout, userAuth }) {
{ title: 'Machine-readable', links: [
{ title: 'Sitemap', url: `${config.baseUrl}/sitemap.xml`, note: 'the public pages, with lastmod' },
{ title: 'robots.txt', url: `${config.baseUrl}/robots.txt` },
++ { title: 'llms-full.txt', url: `${config.baseUrl}/llms-full.txt`, note: 'the full text of every public page' },
] },
],
})));
++ // /llms-full.txt: /llms.txt with the full plain text of every public page — the home copy, each
++ // creator's plan page and every published plan's terms. Built from public data only and cached
++ // in memory for an hour; the response says the same, so caches and crawlers may keep it too.
++ function termsText(v) {
++ const out = [];
++ if (v.description) out.push(v.description);
++ if (v.benefits.length) out.push(`Benefits: ${v.benefits.join('; ')}.`);
++ if (v.perks.length) out.push(`Perks: ${v.perks.map((p) => p.name).join(', ')}.`);
++ if (v.change_note) out.push(`Change: ${v.change_note}.`);
++ return out.join('\n');
++ }
++ async function buildLlmsFull() {
++ const creatorPages = [];
++ const planPages = [];
++ for (const row of await creators.listPublic({ limit: 5000 })) {
++ const c = creators.present(row);
++ const name = c.display_name || c.username;
++ const planRows = await Promise.all((await plans.list({ creatorId: c.id })).map(async (p) => await plans.present(p, { withVersions: true })));
++ const about = [`${name} (@${c.username}) offers membership plans on OpenVibe.VIP.`];
++ if (c.bio) about.push(c.bio);
++ for (const p of planRows) {
++ const v = p.current_version;
++ about.push(`Plan ${v.name} (version ${v.version})${v.description ? `: ${v.description}` : ''}`);
++ }
++ creatorPages.push({ title: `${name} memberships`, url: `/${c.username}`, text: about.join('\n\n') });
++ for (const p of planRows) {
++ const body = [`The published terms of ${name}'s ${p.current_version.name} plan on OpenVibe.VIP. Each edit is a new version; a member keeps the version they joined under.`];
++ for (const v of (p.versions || []).filter((x) => x.published_at)) {
++ const terms = termsText(v);
++ body.push(`Version ${v.version}${v.published_at ? ` (published ${String(v.published_at).slice(0, 10)})` : ''}${terms ? `\n${terms}` : ''}`);
++ }
++ planPages.push({ title: `${name}: ${p.current_version.name} terms`, url: `/${c.username}/plans/${p.slug}`, text: body.join('\n\n') });
++ }
++ }
++ return seo.llmsFull({
++ site: { name: LLMS_NAME, url: config.baseUrl },
++ summary: `${LLMS_SUMMARY} ${LLMS_DETAILS}`,
++ sections: [
++ { title: 'Home', pages: [{ title: `${LLMS_NAME} home`, url: '/', text: `${LLMS_SUMMARY}\n\n${LLMS_DETAILS}\n\nThe home page lists every creator with a published plan.` }] },
++ { title: 'Creators', pages: creatorPages },
++ { title: 'Plans', pages: planPages },
++ ],
++ });
++ }
++ let llmsFullCache = { at: 0, value: null };
++ router.get('/llms-full.txt', async (req, res, next) => {
++ try {
++ if (!llmsFullCache.value || domain.now() - llmsFullCache.at >= LLMS_FULL_TTL_MS) llmsFullCache = { at: domain.now(), value: await buildLlmsFull() };
++ res.setHeader('Cache-Control', 'public, max-age=3600');
++ res.type('text/plain').send(llmsFullCache.value);
++ } catch (e) { next(e); }
++ });
++
// What shipped on OpenVibe.VIP: the shared update log every OpenVibe site has.
router.get('/updates', (req, res) => send(res, 200, { viewer: req.viewer, canonicalPath: '/updates', title: 'What shipped on OpenVibe.VIP', body: frame.updatesBody({ service: 'vip', siteName: 'OpenVibe.VIP' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>` }));
router.get('/', async (req, res) => send(res, 200, {
ddiff --git a/test/api.test.js b/test/api.test.js
iindex ae8abd9..6df4c36 100644
---- a/test/api.test.js
++++ b/test/api.test.js
@@@ -226,6 +226,16 @@ const { test, run } = harness('api');
assert.match(llms.headers.get('content-type'), /^text\/plain/);
assert.match(llms.text, /^# /, 'llms.txt starts with a markdown heading');
assert.match(llms.text, /\[Creators with published plans\]\(http:\/\/vip\.test\/\)/, 'llms.txt lists the main public pages');
++ const full = await t.call('GET', '/llms-full.txt', { token: null });
++ assert.strictEqual(full.status, 200);
++ assert.match(full.headers.get('content-type'), /^text\/plain/);
++ assert.strictEqual(full.headers.get('cache-control'), 'public, max-age=3600', 'llms-full.txt is cached for an hour');
++ assert.match(full.text, /^# OpenVibe\.VIP/, 'llms-full.txt starts with the llms.txt header');
++ assert.match(full.text, /URL: http:\/\/vip\.test\/\n/, 'the home page');
++ assert.match(full.text, /URL: http:\/\/vip\.test\/lena\n/, 'the public creator page');
++ assert.match(full.text, new RegExp(`URL: http:\\/\\/vip\\.test\\/lena\\/plans\\/${plan.slug}\\n`), 'the plan terms page');
++ assert.match(full.text, /Front row/, 'the full plan text');
++ assert.match(full.text, /Monthly stream/, 'a benefit of the current version');
});
test('every rendered page carries the boost marker, boost script (data-main) and the {path} sign-in template', async () => {
ddiff --git a/test/helpers/app.js b/test/helpers/app.js
iindex 15d84cc..eace4ef 100644
---- a/test/helpers/app.js
++++ b/test/helpers/app.js
@@@ -47,7 +47,7 @@ async function boot(opts = {}) {
const log = { log: (...a) => logs.push(a.join(' ')), warn: (...a) => logs.push(a.join(' ')), error: (...a) => logs.push(a.join(' ')), debug() {} };
// One database per boot (PGlite, or VIP_TEST_STORE=pg: the containers), dropped when the boot closes.
const testdb = opts.testdb || await require('./db').testDb();
-- const app = await createApp({ config, db: testdb.db, now: clock.now, log });
++ const app = await createApp({ config, db: testdb.db, now: clock.now, log, indexnow: opts.indexnow });
const server = await new Promise((resolve) => { const s = http.createServer(app); s.listen(0, '127.0.0.1', () => resolve(s)); });
const base = `http://127.0.0.1:${server.address().port}`;
const domain = app.locals.domain;
ddiff --git a/test/indexnow.test.js b/test/indexnow.test.js
nnew file mode 100644
iindex 0000000..79ec71e
---- /dev/null
++++ b/test/indexnow.test.js
@@@ -0,0 +1,80 @@
++'use strict';
++/**
++ * IndexNow (openvibe-shared/indexnow): INDEXNOW_KEY unset → the feature is off (no key route,
++ * nothing sent). With a key, the key file is served at /<key>.txt as text/plain and publishing,
++ * editing or archiving a plan pings the engines with the creator's page, the plan's terms page and
++ * the sitemap. A draft, an unchanged edit and a replay never ping.
++ */
++const assert = require('assert');
++const { boot, harness } = require('./helpers/app');
++
++const { test, run } = harness('indexnow');
++const KEY = 'k'.repeat(32);
++
++(async () => {
++ const off = await boot();
++ const on = await boot({ env: { INDEXNOW_KEY: KEY } });
++
++ // A spy in place of the module's HTTP send: records every pingSoon batch.
++ const pings = [];
++ const spy = {
++ enabled: true,
++ keyFile: (_req, _res, next) => next(),
++ pingSoon: (urls) => { const a = Array.isArray(urls) ? urls : [urls]; pings.push(...a); return a.length; },
++ ping: async () => ({ sent: 0, status: 0 }),
++ flush: async () => ({ sent: 0, status: 0 }),
++ };
++ const t = await boot({ indexnow: spy });
++ const creator = t.network.newUser('alice', { role: 'streamer' });
++ let plan;
++
++ test('without a key IndexNow is off: no key route and nothing sent', async () => {
++ assert.strictEqual(off.app.locals.indexnow.enabled, false);
++ const res = await off.call('GET', `/${KEY}.txt`, { token: null });
++ assert.strictEqual(res.status, 404, res.text);
++ });
++
++ test('with a key the key file answers text/plain with the key, before static', async () => {
++ assert.strictEqual(on.app.locals.indexnow.enabled, true);
++ const res = await on.call('GET', `/${KEY}.txt`, { token: null });
++ assert.strictEqual(res.status, 200, res.text);
++ assert.match(res.headers.get('content-type'), /^text\/plain/);
++ assert.strictEqual(res.headers.get('cache-control'), 'public, max-age=3600');
++ assert.strictEqual(res.text, KEY);
++ });
++
++ test('a draft never pings', async () => {
++ const res = await t.call('POST', '/api/v1/plans', { user: creator, body: { name: 'First light' } });
++ assert.strictEqual(res.status, 201, res.text);
++ plan = res.json.plan;
++ assert.deepStrictEqual(pings, []);
++ });
++
++ test('publishing pings the creator page, the terms page and the sitemap', async () => {
++ const res = await t.call('POST', `/api/v1/plans/${plan.id}/publish`, { user: creator });
++ assert.strictEqual(res.status, 200, res.text);
++ assert.ok(pings.includes('http://vip.test/alice'), JSON.stringify(pings));
++ assert.ok(pings.includes(`http://vip.test/alice/plans/${plan.slug}`), JSON.stringify(pings));
++ assert.ok(pings.includes('http://vip.test/sitemap.xml'), JSON.stringify(pings));
++ });
++
++ test('an unchanged edit does not ping; a real edit of a published plan does', async () => {
++ pings.length = 0;
++ await t.call('PATCH', `/api/v1/plans/${plan.id}`, { user: creator, body: { name: 'First light' } });
++ assert.deepStrictEqual(pings, [], 'an unchanged edit is not a change');
++ const res = await t.call('PATCH', `/api/v1/plans/${plan.id}`, { user: creator, body: { description: 'Now with more.' } });
++ assert.strictEqual(res.status, 200, res.text);
++ assert.ok(pings.includes('http://vip.test/alice'), JSON.stringify(pings));
++ assert.ok(pings.includes(`http://vip.test/alice/plans/${plan.slug}`), JSON.stringify(pings));
++ });
++
++ test('archiving pings the creator page and the sitemap', async () => {
++ pings.length = 0;
++ const res = await t.call('POST', `/api/v1/plans/${plan.id}/archive`, { user: creator });
++ assert.strictEqual(res.status, 200, res.text);
++ assert.ok(pings.includes('http://vip.test/alice'), JSON.stringify(pings));
++ assert.ok(pings.includes('http://vip.test/sitemap.xml'), JSON.stringify(pings));
++ });
++
++ await run().finally(async () => { await t.close(); await on.close(); await off.close(); });
++})();
ddiff --git a/test/llms-full.test.js b/test/llms-full.test.js
nnew file mode 100644
iindex 0000000..7c30b7e
---- /dev/null
++++ b/test/llms-full.test.js
@@@ -0,0 +1,48 @@
++'use strict';
++/**
++ * /llms-full.txt: the /llms.txt header plus the full text of every public page — the home copy,
++ * each creator's plan page and every published plan's terms. It is public data, cached for an hour.
++ */
++const assert = require('assert');
++const { boot, harness } = require('./helpers/app');
++
++const { test, run } = harness('llms-full');
++
++(async () => {
++ const t = await boot();
++ const creator = t.network.newUser('dana', { role: 'streamer' });
++ let plan;
++
++ test('the full text covers the home, the creator page and the plan terms', async () => {
++ const res = await t.call('POST', '/api/v1/plans', { user: creator, body: { name: 'Backstage', description: 'First description', benefits: ['A badge'], publish: true } });
++ assert.strictEqual(res.status, 201, res.text);
++ plan = res.json.plan;
++ const full = await t.call('GET', '/llms-full.txt', { token: null });
++ assert.strictEqual(full.status, 200);
++ assert.match(full.headers.get('content-type'), /^text\/plain/);
++ assert.strictEqual(full.headers.get('cache-control'), 'public, max-age=3600');
++ assert.match(full.text, /^# OpenVibe\.VIP/, 'the llms.txt header comes first');
++ assert.match(full.text, /## Home/);
++ assert.match(full.text, /## Creators/);
++ assert.match(full.text, /## Plans/);
++ assert.match(full.text, /URL: http:\/\/vip\.test\/\n/, 'the home page');
++ assert.match(full.text, /URL: http:\/\/vip\.test\/dana\n/, 'the public creator page');
++ assert.match(full.text, new RegExp(`URL: http:\\/\\/vip\\.test\\/dana\\/plans\\/${plan.slug}\\n`), 'the plan terms page');
++ assert.match(full.text, /First description/, 'the full plan text, never clipped');
++ assert.match(full.text, /@dana/);
++ });
++
++ test('the body is built once and cached for an hour', async () => {
++ const before = (await t.call('GET', '/llms-full.txt', { token: null })).text;
++ const edit = await t.call('PATCH', `/api/v1/plans/${plan.id}`, { user: creator, body: { description: 'Second description' } });
++ assert.strictEqual(edit.status, 200, edit.text);
++ const cached = (await t.call('GET', '/llms-full.txt', { token: null })).text;
++ assert.strictEqual(cached, before, 'within the hour the cached copy is served unchanged');
++ assert.doesNotMatch(cached, /Second description/);
++ t.clock.advance(3600 * 1000 + 1);
++ const fresh = (await t.call('GET', '/llms-full.txt', { token: null })).text;
++ assert.match(fresh, /Second description/, 'after the hour the body is rebuilt from the database');
++ });
++
++ await run().finally(async () => await t.close());
++})();