openvibers/openvibe.host Commits

Compose Host's own pages with shell.page and serve llms.txt files

OP OpenVibers committed 4 Oct 2026 · 0b3f7c7575 · 7 files, +131 −35

Asked: T11 step 7 (Host half) + step 4(c) for Host — OpenVibe.Host: adopt the openvibe-shared v2.6.0 page shell for Host's own public pages and add its crawl files. ONE pull request from current origin/main (git fetch first; the local checkout is stale). The dossier section 'T11 step 4' is attached by the…
Harness job host-t11-step-7-host-half-221039 (plan T11).

Changes

ddiff --git a/README.md b/README.md
iindex 95816de..b882888 100644
---- a/README.md
++++ b/README.md
@@@ -52,7 +52,7 @@ Stage A adds a few safety rules of its own:
- OpenVibe.Contracts (`openvibe-contracts` v0.79.0; `host.*` capabilities and the `host` manifest were released in v0.24.0, and v0.32.0 adds the takedown routes to `host.site.manage`): service manifests (vhost rendering, snapshots, the first-party domain list), ids, problem+json, service-token verification, capability checks.
- OpenVibe.Network (Stage B): SSO for the dashboard, the JWKS that verifies user and service tokens, client-credentials tokens for the outbox relay.
- OpenVibe.Events (Stage B): `host.*` events through the `openvibe-sdk` v0.25.0 transactional outbox (openvibe-sdk/limits for the per-actor limits).
--- `openvibe-shared` v2.2.0 (Stage B): shared chrome, legal pages, `/release.json`, `/metrics`, `/api/ready`.
++- `openvibe-shared` v2.6.0 (Stage B): shared chrome (Host's own pages are composed with `openvibe-shared/shell`), legal pages, `/release.json`, `/metrics`, `/api/ready`.
- OpenVibe.Media: not yet. The roadmap stores artifacts "through Media where practical"; Stage B keeps them on local disk for now (see [Not done yet](#not-done-yet-stage-b)).
## Capabilities
@@@ -249,7 +249,7 @@ A Node service (`server/`, Express 4, better-sqlite3, port **4910**, service id
| Host | What answers |
|---|---|
--| `openvibe.host` (the `BASE_URL` host) and loopback | the dashboard, `/api/v1`, `/auth/*`, `/api/ready`, `/api/health`, `/release.json`, `/limits.json` (the default project limits, from config), `/metrics` (direct loopback only), legal pages |
++| `openvibe.host` (the `BASE_URL` host) and loopback | the dashboard, `/api/v1`, `/auth/*`, `/api/ready`, `/api/health`, `/release.json`, `/limits.json` (the default project limits, from config), `/metrics` (direct loopback only), legal pages, and the crawl files `/robots.txt`, `/sitemap.xml`, `/llms.txt`, `/llms-full.txt` (`text/plain`, cached like the sitemap; they list only Host's own public pages: the front page, `/updates` and the legal pages, never a customer site, a preview or an operator path) |
| `<site>.openvibe.host` (one label) | that site's active deploy, and nothing else: no API, no sign-in, no cookies |
| a custom domain with `status = verified` | the site it was verified for |
| anything else (including pending, failed or lapsed custom domains, and `a.b.openvibe.host`) | `404 Unknown host`, with no tenant content |
@@@ -370,7 +370,7 @@ These tenant events have subject `deploy` and visibility `internal`; `host.deplo
### Dashboard
--Server-rendered pages with the shared chrome (`openvibe-shared` v2.2.0: navbar and theme loader from the Network, `<noscript>` navigation, SSR footer, app icon, legal pages). Every action is a plain form, so it works without JavaScript: projects, quotas and usage, members, sites, folder or archive upload, deploys with activate/rollback/delete, activation history, upload logs and file lists, domains with their DNS records and a "check DNS now" button. Pages are `private, no-store`, `noindex`, `frame-ancestors 'none'`.
++Server-rendered pages with the shared chrome (`openvibe-shared` v2.6.0 `shell.page`: navbar and theme loader from the Network, `<noscript>` navigation, SSR footer, app icon, legal pages). Every action is a plain form, so it works without JavaScript: projects, quotas and usage, members, sites, folder or archive upload, deploys with activate/rollback/delete, activation history, upload logs and file lists, domains with their DNS records and a "check DNS now" button. Pages are `private, no-store`, `noindex`, `frame-ancestors 'none'`.
### Observability
ddiff --git a/STATUS.json b/STATUS.json
iindex 1ccee00..2259c19 100644
---- a/STATUS.json
++++ b/STATUS.json
@@@ -2,6 +2,7 @@
"repository": "OpenVibers/OpenVibe.Host",
"stage": "alpha",
"code": true,
++ "crawlRoutes": ["/robots.txt", "/sitemap.xml", "/llms.txt", "/llms-full.txt"],
"wave": "W21 Stage A + Stage B",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
"runtime": "node 22; Stage A: CLI (ovhost); Stage B: express 4 + PostgreSQL 18 (openvibe-sdk/db, async) + Valkey 9 service",
@@@ -12,7 +13,7 @@
"packages": [
"openvibe-contracts v0.79.0",
"openvibe-sdk v0.25.0",
-- "openvibe-shared v2.2.0"
++ "openvibe-shared v2.6.0"
],
"capabilities": [
"host.site.manage",
ddiff --git a/package-lock.json b/package-lock.json
iindex 457aed2..a67654b 100644
---- a/package-lock.json
++++ b/package-lock.json
@@@ -19,7 +19,7 @@
"iovalkey": "^0.4.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.83.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.26.0",
-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.5.0",
++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.6.0",
"pg": "^8.23.0"
},
"bin": {
@@@ -1546,9 +1546,9 @@
}
},
"node_modules/openvibe-shared": {
-- "version": "2.5.0",
-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.5.0",
-- "integrity": "sha512-fHuUS6SXM7ezatfAyqLm1V8B6puV9YXSnz6aG6XQbWx28qghRCCHeUF3lr4nxiCGdn9SyikYtIyLGL5r1ycp9Q==",
++ "version": "2.6.0",
++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.6.0",
++ "integrity": "sha512-IKKvWTM/9UonQ4msrzM4yir0QcTUNbhyqeN1r5ywWavU4VTsL1sI5QdHQlxge7BAsMrQ4OvMEPlHjuAfUJM3rg==",
"license": "MIT",
"dependencies": {
"jsonwebtoken": "^9.0.0"
ddiff --git a/package.json b/package.json
iindex e2035df..6712003 100644
---- a/package.json
++++ b/package.json
@@@ -35,7 +35,7 @@
"iovalkey": "^0.4.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.83.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.26.0",
-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.5.0",
++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v2.6.0",
"pg": "^8.23.0"
},
"engines": {
ddiff --git a/server/app.js b/server/app.js
iindex ac79189..8478059 100644
---- a/server/app.js
++++ b/server/app.js
@@@ -20,6 +20,7 @@ const rateLimit = require('express-rate-limit');
const contracts = require('openvibe-contracts');
const sharedMetrics = require('openvibe-shared/metrics');
const cache = require('openvibe-shared/cache-policy');
++const sharedSeo = require('openvibe-shared/seo');
const { createIndexNow } = require('openvibe-shared/indexnow');
const configLib = require('./config');
@@@ -210,6 +211,35 @@ async function createApp(opts = {}) {
.send(['User-agent: *', 'Allow: /$', ...legalPaths.map((p) => `Allow: ${p}$`), 'Disallow: /', '', `Sitemap: ${config.baseUrl}/sitemap.xml`, ''].join('\n')));
app.get('/sitemap.xml', (_req, res) => res.type('application/xml').set('Cache-Control', cache.htmlHeaders({ maxAge: 3600 }))
.send(`<?xml version="1.0" encoding="UTF-8"?>\n<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n${['/', ...legalPaths].map((p) => ` <url><loc>${config.baseUrl}${p}</loc></url>`).join('\n')}\n</urlset>\n`));
++ // /llms.txt and /llms-full.txt (llmstxt.org): a map of Host's own public pages only. A tenant site, a preview
++ // URL, the dashboard, /api and /internal are never listed (a tenant's own /llms.txt is its deploy's file).
++ const SITE_SUMMARY = 'OpenVibe.Host: static site hosting for OpenVibe projects (alpha). Immutable deploys, custom domains and one-step rollback.';
++ const legalTitles = require('openvibe-shared/legal').TITLES;
++ const publicPages = [
++ { title: 'OpenVibe.Host', url: '/', text: 'The public front page of OpenVibe.Host: static site hosting for OpenVibe projects. Upload a folder or a .tar.gz of HTML, CSS, JavaScript, images and fonts; Host keeps every deploy as an immutable artifact, serves the active one on a <site>.openvibe.host address or a verified custom domain, and rolls back in one step. Alpha: static files only, no build step and no server-side code.' },
++ { title: 'What shipped on OpenVibe.Host', url: '/updates', text: 'The public update log for OpenVibe.Host: what shipped, newest first.' },
++ ...legalPaths.map((p) => ({ title: legalTitles[p.slice(1)] || p.slice(1), url: p, text: `The ${legalTitles[p.slice(1)] || p.slice(1)} of OpenVibe.Host.` })),
++ ];
++ app.get('/llms.txt', (_req, res) => res.type('text/plain').set('Cache-Control', cache.htmlHeaders({ maxAge: 3600 })).send(sharedSeo.llmsTxt({
++ name: 'OpenVibe.Host',
++ summary: SITE_SUMMARY,
++ details: 'The sites Host serves belong to their owners and are not listed here; the dashboard, sign-in and the API are per-person.',
++ sections: [
++ { title: 'Public pages', links: publicPages.map((p) => ({ title: p.title, url: `${config.baseUrl}${p.url}` })) },
++ { title: 'Machine-readable', links: [
++ { title: 'Sitemap', url: `${config.baseUrl}/sitemap.xml`, note: 'the public pages' },
++ { title: 'robots.txt', url: `${config.baseUrl}/robots.txt` },
++ { title: 'llms-full.txt', url: `${config.baseUrl}/llms-full.txt`, note: 'this same map with a description of each public page' },
++ ] },
++ ],
++ })));
++ app.get('/llms-full.txt', (_req, res) => res.type('text/plain').set('Cache-Control', cache.htmlHeaders({ maxAge: 3600 })).send(sharedSeo.llmsFull({
++ site: { name: 'OpenVibe.Host', url: config.baseUrl },
++ summary: SITE_SUMMARY,
++ base: config.baseUrl,
++ maxBytes: 512 * 1024,
++ sections: [{ title: 'Public pages', pages: publicPages }],
++ })));
// ── Sign-in (OAuth2 client of OpenVibe.Network) ─────────
app.use('/auth/', rateLimit({ windowMs: 15 * 60_000, max: 60, standardHeaders: true, legacyHeaders: false }));
ddiff --git a/server/render/layout.js b/server/render/layout.js
iindex 1ecd82b..041d7f5 100644
---- a/server/render/layout.js
++++ b/server/render/layout.js
@@@ -11,6 +11,7 @@ const fs = require('fs');
const path = require('path');
const appIcon = require('openvibe-shared/app-icon');
const frame = require('openvibe-shared/frame');
++const shell = require('openvibe-shared/shell');
const NETWORK_URL = 'https://openvibe.network';
const SITE_NAME = 'OpenVibe.Host';
@@@ -58,43 +59,40 @@ function renderPage(o) {
? `Signed in as ${who} · <a href="/auth/logout?next=%2F">Sign out</a>`
: `<a href="/auth/login?next=${loginNext}">Sign in with OpenVibe</a>`;
const notice = o.notice ? `<p class="notice notice-${esc(o.notice.kind || 'info')}" role="status">${esc(o.notice.text)}</p>` : '';
-- return `<!DOCTYPE html>
--<html lang="en">
--<head>
--<meta charset="utf-8">
--<meta name="viewport" content="width=device-width, initial-scale=1">
--<title>${esc(o.title ? `${o.title} · ${SITE_NAME}` : SITE_NAME)}</title>
--<meta name="robots" content="${o.indexable ? 'index, follow' : 'noindex, nofollow'}">
--${o.indexable ? `<link rel="canonical" href="${esc(`${o.config.baseUrl}${o.path && o.path.split('?')[0] !== '/' ? o.path.split('?')[0] : '/'}`)}">` : ''}
--<meta name="description" content="OpenVibe.Host: static site hosting for OpenVibe projects (alpha).">
--${appIcon.headTags({ site: 'host' })}
--<link rel="stylesheet" href="${asset('css/host.css')}">
--<script src="${ovServe.url('theme-loader.js')}" defer></script>
--<script src="${ovServe.url('navbar.js')}" defer></script>
--<script src="${ovServe.url('footer.js')}" defer></script>
--<meta name="ov-boost" content="host@${esc(RELEASE)}">
--<script src="${ovServe.url('boost.js')}" data-main="#main" defer></script>
--</head>
--<body>
--<a class="skip" href="#main">Skip to content</a>
++ // shell.page (openvibe-shared/shell) writes the document, the SEO head, the theme-loader, navbar.js and
++ // footer.js with the navbar init, the noscript nav and the SSR footer; the rest of the head and the body
++ // are this site's own. Only Host's own pages come through here, never a tenant site (http/tenant.js).
++ const canonical = o.indexable ? `${o.config.baseUrl}${o.path && o.path.split('?')[0] !== '/' ? o.path.split('?')[0] : '/'}` : undefined;
++ return shell.page({
++ name: SITE_NAME, service: 'host', lang: 'en',
++ title: o.title || SITE_NAME,
++ titleSuffix: o.title ? ` · ${SITE_NAME}` : undefined,
++ siteName: SITE_NAME,
++ description: 'OpenVibe.Host: static site hosting for OpenVibe projects (alpha).',
++ canonical,
++ robots: o.indexable ? 'index, follow' : 'noindex, nofollow',
++ navbar: nav, footer, home: '/', navLinks: [{ label: 'Projects', href: '/' }],
++ head: [
++ appIcon.headTags({ site: 'host' }),
++ `<link rel="stylesheet" href="${asset('css/host.css')}">`,
++ `<meta name="ov-boost" content="host@${esc(RELEASE)}">`,
++ `<script src="${ovServe.url('boost.js')}" data-main="#main" defer></script>`,
++ ].join('\n'),
++ body: `<a class="skip" href="#main">Skip to content</a>
<div id="navbar-mount"></div>
--${frame.noscriptNav({ name: SITE_NAME, home: '/', links: [{ label: 'Projects', href: '/' }] })}
<noscript><div class="account-bar" role="navigation" aria-label="Account">${account}</div></noscript>
<main id="main" class="page">
${notice}
${o.body || ''}
${o.path === '/' ? frame.shipped({ service: 'host', title: `Recently shipped on ${SITE_NAME}` }) : ''}
</main>
--${frame.footer(footer)}
<script>
window.__OV_PAGE = ${JSON.stringify({ navbar: nav, footer }).replace(/</g, '\\u003c')};
document.addEventListener('DOMContentLoaded', function () {
-- try { if (window.OpenVibeNavbar) OpenVibeNavbar.init(window.__OV_PAGE.navbar); } catch (e) { /* the Frame is optional */ }
-- try { if (window.OpenVibeFooter) OpenVibeFooter.init(window.__OV_PAGE.footer); } catch (e) { /* */ }
++ try { if (window.OpenVibeFooter) OpenVibeFooter.init(window.__OV_PAGE.footer); } catch (e) { /* the SSR footer stays */ }
});
--</script>
--</body>
--</html>`;
++</script>`,
++ });
}
module.exports = { renderPage, esc, asset, assetVersion, setRelease, SITE_NAME, NETWORK_URL };
ddiff --git a/test/host-llms.test.js b/test/host-llms.test.js
nnew file mode 100644
iindex 0000000..802b752
---- /dev/null
++++ b/test/host-llms.test.js
@@@ -0,0 +1,67 @@
++'use strict';
++/**
++ * /llms.txt and /llms-full.txt (openvibe-shared/seo, llmstxt.org) list Host's own public pages only:
++ * text/plain, cached like the sitemap, never a customer site, a preview URL, /internal or the dashboard.
++ * Host's own pages come through openvibe-shared/shell, so the public front page keeps its title,
++ * canonical and robots and now carries the shell's markers (the runtime boot, the Open Graph head).
++ */
++const assert = require('assert');
++const { boot, check, done, DASHBOARD } = require('./stageb/boot');
++
++(async () => {
++ const t = await boot();
++ const alice = t.user('alice', { display_name: 'Alice' });
++ const project = await t.project(alice, 'Widgets');
++ const site = await t.site(alice, project.id, 'widgets');
++ await t.deploy(alice, site.id, { 'index.html': '<h1>Widgets</h1>' }, { activate: true });
++ const sitemap = await t.get(DASHBOARD, '/sitemap.xml');
++
++ for (const file of ['/llms.txt', '/llms-full.txt']) {
++ await check(`${file} is 200 text/plain, cached like the sitemap, and lists only Host's public pages`, async () => {
++ const r = await t.get(DASHBOARD, file);
++ assert.strictEqual(r.status, 200, r.text);
++ assert.match(r.headers['content-type'], /^text\/plain/);
++ assert.strictEqual(r.headers['cache-control'], sitemap.headers['cache-control']);
++ assert.match(r.text, /^# OpenVibe\.Host/);
++ const urls = new Set([...r.text.matchAll(/https?:\/\/[^\s)>\]]+/g)].map((m) => m[0]));
++ for (const url of urls) assert.match(url, /^https:\/\/openvibe\.host\//, `${url} is not a Host page`);
++ for (const page of ['/', '/updates', '/terms', '/privacy', '/dmca']) assert.ok(urls.has(`https://openvibe.host${page}`), `${page} missing`);
++ for (const hidden of ['widgets', 'Widgets', 'alice', 'preview', '/internal', '/api/', '/projects', '/sites/', '/auth/']) {
++ assert.ok(!r.text.includes(hidden), `${hidden} in ${file}`);
++ }
++ });
++ }
++
++ await check('/llms-full.txt carries a description of each public page and stays within 512 KiB', async () => {
++ const r = await t.get(DASHBOARD, '/llms-full.txt');
++ assert.match(r.text, /## Public pages/);
++ assert.match(r.text, /immutable artifact/);
++ assert.ok(Buffer.byteLength(r.text) <= 512 * 1024);
++ });
++
++ await check("the public front page goes through the shell: title, canonical, robots and the shell's markers", async () => {
++ const r = await t.api('GET', '/');
++ assert.strictEqual(r.status, 200, r.text);
++ assert.match(r.text, /^<!doctype html>/i);
++ assert.match(r.text, /<title>OpenVibe\.Host<\/title>/);
++ assert.match(r.text, /<link rel="canonical" href="https:\/\/openvibe\.host\/">/);
++ assert.match(r.text, /<meta name="robots" content="index, follow">/);
++ assert.match(r.text, /<meta property="og:site_name" content="OpenVibe\.Host">/);
++ assert.match(r.text, /<script src="\/shared\/web-runtime\.js\?v=[0-9a-f]+" defer><\/script>/);
++ assert.match(r.text, /OVWebRuntime\.create\(/);
++ assert.match(r.text, /<main id="main" class="page">/);
++ });
++
++ await check('the updates page keeps its suffixed title and canonical; a private page stays noindex without one', async () => {
++ const u = await t.api('GET', '/updates');
++ assert.match(u.text, /<title>What shipped on OpenVibe\.Host · OpenVibe\.Host<\/title>/);
++ assert.match(u.text, /<link rel="canonical" href="https:\/\/openvibe\.host\/updates">/);
++ const nf = await t.api('GET', '/no-such-page');
++ assert.strictEqual(nf.status, 404);
++ assert.match(nf.text, /<meta name="robots" content="noindex, nofollow">/);
++ assert.ok(!/rel="canonical"/.test(nf.text));
++ });
++
++ await t.close();
++ done();
++})().catch((err) => { console.error(err); process.exit(1); });