openvibers/openvibe.shared Commits

browser-harness: a noindex page (meta robots or X-Robots-Tag) needs no canonical link (Sources' landing page is noindex); a canonical it has is still judged

OP OpenVibers committed 26 Sept 2026 · 6d4420f272 · 4 files, +24 −8


Changes

ddiff --git a/CHANGELOG.md b/CHANGELOG.md
iindex 91b7213..5773ab7 100644
---- a/CHANGELOG.md
++++ b/CHANGELOG.md
@@@ -6,7 +6,7 @@ A release is the git tag `vX.Y.Z`; consumers pin the tag's tarball (see README).
## 1.16.0 — 2026-09-26
--**`openvibe-shared/browser-harness`** (roadmap WS-Q task 3, WS-T tasks 3 and 4): Live's browser smoke suite, generalised for any site. It drives headless Chrome over the DevTools protocol with Node's own WebSocket (Node 22+, no dependency), one browser context per route. `run({ base, routes, widths, checks, navigation, … })` checks every route at 390, 768 and 1280 px: the HTTP status (a route can expect 404), console errors (exceptions, `console.error`, failed loads, CSP), horizontal overflow (naming the elements) and scripts requested twice. With JavaScript off it checks the visible text (200 characters), exactly one absolute canonical link, and that JSON-LD parses and its headline or name is in the visible text. For an ItemList, at least 80% of the item names must be visible. axe-core runs at the widest width: serious and critical violations fail, moderate is reported. Once per site, A → B → A navigation runs five times, clicking a link when there is one so a single-page app navigates in place. After a forced GC it samples heap, DOM nodes, listeners and documents (`Performance.getMetrics`), plus live intervals, pending timeouts and open WebSockets (a probe installed before page scripts). A measure fails when it grows past its budget from lap 2 to the last lap and is still growing (`BUDGETS`). CPU, requests and bytes over 5 idle seconds are reported, not failed. `ignoreErrors` rules are counted per label, never dropped silently. `format()` gives text or Markdown, and `summarize()` gives per-check counts. axe-core is not vendored: an installed `axe-core` is used, else 4.13.0 comes from jsDelivr, checked against a pinned sha384 (the same bytes as the npm tarball) and cached. `test/browser-harness-chrome.test.js` runs it in real Chrome against a fixture and skips without Chrome. Additive.
++**`openvibe-shared/browser-harness`** (roadmap WS-Q task 3, WS-T tasks 3 and 4): Live's browser smoke suite, generalised for any site. It drives headless Chrome over the DevTools protocol with Node's own WebSocket (Node 22+, no dependency), one browser context per route. `run({ base, routes, widths, checks, navigation, … })` checks every route at 390, 768 and 1280 px: the HTTP status (a route can expect 404), console errors (exceptions, `console.error`, failed loads, CSP), horizontal overflow (naming the elements) and scripts requested twice. With JavaScript off it checks the visible text (200 characters), exactly one absolute canonical link (a noindex page may have none), and that JSON-LD parses and its headline or name is in the visible text. For an ItemList, at least 80% of the item names must be visible. axe-core runs at the widest width: serious and critical violations fail, moderate is reported. Once per site, A → B → A navigation runs five times, clicking a link when there is one so a single-page app navigates in place. After a forced GC it samples heap, DOM nodes, listeners and documents (`Performance.getMetrics`), plus live intervals, pending timeouts and open WebSockets (a probe installed before page scripts). A measure fails when it grows past its budget from lap 2 to the last lap and is still growing (`BUDGETS`). CPU, requests and bytes over 5 idle seconds are reported, not failed. `ignoreErrors` rules are counted per label, never dropped silently. `format()` gives text or Markdown, and `summarize()` gives per-check counts. axe-core is not vendored: an installed `axe-core` is used, else 4.13.0 comes from jsDelivr, checked against a pinned sha384 (the same bytes as the npm tarball) and cached. `test/browser-harness-chrome.test.js` runs it in real Chrome against a fixture and skips without Chrome. Additive.
Fix: the selected filter chip of the shared update log (`shipped.js`, every site's `/updates`) was white on `--accent` (3.67:1). It now uses `--accent-strong` / `--on-accent-strong` (4.5:1 or better in every theme), like the navbar's Sign In button. The harness found it.
ddiff --git a/browser-harness.js b/browser-harness.js
iindex 2d51028..c6a49f5 100644
---- a/browser-harness.js
++++ b/browser-harness.js
@@@ -17,7 +17,7 @@
* once per route, with JavaScript disabled (the initial HTML, what a crawler reads):
* nojs at least `minText` (200) characters of visible text
* canonical exactly one <link rel=canonical>, an absolute URL; another origin or a different URL with
-- * JavaScript on is a warning
++ * JavaScript on is a warning; a noindex page (meta robots or X-Robots-Tag) may have none
* jsonld every application/ld+json block parses, and the headline (else name) of each top-level entity
* (of an ItemList: 80 % of its item names) is in the visible text: without JavaScript passes, only
* with JavaScript warns, nowhere fails
@@@ -240,7 +240,10 @@ async function openPage(browser, context, { width = 1280, height = 900, js = tru
});
on('Network.responseReceived', (p) => {
activity();
-- if (p.type === 'Document' && p.frameId === st.mainFrame) st.doc = { status: p.response.status, url: p.response.url, mimeType: p.response.mimeType };
++ if (p.type === 'Document' && p.frameId === st.mainFrame) {
++ const hdr = Object.fromEntries(Object.entries(p.response.headers || {}).map(([k, v]) => [k.toLowerCase(), v]));
++ st.doc = { status: p.response.status, url: p.response.url, mimeType: p.response.mimeType, xRobots: hdr['x-robots-tag'] || '' };
++ }
});
on('Network.dataReceived', (p) => { st.bytes += p.encodedDataLength || 0; });
on('Network.loadingFinished', (p) => { activity(); st.inflight.delete(p.requestId); });
@@@ -333,6 +336,7 @@ const READ_PAGE = `(() => {
url: location.href, title: document.title, vw, scrollWidth, scrollX,
offenders: offenders.map((o) => o.desc),
canonical: [...document.querySelectorAll('link[rel~="canonical" i]')].map((l) => l.getAttribute('href')),
++ robots: [...document.querySelectorAll('meta[name="robots" i]')].map((m) => m.getAttribute('content') || '').join(', '),
jsonld: [...document.querySelectorAll('script[type="application/ld+json" i]')].map((s) => s.textContent),
h1: document.querySelectorAll('h1').length, links: document.querySelectorAll('a[href]').length,
textChars: text.length, text: text.slice(0, 300000),
@@@ -470,6 +474,8 @@ function checkRoute(route, r, { minText, ignoreErrors }) {
const n = r.nojs;
checks.nojs = !want('nojs') ? 'skip' : n.textChars >= minText ? 'pass' : 'fail';
if (!want('canonical')) checks.canonical = 'skip';
++ // A page that asks not to be indexed needs no canonical URL (one it has is still checked).
++ else if (!n.canonical.length && /noindex/i.test(n.robots || '')) { checks.canonical = 'skip'; r.canonical = { found: 0, note: 'noindex page: no canonical needed' }; }
else if (n.canonical.length !== 1) { checks.canonical = 'fail'; r.canonical = { found: n.canonical.length, note: n.canonical.length ? 'more than one canonical link' : 'no canonical link' }; }
else {
const href = n.canonical[0];
@@@ -672,7 +678,8 @@ async function checkOneRoute(browser, context, route, o) {
try {
await page.goto(url, { quietMs: o.settleMs, maxMs: o.maxSettleMs });
const s = await page.evaluate(READ_PAGE);
-- r.nojs = { status: page.state.doc ? page.state.doc.status : null, textChars: s.textChars, h1: s.h1, links: s.links, canonical: s.canonical, jsonld: s.jsonld, text: s.text, title: s.title };
++ r.nojs = { status: page.state.doc ? page.state.doc.status : null, textChars: s.textChars, h1: s.h1, links: s.links, canonical: s.canonical, jsonld: s.jsonld, text: s.text, title: s.title,
++ robots: [s.robots, page.state.doc && page.state.doc.xRobots].filter(Boolean).join(', ') };
if (page.state.doc && page.state.doc.error) r.nojs.error = page.state.doc.error;
} catch (e) { r.nojs = { error: e.message }; } finally { await page.close(); }
}
ddiff --git a/test/browser-harness-chrome.test.js b/test/browser-harness-chrome.test.js
iindex 238254d..1243337 100644
---- a/test/browser-harness-chrome.test.js
++++ b/test/browser-harness-chrome.test.js
@@@ -27,6 +27,7 @@ const ROUTES = {
<script>document.addEventListener('click', (e) => { const a = e.target.closest('a'); if (!a) return; e.preventDefault(); history.pushState(null, '', a.getAttribute('href'));
const j = document.getElementById('junk'); for (let i = 0; i < 200; i++) j.appendChild(document.createElement('span')); setInterval(() => {}, 100000); });</script></main>`,
'<link rel="canonical" href="ORIGIN/spa">'),
++ '/private': () => page(`<main><h1>Private</h1><p>${words}</p></main>`),
'/s.js': () => 'window.__s = (window.__s || 0) + 1;',
};
// axe stand-in: one critical violation on /b, none elsewhere.
@@@ -39,17 +40,17 @@ const AXE_STUB = `window.axe = { run: () => Promise.resolve({ violations: locati
const origin = `http://127.0.0.1:${server.address().port}`;
const route = ROUTES[u.pathname];
if (!route) { res.writeHead(404, { 'content-type': 'text/html' }); res.end(page(`<h1>Not found</h1><p>${words}</p>`)); return; }
-- res.writeHead(200, { 'content-type': u.pathname.endsWith('.js') ? 'application/javascript' : 'text/html; charset=utf-8' });
++ res.writeHead(200, { 'content-type': u.pathname.endsWith('.js') ? 'application/javascript' : 'text/html; charset=utf-8', ...(u.pathname === '/private' ? { 'x-robots-tag': 'noindex, nofollow' } : {}) });
res.end(route().replace(/ORIGIN/g, origin));
});
await new Promise((ok) => server.listen(0, '127.0.0.1', ok));
const base = `http://127.0.0.1:${server.address().port}`;
const report = await h.run({
-- base, routes: ['/', '/b', { path: '/missing', status: 404 }], widths: [390, 1280], settleMs: 300, idleMs: 1000,
++ base, routes: ['/', '/b', { path: '/missing', status: 404 }, '/private'], widths: [390, 1280], settleMs: 300, idleMs: 1000,
navigation: { from: '/spa', to: '/spa?b', laps: 4 }, axe: { source: AXE_STUB },
});
server.close();
-- const [good, bad, missing] = report.routes;
++ const [good, bad, missing, priv] = report.routes;
const text = h.format(report);
assert.ok(/^Chrome\/|HeadlessChrome\//.test(report.chrome), report.chrome);
@@@ -71,6 +72,9 @@ const AXE_STUB = `window.axe = { run: () => Promise.resolve({ violations: locati
// The 404 page: the expected status, and its own 404 is not counted as a console error.
assert.deepStrictEqual([missing.checks.status, missing.checks.errors, missing.widths[0].status], ['pass', 'pass', 404], text);
++ // X-Robots-Tag: noindex, so no canonical is needed.
++ assert.deepStrictEqual([priv.checks.canonical, priv.ok], ['skip', true], text);
++
// In-page navigation (the link is clicked, the document stays), growth caught, idle measured.
const nav = report.navigation;
assert.strictEqual(nav.mode, 'in-page');
@@@ -82,7 +86,7 @@ const AXE_STUB = `window.axe = { run: () => Promise.resolve({ violations: locati
assert.deepStrictEqual(nav.idle.animations, { running: 1, infinite: 1, list: ['spin on i.spinner'] }, 'the idle report names what keeps painting');
assert.strictEqual(report.ok, false);
-- assert.deepStrictEqual(report.summary.checks.overflow, { pass: 2, warn: 0, fail: 1, skip: 0 });
++ assert.deepStrictEqual(report.summary.checks.overflow, { pass: 3, warn: 0, fail: 1, skip: 0 });
assert.ok(text.includes('nodes grew'), text);
console.log('browser-harness-chrome: ok');
})().catch((e) => { console.error(e); process.exit(1); });
ddiff --git a/test/browser-harness.test.js b/test/browser-harness.test.js
iindex 8549fec..e19289c 100644
---- a/test/browser-harness.test.js
++++ b/test/browser-harness.test.js
@@@ -107,6 +107,11 @@ const h = require('../browser-harness');
assert.strictEqual(r.canonical.note, note);
assert.strictEqual(r.checks.axe, 'skip');
}
++ // A noindex page (meta robots or X-Robots-Tag) needs no canonical; one it has is still judged.
++ r = h.checkRoute(h.normalizeRoutes(['/p'])[0], { url: 'https://a.example/p', widths: [base], nojs: { ...nojs, canonical: [], robots: 'noindex, nofollow' }, axe: { skipped: true } }, { minText: 200 });
++ assert.deepStrictEqual([r.checks.canonical, r.canonical.note], ['skip', 'noindex page: no canonical needed']);
++ r = h.checkRoute(h.normalizeRoutes(['/p'])[0], { url: 'https://a.example/p', widths: [base], nojs: { ...nojs, canonical: ['/p'], robots: 'noindex' }, axe: { skipped: true } }, { minText: 200 });
++ assert.strictEqual(r.checks.canonical, 'fail');
// A 404 route: the status is what counts; no crawler checks; a no-JS load that failed fails the status.
r = h.checkRoute(h.normalizeRoutes([{ path: '/gone', status: 404 }])[0], { url: 'https://a.example/gone', widths: [{ ...base, status: 404 }], nojs: null, axe: { skipped: true } }, { minText: 200 });
assert.deepStrictEqual([r.checks.status, r.checks.nojs, r.checks.canonical, r.checks.jsonld], ['pass', 'skip', 'skip', 'skip']);