openvibers/openvibe.news Commits
News: AI briefs with tone and outlet comparisons, and a live breaking view (#68)
Changes
@@@ -26,6 +26,26 @@ shown, fed or indexed.News publishes nothing until an editor does: the only seed is the topic list. Until then, and beside the stories after, **the wire** (`/wire`, and "From the wire" on the front page) shows what the sources report, grouped by cluster: each line is the outlet's own headline, outlet, date and link (the fields the licensing rules let News keep), never News' writing; duplicates and reports removed upstream leave it; `/wire` is `noindex, follow`.++## AI briefs and Live++++The owner's April ask for News was multi-source summaries with tone and perspective controls and a live breaking-news++view. Stories stay editor-written (below); beside them News now has:++++- **AI briefs** (`/briefs`, `/briefs/:cluster`; [server/domain/briefs.js](server/domain/briefs.js),++ `migrations/0004_cluster_briefs.sql`): what a cluster's reports say, written by OpenVibe.AI and labelled as AI on++ every page. Every paragraph cites the reports it rests on, numbered in the table below it; claims the model did not++ cite are dropped and listed under "What the AI left out"; a run with no cited text makes no brief; the stub++ provider's output is never shown. Only News' stored fields are sent (headline, outlet, date, URL and the short summary++ the terms allow). The worker writes a neutral summary for every open cluster with two or more reports in the last 72++ hours, six a tick every `NEWS_AI_BRIEFS_INTERVAL_MS` (10 minutes), and writes it again once the cluster changes.++- **Tone** (`?tone=brief|neutral|explained`, `news.summarize_story`'s `tone`) and **how the outlets put it**++ (`news.compare_perspectives`, two outlets or more): made when a reader asks, with a POST from this site (five every ten++ minutes an address, 40 an hour in all), never by a link a crawler follows.++- **Live** (`/live`): the newest reports as they arrive, grouped, refreshing itself every minute (`Refresh: 60`), with++ **Breaking** on a subject three or more reports covered in the last two hours, and links to their AI briefs.++- Briefs and Live are `noindex`: they are AI text and other publishers' headlines, not News' writing.++ `NEWS_AI_BRIEFS=off` turns briefs off.++## OwnsThe nine charter tables live in News' own PostgreSQL database (`ov_news` on the host's data role, ADR-035; schema in [migrations/](migrations/)):nnew file mode 100644@@@ -0,0 +1,23 @@++-- phase: expand++-- AI briefs (server/domain/briefs.js; the owner's April ask: multi-source summaries with tone and perspective controls).++-- One row per cluster, kind and tone: a summary of what the cluster's reports say (tone brief | neutral | explained),++-- or how its outlets frame it (kind perspectives). Every paragraph cites the source items it rests on; the model's++-- uncited claims are dropped and listed in gaps. A brief is AI text, labelled as such, never a story: stories stay++-- editor-written. A brief is stale once its cluster changes (item_count or cluster_updated_at) and is made again.++CREATE TABLE IF NOT EXISTS news_cluster_briefs (++ cluster_id text COLLATE "C" NOT NULL REFERENCES news_story_clusters(id) ON DELETE CASCADE,++ kind text COLLATE "C" NOT NULL CHECK (kind IN ('summary', 'perspectives')),++ tone text COLLATE "C" NOT NULL DEFAULT 'neutral' CHECK (tone IN ('brief', 'neutral', 'explained')),++ headline text COLLATE "C",++ paragraphs text COLLATE "C" NOT NULL, -- JSON [{ text, sources: [nsi_…] }]++ source_ids text COLLATE "C" NOT NULL, -- JSON [nsi_…] the reports it was made from, in order++ gaps text COLLATE "C" NOT NULL DEFAULT '[]',++ run_id text COLLATE "C",++ model text COLLATE "C",++ stub boolean NOT NULL DEFAULT false, -- the stub provider's synthetic output: never shown++ item_count integer NOT NULL,++ cluster_updated_at bigint NOT NULL,++ created_at bigint NOT NULL,++ PRIMARY KEY (cluster_id, kind, tone)++);++CREATE INDEX IF NOT EXISTS news_cluster_briefs_recent ON news_cluster_briefs (kind, tone, created_at DESC);nnew file mode 100644@@@ -0,0 +1,17 @@++/* OpenVibe.News — the AI briefs and live pages only (render/briefs.js), so news.css stays inside its budget. */++.badge-ai { background: color-mix(in srgb, #a78bfa 18%, transparent); color: color-mix(in srgb, #8b5cf6 70%, var(--text-primary, #e6edf7)); border: 1px solid color-mix(in srgb, #a78bfa 45%, transparent); }++.badge-breaking { background: color-mix(in srgb, #ef4444 16%, transparent); color: color-mix(in srgb, #ef4444 75%, var(--text-primary, #e6edf7)); border: 1px solid color-mix(in srgb, #ef4444 45%, transparent); }++.eyebrow { margin: 0 0 .4rem; font-size: .9rem; }++.brief-tones { display: flex; flex-wrap: wrap; gap: 8px; margin: 12px 0 16px; }++.brief-tone { padding: 5px 12px; border-radius: 999px; border: 1px solid var(--border-light, #2c3d5c); text-decoration: none; color: inherit; font-size: .9rem; }++.brief-tone.on { border-color: var(--accent, #3b82f6); font-weight: 700; }++.brief .brief-p, .brief-frames .brief-p { max-width: 72ch; line-height: 1.6; }++.brief-make { display: inline; }++.brief-make button { font: inherit; padding: 5px 12px; border-radius: 8px; border: 1px solid var(--border-light, #2c3d5c); background: var(--bg-tertiary, #16223a); color: inherit; cursor: pointer; }++.brief-make button:hover, .brief-make button:focus-visible { border-color: var(--accent, #3b82f6); }++.brief-frames { margin-top: 1.6rem; }++.brief-gaps { margin-top: 1rem; font-size: .9rem; }++.live-dot { display: inline-block; width: .55em; height: .55em; border-radius: 50%; background: #ef4444; vertical-align: middle; animation: live-pulse 1.6s ease-in-out infinite; }++@keyframes live-pulse { 50% { opacity: .35; } }++@media (prefers-reduced-motion: reduce) { .live-dot { animation: none; } }++.wire-cluster.is-breaking { border-left: 3px solid #ef4444; padding-left: 10px; }@@@ -26,6 +26,7 @@ const { openStore } = require('./db');const { createViewerResolver } = require('./auth/viewer');const { createPeople } = require('./clients/network');const { createAi } = require('./clients/ai');++const { createBriefs } = require('./domain/briefs');const { createCommunity } = require('./clients/community');const { createPublication } = require('./domain/publication');const { createDiscussion } = require('./domain/discussion');@@@ -97,9 +98,10 @@ async function createApp(opts = {}) {secureCookies: config.cookies.secure,});const viewers = createViewerResolver({ auth, config, people });-- const worker = createWorker({ config, ingest, outbox, log });++ const briefs = createBriefs({ store, ai: config.briefs.enabled ? ai : null, log });++ const worker = createWorker({ config, ingest, outbox, briefs, log });-- const ctx = { config, store, outbox, publication, clusters, sources, ai, community, discussion, ingest, stories, topics, people, reading, auth, viewers, worker, indexnow };++ const ctx = { config, store, outbox, publication, clusters, sources, ai, community, discussion, ingest, stories, topics, people, reading, auth, viewers, worker, indexnow, briefs };const app = express();app.disable('x-powered-by');@@@ -29,11 +29,12 @@ function createAi({ config, fetchImpl = globalThis.fetch }) {}) : null;/** sources: [{ id (nsi_…), sources_item_id, headline, url, outlet, authors, published_at, retrieved_at, summary }] in citation order. */-- async function run(workflow, { topic, sources }) {++ async function run(workflow, { topic, sources, tone = null }) {if (!enabled) throw new AiError('ai.not_configured', 'OpenVibe.AI is not configured (OV_AI_INTERNAL_URL)', 503);if (!WORKFLOWS.includes(workflow)) throw new AiError('ai.unknown_workflow', `workflow must be one of ${WORKFLOWS.join(', ')}`, 422);const input = {...(topic ? { topic: String(topic).slice(0, 300) } : {}),++ ...(tone && workflow === 'news.summarize_story' ? { tone } : {}),sources: sources.slice(0, 50).map((s) => ({source_type: 'news.article', source_id: s.sources_item_id, title: s.headline, ...(s.url ? { url: s.url } : {}),...(s.authors && s.authors.length ? { author: s.authors.join(', ').slice(0, 200) } : {}),@@@ -119,6 +119,13 @@ function load(env = process.env) {enabled: env.NEWS_WORKER !== 'off',},++ // AI briefs (server/domain/briefs.js): the sweep summarizes busy clusters every intervalMs; off with++ // NEWS_AI_BRIEFS=off (readers' requests for a tone or the perspectives then answer 503 too).++ briefs: {++ enabled: env.NEWS_AI_BRIEFS !== 'off',++ intervalMs: Math.max(60_000, int(env.NEWS_AI_BRIEFS_INTERVAL_MS, 600_000)),++ },++// Browser origins that may call /api/v1 with a Bearer Network JWT (no cookies cross origins).apiCorsOrigins: list(env.API_CORS_ORIGINS || 'https://openvibe.network,https://openvibe.live,https://openvibe.community'),};nnew file mode 100644@@@ -0,0 +1,112 @@++'use strict';++++/**++ * AI briefs: what a cluster's reports say, written by OpenVibe.AI and labelled as AI (migrations/0004_cluster_briefs.sql).++ * The owner's April ask for News: multi-source summarization, tone controls and perspective/slant controls, and a live++ * breaking view. Stories stay what they were: editor-written, every paragraph cited, never AI text without a person's++ * approval. A brief is the other thing: an AI summary of a cluster, shown as such, beside the wire.++ *++ * summary news.summarize_story over the cluster's live reports, in a tone: brief, neutral (made by the sweep++ * for every cluster with at least MIN_REPORTS reports) or explained (made when a reader asks)++ * perspectives news.compare_perspectives: how the outlets frame it (when a reader asks; two outlets or more)++ *++ * Only News' stored fields go to AI (headline, outlet, date, URL, and the short summary its terms allow). A paragraph the++ * model did not cite is dropped (clients/ai.js draftFromOutput) and listed as a gap; a run with no cited text makes no++ * brief. The stub provider's synthetic output is stored but never shown. A brief made from a cluster that has since++ * changed is stale and is made again (the sweep for neutral summaries, the next reader's request for the rest).++ */++++const { draftFromOutput } = require('../clients/ai');++++const TONES = ['brief', 'neutral', 'explained'];++const TONE_TEXT = { brief: 'In brief', neutral: 'Summary', explained: 'Explained' };++const KINDS = ['summary', 'perspectives'];++const MIN_REPORTS = 2;++const SWEEP_PER_TICK = 6;++const SWEEP_WINDOW_MS = 72 * 3_600_000;++++const parse = (v, d) => { try { return typeof v === 'string' ? JSON.parse(v) : (v == null ? d : v); } catch { return d; } };++++function createBriefs({ store, ai, log = console }) {++ const db = store.db;++ const q = {++ cluster: db.prepare("SELECT * FROM news_story_clusters WHERE id = ? AND status = 'open'"),++ reports: db.prepare("SELECT * FROM news_source_items WHERE cluster_id = ? AND status = 'active' AND removed_at IS NULL ORDER BY published_at, first_seen_at, seq"),++ get: db.prepare('SELECT * FROM news_cluster_briefs WHERE cluster_id = ? AND kind = ? AND tone = ?'),++ put: db.prepare(`INSERT INTO news_cluster_briefs (cluster_id, kind, tone, headline, paragraphs, source_ids, gaps, run_id, model, stub, item_count, cluster_updated_at, created_at)++ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)++ ON CONFLICT (cluster_id, kind, tone) DO UPDATE SET headline = excluded.headline, paragraphs = excluded.paragraphs, source_ids = excluded.source_ids,++ gaps = excluded.gaps, run_id = excluded.run_id, model = excluded.model, stub = excluded.stub, item_count = excluded.item_count,++ cluster_updated_at = excluded.cluster_updated_at, created_at = excluded.created_at`),++ latest: db.prepare(`SELECT b.*, c.label, c.window_end FROM news_cluster_briefs b JOIN news_story_clusters c ON c.id = b.cluster_id++ WHERE b.kind = 'summary' AND b.tone = 'neutral' AND b.stub = false AND c.status = 'open' ORDER BY c.window_end DESC, b.created_at DESC LIMIT ?`),++ // Open clusters active in the window with at least MIN_REPORTS live reports, newest activity first.++ due: db.prepare(`SELECT c.id, c.updated_at, count(i.id) AS n FROM news_story_clusters c JOIN news_source_items i ON i.cluster_id = c.id AND i.status = 'active' AND i.removed_at IS NULL++ WHERE c.status = 'open' AND c.window_end >= ? GROUP BY c.id, c.updated_at, c.window_end HAVING count(i.id) >= ? ORDER BY c.window_end DESC LIMIT 200`),++ };++++ const fresh = (row, cluster, n) => Boolean(row && Number(row.item_count) === n && Number(row.cluster_updated_at) === Number(cluster.updated_at));++++ /** The stored brief, and whether it still matches its cluster. */++ async function get(clusterId, { kind = 'summary', tone = 'neutral' } = {}) {++ const cluster = await q.cluster.get(clusterId);++ if (!cluster) return null;++ const reports = await q.reports.all(cluster.id);++ const row = await q.get.get(cluster.id, kind, tone);++ return { cluster, reports, brief: row ? shape(row) : null, fresh: fresh(row, cluster, reports.length) };++ }++++ function shape(row) {++ return {++ cluster_id: row.cluster_id, kind: row.kind, tone: row.tone, headline: row.headline || null,++ paragraphs: parse(row.paragraphs, []), source_ids: parse(row.source_ids, []), gaps: parse(row.gaps, []),++ run_id: row.run_id || null, model: row.model || null, stub: Boolean(row.stub), item_count: Number(row.item_count), created_at: Number(row.created_at),++ };++ }++++ /** Make (or remake) one brief. Throws the AI client's error when the run fails; makes nothing without cited text. */++ async function make(clusterId, { kind = 'summary', tone = 'neutral' } = {}) {++ if (!ai || !ai.enabled) throw Object.assign(new Error('OpenVibe.AI is not configured on this server'), { code: 'ai.not_configured', status: 503 });++ if (!KINDS.includes(kind) || !TONES.includes(tone)) throw Object.assign(new Error('kind or tone not known'), { code: 'brief.invalid', status: 422 });++ const cluster = await q.cluster.get(clusterId);++ if (!cluster) throw Object.assign(new Error('No such cluster'), { code: 'cluster.not_found', status: 404 });++ const reports = await q.reports.all(cluster.id);++ if (!reports.length) throw Object.assign(new Error('This cluster has no live reports'), { code: 'brief.no_reports', status: 422 });++ const outlets = new Set(reports.map((r) => r.outlet));++ if (kind === 'perspectives' && outlets.size < 2) throw Object.assign(new Error('Comparing how outlets frame it needs two outlets or more'), { code: 'brief.one_outlet', status: 422 });++ const workflow = kind === 'perspectives' ? 'news.compare_perspectives' : 'news.summarize_story';++ const result = await ai.run(workflow, {++ topic: cluster.label, tone: kind === 'summary' ? tone : null,++ sources: reports.map((r) => ({ id: r.id, sources_item_id: r.sources_item_id, headline: r.headline, url: r.canonical_url, outlet: r.outlet,++ authors: parse(r.authors, []), published_at: r.published_at, retrieved_at: r.retrieved_at, summary: r.summary })),++ });++ const ids = reports.map((r) => r.id);++ const draft = draftFromOutput(workflow, result.output, ids);++ if (!draft.paragraphs.length) throw Object.assign(new Error('The AI run returned no cited text; no brief was made'), { code: 'brief.nothing_cited', status: 422 });++ await q.put.run(cluster.id, kind, tone, draft.headline, JSON.stringify(draft.paragraphs), JSON.stringify(ids), JSON.stringify(draft.gaps),++ result.runId, (result.workflow && result.workflow.model) || null, Boolean(result.stub), reports.length, Number(cluster.updated_at), store.now());++ return get(cluster.id, { kind, tone });++ }++++ /** The sweep: neutral summaries for the busiest recent clusters that have none, or a stale one. */++ async function sweep({ limit = SWEEP_PER_TICK } = {}) {++ if (!ai || !ai.enabled) return 0;++ let made = 0;++ for (const c of await q.due.all(store.now() - SWEEP_WINDOW_MS, MIN_REPORTS)) {++ if (made >= limit) break;++ const row = await q.get.get(c.id, 'summary', 'neutral');++ if (fresh(row, c, Number(c.n))) continue;++ try { await make(c.id, { kind: 'summary', tone: 'neutral' }); made += 1; } catch (err) {++ (log.warn || log.log)(`[News] brief for ${c.id}: ${err.code || err.message}`);++ if (String(err.code || '').startsWith('ai.')) break; // AI is down or refusing: try again next tick++ }++ }++ return made;++ }++++ const latest = async (limit = 12) => (await q.latest.all(limit)).map((r) => ({ ...shape(r), label: r.label, window_end: Number(r.window_end) }));++++ return { TONES, TONE_TEXT, KINDS, MIN_REPORTS, get, make, sweep, latest };++}++++module.exports = { createBriefs, TONES, TONE_TEXT, MIN_REPORTS };@@@ -28,6 +28,7 @@ const seo = require('openvibe-publishing/seo');const ssr = require('openvibe-publishing/ssr');const { renderPage } = require('../render/layout');const pages = require('../render/pages');++const briefPages = require('../render/briefs');const access = require('../domain/access');const { csrfToken, checkCsrf } = require('../auth/forms');const cache = require('openvibe-shared/cache-policy');@@@ -36,7 +37,7 @@ const { SITE_SUMMARY } = require('./discovery');const PER_PAGE = 20;function createPublicRoutes(ctx) {-- const { config, store, stories, publication, reading, topics, viewers, community, discussion, clusters } = ctx;++ const { config, store, stories, publication, reading, topics, viewers, community, discussion, clusters, briefs } = ctx;const router = express.Router();router.use(viewers.middleware({ services: false }));@@@ -136,6 +137,65 @@ function createPublicRoutes(ctx) {}, { cacheable: true });}));++ // ── AI briefs and the live view (domain/briefs.js, render/briefs.js) ──++ // Briefs are AI text: labelled, every paragraph cited, never a story and never indexed. A reader may ask for another++ // tone or the outlets' comparison (a POST, from this site, a few an address and a few dozen an hour in all), never a++ // crawler following a link.++ const noindex = (path) => seo.evaluate({ state: 'published', visibility: 'public', canonicalUrl: seo.canonicalUrl(config.baseUrl, path), wordCount: 0, noindex: true }, { policy: { minWords: 0 }, now: store.now() });++ const BRIEF_ID = /^clu_[0-9A-HJKMNP-TV-Z]{26}$/;++ const makeLimit = rateLimit({ windowMs: 10 * 60_000, limit: 5, standardHeaders: true, legacyHeaders: false, keyGenerator: (req) => req.ip,++ handler: (req, res) => messagePage(req, res, 429, 'Too many requests', 'Wait a few minutes, then ask again.', { href: `/briefs/${req.params.id}`, label: 'Back to the brief' }) });++ const runs = { hour: 0, at: 0 };++ const RUNS_AN_HOUR = 40;++ const sameSite = (req) => { const site = String(req.get('sec-fetch-site') || ''); return site ? site === 'same-origin' : String(req.get('origin') || '') === config.baseUrl; };++ const canMake = () => Boolean(briefs && config.briefs.enabled && ctx.ai && ctx.ai.enabled);++++ router.get('/briefs', wrap(async (req, res) => {++ send(req, res, 200, { title: 'AI briefs', description: 'What the reports on a subject say, summarized by AI with every paragraph linked to its reports.', decision: noindex('/briefs'), canonical: `${config.baseUrl}/briefs`, localStyles: ['css/briefs.css'], body: briefPages.briefsIndex({ briefs: briefs ? await briefs.latest(40) : [] }) }, { cacheable: true });++ }));++++ router.get('/briefs/:id', wrap(async (req, res) => {++ const id = String(req.params.id || '');++ const tone = briefPages.TONES.some(([t]) => t === req.query.tone) ? String(req.query.tone) : 'neutral';++ const got = BRIEF_ID.test(id) && briefs ? await briefs.get(id, { kind: 'summary', tone }) : null;++ if (!got || !got.reports.length) return notFound(req, res);++ const persp = await briefs.get(id, { kind: 'perspectives', tone: 'neutral' });++ const notice = req.query.made === '1' ? 'Written just now.' : (req.query.failed ? String(req.query.failed).slice(0, 200) : null);++ send(req, res, 200, {++ title: `${got.brief && got.brief.headline ? got.brief.headline : got.cluster.label} (AI brief)`,++ description: 'An AI summary of the reports on this subject, every paragraph linked to its reports.',++ decision: noindex(`/briefs/${id}`), canonical: `${config.baseUrl}/briefs/${id}`, localStyles: ['css/briefs.css'],++ body: briefPages.briefPage({ ...got, tone, perspectives: persp ? persp.brief : null, canMake: canMake(), notice }),++ }, { cacheable: !notice });++ }));++++ router.post('/briefs/:id/make', makeLimit, express.urlencoded({ extended: false, limit: '4kb' }), wrap(async (req, res) => {++ const id = String(req.params.id || '');++ if (!BRIEF_ID.test(id) || !canMake()) return notFound(req, res);++ if (!sameSite(req)) return messagePage(req, res, 403, 'Not from this site', 'Ask from the brief\'s own page.', { href: `/briefs/${id}`, label: 'Back to the brief' });++ const kind = req.body && req.body.kind === 'perspectives' ? 'perspectives' : 'summary';++ const tone = briefPages.TONES.some(([t]) => t === (req.body && req.body.tone)) ? req.body.tone : 'neutral';++ const back = `/briefs/${id}${tone !== 'neutral' ? `?tone=${tone}&` : '?'}`;++ if (Date.now() - runs.at > 3_600_000) { runs.at = Date.now(); runs.hour = 0; }++ if (runs.hour >= RUNS_AN_HOUR) return res.redirect(303, `${back}failed=${encodeURIComponent('Many briefs were asked for this hour. Try again later.')}`);++ runs.hour += 1;++ try {++ await briefs.make(id, { kind, tone });++ } catch (err) {++ return res.redirect(303, `${back}failed=${encodeURIComponent(err.status === 422 ? err.message : 'OpenVibe.AI could not write it just now. Try again in a few minutes.')}`);++ }++ return res.redirect(303, `${back}made=1`);++ }));++++ router.get('/live', wrap(async (req, res) => {++ const list = await clusters.wire({ limit: 40 });++ const now = store.now();++ const breaking = new Set(list.filter((c) => c.items.filter((i) => i.published_at && now - Date.parse(i.published_at) <= 2 * 3_600_000).length >= 3).map((c) => c.id));++ const briefIds = new Set(briefs ? (await briefs.latest(200)).map((b) => b.cluster_id) : []);++ res.set('Refresh', '60');++ send(req, res, 200, { title: 'Live', description: 'The newest reports from the sources OpenVibe.News follows, as they arrive, with what is breaking.', decision: noindex('/live'), canonical: `${config.baseUrl}/live`, localStyles: ['css/briefs.css'], body: briefPages.livePage({ clusters: list, briefIds, breaking }) }, { cacheable: true });++ }));++router.get('/topics', wrap(async (req, res) => {send(req, res, 200, { title: 'Topics', description: 'Topics on OpenVibe.News.', decision: pageDecision('/topics'), canonical: publication.abs('/topics'), body: pages.topicsIndex({ topics: await topics.list() }) }, { cacheable: true });}));nnew file mode 100644@@@ -0,0 +1,77 @@++'use strict';++++/**++ * AI briefs and the live view (server/domain/briefs.js; routes in http/public.js). Everything here that OpenVibe.AI++ * wrote is labelled as AI, and every paragraph links the reports it rests on, numbered in the table below it. The++ * reports' own headlines and links are the outlets'.++ */++const ssr = require('openvibe-publishing/ssr');++const { time, httpHref } = require('./pages');++++const { html: h, raw } = ssr;++const TONES = [['brief', 'In brief'], ['neutral', 'Summary'], ['explained', 'Explained']];++++const AI_NOTE = 'Written by OpenVibe.AI from the reports listed below, not by an editor. Each paragraph links the reports it rests on; read them before relying on it.';++++function sourceTable(reports) {++ return h`<table class="sources brief-sources"><thead><tr><th>#</th><th>Report</th><th>Outlet</th><th>Date</th></tr></thead><tbody>++${reports.map((r, i) => h`<tr id="source-${i + 1}"><td>${i + 1}</td><td>${httpHref(r.canonical_url) ? h`<a href="${httpHref(r.canonical_url)}" rel="noopener nofollow" target="_blank">${r.headline}</a>` : r.headline}${r.summary ? h`<p class="source-summary">${r.summary}</p>` : ''}</td><td>${r.outlet}</td><td>${time(r.published_at ? Date.parse(r.published_at) : null)}</td></tr>`)}++</tbody></table>`;++}++++/** Paragraphs whose sources are news item ids, cited by their place in `reports` (1-based). */++function paragraphs(brief, reports) {++ const n = new Map(reports.map((r, i) => [r.id, i + 1]));++ return brief.paragraphs.map((p) => {++ const nums = p.sources.map((id) => n.get(id)).filter(Boolean);++ return h`<p class="brief-p">${p.text}${nums.length ? h` <sup class="cites">${nums.map((x, i) => h`${i ? ',' : ''}<a href="#source-${x}" aria-label="report ${x}">${x}</a>`)}</sup>` : ''}</p>`;++ });++}++++function makeButton(clusterId, { kind, tone, label }) {++ return h`<form class="brief-make" method="post" action="/briefs/${clusterId}/make"><input type="hidden" name="kind" value="${kind}"><input type="hidden" name="tone" value="${tone}"><button type="submit">${label}</button></form>`;++}++++/** /briefs/:id */++function briefPage({ cluster, reports, brief, fresh, tone, perspectives, canMake, notice = null }) {++ const outlets = [...new Set(reports.map((r) => r.outlet))];++ const shown = brief && !brief.stub ? brief : null;++ const tabs = h`<nav class="brief-tones" aria-label="How to tell it">${TONES.map(([t, label]) => (t === tone++ ? h`<span class="brief-tone on" aria-current="page">${label}</span>`++ : h`<a class="brief-tone" href="/briefs/${cluster.id}${t === 'neutral' ? '' : `?tone=${t}`}">${label}</a>`))}</nav>`;++ const persp = perspectives && !perspectives.stub ? perspectives : null;++ return String(h`<header class="site-header"><p class="eyebrow"><span class="badge badge-ai">AI brief</span> from ${reports.length} ${reports.length === 1 ? 'report' : 'reports'}${outlets.length ? h` (${outlets.slice(0, 5).join(', ')}${outlets.length > 5 ? ` and ${outlets.length - 5} more` : ''})` : ''}</p>++<h1>${shown && shown.headline ? shown.headline : cluster.label}</h1>++<p class="meta">${AI_NOTE}</p></header>++${notice ? h`<p class="notice">${notice}</p>` : ''}++${tabs}++<section class="brief" aria-label="The brief">++${shown ? h`${paragraphs(shown, reports)}${!fresh ? h`<p class="notice small">New reports arrived since this was written. ${canMake ? makeButton(cluster.id, { kind: 'summary', tone, label: 'Write it again' }) : ''}</p>` : ''}`++ : h`<p class="empty">${canMake ? h`Not written in this tone yet. ${makeButton(cluster.id, { kind: 'summary', tone, label: 'Write it with AI' })}` : 'No brief yet.'}</p>`}++</section>++${outlets.length >= 2 ? h`<section class="brief-frames" aria-labelledby="frames-h"><h2 id="frames-h">How the outlets put it</h2>++${persp ? h`<p class="meta">${AI_NOTE}</p>${paragraphs(persp, reports)}` : h`<p class="meta">How each outlet frames this, side by side, with what they agree and differ on.</p>${canMake ? makeButton(cluster.id, { kind: 'perspectives', tone: 'neutral', label: 'Compare the outlets with AI' }) : ''}`}++</section>` : ''}++<section aria-labelledby="reports-h"><h2 id="reports-h">The reports</h2>${sourceTable(reports)}</section>++${shown && shown.gaps.length ? h`<details class="brief-gaps"><summary>What the AI left out (${shown.gaps.length})</summary><ul>${shown.gaps.map((g) => h`<li>${g}</li>`)}</ul></details>` : ''}++<p class="meta"><a href="/briefs">All AI briefs</a> · <a href="/live">Live</a> · <a href="/wire">The wire</a></p>`);++}++++/** /briefs */++function briefsIndex({ briefs }) {++ return String(h`<header class="site-header"><h1>AI briefs</h1><p class="lede">What the reports on a subject say, summarized by OpenVibe.AI with every paragraph linked to its reports, and labelled as AI. Editors' stories are on <a href="/">the front page</a>.</p></header>++${briefs.length ? h`<ol class="story-list">${briefs.map((b) => h`<li class="story-item"><h2 class="story-item-title"><a href="/briefs/${b.cluster_id}">${b.headline || b.label}</a> <span class="badge badge-ai">AI</span></h2>++<p class="meta">${time(b.window_end)} · from ${b.item_count} reports</p>${b.paragraphs[0] ? h`<p class="summary">${b.paragraphs[0].text.slice(0, 280)}${b.paragraphs[0].text.length > 280 ? '…' : ''}</p>` : ''}</li>`)}</ol>`++ : h`<p class="empty">No briefs yet. They are written as reports on a subject come in from two or more of the sources.</p>`}`);++}++++/** /live: the newest reports as they arrive, grouped, with what is breaking. */++function livePage({ clusters, briefIds, breaking }) {++ return String(h`<header class="site-header"><h1>Live <span class="live-dot" aria-hidden="true"></span></h1><p class="lede">The newest reports from the sources OpenVibe.News follows, as they arrive. This page refreshes itself every minute. <span class="badge badge-breaking">Breaking</span> marks a subject three or more reports covered in the last two hours.</p></header>++${clusters.length ? h`<ol class="wire-list wide live-list">${clusters.map((c) => h`<li class="wire-cluster${breaking.has(c.id) ? ' is-breaking' : ''}">++<p class="wire-lead">${breaking.has(c.id) ? h`<span class="badge badge-breaking">Breaking</span> ` : ''}<strong>${c.label}</strong>${briefIds.has(c.id) ? h` · <a href="/briefs/${c.id}">AI brief</a>` : ''}</p>++<ul class="wire-items">${c.items.slice(0, 5).map((i) => h`<li>${httpHref(i.url) ? h`<a href="${httpHref(i.url)}" rel="nofollow noopener" target="_blank">${i.headline}</a>` : i.headline} <span class="meta">${i.outlet} · ${time(i.published_at ? Date.parse(i.published_at) : null)}</span></li>`)}</ul></li>`)}</ol>`++ : h`<p class="empty">Nothing new from the sources yet.</p>`}`);++}++++module.exports = { briefPage, briefsIndex, livePage, AI_NOTE, TONES };@@@ -22,6 +22,8 @@ const NAV_LINKS = [{ label: 'News', href: '/' },{ label: 'Topics', href: '/topics' },{ label: 'Wire', href: '/wire' },++ { label: 'Live', href: '/live' },++ { label: 'AI briefs', href: '/briefs' },];const hashes = new Map();@@@ -99,6 +101,8 @@ function renderPage(o) {home: '/',css: asset('css/news.css'),styles: o.styles, // openvibe-shared stylesheet names (the home page's showcase.css)++ // This service's own extra stylesheets for one kind of page (the briefs and live pages' css/briefs.css).++ head: (o.localStyles || []).map((rel) => `<link rel="stylesheet" href="${asset(rel)}">`).join('\n'),release: RELEASE,account,body: o.body,@@@ -7,8 +7,15 @@* (idempotent), the backstop for webhook deliveries that never arrived.* A failed pull is recorded and relayed once per outage; it creates nothing.*/--function createWorker({ config, ingest, outbox, log = console }) {++function createWorker({ config, ingest, outbox, briefs = null, log = console }) {let timer = null;++ let briefTimer = null;++++ /** AI briefs for the busiest recent clusters (server/domain/briefs.js); a failure waits for the next tick. */++ async function briefTick() {++ if (!briefs) return 0;++ try { return await briefs.sweep(); } catch (err) { log.error('[News] brief tick failed:', err.message); return 0; }++ }async function pullTick() {if (!ingest) return null;@@@ -24,13 +31,20 @@ function createWorker({ config, ingest, outbox, log = console }) {return {pullTick,++ briefTick,start() {-- if (!config.worker.enabled || !config.sources.pullIntervalMs) return;++ if (!config.worker.enabled) return;++ if (config.briefs && config.briefs.enabled && briefs) {++ briefTimer = setInterval(briefTick, config.briefs.intervalMs);++ briefTimer.unref();++ setTimeout(briefTick, 30_000).unref();++ }++ if (!config.sources.pullIntervalMs) return;timer = setInterval(pullTick, config.sources.pullIntervalMs);timer.unref();setTimeout(pullTick, 2000).unref();},-- stop() { clearInterval(timer); },++ stop() { clearInterval(timer); clearInterval(briefTimer); },};}nnew file mode 100644@@@ -0,0 +1,90 @@++'use strict';++/**++ * AI briefs and the live view (server/domain/briefs.js, render/briefs.js; the owner's April ask: multi-source summaries++ * with tone and perspective controls, and a breaking-news live updater). The sweep summarizes a cluster with two or++ * more reports; every paragraph cites its reports and uncited claims are dropped; the page labels it as AI, is never++ * indexed, and shows the reports; a reader's POST asks for another tone or the outlets' comparison (same-origin only);++ * a stub provider's output is never shown; a changed cluster's brief is stale and made again; /live refreshes itself++ * and marks what is breaking.++ */++const assert = require('assert');++const { boot, check, done, launchReports } = require('./helpers/boot');++++const run = (output, id = 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E1', synthetic = false) => ({ status: 201, run: { id, status: 'succeeded', workflow: { key: 'news.summarize_story', version: 3 }, synthetic, provenance: { model: 'test-model' }, output } });++const SAME = { 'sec-fetch-site': 'same-origin' };++++(async () => {++ const t = await boot({ ai: true });++ try {++ launchReports(t.sources);++ await t.pull();++ const clusterId = (await t.db().prepare("SELECT cluster_id, count(*) AS n FROM news_source_items WHERE status = 'active' GROUP BY cluster_id ORDER BY n DESC").get()).cluster_id;++++ await check('the sweep writes a cited summary for a cluster with two or more reports; uncited claims are dropped', async () => {++ t.ai.setNext(run({ headline: 'Europa Clipper is on its way', summary: 'NASA launched Europa Clipper toward Jupiter.', key_points: [{ text: 'It reaches Jupiter in 2030.', citations: [1] }, { text: 'A claim nobody reported.', citations: [] }], timeline: [], citations: [0, 1], gaps: [] }));++ const made = await t.ctx.briefs.sweep();++ assert.strictEqual(made, 1);++ const sent = t.ai.requests.pop();++ assert.strictEqual(sent.workflow, 'news.summarize_story');++ assert.strictEqual(sent.input.tone, 'neutral');++ const got = await t.ctx.briefs.get(clusterId);++ assert.ok(got.fresh);++ assert.deepStrictEqual(got.brief.paragraphs.map((p) => p.text), ['NASA launched Europa Clipper toward Jupiter.', 'It reaches Jupiter in 2030.']);++ assert.ok(got.brief.gaps.some((g) => /uncited/.test(g)));++ assert.strictEqual(await t.ctx.briefs.sweep(), 0, 'a fresh brief is not made again');++ });++++ await check('/briefs/:id: labelled AI, cited to the reports below it, never indexed; listed on /briefs', async () => {++ const r = await t.get(`/briefs/${clusterId}`);++ assert.strictEqual(r.status, 200, r.text.slice(0, 200));++ assert.ok(r.text.includes('AI brief') && r.text.includes('Written by OpenVibe.AI'));++ assert.ok(r.text.includes('Europa Clipper is on its way'));++ assert.match(r.text, /href="#source-2"/);++ assert.ok(r.text.includes('id="source-1"'), 'the reports are in the page');++ assert.ok(r.text.includes('noindex'));++ assert.ok(!r.text.includes('A claim nobody reported') || r.text.includes('What the AI left out'));++ const list = await t.get('/briefs');++ assert.ok(list.text.includes(`/briefs/${clusterId}`));++ });++++ await check('a reader asks for another tone with a POST from this site; a GET never runs AI', async () => {++ const before = t.ai.requests.length;++ const page = await t.get(`/briefs/${clusterId}?tone=brief`);++ assert.ok(page.text.includes('Write it with AI'));++ assert.strictEqual(t.ai.requests.length, before, 'reading never runs AI');++ assert.strictEqual((await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'brief' }, headers: { 'sec-fetch-site': 'cross-site' } })).status, 403);++ t.ai.setNext(run({ headline: 'Europa Clipper launched', summary: 'NASA sent Europa Clipper to Jupiter.', key_points: [], timeline: [], citations: [0], gaps: [] }, 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E2'));++ const made = await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'brief' }, headers: SAME });++ assert.strictEqual(made.status, 303);++ assert.strictEqual(t.ai.requests.pop().input.tone, 'brief');++ assert.ok((await t.get(`/briefs/${clusterId}?tone=brief`)).text.includes('NASA sent Europa Clipper to Jupiter.'));++ });++++ await check('how the outlets put it: compare_perspectives on request, cited', async () => {++ t.ai.setNext(run({ question: 'How was the launch covered?', perspectives: [{ label: 'Wire A', summary: 'Focus on the rocket.', citations: [0] }, { label: 'Paper B', summary: 'Focus on the science.', citations: [1] }], agreements: [], disagreements: [], citations: [0, 1], gaps: [] }, 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E3'));++ const made = await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'perspectives', tone: 'neutral' }, headers: SAME });++ assert.strictEqual(made.status, 303);++ assert.strictEqual(t.ai.requests.pop().workflow, 'news.compare_perspectives');++ assert.ok((await t.get(`/briefs/${clusterId}`)).text.includes('Paper B: Focus on the science.'));++ });++++ await check('stub output is never shown; a failed run makes nothing and says so', async () => {++ t.ai.setNext(run({ headline: 'STUB', summary: 'stub text', key_points: [], timeline: [], citations: [0], gaps: [] }, 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E4', true));++ await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'explained' }, headers: SAME });++ assert.ok(!(await t.get(`/briefs/${clusterId}?tone=explained`)).text.includes('stub text'));++ t.ai.setNext(null);++ const failed = await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'explained' }, headers: SAME });++ assert.match(failed.headers.get('location'), /failed=/);++ });++++ await check('/live refreshes itself and links the AI briefs; it is not indexed', async () => {++ const r = await t.get('/live');++ assert.strictEqual(r.status, 200);++ assert.strictEqual(r.headers.get('refresh'), '60');++ assert.ok(r.text.includes(`/briefs/${clusterId}`) && r.text.includes('noindex'));++ const home = await t.get('/');++ assert.ok(home.text.includes('href="/live"') && home.text.includes('href="/briefs"'), 'in the navigation');++ });++ } finally { await t.close(); }++ done();++})().catch((e) => { console.error(e); process.exit(1); });
@@ -26,6 +26,26 @@ shown, fed or indexed.
@@ -26,6 +26,26 @@ shown, fed or indexed.
News publishes nothing until an editor does: the only seed is the topic list. Until then, and beside the stories after, **the wire** (`/wire`, and "From the wire" on the front page) shows what the sources report, grouped by cluster: each line is the outlet's own headline, outlet, date and link (the fields the licensing rules let News keep), never News' writing; duplicates and reports removed upstream leave it; `/wire` is `noindex, follow`.
News publishes nothing until an editor does: the only seed is the topic list. Until then, and beside the stories after, **the wire** (`/wire`, and "From the wire" on the front page) shows what the sources report, grouped by cluster: each line is the outlet's own headline, outlet, date and link (the fields the licensing rules let News keep), never News' writing; duplicates and reports removed upstream leave it; `/wire` is `noindex, follow`.
+## AI briefs and Live
+
+The owner's April ask for News was multi-source summaries with tone and perspective controls and a live breaking-news
+view. Stories stay editor-written (below); beside them News now has:
+
+- **AI briefs** (`/briefs`, `/briefs/:cluster`; [server/domain/briefs.js](server/domain/briefs.js),
+ `migrations/0004_cluster_briefs.sql`): what a cluster's reports say, written by OpenVibe.AI and labelled as AI on
+ every page. Every paragraph cites the reports it rests on, numbered in the table below it; claims the model did not
+ cite are dropped and listed under "What the AI left out"; a run with no cited text makes no brief; the stub
+ provider's output is never shown. Only News' stored fields are sent (headline, outlet, date, URL and the short summary
+ the terms allow). The worker writes a neutral summary for every open cluster with two or more reports in the last 72
+ hours, six a tick every `NEWS_AI_BRIEFS_INTERVAL_MS` (10 minutes), and writes it again once the cluster changes.
+- **Tone** (`?tone=brief|neutral|explained`, `news.summarize_story`'s `tone`) and **how the outlets put it**
+ (`news.compare_perspectives`, two outlets or more): made when a reader asks, with a POST from this site (five every ten
+ minutes an address, 40 an hour in all), never by a link a crawler follows.
+- **Live** (`/live`): the newest reports as they arrive, grouped, refreshing itself every minute (`Refresh: 60`), with
+ **Breaking** on a subject three or more reports covered in the last two hours, and links to their AI briefs.
+- Briefs and Live are `noindex`: they are AI text and other publishers' headlines, not News' writing.
+ `NEWS_AI_BRIEFS=off` turns briefs off.
+
## Owns
## Owns
The nine charter tables live in News' own PostgreSQL database (`ov_news` on the host's data role, ADR-035; schema in [migrations/](migrations/)):
The nine charter tables live in News' own PostgreSQL database (`ov_news` on the host's data role, ADR-035; schema in [migrations/](migrations/)):
new file mode 100644
new file mode 100644
@@ -0,0 +1,23 @@
@@ -0,0 +1,23 @@
+-- phase: expand
+-- AI briefs (server/domain/briefs.js; the owner's April ask: multi-source summaries with tone and perspective controls).
+-- One row per cluster, kind and tone: a summary of what the cluster's reports say (tone brief | neutral | explained),
+-- or how its outlets frame it (kind perspectives). Every paragraph cites the source items it rests on; the model's
+-- uncited claims are dropped and listed in gaps. A brief is AI text, labelled as such, never a story: stories stay
+-- editor-written. A brief is stale once its cluster changes (item_count or cluster_updated_at) and is made again.
+CREATE TABLE IF NOT EXISTS news_cluster_briefs (
+ cluster_id text COLLATE "C" NOT NULL REFERENCES news_story_clusters(id) ON DELETE CASCADE,
+ kind text COLLATE "C" NOT NULL CHECK (kind IN ('summary', 'perspectives')),
+ tone text COLLATE "C" NOT NULL DEFAULT 'neutral' CHECK (tone IN ('brief', 'neutral', 'explained')),
+ headline text COLLATE "C",
+ paragraphs text COLLATE "C" NOT NULL, -- JSON [{ text, sources: [nsi_…] }]
+ source_ids text COLLATE "C" NOT NULL, -- JSON [nsi_…] the reports it was made from, in order
+ gaps text COLLATE "C" NOT NULL DEFAULT '[]',
+ run_id text COLLATE "C",
+ model text COLLATE "C",
+ stub boolean NOT NULL DEFAULT false, -- the stub provider's synthetic output: never shown
+ item_count integer NOT NULL,
+ cluster_updated_at bigint NOT NULL,
+ created_at bigint NOT NULL,
+ PRIMARY KEY (cluster_id, kind, tone)
+);
+CREATE INDEX IF NOT EXISTS news_cluster_briefs_recent ON news_cluster_briefs (kind, tone, created_at DESC);
new file mode 100644
new file mode 100644
@@ -0,0 +1,17 @@
@@ -0,0 +1,17 @@
+/* OpenVibe.News — the AI briefs and live pages only (render/briefs.js), so news.css stays inside its budget. */
+.badge-ai { background: color-mix(in srgb, #a78bfa 18%, transparent); color: color-mix(in srgb, #8b5cf6 70%, var(--text-primary, #e6edf7)); border: 1px solid color-mix(in srgb, #a78bfa 45%, transparent); }
+.badge-breaking { background: color-mix(in srgb, #ef4444 16%, transparent); color: color-mix(in srgb, #ef4444 75%, var(--text-primary, #e6edf7)); border: 1px solid color-mix(in srgb, #ef4444 45%, transparent); }
+.eyebrow { margin: 0 0 .4rem; font-size: .9rem; }
+.brief-tones { display: flex; flex-wrap: wrap; gap: 8px; margin: 12px 0 16px; }
+.brief-tone { padding: 5px 12px; border-radius: 999px; border: 1px solid var(--border-light, #2c3d5c); text-decoration: none; color: inherit; font-size: .9rem; }
+.brief-tone.on { border-color: var(--accent, #3b82f6); font-weight: 700; }
+.brief .brief-p, .brief-frames .brief-p { max-width: 72ch; line-height: 1.6; }
+.brief-make { display: inline; }
+.brief-make button { font: inherit; padding: 5px 12px; border-radius: 8px; border: 1px solid var(--border-light, #2c3d5c); background: var(--bg-tertiary, #16223a); color: inherit; cursor: pointer; }
+.brief-make button:hover, .brief-make button:focus-visible { border-color: var(--accent, #3b82f6); }
+.brief-frames { margin-top: 1.6rem; }
+.brief-gaps { margin-top: 1rem; font-size: .9rem; }
+.live-dot { display: inline-block; width: .55em; height: .55em; border-radius: 50%; background: #ef4444; vertical-align: middle; animation: live-pulse 1.6s ease-in-out infinite; }
+@keyframes live-pulse { 50% { opacity: .35; } }
+@media (prefers-reduced-motion: reduce) { .live-dot { animation: none; } }
+.wire-cluster.is-breaking { border-left: 3px solid #ef4444; padding-left: 10px; }
@@ -26,6 +26,7 @@ const { openStore } = require('./db');
@@ -26,6 +26,7 @@ const { openStore } = require('./db');
const { createViewerResolver } = require('./auth/viewer');
const { createViewerResolver } = require('./auth/viewer');
const { createPeople } = require('./clients/network');
const { createPeople } = require('./clients/network');
const { createAi } = require('./clients/ai');
const { createAi } = require('./clients/ai');
+const { createBriefs } = require('./domain/briefs');
const { createCommunity } = require('./clients/community');
const { createCommunity } = require('./clients/community');
const { createPublication } = require('./domain/publication');
const { createPublication } = require('./domain/publication');
const { createDiscussion } = require('./domain/discussion');
const { createDiscussion } = require('./domain/discussion');
@@ -97,9 +98,10 @@ async function createApp(opts = {}) {
@@ -97,9 +98,10 @@ async function createApp(opts = {}) {
secureCookies: config.cookies.secure,
secureCookies: config.cookies.secure,
});
});
const viewers = createViewerResolver({ auth, config, people });
const viewers = createViewerResolver({ auth, config, people });
- const worker = createWorker({ config, ingest, outbox, log });
+ const briefs = createBriefs({ store, ai: config.briefs.enabled ? ai : null, log });
+ const worker = createWorker({ config, ingest, outbox, briefs, log });
- const ctx = { config, store, outbox, publication, clusters, sources, ai, community, discussion, ingest, stories, topics, people, reading, auth, viewers, worker, indexnow };
+ const ctx = { config, store, outbox, publication, clusters, sources, ai, community, discussion, ingest, stories, topics, people, reading, auth, viewers, worker, indexnow, briefs };
const app = express();
const app = express();
app.disable('x-powered-by');
app.disable('x-powered-by');
@@ -29,11 +29,12 @@ function createAi({ config, fetchImpl = globalThis.fetch }) {
@@ -29,11 +29,12 @@ function createAi({ config, fetchImpl = globalThis.fetch }) {
}) : null;
}) : null;
/** sources: [{ id (nsi_…), sources_item_id, headline, url, outlet, authors, published_at, retrieved_at, summary }] in citation order. */
/** sources: [{ id (nsi_…), sources_item_id, headline, url, outlet, authors, published_at, retrieved_at, summary }] in citation order. */
- async function run(workflow, { topic, sources }) {
+ async function run(workflow, { topic, sources, tone = null }) {
if (!enabled) throw new AiError('ai.not_configured', 'OpenVibe.AI is not configured (OV_AI_INTERNAL_URL)', 503);
if (!enabled) throw new AiError('ai.not_configured', 'OpenVibe.AI is not configured (OV_AI_INTERNAL_URL)', 503);
if (!WORKFLOWS.includes(workflow)) throw new AiError('ai.unknown_workflow', `workflow must be one of ${WORKFLOWS.join(', ')}`, 422);
if (!WORKFLOWS.includes(workflow)) throw new AiError('ai.unknown_workflow', `workflow must be one of ${WORKFLOWS.join(', ')}`, 422);
const input = {
const input = {
...(topic ? { topic: String(topic).slice(0, 300) } : {}),
...(topic ? { topic: String(topic).slice(0, 300) } : {}),
+ ...(tone && workflow === 'news.summarize_story' ? { tone } : {}),
sources: sources.slice(0, 50).map((s) => ({
sources: sources.slice(0, 50).map((s) => ({
source_type: 'news.article', source_id: s.sources_item_id, title: s.headline, ...(s.url ? { url: s.url } : {}),
source_type: 'news.article', source_id: s.sources_item_id, title: s.headline, ...(s.url ? { url: s.url } : {}),
...(s.authors && s.authors.length ? { author: s.authors.join(', ').slice(0, 200) } : {}),
...(s.authors && s.authors.length ? { author: s.authors.join(', ').slice(0, 200) } : {}),
@@ -119,6 +119,13 @@ function load(env = process.env) {
@@ -119,6 +119,13 @@ function load(env = process.env) {
enabled: env.NEWS_WORKER !== 'off',
enabled: env.NEWS_WORKER !== 'off',
},
},
+ // AI briefs (server/domain/briefs.js): the sweep summarizes busy clusters every intervalMs; off with
+ // NEWS_AI_BRIEFS=off (readers' requests for a tone or the perspectives then answer 503 too).
+ briefs: {
+ enabled: env.NEWS_AI_BRIEFS !== 'off',
+ intervalMs: Math.max(60_000, int(env.NEWS_AI_BRIEFS_INTERVAL_MS, 600_000)),
+ },
+
// Browser origins that may call /api/v1 with a Bearer Network JWT (no cookies cross origins).
// Browser origins that may call /api/v1 with a Bearer Network JWT (no cookies cross origins).
apiCorsOrigins: list(env.API_CORS_ORIGINS || 'https://openvibe.network,https://openvibe.live,https://openvibe.community'),
apiCorsOrigins: list(env.API_CORS_ORIGINS || 'https://openvibe.network,https://openvibe.live,https://openvibe.community'),
};
};
new file mode 100644
new file mode 100644
@@ -0,0 +1,112 @@
@@ -0,0 +1,112 @@
+'use strict';
+
+/**
+ * AI briefs: what a cluster's reports say, written by OpenVibe.AI and labelled as AI (migrations/0004_cluster_briefs.sql).
+ * The owner's April ask for News: multi-source summarization, tone controls and perspective/slant controls, and a live
+ * breaking view. Stories stay what they were: editor-written, every paragraph cited, never AI text without a person's
+ * approval. A brief is the other thing: an AI summary of a cluster, shown as such, beside the wire.
+ *
+ * summary news.summarize_story over the cluster's live reports, in a tone: brief, neutral (made by the sweep
+ * for every cluster with at least MIN_REPORTS reports) or explained (made when a reader asks)
+ * perspectives news.compare_perspectives: how the outlets frame it (when a reader asks; two outlets or more)
+ *
+ * Only News' stored fields go to AI (headline, outlet, date, URL, and the short summary its terms allow). A paragraph the
+ * model did not cite is dropped (clients/ai.js draftFromOutput) and listed as a gap; a run with no cited text makes no
+ * brief. The stub provider's synthetic output is stored but never shown. A brief made from a cluster that has since
+ * changed is stale and is made again (the sweep for neutral summaries, the next reader's request for the rest).
+ */
+
+const { draftFromOutput } = require('../clients/ai');
+
+const TONES = ['brief', 'neutral', 'explained'];
+const TONE_TEXT = { brief: 'In brief', neutral: 'Summary', explained: 'Explained' };
+const KINDS = ['summary', 'perspectives'];
+const MIN_REPORTS = 2;
+const SWEEP_PER_TICK = 6;
+const SWEEP_WINDOW_MS = 72 * 3_600_000;
+
+const parse = (v, d) => { try { return typeof v === 'string' ? JSON.parse(v) : (v == null ? d : v); } catch { return d; } };
+
+function createBriefs({ store, ai, log = console }) {
+ const db = store.db;
+ const q = {
+ cluster: db.prepare("SELECT * FROM news_story_clusters WHERE id = ? AND status = 'open'"),
+ reports: db.prepare("SELECT * FROM news_source_items WHERE cluster_id = ? AND status = 'active' AND removed_at IS NULL ORDER BY published_at, first_seen_at, seq"),
+ get: db.prepare('SELECT * FROM news_cluster_briefs WHERE cluster_id = ? AND kind = ? AND tone = ?'),
+ put: db.prepare(`INSERT INTO news_cluster_briefs (cluster_id, kind, tone, headline, paragraphs, source_ids, gaps, run_id, model, stub, item_count, cluster_updated_at, created_at)
+ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+ ON CONFLICT (cluster_id, kind, tone) DO UPDATE SET headline = excluded.headline, paragraphs = excluded.paragraphs, source_ids = excluded.source_ids,
+ gaps = excluded.gaps, run_id = excluded.run_id, model = excluded.model, stub = excluded.stub, item_count = excluded.item_count,
+ cluster_updated_at = excluded.cluster_updated_at, created_at = excluded.created_at`),
+ latest: db.prepare(`SELECT b.*, c.label, c.window_end FROM news_cluster_briefs b JOIN news_story_clusters c ON c.id = b.cluster_id
+ WHERE b.kind = 'summary' AND b.tone = 'neutral' AND b.stub = false AND c.status = 'open' ORDER BY c.window_end DESC, b.created_at DESC LIMIT ?`),
+ // Open clusters active in the window with at least MIN_REPORTS live reports, newest activity first.
+ due: db.prepare(`SELECT c.id, c.updated_at, count(i.id) AS n FROM news_story_clusters c JOIN news_source_items i ON i.cluster_id = c.id AND i.status = 'active' AND i.removed_at IS NULL
+ WHERE c.status = 'open' AND c.window_end >= ? GROUP BY c.id, c.updated_at, c.window_end HAVING count(i.id) >= ? ORDER BY c.window_end DESC LIMIT 200`),
+ };
+
+ const fresh = (row, cluster, n) => Boolean(row && Number(row.item_count) === n && Number(row.cluster_updated_at) === Number(cluster.updated_at));
+
+ /** The stored brief, and whether it still matches its cluster. */
+ async function get(clusterId, { kind = 'summary', tone = 'neutral' } = {}) {
+ const cluster = await q.cluster.get(clusterId);
+ if (!cluster) return null;
+ const reports = await q.reports.all(cluster.id);
+ const row = await q.get.get(cluster.id, kind, tone);
+ return { cluster, reports, brief: row ? shape(row) : null, fresh: fresh(row, cluster, reports.length) };
+ }
+
+ function shape(row) {
+ return {
+ cluster_id: row.cluster_id, kind: row.kind, tone: row.tone, headline: row.headline || null,
+ paragraphs: parse(row.paragraphs, []), source_ids: parse(row.source_ids, []), gaps: parse(row.gaps, []),
+ run_id: row.run_id || null, model: row.model || null, stub: Boolean(row.stub), item_count: Number(row.item_count), created_at: Number(row.created_at),
+ };
+ }
+
+ /** Make (or remake) one brief. Throws the AI client's error when the run fails; makes nothing without cited text. */
+ async function make(clusterId, { kind = 'summary', tone = 'neutral' } = {}) {
+ if (!ai || !ai.enabled) throw Object.assign(new Error('OpenVibe.AI is not configured on this server'), { code: 'ai.not_configured', status: 503 });
+ if (!KINDS.includes(kind) || !TONES.includes(tone)) throw Object.assign(new Error('kind or tone not known'), { code: 'brief.invalid', status: 422 });
+ const cluster = await q.cluster.get(clusterId);
+ if (!cluster) throw Object.assign(new Error('No such cluster'), { code: 'cluster.not_found', status: 404 });
+ const reports = await q.reports.all(cluster.id);
+ if (!reports.length) throw Object.assign(new Error('This cluster has no live reports'), { code: 'brief.no_reports', status: 422 });
+ const outlets = new Set(reports.map((r) => r.outlet));
+ if (kind === 'perspectives' && outlets.size < 2) throw Object.assign(new Error('Comparing how outlets frame it needs two outlets or more'), { code: 'brief.one_outlet', status: 422 });
+ const workflow = kind === 'perspectives' ? 'news.compare_perspectives' : 'news.summarize_story';
+ const result = await ai.run(workflow, {
+ topic: cluster.label, tone: kind === 'summary' ? tone : null,
+ sources: reports.map((r) => ({ id: r.id, sources_item_id: r.sources_item_id, headline: r.headline, url: r.canonical_url, outlet: r.outlet,
+ authors: parse(r.authors, []), published_at: r.published_at, retrieved_at: r.retrieved_at, summary: r.summary })),
+ });
+ const ids = reports.map((r) => r.id);
+ const draft = draftFromOutput(workflow, result.output, ids);
+ if (!draft.paragraphs.length) throw Object.assign(new Error('The AI run returned no cited text; no brief was made'), { code: 'brief.nothing_cited', status: 422 });
+ await q.put.run(cluster.id, kind, tone, draft.headline, JSON.stringify(draft.paragraphs), JSON.stringify(ids), JSON.stringify(draft.gaps),
+ result.runId, (result.workflow && result.workflow.model) || null, Boolean(result.stub), reports.length, Number(cluster.updated_at), store.now());
+ return get(cluster.id, { kind, tone });
+ }
+
+ /** The sweep: neutral summaries for the busiest recent clusters that have none, or a stale one. */
+ async function sweep({ limit = SWEEP_PER_TICK } = {}) {
+ if (!ai || !ai.enabled) return 0;
+ let made = 0;
+ for (const c of await q.due.all(store.now() - SWEEP_WINDOW_MS, MIN_REPORTS)) {
+ if (made >= limit) break;
+ const row = await q.get.get(c.id, 'summary', 'neutral');
+ if (fresh(row, c, Number(c.n))) continue;
+ try { await make(c.id, { kind: 'summary', tone: 'neutral' }); made += 1; } catch (err) {
+ (log.warn || log.log)(`[News] brief for ${c.id}: ${err.code || err.message}`);
+ if (String(err.code || '').startsWith('ai.')) break; // AI is down or refusing: try again next tick
+ }
+ }
+ return made;
+ }
+
+ const latest = async (limit = 12) => (await q.latest.all(limit)).map((r) => ({ ...shape(r), label: r.label, window_end: Number(r.window_end) }));
+
+ return { TONES, TONE_TEXT, KINDS, MIN_REPORTS, get, make, sweep, latest };
+}
+
+module.exports = { createBriefs, TONES, TONE_TEXT, MIN_REPORTS };
@@ -28,6 +28,7 @@ const seo = require('openvibe-publishing/seo');
@@ -28,6 +28,7 @@ const seo = require('openvibe-publishing/seo');
const ssr = require('openvibe-publishing/ssr');
const ssr = require('openvibe-publishing/ssr');
const { renderPage } = require('../render/layout');
const { renderPage } = require('../render/layout');
const pages = require('../render/pages');
const pages = require('../render/pages');
+const briefPages = require('../render/briefs');
const access = require('../domain/access');
const access = require('../domain/access');
const { csrfToken, checkCsrf } = require('../auth/forms');
const { csrfToken, checkCsrf } = require('../auth/forms');
const cache = require('openvibe-shared/cache-policy');
const cache = require('openvibe-shared/cache-policy');
@@ -36,7 +37,7 @@ const { SITE_SUMMARY } = require('./discovery');
@@ -36,7 +37,7 @@ const { SITE_SUMMARY } = require('./discovery');
const PER_PAGE = 20;
const PER_PAGE = 20;
function createPublicRoutes(ctx) {
function createPublicRoutes(ctx) {
- const { config, store, stories, publication, reading, topics, viewers, community, discussion, clusters } = ctx;
+ const { config, store, stories, publication, reading, topics, viewers, community, discussion, clusters, briefs } = ctx;
const router = express.Router();
const router = express.Router();
router.use(viewers.middleware({ services: false }));
router.use(viewers.middleware({ services: false }));
@@ -136,6 +137,65 @@ function createPublicRoutes(ctx) {
@@ -136,6 +137,65 @@ function createPublicRoutes(ctx) {
}, { cacheable: true });
}, { cacheable: true });
}));
}));
+ // ── AI briefs and the live view (domain/briefs.js, render/briefs.js) ──
+ // Briefs are AI text: labelled, every paragraph cited, never a story and never indexed. A reader may ask for another
+ // tone or the outlets' comparison (a POST, from this site, a few an address and a few dozen an hour in all), never a
+ // crawler following a link.
+ const noindex = (path) => seo.evaluate({ state: 'published', visibility: 'public', canonicalUrl: seo.canonicalUrl(config.baseUrl, path), wordCount: 0, noindex: true }, { policy: { minWords: 0 }, now: store.now() });
+ const BRIEF_ID = /^clu_[0-9A-HJKMNP-TV-Z]{26}$/;
+ const makeLimit = rateLimit({ windowMs: 10 * 60_000, limit: 5, standardHeaders: true, legacyHeaders: false, keyGenerator: (req) => req.ip,
+ handler: (req, res) => messagePage(req, res, 429, 'Too many requests', 'Wait a few minutes, then ask again.', { href: `/briefs/${req.params.id}`, label: 'Back to the brief' }) });
+ const runs = { hour: 0, at: 0 };
+ const RUNS_AN_HOUR = 40;
+ const sameSite = (req) => { const site = String(req.get('sec-fetch-site') || ''); return site ? site === 'same-origin' : String(req.get('origin') || '') === config.baseUrl; };
+ const canMake = () => Boolean(briefs && config.briefs.enabled && ctx.ai && ctx.ai.enabled);
+
+ router.get('/briefs', wrap(async (req, res) => {
+ send(req, res, 200, { title: 'AI briefs', description: 'What the reports on a subject say, summarized by AI with every paragraph linked to its reports.', decision: noindex('/briefs'), canonical: `${config.baseUrl}/briefs`, localStyles: ['css/briefs.css'], body: briefPages.briefsIndex({ briefs: briefs ? await briefs.latest(40) : [] }) }, { cacheable: true });
+ }));
+
+ router.get('/briefs/:id', wrap(async (req, res) => {
+ const id = String(req.params.id || '');
+ const tone = briefPages.TONES.some(([t]) => t === req.query.tone) ? String(req.query.tone) : 'neutral';
+ const got = BRIEF_ID.test(id) && briefs ? await briefs.get(id, { kind: 'summary', tone }) : null;
+ if (!got || !got.reports.length) return notFound(req, res);
+ const persp = await briefs.get(id, { kind: 'perspectives', tone: 'neutral' });
+ const notice = req.query.made === '1' ? 'Written just now.' : (req.query.failed ? String(req.query.failed).slice(0, 200) : null);
+ send(req, res, 200, {
+ title: `${got.brief && got.brief.headline ? got.brief.headline : got.cluster.label} (AI brief)`,
+ description: 'An AI summary of the reports on this subject, every paragraph linked to its reports.',
+ decision: noindex(`/briefs/${id}`), canonical: `${config.baseUrl}/briefs/${id}`, localStyles: ['css/briefs.css'],
+ body: briefPages.briefPage({ ...got, tone, perspectives: persp ? persp.brief : null, canMake: canMake(), notice }),
+ }, { cacheable: !notice });
+ }));
+
+ router.post('/briefs/:id/make', makeLimit, express.urlencoded({ extended: false, limit: '4kb' }), wrap(async (req, res) => {
+ const id = String(req.params.id || '');
+ if (!BRIEF_ID.test(id) || !canMake()) return notFound(req, res);
+ if (!sameSite(req)) return messagePage(req, res, 403, 'Not from this site', 'Ask from the brief\'s own page.', { href: `/briefs/${id}`, label: 'Back to the brief' });
+ const kind = req.body && req.body.kind === 'perspectives' ? 'perspectives' : 'summary';
+ const tone = briefPages.TONES.some(([t]) => t === (req.body && req.body.tone)) ? req.body.tone : 'neutral';
+ const back = `/briefs/${id}${tone !== 'neutral' ? `?tone=${tone}&` : '?'}`;
+ if (Date.now() - runs.at > 3_600_000) { runs.at = Date.now(); runs.hour = 0; }
+ if (runs.hour >= RUNS_AN_HOUR) return res.redirect(303, `${back}failed=${encodeURIComponent('Many briefs were asked for this hour. Try again later.')}`);
+ runs.hour += 1;
+ try {
+ await briefs.make(id, { kind, tone });
+ } catch (err) {
+ return res.redirect(303, `${back}failed=${encodeURIComponent(err.status === 422 ? err.message : 'OpenVibe.AI could not write it just now. Try again in a few minutes.')}`);
+ }
+ return res.redirect(303, `${back}made=1`);
+ }));
+
+ router.get('/live', wrap(async (req, res) => {
+ const list = await clusters.wire({ limit: 40 });
+ const now = store.now();
+ const breaking = new Set(list.filter((c) => c.items.filter((i) => i.published_at && now - Date.parse(i.published_at) <= 2 * 3_600_000).length >= 3).map((c) => c.id));
+ const briefIds = new Set(briefs ? (await briefs.latest(200)).map((b) => b.cluster_id) : []);
+ res.set('Refresh', '60');
+ send(req, res, 200, { title: 'Live', description: 'The newest reports from the sources OpenVibe.News follows, as they arrive, with what is breaking.', decision: noindex('/live'), canonical: `${config.baseUrl}/live`, localStyles: ['css/briefs.css'], body: briefPages.livePage({ clusters: list, briefIds, breaking }) }, { cacheable: true });
+ }));
+
router.get('/topics', wrap(async (req, res) => {
router.get('/topics', wrap(async (req, res) => {
send(req, res, 200, { title: 'Topics', description: 'Topics on OpenVibe.News.', decision: pageDecision('/topics'), canonical: publication.abs('/topics'), body: pages.topicsIndex({ topics: await topics.list() }) }, { cacheable: true });
send(req, res, 200, { title: 'Topics', description: 'Topics on OpenVibe.News.', decision: pageDecision('/topics'), canonical: publication.abs('/topics'), body: pages.topicsIndex({ topics: await topics.list() }) }, { cacheable: true });
}));
}));
new file mode 100644
new file mode 100644
@@ -0,0 +1,77 @@
@@ -0,0 +1,77 @@
+'use strict';
+
+/**
+ * AI briefs and the live view (server/domain/briefs.js; routes in http/public.js). Everything here that OpenVibe.AI
+ * wrote is labelled as AI, and every paragraph links the reports it rests on, numbered in the table below it. The
+ * reports' own headlines and links are the outlets'.
+ */
+const ssr = require('openvibe-publishing/ssr');
+const { time, httpHref } = require('./pages');
+
+const { html: h, raw } = ssr;
+const TONES = [['brief', 'In brief'], ['neutral', 'Summary'], ['explained', 'Explained']];
+
+const AI_NOTE = 'Written by OpenVibe.AI from the reports listed below, not by an editor. Each paragraph links the reports it rests on; read them before relying on it.';
+
+function sourceTable(reports) {
+ return h`<table class="sources brief-sources"><thead><tr><th>#</th><th>Report</th><th>Outlet</th><th>Date</th></tr></thead><tbody>
+${reports.map((r, i) => h`<tr id="source-${i + 1}"><td>${i + 1}</td><td>${httpHref(r.canonical_url) ? h`<a href="${httpHref(r.canonical_url)}" rel="noopener nofollow" target="_blank">${r.headline}</a>` : r.headline}${r.summary ? h`<p class="source-summary">${r.summary}</p>` : ''}</td><td>${r.outlet}</td><td>${time(r.published_at ? Date.parse(r.published_at) : null)}</td></tr>`)}
+</tbody></table>`;
+}
+
+/** Paragraphs whose sources are news item ids, cited by their place in `reports` (1-based). */
+function paragraphs(brief, reports) {
+ const n = new Map(reports.map((r, i) => [r.id, i + 1]));
+ return brief.paragraphs.map((p) => {
+ const nums = p.sources.map((id) => n.get(id)).filter(Boolean);
+ return h`<p class="brief-p">${p.text}${nums.length ? h` <sup class="cites">${nums.map((x, i) => h`${i ? ',' : ''}<a href="#source-${x}" aria-label="report ${x}">${x}</a>`)}</sup>` : ''}</p>`;
+ });
+}
+
+function makeButton(clusterId, { kind, tone, label }) {
+ return h`<form class="brief-make" method="post" action="/briefs/${clusterId}/make"><input type="hidden" name="kind" value="${kind}"><input type="hidden" name="tone" value="${tone}"><button type="submit">${label}</button></form>`;
+}
+
+/** /briefs/:id */
+function briefPage({ cluster, reports, brief, fresh, tone, perspectives, canMake, notice = null }) {
+ const outlets = [...new Set(reports.map((r) => r.outlet))];
+ const shown = brief && !brief.stub ? brief : null;
+ const tabs = h`<nav class="brief-tones" aria-label="How to tell it">${TONES.map(([t, label]) => (t === tone
+ ? h`<span class="brief-tone on" aria-current="page">${label}</span>`
+ : h`<a class="brief-tone" href="/briefs/${cluster.id}${t === 'neutral' ? '' : `?tone=${t}`}">${label}</a>`))}</nav>`;
+ const persp = perspectives && !perspectives.stub ? perspectives : null;
+ return String(h`<header class="site-header"><p class="eyebrow"><span class="badge badge-ai">AI brief</span> from ${reports.length} ${reports.length === 1 ? 'report' : 'reports'}${outlets.length ? h` (${outlets.slice(0, 5).join(', ')}${outlets.length > 5 ? ` and ${outlets.length - 5} more` : ''})` : ''}</p>
+<h1>${shown && shown.headline ? shown.headline : cluster.label}</h1>
+<p class="meta">${AI_NOTE}</p></header>
+${notice ? h`<p class="notice">${notice}</p>` : ''}
+${tabs}
+<section class="brief" aria-label="The brief">
+${shown ? h`${paragraphs(shown, reports)}${!fresh ? h`<p class="notice small">New reports arrived since this was written. ${canMake ? makeButton(cluster.id, { kind: 'summary', tone, label: 'Write it again' }) : ''}</p>` : ''}`
+ : h`<p class="empty">${canMake ? h`Not written in this tone yet. ${makeButton(cluster.id, { kind: 'summary', tone, label: 'Write it with AI' })}` : 'No brief yet.'}</p>`}
+</section>
+${outlets.length >= 2 ? h`<section class="brief-frames" aria-labelledby="frames-h"><h2 id="frames-h">How the outlets put it</h2>
+${persp ? h`<p class="meta">${AI_NOTE}</p>${paragraphs(persp, reports)}` : h`<p class="meta">How each outlet frames this, side by side, with what they agree and differ on.</p>${canMake ? makeButton(cluster.id, { kind: 'perspectives', tone: 'neutral', label: 'Compare the outlets with AI' }) : ''}`}
+</section>` : ''}
+<section aria-labelledby="reports-h"><h2 id="reports-h">The reports</h2>${sourceTable(reports)}</section>
+${shown && shown.gaps.length ? h`<details class="brief-gaps"><summary>What the AI left out (${shown.gaps.length})</summary><ul>${shown.gaps.map((g) => h`<li>${g}</li>`)}</ul></details>` : ''}
+<p class="meta"><a href="/briefs">All AI briefs</a> · <a href="/live">Live</a> · <a href="/wire">The wire</a></p>`);
+}
+
+/** /briefs */
+function briefsIndex({ briefs }) {
+ return String(h`<header class="site-header"><h1>AI briefs</h1><p class="lede">What the reports on a subject say, summarized by OpenVibe.AI with every paragraph linked to its reports, and labelled as AI. Editors' stories are on <a href="/">the front page</a>.</p></header>
+${briefs.length ? h`<ol class="story-list">${briefs.map((b) => h`<li class="story-item"><h2 class="story-item-title"><a href="/briefs/${b.cluster_id}">${b.headline || b.label}</a> <span class="badge badge-ai">AI</span></h2>
+<p class="meta">${time(b.window_end)} · from ${b.item_count} reports</p>${b.paragraphs[0] ? h`<p class="summary">${b.paragraphs[0].text.slice(0, 280)}${b.paragraphs[0].text.length > 280 ? '…' : ''}</p>` : ''}</li>`)}</ol>`
+ : h`<p class="empty">No briefs yet. They are written as reports on a subject come in from two or more of the sources.</p>`}`);
+}
+
+/** /live: the newest reports as they arrive, grouped, with what is breaking. */
+function livePage({ clusters, briefIds, breaking }) {
+ return String(h`<header class="site-header"><h1>Live <span class="live-dot" aria-hidden="true"></span></h1><p class="lede">The newest reports from the sources OpenVibe.News follows, as they arrive. This page refreshes itself every minute. <span class="badge badge-breaking">Breaking</span> marks a subject three or more reports covered in the last two hours.</p></header>
+${clusters.length ? h`<ol class="wire-list wide live-list">${clusters.map((c) => h`<li class="wire-cluster${breaking.has(c.id) ? ' is-breaking' : ''}">
+<p class="wire-lead">${breaking.has(c.id) ? h`<span class="badge badge-breaking">Breaking</span> ` : ''}<strong>${c.label}</strong>${briefIds.has(c.id) ? h` · <a href="/briefs/${c.id}">AI brief</a>` : ''}</p>
+<ul class="wire-items">${c.items.slice(0, 5).map((i) => h`<li>${httpHref(i.url) ? h`<a href="${httpHref(i.url)}" rel="nofollow noopener" target="_blank">${i.headline}</a>` : i.headline} <span class="meta">${i.outlet} · ${time(i.published_at ? Date.parse(i.published_at) : null)}</span></li>`)}</ul></li>`)}</ol>`
+ : h`<p class="empty">Nothing new from the sources yet.</p>`}`);
+}
+
+module.exports = { briefPage, briefsIndex, livePage, AI_NOTE, TONES };
@@ -22,6 +22,8 @@ const NAV_LINKS = [
@@ -22,6 +22,8 @@ const NAV_LINKS = [
{ label: 'News', href: '/' },
{ label: 'News', href: '/' },
{ label: 'Topics', href: '/topics' },
{ label: 'Topics', href: '/topics' },
{ label: 'Wire', href: '/wire' },
{ label: 'Wire', href: '/wire' },
+ { label: 'Live', href: '/live' },
+ { label: 'AI briefs', href: '/briefs' },
];
];
const hashes = new Map();
const hashes = new Map();
@@ -99,6 +101,8 @@ function renderPage(o) {
@@ -99,6 +101,8 @@ function renderPage(o) {
home: '/',
home: '/',
css: asset('css/news.css'),
css: asset('css/news.css'),
styles: o.styles, // openvibe-shared stylesheet names (the home page's showcase.css)
styles: o.styles, // openvibe-shared stylesheet names (the home page's showcase.css)
+ // This service's own extra stylesheets for one kind of page (the briefs and live pages' css/briefs.css).
+ head: (o.localStyles || []).map((rel) => `<link rel="stylesheet" href="${asset(rel)}">`).join('\n'),
release: RELEASE,
release: RELEASE,
account,
account,
body: o.body,
body: o.body,
@@ -7,8 +7,15 @@
@@ -7,8 +7,15 @@
* (idempotent), the backstop for webhook deliveries that never arrived.
* (idempotent), the backstop for webhook deliveries that never arrived.
* A failed pull is recorded and relayed once per outage; it creates nothing.
* A failed pull is recorded and relayed once per outage; it creates nothing.
*/
*/
-function createWorker({ config, ingest, outbox, log = console }) {
+function createWorker({ config, ingest, outbox, briefs = null, log = console }) {
let timer = null;
let timer = null;
+ let briefTimer = null;
+
+ /** AI briefs for the busiest recent clusters (server/domain/briefs.js); a failure waits for the next tick. */
+ async function briefTick() {
+ if (!briefs) return 0;
+ try { return await briefs.sweep(); } catch (err) { log.error('[News] brief tick failed:', err.message); return 0; }
+ }
async function pullTick() {
async function pullTick() {
if (!ingest) return null;
if (!ingest) return null;
@@ -24,13 +31,20 @@ function createWorker({ config, ingest, outbox, log = console }) {
@@ -24,13 +31,20 @@ function createWorker({ config, ingest, outbox, log = console }) {
return {
return {
pullTick,
pullTick,
+ briefTick,
start() {
start() {
- if (!config.worker.enabled || !config.sources.pullIntervalMs) return;
+ if (!config.worker.enabled) return;
+ if (config.briefs && config.briefs.enabled && briefs) {
+ briefTimer = setInterval(briefTick, config.briefs.intervalMs);
+ briefTimer.unref();
+ setTimeout(briefTick, 30_000).unref();
+ }
+ if (!config.sources.pullIntervalMs) return;
timer = setInterval(pullTick, config.sources.pullIntervalMs);
timer = setInterval(pullTick, config.sources.pullIntervalMs);
timer.unref();
timer.unref();
setTimeout(pullTick, 2000).unref();
setTimeout(pullTick, 2000).unref();
},
},
- stop() { clearInterval(timer); },
+ stop() { clearInterval(timer); clearInterval(briefTimer); },
};
};
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,90 @@
@@ -0,0 +1,90 @@
+'use strict';
+/**
+ * AI briefs and the live view (server/domain/briefs.js, render/briefs.js; the owner's April ask: multi-source summaries
+ * with tone and perspective controls, and a breaking-news live updater). The sweep summarizes a cluster with two or
+ * more reports; every paragraph cites its reports and uncited claims are dropped; the page labels it as AI, is never
+ * indexed, and shows the reports; a reader's POST asks for another tone or the outlets' comparison (same-origin only);
+ * a stub provider's output is never shown; a changed cluster's brief is stale and made again; /live refreshes itself
+ * and marks what is breaking.
+ */
+const assert = require('assert');
+const { boot, check, done, launchReports } = require('./helpers/boot');
+
+const run = (output, id = 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E1', synthetic = false) => ({ status: 201, run: { id, status: 'succeeded', workflow: { key: 'news.summarize_story', version: 3 }, synthetic, provenance: { model: 'test-model' }, output } });
+const SAME = { 'sec-fetch-site': 'same-origin' };
+
+(async () => {
+ const t = await boot({ ai: true });
+ try {
+ launchReports(t.sources);
+ await t.pull();
+ const clusterId = (await t.db().prepare("SELECT cluster_id, count(*) AS n FROM news_source_items WHERE status = 'active' GROUP BY cluster_id ORDER BY n DESC").get()).cluster_id;
+
+ await check('the sweep writes a cited summary for a cluster with two or more reports; uncited claims are dropped', async () => {
+ t.ai.setNext(run({ headline: 'Europa Clipper is on its way', summary: 'NASA launched Europa Clipper toward Jupiter.', key_points: [{ text: 'It reaches Jupiter in 2030.', citations: [1] }, { text: 'A claim nobody reported.', citations: [] }], timeline: [], citations: [0, 1], gaps: [] }));
+ const made = await t.ctx.briefs.sweep();
+ assert.strictEqual(made, 1);
+ const sent = t.ai.requests.pop();
+ assert.strictEqual(sent.workflow, 'news.summarize_story');
+ assert.strictEqual(sent.input.tone, 'neutral');
+ const got = await t.ctx.briefs.get(clusterId);
+ assert.ok(got.fresh);
+ assert.deepStrictEqual(got.brief.paragraphs.map((p) => p.text), ['NASA launched Europa Clipper toward Jupiter.', 'It reaches Jupiter in 2030.']);
+ assert.ok(got.brief.gaps.some((g) => /uncited/.test(g)));
+ assert.strictEqual(await t.ctx.briefs.sweep(), 0, 'a fresh brief is not made again');
+ });
+
+ await check('/briefs/:id: labelled AI, cited to the reports below it, never indexed; listed on /briefs', async () => {
+ const r = await t.get(`/briefs/${clusterId}`);
+ assert.strictEqual(r.status, 200, r.text.slice(0, 200));
+ assert.ok(r.text.includes('AI brief') && r.text.includes('Written by OpenVibe.AI'));
+ assert.ok(r.text.includes('Europa Clipper is on its way'));
+ assert.match(r.text, /href="#source-2"/);
+ assert.ok(r.text.includes('id="source-1"'), 'the reports are in the page');
+ assert.ok(r.text.includes('noindex'));
+ assert.ok(!r.text.includes('A claim nobody reported') || r.text.includes('What the AI left out'));
+ const list = await t.get('/briefs');
+ assert.ok(list.text.includes(`/briefs/${clusterId}`));
+ });
+
+ await check('a reader asks for another tone with a POST from this site; a GET never runs AI', async () => {
+ const before = t.ai.requests.length;
+ const page = await t.get(`/briefs/${clusterId}?tone=brief`);
+ assert.ok(page.text.includes('Write it with AI'));
+ assert.strictEqual(t.ai.requests.length, before, 'reading never runs AI');
+ assert.strictEqual((await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'brief' }, headers: { 'sec-fetch-site': 'cross-site' } })).status, 403);
+ t.ai.setNext(run({ headline: 'Europa Clipper launched', summary: 'NASA sent Europa Clipper to Jupiter.', key_points: [], timeline: [], citations: [0], gaps: [] }, 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E2'));
+ const made = await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'brief' }, headers: SAME });
+ assert.strictEqual(made.status, 303);
+ assert.strictEqual(t.ai.requests.pop().input.tone, 'brief');
+ assert.ok((await t.get(`/briefs/${clusterId}?tone=brief`)).text.includes('NASA sent Europa Clipper to Jupiter.'));
+ });
+
+ await check('how the outlets put it: compare_perspectives on request, cited', async () => {
+ t.ai.setNext(run({ question: 'How was the launch covered?', perspectives: [{ label: 'Wire A', summary: 'Focus on the rocket.', citations: [0] }, { label: 'Paper B', summary: 'Focus on the science.', citations: [1] }], agreements: [], disagreements: [], citations: [0, 1], gaps: [] }, 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E3'));
+ const made = await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'perspectives', tone: 'neutral' }, headers: SAME });
+ assert.strictEqual(made.status, 303);
+ assert.strictEqual(t.ai.requests.pop().workflow, 'news.compare_perspectives');
+ assert.ok((await t.get(`/briefs/${clusterId}`)).text.includes('Paper B: Focus on the science.'));
+ });
+
+ await check('stub output is never shown; a failed run makes nothing and says so', async () => {
+ t.ai.setNext(run({ headline: 'STUB', summary: 'stub text', key_points: [], timeline: [], citations: [0], gaps: [] }, 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E4', true));
+ await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'explained' }, headers: SAME });
+ assert.ok(!(await t.get(`/briefs/${clusterId}?tone=explained`)).text.includes('stub text'));
+ t.ai.setNext(null);
+ const failed = await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'explained' }, headers: SAME });
+ assert.match(failed.headers.get('location'), /failed=/);
+ });
+
+ await check('/live refreshes itself and links the AI briefs; it is not indexed', async () => {
+ const r = await t.get('/live');
+ assert.strictEqual(r.status, 200);
+ assert.strictEqual(r.headers.get('refresh'), '60');
+ assert.ok(r.text.includes(`/briefs/${clusterId}`) && r.text.includes('noindex'));
+ const home = await t.get('/');
+ assert.ok(home.text.includes('href="/live"') && home.text.includes('href="/briefs"'), 'in the navigation');
+ });
+ } finally { await t.close(); }
+ done();
+})().catch((e) => { console.error(e); process.exit(1); });