openvibers/openvibe.news Commits

News: AI briefs with tone and outlet comparisons, and a live breaking view (#68)

OP OpenVibers committed 29 days ago · de2a634df2 · 12 files, +434 −7

The owner's April ask: multi-source summaries with tone and perspective controls, and a breaking-news live updater. Stories stay editor-written. Beside them: /briefs, an AI summary of each busy cluster (two reports or more), labelled as AI, every paragraph cited to the reports numbered below it, uncited claims dropped and listed, stub output never shown, written by the worker and again when the cluster changes; tone (brief, neutral, explained) and how the outlets put it (compare_perspectives) on a reader's same-origin POST, rate-limited; /live, the newest reports refreshing every minute with Breaking marks. Both noindex. Migration 0004; css/briefs.css loads only on these pages (news.css stays in budget).

Changes

ddiff --git a/README.md b/README.md
iindex 2359972..fd56aec 100644
---- a/README.md
++++ b/README.md
@@@ -26,6 +26,26 @@ shown, fed or indexed.
News publishes nothing until an editor does: the only seed is the topic list. Until then, and beside the stories after, **the wire** (`/wire`, and "From the wire" on the front page) shows what the sources report, grouped by cluster: each line is the outlet's own headline, outlet, date and link (the fields the licensing rules let News keep), never News' writing; duplicates and reports removed upstream leave it; `/wire` is `noindex, follow`.
++## AI briefs and Live
++
++The owner's April ask for News was multi-source summaries with tone and perspective controls and a live breaking-news
++view. Stories stay editor-written (below); beside them News now has:
++
++- **AI briefs** (`/briefs`, `/briefs/:cluster`; [server/domain/briefs.js](server/domain/briefs.js),
++ `migrations/0004_cluster_briefs.sql`): what a cluster's reports say, written by OpenVibe.AI and labelled as AI on
++ every page. Every paragraph cites the reports it rests on, numbered in the table below it; claims the model did not
++ cite are dropped and listed under "What the AI left out"; a run with no cited text makes no brief; the stub
++ provider's output is never shown. Only News' stored fields are sent (headline, outlet, date, URL and the short summary
++ the terms allow). The worker writes a neutral summary for every open cluster with two or more reports in the last 72
++ hours, six a tick every `NEWS_AI_BRIEFS_INTERVAL_MS` (10 minutes), and writes it again once the cluster changes.
++- **Tone** (`?tone=brief|neutral|explained`, `news.summarize_story`'s `tone`) and **how the outlets put it**
++ (`news.compare_perspectives`, two outlets or more): made when a reader asks, with a POST from this site (five every ten
++ minutes an address, 40 an hour in all), never by a link a crawler follows.
++- **Live** (`/live`): the newest reports as they arrive, grouped, refreshing itself every minute (`Refresh: 60`), with
++ **Breaking** on a subject three or more reports covered in the last two hours, and links to their AI briefs.
++- Briefs and Live are `noindex`: they are AI text and other publishers' headlines, not News' writing.
++ `NEWS_AI_BRIEFS=off` turns briefs off.
++
## Owns
The nine charter tables live in News' own PostgreSQL database (`ov_news` on the host's data role, ADR-035; schema in [migrations/](migrations/)):
ddiff --git a/migrations/0004_cluster_briefs.sql b/migrations/0004_cluster_briefs.sql
nnew file mode 100644
iindex 0000000..e356092
---- /dev/null
++++ b/migrations/0004_cluster_briefs.sql
@@@ -0,0 +1,23 @@
++-- phase: expand
++-- AI briefs (server/domain/briefs.js; the owner's April ask: multi-source summaries with tone and perspective controls).
++-- One row per cluster, kind and tone: a summary of what the cluster's reports say (tone brief | neutral | explained),
++-- or how its outlets frame it (kind perspectives). Every paragraph cites the source items it rests on; the model's
++-- uncited claims are dropped and listed in gaps. A brief is AI text, labelled as such, never a story: stories stay
++-- editor-written. A brief is stale once its cluster changes (item_count or cluster_updated_at) and is made again.
++CREATE TABLE IF NOT EXISTS news_cluster_briefs (
++ cluster_id text COLLATE "C" NOT NULL REFERENCES news_story_clusters(id) ON DELETE CASCADE,
++ kind text COLLATE "C" NOT NULL CHECK (kind IN ('summary', 'perspectives')),
++ tone text COLLATE "C" NOT NULL DEFAULT 'neutral' CHECK (tone IN ('brief', 'neutral', 'explained')),
++ headline text COLLATE "C",
++ paragraphs text COLLATE "C" NOT NULL, -- JSON [{ text, sources: [nsi_…] }]
++ source_ids text COLLATE "C" NOT NULL, -- JSON [nsi_…] the reports it was made from, in order
++ gaps text COLLATE "C" NOT NULL DEFAULT '[]',
++ run_id text COLLATE "C",
++ model text COLLATE "C",
++ stub boolean NOT NULL DEFAULT false, -- the stub provider's synthetic output: never shown
++ item_count integer NOT NULL,
++ cluster_updated_at bigint NOT NULL,
++ created_at bigint NOT NULL,
++ PRIMARY KEY (cluster_id, kind, tone)
++);
++CREATE INDEX IF NOT EXISTS news_cluster_briefs_recent ON news_cluster_briefs (kind, tone, created_at DESC);
ddiff --git a/public/css/briefs.css b/public/css/briefs.css
nnew file mode 100644
iindex 0000000..adef62b
---- /dev/null
++++ b/public/css/briefs.css
@@@ -0,0 +1,17 @@
++/* OpenVibe.News — the AI briefs and live pages only (render/briefs.js), so news.css stays inside its budget. */
++.badge-ai { background: color-mix(in srgb, #a78bfa 18%, transparent); color: color-mix(in srgb, #8b5cf6 70%, var(--text-primary, #e6edf7)); border: 1px solid color-mix(in srgb, #a78bfa 45%, transparent); }
++.badge-breaking { background: color-mix(in srgb, #ef4444 16%, transparent); color: color-mix(in srgb, #ef4444 75%, var(--text-primary, #e6edf7)); border: 1px solid color-mix(in srgb, #ef4444 45%, transparent); }
++.eyebrow { margin: 0 0 .4rem; font-size: .9rem; }
++.brief-tones { display: flex; flex-wrap: wrap; gap: 8px; margin: 12px 0 16px; }
++.brief-tone { padding: 5px 12px; border-radius: 999px; border: 1px solid var(--border-light, #2c3d5c); text-decoration: none; color: inherit; font-size: .9rem; }
++.brief-tone.on { border-color: var(--accent, #3b82f6); font-weight: 700; }
++.brief .brief-p, .brief-frames .brief-p { max-width: 72ch; line-height: 1.6; }
++.brief-make { display: inline; }
++.brief-make button { font: inherit; padding: 5px 12px; border-radius: 8px; border: 1px solid var(--border-light, #2c3d5c); background: var(--bg-tertiary, #16223a); color: inherit; cursor: pointer; }
++.brief-make button:hover, .brief-make button:focus-visible { border-color: var(--accent, #3b82f6); }
++.brief-frames { margin-top: 1.6rem; }
++.brief-gaps { margin-top: 1rem; font-size: .9rem; }
++.live-dot { display: inline-block; width: .55em; height: .55em; border-radius: 50%; background: #ef4444; vertical-align: middle; animation: live-pulse 1.6s ease-in-out infinite; }
++@keyframes live-pulse { 50% { opacity: .35; } }
++@media (prefers-reduced-motion: reduce) { .live-dot { animation: none; } }
++.wire-cluster.is-breaking { border-left: 3px solid #ef4444; padding-left: 10px; }
ddiff --git a/server/app.js b/server/app.js
iindex 10d5ca3..4188dbb 100644
---- a/server/app.js
++++ b/server/app.js
@@@ -26,6 +26,7 @@ const { openStore } = require('./db');
const { createViewerResolver } = require('./auth/viewer');
const { createPeople } = require('./clients/network');
const { createAi } = require('./clients/ai');
++const { createBriefs } = require('./domain/briefs');
const { createCommunity } = require('./clients/community');
const { createPublication } = require('./domain/publication');
const { createDiscussion } = require('./domain/discussion');
@@@ -97,9 +98,10 @@ async function createApp(opts = {}) {
secureCookies: config.cookies.secure,
});
const viewers = createViewerResolver({ auth, config, people });
-- const worker = createWorker({ config, ingest, outbox, log });
++ const briefs = createBriefs({ store, ai: config.briefs.enabled ? ai : null, log });
++ const worker = createWorker({ config, ingest, outbox, briefs, log });
-- const ctx = { config, store, outbox, publication, clusters, sources, ai, community, discussion, ingest, stories, topics, people, reading, auth, viewers, worker, indexnow };
++ const ctx = { config, store, outbox, publication, clusters, sources, ai, community, discussion, ingest, stories, topics, people, reading, auth, viewers, worker, indexnow, briefs };
const app = express();
app.disable('x-powered-by');
ddiff --git a/server/clients/ai.js b/server/clients/ai.js
iindex 10ab1f3..4157581 100644
---- a/server/clients/ai.js
++++ b/server/clients/ai.js
@@@ -29,11 +29,12 @@ function createAi({ config, fetchImpl = globalThis.fetch }) {
}) : null;
/** sources: [{ id (nsi_…), sources_item_id, headline, url, outlet, authors, published_at, retrieved_at, summary }] in citation order. */
-- async function run(workflow, { topic, sources }) {
++ async function run(workflow, { topic, sources, tone = null }) {
if (!enabled) throw new AiError('ai.not_configured', 'OpenVibe.AI is not configured (OV_AI_INTERNAL_URL)', 503);
if (!WORKFLOWS.includes(workflow)) throw new AiError('ai.unknown_workflow', `workflow must be one of ${WORKFLOWS.join(', ')}`, 422);
const input = {
...(topic ? { topic: String(topic).slice(0, 300) } : {}),
++ ...(tone && workflow === 'news.summarize_story' ? { tone } : {}),
sources: sources.slice(0, 50).map((s) => ({
source_type: 'news.article', source_id: s.sources_item_id, title: s.headline, ...(s.url ? { url: s.url } : {}),
...(s.authors && s.authors.length ? { author: s.authors.join(', ').slice(0, 200) } : {}),
ddiff --git a/server/config.js b/server/config.js
iindex e8e9add..07c450f 100644
---- a/server/config.js
++++ b/server/config.js
@@@ -119,6 +119,13 @@ function load(env = process.env) {
enabled: env.NEWS_WORKER !== 'off',
},
++ // AI briefs (server/domain/briefs.js): the sweep summarizes busy clusters every intervalMs; off with
++ // NEWS_AI_BRIEFS=off (readers' requests for a tone or the perspectives then answer 503 too).
++ briefs: {
++ enabled: env.NEWS_AI_BRIEFS !== 'off',
++ intervalMs: Math.max(60_000, int(env.NEWS_AI_BRIEFS_INTERVAL_MS, 600_000)),
++ },
++
// Browser origins that may call /api/v1 with a Bearer Network JWT (no cookies cross origins).
apiCorsOrigins: list(env.API_CORS_ORIGINS || 'https://openvibe.network,https://openvibe.live,https://openvibe.community'),
};
ddiff --git a/server/domain/briefs.js b/server/domain/briefs.js
nnew file mode 100644
iindex 0000000..77deaee
---- /dev/null
++++ b/server/domain/briefs.js
@@@ -0,0 +1,112 @@
++'use strict';
++
++/**
++ * AI briefs: what a cluster's reports say, written by OpenVibe.AI and labelled as AI (migrations/0004_cluster_briefs.sql).
++ * The owner's April ask for News: multi-source summarization, tone controls and perspective/slant controls, and a live
++ * breaking view. Stories stay what they were: editor-written, every paragraph cited, never AI text without a person's
++ * approval. A brief is the other thing: an AI summary of a cluster, shown as such, beside the wire.
++ *
++ * summary news.summarize_story over the cluster's live reports, in a tone: brief, neutral (made by the sweep
++ * for every cluster with at least MIN_REPORTS reports) or explained (made when a reader asks)
++ * perspectives news.compare_perspectives: how the outlets frame it (when a reader asks; two outlets or more)
++ *
++ * Only News' stored fields go to AI (headline, outlet, date, URL, and the short summary its terms allow). A paragraph the
++ * model did not cite is dropped (clients/ai.js draftFromOutput) and listed as a gap; a run with no cited text makes no
++ * brief. The stub provider's synthetic output is stored but never shown. A brief made from a cluster that has since
++ * changed is stale and is made again (the sweep for neutral summaries, the next reader's request for the rest).
++ */
++
++const { draftFromOutput } = require('../clients/ai');
++
++const TONES = ['brief', 'neutral', 'explained'];
++const TONE_TEXT = { brief: 'In brief', neutral: 'Summary', explained: 'Explained' };
++const KINDS = ['summary', 'perspectives'];
++const MIN_REPORTS = 2;
++const SWEEP_PER_TICK = 6;
++const SWEEP_WINDOW_MS = 72 * 3_600_000;
++
++const parse = (v, d) => { try { return typeof v === 'string' ? JSON.parse(v) : (v == null ? d : v); } catch { return d; } };
++
++function createBriefs({ store, ai, log = console }) {
++ const db = store.db;
++ const q = {
++ cluster: db.prepare("SELECT * FROM news_story_clusters WHERE id = ? AND status = 'open'"),
++ reports: db.prepare("SELECT * FROM news_source_items WHERE cluster_id = ? AND status = 'active' AND removed_at IS NULL ORDER BY published_at, first_seen_at, seq"),
++ get: db.prepare('SELECT * FROM news_cluster_briefs WHERE cluster_id = ? AND kind = ? AND tone = ?'),
++ put: db.prepare(`INSERT INTO news_cluster_briefs (cluster_id, kind, tone, headline, paragraphs, source_ids, gaps, run_id, model, stub, item_count, cluster_updated_at, created_at)
++ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
++ ON CONFLICT (cluster_id, kind, tone) DO UPDATE SET headline = excluded.headline, paragraphs = excluded.paragraphs, source_ids = excluded.source_ids,
++ gaps = excluded.gaps, run_id = excluded.run_id, model = excluded.model, stub = excluded.stub, item_count = excluded.item_count,
++ cluster_updated_at = excluded.cluster_updated_at, created_at = excluded.created_at`),
++ latest: db.prepare(`SELECT b.*, c.label, c.window_end FROM news_cluster_briefs b JOIN news_story_clusters c ON c.id = b.cluster_id
++ WHERE b.kind = 'summary' AND b.tone = 'neutral' AND b.stub = false AND c.status = 'open' ORDER BY c.window_end DESC, b.created_at DESC LIMIT ?`),
++ // Open clusters active in the window with at least MIN_REPORTS live reports, newest activity first.
++ due: db.prepare(`SELECT c.id, c.updated_at, count(i.id) AS n FROM news_story_clusters c JOIN news_source_items i ON i.cluster_id = c.id AND i.status = 'active' AND i.removed_at IS NULL
++ WHERE c.status = 'open' AND c.window_end >= ? GROUP BY c.id, c.updated_at, c.window_end HAVING count(i.id) >= ? ORDER BY c.window_end DESC LIMIT 200`),
++ };
++
++ const fresh = (row, cluster, n) => Boolean(row && Number(row.item_count) === n && Number(row.cluster_updated_at) === Number(cluster.updated_at));
++
++ /** The stored brief, and whether it still matches its cluster. */
++ async function get(clusterId, { kind = 'summary', tone = 'neutral' } = {}) {
++ const cluster = await q.cluster.get(clusterId);
++ if (!cluster) return null;
++ const reports = await q.reports.all(cluster.id);
++ const row = await q.get.get(cluster.id, kind, tone);
++ return { cluster, reports, brief: row ? shape(row) : null, fresh: fresh(row, cluster, reports.length) };
++ }
++
++ function shape(row) {
++ return {
++ cluster_id: row.cluster_id, kind: row.kind, tone: row.tone, headline: row.headline || null,
++ paragraphs: parse(row.paragraphs, []), source_ids: parse(row.source_ids, []), gaps: parse(row.gaps, []),
++ run_id: row.run_id || null, model: row.model || null, stub: Boolean(row.stub), item_count: Number(row.item_count), created_at: Number(row.created_at),
++ };
++ }
++
++ /** Make (or remake) one brief. Throws the AI client's error when the run fails; makes nothing without cited text. */
++ async function make(clusterId, { kind = 'summary', tone = 'neutral' } = {}) {
++ if (!ai || !ai.enabled) throw Object.assign(new Error('OpenVibe.AI is not configured on this server'), { code: 'ai.not_configured', status: 503 });
++ if (!KINDS.includes(kind) || !TONES.includes(tone)) throw Object.assign(new Error('kind or tone not known'), { code: 'brief.invalid', status: 422 });
++ const cluster = await q.cluster.get(clusterId);
++ if (!cluster) throw Object.assign(new Error('No such cluster'), { code: 'cluster.not_found', status: 404 });
++ const reports = await q.reports.all(cluster.id);
++ if (!reports.length) throw Object.assign(new Error('This cluster has no live reports'), { code: 'brief.no_reports', status: 422 });
++ const outlets = new Set(reports.map((r) => r.outlet));
++ if (kind === 'perspectives' && outlets.size < 2) throw Object.assign(new Error('Comparing how outlets frame it needs two outlets or more'), { code: 'brief.one_outlet', status: 422 });
++ const workflow = kind === 'perspectives' ? 'news.compare_perspectives' : 'news.summarize_story';
++ const result = await ai.run(workflow, {
++ topic: cluster.label, tone: kind === 'summary' ? tone : null,
++ sources: reports.map((r) => ({ id: r.id, sources_item_id: r.sources_item_id, headline: r.headline, url: r.canonical_url, outlet: r.outlet,
++ authors: parse(r.authors, []), published_at: r.published_at, retrieved_at: r.retrieved_at, summary: r.summary })),
++ });
++ const ids = reports.map((r) => r.id);
++ const draft = draftFromOutput(workflow, result.output, ids);
++ if (!draft.paragraphs.length) throw Object.assign(new Error('The AI run returned no cited text; no brief was made'), { code: 'brief.nothing_cited', status: 422 });
++ await q.put.run(cluster.id, kind, tone, draft.headline, JSON.stringify(draft.paragraphs), JSON.stringify(ids), JSON.stringify(draft.gaps),
++ result.runId, (result.workflow && result.workflow.model) || null, Boolean(result.stub), reports.length, Number(cluster.updated_at), store.now());
++ return get(cluster.id, { kind, tone });
++ }
++
++ /** The sweep: neutral summaries for the busiest recent clusters that have none, or a stale one. */
++ async function sweep({ limit = SWEEP_PER_TICK } = {}) {
++ if (!ai || !ai.enabled) return 0;
++ let made = 0;
++ for (const c of await q.due.all(store.now() - SWEEP_WINDOW_MS, MIN_REPORTS)) {
++ if (made >= limit) break;
++ const row = await q.get.get(c.id, 'summary', 'neutral');
++ if (fresh(row, c, Number(c.n))) continue;
++ try { await make(c.id, { kind: 'summary', tone: 'neutral' }); made += 1; } catch (err) {
++ (log.warn || log.log)(`[News] brief for ${c.id}: ${err.code || err.message}`);
++ if (String(err.code || '').startsWith('ai.')) break; // AI is down or refusing: try again next tick
++ }
++ }
++ return made;
++ }
++
++ const latest = async (limit = 12) => (await q.latest.all(limit)).map((r) => ({ ...shape(r), label: r.label, window_end: Number(r.window_end) }));
++
++ return { TONES, TONE_TEXT, KINDS, MIN_REPORTS, get, make, sweep, latest };
++}
++
++module.exports = { createBriefs, TONES, TONE_TEXT, MIN_REPORTS };
ddiff --git a/server/http/public.js b/server/http/public.js
iindex 1ebb8bb..3716a5e 100644
---- a/server/http/public.js
++++ b/server/http/public.js
@@@ -28,6 +28,7 @@ const seo = require('openvibe-publishing/seo');
const ssr = require('openvibe-publishing/ssr');
const { renderPage } = require('../render/layout');
const pages = require('../render/pages');
++const briefPages = require('../render/briefs');
const access = require('../domain/access');
const { csrfToken, checkCsrf } = require('../auth/forms');
const cache = require('openvibe-shared/cache-policy');
@@@ -36,7 +37,7 @@ const { SITE_SUMMARY } = require('./discovery');
const PER_PAGE = 20;
function createPublicRoutes(ctx) {
-- const { config, store, stories, publication, reading, topics, viewers, community, discussion, clusters } = ctx;
++ const { config, store, stories, publication, reading, topics, viewers, community, discussion, clusters, briefs } = ctx;
const router = express.Router();
router.use(viewers.middleware({ services: false }));
@@@ -136,6 +137,65 @@ function createPublicRoutes(ctx) {
}, { cacheable: true });
}));
++ // ── AI briefs and the live view (domain/briefs.js, render/briefs.js) ──
++ // Briefs are AI text: labelled, every paragraph cited, never a story and never indexed. A reader may ask for another
++ // tone or the outlets' comparison (a POST, from this site, a few an address and a few dozen an hour in all), never a
++ // crawler following a link.
++ const noindex = (path) => seo.evaluate({ state: 'published', visibility: 'public', canonicalUrl: seo.canonicalUrl(config.baseUrl, path), wordCount: 0, noindex: true }, { policy: { minWords: 0 }, now: store.now() });
++ const BRIEF_ID = /^clu_[0-9A-HJKMNP-TV-Z]{26}$/;
++ const makeLimit = rateLimit({ windowMs: 10 * 60_000, limit: 5, standardHeaders: true, legacyHeaders: false, keyGenerator: (req) => req.ip,
++ handler: (req, res) => messagePage(req, res, 429, 'Too many requests', 'Wait a few minutes, then ask again.', { href: `/briefs/${req.params.id}`, label: 'Back to the brief' }) });
++ const runs = { hour: 0, at: 0 };
++ const RUNS_AN_HOUR = 40;
++ const sameSite = (req) => { const site = String(req.get('sec-fetch-site') || ''); return site ? site === 'same-origin' : String(req.get('origin') || '') === config.baseUrl; };
++ const canMake = () => Boolean(briefs && config.briefs.enabled && ctx.ai && ctx.ai.enabled);
++
++ router.get('/briefs', wrap(async (req, res) => {
++ send(req, res, 200, { title: 'AI briefs', description: 'What the reports on a subject say, summarized by AI with every paragraph linked to its reports.', decision: noindex('/briefs'), canonical: `${config.baseUrl}/briefs`, localStyles: ['css/briefs.css'], body: briefPages.briefsIndex({ briefs: briefs ? await briefs.latest(40) : [] }) }, { cacheable: true });
++ }));
++
++ router.get('/briefs/:id', wrap(async (req, res) => {
++ const id = String(req.params.id || '');
++ const tone = briefPages.TONES.some(([t]) => t === req.query.tone) ? String(req.query.tone) : 'neutral';
++ const got = BRIEF_ID.test(id) && briefs ? await briefs.get(id, { kind: 'summary', tone }) : null;
++ if (!got || !got.reports.length) return notFound(req, res);
++ const persp = await briefs.get(id, { kind: 'perspectives', tone: 'neutral' });
++ const notice = req.query.made === '1' ? 'Written just now.' : (req.query.failed ? String(req.query.failed).slice(0, 200) : null);
++ send(req, res, 200, {
++ title: `${got.brief && got.brief.headline ? got.brief.headline : got.cluster.label} (AI brief)`,
++ description: 'An AI summary of the reports on this subject, every paragraph linked to its reports.',
++ decision: noindex(`/briefs/${id}`), canonical: `${config.baseUrl}/briefs/${id}`, localStyles: ['css/briefs.css'],
++ body: briefPages.briefPage({ ...got, tone, perspectives: persp ? persp.brief : null, canMake: canMake(), notice }),
++ }, { cacheable: !notice });
++ }));
++
++ router.post('/briefs/:id/make', makeLimit, express.urlencoded({ extended: false, limit: '4kb' }), wrap(async (req, res) => {
++ const id = String(req.params.id || '');
++ if (!BRIEF_ID.test(id) || !canMake()) return notFound(req, res);
++ if (!sameSite(req)) return messagePage(req, res, 403, 'Not from this site', 'Ask from the brief\'s own page.', { href: `/briefs/${id}`, label: 'Back to the brief' });
++ const kind = req.body && req.body.kind === 'perspectives' ? 'perspectives' : 'summary';
++ const tone = briefPages.TONES.some(([t]) => t === (req.body && req.body.tone)) ? req.body.tone : 'neutral';
++ const back = `/briefs/${id}${tone !== 'neutral' ? `?tone=${tone}&` : '?'}`;
++ if (Date.now() - runs.at > 3_600_000) { runs.at = Date.now(); runs.hour = 0; }
++ if (runs.hour >= RUNS_AN_HOUR) return res.redirect(303, `${back}failed=${encodeURIComponent('Many briefs were asked for this hour. Try again later.')}`);
++ runs.hour += 1;
++ try {
++ await briefs.make(id, { kind, tone });
++ } catch (err) {
++ return res.redirect(303, `${back}failed=${encodeURIComponent(err.status === 422 ? err.message : 'OpenVibe.AI could not write it just now. Try again in a few minutes.')}`);
++ }
++ return res.redirect(303, `${back}made=1`);
++ }));
++
++ router.get('/live', wrap(async (req, res) => {
++ const list = await clusters.wire({ limit: 40 });
++ const now = store.now();
++ const breaking = new Set(list.filter((c) => c.items.filter((i) => i.published_at && now - Date.parse(i.published_at) <= 2 * 3_600_000).length >= 3).map((c) => c.id));
++ const briefIds = new Set(briefs ? (await briefs.latest(200)).map((b) => b.cluster_id) : []);
++ res.set('Refresh', '60');
++ send(req, res, 200, { title: 'Live', description: 'The newest reports from the sources OpenVibe.News follows, as they arrive, with what is breaking.', decision: noindex('/live'), canonical: `${config.baseUrl}/live`, localStyles: ['css/briefs.css'], body: briefPages.livePage({ clusters: list, briefIds, breaking }) }, { cacheable: true });
++ }));
++
router.get('/topics', wrap(async (req, res) => {
send(req, res, 200, { title: 'Topics', description: 'Topics on OpenVibe.News.', decision: pageDecision('/topics'), canonical: publication.abs('/topics'), body: pages.topicsIndex({ topics: await topics.list() }) }, { cacheable: true });
}));
ddiff --git a/server/render/briefs.js b/server/render/briefs.js
nnew file mode 100644
iindex 0000000..c39f30c
---- /dev/null
++++ b/server/render/briefs.js
@@@ -0,0 +1,77 @@
++'use strict';
++
++/**
++ * AI briefs and the live view (server/domain/briefs.js; routes in http/public.js). Everything here that OpenVibe.AI
++ * wrote is labelled as AI, and every paragraph links the reports it rests on, numbered in the table below it. The
++ * reports' own headlines and links are the outlets'.
++ */
++const ssr = require('openvibe-publishing/ssr');
++const { time, httpHref } = require('./pages');
++
++const { html: h, raw } = ssr;
++const TONES = [['brief', 'In brief'], ['neutral', 'Summary'], ['explained', 'Explained']];
++
++const AI_NOTE = 'Written by OpenVibe.AI from the reports listed below, not by an editor. Each paragraph links the reports it rests on; read them before relying on it.';
++
++function sourceTable(reports) {
++ return h`<table class="sources brief-sources"><thead><tr><th>#</th><th>Report</th><th>Outlet</th><th>Date</th></tr></thead><tbody>
++${reports.map((r, i) => h`<tr id="source-${i + 1}"><td>${i + 1}</td><td>${httpHref(r.canonical_url) ? h`<a href="${httpHref(r.canonical_url)}" rel="noopener nofollow" target="_blank">${r.headline}</a>` : r.headline}${r.summary ? h`<p class="source-summary">${r.summary}</p>` : ''}</td><td>${r.outlet}</td><td>${time(r.published_at ? Date.parse(r.published_at) : null)}</td></tr>`)}
++</tbody></table>`;
++}
++
++/** Paragraphs whose sources are news item ids, cited by their place in `reports` (1-based). */
++function paragraphs(brief, reports) {
++ const n = new Map(reports.map((r, i) => [r.id, i + 1]));
++ return brief.paragraphs.map((p) => {
++ const nums = p.sources.map((id) => n.get(id)).filter(Boolean);
++ return h`<p class="brief-p">${p.text}${nums.length ? h` <sup class="cites">${nums.map((x, i) => h`${i ? ',' : ''}<a href="#source-${x}" aria-label="report ${x}">${x}</a>`)}</sup>` : ''}</p>`;
++ });
++}
++
++function makeButton(clusterId, { kind, tone, label }) {
++ return h`<form class="brief-make" method="post" action="/briefs/${clusterId}/make"><input type="hidden" name="kind" value="${kind}"><input type="hidden" name="tone" value="${tone}"><button type="submit">${label}</button></form>`;
++}
++
++/** /briefs/:id */
++function briefPage({ cluster, reports, brief, fresh, tone, perspectives, canMake, notice = null }) {
++ const outlets = [...new Set(reports.map((r) => r.outlet))];
++ const shown = brief && !brief.stub ? brief : null;
++ const tabs = h`<nav class="brief-tones" aria-label="How to tell it">${TONES.map(([t, label]) => (t === tone
++ ? h`<span class="brief-tone on" aria-current="page">${label}</span>`
++ : h`<a class="brief-tone" href="/briefs/${cluster.id}${t === 'neutral' ? '' : `?tone=${t}`}">${label}</a>`))}</nav>`;
++ const persp = perspectives && !perspectives.stub ? perspectives : null;
++ return String(h`<header class="site-header"><p class="eyebrow"><span class="badge badge-ai">AI brief</span> from ${reports.length} ${reports.length === 1 ? 'report' : 'reports'}${outlets.length ? h` (${outlets.slice(0, 5).join(', ')}${outlets.length > 5 ? ` and ${outlets.length - 5} more` : ''})` : ''}</p>
++<h1>${shown && shown.headline ? shown.headline : cluster.label}</h1>
++<p class="meta">${AI_NOTE}</p></header>
++${notice ? h`<p class="notice">${notice}</p>` : ''}
++${tabs}
++<section class="brief" aria-label="The brief">
++${shown ? h`${paragraphs(shown, reports)}${!fresh ? h`<p class="notice small">New reports arrived since this was written. ${canMake ? makeButton(cluster.id, { kind: 'summary', tone, label: 'Write it again' }) : ''}</p>` : ''}`
++ : h`<p class="empty">${canMake ? h`Not written in this tone yet. ${makeButton(cluster.id, { kind: 'summary', tone, label: 'Write it with AI' })}` : 'No brief yet.'}</p>`}
++</section>
++${outlets.length >= 2 ? h`<section class="brief-frames" aria-labelledby="frames-h"><h2 id="frames-h">How the outlets put it</h2>
++${persp ? h`<p class="meta">${AI_NOTE}</p>${paragraphs(persp, reports)}` : h`<p class="meta">How each outlet frames this, side by side, with what they agree and differ on.</p>${canMake ? makeButton(cluster.id, { kind: 'perspectives', tone: 'neutral', label: 'Compare the outlets with AI' }) : ''}`}
++</section>` : ''}
++<section aria-labelledby="reports-h"><h2 id="reports-h">The reports</h2>${sourceTable(reports)}</section>
++${shown && shown.gaps.length ? h`<details class="brief-gaps"><summary>What the AI left out (${shown.gaps.length})</summary><ul>${shown.gaps.map((g) => h`<li>${g}</li>`)}</ul></details>` : ''}
++<p class="meta"><a href="/briefs">All AI briefs</a> · <a href="/live">Live</a> · <a href="/wire">The wire</a></p>`);
++}
++
++/** /briefs */
++function briefsIndex({ briefs }) {
++ return String(h`<header class="site-header"><h1>AI briefs</h1><p class="lede">What the reports on a subject say, summarized by OpenVibe.AI with every paragraph linked to its reports, and labelled as AI. Editors' stories are on <a href="/">the front page</a>.</p></header>
++${briefs.length ? h`<ol class="story-list">${briefs.map((b) => h`<li class="story-item"><h2 class="story-item-title"><a href="/briefs/${b.cluster_id}">${b.headline || b.label}</a> <span class="badge badge-ai">AI</span></h2>
++<p class="meta">${time(b.window_end)} · from ${b.item_count} reports</p>${b.paragraphs[0] ? h`<p class="summary">${b.paragraphs[0].text.slice(0, 280)}${b.paragraphs[0].text.length > 280 ? '…' : ''}</p>` : ''}</li>`)}</ol>`
++ : h`<p class="empty">No briefs yet. They are written as reports on a subject come in from two or more of the sources.</p>`}`);
++}
++
++/** /live: the newest reports as they arrive, grouped, with what is breaking. */
++function livePage({ clusters, briefIds, breaking }) {
++ return String(h`<header class="site-header"><h1>Live <span class="live-dot" aria-hidden="true"></span></h1><p class="lede">The newest reports from the sources OpenVibe.News follows, as they arrive. This page refreshes itself every minute. <span class="badge badge-breaking">Breaking</span> marks a subject three or more reports covered in the last two hours.</p></header>
++${clusters.length ? h`<ol class="wire-list wide live-list">${clusters.map((c) => h`<li class="wire-cluster${breaking.has(c.id) ? ' is-breaking' : ''}">
++<p class="wire-lead">${breaking.has(c.id) ? h`<span class="badge badge-breaking">Breaking</span> ` : ''}<strong>${c.label}</strong>${briefIds.has(c.id) ? h` · <a href="/briefs/${c.id}">AI brief</a>` : ''}</p>
++<ul class="wire-items">${c.items.slice(0, 5).map((i) => h`<li>${httpHref(i.url) ? h`<a href="${httpHref(i.url)}" rel="nofollow noopener" target="_blank">${i.headline}</a>` : i.headline} <span class="meta">${i.outlet} · ${time(i.published_at ? Date.parse(i.published_at) : null)}</span></li>`)}</ul></li>`)}</ol>`
++ : h`<p class="empty">Nothing new from the sources yet.</p>`}`);
++}
++
++module.exports = { briefPage, briefsIndex, livePage, AI_NOTE, TONES };
ddiff --git a/server/render/layout.js b/server/render/layout.js
iindex d0e2dbc..f6b1bc7 100644
---- a/server/render/layout.js
++++ b/server/render/layout.js
@@@ -22,6 +22,8 @@ const NAV_LINKS = [
{ label: 'News', href: '/' },
{ label: 'Topics', href: '/topics' },
{ label: 'Wire', href: '/wire' },
++ { label: 'Live', href: '/live' },
++ { label: 'AI briefs', href: '/briefs' },
];
const hashes = new Map();
@@@ -99,6 +101,8 @@ function renderPage(o) {
home: '/',
css: asset('css/news.css'),
styles: o.styles, // openvibe-shared stylesheet names (the home page's showcase.css)
++ // This service's own extra stylesheets for one kind of page (the briefs and live pages' css/briefs.css).
++ head: (o.localStyles || []).map((rel) => `<link rel="stylesheet" href="${asset(rel)}">`).join('\n'),
release: RELEASE,
account,
body: o.body,
ddiff --git a/server/worker.js b/server/worker.js
iindex 2ee1496..932f240 100644
---- a/server/worker.js
++++ b/server/worker.js
@@@ -7,8 +7,15 @@
* (idempotent), the backstop for webhook deliveries that never arrived.
* A failed pull is recorded and relayed once per outage; it creates nothing.
*/
--function createWorker({ config, ingest, outbox, log = console }) {
++function createWorker({ config, ingest, outbox, briefs = null, log = console }) {
let timer = null;
++ let briefTimer = null;
++
++ /** AI briefs for the busiest recent clusters (server/domain/briefs.js); a failure waits for the next tick. */
++ async function briefTick() {
++ if (!briefs) return 0;
++ try { return await briefs.sweep(); } catch (err) { log.error('[News] brief tick failed:', err.message); return 0; }
++ }
async function pullTick() {
if (!ingest) return null;
@@@ -24,13 +31,20 @@ function createWorker({ config, ingest, outbox, log = console }) {
return {
pullTick,
++ briefTick,
start() {
-- if (!config.worker.enabled || !config.sources.pullIntervalMs) return;
++ if (!config.worker.enabled) return;
++ if (config.briefs && config.briefs.enabled && briefs) {
++ briefTimer = setInterval(briefTick, config.briefs.intervalMs);
++ briefTimer.unref();
++ setTimeout(briefTick, 30_000).unref();
++ }
++ if (!config.sources.pullIntervalMs) return;
timer = setInterval(pullTick, config.sources.pullIntervalMs);
timer.unref();
setTimeout(pullTick, 2000).unref();
},
-- stop() { clearInterval(timer); },
++ stop() { clearInterval(timer); clearInterval(briefTimer); },
};
}
ddiff --git a/test/briefs.test.js b/test/briefs.test.js
nnew file mode 100644
iindex 0000000..ec0f9e4
---- /dev/null
++++ b/test/briefs.test.js
@@@ -0,0 +1,90 @@
++'use strict';
++/**
++ * AI briefs and the live view (server/domain/briefs.js, render/briefs.js; the owner's April ask: multi-source summaries
++ * with tone and perspective controls, and a breaking-news live updater). The sweep summarizes a cluster with two or
++ * more reports; every paragraph cites its reports and uncited claims are dropped; the page labels it as AI, is never
++ * indexed, and shows the reports; a reader's POST asks for another tone or the outlets' comparison (same-origin only);
++ * a stub provider's output is never shown; a changed cluster's brief is stale and made again; /live refreshes itself
++ * and marks what is breaking.
++ */
++const assert = require('assert');
++const { boot, check, done, launchReports } = require('./helpers/boot');
++
++const run = (output, id = 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E1', synthetic = false) => ({ status: 201, run: { id, status: 'succeeded', workflow: { key: 'news.summarize_story', version: 3 }, synthetic, provenance: { model: 'test-model' }, output } });
++const SAME = { 'sec-fetch-site': 'same-origin' };
++
++(async () => {
++ const t = await boot({ ai: true });
++ try {
++ launchReports(t.sources);
++ await t.pull();
++ const clusterId = (await t.db().prepare("SELECT cluster_id, count(*) AS n FROM news_source_items WHERE status = 'active' GROUP BY cluster_id ORDER BY n DESC").get()).cluster_id;
++
++ await check('the sweep writes a cited summary for a cluster with two or more reports; uncited claims are dropped', async () => {
++ t.ai.setNext(run({ headline: 'Europa Clipper is on its way', summary: 'NASA launched Europa Clipper toward Jupiter.', key_points: [{ text: 'It reaches Jupiter in 2030.', citations: [1] }, { text: 'A claim nobody reported.', citations: [] }], timeline: [], citations: [0, 1], gaps: [] }));
++ const made = await t.ctx.briefs.sweep();
++ assert.strictEqual(made, 1);
++ const sent = t.ai.requests.pop();
++ assert.strictEqual(sent.workflow, 'news.summarize_story');
++ assert.strictEqual(sent.input.tone, 'neutral');
++ const got = await t.ctx.briefs.get(clusterId);
++ assert.ok(got.fresh);
++ assert.deepStrictEqual(got.brief.paragraphs.map((p) => p.text), ['NASA launched Europa Clipper toward Jupiter.', 'It reaches Jupiter in 2030.']);
++ assert.ok(got.brief.gaps.some((g) => /uncited/.test(g)));
++ assert.strictEqual(await t.ctx.briefs.sweep(), 0, 'a fresh brief is not made again');
++ });
++
++ await check('/briefs/:id: labelled AI, cited to the reports below it, never indexed; listed on /briefs', async () => {
++ const r = await t.get(`/briefs/${clusterId}`);
++ assert.strictEqual(r.status, 200, r.text.slice(0, 200));
++ assert.ok(r.text.includes('AI brief') && r.text.includes('Written by OpenVibe.AI'));
++ assert.ok(r.text.includes('Europa Clipper is on its way'));
++ assert.match(r.text, /href="#source-2"/);
++ assert.ok(r.text.includes('id="source-1"'), 'the reports are in the page');
++ assert.ok(r.text.includes('noindex'));
++ assert.ok(!r.text.includes('A claim nobody reported') || r.text.includes('What the AI left out'));
++ const list = await t.get('/briefs');
++ assert.ok(list.text.includes(`/briefs/${clusterId}`));
++ });
++
++ await check('a reader asks for another tone with a POST from this site; a GET never runs AI', async () => {
++ const before = t.ai.requests.length;
++ const page = await t.get(`/briefs/${clusterId}?tone=brief`);
++ assert.ok(page.text.includes('Write it with AI'));
++ assert.strictEqual(t.ai.requests.length, before, 'reading never runs AI');
++ assert.strictEqual((await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'brief' }, headers: { 'sec-fetch-site': 'cross-site' } })).status, 403);
++ t.ai.setNext(run({ headline: 'Europa Clipper launched', summary: 'NASA sent Europa Clipper to Jupiter.', key_points: [], timeline: [], citations: [0], gaps: [] }, 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E2'));
++ const made = await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'brief' }, headers: SAME });
++ assert.strictEqual(made.status, 303);
++ assert.strictEqual(t.ai.requests.pop().input.tone, 'brief');
++ assert.ok((await t.get(`/briefs/${clusterId}?tone=brief`)).text.includes('NASA sent Europa Clipper to Jupiter.'));
++ });
++
++ await check('how the outlets put it: compare_perspectives on request, cited', async () => {
++ t.ai.setNext(run({ question: 'How was the launch covered?', perspectives: [{ label: 'Wire A', summary: 'Focus on the rocket.', citations: [0] }, { label: 'Paper B', summary: 'Focus on the science.', citations: [1] }], agreements: [], disagreements: [], citations: [0, 1], gaps: [] }, 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E3'));
++ const made = await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'perspectives', tone: 'neutral' }, headers: SAME });
++ assert.strictEqual(made.status, 303);
++ assert.strictEqual(t.ai.requests.pop().workflow, 'news.compare_perspectives');
++ assert.ok((await t.get(`/briefs/${clusterId}`)).text.includes('Paper B: Focus on the science.'));
++ });
++
++ await check('stub output is never shown; a failed run makes nothing and says so', async () => {
++ t.ai.setNext(run({ headline: 'STUB', summary: 'stub text', key_points: [], timeline: [], citations: [0], gaps: [] }, 'run_01J8Z3V9Q6N1X2Y3Z4A5B6C7E4', true));
++ await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'explained' }, headers: SAME });
++ assert.ok(!(await t.get(`/briefs/${clusterId}?tone=explained`)).text.includes('stub text'));
++ t.ai.setNext(null);
++ const failed = await t.get(`/briefs/${clusterId}/make`, { method: 'POST', form: { kind: 'summary', tone: 'explained' }, headers: SAME });
++ assert.match(failed.headers.get('location'), /failed=/);
++ });
++
++ await check('/live refreshes itself and links the AI briefs; it is not indexed', async () => {
++ const r = await t.get('/live');
++ assert.strictEqual(r.status, 200);
++ assert.strictEqual(r.headers.get('refresh'), '60');
++ assert.ok(r.text.includes(`/briefs/${clusterId}`) && r.text.includes('noindex'));
++ const home = await t.get('/');
++ assert.ok(home.text.includes('href="/live"') && home.text.includes('href="/briefs"'), 'in the navigation');
++ });
++ } finally { await t.close(); }
++ done();
++})().catch((e) => { console.error(e); process.exit(1); });