openvibers/openvibe.blog Commits
Tag changelog entries: kind, areas and plain words for /updates (#254)
Changes
@@@ -3,6 +3,12 @@## Unreleased- Weekly devblogs use the content.devblog AI agent and blog.devblog.write task: meaningful merged changes, stored screenshots, rich blocks, and a person-only review gate. Owner requests and the opt-in scheduler share durable weekly reservations. Per-site and network indexes use the first figure and summary; the back-catalogue script defaults to a read-only dry run.++- The network changelog's entries carry `kind` (new, better, fixed or internal), one to three fixed `areas` and the++ subject in `plain` words. Library pins, merges and test/docs/CI-only work become internal without a model; the rest++ is tagged by OpenVibe.AI's `blog.changelog.tag` task. `GET /api/v1/changelog` filters by `kind`, `area` and `q` and++ returns facets and the vocabulary; `GET /api/v1/changelog/devblog?service=` answers a site's newest devblog post++ (title, teaser, date, hero image, URL) for the home card and /updates. `scripts/backfill-changelog-tags.js` tags++ existing entries, dry run first.- The screenshot pass, from the Blog's side (plan L9, devblog §3.3): when the patch notes are drafted, Blog reads the shotsOpenVibe.Host captured (the newest one taken before the release and the first one taken after it, per page and viewport).@@@ -226,16 +226,31 @@ draft is the only artifact, and publishing it needs a person's approval of the r`node scripts/patch-notes.js` files a draft from the pending commits (`--dry-run` prints it,`--no-ai` uses the deterministic post).--`GET /api/v1/changelog?service=&limit=&before=` is public (CORS `*`, cached for 60 s). It returns:++`GET /api/v1/changelog?service=&kind=&area=&q=&limit=&before=` is public (CORS `*`, cached for 60 s). It returns:| Field | Contents ||---|---|--| `entries` | Each entry has its site, sha, subject, author, time and commit link |++| `entries` | Each entry has its site, sha, plain words, kind, areas, author, time, commit link and pull request || `next` | The cursor for the next page |++| `facets` | The kind and area counts in the same service and `q` scope (the filter chips) |++| `vocabulary` | The four kinds and the 30 fixed areas with their display labels || `latest_post` | The newest patch notes post || `posts` | The recent patch notes posts || `sites` | Only without `service`: each site with its count and latest time |++Every entry carries `kind` (`new`, `better`, `fixed` or `internal`), one to three `areas` from the++fixed list (`server/changelog/tags.js`), and `plain` — the subject without its PR number or stage tag.++Library pins, merges and test-only, docs-only or CI-only commits are `internal` at collection without++a model; everything else is tagged by OpenVibe.AI's `blog.changelog.tag` task++(`server/changelog/tagger.js`), whose owner-chosen model answers (the cheapest good one by default;++the owner changes it on Exchange). `CHANGELOG_AI_TAG=off` leaves new entries for the backfill.++`node scripts/backfill-changelog-tags.js` tags existing entries: a dry run prints what it would do,++and `--apply` writes (a production database also needs `--production`).++++`GET /api/v1/changelog/devblog?service=live` is public too (CORS `*`, cached 5 minutes): the site's++newest devblog post as `{ service, title, teaser, date, url, hero_image_url }`, and without `service`++the newest devblog of each site under `devblogs` — the card and /updates data of plan §9.++OpenVibe.Network proxies this feed at `openvibe.network/api/v1/changelog`. Every site shows it through`openvibe-shared/shipped.js`:nnew file mode 100644@@@ -0,0 +1,18 @@++-- phase: expand++-- The network changelog's tags (plan §9, owner items 61–62): every entry collected after this carries++-- plain words (the commit subject without its PR number or stage tag), a kind (new, better, fixed or++-- internal) and one to three areas from Blog's fixed list (server/changelog/tags.js).++--++-- Entries collected before this stay NULL in kind/plain and empty in areas; scripts/backfill-changelog-++-- tags.js tags them later. Housekeeping (library pins, merges, test-only and docs-only work) is written++-- kind internal at collection without a model; everything else stays NULL until blog.changelog.tag++-- answers. Nothing here is a constraint on the old rows, so the migration only widens the table.++++ALTER TABLE changelog_entries ADD COLUMN IF NOT EXISTS kind text;++ALTER TABLE changelog_entries ADD COLUMN IF NOT EXISTS areas text[] NOT NULL DEFAULT '{}'::text[];++ALTER TABLE changelog_entries ADD COLUMN IF NOT EXISTS plain text;++ALTER TABLE changelog_entries ADD COLUMN IF NOT EXISTS tagged_at bigint;++ALTER TABLE changelog_entries ADD COLUMN IF NOT EXISTS tagged_by text;++++CREATE INDEX IF NOT EXISTS idx_changelog_kind ON changelog_entries (kind, deployed_at DESC, id DESC);++CREATE INDEX IF NOT EXISTS idx_changelog_areas ON changelog_entries USING gin (areas);@@@ -9,6 +9,7 @@"dev": "NODE_ENV=development node server/index.js","test": "node test/run.js","seed": "node scripts/seed.js",++ "backfill:changelog-tags": "node scripts/backfill-changelog-tags.js","test:pg": "BLOG_TEST_STORE=pg node test/run.js"},"keywords": [nnew file mode 100644@@@ -0,0 +1,94 @@++#!/usr/bin/env node++'use strict';++/**++ * Backfill the network changelog's tags (plan §9): existing entries with no kind get plain words,++ * a kind and one to three areas. Housekeeping (library pins, merges, test-only and docs-only work)++ * is decided here without a model; everything else goes to OpenVibe.AI's blog.changelog.tag task in++ * batches, exactly as collection does (server/changelog/tagger.js).++ *++ * node scripts/backfill-changelog-tags.js # dry run: print what it would do++ * node scripts/backfill-changelog-tags.js --service live # one site++ * node scripts/backfill-changelog-tags.js --limit 200 # at most this many rows (default 500)++ * node scripts/backfill-changelog-tags.js --apply # write, spending the AI task's calls++ * node scripts/backfill-changelog-tags.js --apply --production++ *++ * The dry run writes nothing and never calls the model. --apply needs OV_AI_INTERNAL_URL and the++ * service secret; a production database also needs --production, so it is never an accident.++ */++const tags = require('../server/changelog/tags');++const { createChangelogTagger } = require('../server/changelog/tagger');++++const args = process.argv.slice(2);++const opt = (name) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : null; };++const has = (name) => args.includes(`--${name}`);++++async function main() {++ if (has('help')) {++ console.log('usage: node scripts/backfill-changelog-tags.js [--service <id>] [--limit 500] [--chunk 20] [--apply] [--production]');++ return 0;++ }++ const config = require('../server/config').load();++ if (config.isProduction && has('apply') && !has('production')) {++ console.error('[backfill] refusing to write to a production database without --production');++ return 1;++ }++ const apply = has('apply');++ const service = opt('service');++ const limit = Math.max(1, Math.min(10000, parseInt(opt('limit'), 10) || 500));++ const chunk = Math.max(1, Math.min(25, parseInt(opt('chunk'), 10) || 20));++ const { openStore } = require('../server/db');++ const store = await openStore(config, { log: console });++ const db = store.db;++ const where = ['kind IS NULL'];++ const params = [];++ if (service) { where.push('service = ?'); params.push(service); }++ const rows = await db.prepare(`SELECT id, service, repo, sha, subject FROM changelog_entries WHERE ${where.join(' AND ')} ORDER BY deployed_at, id LIMIT ?`).all(...params, limit);++++ const housekeeping = [];++ const needModel = [];++ for (const row of rows) {++ const initial = tags.initial(row.subject);++ if (initial.kind === 'internal') housekeeping.push({ ...row, tag: initial });++ else needModel.push({ ...row, plain: initial.plain });++ }++ console.log(`[backfill] ${rows.length} untagged entr${rows.length === 1 ? 'y' : 'ies'}${service ? ` on ${service}` : ''} (limit ${limit})`);++ console.log(`[backfill] ${housekeeping.length} housekeeping → internal without a model; ${needModel.length} need blog.changelog.tag (${Math.ceil(needModel.length / chunk)} run${Math.ceil(needModel.length / chunk) === 1 ? '' : 's'})`);++ for (const row of housekeeping.slice(0, 40)) console.log(` internal ${row.service} ${row.sha.slice(0, 7)} ${row.tag.plain}`);++ if (housekeeping.length > 40) console.log(` … ${housekeeping.length - 40} more housekeeping`);++ for (const row of needModel.slice(0, 40)) console.log(` model ${row.service} ${row.sha.slice(0, 7)} ${row.plain}`);++ if (needModel.length > 40) console.log(` … ${needModel.length - 40} more for the model`);++++ if (!apply) {++ console.log('[backfill] dry run: no writes, no AI calls. Re-run with --apply to write.');++ await store.close();++ return 0;++ }++ const tagger = createChangelogTagger({ config, log: console });++ if (!tagger.enabled) {++ console.error('[backfill] blog.changelog.tag is not configured (OV_AI_INTERNAL_URL and OV_OAUTH_CLIENT_SECRET)');++ await store.close();++ return 1;++ }++ const tagStmt = db.prepare('UPDATE changelog_entries SET kind = ?, areas = ?, plain = ?, tagged_at = ?, tagged_by = ? WHERE id = ? AND kind IS NULL');++ let tagged = 0;++ for (const row of housekeeping) tagged += (await tagStmt.run('internal', [], row.tag.plain, store.now(), 'housekeeping', row.id)).changes;++ for (let i = 0; i < needModel.length; i += chunk) {++ const batch = needModel.slice(i, i + chunk);++ try {++ const answer = await tagger.tag(batch);++ for (const row of batch) {++ const t = answer.get(String(row.id));++ if (!t) continue;++ tagged += (await tagStmt.run(t.kind, t.areas, t.plain, store.now(), 'ai', row.id)).changes;++ }++ console.log(`[backfill] tagged ${Math.min(i + chunk, needModel.length)}/${needModel.length} model entries`);++ } catch (err) {++ console.error(`[backfill] batch ${i / chunk + 1} failed, left untagged: ${err.message}`);++ }++ }++ console.log(`[backfill] wrote ${tagged} of ${rows.length} entries`);++ await store.close();++ return tagged === rows.length ? 0 : 1;++}++++main().then((code) => { process.exitCode = code; }, (err) => { console.error(`[backfill] ${err.message}`); process.exitCode = 1; });@@@ -103,7 +103,10 @@ async function createApp(opts = {}) {// Patch notes v2: the story map, the template, the voice cards and the review gate. The changelog files drafts// through it; posts.publish refuses a draft with a blocking check or no person's approval.const patchNotes = opts.patchNotes || require('./patch-notes').createPatchNotes({ config, store, blogs, posts, publication, aiDrafts, media, fetchImpl, log });-- const changelog = opts.changelog || require('./changelog').createChangelog({ config, store, blogs, posts, patchNotes, fetchImpl, log });++ // The changelog's tags: housekeeping is internal without a model; the rest ask OpenVibe.AI's++ // blog.changelog.tag task (server/changelog/tagger.js), whose owner-chosen model answers.++ const changelogTagger = opts.changelogTagger || (opts.changelog ? null : require('./changelog/tagger').createChangelogTagger({ config, fetchImpl, log }));++ const changelog = opts.changelog || require('./changelog').createChangelog({ config, store, blogs, posts, reading, tagger: changelogTagger, patchNotes, fetchImpl, log });const devblog = require('./devblog').createDevblog({ config, store, blogs, posts, access, aiDrafts, media, log });if (changelog.attachDevblog) changelog.attachDevblog(devblog);if (patchNotes.attach) patchNotes.attach({ changelog });@@@ -20,11 +20,17 @@* template, with OpenVibe.AI's prose when it is configured, and a person reviews the draft before it* can be published. The draft owns its entries; the public feed links them only once the post is live.*-- * Read: GET /api/v1/changelog?service=&limit= (public, CORS *) is what every site's "recently-- * shipped" shows, with the latest patch notes post to link to.++ * Read: GET /api/v1/changelog?service=&kind=&area=&q=&limit= (public, CORS *) is what every site's++ * "recently shipped" shows, with the latest patch notes post to link to. Each entry carries its kind++ * (new, better, fixed or internal), one to three areas and plain words; internal housekeeping is++ * filterable and counted but the sites hide it behind "Show housekeeping". GET++ * /api/v1/changelog/devblog?service= answers the site's newest devblog post (title, teaser, date,++ * hero image and URL) for the home card. The fixed vocabulary is server/changelog/tags.js; the++ * blog.changelog.tag task writes the tags (server/changelog/tagger.js), housekeeping without a model.*/const crypto = require('crypto');const { serviceAuth } = require('openvibe-contracts');++const tags = require('./changelog/tags');const GITHUB = 'https://api.github.com';const clean = (s, n) => String(s == null ? '' : s).replace(/\s+/g, ' ').trim().slice(0, n);@@@ -49,26 +55,27 @@ function subjectOf(message, n = 240) {return `${cut.slice(0, Math.max(cut.lastIndexOf(' '), n - 40)).replace(/[,;:(\s]+$/, '')}…`;}const isVersionBump = (subject) => /^\d+\.\d+\.\d+\b/.test(subject);--const isNoise = (subject) => /^Merge (branch|pull request|remote)/i.test(subject);++const TAG_BATCH = 20;--function createChangelog({ config, store, blogs, posts, patchNotes = null, fetchImpl = globalThis.fetch, log = console }) {++function createChangelog({ config, store, blogs, posts, reading = null, tagger = null, patchNotes = null, fetchImpl = globalThis.fetch, log = console }) {const c = config.changelog;const db = store.db;const nowMs = () => store.now();-- const stats = { runs: 0, entries: 0, posts: 0, drafts: 0, lastError: null, lastRunAt: null };++ const stats = { runs: 0, entries: 0, posts: 0, drafts: 0, tagged: 0, lastError: null, lastRunAt: null };const q = {state: db.prepare('SELECT * FROM changelog_state WHERE service = ?'),putState: db.prepare(`INSERT INTO changelog_state (service, release, head, checked_at) VALUES (@service, @release, @head, @at)ON CONFLICT(service) DO UPDATE SET release = excluded.release, head = excluded.head, checked_at = excluded.checked_at`),-- insert: db.prepare(`INSERT INTO changelog_entries (service, repo, sha, subject, author, committed_at, deployed_at, release, release_lines, major, post_id)-- VALUES (@service, @repo, @sha, @subject, @author, @committed_at, @deployed_at, @release, @release_lines, @major, @post_id) ON CONFLICT (service, sha) DO NOTHING`),++ insert: db.prepare(`INSERT INTO changelog_entries (service, repo, sha, subject, author, committed_at, deployed_at, release, release_lines, major, post_id, kind, areas, plain, tagged_at, tagged_by)++ VALUES (@service, @repo, @sha, @subject, @author, @committed_at, @deployed_at, @release, @release_lines, @major, @post_id, @kind, @areas, @plain, @tagged_at, @tagged_by) ON CONFLICT (service, sha) DO NOTHING RETURNING id`),++ tag: db.prepare('UPDATE changelog_entries SET kind = ?, areas = ?, plain = ?, tagged_at = ?, tagged_by = ? WHERE id = ? AND kind IS NULL'),fillAuthor: db.prepare('UPDATE changelog_entries SET author = ? WHERE service = ? AND sha = ? AND author IS NULL'),history: db.prepare('SELECT * FROM changelog_history WHERE service = ?'),putHistory: db.prepare('INSERT INTO changelog_history (service, imported, imported_at) VALUES (?, ?, ?) ON CONFLICT (service) DO UPDATE SET imported = excluded.imported, imported_at = excluded.imported_at'),recentPosts: db.prepare('SELECT * FROM changelog_posts ORDER BY created_at DESC LIMIT ?'),sites: db.prepare(`SELECT service, COUNT(*) AS entries, MAX(deployed_at) AS latest_at FROM changelog_entries GROUP BY service ORDER BY latest_at DESC`),-- pending: db.prepare('SELECT * FROM changelog_entries WHERE post_id IS NULL ORDER BY deployed_at, committed_at, id'),++ pending: db.prepare("SELECT * FROM changelog_entries WHERE post_id IS NULL AND (kind IS NULL OR kind <> 'internal') ORDER BY deployed_at, committed_at, id"),mark: db.prepare('UPDATE changelog_entries SET post_id = ? WHERE id = ? AND post_id IS NULL'),ownedBy: db.prepare('SELECT * FROM changelog_entries WHERE post_id = ? ORDER BY deployed_at, committed_at, id'),move: db.prepare('UPDATE changelog_entries SET post_id = ? WHERE post_id = ?'),@@@ -140,27 +147,60 @@ function createChangelog({ config, store, blogs, posts, patchNotes = null, fetchconst authorOf = (cm) => clean((cm.commit && cm.commit.author && cm.commit.author.name) || (cm.author && cm.author.login) || '', 80) || null;++ /**++ * Tag the entries just collected: housekeeping already carries kind internal (plain words and++ * no model call); the rest go to the blog.changelog.tag task in one run. A failed run leaves++ * them untagged for the backfill and never fails collection. History is never sent (it can be++ * hundreds of commits per site): the backfill tags it.++ */++ async function tagPending(list) {++ if (!list.length || !tagger || !tagger.enabled) return 0;++ let n = 0;++ for (let i = 0; i < list.length; i += TAG_BATCH) {++ const chunk = list.slice(i, i + TAG_BATCH);++ try {++ const answer = await tagger.tag(chunk);++ for (const e of chunk) {++ const t = answer.get(String(e.id));++ if (!t) continue;++ n += (await q.tag.run(t.kind, t.areas, t.plain, nowMs(), 'ai', e.id)).changes;++ }++ } catch (err) {++ stats.lastError = `tag: ${err.message}`;++ log.warn && log.warn(`[changelog] ${list.length} entries left untagged: ${err.message}`);++ }++ }++ stats.tagged += n;++ return n;++ }++// history: the commits a site shipped before the changelog watched it, so its updates page is not// empty on day one. They carry post_id 'history' (never pending, never in a patch notes post) and// their commit time stands in for the deploy time nobody recorded.async function addEntries(svc, commits, { releaseLines = 0, major = false, history = false } = {}) {const deployedAt = svc.releasedAt || new Date(nowMs()).toISOString();let n = 0;++ const untagged = [];for (const cm of commits) {const subject = subjectOf(cm.commit && cm.commit.message);-- if (!subject || isNoise(subject)) continue;++ if (!subject) continue;++ const meta = tags.initial(subject);const flagged = major || isVersionBump(subject) || /\[major\]/i.test((cm.commit && cm.commit.message) || '');const committedAt = (cm.commit && cm.commit.author && cm.commit.author.date) || null;if (history && !committedAt) continue;const author = authorOf(cm);-- const added = (await q.insert.run({++ const row = await q.insert.get({service: svc.id, repo: svc.repo, sha: cm.sha, subject, author, committed_at: committedAt,deployed_at: history ? new Date(committedAt).toISOString() : deployedAt, release: svc.release, release_lines: releaseLines, major: flagged ? 1 : 0,-- post_id: history ? 'history' : null,-- })).changes;-- if (!added && author) await q.fillAuthor.run(author, svc.id, cm.sha);-- n += added;++ post_id: history ? 'history' : null, kind: meta.kind, areas: meta.areas, plain: meta.plain,++ tagged_at: meta.kind ? nowMs() : null, tagged_by: meta.kind ? 'housekeeping' : null,++ });++ if (!row) { if (author) await q.fillAuthor.run(author, svc.id, cm.sha); continue; }++ n += 1;++ // History is tagged by the backfill, not inside a collection run.++ if (!history && !meta.kind) untagged.push({ id: row.id, service: svc.id, repo: svc.repo, subject, plain: meta.plain });}++ await tagPending(untagged);return n;}@@@ -340,25 +380,56 @@ function createChangelog({ config, store, blogs, posts, patchNotes = null, fetch}function stop() { if (timer) clearInterval(timer); timer = null; }-- /** The public feed, newest first: { entries, next } (next = the cursor for the page after, or null). */-- async function page({ service = null, limit = 20, before = null } = {}) {++ /**++ * The public feed, newest first: { entries, next, facets, vocabulary } (next = the cursor for the++ * page after, or null). Filters: service, kind, area and q (the plain words or the subject);++ * facets count the kinds and areas in the same scope (service and q) so the chips can show them.++ */++ async function page({ service = null, limit = 20, before = null, kind = null, area = null, q: search = null } = {}) {const n = Math.min(Math.max(parseInt(limit, 10) || 20, 1), 100);const cur = decodeCursor(before);-- const where = [];-- const args = [];-- if (service) { where.push('service = ?'); args.push(String(service)); }-- if (cur) { where.push('(deployed_at < ? OR (deployed_at = ? AND id < ?))'); args.push(cur.at, cur.at, cur.id); }++ const escapeLike = (s) => String(s).replace(/[\\%_]/g, '\\$&');++ const scope = [];++ const scopeArgs = [];++ if (service) { scope.push('e.service = ?'); scopeArgs.push(String(service)); }++ if (search) {++ const pattern = `%${escapeLike(search).slice(0, 200)}%`;++ scope.push("(e.plain ILIKE ? ESCAPE '\\' OR e.subject ILIKE ? ESCAPE '\\')");++ scopeArgs.push(pattern, pattern);++ }++ const where = [...scope];++ const args = [...scopeArgs];++ if (kind) { where.push('e.kind = ?'); args.push(String(kind)); }++ if (area) { where.push('e.areas @> ARRAY[?]::text[]'); args.push(String(area)); }++ if (cur) { where.push('(e.deployed_at < ? OR (e.deployed_at = ? AND e.id < ?))'); args.push(cur.at, cur.at, cur.id); }const rows = await db.prepare(`SELECT e.*, (SELECT p.state FROM blog_posts p WHERE p.id = e.post_id) AS post_stateFROM changelog_entries e ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY e.deployed_at DESC, e.id DESC LIMIT ?`).all(...args, n + 1);const more = rows.length > n;const shown = rows.slice(0, n);++ const scoped = scope.length ? `WHERE ${scope.join(' AND ')}` : '';++ const kindCounts = Object.fromEntries(tags.KINDS.map((k) => [k, 0]));++ for (const row of await db.prepare(`SELECT e.kind, COUNT(*) AS n FROM changelog_entries e ${scoped} GROUP BY e.kind`).all(...scopeArgs)) {++ kindCounts[row.kind && tags.isKind(row.kind) ? row.kind : 'better'] += Number(row.n) || 0;++ }++ const areaCounts = {};++ for (const row of await db.prepare(`SELECT a AS area, COUNT(*) AS n FROM changelog_entries e CROSS JOIN LATERAL unnest(e.areas) AS a ${scoped} GROUP BY a`).all(...scopeArgs)) {++ if (tags.isArea(row.area)) areaCounts[row.area] = Number(row.n) || 0;++ }return {-- entries: shown.map((e) => ({-- service: e.service, sha: e.sha, short: e.sha.slice(0, 7), subject: e.subject, author: e.author || null, committed_at: e.committed_at, deployed_at: e.deployed_at,-- major: Boolean(e.major), url: `https://github.com/${e.repo}/commit/${e.sha}`,-- post_id: e.post_id && e.post_id !== 'history' && e.post_state === 'published' ? e.post_id : null,-- })),++ entries: shown.map((e) => {++ const pr = Number(e.pr_number) || tags.prOf(e.subject);++ return {++ service: e.service, repo: e.repo, sha: e.sha, short: e.sha.slice(0, 7), subject: e.subject, plain: e.plain || tags.plainOf(e.subject),++ kind: e.kind && tags.isKind(e.kind) ? e.kind : 'better', areas: Array.isArray(e.areas) ? e.areas.filter(tags.isArea).slice(0, 3) : [],++ author: e.author || null, committed_at: e.committed_at, deployed_at: e.deployed_at,++ major: Boolean(e.major), url: `https://github.com/${e.repo}/commit/${e.sha}`,++ post_id: e.post_id && e.post_id !== 'history' && e.post_state === 'published' ? e.post_id : null,++ pr: pr || null, pr_url: pr ? `https://github.com/${e.repo}/pull/${pr}` : null,++ };++ }),next: more && shown.length ? encodeCursor(shown[shown.length - 1]) : null,++ facets: { kinds: kindCounts, areas: areaCounts },++ vocabulary: tags.vocabulary(),};}async function entries(opts = {}) { return (await page(opts)).entries; }@@@ -367,7 +438,45 @@ function createChangelog({ config, store, blogs, posts, patchNotes = null, fetch/** Every site with entries: [{ service, entries, latest_at }], most recently shipped first. */async function sites() { return await q.sites.all(); }-- return { attachDevblog: value => { devblog = value; }, tick, collect, due, fileDraft, enrich, pendingEntries, markEntries, entriesOf, moveEntries, openPulls, start, stop, entries, page, latestPost, recentPosts, sites, importHistory, stats: async () => ({ enabled: c.enabled, ...stats, github_token: cachedToken.source, pending: (await q.pending.all()).length }) };++ /**++ * One site's newest devblog post, for the Rich Content §9 home card and the day's row in++ * /updates: { service, title, teaser, date, url, hero_image_url } or null. Devblog posts live++ * on the official blog, tagged with the service and categorized Devblog; the hero block's art++ * is the card image. `reading` extracts the art and the URL from the revision.++ */++ async function newestDevblog(service) {++ if (!service || !reading || !posts || !blogs) return null;++ const blog = await blogs.byHandle(c.blogHandle) || (c.blogHandle === config.official.handle ? await blogs.ensureOfficial() : null);++ if (!blog) return null;++ const tag = await store.taxonomy.bySlug('tag', String(service));++ if (!tag) return null;++ const { posts: rows } = await posts.listPublished({ blogId: blog.id, termIds: [tag.id], limit: 10 });++ const items = await reading.listItems(rows);++ const item = items.find((i) => (i.categories || []).some((x) => /^devblog$/i.test(x.name || x.slug || ''))++ || (i.tags || []).some((x) => /^devblog$/i.test(x.slug || x.name || '')));++ if (!item || !item.rev) return null;++ const when = item.post.published_at || item.post.first_published_at || null;++ return {++ service: String(service),++ title: item.rev.fields.title || null,++ teaser: item.rev.fields.summary || null,++ date: when ? new Date(when).toISOString() : null,++ url: `${String(config.baseUrl || '').replace(/\/+$/, '')}${item.url}`,++ hero_image_url: item.art ? item.art.url : null,++ };++ }++++ /** The newest devblog of each given site (the /updates page's day sections), newest first. */++ async function newestDevblogs(services = []) {++ const out = [];++ for (const service of services.slice(0, 20)) {++ const devblog = await newestDevblog(service).catch(() => null);++ if (devblog) out.push(devblog);++ }++ return out.sort((a, b) => String(b.date || '').localeCompare(String(a.date || '')));++ }++++ return { attachDevblog: value => { devblog = value; }, tick, collect, due, fileDraft, enrich, pendingEntries, markEntries, entriesOf, moveEntries, openPulls, start, stop, entries, page, latestPost, recentPosts, sites, newestDevblog, newestDevblogs, importHistory, stats: async () => ({ enabled: c.enabled, ...stats, github_token: cachedToken.source, pending: (await q.pending.all()).length }) };}/** A stable id for a batch (tests). */nnew file mode 100644@@@ -0,0 +1,114 @@++'use strict';++/**++ * The AI tagger for the network changelog (plan §9): one blog.changelog.tag run per batch of new++ * entries. The task is OpenVibe.AI's: its owner-chosen model (the cheapest good one by default)++ * answers kind, one to three areas and the plain words; Blog never names a model here.++ *++ * A service token with ai.run.create in the blog namespace starts the run, as itself (collection++ * is not a person's act). The run's answer is either the task workflow's { tags } or the general++ * ai.generate { text } holding the same JSON (the task's fallback while it names ai.generate).++ * Nothing here throws into collection: a missing or failed AI leaves the entries untagged for the++ * backfill (scripts/backfill-changelog-tags.js).++ */++const crypto = require('crypto');++const { serviceAuth } = require('openvibe-contracts');++const { normalizeTag } = require('./tags');++++const WAIT_MS = 20000;++const TIMEOUT_MS = 60000;++const POLL_MS = 1500;++++function parseLoose(text) {++ const s = String(text || '').trim().replace(/^```(?:json)?\s*|\s*```$/g, '');++ try { return JSON.parse(s); } catch {}++ const a = s.indexOf('{');++ const b = s.lastIndexOf('}');++ if (a >= 0 && b > a) { try { return JSON.parse(s.slice(a, b + 1)); } catch {} }++ return null;++}++++/** { tags } from either workflow's output. */++function tagsFrom(output) {++ if (!output || typeof output !== 'object') return [];++ if (Array.isArray(output.tags)) return output.tags;++ const parsed = typeof output.text === 'string' ? parseLoose(output.text) : null;++ return parsed && Array.isArray(parsed.tags) ? parsed.tags : [];++}++++function createChangelogTagger({ config, fetchImpl = globalThis.fetch, log = console } = {}) {++ const tokenClient = config.aiUrl && config.oauth && config.oauth.clientSecret ? serviceAuth.createTokenClient({++ tokenUrl: `${config.networkInternalUrl}/oauth/token`, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret,++ audience: 'openvibe.ai', scope: 'ai.run.create ai.run.read', fetchImpl,++ }) : null;++ const enabled = Boolean(config.changelog && config.changelog.tag !== false && config.aiUrl && tokenClient);++++ async function call(method, path, body, { retried = false } = {}) {++ const res = await fetchImpl(`${String(config.aiUrl).replace(/\/+$/, '')}${path}`, {++ method,++ headers: { Accept: 'application/json', ...(body ? { 'Content-Type': 'application/json' } : {}), ...(await tokenClient.authHeaders()) },++ body: body ? JSON.stringify(body) : undefined,++ signal: AbortSignal.timeout(WAIT_MS + 10000),++ });++ if (res.status === 401 && !retried && typeof tokenClient.invalidate === 'function') {++ tokenClient.invalidate();++ return call(method, path, body, { retried: true });++ }++ const json = await res.json().catch(() => ({}));++ return { status: res.status, ok: res.ok, body: json };++ }++++ const runOf = (body) => (body && body.run ? body.run : body);++++ async function run(body) {++ const started = await call('POST', `/api/v1/runs?wait=${WAIT_MS}`, body);++ // Older AI (before tasks and choices) refuses a task: the same workflow by name still runs.++ if (!started.ok && body.task && [404, 422].includes(started.status)++ && /task|workflow/i.test(`${started.body.code || ''} ${started.body.detail || ''}`)) {++ const { task, ...rest } = body;++ return { ...(await call('POST', `/api/v1/runs?wait=${WAIT_MS}`, { ...rest, workflow: 'blog.changelog.tag' })), workflowFallback: true };++ }++ if (!started.ok) throw Object.assign(new Error(`AI refused the run (${started.body.code || started.status})`), { status: started.status });++ let run = runOf(started.body);++ const deadline = Date.now() + TIMEOUT_MS;++ while (run && (run.status === 'queued' || run.status === 'running')) {++ if (Date.now() > deadline) throw Object.assign(new Error(`run ${run.id} did not finish`), { status: 504 });++ await new Promise((ok) => setTimeout(ok, POLL_MS));++ const got = await call('GET', `/api/v1/runs/${encodeURIComponent(run.id)}`);++ if (!got.ok) throw Object.assign(new Error(`reading run ${run.id} failed (${got.status})`), { status: got.status });++ run = runOf(got.body);++ }++ if (!run || !['succeeded', 'cached'].includes(run.status)) {++ throw Object.assign(new Error(`run ${run && run.id} ${run && run.status}`), { status: 502 });++ }++ return { output: run.output || {}, workflow: run.workflow || null, runId: run.id };++ }++++ /**++ * Tag these entries: [{ id, service, subject, repo, plain }] → Map(id → { kind, areas, plain }).++ * One run for the whole batch (at most the caller's slice). Rejects on failure so the caller++ * can log it; the entries stay untagged.++ */++ async function tag(entries) {++ if (!enabled || !entries.length) return new Map();++ const input = {++ entries: entries.slice(0, 25).map((e) => ({ id: String(e.id), service: e.service, subject: e.subject, ...(e.repo ? { repo: e.repo } : {}), ...(e.plain ? { plain: e.plain } : {}) })),++ };++ const idempotency = crypto.createHash('sha256').update(input.entries.map((e) => `${e.service}:${e.id}`).join(',')).digest('hex');++ const answer = await run({++ task: 'blog.changelog.tag', input,++ target: { service: 'blog', type: 'changelog', id: 'network' },++ idempotency_key: `changelog-tags:${idempotency}`,++ });++ const out = new Map();++ for (const t of tagsFrom(answer.output)) {++ const safe = normalizeTag(t);++ if (safe && input.entries.some((e) => e.id === String(t.id))) out.set(String(t.id), safe);++ }++ if (answer.workflowFallback) log.warn && log.warn('[changelog] AI does not know the blog.changelog.tag task yet; ran the workflow by name');++ return out;++ }++++ return { enabled, tag };++}++++module.exports = { createChangelogTagger, tagsFrom, parseLoose };nnew file mode 100644@@@ -0,0 +1,125 @@++'use strict';++/**++ * The network changelog's fixed vocabulary (plan §9): what kind of change an entry is and which++ * parts of the product it touched. One module owns both lists: the AI tagger, the backfill, the++ * API filters and the contract (blog.changelog-entry@2) all read them from here or repeat this++ * exact order.++ *++ * Housekeeping is decided without a model: library pins ("openvibe-shared v4.175.0 (was++ * v4.173.0)"), merges, and test-only, docs-only or CI-only commits are kind internal. Everything++ * else is offered to the blog.changelog.tag task, whose owner-chosen model answers kind, areas and++ * the plain words; the deterministic fallbacks here are what is stored until it answers.++ */++++const KINDS = ['new', 'better', 'fixed', 'internal'];++++/** The fixed areas, in the order the filters show them. `id` is what the API and contract carry. */++const AREAS = [++ { id: 'phone', label: 'Phone' },++ { id: 'chat', label: 'Chat' },++ { id: 'go-live', label: 'Go live' },++ { id: 'video', label: 'Video' },++ { id: 'menu', label: 'Menu' },++ { id: 'accessibility', label: 'Accessibility' },++ { id: 'speed', label: 'Speed' },++ { id: 'sign-in', label: 'Sign-in' },++ { id: 'wallet', label: 'Wallet' },++ { id: 'ai', label: 'AI' },++ { id: 'search', label: 'Search' },++ { id: 'files', label: 'Files' },++ { id: 'pictures', label: 'Pictures' },++ { id: 'games', label: 'Games' },++ { id: 'design', label: 'Design' },++ { id: 'developer', label: 'Developer' },++ { id: 'machines', label: 'Machines (Link)' },++ { id: 'money', label: 'Money' },++ { id: 'privacy', label: 'Privacy' },++ { id: 'moderation', label: 'Moderation' },++ { id: 'notifications', label: 'Notifications' },++ { id: 'feed', label: 'Feed' },++ { id: 'wiki', label: 'Wiki' },++ { id: 'blog', label: 'Blog' },++ { id: 'community', label: 'Community' },++ { id: 'meme', label: 'Meme' },++ { id: 'shop', label: 'Shop' },++ { id: 'tools', label: 'Tools' },++ { id: 'exchange', label: 'Exchange' },++ { id: 'git', label: 'Git' },++];++const AREA_IDS = AREAS.map((a) => a.id);++const AREA_LABELS = new Map(AREAS.map((a) => [a.id, a.label]));++++const clean = (s, n) => String(s == null ? '' : s).replace(/\s+/g, ' ').trim().slice(0, n).trim();++const isKind = (v) => KINDS.includes(v);++const isArea = (v) => AREA_IDS.includes(v);++++/** The PR number a subject names in "(#714)"; null when it names none. */++function prOf(subject) {++ const m = /\(#(\d+)\)\s*$/.exec(String(subject || '').trim());++ return m ? Number(m[1]) : null;++}++++/** Leading stage tags ("[A6]", "[stage 2]", "[C2]") and conventional prefixes are not the subject. */++function withoutStage(subject) {++ return String(subject || '')++ .replace(/^\s*(?:\[[^\]]{1,40}\]\s*)+/, '')++ .replace(/^\s*[A-Za-z][A-Za-z0-9._-]*\([^)]*\)!?:\s*/, '') // feat(chat): / fix(menu)!:++ .replace(/^\s*(?:feat|fix|perf|refactor|style|docs?|test|tests|ci|build|chore|deps|revert)!?:\s*/i, '')++ .replace(/^\s*\d+\.\d+\.\d+:\s*/, '') // 0.376.0: the release subject++ .replace(/\s*\(#\d+\)\s*$/, '')++ .replace(/\s+/g, ' ')++ .trim();++}++++/** A subject in plain words: no PR number, no stage tag, no release-version prefix. */++function plainOf(subject, n = 240) {++ let s = withoutStage(subject);++ s = s.replace(/\s*\(#\d+\)\s*/g, ' ').replace(/\s*\[[^\]]{1,40}\]\s*/g, ' ').replace(/\s+/g, ' ').trim();++ return clean(s, n) || clean(subject, n) || 'A change';++}++++/** Library pins, merges and test/docs/CI-only work: internal without asking a model. */++function isHousekeeping(subject) {++ const s = String(subject || '').replace(/^\s*(?:\[[^\]]{1,40}\]\s*)+/, '').trim();++ return /^Merge (?:branch|pull request|remote|tag)\b/i.test(s)++ || /^Revert "?Merge\b/i.test(s)++ || /^openvibe-[a-z0-9-]+\s+v?\d+\.\d+\.\d+/i.test(s)++ || /^(?:Bump|Pin|Repin)\s+/i.test(s)++ || /^(?:Bump|Pin|Repin)\b.*\bopenvibe-/i.test(s)++ || /^\d+\.\d+\.\d+[:\s]/.test(s)++ || /^(?:test|tests|docs?|ci|build|chore|deps|dependencies|release|lint|format)(?:\([^)]*\))?!?:\s/i.test(s)++ || /^(?:Update|Refresh|Tidy|Normalise|Normalize)\s+(?:the\s+)?(?:deps|dependencies|lockfile|package-lock|docs?|tests?|changelog)\b/i.test(s)++ || /^(?:Add|Update|Fix)\s+(?:a\s+)?(?:test|tests|docs?)\s+(?:only|for)\b/i.test(s);++}++++/**++ * What is stored when an entry is first collected: plain words always, and kind internal for++ * housekeeping. `kind: null` means the AI task has not answered yet.++ */++function initial(subject) {++ return {++ plain: plainOf(subject),++ kind: isHousekeeping(subject) ? 'internal' : null,++ areas: isHousekeeping(subject) ? [] : [],++ };++}++++/** A model's answer, made safe: a valid kind, 1–3 known areas (none for internal) and plain words. */++function normalizeTag(value) {++ if (!value || typeof value !== 'object') return null;++ const kind = isKind(value.kind) ? value.kind : null;++ if (!kind) return null;++ let areas = Array.isArray(value.areas) ? value.areas.filter(isArea) : [];++ areas = [...new Set(areas)].slice(0, 3);++ if (kind !== 'internal' && !areas.length) return null;++ const plain = plainOf(value.plain, 240);++ if (!plain) return null;++ return { kind, areas, plain };++}++++/** The vocabulary as the API shows it (ids and their display labels). */++function vocabulary() {++ return { kinds: [...KINDS], areas: AREAS.map((a) => ({ ...a })) };++}++++module.exports = { KINDS, AREAS, AREA_IDS, AREA_LABELS, vocabulary, isKind, isArea, prOf, plainOf, isHousekeeping, initial, normalizeTag };@@@ -59,6 +59,9 @@ function load(env = process.env) {history: Math.max(0, Math.min(100, parseInt(env.CHANGELOG_HISTORY ?? '60', 10) || 0)),firstDelayMs: 60 * 1000,since: env.CHANGELOG_SINCE && !Number.isNaN(Date.parse(env.CHANGELOG_SINCE)) ? new Date(env.CHANGELOG_SINCE).toISOString() : null,++ // blog.changelog.tag: on when OV_AI_INTERNAL_URL and the service secret are set; off leaves++ // entries untagged for the backfill (the deterministic plain words are still stored).++ tag: env.CHANGELOG_AI_TAG !== 'off',},// Patch notes v2 (server/patch-notes/): how many workstreams get an H2, and how many published posts the AI// prompt gets as voice samples. The draft is always a draft; CHANGELOG_AI_DRAFT=off skips the AI prose and@@@ -37,6 +37,7 @@ const { run, jsonBody, ApiError, privateNoStore } = require('./errors');const { guard } = require('../auth/viewer');const { checkCapability, CAPABILITIES } = require('../auth/capabilities');const blockRender = require('../render/blocks');++const tags = require('../changelog/tags');function cors(origins) {const allowed = new Set(origins);@@@ -210,21 +211,29 @@ function createApi(ctx) {}));// The network changelog (server/changelog.js): what shipped on every OpenVibe site, newest first, and-- // the latest patch notes post. Public: every site's "recently shipped" reads it.++ // the latest patch notes post. Public: every site's "recently shipped" reads it. Filters: kind, area++ // (from the fixed vocabulary) and q (plain words or subject). /changelog/devblog answers the site's++ // newest devblog post for the home card and /updates.router.get('/changelog', run(async (req, res) => {res.set('Access-Control-Allow-Origin', '*');res.set('Cache-Control', 'public, max-age=60');const service = /^[a-z][a-z0-9-]{1,39}$/.test(String(req.query.service || '')) ? String(req.query.service) : null;++ const kind = req.query.kind == null || req.query.kind === '' ? null : String(req.query.kind);++ const area = req.query.area == null || req.query.area === '' ? null : String(req.query.area);++ if (kind && !tags.isKind(kind)) throw new ApiError(400, 'changelog.kind_invalid', `kind is one of ${tags.KINDS.join(', ')}`);++ if (area && !tags.isArea(area)) throw new ApiError(400, 'changelog.area_invalid', `area is one of ${tags.AREA_IDS.join(', ')}`);++ const q = req.query.q == null ? null : String(req.query.q).trim().slice(0, 200) || null;const postOf = async (row) => {const p = row && await posts.get(row.post_id);const blog = p ? await blogs.get(p.blog_id) : null;if (!p || !blog || p.state !== 'published') return null;return { id: p.id, title: (await posts.head(p) || { fields: {} }).fields.title || null, url: publication.abs(publication.postPath(blog, p)), published_at: p.published_at || null, entries: row.entries };};-- const page = await ctx.changelog.page({ service, limit: req.query.limit, before: req.query.before });++ const page = await ctx.changelog.page({ service, limit: req.query.limit, before: req.query.before, kind, area, q });const recent = (await Promise.all((await ctx.changelog.recentPosts(5)).map(postOf))).filter(Boolean);return {-- service, entries: page.entries, next: page.next, latest_post: recent[0] || null, posts: recent,++ service, entries: page.entries, next: page.next, facets: page.facets, vocabulary: page.vocabulary,++ latest_post: recent[0] || null, posts: recent,...(service ? {} : { sites: await ctx.changelog.sites() }),};}));@@@ -236,6 +245,19 @@ function createApi(ctx) {return { post: await postDto(result.post, { full: true }), created: result.created };}, 201));++ // The site's newest devblog post (Rich Content §9): title, teaser, date, hero image and URL.++ // With service it is that site's card; without one it lists the newest devblog of each site with++ // a devblog, for /updates' day sections. Public and cached like the changelog.++ router.get('/changelog/devblog', run(async (req, res) => {++ res.set('Access-Control-Allow-Origin', '*');++ res.set('Cache-Control', 'public, max-age=300');++ const service = req.query.service == null || req.query.service === '' ? null : String(req.query.service);++ if (service && !/^[a-z][a-z0-9-]{1,39}$/.test(service)) throw new ApiError(400, 'changelog.service_invalid', 'service is a site id (live, chat, blog, …)');++ if (service) return { service, devblog: await ctx.changelog.newestDevblog(service) };++ const services = (await ctx.changelog.sites()).slice(0, 20).map((s) => s.service);++ return { devblogs: await ctx.changelog.newestDevblogs(services) };++ }));++// Draft with AI: OpenVibe.AI's blog.draft_post writes a draft (AI-authored, noindex until reviewed).router.post('/blogs/:handle/posts/ai-draft', guard('blog.post.create'), B('blog.post.ai_draft'), jsonBody, run(async (req) => {const blog = await mustBlog(req);@@@ -118,10 +118,15 @@ route('get', '/blogs/{handle}/posts', {route('get', '/changelog', {operationId: 'getChangelog', summary: 'What shipped on every OpenVibe site, newest first', tags: ['changelog'], security: ANONYMOUS, parameters: [query('service', str({ pattern: '^[a-z][a-z0-9-]{1,39}$' }), 'Only this service\'s entries (default: every site)'),++ query('kind', { type: 'string', enum: ['new', 'better', 'fixed', 'internal'] }, 'Only this kind of change'),++ query('area', str({ pattern: '^[a-z][a-z0-9-]{0,39}$' }), 'Only changes touching this area (the response vocabulary lists them)'),++ query('q', str({ maxLength: 200 }), 'The plain words or the commit subject'),query('limit', { type: 'integer', minimum: 1 }), query('before', str(), 'The cursor from the previous page')],-- description: 'Public and CORS *, cached 60 seconds: each site\'s "recently shipped" reads it.',-- responses: { 200: json('Entries, the cursor for the next page and the latest patch notes post', { type: 'object', properties: {++ description: 'Public and CORS *, cached 60 seconds: each site\'s "recently shipped" reads it. Entries are blog.changelog-entry@2: plain words, kind, areas, time, commit and pull request. facets counts the kinds and areas in the same service and q scope; internal housekeeping is filterable with kind=internal.',++ responses: { 200: json('Entries, the cursor for the next page, the facets, the vocabulary and the latest patch notes post', { type: 'object', properties: {service: nullable('string'), entries: arrayOf({ type: 'object' }), next: nullable('string'),++ facets: { type: 'object', properties: { kinds: { type: 'object' }, areas: { type: 'object' } } },++ vocabulary: { type: 'object', properties: { kinds: arrayOf(str()), areas: arrayOf({ type: 'object', properties: { id: str(), label: str() } }) } },latest_post: nullable('object'), posts: arrayOf({ type: 'object' }), sites: arrayOf({ type: 'object' }),} }), ...errors([400, 429]) },});@@@ -136,6 +141,15 @@ route('post', '/blogs/{handle}/posts/devblog', {requestBody: body({ type: 'object', required: ['service'], properties: { service: str({ pattern: '^[a-z][a-z0-9-]{0,63}$' }) } }),responses: { 201: json('The draft', { type: 'object', properties: { post: ref('Post'), created: { type: 'boolean' } } }), 409: problem, 422: problem, 503: problem, ...errors() },});++++route('get', '/changelog/devblog', {++ operationId: 'getChangelogDevblog', summary: 'The newest devblog post of a site (or of each site)', tags: ['changelog'], security: ANONYMOUS, parameters: [++ query('service', str({ pattern: '^[a-z][a-z0-9-]{1,39}$' }), 'That site\'s newest devblog post; without it, the newest of each site')],++ description: 'Public and CORS *: the Rich Content §9 home card and /updates day section. A devblog post is on the official blog, tagged with the service and categorized Devblog; the hero block\'s art is hero_image_url.',++ responses: { 200: json('{ devblog } for one service, { devblogs } for all', { type: 'object', properties: {++ service: nullable('string'), devblog: nullable('object'), devblogs: arrayOf({ type: 'object' }),++ } }), ...errors([400, 429]) },++});route('post', '/blogs/{handle}/posts/ai-draft', {operationId: 'draftPostWithAi', summary: 'Have OpenVibe.AI write a draft', tags: ['posts'], security: PERSON, parameters: [HANDLE],description: 'blog.post.create. The draft is AI-authored and cannot publish until a person reviews it; 503 when AI drafting is not configured here.',nnew file mode 100644@@@ -0,0 +1,120 @@++'use strict';++/**++ * The changelog vocabulary (plan §9): the fixed kinds and areas, the deterministic plain words,++ * the housekeeping rule that never asks a model, and the tagger's answer parser. The feed's filters++ * and the blog.changelog-entry@2 contract repeat this exact list, so the count matters.++ */++const assert = require('assert');++const { execFileSync } = require('child_process');++const path = require('path');++const { check, done } = require('./helpers/boot');++const tags = require('../server/changelog/tags');++const { tagsFrom, parseLoose, createChangelogTagger } = require('../server/changelog/tagger');++++(async () => {++ await check('the vocabulary is 30 fixed areas and four kinds, with no duplicates', async () => {++ assert.deepStrictEqual(tags.KINDS, ['new', 'better', 'fixed', 'internal']);++ assert.strictEqual(tags.AREAS.length, 30);++ assert.strictEqual(new Set(tags.AREA_IDS).size, 30);++ assert.strictEqual(tags.AREAS.filter((a) => a.id === 'go-live')[0].label, 'Go live');++ assert.strictEqual(tags.AREAS.filter((a) => a.id === 'machines')[0].label, 'Machines (Link)');++ assert.ok(tags.AREAS.every((a) => /^[a-z][a-z0-9-]*$/.test(a.id)));++ const v = tags.vocabulary();++ assert.deepStrictEqual(v.kinds, tags.KINDS);++ assert.strictEqual(v.areas.length, 30);++ });++++ await check('plain words drop the PR number and stage tag, and keep the subject', async () => {++ assert.strictEqual(tags.plainOf('Recent changes card: show the newest devblog with its art (#714)'), 'Recent changes card: show the newest devblog with its art');++ assert.strictEqual(tags.plainOf('[A6] Add the changelog tags (#61)'), 'Add the changelog tags');++ assert.strictEqual(tags.plainOf('feat(chat): show the newest devblog'), 'show the newest devblog');++ assert.strictEqual(tags.plainOf('0.376.0: Complete the Feed contract'), 'Complete the Feed contract');++ assert.strictEqual(tags.plainOf(''), 'A change');++ assert.strictEqual(tags.prOf('A change (#714)'), 714);++ assert.strictEqual(tags.prOf('A change'), null);++ });++++ await check('housekeeping is internal without a model: pins, merges, test/docs/CI-only work', async () => {++ for (const s of [++ 'openvibe-shared v4.175.0 (was v4.173.0) (#401)',++ 'Pin openvibe-contracts 0.379.0',++ 'Merge branch main',++ 'Merge pull request #12 from x/y',++ 'test: cover the filters',++ 'docs(readme): the new call',++ 'ci: rerun',++ 'chore: deps',++ '0.376.0: Complete the Feed contract',++ 'Tidy the docs',++ ]) assert.strictEqual(tags.isHousekeeping(s), true, s);++ for (const s of ['Fix the chat scroll', 'Recent tools in the launcher', 'Add devblog cards', 'Make sign-in faster']) {++ assert.strictEqual(tags.isHousekeeping(s), false, s);++ const initial = tags.initial(s);++ assert.strictEqual(initial.kind, null, `${s} waits for the model`);++ assert.ok(initial.plain.length > 0);++ }++ const hk = tags.initial('openvibe-shared v4.175.0 (was v4.173.0) (#401)');++ assert.strictEqual(hk.kind, 'internal');++ assert.deepStrictEqual(hk.areas, []);++ assert.strictEqual(hk.plain, 'openvibe-shared v4.175.0 (was v4.173.0)');++ });++++ await check('a model answer is made safe: a known kind, up to three known areas, plain words', async () => {++ assert.deepStrictEqual(tags.normalizeTag({ kind: 'new', areas: ['video', 'chat'], plain: 'Watch the clip' }), { kind: 'new', areas: ['video', 'chat'], plain: 'Watch the clip' });++ assert.deepStrictEqual(tags.normalizeTag({ kind: 'fixed', areas: ['chat', 'chat', 'podcast'], plain: 'Fix the chat (#2)' }), { kind: 'fixed', areas: ['chat'], plain: 'Fix the chat' });++ assert.deepStrictEqual(tags.normalizeTag({ kind: 'internal', areas: ['chat'], plain: 'Pin a library' }), { kind: 'internal', areas: ['chat'], plain: 'Pin a library' });++ assert.strictEqual(tags.normalizeTag({ kind: 'improvement', areas: ['chat'], plain: 'x' }), null);++ assert.strictEqual(tags.normalizeTag({ kind: 'new', areas: [], plain: 'x' }), null, 'a non-internal change has an area');++ assert.strictEqual(tags.normalizeTag({ kind: 'new', areas: ['podcast'], plain: 'x' }), null, 'unknown areas are dropped, leaving none');++ assert.strictEqual(tags.normalizeTag(null), null);++ });++++ await check('a tagger answer is read from { tags } or from ai.generate\'s JSON text', async () => {++ const tagsList = [{ id: '1', kind: 'new', areas: ['video'], plain: 'Watch the clip' }];++ assert.deepStrictEqual(tagsFrom({ tags: tagsList }), tagsList);++ assert.deepStrictEqual(tagsFrom({ text: JSON.stringify({ tags: tagsList }) }), tagsList);++ assert.deepStrictEqual(tagsFrom({ text: '```json\n{"tags":[]}\n```' }), []);++ assert.deepStrictEqual(parseLoose('not json'), null);++ assert.deepStrictEqual(tagsFrom(null), []);++ });++++ await check('the tagger is on by default and CHANGELOG_AI_TAG=off turns it off', async () => {++ const config = require('../server/config');++ assert.strictEqual(config.load({}).changelog.tag, true);++ assert.strictEqual(config.load({ CHANGELOG_AI_TAG: 'off' }).changelog.tag, false);++ });++++ await check('the tagger asks AI for the blog.changelog.tag task, as a service, and reads its answer', async () => {++ const seen = { token: null, run: null };++ const json = (body, status = 200) => ({ ok: status < 300, status, json: async () => body });++ const fetchImpl = async (url, opts = {}) => {++ if (String(url).includes('/oauth/token')) { seen.token = String(opts.body); return await json({ access_token: 'svc', token_type: 'Bearer', expires_in: 300 }); }++ if (String(url).includes('/api/v1/runs')) { seen.run = { url: String(url), body: JSON.parse(opts.body), auth: opts.headers.Authorization }; return await json({ run: { id: 'run_1', status: 'succeeded', workflow: { key: 'blog.changelog.tag' }, output: { tags: [++ { id: '7', kind: 'new', areas: ['chat', 'video'], plain: 'A calmer chat' },++ { id: '8', kind: 'improvement', areas: ['chat'], plain: 'dropped: unknown kind' },++ ] } } }, 201); }++ return await json({}, 404);++ };++ const tagger = createChangelogTagger({ config: {++ aiUrl: 'http://ai.test', networkInternalUrl: 'http://network.test', oauth: { clientId: 'blog', clientSecret: 'shh' }, changelog: { tag: true },++ }, fetchImpl });++ assert.strictEqual(tagger.enabled, true);++ const answer = await tagger.tag([{ id: 7, service: 'live', subject: 'Add a calmer chat (#9)', repo: 'OpenVibers/OpenVibe.Live' }]);++ assert.strictEqual(seen.run.body.task, 'blog.changelog.tag');++ assert.strictEqual(seen.run.body.target.service, 'blog');++ assert.deepStrictEqual(seen.run.body.input.entries, [{ id: '7', service: 'live', subject: 'Add a calmer chat (#9)', repo: 'OpenVibers/OpenVibe.Live' }]);++ assert.match(seen.token, /audience=openvibe\.ai/);++ assert.strictEqual(seen.run.auth, 'Bearer svc');++ assert.deepStrictEqual(answer.get('7'), { kind: 'new', areas: ['chat', 'video'], plain: 'A calmer chat' });++ assert.strictEqual(answer.has('8'), false, 'an unknown kind is not written');++ assert.strictEqual((await createChangelogTagger({ config: { aiUrl: '', networkInternalUrl: '', oauth: {}, changelog: { tag: true } }, fetchImpl }).tag([{ id: 1 }])).size, 0, 'unconfigured is off');++ });++++ await check('the backfill script is safe by default: it prints what it would do and writes nothing', async () => {++ const out = execFileSync(process.execPath, [path.join(__dirname, '..', 'scripts', 'backfill-changelog-tags.js'), '--help'], { encoding: 'utf8' });++ assert.match(out, /--apply/);++ assert.match(out, /--production/);++ });++++ done();++})();@@@ -38,7 +38,13 @@ const H = (n) => n.toString(16).padStart(40, 'a');// Patch notes v2: the changelog files a DRAFT through the patch-notes module and never publishes.const drafts = [];const patchNotes = { draftPending: async ({ entries, reason }) => { drafts.push({ entries, reason }); return { created: true, post: { id: `pst_draft${drafts.length}`, blog_id: 'blg_test' }, checks: { blocking: [], warnings: [] } }; } };-- const cl = createChangelog({ config, store: t.ctx.store, blogs: t.ctx.blogs, posts: t.ctx.posts, aiDrafts: null, patchNotes, fetchImpl, log: { log() {}, warn() {} } });++ // The model half is faked: housekeeping must never reach it, and every other entry gets an answer.++ const tagCalls = [];++ const tagger = { enabled: true, tag: async (entries) => {++ tagCalls.push(entries);++ return new Map(entries.map((e) => [String(e.id), { kind: /^Fix/.test(e.subject) ? 'fixed' : 'better', areas: ['chat'], plain: `plain: ${e.plain}` }]));++ } };++ const cl = createChangelog({ config, store: t.ctx.store, blogs: t.ctx.blogs, posts: t.ctx.posts, reading: t.ctx.reading, tagger, aiDrafts: null, patchNotes, fetchImpl, log: { log() {}, warn() {} } });const commit = (i, msg) => ({ sha: H(i), commit: { message: msg, author: { date: new Date(t.clock.now()).toISOString() } } });await check('a service seen for the first time only records its release', async () => {@@@ -53,9 +59,16 @@ const H = (n) => n.toString(16).padStart(40, 'a');const r = await cl.tick();assert.strictEqual(r, null, 'not due yet');const e = await cl.entries({ service: 'live' });-- assert.deepStrictEqual(e.map((x) => x.subject), ['Fix the chat scroll'], 'first lines only; merges skipped');-- assert.strictEqual(e[0].url, `https://github.com/OpenVibers/OpenVibe.Live/commit/${H(1)}`);-- assert.strictEqual(e[0].major, false);++ const bySubject = new Map(e.map((x) => [x.subject, x]));++ assert.strictEqual(bySubject.size, 2, 'first lines only');++ assert.strictEqual(bySubject.get('Fix the chat scroll').kind, 'fixed');++ assert.deepStrictEqual(bySubject.get('Fix the chat scroll').areas, ['chat']);++ assert.strictEqual(bySubject.get('Fix the chat scroll').plain, 'plain: Fix the chat scroll');++ assert.strictEqual(bySubject.get('Merge branch main').kind, 'internal', 'merges are housekeeping');++ assert.deepStrictEqual(bySubject.get('Merge branch main').areas, []);++ assert.ok(!tagCalls.flat().some((x) => /^Merge/.test(x.subject)), 'the model is never asked about housekeeping');++ assert.strictEqual(bySubject.get('Fix the chat scroll').url, `https://github.com/OpenVibers/OpenVibe.Live/commit/${H(1)}`);++ assert.strictEqual(bySubject.get('Fix the chat scroll').major, false);});await check('a major release after a quiet spell files a Patch notes draft, never a published post', async () => {@@@ -67,7 +80,7 @@ const H = (n) => n.toString(16).padStart(40, 'a');assert.ok(r && r.post, 'drafted');assert.strictEqual(drafts.length, 1);assert.strictEqual(drafts[0].reason, 'major');-- assert.deepStrictEqual(drafts[0].entries.map((e) => e.subject).sort(), ['Fix the chat scroll', 'Recent tools in the launcher: anonymous and account history', 'Tidy the docs']);++ assert.deepStrictEqual(drafts[0].entries.map((e) => e.subject).sort(), ['Fix the chat scroll', 'Recent tools in the launcher: anonymous and account history'], 'housekeeping (the merge and docs-only) stays out of the draft');assert.strictEqual(r.post.state, undefined, 'a draft object from the fake; the real module creates a draft post');assert.strictEqual((await cl.entries()).filter((x) => x.post_id).length, 0, 'nothing is published, so the feed has no post link');});@@@ -82,6 +95,50 @@ const H = (n) => n.toString(16).padStart(40, 'a');assert.strictEqual(j.latest_post, null, 'nothing is published from the pipeline');});++ await check('the feed filters by kind, area and q, counts the facets and names the vocabulary', async () => {++ const one = async (q) => (await t.get(`/api/v1/changelog?service=live&${q}`)).json();++ assert.deepStrictEqual((await one('kind=fixed')).entries.map((x) => x.subject), ['Fix the chat scroll']);++ assert.deepStrictEqual((await one('area=chat')).entries.map((x) => x.subject), ['Fix the chat scroll'], 'the merge has no area');++ assert.deepStrictEqual((await one('kind=internal')).entries.map((x) => x.subject), ['Merge branch main']);++ assert.deepStrictEqual((await one('q=scroll')).entries.map((x) => x.subject), ['Fix the chat scroll']);++ const all = await one('limit=20');++ assert.strictEqual(all.facets.kinds.fixed, 1);++ assert.strictEqual(all.facets.kinds.internal, 1);++ assert.strictEqual(all.facets.areas.chat, 1);++ assert.strictEqual(all.vocabulary.areas.length, 30);++ assert.ok(all.vocabulary.areas.some((a) => a.id === 'go-live' && a.label === 'Go live'));++ const badKind = await t.get('/api/v1/changelog?service=live&kind=improvement');++ assert.strictEqual(badKind.status, 400, badKind.text);++ assert.strictEqual(badKind.json().code, 'changelog.kind_invalid');++ const badArea = await t.get('/api/v1/changelog?service=live&area=podcast');++ assert.strictEqual(badArea.status, 400, badArea.text);++ assert.strictEqual(badArea.json().code, 'changelog.area_invalid');++ });++++ await check('the newest devblog post of a site is an API call: title, teaser, date, hero image, url', async () => {++ const MED = 'med_01J8ZQ4X3M000000000000000A';++ t.media.objects.set(MED, 'ready');++ const made = await t.get('/api/v1/blogs/openvibe/posts', { as: t.official, json: {++ title: 'Live this week', summary: 'One encoder, a calmer chat and the new recent-changes card.', categories: ['Devblog'], tags: ['devblog', 'live'],++ body: `:::hero{art="${MED}" kicker="Devblog"}\n# Live this week\nWhat shipped.\n:::\n\n` + 'The details. '.repeat(30),++ } });++ assert.strictEqual(made.status, 201, made.text);++ assert.strictEqual((await t.get(`/api/v1/posts/${made.json().post.id}/publish`, { as: t.official, json: {} })).status, 200);++ const r = await t.get('/api/v1/changelog/devblog?service=live');++ assert.strictEqual(r.status, 200, r.text);++ assert.strictEqual(r.headers.get('access-control-allow-origin'), '*');++ const d = r.json().devblog;++ assert.strictEqual(d.title, 'Live this week');++ assert.match(d.teaser, /calmer chat/);++ assert.match(d.hero_image_url, new RegExp(MED));++ assert.match(d.url, /\/@openvibe\/live-this-week$/);++ assert.ok(!Number.isNaN(Date.parse(d.date)));++ const all = (await t.get('/api/v1/changelog/devblog')).json();++ assert.ok(all.devblogs.some((x) => x.service === 'live' && x.title === 'Live this week'));++ assert.strictEqual((await t.get('/api/v1/changelog/devblog?service=nope')).json().devblog, null);++ assert.strictEqual((await t.get('/api/v1/changelog/devblog?service=NOT OK')).status, 400);++ });++await check('a batch publishes without a major, but never sooner than the gap after the last post', async () => {releases.live = 'aaaaaaa00003';compares.aaaaaaa00003 = { status: 'ahead', commits: [5, 6, 7, 8, 9].map((i) => commit(i, `Small change ${i}`)), files: [{ additions: 5, deletions: 5 }] };@@@ -62,6 +62,7 @@ const LONG = Array.from({ length: 100 }, (_, i) => `w${i}`).join(' ');// openvibe-sdk/events and are no longer imported; they await deletion (the sandbox denies rm).'server/auth/sso.js': 'Network JWKS, OAuth token and revoke (configured); dead after the SDK move','server/changelog.js': 'Network registry (configured) and api.github.com (fixed host) for the repositories the registry names',++ 'server/changelog/tagger.js': 'OpenVibe.AI\'s run endpoint (OV_AI_INTERNAL_URL, configured) for the blog.changelog.tag task','server/clients/community.js': 'OpenVibe.Community (configured)','server/desks/desks.js': 'OpenVibe.Food\'s recipe library feed (OV_FOOD_URL, configured; the budget-recipes desk)','server/domain/ai-drafts.js': 'Publishing AI client calls only configured OV_AI_INTERNAL_URL; the devblog adapter adds a task field, never a caller URL',
@@ -3,6 +3,12 @@
@@ -3,6 +3,12 @@
## Unreleased
## Unreleased
- Weekly devblogs use the content.devblog AI agent and blog.devblog.write task: meaningful merged changes, stored screenshots, rich blocks, and a person-only review gate. Owner requests and the opt-in scheduler share durable weekly reservations. Per-site and network indexes use the first figure and summary; the back-catalogue script defaults to a read-only dry run.
- Weekly devblogs use the content.devblog AI agent and blog.devblog.write task: meaningful merged changes, stored screenshots, rich blocks, and a person-only review gate. Owner requests and the opt-in scheduler share durable weekly reservations. Per-site and network indexes use the first figure and summary; the back-catalogue script defaults to a read-only dry run.
+- The network changelog's entries carry `kind` (new, better, fixed or internal), one to three fixed `areas` and the
+ subject in `plain` words. Library pins, merges and test/docs/CI-only work become internal without a model; the rest
+ is tagged by OpenVibe.AI's `blog.changelog.tag` task. `GET /api/v1/changelog` filters by `kind`, `area` and `q` and
+ returns facets and the vocabulary; `GET /api/v1/changelog/devblog?service=` answers a site's newest devblog post
+ (title, teaser, date, hero image, URL) for the home card and /updates. `scripts/backfill-changelog-tags.js` tags
+ existing entries, dry run first.
- The screenshot pass, from the Blog's side (plan L9, devblog §3.3): when the patch notes are drafted, Blog reads the shots
- The screenshot pass, from the Blog's side (plan L9, devblog §3.3): when the patch notes are drafted, Blog reads the shots
OpenVibe.Host captured (the newest one taken before the release and the first one taken after it, per page and viewport).
OpenVibe.Host captured (the newest one taken before the release and the first one taken after it, per page and viewport).
@@ -226,16 +226,31 @@ draft is the only artifact, and publishing it needs a person's approval of the r
@@ -226,16 +226,31 @@ draft is the only artifact, and publishing it needs a person's approval of the r
`node scripts/patch-notes.js` files a draft from the pending commits (`--dry-run` prints it,
`node scripts/patch-notes.js` files a draft from the pending commits (`--dry-run` prints it,
`--no-ai` uses the deterministic post).
`--no-ai` uses the deterministic post).
-`GET /api/v1/changelog?service=&limit=&before=` is public (CORS `*`, cached for 60 s). It returns:
+`GET /api/v1/changelog?service=&kind=&area=&q=&limit=&before=` is public (CORS `*`, cached for 60 s). It returns:
| Field | Contents |
| Field | Contents |
|---|---|
|---|---|
-| `entries` | Each entry has its site, sha, subject, author, time and commit link |
+| `entries` | Each entry has its site, sha, plain words, kind, areas, author, time, commit link and pull request |
| `next` | The cursor for the next page |
| `next` | The cursor for the next page |
+| `facets` | The kind and area counts in the same service and `q` scope (the filter chips) |
+| `vocabulary` | The four kinds and the 30 fixed areas with their display labels |
| `latest_post` | The newest patch notes post |
| `latest_post` | The newest patch notes post |
| `posts` | The recent patch notes posts |
| `posts` | The recent patch notes posts |
| `sites` | Only without `service`: each site with its count and latest time |
| `sites` | Only without `service`: each site with its count and latest time |
+Every entry carries `kind` (`new`, `better`, `fixed` or `internal`), one to three `areas` from the
+fixed list (`server/changelog/tags.js`), and `plain` — the subject without its PR number or stage tag.
+Library pins, merges and test-only, docs-only or CI-only commits are `internal` at collection without
+a model; everything else is tagged by OpenVibe.AI's `blog.changelog.tag` task
+(`server/changelog/tagger.js`), whose owner-chosen model answers (the cheapest good one by default;
+the owner changes it on Exchange). `CHANGELOG_AI_TAG=off` leaves new entries for the backfill.
+`node scripts/backfill-changelog-tags.js` tags existing entries: a dry run prints what it would do,
+and `--apply` writes (a production database also needs `--production`).
+
+`GET /api/v1/changelog/devblog?service=live` is public too (CORS `*`, cached 5 minutes): the site's
+newest devblog post as `{ service, title, teaser, date, url, hero_image_url }`, and without `service`
+the newest devblog of each site under `devblogs` — the card and /updates data of plan §9.
+
OpenVibe.Network proxies this feed at `openvibe.network/api/v1/changelog`. Every site shows it through
OpenVibe.Network proxies this feed at `openvibe.network/api/v1/changelog`. Every site shows it through
`openvibe-shared/shipped.js`:
`openvibe-shared/shipped.js`:
new file mode 100644
new file mode 100644
@@ -0,0 +1,18 @@
@@ -0,0 +1,18 @@
+-- phase: expand
+-- The network changelog's tags (plan §9, owner items 61–62): every entry collected after this carries
+-- plain words (the commit subject without its PR number or stage tag), a kind (new, better, fixed or
+-- internal) and one to three areas from Blog's fixed list (server/changelog/tags.js).
+--
+-- Entries collected before this stay NULL in kind/plain and empty in areas; scripts/backfill-changelog-
+-- tags.js tags them later. Housekeeping (library pins, merges, test-only and docs-only work) is written
+-- kind internal at collection without a model; everything else stays NULL until blog.changelog.tag
+-- answers. Nothing here is a constraint on the old rows, so the migration only widens the table.
+
+ALTER TABLE changelog_entries ADD COLUMN IF NOT EXISTS kind text;
+ALTER TABLE changelog_entries ADD COLUMN IF NOT EXISTS areas text[] NOT NULL DEFAULT '{}'::text[];
+ALTER TABLE changelog_entries ADD COLUMN IF NOT EXISTS plain text;
+ALTER TABLE changelog_entries ADD COLUMN IF NOT EXISTS tagged_at bigint;
+ALTER TABLE changelog_entries ADD COLUMN IF NOT EXISTS tagged_by text;
+
+CREATE INDEX IF NOT EXISTS idx_changelog_kind ON changelog_entries (kind, deployed_at DESC, id DESC);
+CREATE INDEX IF NOT EXISTS idx_changelog_areas ON changelog_entries USING gin (areas);
@@ -9,6 +9,7 @@
@@ -9,6 +9,7 @@
"dev": "NODE_ENV=development node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
"test": "node test/run.js",
"test": "node test/run.js",
"seed": "node scripts/seed.js",
"seed": "node scripts/seed.js",
+ "backfill:changelog-tags": "node scripts/backfill-changelog-tags.js",
"test:pg": "BLOG_TEST_STORE=pg node test/run.js"
"test:pg": "BLOG_TEST_STORE=pg node test/run.js"
},
},
"keywords": [
"keywords": [
new file mode 100644
new file mode 100644
@@ -0,0 +1,94 @@
@@ -0,0 +1,94 @@
+#!/usr/bin/env node
+'use strict';
+/**
+ * Backfill the network changelog's tags (plan §9): existing entries with no kind get plain words,
+ * a kind and one to three areas. Housekeeping (library pins, merges, test-only and docs-only work)
+ * is decided here without a model; everything else goes to OpenVibe.AI's blog.changelog.tag task in
+ * batches, exactly as collection does (server/changelog/tagger.js).
+ *
+ * node scripts/backfill-changelog-tags.js # dry run: print what it would do
+ * node scripts/backfill-changelog-tags.js --service live # one site
+ * node scripts/backfill-changelog-tags.js --limit 200 # at most this many rows (default 500)
+ * node scripts/backfill-changelog-tags.js --apply # write, spending the AI task's calls
+ * node scripts/backfill-changelog-tags.js --apply --production
+ *
+ * The dry run writes nothing and never calls the model. --apply needs OV_AI_INTERNAL_URL and the
+ * service secret; a production database also needs --production, so it is never an accident.
+ */
+const tags = require('../server/changelog/tags');
+const { createChangelogTagger } = require('../server/changelog/tagger');
+
+const args = process.argv.slice(2);
+const opt = (name) => { const i = args.indexOf(`--${name}`); return i >= 0 ? args[i + 1] : null; };
+const has = (name) => args.includes(`--${name}`);
+
+async function main() {
+ if (has('help')) {
+ console.log('usage: node scripts/backfill-changelog-tags.js [--service <id>] [--limit 500] [--chunk 20] [--apply] [--production]');
+ return 0;
+ }
+ const config = require('../server/config').load();
+ if (config.isProduction && has('apply') && !has('production')) {
+ console.error('[backfill] refusing to write to a production database without --production');
+ return 1;
+ }
+ const apply = has('apply');
+ const service = opt('service');
+ const limit = Math.max(1, Math.min(10000, parseInt(opt('limit'), 10) || 500));
+ const chunk = Math.max(1, Math.min(25, parseInt(opt('chunk'), 10) || 20));
+ const { openStore } = require('../server/db');
+ const store = await openStore(config, { log: console });
+ const db = store.db;
+ const where = ['kind IS NULL'];
+ const params = [];
+ if (service) { where.push('service = ?'); params.push(service); }
+ const rows = await db.prepare(`SELECT id, service, repo, sha, subject FROM changelog_entries WHERE ${where.join(' AND ')} ORDER BY deployed_at, id LIMIT ?`).all(...params, limit);
+
+ const housekeeping = [];
+ const needModel = [];
+ for (const row of rows) {
+ const initial = tags.initial(row.subject);
+ if (initial.kind === 'internal') housekeeping.push({ ...row, tag: initial });
+ else needModel.push({ ...row, plain: initial.plain });
+ }
+ console.log(`[backfill] ${rows.length} untagged entr${rows.length === 1 ? 'y' : 'ies'}${service ? ` on ${service}` : ''} (limit ${limit})`);
+ console.log(`[backfill] ${housekeeping.length} housekeeping → internal without a model; ${needModel.length} need blog.changelog.tag (${Math.ceil(needModel.length / chunk)} run${Math.ceil(needModel.length / chunk) === 1 ? '' : 's'})`);
+ for (const row of housekeeping.slice(0, 40)) console.log(` internal ${row.service} ${row.sha.slice(0, 7)} ${row.tag.plain}`);
+ if (housekeeping.length > 40) console.log(` … ${housekeeping.length - 40} more housekeeping`);
+ for (const row of needModel.slice(0, 40)) console.log(` model ${row.service} ${row.sha.slice(0, 7)} ${row.plain}`);
+ if (needModel.length > 40) console.log(` … ${needModel.length - 40} more for the model`);
+
+ if (!apply) {
+ console.log('[backfill] dry run: no writes, no AI calls. Re-run with --apply to write.');
+ await store.close();
+ return 0;
+ }
+ const tagger = createChangelogTagger({ config, log: console });
+ if (!tagger.enabled) {
+ console.error('[backfill] blog.changelog.tag is not configured (OV_AI_INTERNAL_URL and OV_OAUTH_CLIENT_SECRET)');
+ await store.close();
+ return 1;
+ }
+ const tagStmt = db.prepare('UPDATE changelog_entries SET kind = ?, areas = ?, plain = ?, tagged_at = ?, tagged_by = ? WHERE id = ? AND kind IS NULL');
+ let tagged = 0;
+ for (const row of housekeeping) tagged += (await tagStmt.run('internal', [], row.tag.plain, store.now(), 'housekeeping', row.id)).changes;
+ for (let i = 0; i < needModel.length; i += chunk) {
+ const batch = needModel.slice(i, i + chunk);
+ try {
+ const answer = await tagger.tag(batch);
+ for (const row of batch) {
+ const t = answer.get(String(row.id));
+ if (!t) continue;
+ tagged += (await tagStmt.run(t.kind, t.areas, t.plain, store.now(), 'ai', row.id)).changes;
+ }
+ console.log(`[backfill] tagged ${Math.min(i + chunk, needModel.length)}/${needModel.length} model entries`);
+ } catch (err) {
+ console.error(`[backfill] batch ${i / chunk + 1} failed, left untagged: ${err.message}`);
+ }
+ }
+ console.log(`[backfill] wrote ${tagged} of ${rows.length} entries`);
+ await store.close();
+ return tagged === rows.length ? 0 : 1;
+}
+
+main().then((code) => { process.exitCode = code; }, (err) => { console.error(`[backfill] ${err.message}`); process.exitCode = 1; });
@@ -103,7 +103,10 @@ async function createApp(opts = {}) {
@@ -103,7 +103,10 @@ async function createApp(opts = {}) {
// Patch notes v2: the story map, the template, the voice cards and the review gate. The changelog files drafts
// Patch notes v2: the story map, the template, the voice cards and the review gate. The changelog files drafts
// through it; posts.publish refuses a draft with a blocking check or no person's approval.
// through it; posts.publish refuses a draft with a blocking check or no person's approval.
const patchNotes = opts.patchNotes || require('./patch-notes').createPatchNotes({ config, store, blogs, posts, publication, aiDrafts, media, fetchImpl, log });
const patchNotes = opts.patchNotes || require('./patch-notes').createPatchNotes({ config, store, blogs, posts, publication, aiDrafts, media, fetchImpl, log });
- const changelog = opts.changelog || require('./changelog').createChangelog({ config, store, blogs, posts, patchNotes, fetchImpl, log });
+ // The changelog's tags: housekeeping is internal without a model; the rest ask OpenVibe.AI's
+ // blog.changelog.tag task (server/changelog/tagger.js), whose owner-chosen model answers.
+ const changelogTagger = opts.changelogTagger || (opts.changelog ? null : require('./changelog/tagger').createChangelogTagger({ config, fetchImpl, log }));
+ const changelog = opts.changelog || require('./changelog').createChangelog({ config, store, blogs, posts, reading, tagger: changelogTagger, patchNotes, fetchImpl, log });
const devblog = require('./devblog').createDevblog({ config, store, blogs, posts, access, aiDrafts, media, log });
const devblog = require('./devblog').createDevblog({ config, store, blogs, posts, access, aiDrafts, media, log });
if (changelog.attachDevblog) changelog.attachDevblog(devblog);
if (changelog.attachDevblog) changelog.attachDevblog(devblog);
if (patchNotes.attach) patchNotes.attach({ changelog });
if (patchNotes.attach) patchNotes.attach({ changelog });
@@ -20,11 +20,17 @@
@@ -20,11 +20,17 @@
* template, with OpenVibe.AI's prose when it is configured, and a person reviews the draft before it
* template, with OpenVibe.AI's prose when it is configured, and a person reviews the draft before it
* can be published. The draft owns its entries; the public feed links them only once the post is live.
* can be published. The draft owns its entries; the public feed links them only once the post is live.
*
*
- * Read: GET /api/v1/changelog?service=&limit= (public, CORS *) is what every site's "recently
- * shipped" shows, with the latest patch notes post to link to.
+ * Read: GET /api/v1/changelog?service=&kind=&area=&q=&limit= (public, CORS *) is what every site's
+ * "recently shipped" shows, with the latest patch notes post to link to. Each entry carries its kind
+ * (new, better, fixed or internal), one to three areas and plain words; internal housekeeping is
+ * filterable and counted but the sites hide it behind "Show housekeeping". GET
+ * /api/v1/changelog/devblog?service= answers the site's newest devblog post (title, teaser, date,
+ * hero image and URL) for the home card. The fixed vocabulary is server/changelog/tags.js; the
+ * blog.changelog.tag task writes the tags (server/changelog/tagger.js), housekeeping without a model.
*/
*/
const crypto = require('crypto');
const crypto = require('crypto');
const { serviceAuth } = require('openvibe-contracts');
const { serviceAuth } = require('openvibe-contracts');
+const tags = require('./changelog/tags');
const GITHUB = 'https://api.github.com';
const GITHUB = 'https://api.github.com';
const clean = (s, n) => String(s == null ? '' : s).replace(/\s+/g, ' ').trim().slice(0, n);
const clean = (s, n) => String(s == null ? '' : s).replace(/\s+/g, ' ').trim().slice(0, n);
@@ -49,26 +55,27 @@ function subjectOf(message, n = 240) {
@@ -49,26 +55,27 @@ function subjectOf(message, n = 240) {
return `${cut.slice(0, Math.max(cut.lastIndexOf(' '), n - 40)).replace(/[,;:(\s]+$/, '')}…`;
return `${cut.slice(0, Math.max(cut.lastIndexOf(' '), n - 40)).replace(/[,;:(\s]+$/, '')}…`;
}
}
const isVersionBump = (subject) => /^\d+\.\d+\.\d+\b/.test(subject);
const isVersionBump = (subject) => /^\d+\.\d+\.\d+\b/.test(subject);
-const isNoise = (subject) => /^Merge (branch|pull request|remote)/i.test(subject);
+const TAG_BATCH = 20;
-function createChangelog({ config, store, blogs, posts, patchNotes = null, fetchImpl = globalThis.fetch, log = console }) {
+function createChangelog({ config, store, blogs, posts, reading = null, tagger = null, patchNotes = null, fetchImpl = globalThis.fetch, log = console }) {
const c = config.changelog;
const c = config.changelog;
const db = store.db;
const db = store.db;
const nowMs = () => store.now();
const nowMs = () => store.now();
- const stats = { runs: 0, entries: 0, posts: 0, drafts: 0, lastError: null, lastRunAt: null };
+ const stats = { runs: 0, entries: 0, posts: 0, drafts: 0, tagged: 0, lastError: null, lastRunAt: null };
const q = {
const q = {
state: db.prepare('SELECT * FROM changelog_state WHERE service = ?'),
state: db.prepare('SELECT * FROM changelog_state WHERE service = ?'),
putState: db.prepare(`INSERT INTO changelog_state (service, release, head, checked_at) VALUES (@service, @release, @head, @at)
putState: db.prepare(`INSERT INTO changelog_state (service, release, head, checked_at) VALUES (@service, @release, @head, @at)
ON CONFLICT(service) DO UPDATE SET release = excluded.release, head = excluded.head, checked_at = excluded.checked_at`),
ON CONFLICT(service) DO UPDATE SET release = excluded.release, head = excluded.head, checked_at = excluded.checked_at`),
- insert: db.prepare(`INSERT INTO changelog_entries (service, repo, sha, subject, author, committed_at, deployed_at, release, release_lines, major, post_id)
- VALUES (@service, @repo, @sha, @subject, @author, @committed_at, @deployed_at, @release, @release_lines, @major, @post_id) ON CONFLICT (service, sha) DO NOTHING`),
+ insert: db.prepare(`INSERT INTO changelog_entries (service, repo, sha, subject, author, committed_at, deployed_at, release, release_lines, major, post_id, kind, areas, plain, tagged_at, tagged_by)
+ VALUES (@service, @repo, @sha, @subject, @author, @committed_at, @deployed_at, @release, @release_lines, @major, @post_id, @kind, @areas, @plain, @tagged_at, @tagged_by) ON CONFLICT (service, sha) DO NOTHING RETURNING id`),
+ tag: db.prepare('UPDATE changelog_entries SET kind = ?, areas = ?, plain = ?, tagged_at = ?, tagged_by = ? WHERE id = ? AND kind IS NULL'),
fillAuthor: db.prepare('UPDATE changelog_entries SET author = ? WHERE service = ? AND sha = ? AND author IS NULL'),
fillAuthor: db.prepare('UPDATE changelog_entries SET author = ? WHERE service = ? AND sha = ? AND author IS NULL'),
history: db.prepare('SELECT * FROM changelog_history WHERE service = ?'),
history: db.prepare('SELECT * FROM changelog_history WHERE service = ?'),
putHistory: db.prepare('INSERT INTO changelog_history (service, imported, imported_at) VALUES (?, ?, ?) ON CONFLICT (service) DO UPDATE SET imported = excluded.imported, imported_at = excluded.imported_at'),
putHistory: db.prepare('INSERT INTO changelog_history (service, imported, imported_at) VALUES (?, ?, ?) ON CONFLICT (service) DO UPDATE SET imported = excluded.imported, imported_at = excluded.imported_at'),
recentPosts: db.prepare('SELECT * FROM changelog_posts ORDER BY created_at DESC LIMIT ?'),
recentPosts: db.prepare('SELECT * FROM changelog_posts ORDER BY created_at DESC LIMIT ?'),
sites: db.prepare(`SELECT service, COUNT(*) AS entries, MAX(deployed_at) AS latest_at FROM changelog_entries GROUP BY service ORDER BY latest_at DESC`),
sites: db.prepare(`SELECT service, COUNT(*) AS entries, MAX(deployed_at) AS latest_at FROM changelog_entries GROUP BY service ORDER BY latest_at DESC`),
- pending: db.prepare('SELECT * FROM changelog_entries WHERE post_id IS NULL ORDER BY deployed_at, committed_at, id'),
+ pending: db.prepare("SELECT * FROM changelog_entries WHERE post_id IS NULL AND (kind IS NULL OR kind <> 'internal') ORDER BY deployed_at, committed_at, id"),
mark: db.prepare('UPDATE changelog_entries SET post_id = ? WHERE id = ? AND post_id IS NULL'),
mark: db.prepare('UPDATE changelog_entries SET post_id = ? WHERE id = ? AND post_id IS NULL'),
ownedBy: db.prepare('SELECT * FROM changelog_entries WHERE post_id = ? ORDER BY deployed_at, committed_at, id'),
ownedBy: db.prepare('SELECT * FROM changelog_entries WHERE post_id = ? ORDER BY deployed_at, committed_at, id'),
move: db.prepare('UPDATE changelog_entries SET post_id = ? WHERE post_id = ?'),
move: db.prepare('UPDATE changelog_entries SET post_id = ? WHERE post_id = ?'),
@@ -140,27 +147,60 @@ function createChangelog({ config, store, blogs, posts, patchNotes = null, fetch
@@ -140,27 +147,60 @@ function createChangelog({ config, store, blogs, posts, patchNotes = null, fetch
const authorOf = (cm) => clean((cm.commit && cm.commit.author && cm.commit.author.name) || (cm.author && cm.author.login) || '', 80) || null;
const authorOf = (cm) => clean((cm.commit && cm.commit.author && cm.commit.author.name) || (cm.author && cm.author.login) || '', 80) || null;
+ /**
+ * Tag the entries just collected: housekeeping already carries kind internal (plain words and
+ * no model call); the rest go to the blog.changelog.tag task in one run. A failed run leaves
+ * them untagged for the backfill and never fails collection. History is never sent (it can be
+ * hundreds of commits per site): the backfill tags it.
+ */
+ async function tagPending(list) {
+ if (!list.length || !tagger || !tagger.enabled) return 0;
+ let n = 0;
+ for (let i = 0; i < list.length; i += TAG_BATCH) {
+ const chunk = list.slice(i, i + TAG_BATCH);
+ try {
+ const answer = await tagger.tag(chunk);
+ for (const e of chunk) {
+ const t = answer.get(String(e.id));
+ if (!t) continue;
+ n += (await q.tag.run(t.kind, t.areas, t.plain, nowMs(), 'ai', e.id)).changes;
+ }
+ } catch (err) {
+ stats.lastError = `tag: ${err.message}`;
+ log.warn && log.warn(`[changelog] ${list.length} entries left untagged: ${err.message}`);
+ }
+ }
+ stats.tagged += n;
+ return n;
+ }
+
// history: the commits a site shipped before the changelog watched it, so its updates page is not
// history: the commits a site shipped before the changelog watched it, so its updates page is not
// empty on day one. They carry post_id 'history' (never pending, never in a patch notes post) and
// empty on day one. They carry post_id 'history' (never pending, never in a patch notes post) and
// their commit time stands in for the deploy time nobody recorded.
// their commit time stands in for the deploy time nobody recorded.
async function addEntries(svc, commits, { releaseLines = 0, major = false, history = false } = {}) {
async function addEntries(svc, commits, { releaseLines = 0, major = false, history = false } = {}) {
const deployedAt = svc.releasedAt || new Date(nowMs()).toISOString();
const deployedAt = svc.releasedAt || new Date(nowMs()).toISOString();
let n = 0;
let n = 0;
+ const untagged = [];
for (const cm of commits) {
for (const cm of commits) {
const subject = subjectOf(cm.commit && cm.commit.message);
const subject = subjectOf(cm.commit && cm.commit.message);
- if (!subject || isNoise(subject)) continue;
+ if (!subject) continue;
+ const meta = tags.initial(subject);
const flagged = major || isVersionBump(subject) || /\[major\]/i.test((cm.commit && cm.commit.message) || '');
const flagged = major || isVersionBump(subject) || /\[major\]/i.test((cm.commit && cm.commit.message) || '');
const committedAt = (cm.commit && cm.commit.author && cm.commit.author.date) || null;
const committedAt = (cm.commit && cm.commit.author && cm.commit.author.date) || null;
if (history && !committedAt) continue;
if (history && !committedAt) continue;
const author = authorOf(cm);
const author = authorOf(cm);
- const added = (await q.insert.run({
+ const row = await q.insert.get({
service: svc.id, repo: svc.repo, sha: cm.sha, subject, author, committed_at: committedAt,
service: svc.id, repo: svc.repo, sha: cm.sha, subject, author, committed_at: committedAt,
deployed_at: history ? new Date(committedAt).toISOString() : deployedAt, release: svc.release, release_lines: releaseLines, major: flagged ? 1 : 0,
deployed_at: history ? new Date(committedAt).toISOString() : deployedAt, release: svc.release, release_lines: releaseLines, major: flagged ? 1 : 0,
- post_id: history ? 'history' : null,
- })).changes;
- if (!added && author) await q.fillAuthor.run(author, svc.id, cm.sha);
- n += added;
+ post_id: history ? 'history' : null, kind: meta.kind, areas: meta.areas, plain: meta.plain,
+ tagged_at: meta.kind ? nowMs() : null, tagged_by: meta.kind ? 'housekeeping' : null,
+ });
+ if (!row) { if (author) await q.fillAuthor.run(author, svc.id, cm.sha); continue; }
+ n += 1;
+ // History is tagged by the backfill, not inside a collection run.
+ if (!history && !meta.kind) untagged.push({ id: row.id, service: svc.id, repo: svc.repo, subject, plain: meta.plain });
}
}
+ await tagPending(untagged);
return n;
return n;
}
}
@@ -340,25 +380,56 @@ function createChangelog({ config, store, blogs, posts, patchNotes = null, fetch
@@ -340,25 +380,56 @@ function createChangelog({ config, store, blogs, posts, patchNotes = null, fetch
}
}
function stop() { if (timer) clearInterval(timer); timer = null; }
function stop() { if (timer) clearInterval(timer); timer = null; }
- /** The public feed, newest first: { entries, next } (next = the cursor for the page after, or null). */
- async function page({ service = null, limit = 20, before = null } = {}) {
+ /**
+ * The public feed, newest first: { entries, next, facets, vocabulary } (next = the cursor for the
+ * page after, or null). Filters: service, kind, area and q (the plain words or the subject);
+ * facets count the kinds and areas in the same scope (service and q) so the chips can show them.
+ */
+ async function page({ service = null, limit = 20, before = null, kind = null, area = null, q: search = null } = {}) {
const n = Math.min(Math.max(parseInt(limit, 10) || 20, 1), 100);
const n = Math.min(Math.max(parseInt(limit, 10) || 20, 1), 100);
const cur = decodeCursor(before);
const cur = decodeCursor(before);
- const where = [];
- const args = [];
- if (service) { where.push('service = ?'); args.push(String(service)); }
- if (cur) { where.push('(deployed_at < ? OR (deployed_at = ? AND id < ?))'); args.push(cur.at, cur.at, cur.id); }
+ const escapeLike = (s) => String(s).replace(/[\\%_]/g, '\\$&');
+ const scope = [];
+ const scopeArgs = [];
+ if (service) { scope.push('e.service = ?'); scopeArgs.push(String(service)); }
+ if (search) {
+ const pattern = `%${escapeLike(search).slice(0, 200)}%`;
+ scope.push("(e.plain ILIKE ? ESCAPE '\\' OR e.subject ILIKE ? ESCAPE '\\')");
+ scopeArgs.push(pattern, pattern);
+ }
+ const where = [...scope];
+ const args = [...scopeArgs];
+ if (kind) { where.push('e.kind = ?'); args.push(String(kind)); }
+ if (area) { where.push('e.areas @> ARRAY[?]::text[]'); args.push(String(area)); }
+ if (cur) { where.push('(e.deployed_at < ? OR (e.deployed_at = ? AND e.id < ?))'); args.push(cur.at, cur.at, cur.id); }
const rows = await db.prepare(`SELECT e.*, (SELECT p.state FROM blog_posts p WHERE p.id = e.post_id) AS post_state
const rows = await db.prepare(`SELECT e.*, (SELECT p.state FROM blog_posts p WHERE p.id = e.post_id) AS post_state
FROM changelog_entries e ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY e.deployed_at DESC, e.id DESC LIMIT ?`).all(...args, n + 1);
FROM changelog_entries e ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY e.deployed_at DESC, e.id DESC LIMIT ?`).all(...args, n + 1);
const more = rows.length > n;
const more = rows.length > n;
const shown = rows.slice(0, n);
const shown = rows.slice(0, n);
+ const scoped = scope.length ? `WHERE ${scope.join(' AND ')}` : '';
+ const kindCounts = Object.fromEntries(tags.KINDS.map((k) => [k, 0]));
+ for (const row of await db.prepare(`SELECT e.kind, COUNT(*) AS n FROM changelog_entries e ${scoped} GROUP BY e.kind`).all(...scopeArgs)) {
+ kindCounts[row.kind && tags.isKind(row.kind) ? row.kind : 'better'] += Number(row.n) || 0;
+ }
+ const areaCounts = {};
+ for (const row of await db.prepare(`SELECT a AS area, COUNT(*) AS n FROM changelog_entries e CROSS JOIN LATERAL unnest(e.areas) AS a ${scoped} GROUP BY a`).all(...scopeArgs)) {
+ if (tags.isArea(row.area)) areaCounts[row.area] = Number(row.n) || 0;
+ }
return {
return {
- entries: shown.map((e) => ({
- service: e.service, sha: e.sha, short: e.sha.slice(0, 7), subject: e.subject, author: e.author || null, committed_at: e.committed_at, deployed_at: e.deployed_at,
- major: Boolean(e.major), url: `https://github.com/${e.repo}/commit/${e.sha}`,
- post_id: e.post_id && e.post_id !== 'history' && e.post_state === 'published' ? e.post_id : null,
- })),
+ entries: shown.map((e) => {
+ const pr = Number(e.pr_number) || tags.prOf(e.subject);
+ return {
+ service: e.service, repo: e.repo, sha: e.sha, short: e.sha.slice(0, 7), subject: e.subject, plain: e.plain || tags.plainOf(e.subject),
+ kind: e.kind && tags.isKind(e.kind) ? e.kind : 'better', areas: Array.isArray(e.areas) ? e.areas.filter(tags.isArea).slice(0, 3) : [],
+ author: e.author || null, committed_at: e.committed_at, deployed_at: e.deployed_at,
+ major: Boolean(e.major), url: `https://github.com/${e.repo}/commit/${e.sha}`,
+ post_id: e.post_id && e.post_id !== 'history' && e.post_state === 'published' ? e.post_id : null,
+ pr: pr || null, pr_url: pr ? `https://github.com/${e.repo}/pull/${pr}` : null,
+ };
+ }),
next: more && shown.length ? encodeCursor(shown[shown.length - 1]) : null,
next: more && shown.length ? encodeCursor(shown[shown.length - 1]) : null,
+ facets: { kinds: kindCounts, areas: areaCounts },
+ vocabulary: tags.vocabulary(),
};
};
}
}
async function entries(opts = {}) { return (await page(opts)).entries; }
async function entries(opts = {}) { return (await page(opts)).entries; }
@@ -367,7 +438,45 @@ function createChangelog({ config, store, blogs, posts, patchNotes = null, fetch
@@ -367,7 +438,45 @@ function createChangelog({ config, store, blogs, posts, patchNotes = null, fetch
/** Every site with entries: [{ service, entries, latest_at }], most recently shipped first. */
/** Every site with entries: [{ service, entries, latest_at }], most recently shipped first. */
async function sites() { return await q.sites.all(); }
async function sites() { return await q.sites.all(); }
- return { attachDevblog: value => { devblog = value; }, tick, collect, due, fileDraft, enrich, pendingEntries, markEntries, entriesOf, moveEntries, openPulls, start, stop, entries, page, latestPost, recentPosts, sites, importHistory, stats: async () => ({ enabled: c.enabled, ...stats, github_token: cachedToken.source, pending: (await q.pending.all()).length }) };
+ /**
+ * One site's newest devblog post, for the Rich Content §9 home card and the day's row in
+ * /updates: { service, title, teaser, date, url, hero_image_url } or null. Devblog posts live
+ * on the official blog, tagged with the service and categorized Devblog; the hero block's art
+ * is the card image. `reading` extracts the art and the URL from the revision.
+ */
+ async function newestDevblog(service) {
+ if (!service || !reading || !posts || !blogs) return null;
+ const blog = await blogs.byHandle(c.blogHandle) || (c.blogHandle === config.official.handle ? await blogs.ensureOfficial() : null);
+ if (!blog) return null;
+ const tag = await store.taxonomy.bySlug('tag', String(service));
+ if (!tag) return null;
+ const { posts: rows } = await posts.listPublished({ blogId: blog.id, termIds: [tag.id], limit: 10 });
+ const items = await reading.listItems(rows);
+ const item = items.find((i) => (i.categories || []).some((x) => /^devblog$/i.test(x.name || x.slug || ''))
+ || (i.tags || []).some((x) => /^devblog$/i.test(x.slug || x.name || '')));
+ if (!item || !item.rev) return null;
+ const when = item.post.published_at || item.post.first_published_at || null;
+ return {
+ service: String(service),
+ title: item.rev.fields.title || null,
+ teaser: item.rev.fields.summary || null,
+ date: when ? new Date(when).toISOString() : null,
+ url: `${String(config.baseUrl || '').replace(/\/+$/, '')}${item.url}`,
+ hero_image_url: item.art ? item.art.url : null,
+ };
+ }
+
+ /** The newest devblog of each given site (the /updates page's day sections), newest first. */
+ async function newestDevblogs(services = []) {
+ const out = [];
+ for (const service of services.slice(0, 20)) {
+ const devblog = await newestDevblog(service).catch(() => null);
+ if (devblog) out.push(devblog);
+ }
+ return out.sort((a, b) => String(b.date || '').localeCompare(String(a.date || '')));
+ }
+
+ return { attachDevblog: value => { devblog = value; }, tick, collect, due, fileDraft, enrich, pendingEntries, markEntries, entriesOf, moveEntries, openPulls, start, stop, entries, page, latestPost, recentPosts, sites, newestDevblog, newestDevblogs, importHistory, stats: async () => ({ enabled: c.enabled, ...stats, github_token: cachedToken.source, pending: (await q.pending.all()).length }) };
}
}
/** A stable id for a batch (tests). */
/** A stable id for a batch (tests). */
new file mode 100644
new file mode 100644
@@ -0,0 +1,114 @@
@@ -0,0 +1,114 @@
+'use strict';
+/**
+ * The AI tagger for the network changelog (plan §9): one blog.changelog.tag run per batch of new
+ * entries. The task is OpenVibe.AI's: its owner-chosen model (the cheapest good one by default)
+ * answers kind, one to three areas and the plain words; Blog never names a model here.
+ *
+ * A service token with ai.run.create in the blog namespace starts the run, as itself (collection
+ * is not a person's act). The run's answer is either the task workflow's { tags } or the general
+ * ai.generate { text } holding the same JSON (the task's fallback while it names ai.generate).
+ * Nothing here throws into collection: a missing or failed AI leaves the entries untagged for the
+ * backfill (scripts/backfill-changelog-tags.js).
+ */
+const crypto = require('crypto');
+const { serviceAuth } = require('openvibe-contracts');
+const { normalizeTag } = require('./tags');
+
+const WAIT_MS = 20000;
+const TIMEOUT_MS = 60000;
+const POLL_MS = 1500;
+
+function parseLoose(text) {
+ const s = String(text || '').trim().replace(/^```(?:json)?\s*|\s*```$/g, '');
+ try { return JSON.parse(s); } catch {}
+ const a = s.indexOf('{');
+ const b = s.lastIndexOf('}');
+ if (a >= 0 && b > a) { try { return JSON.parse(s.slice(a, b + 1)); } catch {} }
+ return null;
+}
+
+/** { tags } from either workflow's output. */
+function tagsFrom(output) {
+ if (!output || typeof output !== 'object') return [];
+ if (Array.isArray(output.tags)) return output.tags;
+ const parsed = typeof output.text === 'string' ? parseLoose(output.text) : null;
+ return parsed && Array.isArray(parsed.tags) ? parsed.tags : [];
+}
+
+function createChangelogTagger({ config, fetchImpl = globalThis.fetch, log = console } = {}) {
+ const tokenClient = config.aiUrl && config.oauth && config.oauth.clientSecret ? serviceAuth.createTokenClient({
+ tokenUrl: `${config.networkInternalUrl}/oauth/token`, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret,
+ audience: 'openvibe.ai', scope: 'ai.run.create ai.run.read', fetchImpl,
+ }) : null;
+ const enabled = Boolean(config.changelog && config.changelog.tag !== false && config.aiUrl && tokenClient);
+
+ async function call(method, path, body, { retried = false } = {}) {
+ const res = await fetchImpl(`${String(config.aiUrl).replace(/\/+$/, '')}${path}`, {
+ method,
+ headers: { Accept: 'application/json', ...(body ? { 'Content-Type': 'application/json' } : {}), ...(await tokenClient.authHeaders()) },
+ body: body ? JSON.stringify(body) : undefined,
+ signal: AbortSignal.timeout(WAIT_MS + 10000),
+ });
+ if (res.status === 401 && !retried && typeof tokenClient.invalidate === 'function') {
+ tokenClient.invalidate();
+ return call(method, path, body, { retried: true });
+ }
+ const json = await res.json().catch(() => ({}));
+ return { status: res.status, ok: res.ok, body: json };
+ }
+
+ const runOf = (body) => (body && body.run ? body.run : body);
+
+ async function run(body) {
+ const started = await call('POST', `/api/v1/runs?wait=${WAIT_MS}`, body);
+ // Older AI (before tasks and choices) refuses a task: the same workflow by name still runs.
+ if (!started.ok && body.task && [404, 422].includes(started.status)
+ && /task|workflow/i.test(`${started.body.code || ''} ${started.body.detail || ''}`)) {
+ const { task, ...rest } = body;
+ return { ...(await call('POST', `/api/v1/runs?wait=${WAIT_MS}`, { ...rest, workflow: 'blog.changelog.tag' })), workflowFallback: true };
+ }
+ if (!started.ok) throw Object.assign(new Error(`AI refused the run (${started.body.code || started.status})`), { status: started.status });
+ let run = runOf(started.body);
+ const deadline = Date.now() + TIMEOUT_MS;
+ while (run && (run.status === 'queued' || run.status === 'running')) {
+ if (Date.now() > deadline) throw Object.assign(new Error(`run ${run.id} did not finish`), { status: 504 });
+ await new Promise((ok) => setTimeout(ok, POLL_MS));
+ const got = await call('GET', `/api/v1/runs/${encodeURIComponent(run.id)}`);
+ if (!got.ok) throw Object.assign(new Error(`reading run ${run.id} failed (${got.status})`), { status: got.status });
+ run = runOf(got.body);
+ }
+ if (!run || !['succeeded', 'cached'].includes(run.status)) {
+ throw Object.assign(new Error(`run ${run && run.id} ${run && run.status}`), { status: 502 });
+ }
+ return { output: run.output || {}, workflow: run.workflow || null, runId: run.id };
+ }
+
+ /**
+ * Tag these entries: [{ id, service, subject, repo, plain }] → Map(id → { kind, areas, plain }).
+ * One run for the whole batch (at most the caller's slice). Rejects on failure so the caller
+ * can log it; the entries stay untagged.
+ */
+ async function tag(entries) {
+ if (!enabled || !entries.length) return new Map();
+ const input = {
+ entries: entries.slice(0, 25).map((e) => ({ id: String(e.id), service: e.service, subject: e.subject, ...(e.repo ? { repo: e.repo } : {}), ...(e.plain ? { plain: e.plain } : {}) })),
+ };
+ const idempotency = crypto.createHash('sha256').update(input.entries.map((e) => `${e.service}:${e.id}`).join(',')).digest('hex');
+ const answer = await run({
+ task: 'blog.changelog.tag', input,
+ target: { service: 'blog', type: 'changelog', id: 'network' },
+ idempotency_key: `changelog-tags:${idempotency}`,
+ });
+ const out = new Map();
+ for (const t of tagsFrom(answer.output)) {
+ const safe = normalizeTag(t);
+ if (safe && input.entries.some((e) => e.id === String(t.id))) out.set(String(t.id), safe);
+ }
+ if (answer.workflowFallback) log.warn && log.warn('[changelog] AI does not know the blog.changelog.tag task yet; ran the workflow by name');
+ return out;
+ }
+
+ return { enabled, tag };
+}
+
+module.exports = { createChangelogTagger, tagsFrom, parseLoose };
new file mode 100644
new file mode 100644
@@ -0,0 +1,125 @@
@@ -0,0 +1,125 @@
+'use strict';
+/**
+ * The network changelog's fixed vocabulary (plan §9): what kind of change an entry is and which
+ * parts of the product it touched. One module owns both lists: the AI tagger, the backfill, the
+ * API filters and the contract (blog.changelog-entry@2) all read them from here or repeat this
+ * exact order.
+ *
+ * Housekeeping is decided without a model: library pins ("openvibe-shared v4.175.0 (was
+ * v4.173.0)"), merges, and test-only, docs-only or CI-only commits are kind internal. Everything
+ * else is offered to the blog.changelog.tag task, whose owner-chosen model answers kind, areas and
+ * the plain words; the deterministic fallbacks here are what is stored until it answers.
+ */
+
+const KINDS = ['new', 'better', 'fixed', 'internal'];
+
+/** The fixed areas, in the order the filters show them. `id` is what the API and contract carry. */
+const AREAS = [
+ { id: 'phone', label: 'Phone' },
+ { id: 'chat', label: 'Chat' },
+ { id: 'go-live', label: 'Go live' },
+ { id: 'video', label: 'Video' },
+ { id: 'menu', label: 'Menu' },
+ { id: 'accessibility', label: 'Accessibility' },
+ { id: 'speed', label: 'Speed' },
+ { id: 'sign-in', label: 'Sign-in' },
+ { id: 'wallet', label: 'Wallet' },
+ { id: 'ai', label: 'AI' },
+ { id: 'search', label: 'Search' },
+ { id: 'files', label: 'Files' },
+ { id: 'pictures', label: 'Pictures' },
+ { id: 'games', label: 'Games' },
+ { id: 'design', label: 'Design' },
+ { id: 'developer', label: 'Developer' },
+ { id: 'machines', label: 'Machines (Link)' },
+ { id: 'money', label: 'Money' },
+ { id: 'privacy', label: 'Privacy' },
+ { id: 'moderation', label: 'Moderation' },
+ { id: 'notifications', label: 'Notifications' },
+ { id: 'feed', label: 'Feed' },
+ { id: 'wiki', label: 'Wiki' },
+ { id: 'blog', label: 'Blog' },
+ { id: 'community', label: 'Community' },
+ { id: 'meme', label: 'Meme' },
+ { id: 'shop', label: 'Shop' },
+ { id: 'tools', label: 'Tools' },
+ { id: 'exchange', label: 'Exchange' },
+ { id: 'git', label: 'Git' },
+];
+const AREA_IDS = AREAS.map((a) => a.id);
+const AREA_LABELS = new Map(AREAS.map((a) => [a.id, a.label]));
+
+const clean = (s, n) => String(s == null ? '' : s).replace(/\s+/g, ' ').trim().slice(0, n).trim();
+const isKind = (v) => KINDS.includes(v);
+const isArea = (v) => AREA_IDS.includes(v);
+
+/** The PR number a subject names in "(#714)"; null when it names none. */
+function prOf(subject) {
+ const m = /\(#(\d+)\)\s*$/.exec(String(subject || '').trim());
+ return m ? Number(m[1]) : null;
+}
+
+/** Leading stage tags ("[A6]", "[stage 2]", "[C2]") and conventional prefixes are not the subject. */
+function withoutStage(subject) {
+ return String(subject || '')
+ .replace(/^\s*(?:\[[^\]]{1,40}\]\s*)+/, '')
+ .replace(/^\s*[A-Za-z][A-Za-z0-9._-]*\([^)]*\)!?:\s*/, '') // feat(chat): / fix(menu)!:
+ .replace(/^\s*(?:feat|fix|perf|refactor|style|docs?|test|tests|ci|build|chore|deps|revert)!?:\s*/i, '')
+ .replace(/^\s*\d+\.\d+\.\d+:\s*/, '') // 0.376.0: the release subject
+ .replace(/\s*\(#\d+\)\s*$/, '')
+ .replace(/\s+/g, ' ')
+ .trim();
+}
+
+/** A subject in plain words: no PR number, no stage tag, no release-version prefix. */
+function plainOf(subject, n = 240) {
+ let s = withoutStage(subject);
+ s = s.replace(/\s*\(#\d+\)\s*/g, ' ').replace(/\s*\[[^\]]{1,40}\]\s*/g, ' ').replace(/\s+/g, ' ').trim();
+ return clean(s, n) || clean(subject, n) || 'A change';
+}
+
+/** Library pins, merges and test/docs/CI-only work: internal without asking a model. */
+function isHousekeeping(subject) {
+ const s = String(subject || '').replace(/^\s*(?:\[[^\]]{1,40}\]\s*)+/, '').trim();
+ return /^Merge (?:branch|pull request|remote|tag)\b/i.test(s)
+ || /^Revert "?Merge\b/i.test(s)
+ || /^openvibe-[a-z0-9-]+\s+v?\d+\.\d+\.\d+/i.test(s)
+ || /^(?:Bump|Pin|Repin)\s+/i.test(s)
+ || /^(?:Bump|Pin|Repin)\b.*\bopenvibe-/i.test(s)
+ || /^\d+\.\d+\.\d+[:\s]/.test(s)
+ || /^(?:test|tests|docs?|ci|build|chore|deps|dependencies|release|lint|format)(?:\([^)]*\))?!?:\s/i.test(s)
+ || /^(?:Update|Refresh|Tidy|Normalise|Normalize)\s+(?:the\s+)?(?:deps|dependencies|lockfile|package-lock|docs?|tests?|changelog)\b/i.test(s)
+ || /^(?:Add|Update|Fix)\s+(?:a\s+)?(?:test|tests|docs?)\s+(?:only|for)\b/i.test(s);
+}
+
+/**
+ * What is stored when an entry is first collected: plain words always, and kind internal for
+ * housekeeping. `kind: null` means the AI task has not answered yet.
+ */
+function initial(subject) {
+ return {
+ plain: plainOf(subject),
+ kind: isHousekeeping(subject) ? 'internal' : null,
+ areas: isHousekeeping(subject) ? [] : [],
+ };
+}
+
+/** A model's answer, made safe: a valid kind, 1–3 known areas (none for internal) and plain words. */
+function normalizeTag(value) {
+ if (!value || typeof value !== 'object') return null;
+ const kind = isKind(value.kind) ? value.kind : null;
+ if (!kind) return null;
+ let areas = Array.isArray(value.areas) ? value.areas.filter(isArea) : [];
+ areas = [...new Set(areas)].slice(0, 3);
+ if (kind !== 'internal' && !areas.length) return null;
+ const plain = plainOf(value.plain, 240);
+ if (!plain) return null;
+ return { kind, areas, plain };
+}
+
+/** The vocabulary as the API shows it (ids and their display labels). */
+function vocabulary() {
+ return { kinds: [...KINDS], areas: AREAS.map((a) => ({ ...a })) };
+}
+
+module.exports = { KINDS, AREAS, AREA_IDS, AREA_LABELS, vocabulary, isKind, isArea, prOf, plainOf, isHousekeeping, initial, normalizeTag };
@@ -59,6 +59,9 @@ function load(env = process.env) {
@@ -59,6 +59,9 @@ function load(env = process.env) {
history: Math.max(0, Math.min(100, parseInt(env.CHANGELOG_HISTORY ?? '60', 10) || 0)),
history: Math.max(0, Math.min(100, parseInt(env.CHANGELOG_HISTORY ?? '60', 10) || 0)),
firstDelayMs: 60 * 1000,
firstDelayMs: 60 * 1000,
since: env.CHANGELOG_SINCE && !Number.isNaN(Date.parse(env.CHANGELOG_SINCE)) ? new Date(env.CHANGELOG_SINCE).toISOString() : null,
since: env.CHANGELOG_SINCE && !Number.isNaN(Date.parse(env.CHANGELOG_SINCE)) ? new Date(env.CHANGELOG_SINCE).toISOString() : null,
+ // blog.changelog.tag: on when OV_AI_INTERNAL_URL and the service secret are set; off leaves
+ // entries untagged for the backfill (the deterministic plain words are still stored).
+ tag: env.CHANGELOG_AI_TAG !== 'off',
},
},
// Patch notes v2 (server/patch-notes/): how many workstreams get an H2, and how many published posts the AI
// Patch notes v2 (server/patch-notes/): how many workstreams get an H2, and how many published posts the AI
// prompt gets as voice samples. The draft is always a draft; CHANGELOG_AI_DRAFT=off skips the AI prose and
// prompt gets as voice samples. The draft is always a draft; CHANGELOG_AI_DRAFT=off skips the AI prose and
@@ -37,6 +37,7 @@ const { run, jsonBody, ApiError, privateNoStore } = require('./errors');
@@ -37,6 +37,7 @@ const { run, jsonBody, ApiError, privateNoStore } = require('./errors');
const { guard } = require('../auth/viewer');
const { guard } = require('../auth/viewer');
const { checkCapability, CAPABILITIES } = require('../auth/capabilities');
const { checkCapability, CAPABILITIES } = require('../auth/capabilities');
const blockRender = require('../render/blocks');
const blockRender = require('../render/blocks');
+const tags = require('../changelog/tags');
function cors(origins) {
function cors(origins) {
const allowed = new Set(origins);
const allowed = new Set(origins);
@@ -210,21 +211,29 @@ function createApi(ctx) {
@@ -210,21 +211,29 @@ function createApi(ctx) {
}));
}));
// The network changelog (server/changelog.js): what shipped on every OpenVibe site, newest first, and
// The network changelog (server/changelog.js): what shipped on every OpenVibe site, newest first, and
- // the latest patch notes post. Public: every site's "recently shipped" reads it.
+ // the latest patch notes post. Public: every site's "recently shipped" reads it. Filters: kind, area
+ // (from the fixed vocabulary) and q (plain words or subject). /changelog/devblog answers the site's
+ // newest devblog post for the home card and /updates.
router.get('/changelog', run(async (req, res) => {
router.get('/changelog', run(async (req, res) => {
res.set('Access-Control-Allow-Origin', '*');
res.set('Access-Control-Allow-Origin', '*');
res.set('Cache-Control', 'public, max-age=60');
res.set('Cache-Control', 'public, max-age=60');
const service = /^[a-z][a-z0-9-]{1,39}$/.test(String(req.query.service || '')) ? String(req.query.service) : null;
const service = /^[a-z][a-z0-9-]{1,39}$/.test(String(req.query.service || '')) ? String(req.query.service) : null;
+ const kind = req.query.kind == null || req.query.kind === '' ? null : String(req.query.kind);
+ const area = req.query.area == null || req.query.area === '' ? null : String(req.query.area);
+ if (kind && !tags.isKind(kind)) throw new ApiError(400, 'changelog.kind_invalid', `kind is one of ${tags.KINDS.join(', ')}`);
+ if (area && !tags.isArea(area)) throw new ApiError(400, 'changelog.area_invalid', `area is one of ${tags.AREA_IDS.join(', ')}`);
+ const q = req.query.q == null ? null : String(req.query.q).trim().slice(0, 200) || null;
const postOf = async (row) => {
const postOf = async (row) => {
const p = row && await posts.get(row.post_id);
const p = row && await posts.get(row.post_id);
const blog = p ? await blogs.get(p.blog_id) : null;
const blog = p ? await blogs.get(p.blog_id) : null;
if (!p || !blog || p.state !== 'published') return null;
if (!p || !blog || p.state !== 'published') return null;
return { id: p.id, title: (await posts.head(p) || { fields: {} }).fields.title || null, url: publication.abs(publication.postPath(blog, p)), published_at: p.published_at || null, entries: row.entries };
return { id: p.id, title: (await posts.head(p) || { fields: {} }).fields.title || null, url: publication.abs(publication.postPath(blog, p)), published_at: p.published_at || null, entries: row.entries };
};
};
- const page = await ctx.changelog.page({ service, limit: req.query.limit, before: req.query.before });
+ const page = await ctx.changelog.page({ service, limit: req.query.limit, before: req.query.before, kind, area, q });
const recent = (await Promise.all((await ctx.changelog.recentPosts(5)).map(postOf))).filter(Boolean);
const recent = (await Promise.all((await ctx.changelog.recentPosts(5)).map(postOf))).filter(Boolean);
return {
return {
- service, entries: page.entries, next: page.next, latest_post: recent[0] || null, posts: recent,
+ service, entries: page.entries, next: page.next, facets: page.facets, vocabulary: page.vocabulary,
+ latest_post: recent[0] || null, posts: recent,
...(service ? {} : { sites: await ctx.changelog.sites() }),
...(service ? {} : { sites: await ctx.changelog.sites() }),
};
};
}));
}));
@@ -236,6 +245,19 @@ function createApi(ctx) {
@@ -236,6 +245,19 @@ function createApi(ctx) {
return { post: await postDto(result.post, { full: true }), created: result.created };
return { post: await postDto(result.post, { full: true }), created: result.created };
}, 201));
}, 201));
+ // The site's newest devblog post (Rich Content §9): title, teaser, date, hero image and URL.
+ // With service it is that site's card; without one it lists the newest devblog of each site with
+ // a devblog, for /updates' day sections. Public and cached like the changelog.
+ router.get('/changelog/devblog', run(async (req, res) => {
+ res.set('Access-Control-Allow-Origin', '*');
+ res.set('Cache-Control', 'public, max-age=300');
+ const service = req.query.service == null || req.query.service === '' ? null : String(req.query.service);
+ if (service && !/^[a-z][a-z0-9-]{1,39}$/.test(service)) throw new ApiError(400, 'changelog.service_invalid', 'service is a site id (live, chat, blog, …)');
+ if (service) return { service, devblog: await ctx.changelog.newestDevblog(service) };
+ const services = (await ctx.changelog.sites()).slice(0, 20).map((s) => s.service);
+ return { devblogs: await ctx.changelog.newestDevblogs(services) };
+ }));
+
// Draft with AI: OpenVibe.AI's blog.draft_post writes a draft (AI-authored, noindex until reviewed).
// Draft with AI: OpenVibe.AI's blog.draft_post writes a draft (AI-authored, noindex until reviewed).
router.post('/blogs/:handle/posts/ai-draft', guard('blog.post.create'), B('blog.post.ai_draft'), jsonBody, run(async (req) => {
router.post('/blogs/:handle/posts/ai-draft', guard('blog.post.create'), B('blog.post.ai_draft'), jsonBody, run(async (req) => {
const blog = await mustBlog(req);
const blog = await mustBlog(req);
@@ -118,10 +118,15 @@ route('get', '/blogs/{handle}/posts', {
@@ -118,10 +118,15 @@ route('get', '/blogs/{handle}/posts', {
route('get', '/changelog', {
route('get', '/changelog', {
operationId: 'getChangelog', summary: 'What shipped on every OpenVibe site, newest first', tags: ['changelog'], security: ANONYMOUS, parameters: [
operationId: 'getChangelog', summary: 'What shipped on every OpenVibe site, newest first', tags: ['changelog'], security: ANONYMOUS, parameters: [
query('service', str({ pattern: '^[a-z][a-z0-9-]{1,39}$' }), 'Only this service\'s entries (default: every site)'),
query('service', str({ pattern: '^[a-z][a-z0-9-]{1,39}$' }), 'Only this service\'s entries (default: every site)'),
+ query('kind', { type: 'string', enum: ['new', 'better', 'fixed', 'internal'] }, 'Only this kind of change'),
+ query('area', str({ pattern: '^[a-z][a-z0-9-]{0,39}$' }), 'Only changes touching this area (the response vocabulary lists them)'),
+ query('q', str({ maxLength: 200 }), 'The plain words or the commit subject'),
query('limit', { type: 'integer', minimum: 1 }), query('before', str(), 'The cursor from the previous page')],
query('limit', { type: 'integer', minimum: 1 }), query('before', str(), 'The cursor from the previous page')],
- description: 'Public and CORS *, cached 60 seconds: each site\'s "recently shipped" reads it.',
- responses: { 200: json('Entries, the cursor for the next page and the latest patch notes post', { type: 'object', properties: {
+ description: 'Public and CORS *, cached 60 seconds: each site\'s "recently shipped" reads it. Entries are blog.changelog-entry@2: plain words, kind, areas, time, commit and pull request. facets counts the kinds and areas in the same service and q scope; internal housekeeping is filterable with kind=internal.',
+ responses: { 200: json('Entries, the cursor for the next page, the facets, the vocabulary and the latest patch notes post', { type: 'object', properties: {
service: nullable('string'), entries: arrayOf({ type: 'object' }), next: nullable('string'),
service: nullable('string'), entries: arrayOf({ type: 'object' }), next: nullable('string'),
+ facets: { type: 'object', properties: { kinds: { type: 'object' }, areas: { type: 'object' } } },
+ vocabulary: { type: 'object', properties: { kinds: arrayOf(str()), areas: arrayOf({ type: 'object', properties: { id: str(), label: str() } }) } },
latest_post: nullable('object'), posts: arrayOf({ type: 'object' }), sites: arrayOf({ type: 'object' }),
latest_post: nullable('object'), posts: arrayOf({ type: 'object' }), sites: arrayOf({ type: 'object' }),
} }), ...errors([400, 429]) },
} }), ...errors([400, 429]) },
});
});
@@ -136,6 +141,15 @@ route('post', '/blogs/{handle}/posts/devblog', {
@@ -136,6 +141,15 @@ route('post', '/blogs/{handle}/posts/devblog', {
requestBody: body({ type: 'object', required: ['service'], properties: { service: str({ pattern: '^[a-z][a-z0-9-]{0,63}$' }) } }),
requestBody: body({ type: 'object', required: ['service'], properties: { service: str({ pattern: '^[a-z][a-z0-9-]{0,63}$' }) } }),
responses: { 201: json('The draft', { type: 'object', properties: { post: ref('Post'), created: { type: 'boolean' } } }), 409: problem, 422: problem, 503: problem, ...errors() },
responses: { 201: json('The draft', { type: 'object', properties: { post: ref('Post'), created: { type: 'boolean' } } }), 409: problem, 422: problem, 503: problem, ...errors() },
});
});
+
+route('get', '/changelog/devblog', {
+ operationId: 'getChangelogDevblog', summary: 'The newest devblog post of a site (or of each site)', tags: ['changelog'], security: ANONYMOUS, parameters: [
+ query('service', str({ pattern: '^[a-z][a-z0-9-]{1,39}$' }), 'That site\'s newest devblog post; without it, the newest of each site')],
+ description: 'Public and CORS *: the Rich Content §9 home card and /updates day section. A devblog post is on the official blog, tagged with the service and categorized Devblog; the hero block\'s art is hero_image_url.',
+ responses: { 200: json('{ devblog } for one service, { devblogs } for all', { type: 'object', properties: {
+ service: nullable('string'), devblog: nullable('object'), devblogs: arrayOf({ type: 'object' }),
+ } }), ...errors([400, 429]) },
+});
route('post', '/blogs/{handle}/posts/ai-draft', {
route('post', '/blogs/{handle}/posts/ai-draft', {
operationId: 'draftPostWithAi', summary: 'Have OpenVibe.AI write a draft', tags: ['posts'], security: PERSON, parameters: [HANDLE],
operationId: 'draftPostWithAi', summary: 'Have OpenVibe.AI write a draft', tags: ['posts'], security: PERSON, parameters: [HANDLE],
description: 'blog.post.create. The draft is AI-authored and cannot publish until a person reviews it; 503 when AI drafting is not configured here.',
description: 'blog.post.create. The draft is AI-authored and cannot publish until a person reviews it; 503 when AI drafting is not configured here.',
new file mode 100644
new file mode 100644
@@ -0,0 +1,120 @@
@@ -0,0 +1,120 @@
+'use strict';
+/**
+ * The changelog vocabulary (plan §9): the fixed kinds and areas, the deterministic plain words,
+ * the housekeeping rule that never asks a model, and the tagger's answer parser. The feed's filters
+ * and the blog.changelog-entry@2 contract repeat this exact list, so the count matters.
+ */
+const assert = require('assert');
+const { execFileSync } = require('child_process');
+const path = require('path');
+const { check, done } = require('./helpers/boot');
+const tags = require('../server/changelog/tags');
+const { tagsFrom, parseLoose, createChangelogTagger } = require('../server/changelog/tagger');
+
+(async () => {
+ await check('the vocabulary is 30 fixed areas and four kinds, with no duplicates', async () => {
+ assert.deepStrictEqual(tags.KINDS, ['new', 'better', 'fixed', 'internal']);
+ assert.strictEqual(tags.AREAS.length, 30);
+ assert.strictEqual(new Set(tags.AREA_IDS).size, 30);
+ assert.strictEqual(tags.AREAS.filter((a) => a.id === 'go-live')[0].label, 'Go live');
+ assert.strictEqual(tags.AREAS.filter((a) => a.id === 'machines')[0].label, 'Machines (Link)');
+ assert.ok(tags.AREAS.every((a) => /^[a-z][a-z0-9-]*$/.test(a.id)));
+ const v = tags.vocabulary();
+ assert.deepStrictEqual(v.kinds, tags.KINDS);
+ assert.strictEqual(v.areas.length, 30);
+ });
+
+ await check('plain words drop the PR number and stage tag, and keep the subject', async () => {
+ assert.strictEqual(tags.plainOf('Recent changes card: show the newest devblog with its art (#714)'), 'Recent changes card: show the newest devblog with its art');
+ assert.strictEqual(tags.plainOf('[A6] Add the changelog tags (#61)'), 'Add the changelog tags');
+ assert.strictEqual(tags.plainOf('feat(chat): show the newest devblog'), 'show the newest devblog');
+ assert.strictEqual(tags.plainOf('0.376.0: Complete the Feed contract'), 'Complete the Feed contract');
+ assert.strictEqual(tags.plainOf(''), 'A change');
+ assert.strictEqual(tags.prOf('A change (#714)'), 714);
+ assert.strictEqual(tags.prOf('A change'), null);
+ });
+
+ await check('housekeeping is internal without a model: pins, merges, test/docs/CI-only work', async () => {
+ for (const s of [
+ 'openvibe-shared v4.175.0 (was v4.173.0) (#401)',
+ 'Pin openvibe-contracts 0.379.0',
+ 'Merge branch main',
+ 'Merge pull request #12 from x/y',
+ 'test: cover the filters',
+ 'docs(readme): the new call',
+ 'ci: rerun',
+ 'chore: deps',
+ '0.376.0: Complete the Feed contract',
+ 'Tidy the docs',
+ ]) assert.strictEqual(tags.isHousekeeping(s), true, s);
+ for (const s of ['Fix the chat scroll', 'Recent tools in the launcher', 'Add devblog cards', 'Make sign-in faster']) {
+ assert.strictEqual(tags.isHousekeeping(s), false, s);
+ const initial = tags.initial(s);
+ assert.strictEqual(initial.kind, null, `${s} waits for the model`);
+ assert.ok(initial.plain.length > 0);
+ }
+ const hk = tags.initial('openvibe-shared v4.175.0 (was v4.173.0) (#401)');
+ assert.strictEqual(hk.kind, 'internal');
+ assert.deepStrictEqual(hk.areas, []);
+ assert.strictEqual(hk.plain, 'openvibe-shared v4.175.0 (was v4.173.0)');
+ });
+
+ await check('a model answer is made safe: a known kind, up to three known areas, plain words', async () => {
+ assert.deepStrictEqual(tags.normalizeTag({ kind: 'new', areas: ['video', 'chat'], plain: 'Watch the clip' }), { kind: 'new', areas: ['video', 'chat'], plain: 'Watch the clip' });
+ assert.deepStrictEqual(tags.normalizeTag({ kind: 'fixed', areas: ['chat', 'chat', 'podcast'], plain: 'Fix the chat (#2)' }), { kind: 'fixed', areas: ['chat'], plain: 'Fix the chat' });
+ assert.deepStrictEqual(tags.normalizeTag({ kind: 'internal', areas: ['chat'], plain: 'Pin a library' }), { kind: 'internal', areas: ['chat'], plain: 'Pin a library' });
+ assert.strictEqual(tags.normalizeTag({ kind: 'improvement', areas: ['chat'], plain: 'x' }), null);
+ assert.strictEqual(tags.normalizeTag({ kind: 'new', areas: [], plain: 'x' }), null, 'a non-internal change has an area');
+ assert.strictEqual(tags.normalizeTag({ kind: 'new', areas: ['podcast'], plain: 'x' }), null, 'unknown areas are dropped, leaving none');
+ assert.strictEqual(tags.normalizeTag(null), null);
+ });
+
+ await check('a tagger answer is read from { tags } or from ai.generate\'s JSON text', async () => {
+ const tagsList = [{ id: '1', kind: 'new', areas: ['video'], plain: 'Watch the clip' }];
+ assert.deepStrictEqual(tagsFrom({ tags: tagsList }), tagsList);
+ assert.deepStrictEqual(tagsFrom({ text: JSON.stringify({ tags: tagsList }) }), tagsList);
+ assert.deepStrictEqual(tagsFrom({ text: '```json\n{"tags":[]}\n```' }), []);
+ assert.deepStrictEqual(parseLoose('not json'), null);
+ assert.deepStrictEqual(tagsFrom(null), []);
+ });
+
+ await check('the tagger is on by default and CHANGELOG_AI_TAG=off turns it off', async () => {
+ const config = require('../server/config');
+ assert.strictEqual(config.load({}).changelog.tag, true);
+ assert.strictEqual(config.load({ CHANGELOG_AI_TAG: 'off' }).changelog.tag, false);
+ });
+
+ await check('the tagger asks AI for the blog.changelog.tag task, as a service, and reads its answer', async () => {
+ const seen = { token: null, run: null };
+ const json = (body, status = 200) => ({ ok: status < 300, status, json: async () => body });
+ const fetchImpl = async (url, opts = {}) => {
+ if (String(url).includes('/oauth/token')) { seen.token = String(opts.body); return await json({ access_token: 'svc', token_type: 'Bearer', expires_in: 300 }); }
+ if (String(url).includes('/api/v1/runs')) { seen.run = { url: String(url), body: JSON.parse(opts.body), auth: opts.headers.Authorization }; return await json({ run: { id: 'run_1', status: 'succeeded', workflow: { key: 'blog.changelog.tag' }, output: { tags: [
+ { id: '7', kind: 'new', areas: ['chat', 'video'], plain: 'A calmer chat' },
+ { id: '8', kind: 'improvement', areas: ['chat'], plain: 'dropped: unknown kind' },
+ ] } } }, 201); }
+ return await json({}, 404);
+ };
+ const tagger = createChangelogTagger({ config: {
+ aiUrl: 'http://ai.test', networkInternalUrl: 'http://network.test', oauth: { clientId: 'blog', clientSecret: 'shh' }, changelog: { tag: true },
+ }, fetchImpl });
+ assert.strictEqual(tagger.enabled, true);
+ const answer = await tagger.tag([{ id: 7, service: 'live', subject: 'Add a calmer chat (#9)', repo: 'OpenVibers/OpenVibe.Live' }]);
+ assert.strictEqual(seen.run.body.task, 'blog.changelog.tag');
+ assert.strictEqual(seen.run.body.target.service, 'blog');
+ assert.deepStrictEqual(seen.run.body.input.entries, [{ id: '7', service: 'live', subject: 'Add a calmer chat (#9)', repo: 'OpenVibers/OpenVibe.Live' }]);
+ assert.match(seen.token, /audience=openvibe\.ai/);
+ assert.strictEqual(seen.run.auth, 'Bearer svc');
+ assert.deepStrictEqual(answer.get('7'), { kind: 'new', areas: ['chat', 'video'], plain: 'A calmer chat' });
+ assert.strictEqual(answer.has('8'), false, 'an unknown kind is not written');
+ assert.strictEqual((await createChangelogTagger({ config: { aiUrl: '', networkInternalUrl: '', oauth: {}, changelog: { tag: true } }, fetchImpl }).tag([{ id: 1 }])).size, 0, 'unconfigured is off');
+ });
+
+ await check('the backfill script is safe by default: it prints what it would do and writes nothing', async () => {
+ const out = execFileSync(process.execPath, [path.join(__dirname, '..', 'scripts', 'backfill-changelog-tags.js'), '--help'], { encoding: 'utf8' });
+ assert.match(out, /--apply/);
+ assert.match(out, /--production/);
+ });
+
+ done();
+})();
@@ -38,7 +38,13 @@ const H = (n) => n.toString(16).padStart(40, 'a');
@@ -38,7 +38,13 @@ const H = (n) => n.toString(16).padStart(40, 'a');
// Patch notes v2: the changelog files a DRAFT through the patch-notes module and never publishes.
// Patch notes v2: the changelog files a DRAFT through the patch-notes module and never publishes.
const drafts = [];
const drafts = [];
const patchNotes = { draftPending: async ({ entries, reason }) => { drafts.push({ entries, reason }); return { created: true, post: { id: `pst_draft${drafts.length}`, blog_id: 'blg_test' }, checks: { blocking: [], warnings: [] } }; } };
const patchNotes = { draftPending: async ({ entries, reason }) => { drafts.push({ entries, reason }); return { created: true, post: { id: `pst_draft${drafts.length}`, blog_id: 'blg_test' }, checks: { blocking: [], warnings: [] } }; } };
- const cl = createChangelog({ config, store: t.ctx.store, blogs: t.ctx.blogs, posts: t.ctx.posts, aiDrafts: null, patchNotes, fetchImpl, log: { log() {}, warn() {} } });
+ // The model half is faked: housekeeping must never reach it, and every other entry gets an answer.
+ const tagCalls = [];
+ const tagger = { enabled: true, tag: async (entries) => {
+ tagCalls.push(entries);
+ return new Map(entries.map((e) => [String(e.id), { kind: /^Fix/.test(e.subject) ? 'fixed' : 'better', areas: ['chat'], plain: `plain: ${e.plain}` }]));
+ } };
+ const cl = createChangelog({ config, store: t.ctx.store, blogs: t.ctx.blogs, posts: t.ctx.posts, reading: t.ctx.reading, tagger, aiDrafts: null, patchNotes, fetchImpl, log: { log() {}, warn() {} } });
const commit = (i, msg) => ({ sha: H(i), commit: { message: msg, author: { date: new Date(t.clock.now()).toISOString() } } });
const commit = (i, msg) => ({ sha: H(i), commit: { message: msg, author: { date: new Date(t.clock.now()).toISOString() } } });
await check('a service seen for the first time only records its release', async () => {
await check('a service seen for the first time only records its release', async () => {
@@ -53,9 +59,16 @@ const H = (n) => n.toString(16).padStart(40, 'a');
@@ -53,9 +59,16 @@ const H = (n) => n.toString(16).padStart(40, 'a');
const r = await cl.tick();
const r = await cl.tick();
assert.strictEqual(r, null, 'not due yet');
assert.strictEqual(r, null, 'not due yet');
const e = await cl.entries({ service: 'live' });
const e = await cl.entries({ service: 'live' });
- assert.deepStrictEqual(e.map((x) => x.subject), ['Fix the chat scroll'], 'first lines only; merges skipped');
- assert.strictEqual(e[0].url, `https://github.com/OpenVibers/OpenVibe.Live/commit/${H(1)}`);
- assert.strictEqual(e[0].major, false);
+ const bySubject = new Map(e.map((x) => [x.subject, x]));
+ assert.strictEqual(bySubject.size, 2, 'first lines only');
+ assert.strictEqual(bySubject.get('Fix the chat scroll').kind, 'fixed');
+ assert.deepStrictEqual(bySubject.get('Fix the chat scroll').areas, ['chat']);
+ assert.strictEqual(bySubject.get('Fix the chat scroll').plain, 'plain: Fix the chat scroll');
+ assert.strictEqual(bySubject.get('Merge branch main').kind, 'internal', 'merges are housekeeping');
+ assert.deepStrictEqual(bySubject.get('Merge branch main').areas, []);
+ assert.ok(!tagCalls.flat().some((x) => /^Merge/.test(x.subject)), 'the model is never asked about housekeeping');
+ assert.strictEqual(bySubject.get('Fix the chat scroll').url, `https://github.com/OpenVibers/OpenVibe.Live/commit/${H(1)}`);
+ assert.strictEqual(bySubject.get('Fix the chat scroll').major, false);
});
});
await check('a major release after a quiet spell files a Patch notes draft, never a published post', async () => {
await check('a major release after a quiet spell files a Patch notes draft, never a published post', async () => {
@@ -67,7 +80,7 @@ const H = (n) => n.toString(16).padStart(40, 'a');
@@ -67,7 +80,7 @@ const H = (n) => n.toString(16).padStart(40, 'a');
assert.ok(r && r.post, 'drafted');
assert.ok(r && r.post, 'drafted');
assert.strictEqual(drafts.length, 1);
assert.strictEqual(drafts.length, 1);
assert.strictEqual(drafts[0].reason, 'major');
assert.strictEqual(drafts[0].reason, 'major');
- assert.deepStrictEqual(drafts[0].entries.map((e) => e.subject).sort(), ['Fix the chat scroll', 'Recent tools in the launcher: anonymous and account history', 'Tidy the docs']);
+ assert.deepStrictEqual(drafts[0].entries.map((e) => e.subject).sort(), ['Fix the chat scroll', 'Recent tools in the launcher: anonymous and account history'], 'housekeeping (the merge and docs-only) stays out of the draft');
assert.strictEqual(r.post.state, undefined, 'a draft object from the fake; the real module creates a draft post');
assert.strictEqual(r.post.state, undefined, 'a draft object from the fake; the real module creates a draft post');
assert.strictEqual((await cl.entries()).filter((x) => x.post_id).length, 0, 'nothing is published, so the feed has no post link');
assert.strictEqual((await cl.entries()).filter((x) => x.post_id).length, 0, 'nothing is published, so the feed has no post link');
});
});
@@ -82,6 +95,50 @@ const H = (n) => n.toString(16).padStart(40, 'a');
@@ -82,6 +95,50 @@ const H = (n) => n.toString(16).padStart(40, 'a');
assert.strictEqual(j.latest_post, null, 'nothing is published from the pipeline');
assert.strictEqual(j.latest_post, null, 'nothing is published from the pipeline');
});
});
+ await check('the feed filters by kind, area and q, counts the facets and names the vocabulary', async () => {
+ const one = async (q) => (await t.get(`/api/v1/changelog?service=live&${q}`)).json();
+ assert.deepStrictEqual((await one('kind=fixed')).entries.map((x) => x.subject), ['Fix the chat scroll']);
+ assert.deepStrictEqual((await one('area=chat')).entries.map((x) => x.subject), ['Fix the chat scroll'], 'the merge has no area');
+ assert.deepStrictEqual((await one('kind=internal')).entries.map((x) => x.subject), ['Merge branch main']);
+ assert.deepStrictEqual((await one('q=scroll')).entries.map((x) => x.subject), ['Fix the chat scroll']);
+ const all = await one('limit=20');
+ assert.strictEqual(all.facets.kinds.fixed, 1);
+ assert.strictEqual(all.facets.kinds.internal, 1);
+ assert.strictEqual(all.facets.areas.chat, 1);
+ assert.strictEqual(all.vocabulary.areas.length, 30);
+ assert.ok(all.vocabulary.areas.some((a) => a.id === 'go-live' && a.label === 'Go live'));
+ const badKind = await t.get('/api/v1/changelog?service=live&kind=improvement');
+ assert.strictEqual(badKind.status, 400, badKind.text);
+ assert.strictEqual(badKind.json().code, 'changelog.kind_invalid');
+ const badArea = await t.get('/api/v1/changelog?service=live&area=podcast');
+ assert.strictEqual(badArea.status, 400, badArea.text);
+ assert.strictEqual(badArea.json().code, 'changelog.area_invalid');
+ });
+
+ await check('the newest devblog post of a site is an API call: title, teaser, date, hero image, url', async () => {
+ const MED = 'med_01J8ZQ4X3M000000000000000A';
+ t.media.objects.set(MED, 'ready');
+ const made = await t.get('/api/v1/blogs/openvibe/posts', { as: t.official, json: {
+ title: 'Live this week', summary: 'One encoder, a calmer chat and the new recent-changes card.', categories: ['Devblog'], tags: ['devblog', 'live'],
+ body: `:::hero{art="${MED}" kicker="Devblog"}\n# Live this week\nWhat shipped.\n:::\n\n` + 'The details. '.repeat(30),
+ } });
+ assert.strictEqual(made.status, 201, made.text);
+ assert.strictEqual((await t.get(`/api/v1/posts/${made.json().post.id}/publish`, { as: t.official, json: {} })).status, 200);
+ const r = await t.get('/api/v1/changelog/devblog?service=live');
+ assert.strictEqual(r.status, 200, r.text);
+ assert.strictEqual(r.headers.get('access-control-allow-origin'), '*');
+ const d = r.json().devblog;
+ assert.strictEqual(d.title, 'Live this week');
+ assert.match(d.teaser, /calmer chat/);
+ assert.match(d.hero_image_url, new RegExp(MED));
+ assert.match(d.url, /\/@openvibe\/live-this-week$/);
+ assert.ok(!Number.isNaN(Date.parse(d.date)));
+ const all = (await t.get('/api/v1/changelog/devblog')).json();
+ assert.ok(all.devblogs.some((x) => x.service === 'live' && x.title === 'Live this week'));
+ assert.strictEqual((await t.get('/api/v1/changelog/devblog?service=nope')).json().devblog, null);
+ assert.strictEqual((await t.get('/api/v1/changelog/devblog?service=NOT OK')).status, 400);
+ });
+
await check('a batch publishes without a major, but never sooner than the gap after the last post', async () => {
await check('a batch publishes without a major, but never sooner than the gap after the last post', async () => {
releases.live = 'aaaaaaa00003';
releases.live = 'aaaaaaa00003';
compares.aaaaaaa00003 = { status: 'ahead', commits: [5, 6, 7, 8, 9].map((i) => commit(i, `Small change ${i}`)), files: [{ additions: 5, deletions: 5 }] };
compares.aaaaaaa00003 = { status: 'ahead', commits: [5, 6, 7, 8, 9].map((i) => commit(i, `Small change ${i}`)), files: [{ additions: 5, deletions: 5 }] };
@@ -62,6 +62,7 @@ const LONG = Array.from({ length: 100 }, (_, i) => `w${i}`).join(' ');
@@ -62,6 +62,7 @@ const LONG = Array.from({ length: 100 }, (_, i) => `w${i}`).join(' ');
// openvibe-sdk/events and are no longer imported; they await deletion (the sandbox denies rm).
// openvibe-sdk/events and are no longer imported; they await deletion (the sandbox denies rm).
'server/auth/sso.js': 'Network JWKS, OAuth token and revoke (configured); dead after the SDK move',
'server/auth/sso.js': 'Network JWKS, OAuth token and revoke (configured); dead after the SDK move',
'server/changelog.js': 'Network registry (configured) and api.github.com (fixed host) for the repositories the registry names',
'server/changelog.js': 'Network registry (configured) and api.github.com (fixed host) for the repositories the registry names',
+ 'server/changelog/tagger.js': 'OpenVibe.AI\'s run endpoint (OV_AI_INTERNAL_URL, configured) for the blog.changelog.tag task',
'server/clients/community.js': 'OpenVibe.Community (configured)',
'server/clients/community.js': 'OpenVibe.Community (configured)',
'server/desks/desks.js': 'OpenVibe.Food\'s recipe library feed (OV_FOOD_URL, configured; the budget-recipes desk)',
'server/desks/desks.js': 'OpenVibe.Food\'s recipe library feed (OV_FOOD_URL, configured; the budget-recipes desk)',
'server/domain/ai-drafts.js': 'Publishing AI client calls only configured OV_AI_INTERNAL_URL; the devblog adapter adds a task field, never a caller URL',
'server/domain/ai-drafts.js': 'Publishing AI client calls only configured OV_AI_INTERNAL_URL; the devblog adapter adds a task field, never a caller URL',