openvibers/openvibe.wiki Commits
Vibe a wiki: any topic → a whole wiki with pictures and sources (#104)
Changes
@@@ -61,6 +61,15 @@ OV_OAUTH_CLIENT_SECRET=#OV_MEDIA_INTERNAL_URL=http://127.0.0.1:4100#OV_MEDIA_URL=https://openvibe.media#WIKI_MEDIA_APP=wiki++# OpenVibe.AI — "vibe a wiki" (research.gather, wiki.generate_space, wiki.generate_page). Unset: /vibe answers that it is++# not available. Limits per person and per day, page counts by size, pages written at a time.++#OV_AI_INTERNAL_URL=http://127.0.0.1:4700++#WIKI_VIBE_PER_PERSON_DAY=3++#WIKI_VIBE_PER_DAY=30++#WIKI_VIBE_PAGES_QUICK=6++#WIKI_VIBE_PAGES_STANDARD=12++#WIKI_VIBE_PAGES_BIG=24++#WIKI_VIBE_CONCURRENCY=2#WIKI_MEDIA_VERIFY_INTERVAL_MS=21600000# Scheduled publication worker.@@@ -66,7 +66,8 @@ and, with the workers on (production), flushes it every minute and runs the rete- **Source items** — OpenVibe.Sources owns them; a citation stores the item id plus the URL, title,retrieval time and license read from the item's provenance.- **The network index** — OpenVibe.Search owns it; Wiki sends `wiki.index_document.*` events.--- **Publication truth by AI** — never. There are no AI provider calls in Wiki.++- **Publication truth by AI** — never. There are no AI provider calls in Wiki: generated wikis run OpenVibe.AI's++ registered workflows, and their pages carry the AI disclosure and stay noindex until a person reviews them.- **Identity** — OpenVibe.Network (subjects `usr_…`, SSO, service principals).## What works@@@ -161,6 +162,26 @@ and, with the workers on (production), flushes it every minute and runs the retethe accountable person and where the text comes from; `ai_assisted: true` keeps every page noindexuntil a person reviews it. Existing slugs stop the import (409) unless `on_existing: "skip"`.Twenty imports per hour per address. Format: [server/wiki/import.js](server/wiki/import.js).++- **Vibe a wiki** (`/vibe`, `POST /api/v1/vibes`; [server/wiki/vibe.js](server/wiki/vibe.js)): a signed-in person names++ any topic ("Kim Possible", "ancient Rome") and gets a whole wiki about it, in a new user space they own:++ 1. *Research*: OpenVibe.AI `research.gather` (MegaSource: Wikipedia, the topic's Fandom wiki, Wikidata, DeepSeek web++ search, Commons, Openverse) for the topic.++ 2. *Plan*: `wiki.generate_space` names the wiki and plans its pages (the main page first; quick 6, standard 12, big 24).++ 3. *Write*: per page, its own research (the page's subject inside the topic) and `wiki.generate_page`. The pictures it++ picks are fetched under openvibe-shared/egress ([server/integrations/image-fetch.js](server/integrations/image-fetch.js):++ https, public addresses, 8 MB, real JPEG/PNG/GIF/WebP bytes), stored in OpenVibe.Media for the person (public,++ with source, licence and author in the object's metadata), placed in the text (`[[media:med_…|caption]]`) or the++ infobox (a `media` row), attached, and cited with their licence. The page is imported like any AI-assisted page:++ published with the disclosure, noindex until a person reviews it; the main page lists every planned page++ ("In this wiki"), and the rest hang under it.++++ The job is a `wiki_vibes` row (migration `0004_vibes.sql`) a worker takes under a lease; progress is saved after every++ step, so a restart resumes and pages already made are skipped. AI's quota or queue (429) waits for its Retry-After; a++ page the sources cannot support is left as a red link. `/vibe/:id` shows it live (and refreshes itself without++ JavaScript); `/vibe/:id.json` and `GET /api/v1/vibes/:id` give the same progress. Limits: `WIKI_VIBE_PER_PERSON_DAY`++ (3) and `WIKI_VIBE_PER_DAY` (30); `WIKI_VIBE_PAGES_QUICK|STANDARD|BIG`, `WIKI_VIBE_CONCURRENCY` (2 pages at a time).++ Needs `OV_AI_INTERNAL_URL` and Media; Network grants `wiki` `ai.run.create`/`ai.run.read` (`wiki.*`, `research.*`)++ and `media.object.upload` (namespace `wiki`). Tests: `test/vibe.test.js`.- **Seed**: `npm run seed` imports the official "OpenVibe" space (`seeds/openvibe.json`): 42 pagesabout the network's repositories, summarised from their README/STATUS files at pinned commits,each cited with a GitHub permalink and retrieval time, authorship recorded as `imported` with@@@ -185,7 +206,7 @@ match; network search is OpenVibe.Search).## Routes--Pages (SSR, useful without JavaScript): `/`, `/recent`, `/search?q=`, `/new-space`, `/s/:space`,++Pages (SSR, useful without JavaScript): `/`, `/vibe`, `/vibe/:id` (`.json`), `/recent`, `/search?q=`, `/new-space`, `/s/:space`,`/s/:space/new`, `/s/:space/settings`, `/s/:space/import`, `/s/:space/proposals`, `/w/:space/:slug` (`?rev=N`),`/w/:space/:slug.json`, `/w/:space/:slug/history`, `/w/:space/:slug/sources` (`?rev=N`), `/w/:space/:slug/diff/:a/:b` (`?mode=line`),`/w/:space/:slug/edit`, `/w/:space/:slug/revert?to=N`, `/w/:space/:slug/settings`,@@@ -302,7 +323,8 @@ Production: `/opt/openvibe.wiki`, env `/etc/openvibe/wiki.env`, unitOpenVibe.Sources, OpenVibe.Media, OpenVibe.VIP (VIP spaces and pages), OpenVibe.Search (consumer ofthe index events). All but the Network key are optional at runtime and degrade to explicit failurestates.--- OpenVibe.AI for proposals (not wired to Wiki yet: the proposal API is the seam).++- OpenVibe.AI: generated wikis (`OV_AI_INTERNAL_URL`; research.gather, wiki.generate_space, wiki.generate_page). AI++ proposals still arrive through the proposal API.- Account export and deletion (ADR-033): Network grants `events.subscription.manage` (openvibe.events) for the twosubscriptions created at boot, then, last and once the release is live, `network.account.export.contribute` and`network.account.deletion.confirm` (openvibe.network).nnew file mode 100644@@@ -0,0 +1,27 @@++-- phase: expand++-- "Vibe a wiki" (server/wiki/vibe.js): one row per generated wiki. A worker takes a queued or stalled job under a lease,++-- researches the topic through OpenVibe.AI (research.gather), plans the pages (wiki.generate_space), writes each one++-- (wiki.generate_page) with its pictures copied into OpenVibe.Media, and records progress here as it goes, so a restart++-- resumes where it stopped (pages already made are skipped).++CREATE TABLE IF NOT EXISTS wiki_vibes (++ id text PRIMARY KEY,++ space_id text NOT NULL REFERENCES wiki_spaces (id) ON DELETE CASCADE,++ topic text NOT NULL,++ lang text NOT NULL DEFAULT 'en',++ size text NOT NULL DEFAULT 'standard' CHECK (size IN ('quick', 'standard', 'big')),++ status text NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'researching', 'planning', 'writing', 'done', 'failed', 'cancelled')),++ requested_by text NOT NULL,++ plan jsonb,++ progress jsonb NOT NULL DEFAULT '{}'::jsonb,++ error text,++ attempts integer NOT NULL DEFAULT 0,++ lease_owner text,++ lease_until bigint,++ retry_at bigint,++ created_at bigint NOT NULL,++ updated_at bigint NOT NULL,++ finished_at bigint++);++CREATE INDEX IF NOT EXISTS wiki_vibes_due_idx ON wiki_vibes (status, retry_at) WHERE status IN ('queued', 'researching', 'planning', 'writing');++CREATE INDEX IF NOT EXISTS wiki_vibes_requested_idx ON wiki_vibes (requested_by, created_at DESC);++CREATE INDEX IF NOT EXISTS wiki_vibes_space_idx ON wiki_vibes (space_id);nnew file mode 100644@@@ -0,0 +1,48 @@++/* Vibe a wiki (/vibe, /vibe/:id): the big form, sizes, the steps, the progress of each page. The home's small form++ reuses its search box (wiki.css); the cards of generated wikis are showcase cards. */++.wk-vibe-form { display: grid; gap: 10px; margin-top: 8px; }++.wk-vibe-label { font-size: 14px; font-weight: 600; }++.wk-vibe-row { display: flex; gap: 8px; }++.wk-vibe-row input { flex: 1; min-width: 0; padding: 10px 12px; font-size: 1.05rem; border-radius: 10px; }++.wk-vibe-row button { padding: 10px 18px; border-radius: 10px; white-space: nowrap; }++.wk-vibe-row input:disabled { opacity: .6; }++@media (max-width: 560px) { .wk-vibe-row { flex-direction: column; } .wk-vibe-row button { width: 100%; } }++.wk-vibe-sizes { display: flex; flex-wrap: wrap; gap: 8px; margin: 0; padding: 0; border: 0; }++.wk-vibe-sizes legend { width: 100%; margin-bottom: 6px; font-size: 14px; font-weight: 600; }++.wk-vibe-sizes label { position: relative; flex: 1 1 140px; cursor: pointer; }++.wk-vibe-sizes input { position: absolute; opacity: 0; inset: 0; }++.wk-vibe-sizes span { display: block; height: 100%; padding: 10px 12px; border: 1px solid var(--wk-line); border-radius: 12px; background: var(--wk-panel); transition: border-color .15s, background .15s; }++.wk-vibe-sizes small { display: block; color: var(--wk-muted); font-size: 13px; }++.wk-vibe-sizes input:checked + span { border-color: var(--wk-accent); background: color-mix(in srgb, var(--wk-accent) 10%, var(--wk-panel)); }++.wk-vibe-sizes input:focus-visible + span { outline: 2px solid var(--wk-accent); outline-offset: 2px; }++.wk-vibe h1 span { background: linear-gradient(90deg, var(--wk-accent), color-mix(in srgb, var(--wk-accent) 40%, #f472b6)); -webkit-background-clip: text; background-clip: text; color: transparent; }++.wk-vibe-lede { max-width: 720px; font-size: 1.08rem; color: var(--wk-muted); }++.wk-vibe-how { list-style: none; counter-reset: wk-how; display: grid; gap: 10px; grid-template-columns: repeat(auto-fit, minmax(min(100%, 200px), 1fr)); margin: 22px 0 10px; padding: 0; }++.wk-vibe-how li { counter-increment: wk-how; padding: 12px 14px; border: 1px solid var(--wk-line); border-radius: 12px; background: var(--wk-panel); }++.wk-vibe-how li::before { content: counter(wk-how); display: inline-grid; place-items: center; width: 24px; height: 24px; margin-right: 8px; border-radius: 50%; background: var(--wk-accent); color: var(--on-accent-strong, #0a0f1c); font-size: 13px; font-weight: 800; }++.wk-vibe-how b { font-size: 1rem; }++.wk-vibe-how span { display: block; margin-top: 4px; font-size: 14px; color: var(--wk-muted); }++.wk-vibe-mine { list-style: none; margin: 0; padding: 0; }++.wk-vibe-mine li { display: flex; flex-wrap: wrap; gap: 4px 10px; align-items: baseline; padding: 8px 0; border-bottom: 1px solid var(--wk-line); }++.wk-vp-steps { list-style: none; display: flex; flex-wrap: wrap; gap: 6px; margin: 4px 0 14px; padding: 0; }++.wk-vp-steps li { padding: 5px 12px; border: 1px solid var(--wk-line); border-radius: 999px; font-size: 14px; color: var(--wk-muted); }++.wk-vp-steps .wk-vp-step-done { border-color: color-mix(in srgb, var(--wk-ok, #34d399) 60%, transparent); color: var(--wk-text); }++.wk-vp-steps .wk-vp-step-done::before { content: "✓ "; color: var(--wk-ok, #34d399); }++.wk-vp-steps .wk-vp-step-now { border-color: var(--wk-accent); color: var(--wk-text); background: color-mix(in srgb, var(--wk-accent) 12%, transparent); animation: wk-pulse 1.6s ease-in-out infinite; }++.wk-vp-status { font-weight: 600; }++.wk-vp-read { color: var(--wk-muted); }++.wk-vp-open { padding: 10px 20px; border-radius: 10px; font-size: 1.05rem; }++.wk-vp-pages { list-style: none; margin: 16px 0 0; padding: 0; display: grid; gap: 8px; grid-template-columns: repeat(auto-fill, minmax(min(100%, 230px), 1fr)); }++.wk-vp-pages li { display: flex; flex-direction: column; gap: 2px; padding: 10px 12px; border: 1px solid var(--wk-line); border-radius: 12px; background: var(--wk-panel); transition: border-color .2s; }++.wk-vp-pages li > span { color: var(--wk-muted); }++.wk-vp-pages small { font-size: 12.5px; color: var(--wk-muted); }++.wk-vp-pages .wk-vp-done { border-color: color-mix(in srgb, var(--wk-ok, #34d399) 45%, var(--wk-line)); }++.wk-vp-pages .wk-vp-done a { font-weight: 700; }++.wk-vp-pages .wk-vp-writing { border-color: var(--wk-accent); }++.wk-vp-pages .wk-vp-writing small::before { content: ""; display: inline-block; width: 8px; height: 8px; margin-right: 6px; border-radius: 50%; background: var(--wk-accent); animation: wk-pulse 1s ease-in-out infinite; }++.wk-vp-pages .wk-vp-failed { opacity: .65; }++.wk-vp-wait { display: flex; gap: 6px; padding: 18px 0; }++.wk-vp-dot { width: 10px; height: 10px; border-radius: 50%; background: var(--wk-accent); animation: wk-pulse 1.2s ease-in-out infinite; }++.wk-vp-dot:nth-child(2) { animation-delay: .2s; }++.wk-vp-dot:nth-child(3) { animation-delay: .4s; }++@keyframes wk-pulse { 0%, 100% { opacity: 1; } 50% { opacity: .35; } }@@@ -145,3 +145,19 @@ main .wk-h-group h3 { display: flex; flex-wrap: wrap; align-items: baseline; gap.wk-h-group .wk-h-host { margin-top: 2px; font-size: 12px; }.wk-h-stats dd { font-size: 1.3rem; }}++++/* Infobox labels never break inside a word ("Reposi|tory"); a picture row spans the box. Pictures in the text sit++ beside the words, full width on phones. */++.wk-infobox th { width: 36%; overflow-wrap: normal; word-break: normal; }++.wk-infobox td { overflow-wrap: anywhere; }++.wk-ib-pic td { padding: 0 !important; }++.wk-ib-pic .ov-media { margin: 0; }++.wk-ib-pic img { max-height: 420px; object-fit: contain; background: var(--wk-bg); }++.ov-media img { display: block; width: 100%; height: auto; border-radius: 8px; }++.ov-media figcaption { padding: 6px 4px; font-size: 13px; line-height: 1.45; color: var(--wk-muted); }++.wk-content .ov-media { float: right; clear: right; width: min(300px, 46%); margin: 4px 0 12px 18px; }++.wk-content h2 { clear: both; }++.wk-media .ov-media { display: inline-block; vertical-align: top; width: min(240px, 100%); margin: 0 12px 12px 0; }++@media (max-width: 640px) { .wk-content .ov-media { float: none; width: 100%; margin: 8px 0 12px; } }++.wk-vibe-panel .wk-h-stats { margin-top: 14px; }++.wk-vibe-ideas { margin: 6px 0 0; font-size: 13.5px; color: var(--wk-muted); }@@@ -23,7 +23,7 @@ const { createActorLimits } = require('./http/actor-limits');const PUBLIC_DIR = path.join(__dirname, '..', 'public');const VERSION = require('../package.json').version;--function createApp({ config, telemetry = null, svc, viewers: baseViewers, platform, keys, db, valkey = null, log = console, rateLimits = true, fetchImpl = globalThis.fetch, limitsNow = null, indexnow = null, accountData = null, accountSend = null }) {++function createApp({ config, telemetry = null, svc, viewers: baseViewers, platform, keys, db, valkey = null, log = console, rateLimits = true, fetchImpl = globalThis.fetch, limitsNow = null, indexnow = null, accountData = null, accountSend = null, vibes = null }) {const app = express();// Every resolved actor carries its wiki roles (read once per request), so the synchronous access checks work.const viewers = { ...baseViewers, resolve: async (req, opts) => svc.loadRoles(await baseViewers.resolve(req, opts)) };@@@ -111,7 +111,7 @@ function createApp({ config, telemetry = null, svc, viewers: baseViewers, platfo// Per-actor limits (http/actor-limits.js) on /api/v1 and the editing forms, counted once each router// resolved req.actor; the per-address limits here stay. limitsNow: the limiter's clock (tests).const limits = createActorLimits({ config, now: limitsNow || (() => Date.now()), registry: metrics.registry, log, enabled: rateLimits, valkey });-- app.use('/api/v1', createApi({ svc, viewers, platform, config, log, limits }));++ app.use('/api/v1', createApi({ svc, viewers, platform, config, log, limits, vibes }));app.use('/api', (req, res) => require('openvibe-contracts').http.sendProblem(res, 404, 'route.not_found', { detail: 'Not found' }));// This site's own pinned copy of the OpenVibe Frame's browser files (openvibe-shared/serve).@@@ -133,7 +133,7 @@ function createApp({ config, telemetry = null, svc, viewers: baseViewers, platfoapp.post(['/new-space', '/s/*', '/w/*', '/proposals/*'], limiter(10 * 60000, 120));// Search scans the text of every published page (LIKE): a person's pace, not a crawler's.app.get('/search', limiter(60000, 60));-- const pages = createPages({ svc, viewers, platform, config, log, limits });++ const pages = createPages({ svc, viewers, platform, config, log, limits, vibes });app.use(pages.router);app.use((req, res) => pages.errorPage(req, res, 404, 'Page not found', 'Nothing lives at that address.'));@@@ -71,6 +71,15 @@ function load(env = process.env) {mediaPublicUrl: trim(env.OV_MEDIA_URL || 'https://openvibe.media'),// Media namespace (app) Wiki's attachments live in, checked with media.object.read.mediaApp: env.WIKI_MEDIA_APP || 'wiki',++ // "Vibe a wiki" (server/wiki/vibe.js): OpenVibe.AI researches and writes; these bound how many and how big.++ aiInternalUrl: trim(env.OV_AI_INTERNAL_URL || ''),++ vibe: {++ perPersonDay: int(env.WIKI_VIBE_PER_PERSON_DAY, 3),++ perDay: int(env.WIKI_VIBE_PER_DAY, 30),++ concurrency: Math.max(1, int(env.WIKI_VIBE_CONCURRENCY, 2)),++ intervalMs: int(env.WIKI_VIBE_INTERVAL_MS, 5000),++ pages: { quick: int(env.WIKI_VIBE_PAGES_QUICK, 6), standard: int(env.WIKI_VIBE_PAGES_STANDARD, 12), big: int(env.WIKI_VIBE_PAGES_BIG, 24) },++ },mediaVerifyIntervalMs: int(env.WIKI_MEDIA_VERIFY_INTERVAL_MS, 6 * 60 * 60 * 1000),// OpenVibe.VIP: who may read a VIP space or page (server/integrations/vip.js, WS-K task 8).@@@ -56,6 +56,8 @@ function serviceItself(req) {const BUDGETS = {// A space is a new tree with its own roles: people start a few.'wiki.space.create': { minute: 10, hour: 60 },++ // A generated wiki is a new space and minutes of research and writing (the daily caps are in server/wiki/vibe.js).++ 'wiki.vibe.create': { minute: 3, hour: 10 },// Space settings, roles, and moves, visibility, media and deletes of a page: a person saves a form// now and then.'wiki.space.update': { minute: 30, hour: 300 },@@@ -34,6 +34,10 @@* GET /proposals/:id wiki.page.read* POST /proposals/:id/review wiki.revision.publish { decision: approved|rejected, note?, publish? } a person* GET /search?q=&space=&limit= wiki.search.query++ * POST /vibes wiki.space.create { topic, size?: quick|standard|big, visibility?: public|private, lang? } a person: a whole++ * generated wiki (server/wiki/vibe.js) → 202 { vibe }, then poll++ * GET /vibes the caller's generated wikis (a person)++ * GET /vibes/:vibe wiki.page.read progress: status, pages written, the wiki's address*/const express = require('express');const contracts = require('openvibe-contracts');@@@ -78,7 +82,7 @@ function serializeTree(nodes, svc, space) {function int(v) { const n = Number(v); return Number.isInteger(n) ? n : undefined; }--function createApi({ svc, viewers, platform, config, log = console, limits }) {++function createApi({ svc, viewers, platform, config, log = console, limits, vibes = null }) {const router = express.Router();const ownOrigin = new URL(config.baseUrl).origin;router.use(http.middleware());@@@ -132,6 +136,20 @@ function createApi({ svc, viewers, platform, config, log = console, limits }) {};const pageOut = async (page) => serializePage(page, svc, await svc.spaceById(page.space_id));++ // Generated wikis ("vibe a wiki")++ const vibesOn = () => { if (!vibes) throw new svc.WikiError(503, 'vibe.unavailable', 'Generated wikis are not available here'); return vibes; };++ router.post('/vibes', guard('wiki.space.create'), B('wiki.vibe.create'), R(async (req) => {++ const b = req.body || {};++ const { vibe, space } = await vibesOn().start({ topic: b.topic, size: b.size || 'standard', visibility: b.visibility || 'public', lang: b.lang || 'en' }, req.actor);++ return { vibe: await vibesOn().get(vibe.id, req.actor), space: serializeSpace(space), progress_url: `${config.baseUrl}/vibe/${vibe.id}` };++ }, 202));++ router.get('/vibes', R(async (req) => ({ vibes: await vibesOn().mine(req.actor) })));++ router.get('/vibes/:vibe', guard('wiki.page.read'), R(async (req) => {++ const v = await vibesOn().get(req.params.vibe, req.actor);++ if (!v) throw new svc.WikiError(404, 'vibe.not_found', 'No such generated wiki');++ return { vibe: v };++ }));++// Spacesrouter.get('/spaces', R(async (req) => ({ spaces: (await svc.listSpaces(req.actor)).map(serializeSpace) })));router.post('/spaces', guard('wiki.space.create'), B('wiki.space.create'), R(async (req) => ({ space: serializeSpace(await svc.createSpace(req.body || {}, req.actor)) }), 201));@@@ -4,6 +4,8 @@* form posts; the shared navbar is progressive enhancement.** GET / spaces + recently published++ * GET|POST /vibe "vibe a wiki": any topic → a generated wiki (server/wiki/vibe.js)++ * GET /vibe/:id, /vibe/:id.json its progress (live with JavaScript; refreshes itself without)* GET /recent, /search?q=* GET|POST /new-space* GET /s/:space the page tree@@@ -31,7 +33,7 @@ const ovServe = require('openvibe-shared/serve');const cache = require('openvibe-shared/cache-policy');const seo = require('openvibe-publishing/seo');const ssr = require('openvibe-publishing/ssr');--const { renderPage, AI_SUMMARY } = require('../render/layout');++const { renderPage, AI_SUMMARY, asset } = require('../render/layout');const views = require('../render/views');const frame = require('openvibe-shared/frame');const showcase = require('openvibe-shared/showcase');@@@ -43,7 +45,7 @@ const { createFrameGroups } = require('./frame-groups');// What the home says in search results and share cards (≤ 160 characters).const HOME_DESCRIPTION = 'Read the OpenVibe wiki or start your own: every page keeps its full revision history, lists the sources behind it and has a discussion thread.';--function createPages({ svc, viewers, platform, config, log = console, limits, frameGroups = createFrameGroups({ url: config.frameUrl }) }) {++function createPages({ svc, viewers, platform, config, log = console, limits, vibes = null, frameGroups = createFrameGroups({ url: config.frameUrl }) }) {const router = express.Router();const form = express.urlencoded({ extended: false, limit: '600kb' });router.use(actorMiddleware(viewers, { services: false }));@@@ -126,19 +128,56 @@ function createPages({ svc, viewers, platform, config, log = console, limits, fr// ── Home, recent, search ─────────────────────────────────router.get('/', wrap(async (req, res) => {if (req.actor.subject) await svc.access.prepareVip(req.actor, (await svc.vipSpaces()).map((space) => ({ space }))); // VIP spaces the viewer may see-- const [spaces, recent, stats, linked, groups] = await Promise.all([svc.listSpaces(req.actor), svc.recentChanges(8), svc.homeStats(), svc.mostLinked(6), frameGroups.get()]);++ const [spaces, recent, stats, linked, groups, generated] = await Promise.all([svc.listSpaces(req.actor), svc.recentChanges(8), svc.homeStats(), svc.mostLinked(6), frameGroups.get(), vibes ? vibes.recent(6).catch(() => []) : []]);// The official wikis' shelves: what is inside them, on the front page (only public spaces: the home is cached).const shelves = new Map(await Promise.all(spaces.filter((s) => s.kind === 'official' && s.visibility === 'public').slice(0, 3).map(async (s) => [s.id, await svc.shelf(s, req.actor, 60)])));// The site's one-line AI summary: the shell turns it into the ai-summary meta and the home's// WebPage JSON-LD (openvibe-publishing/layout v1.3.0 forwards `summary`).-- send(req, res, 200, views.home({ spaces, recent, shelves, stats, linked, groups, actor: req.actor }) + frame.shipped({ service: 'wiki', title: 'Recently shipped on OpenVibe.Wiki' }), {++ send(req, res, 200, views.home({ spaces, recent, shelves, stats, linked, groups, generated, actor: req.actor }) + frame.shipped({ service: 'wiki', title: 'Recently shipped on OpenVibe.Wiki' }), {robots: 'index, follow', cache: 'public', active: 'home', path: '/', styles: [showcase.STYLESHEET],summary: AI_SUMMARY, description: HOME_DESCRIPTION,jsonLd: views.homeJsonLd({ shelves, spaces: spaces.filter((s) => s.kind === 'official' && s.visibility === 'public'), baseUrl: config.baseUrl }),});}));// What shipped on OpenVibe.Wiki: the shared update log every OpenVibe site has.++ // ── Vibe a wiki ──────────────────────────────────────────++ const VIBE_STYLES = [showcase.STYLESHEET];++ const VIBE_HEAD = `<link rel="stylesheet" href="${asset('css/vibe.css')}">`;++ const VIBE_DESCRIPTION = 'Name any topic and OpenVibe researches it across the web, then writes you a whole wiki: pages, pictures and the sources behind every claim.';++ const vibeCtx = async (req) => ({++ mine: vibes && req.actor.subject ? await vibes.mine(req.actor, 10).catch(() => []) : [],++ recent: vibes ? await vibes.recent(12).catch(() => []) : [],++ perPersonDay: (config.vibe && config.vibe.perPersonDay) || 3,++ });++ router.get('/vibe', wrap(async (req, res) => {++ const topic = typeof req.query.topic === 'string' ? req.query.topic.slice(0, 120) : '';++ send(req, res, 200, views.vibePage({ actor: req.actor, topic, ...await vibeCtx(req) }), {++ title: 'Vibe a wiki: any topic, a whole wiki with sources', description: VIBE_DESCRIPTION, robots: topic ? 'noindex, follow' : 'index, follow',++ cache: topic ? null : 'public', path: '/vibe', active: 'vibe', styles: VIBE_STYLES, head: VIBE_HEAD,++ });++ }));++ router.post('/vibe', B('wiki.vibe.create'), form, wrap(async (req, res) => {++ if (!req.actor.subject) return needSignIn(req, res, 'Sign in with your OpenVibe account to vibe a wiki.');++ const b = req.body || {};++ try {++ if (!vibes) throw Object.assign(new Error('Generated wikis are not available here'), { status: 503 });++ const { vibe } = await vibes.start({ topic: b.topic, size: b.size || 'standard', visibility: b.visibility || 'public' }, req.actor);++ res.redirect(303, `/vibe/${vibe.id}`);++ } catch (err) {++ if (!err || !err.status || err.status >= 500 && err.status !== 503) throw err;++ send(req, res, err.status, views.vibePage({ actor: req.actor, topic: String(b.topic || '').slice(0, 120), error: err.message, ...await vibeCtx(req) }), { title: 'Vibe a wiki', robots: 'noindex, nofollow', path: '/vibe', active: 'vibe', styles: VIBE_STYLES, head: VIBE_HEAD });++ }++ }));++ router.get('/vibe/:vibe', wrap(async (req, res) => {++ const json = /\.json$/.test(req.params.vibe);++ const id = json ? req.params.vibe.slice(0, -5) : req.params.vibe;++ const v = vibes ? await vibes.get(id, req.actor) : null;++ if (json) return res.status(v ? 200 : 404).set('Cache-Control', cache.htmlHeaders({ private: true })).json(v || { error: 'not_found' });++ if (!v) return notFound(req, res);++ send(req, res, 200, views.vibeProgress({ vibe: v }), { title: v.status === 'done' ? `A wiki about ${v.topic}` : `Vibing a wiki about ${v.topic}`, robots: 'noindex, nofollow', path: `/vibe/${v.id}`, active: 'vibe', styles: VIBE_STYLES, head: VIBE_HEAD });++ }));++router.get('/updates', (req, res) => send(req, res, 200, frame.updatesBody({ service: 'wiki', siteName: 'OpenVibe.Wiki' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>`, { title: 'What shipped on OpenVibe.Wiki', robots: 'index, follow', cache: 'public', path: '/updates' }));router.get('/recent', wrap(async (req, res) => send(req, res, 200, views.recentPage({ items: await svc.recentChanges(100) }), { title: 'Recent changes', robots: 'noindex, follow', cache: 'public', active: 'recent' })));router.get('/search', wrap(async (req, res) => {@@@ -15,6 +15,7 @@ const { load } = require('./config');const { openDb, migrate, createStores } = require('./db');const { createPlatform } = require('./integrations/platform');const { createWikiService } = require('./wiki/service');++const { createVibes } = require('./wiki/vibe');const { createKeyStore } = require('./auth/keys');const { createViewerResolver } = require('./auth/viewer');const { createApp } = require('./app');@@@ -44,7 +45,7 @@ function createLifecycle({ server, db, valkey = null, platform, timers = [], tel});}--async function start({ config, db: givenDb = null, now = () => Date.now(), fetchImpl = globalThis.fetch, tokens = null, publicKey = null, log = console, telemetrySink = null, listen = true, workers = listen, rateLimits = true, limitsNow = null, indexnow: givenIndexNow = null, signals = false, exit = () => {}, accountSend: givenSend = null } = {}) {++async function start({ config, db: givenDb = null, now = () => Date.now(), fetchImpl = globalThis.fetch, tokens = null, imageFetch = null, publicKey = null, log = console, telemetrySink = null, listen = true, workers = listen, rateLimits = true, limitsNow = null, indexnow: givenIndexNow = null, signals = false, exit = () => {}, accountSend: givenSend = null } = {}) {config = config || load();// PostgreSQL (ADR-035): migrate on the owner's direct connection, then serve on the pooled runtime role.// Tests hand in a migrated database of their own (test/db-helper.js).@@@ -70,6 +71,8 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetchconst valkey = config.valkey.url ? require('openvibe-sdk/valkey').createValkey({ url: config.valkey.url, prefix: config.valkey.prefix, log }) : null;const platform = createPlatform({ config, db, fetchImpl, tokens, now, log });const svc = createWikiService({ db, stores, outbox: platform.outbox, config, community: platform.community, vip: platform.vip, now, log, indexnow });++ // "Vibe a wiki" (server/wiki/vibe.js): generated wikis, researched and written through OpenVibe.AI.++ const vibes = createVibes({ db, svc, platform, config, now, log, ...(imageFetch ? { fetchImg: imageFetch } : {}) });const reconciled = await svc.reconcileIndex();if (reconciled.sent) log.log(`[Wiki] re-sent ${reconciled.sent} Search document(s) whose indexability changed`);const keys = createKeyStore({ config, fetchImpl, log, publicKey });@@@ -86,7 +89,7 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch// on, a one-minute tick writes each interval (the collector has no timer of its own) and the table keeps 14 days.const telemetry = createHttpTelemetry({ service: 'wiki', sink: telemetrySink || createPgTelemetrySink({ db, log }), skipPrefixes: ['/shared'], intervalMs: 60_000 });let retention = null;-- const app = createApp({ config, telemetry, svc, viewers, platform, keys, db, valkey, log, rateLimits, fetchImpl, limitsNow, indexnow, accountData, accountSend });++ const app = createApp({ config, telemetry, svc, viewers, platform, keys, db, valkey, log, rateLimits, fetchImpl, limitsNow, indexnow, accountData, accountSend, vibes });const timers = [];if (workers) {@@@ -105,6 +108,7 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetchtimers.push(setInterval(() => svc.verifyAllMedia(platform.media.resolve).catch((err) => log.warn(`[Wiki] media check: ${err.message}`)), config.mediaVerifyIntervalMs));}if (platform.eventsConfigured) platform.outbox.start();++ if (platform.ai.configured && platform.media.configured) vibes.startWorker();timers.push(setInterval(() => platform.outbox.refreshCounts().catch(() => {}), 30 * 1000));// wiki.projects on Network (Contracts 0.41.0): people whose spaces or roles changed, every minute.const projects = require('./integrations/projects-module').createProjectsModule({ db, config, tokens: platform.tokenClient, fetchImpl, now, log });@@@ -126,12 +130,12 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch});}-- const lifecycle = createLifecycle({ server, db, telemetry, retention, valkey, platform, timers, closeDb: !givenDb, signals, exit, log, extra: [() => { if (subscriptions) subscriptions.stop(); }] });++ const lifecycle = createLifecycle({ server, db, telemetry, retention, valkey, platform, timers, closeDb: !givenDb, signals, exit, log, extra: [() => { if (subscriptions) subscriptions.stop(); }, () => vibes.stopWorker()] });async function stop() {return lifecycle.stop('stop');}-- return { app, db, telemetry, svc, stores, platform, keys, viewers, server, config, indexnow, timers, lifecycle, stop };++ return { app, db, telemetry, svc, stores, platform, keys, viewers, server, config, indexnow, timers, lifecycle, stop, vibes };}module.exports = { start, createLifecycle };nnew file mode 100644@@@ -0,0 +1,72 @@++'use strict';++/**++ * Pictures from the web for generated wikis, fetched under openvibe-shared/egress (the network's one rule for addresses++ * someone else chose): https only, no credentials or IP literals in the URL, the connection only to public addresses++ * (egress.safeLookup checks every DNS answer inside the connection's own lookup, so an answer cannot point it inside the++ * network), at most three redirects each judged the same way, a byte cap, a timeout, and bytes that really are a JPEG,++ * PNG, GIF or WebP whatever the server says.++ */++const net = require('net');++const https = require('https');++const egress = require('openvibe-shared/egress');++++const MAX_BYTES = 8 * 1024 * 1024;++const USER_AGENT = 'OpenVibeWiki/1.0 (+https://openvibe.wiki; [email protected])';++const { isPublicAddress } = egress;++++/** The picture type the first bytes show, or null. */++function sniff(buf) {++ if (buf.length >= 3 && buf[0] === 0xff && buf[1] === 0xd8 && buf[2] === 0xff) return 'image/jpeg';++ if (buf.length >= 8 && buf.slice(0, 8).equals(Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]))) return 'image/png';++ if (buf.length >= 6 && /^GIF8[79]a$/.test(buf.slice(0, 6).toString('latin1'))) return 'image/gif';++ if (buf.length >= 12 && buf.slice(0, 4).toString('latin1') === 'RIFF' && buf.slice(8, 12).toString('latin1') === 'WEBP') return 'image/webp';++ return null;++}++++function checkUrl(raw) {++ let u;++ try { u = new URL(String(raw)); } catch { throw new Error('not a URL'); }++ if (u.protocol !== 'https:') throw new Error('only https pictures are fetched');++ if (u.username || u.password) throw new Error('credentials in a URL are refused');++ if (net.isIP(u.hostname.replace(/^\[|\]$/g, ''))) throw new Error('IP-literal hosts are refused');++ if (egress.isInternalName(u.hostname)) throw new Error(`${u.hostname} is not a public host name`);++ return u;++}++++/** GET a picture → { bytes, mime, url } (url: where it was read after redirects). */++function fetchImage(raw, { timeoutMs = 15000, maxBytes = MAX_BYTES, signal = null, redirects = 3, lookup = egress.safeLookup } = {}) {++ return new Promise((resolve, reject) => {++ let u;++ try { u = checkUrl(raw); } catch (err) { return reject(err); }++ const req = https.request(u, { method: 'GET', lookup, headers: { 'User-Agent': USER_AGENT, Accept: 'image/webp,image/png,image/jpeg,image/gif;q=0.9,*/*;q=0.1' }, timeout: timeoutMs });++ const onAbort = () => { req.destroy(new Error('aborted')); };++ if (signal) { if (signal.aborted) return onAbort(); signal.addEventListener('abort', onAbort, { once: true }); }++ const done = (fn, v) => { if (signal) signal.removeEventListener('abort', onAbort); fn(v); };++ req.on('timeout', () => req.destroy(new Error('timed out')));++ req.on('error', (err) => done(reject, err));++ req.on('response', (res) => {++ if ([301, 302, 303, 307, 308].includes(res.statusCode) && res.headers.location) {++ res.resume();++ if (redirects <= 0) return done(reject, new Error('too many redirects'));++ const next = new URL(res.headers.location, u).toString();++ return fetchImage(next, { timeoutMs, maxBytes, signal, redirects: redirects - 1, lookup }).then((v) => done(resolve, v), (e) => done(reject, e));++ }++ if (res.statusCode !== 200) { res.resume(); return done(reject, new Error(`${u.hostname} answered ${res.statusCode}`)); }++ const declared = Number(res.headers['content-length'] || 0);++ if (declared > maxBytes) { res.destroy(); return done(reject, new Error('picture too large')); }++ const chunks = [];++ let n = 0;++ res.on('data', (c) => { n += c.length; if (n > maxBytes) { res.destroy(); req.destroy(); done(reject, new Error('picture too large')); } else chunks.push(c); });++ res.on('error', (err) => done(reject, err));++ res.on('end', () => {++ const bytes = Buffer.concat(chunks);++ const mime = sniff(bytes);++ if (!mime) return done(reject, new Error('not a JPEG, PNG, GIF or WebP picture'));++ done(resolve, { bytes, mime, url: u.toString() });++ });++ });++ req.end();++ });++}++++module.exports = { fetchImage, sniff, isPublicAddress, checkUrl, MAX_BYTES };@@@ -1,7 +1,8 @@'use strict';/*** Clients for the services Wiki composes (roadmap §29): Events (outbox relay), Community-- * (discussion threads), Sources (citation items) and Media (attachment checks). Each call uses a++ * (discussion threads), Sources (citation items), Media (attachment checks, and pictures a generated wiki++ * uploads) and AI (research and writing for "vibe a wiki"). Each call uses a* Network client-credentials token for svc:wiki, minted per audience by the SDK token client.* Hiding the thread of a page that stopped being public needs community.comment.moderate.*@@@ -12,6 +13,7 @@ const { createClient } = require('openvibe-sdk/core');const { createServiceTokenClient } = require('openvibe-sdk/auth');const { createEventsClient, createPgOutbox } = require('openvibe-sdk/events');const { createDiscussionClient } = require('openvibe-publishing/discussion');++const crypto = require('crypto');class IntegrationError extends Error {constructor(status, code, message) { super(message); this.status = status; this.code = code; }@@@ -180,11 +182,83 @@ function createPlatform({ config, db, fetchImpl = globalThis.fetch, tokens = nul},};++ /**++ * A picture into Media for a page (media.object.upload, namespace WIKI_MEDIA_APP): init, bytes, complete (v2 objects),++ * owned by the person it is uploaded for (X-OV-Subject), public, with where it came from in its metadata.++ * → the object id (med_…). Throws IntegrationError on any refusal.++ */++ media.upload = async function upload({ bytes, mime, filename, owner = null, metadata = {}, timeoutMs = 30000 }) {++ if (!media.configured) throw new IntegrationError(503, 'media.unavailable', 'OpenVibe.Media is not configured (OV_MEDIA_INTERNAL_URL)');++ const base = `${config.mediaInternalUrl}/api/v2/${encodeURIComponent(config.mediaApp)}/objects`;++ const auth = await forAudience('openvibe.media').authHeaders();++ const subject = owner && SUBJECT_RE.test(owner) ? { 'X-OV-Subject': owner } : {};++ const call = async (url, init) => {++ let res;++ try { res = await fetchImpl(url, { ...init, signal: AbortSignal.timeout(timeoutMs) }); } catch (err) {++ throw new IntegrationError(503, 'media.unavailable', `Media did not answer: ${err && err.name === 'TimeoutError' ? 'timeout' : err && err.message}`);++ }++ const body = await res.json().catch(() => null);++ if (res.status === 401) forAudience('openvibe.media').invalidate();++ if (res.status < 200 || res.status > 299) throw new IntegrationError(res.status === 413 || res.status === 415 || res.status === 422 ? 422 : 503, 'media.upload_refused', `Media answered ${res.status}${body && (body.code || body.title) ? ` (${body.code || body.title})` : ''}`);++ return body || {};++ };++ const init = await call(base, {++ method: 'POST', headers: { ...auth, ...subject, 'Content-Type': 'application/json', Accept: 'application/json' },++ body: JSON.stringify({ kind: 'asset', visibility: 'public', mime_type: mime, size_bytes: bytes.length, filename: String(filename || 'picture').slice(0, 120), content_hash: crypto.createHash('sha256').update(bytes).digest('hex'), metadata }),++ });++ const id = init.id || (init.object && init.object.id);++ const up = init.upload || {};++ if (!id || !up.url) throw new IntegrationError(503, 'media.upload_refused', 'Media did not give an upload address');++ // The upload and complete addresses may name Media's public origin; Wiki reaches it on the internal one.++ const internal = (u) => { const x = new URL(u, config.mediaInternalUrl); return `${config.mediaInternalUrl}${x.pathname}${x.search}`; };++ await call(internal(up.url), { method: 'PUT', headers: { ...auth, ...subject, 'Content-Type': mime, ...(up.token ? { 'X-Upload-Token': up.token } : {}) }, body: bytes });++ await call(internal(up.complete_url || `${base}/${encodeURIComponent(id)}/complete`), { method: 'POST', headers: { ...auth, ...subject, 'Content-Type': 'application/json', Accept: 'application/json', ...(up.token ? { 'X-Upload-Token': up.token } : {}) }, body: '{}' });++ return id;++ };++++ // ── AI: research and writing for generated wikis (ai.run.create / ai.run.read, namespaces research.* and wiki.*) ──++ const ai = {++ configured: !!(tokenClient && config.aiInternalUrl),++ /**++ * Run a workflow and wait for it: → the run's output. Long runs are polled. Throws IntegrationError with the run's++ * own code when it failed, 429 with retryAfter (seconds) when AI's quota or queue is full.++ */++ async run(workflow, input, { subject = null, idempotencyKey = null, waitMs = 55000, totalMs = 300000, signal = null } = {}) {++ if (!ai.configured) throw new IntegrationError(503, 'ai.unavailable', 'OpenVibe.AI is not configured (OV_AI_INTERNAL_URL)');++ const headers = async () => ({ ...(await forAudience('openvibe.ai').authHeaders()), 'Content-Type': 'application/json', Accept: 'application/json', ...(idempotencyKey ? { 'Idempotency-Key': idempotencyKey } : {}) });++ const call = async (url, init) => {++ let res;++ try { res = await fetchImpl(url, { ...init, signal: signal ? AbortSignal.any([signal, AbortSignal.timeout(waitMs + 15000)]) : AbortSignal.timeout(waitMs + 15000) }); } catch (err) {++ throw new IntegrationError(503, 'ai.unavailable', `OpenVibe.AI did not answer: ${err && err.name === 'TimeoutError' ? 'timeout' : err && err.message}`);++ }++ const body = await res.json().catch(() => null);++ if (res.status === 401) forAudience('openvibe.ai').invalidate();++ if (res.status === 429) throw Object.assign(new IntegrationError(429, (body && body.code) || 'ai.busy', (body && (body.detail || body.title)) || 'OpenVibe.AI is busy'), { retryAfter: Number(res.headers.get('retry-after')) || 60 });++ if (res.status >= 400) throw new IntegrationError(res.status >= 500 ? 503 : res.status, (body && body.code) || 'ai.refused', (body && (body.detail || body.title)) || `OpenVibe.AI answered ${res.status}`);++ return body && body.run;++ };++ const body = { workflow, input, ...(subject ? { on_behalf_of: { type: 'user', id: subject }, attribution: { service: 'wiki', type: 'user', id: subject } } : {}) };++ let run = await call(`${config.aiInternalUrl}/api/v1/runs?wait=${waitMs}`, { method: 'POST', headers: await headers(), body: JSON.stringify(body) });++ const until = Date.now() + totalMs;++ while (run && !['succeeded', 'cached', 'failed', 'cancelled'].includes(run.status)) {++ if (Date.now() > until) throw new IntegrationError(504, 'ai.timeout', `${workflow} did not finish in ${Math.round(totalMs / 1000)} s`);++ await new Promise((r) => setTimeout(r, 3000));++ run = await call(`${config.aiInternalUrl}/api/v1/runs/${encodeURIComponent(run.id)}`, { method: 'GET', headers: await headers() });++ }++ if (!run) throw new IntegrationError(503, 'ai.unavailable', 'OpenVibe.AI answered without a run');++ if (run.status === 'failed' || run.status === 'cancelled') {++ const e = run.error || {};++ throw new IntegrationError(e.code === 'quota.exceeded' || e.code === 'govern.exceeded' ? 429 : 502, e.code || 'ai.run_failed', e.detail || e.message || `${workflow} ${run.status}`);++ }++ return { output: run.output, runId: run.id, synthetic: Boolean(run.synthetic), provenance: run.provenance || null };++ },++ };++// ── VIP: VIP spaces and pages (WS-K task 8) ──const vip = require('./vip').createVip({ config, tokenClient, fetchImpl, now, log });return {-- tokenClient, sdk, events, outbox, community, sources, media, vip,++ tokenClient, sdk, events, outbox, community, sources, media, ai, vip,eventsConfigured: !!(tokenClient && config.eventsUrl),};}@@@ -50,6 +50,7 @@ function navConfig(o, config) {apiBase: NETWORK_URL,links: [{ label: 'Wikis', href: '/', active: o.active === 'home' },++ { label: 'Vibe a wiki', href: '/vibe', active: o.active === 'vibe' },{ label: 'Recent changes', href: '/recent', active: o.active === 'recent' },{ label: 'Search', href: '/search', active: o.active === 'search' },],@@@ -6,7 +6,7 @@const ssr = require('openvibe-publishing/ssr');const showcase = require('openvibe-shared/showcase');const { figureHtml } = require('openvibe-publishing/media');--const { infoboxToText } = require('../wiki/content');++const { infoboxToText, mediaRefs } = require('../wiki/content');const { html, raw } = ssr;const t = (v) => raw(ssr.timeTag(v));@@@ -64,7 +64,7 @@ const FAQ = [{ q: 'How do I follow changes?', a: 'The whole site and each public wiki have Atom and JSON feeds, and signed-in readers can watch a page. Every public page is also served as JSON: add .json to its address.' },];--function home({ spaces, recent, shelves = new Map(), stats = null, linked = [], groups = null, actor }) {++function home({ spaces, recent, shelves = new Map(), stats = null, linked = [], groups = null, generated = [], actor }) {const official = spaces.filter((s) => s.kind === 'official');const user = spaces.filter((s) => s.kind === 'user');const signedIn = !!(actor && actor.subject);@@@ -112,8 +112,9 @@ ${body}<p class="wk-more"><a href="/s/${e(s.slug)}">Every page of ${s.name}, asstats.links ? ['Links between pages', stats.links] : null,].filter(Boolean) : [];const firstOfficial = shelved[0];-- const aside = stats && stats.pages ? html`<div class="wk-h-panel"><h2 class="wk-h-panel-h">The wiki today</h2><dl class="wk-h-stats">${tiles.map(([k, v]) => html`<div><dt>${k}</dt><dd>${v.toLocaleString('en-US')}</dd></div>`)}</dl>--<form class="wk-h-search" action="/search" method="get" role="search"><label for="wk-hq">Search every public page</label><span><input id="wk-hq" name="q" type="search" placeholder="Search pages"><button type="submit">Search</button></span></form></div>` : '';++ const aside = html`<div class="wk-h-panel wk-vibe-panel"><h2 class="wk-h-panel-h">Vibe a wiki</h2>${vibeForm({ actor, compact: true })}++${stats && stats.pages ? html`<dl class="wk-h-stats">${tiles.slice(0, 4).map(([k, v]) => html`<div><dt>${k}</dt><dd>${v.toLocaleString('en-US')}</dd></div>`)}</dl>` : ''}++<form class="wk-h-search" action="/search" method="get" role="search"><label for="wk-hq">Search every public page</label><span><input id="wk-hq" name="q" type="search" placeholder="Search pages"><button type="submit">Search</button></span></form></div>`;const steps = showcase.steps({title: 'How a page comes together',items: [@@@ -133,11 +134,13 @@ ${body}<p class="wk-more"><a href="/s/${e(s.slug)}">Every page of ${s.name}, as].filter(Boolean);return html`${raw(showcase.hero({eyebrow: 'OpenVibe.Wiki',-- title: 'Knowledge,', accent: 'with sources',-- lede: 'Read the OpenVibe wiki or start one of your own. Every page keeps its full revision history, lists the sources behind it and has a discussion thread.',-- actions: [start, firstOfficial ? { label: `Read the ${firstOfficial.name} wiki`, href: `#inside-${firstOfficial.slug}` } : { label: 'Browse the wikis', href: '#spaces' }],-- aside: aside ? { html: String(aside) } : undefined,++ title: 'Any topic.', accent: 'A whole wiki.',++ lede: 'Name a show, a game, a city, a band or an idea: OpenVibe researches it across the web and writes you a wiki with pages, pictures and sources. Or write one by hand. Every page keeps its full history and cites what it rests on.',++ actions: [{ label: 'Vibe a wiki', href: '/vibe', primary: true }, { ...start, primary: false }],++ aside: { html: String(aside) },}))}++${generated && generated.length ? html`<section class="sc-sec" aria-labelledby="wk-gen-h"><h2 id="wk-gen-h">Wikis people vibed</h2><p class="sc-lede">Each one researched and written by AI from the sources it lists, with its pictures credited. Open one, or make your own.</p>++${raw(vibeCards(generated))}<p class="wk-more"><a href="/vibe">Vibe a wiki</a></p></section>` : ''}<div id="spaces"></div>${inside}${linked.length ? html`<section class="sc-sec" aria-labelledby="wk-linked-h"><h2 id="wk-linked-h">Most linked pages</h2><p class="sc-lede">The pages other pages point to most often: a good place to start reading.</p>@@@ -163,6 +166,86 @@ ${raw(steps)}${raw(showcase.cta({ title: 'Start a wiki of your own', text: 'A team handbook, a game guide, a project\'s docs: write it with history and sources from the first page.', actions: [start] }))}`;}++// ── "Vibe a wiki" (server/wiki/vibe.js) ─────────────────────++const VIBE_SIZES = [['quick', 'Quick', 'about 6 pages'], ['standard', 'Standard', 'about 12 pages'], ['big', 'Big', 'about 24 pages']];++const VIBE_IDEAS = ['Kim Possible', 'The Legend of Zelda', 'Studio Ghibli', 'Ancient Rome', 'Octopuses', 'Tokyo', 'Jazz', 'Minecraft'];++const VIBE_STATUS = { queued: 'Waiting to start', researching: 'Researching', planning: 'Planning the pages', writing: 'Writing pages', done: 'Done', failed: 'Stopped', cancelled: 'Cancelled' };++++/** The form: a topic, a size and who can read it. The home's small one is a search-style box (wiki.css .wk-h-search). */++function vibeForm({ actor, topic = '', compact = false, size = 'standard', visibility = 'public' } = {}) {++ const signedIn = !!(actor && actor.subject);++ const off = signedIn ? '' : ' disabled';++ const id = compact ? 'wk-vibe-topic-c' : 'wk-vibe-topic';++ const input = html`<input id="${id}" name="topic" type="text" required minlength="2" maxlength="120" placeholder="Kim Possible, ancient Rome, octopuses…" value="${topic}" autocomplete="off" enterkeyhint="go"${raw(off)}>`;++ const button = html`<button type="submit">${signedIn ? 'Vibe it' : 'Sign in to vibe'}</button>`;++ const ideas = html`<p class="wk-vibe-ideas">${compact ? 'Try ' : 'Ideas: '}${VIBE_IDEAS.slice(0, compact ? 4 : 8).map((t, i) => html`${i ? ', ' : ''}<a href="/vibe?topic=${e(t)}">${t}</a>`)}</p>`;++ const attrs = signedIn ? { action: '/vibe', method: 'post' } : { action: '/auth/login', method: 'get' };++ const next = signedIn ? '' : html`<input type="hidden" name="next" value="/vibe">`;++ if (compact) return html`<form class="wk-h-search wk-vibe-form" action="${attrs.action}" method="${attrs.method}">${next}<label for="${id}">Name any topic: OpenVibe researches it and writes the wiki</label><span>${input}${button}</span>${ideas}</form>`;++ const radio = (name, v, checked, label, hint) => html`<label><input type="radio" name="${name}" value="${v}"${checked ? ' checked' : ''}${raw(off)}><span><b>${label}</b><small>${hint}</small></span></label>`;++ return html`<form class="wk-vibe-form" action="${attrs.action}" method="${attrs.method}">${next}++<label for="${id}" class="wk-vibe-label">Any topic</label><span class="wk-vibe-row">${input}${button}</span>++<fieldset class="wk-vibe-sizes"><legend>How big</legend>${VIBE_SIZES.map(([v, label, hint]) => radio('size', v, v === size, label, hint))}</fieldset>++<fieldset class="wk-vibe-sizes"><legend>Who can read it</legend>${radio('visibility', 'public', visibility !== 'private', 'Everyone', 'a public wiki')}${radio('visibility', 'private', visibility === 'private', 'Just me', 'add people later')}</fieldset>++${ideas}</form>`;++}++++function vibeCards(list) {++ return html`<div class="sc-grid">${list.map((v) => html`<a class="sc-card wk-h-card" href="${v.main_page || v.space.url}">${ringTile('wiki')}<span><b>${v.space.name}</b><small>${plural(v.done, 'page')}${v.research && v.research.sources ? ` · ${plural(v.research.sources, 'source')} read` : ''}</small></span></a>`)}</div>`;++}++++function vibePage({ actor, topic = '', mine = [], recent = [], error = null, perPersonDay = 3 }) {++ return html`${crumbs([{ name: 'Wiki', url: '/' }, { name: 'Vibe a wiki' }])}++<section class="wk-vibe"><h1>Vibe a wiki</h1>++<p class="wk-vibe-lede">Name anything. OpenVibe reads about it across Wikipedia, its fan wiki, Wikidata, the web and open picture libraries, plans the pages a reader would want, and writes each one with its pictures and the sources it rests on. It takes a few minutes; you can watch it happen.</p>++${error ? notice('error', error) : ''}++${vibeForm({ actor, topic })}++<ol class="wk-vibe-how"><li><b>Research</b><span>Pages and pictures about your topic, each kept with its link, licence and when it was read.</span></li><li><b>Plan</b><span>A main page, then the people, places, works and ideas the sources cover.</span></li><li><b>Write</b><span>Every page cites its sources and credits its pictures. Then the wiki is yours to edit.</span></li></ol>++<p class="wk-muted">The text is written by AI and says so on every page; search engines are asked to wait until a person has checked each page. You can vibe ${String(perPersonDay)} wikis a day.</p>++</section>++${mine.length ? html`<section class="sc-sec" aria-labelledby="wk-mine-h"><h2 id="wk-mine-h">Your wikis</h2><ul class="wk-vibe-mine">${mine.map((v) => html`<li><a href="/vibe/${v.id}">${v.space.name}</a> <span class="wk-tag">${VIBE_STATUS[v.status] || v.status}</span> <span class="wk-muted">${v.done} of ${v.total || '…'} pages</span></li>`)}</ul></section>` : ''}++${recent.length ? html`<section class="sc-sec" aria-labelledby="wk-vrecent-h"><h2 id="wk-vrecent-h">Wikis people vibed</h2>${raw(vibeCards(recent))}</section>` : ''}`;++}++++/** The progress page: the steps, what was read, every planned page as it is written. Live with JavaScript, refreshing without. */++function vibeProgress({ vibe: v }) {++ const steps = [['researching', 'Research'], ['planning', 'Plan'], ['writing', 'Write'], ['done', 'Done']];++ const at = Math.max(0, steps.findIndex(([k]) => k === v.status));++ const active = !['done', 'failed', 'cancelled'].includes(v.status);++ const state = (i) => (v.status === 'done' || i < at ? 'done' : i === at && active ? 'now' : 'next');++ const pageRow = (p) => html`<li class="wk-vp-${p.status}" data-title="${p.title}">${p.url ? html`<a href="${p.url}">${p.title}</a>` : html`<span>${p.title}</span>`}<small>${p.status === 'done' ? `${p.sources} sources${p.images ? ` · ${p.images} picture${p.images === 1 ? '' : 's'}` : ''}` : p.status === 'writing' ? 'writing…' : p.status === 'failed' ? 'not enough found' : 'waiting'}</small></li>`;++ return html`${crumbs([{ name: 'Wiki', url: '/' }, { name: 'Vibe a wiki', url: '/vibe' }, { name: v.topic }])}++<section class="wk-vibe wk-vp" id="wk-vp" data-vibe="${v.id}" data-active="${active ? '1' : '0'}">++<h1>${v.status === 'done' ? html`Your wiki about <span>${v.topic}</span> is ready` : html`Vibing a wiki about <span>${v.topic}</span>`}</h1>++<ol class="wk-vp-steps">${steps.map(([, label], i) => html`<li class="wk-vp-step-${state(i)}">${label}</li>`)}</ol>++<p class="wk-vp-status" role="status" aria-live="polite">${VIBE_STATUS[v.status] || v.status}${v.total ? ` · ${v.done} of ${v.total} pages` : ''}${v.waiting_until ? ' · waiting for the AI service, it will carry on by itself' : ''}</p>++${v.error && v.status === 'failed' ? notice('error', v.error) : ''}++${v.research ? html`<p class="wk-vp-read">Read ${plural(v.research.sources, 'source')} and found ${plural(v.research.images, 'picture')}${v.research.from && v.research.from.length ? html` from ${v.research.from.join(', ')}` : ''}${v.research.topic_wiki ? html`, including <a href="${v.research.topic_wiki.url}" rel="nofollow noopener">${v.research.topic_wiki.name}</a>` : ''}.</p>` : ''}++${v.status === 'done' && v.main_page ? html`<p><a class="wk-button wk-vp-open" href="${v.main_page}">Open the wiki</a> <a href="${v.space.url}">See every page</a></p>` : ''}++${v.pages.length ? html`<ol class="wk-vp-pages">${v.pages.map(pageRow)}</ol>` : html`<div class="wk-vp-wait"><span class="wk-vp-dot"></span><span class="wk-vp-dot"></span><span class="wk-vp-dot"></span></div>`}++<noscript>${active ? raw('<meta http-equiv="refresh" content="8">') : ''}<p class="wk-muted">This page refreshes itself every few seconds while the wiki is written.</p></noscript>++</section>++<script>++(function () {++ var root = document.getElementById('wk-vp');++ if (!root || root.getAttribute('data-active') !== '1') return;++ var id = root.getAttribute('data-vibe'), last = '';++ function poll() {++ if (!document.body.contains(root)) return;++ window.fetch('/vibe/' + id + '.json', { credentials: 'same-origin', cache: 'no-store' }).then(function (r) { return r.ok ? r.json() : null; }).then(function (v) {++ if (!v) return setTimeout(poll, 5000);++ var sig = v.status + ':' + v.done + ':' + v.pages.map(function (p) { return p.status; }).join(',');++ if (sig !== last && last) { location.replace(location.pathname); return; }++ last = sig;++ if (['done', 'failed', 'cancelled'].indexOf(v.status) === -1) setTimeout(poll, 2500);++ else location.replace(location.pathname);++ }).catch(function () { setTimeout(poll, 5000); });++ }++ setTimeout(poll, 2500);++})();++</script>`;++}++/** The home's structured data: the site (with its search), the official pages it lists, and the questions it answers. */function homeJsonLd({ shelves = new Map(), spaces = [], baseUrl }) {const SITE = String(baseUrl).replace(/\/+$/, '');@@@ -196,8 +279,12 @@ ${space.visibility === 'public' ? html`<p class="wk-muted">Follow this wiki: <a</section>`;}--function infoboxHtml(entries, resolve, rel = 'nofollow ugc noopener') {++function infoboxHtml(entries, resolve, rel = 'nofollow ugc noopener', { media = null } = {}) {if (!entries.length) return '';++ // A picture row (type media) whose object the page has attached shows the picture across the box, with its caption.++ const picture = (x) => (x.type === 'media' && typeof media === 'function' ? media(x.value) : null);++ const pics = entries.map(picture);++ const rows = entries.map((x, i) => (pics[i] ? html`<tr class="wk-ib-pic" data-key="${x.key}"><td colspan="2">${raw(pics[i])}</td></tr>` : null));const cell = (x) => {switch (x.type) {case 'url': return html`<a href="${x.value}" rel="${rel}">${x.value.replace(/^https?:\/\//, '')}</a>`;@@@ -209,7 +296,7 @@ function infoboxHtml(entries, resolve, rel = 'nofollow ugc noopener') {default: return x.value;}};-- return html`<aside class="wk-infobox" aria-label="Infobox"><table><tbody>${entries.map((x) => html`<tr data-key="${x.key}" data-type="${x.type}"><th scope="row">${x.label}</th><td>${cell(x)}</td></tr>`)}</tbody></table></aside>`;++ return html`<aside class="wk-infobox" aria-label="Infobox"><table><tbody>${entries.map((x, i) => rows[i] || html`<tr data-key="${x.key}" data-type="${x.type}"><th scope="row">${x.label}</th><td>${cell(x)}</td></tr>`)}</tbody></table></aside>`;}function citationsHtml(cites, inspectHref, rel = 'nofollow ugc noopener') {@@@ -250,7 +337,8 @@ const UNAVAILABLE = {not_found: 'This media is no longer available.',};--function mediaHtml(attachments, urlFor) {++function mediaHtml(attachments, urlFor, shown = new Set()) {++ attachments = attachments.filter((a) => !shown.has(a.mediaId));if (!attachments.length) return '';return html`<section class="wk-media"><h2>Media</h2>${attachments.map((a) => raw(figureHtml(a, { urlFor, unavailableText: UNAVAILABLE[a.brokenReason] || UNAVAILABLE.not_found })))}</section>`;}@@@ -288,9 +376,9 @@ function articlePage(v, { html: contentHtml, discussion, actor, mediaUrl, resolv<article class="wk-article" data-page-id="${page.id}" data-revision="${String(rev.number)}"><header><h1>${rev.fields.title || page.title}</h1><nav class="wk-actions" aria-label="Page actions">${actions.map((a, i) => html`${i ? ' · ' : ''}${a}`)}</nav></header>${banners}--${raw(infoboxHtml(v.infobox, resolve, outRel(space)))}++${raw(infoboxHtml(v.infobox, resolve, outRel(space), { media: (id) => { const a = v.attachments.find((x) => x.mediaId === id); return a && mediaUrl ? figureHtml(a, { urlFor: mediaUrl, unavailableText: UNAVAILABLE[a.brokenReason] || UNAVAILABLE.not_found }) : null; } }))}<div class="wk-content">${raw(contentHtml)}</div>--${raw(mediaHtml(v.attachments, mediaUrl))}++${raw(mediaHtml(v.attachments, mediaUrl, new Set([...mediaRefs(v.revision.content), ...v.infobox.filter((x) => x.type === 'media').map((x) => x.value)])))}${raw(citationsHtml(v.citations, `${base}/sources${v.isPublishedRevision ? '' : `?rev=${rev.number}`}`, outRel(space)))}${v.children.length ? html`<section><h2>Subpages</h2><ul>${v.children.map((c) => html`<li><a href="${wpath(space, c)}">${c.title}</a></li>`)}</ul></section>` : ''}${v.backlinks.length ? html`<section><h2>What links here</h2><ul>${v.backlinks.map((b) => html`<li><a href="${wpath(b.space, b.page)}">${b.page.title}</a>${b.space.id !== space.id ? html` <span class="wk-muted">(${b.space.name})</span>` : ''}</li>`)}</ul></section>` : ''}@@@ -519,6 +607,7 @@ function signInPage({ next, message }) {}module.exports = {++ vibePage, vibeProgress, vibeForm,vipGateBody,home, homeJsonLd, spacePage, articlePage, historyPage, sourcesPage, diffPage, editPage, formValuesFromRevision, newSpacePage,spaceSettingsPage, importPage, pageSettingsPage, revertPage, confirmPublishPage, proposalsPage, searchPage, recentPage,@@@ -29,6 +29,7 @@ const TABLES = [{ table: 'wiki_attachment_origins', subject: 'media_owner', file: null, erase: anonymize },{ table: 'wiki_page_revision_purges', subject: 'purged_by', file: null, erase: anonymize },{ table: 'wiki_citation_purges', subject: 'purged_by', file: null, erase: anonymize },++ { table: 'wiki_vibes', subject: 'requested_by', file: 'generated-wikis.json', columns: ['id', 'space_id', 'topic', 'size', 'status', 'created_at', 'finished_at'], order: 'created_at' },{ table: 'wiki_page_reviews', subject: 'reviewer', file: 'reviews.json', columns: ['entity_id', 'revision', 'decision', 'note', 'reviewed_at'], order: 'reviewed_at', erase: { keep: 'a person\'s approval is what lets reviewed text stay published' } },];@@@ -7,6 +7,8 @@* [[Title|label]] with its own label* [[space-slug:Title]] a page in another space (the prefix must look like a space slug:* lowercase letters, digits and dashes; anything else is part of the title)++ * [[media:med_…|caption]] a picture attached to the page, shown where it stands (on a line of its own it is a++ * figure); one the page does not have, or that Media no longer shares, shows nothing* Markdown itself goes through openvibe-publishing/ssr (escaped first, a fixed tag set, no raw HTML).*/const ssr = require('openvibe-publishing/ssr');@@@ -17,6 +19,7 @@ const SPACE_PREFIX_RE = /^([a-z0-9][a-z0-9-]{0,62}):(.+)$/;const SPACE_SLUG_RE = /^[a-z0-9][a-z0-9-]{1,62}$/;const PAGE_SLUG_RE = /^[\p{Ll}\p{Lo}\p{N}][\p{Ll}\p{Lo}\p{N}-]{0,79}$/u;const MAX_LINKS = 500;++const MEDIA_TOKEN_RE = /^\s*media:(med_[0-9A-HJKMNP-TV-Z]{26})\s*$/;class ContentError extends Error {constructor(code, message) { super(message); this.status = 422; this.code = code; }@@@ -49,11 +52,12 @@ function mapOutsideCode(source, fn) {}).join('\n');}--/** Every distinct [[link]] in the content: [{ space, slug, title, label }]. */++/** Every distinct [[link]] in the content: [{ space, slug, title, label }]. Pictures ([[media:…]]) are not links. */function extractLinks(source, currentSpace) {const seen = new Map();mapOutsideCode(source, (text) => {text.replace(LINK_RE, (_m, target, label) => {++ if (MEDIA_TOKEN_RE.test(target)) return '';const t = parseTarget(target, currentSpace);if (t && seen.size < MAX_LINKS) {const key = `${t.space}\u0000${t.slug}`;@@@ -66,13 +70,32 @@ function extractLinks(source, currentSpace) {return [...seen.values()];}++/** Media object ids placed in the content with [[media:…]], in order of first appearance. */++function mediaRefs(source) {++ const ids = [];++ mapOutsideCode(source, (text) => {++ text.replace(LINK_RE, (_m, target) => { const m = MEDIA_TOKEN_RE.exec(target); if (m && !ids.includes(m[1])) ids.push(m[1]); return ''; });++ return text;++ });++ return ids;++}++/*** Markdown + [[links]] → HTML. resolve({ space, slug }) → { href, exists } (exists false = red link;-- * href null = no link at all, e.g. a reader who cannot create the page).++ * href null = no link at all, e.g. a reader who cannot create the page). media(mediaId, caption) → the figure's HTML,++ * or null when the page has no such picture to show (then nothing is shown).*/--function renderContent(source, { currentSpace, resolve, rel = 'noopener' } = {}) {++function renderContent(source, { currentSpace, resolve, rel = 'noopener', media = null } = {}) {const held = [];++ const figures = new Set();const withTokens = mapOutsideCode(source, (text) => text.replace(LINK_RE, (m, target, label) => {++ const pic = MEDIA_TOKEN_RE.exec(target);++ if (pic) {++ const figure = typeof media === 'function' ? media(pic[1], label ? label.trim() : null) : null;++ held.push(figure || '');++ figures.add(held.length - 1);++ return `\u0001WL${held.length - 1}\u0001`;++ }const t = parseTarget(target, currentSpace);if (!t) return m;const r = resolve(t) || { href: null, exists: false };@@@ -84,7 +107,9 @@ function renderContent(source, { currentSpace, resolve, rel = 'noopener' } = {})held.push(htmlLink);return `\u0001WL${held.length - 1}\u0001`;}));-- const html = ssr.renderMarkdown(withTokens, { rel, headingShift: 0 });++ const html = ssr.renderMarkdown(withTokens, { rel, headingShift: 0 })++ // A picture on a line of its own is a figure, not a paragraph.++ .replace(/<p>\s*\u0001WL(\d+)\u0001\s*<\/p>/g, (m, n) => (figures.has(Number(n)) ? held[Number(n)] : m));return html.replace(/\u0001WL(\d+)\u0001/g, (_m, n) => held[Number(n)] || '');}@@@ -192,7 +217,7 @@ function infoboxToText(entries) {}module.exports = {-- ContentError, pageSlug, parseTarget, extractLinks, renderContent, mapOutsideCode,++ ContentError, pageSlug, parseTarget, extractLinks, renderContent, mapOutsideCode, mediaRefs,normalizeInfobox, parseInfoboxText, infoboxToText, keyFromLabel,TYPES, SPACE_SLUG_RE, PAGE_SLUG_RE, MEDIA_RE,};@@@ -32,7 +32,7 @@ function firstSentence(text) {if (m) return m[1];return t.length <= 160 ? t : `${t.slice(0, 160).replace(/\s+\S*$/, '')}…`;}--const { isMediaId } = require('openvibe-publishing/media');++const { isMediaId, figureHtml } = require('openvibe-publishing/media');const { ulid } = require('openvibe-contracts').ids;const content = require('./content');const importer = require('./import');@@@ -1108,11 +1108,21 @@ function createWikiService({ db, stores, outbox, config, community = null, vip =* are edited by staff, who vouch for their links. Every [[link]] is resolved first (the* renderer itself is synchronous).*/-- async renderRevision(space, rev, actor) {++ /**++ * A revision's HTML. With attachments and mediaUrl, [[media:…]] shows the page's own pictures where they stand++ * (an attachment Media no longer shares shows its placeholder; an id the page does not have shows nothing).++ */++ async renderRevision(space, rev, actor, { attachments = null, mediaUrl = null } = {}) {const targets = content.extractLinks(rev.content, space.slug);const resolved = new Map();for (const t of targets) resolved.set(`${t.space}\u0000${t.slug}`, await svc.resolveLink(t, actor));-- return content.renderContent(rev.content, { currentSpace: space.slug, resolve: (t) => resolved.get(`${t.space}\u0000${t.slug}`) || { href: null, exists: false }, rel: svc.linkRel(space) });++ const byMedia = new Map((attachments || []).map((a) => [a.mediaId, a]));++ const media = mediaUrl ? (id, caption) => {++ const a = byMedia.get(id);++ if (!a) return null;++ return figureHtml({ ...a, caption: caption || a.caption }, { urlFor: mediaUrl, unavailableText: 'This picture is no longer available.' });++ } : null;++ return content.renderContent(rev.content, { currentSpace: space.slug, resolve: (t) => resolved.get(`${t.space}\u0000${t.slug}`) || { href: null, exists: false }, rel: svc.linkRel(space), media });},linkRel(space) { return space && space.kind === 'official' ? 'noopener' : 'nofollow ugc noopener'; },nnew file mode 100644@@@ -0,0 +1,452 @@++'use strict';++/**++ * "Vibe a wiki": a person names any topic and gets a whole wiki about it, researched from the web, written page by page++ * with sources and pictures, in a new wiki they own.++ *++ * 1. research OpenVibe.AI research.gather (MegaSource: Wikipedia, the topic's Fandom wiki, Wikidata, DeepSeek web++ * search, Commons, Openverse) for the topic as a whole++ * 2. plan wiki.generate_space: the wiki's name, summary and pages (main page first), from that research++ * 3. write for each page, its own research (the page's subject inside the topic) and wiki.generate_page; the++ * pictures it picks are fetched safely (integrations/image-fetch.js), stored in OpenVibe.Media for the++ * person (public, with where they came from), placed in the text and attached; the page is imported++ * like any AI-assisted page (published with the disclosure, noindex until a person reviews it)++ *++ * The wiki's pages say the text was written with AI and cite every source they rest on, pictures included (licence and++ * credit). Nothing is invented: a page the sources cannot support is not written (it stays a red link to create).++ *++ * The job is one wiki_vibes row a worker takes under a lease. Progress is saved after every step, so a restart or a++ * refusal resumes where it stopped (pages already made are skipped); AI's quota or queue being full (429) waits for its++ * Retry-After. Limits: WIKI_VIBE_PER_PERSON_DAY and WIKI_VIBE_PER_DAY.++ */++const crypto = require('crypto');++const { ulid } = require('openvibe-contracts').ids;++const content = require('./content');++const importer = require('./import');++const { fetchImage } = require('../integrations/image-fetch');++++const SIZES = ['quick', 'standard', 'big'];++const ACTIVE = ['queued', 'researching', 'planning', 'writing'];++const LEASE_MS = 10 * 60 * 1000;++const DAY_MS = 24 * 60 * 60 * 1000;++const MAX_IMAGES_PER_PAGE = 3;++// How the main page groups the rest, by what the plan says each page is about.++const GROUP = { character: 'Characters', person: 'People', episode: 'Episodes', season: 'Seasons', work: 'Works', place: 'Places', group: 'Groups and organizations',++ object: 'Things', concept: 'Ideas and themes', event: 'Events', production: 'Behind the scenes', reception: 'Reception', list: 'Lists', other: 'More' };++// What research.gather's providers are called where people read them.++const SOURCE_NAMES = { wikipedia: 'Wikipedia', fandom: 'Fandom', wikidata: 'Wikidata', deepseek: 'the web', web: 'the web', commons: 'Wikimedia Commons', openverse: 'Openverse', wikipedia_images: 'Wikipedia', wikidata_image: 'Wikidata' };++const sourceNames = (list) => [...new Set((list || []).map((k) => SOURCE_NAMES[k] || k))];++const GROUP_ORDER = ['character', 'person', 'work', 'season', 'episode', 'place', 'group', 'object', 'concept', 'event', 'production', 'reception', 'list', 'other'];++++class VibeError extends Error {++ constructor(status, code, message, extra) { super(message); this.status = status; this.code = code; if (extra) Object.assign(this, extra); }++}++++const oneLine = (s, n) => String(s == null ? '' : s).replace(/\s+/g, ' ').trim().slice(0, n);++const clip = (s, n) => { const t = String(s || '').trim(); return t.length <= n ? t : `${t.slice(0, n).replace(/\s+\S*$/, '')}…`; };++++/** An ASCII space slug for a topic ("Pokémon Red" → "pokemon-red"), or null when nothing Latin is left. */++function spaceSlugFor(topic) {++ const s = String(topic || '').normalize('NFKD').replace(/[̀-ͯ]/g, '').toLowerCase()++ .replace(/['’]/g, '').replace(/[^a-z0-9]+/g, '-').replace(/^-+|-+$/g, '').slice(0, 50).replace(/-+$/, '');++ return s.length >= 2 ? s : null;++}++++/** The plan as the job keeps it: unique titles (by slug), the main page first, at most `max` pages. */++function normalizePlan(output, topic, max) {++ const pages = [];++ const seen = new Set();++ const add = (p) => {++ const title = oneLine(p && p.title, 120).replace(/[[\]|]/g, '').trim();++ if (!title) return;++ let slug;++ try { slug = content.pageSlug(title); } catch { return; }++ if (seen.has(slug)) return;++ seen.add(slug);++ pages.push({++ title, slug,++ kind: GROUP[p.kind] || p.kind === 'overview' ? p.kind : 'other',++ parent: oneLine(p.parent, 120),++ outline: (Array.isArray(p.outline) ? p.outline : []).map((h) => oneLine(h, 80)).filter(Boolean).slice(0, 8),++ query: oneLine(p.query, 120),++ status: 'waiting',++ });++ };++ const list = Array.isArray(output && output.pages) ? output.pages : [];++ const main = list.find((p) => p && p.kind === 'overview') || list[0] || { title: topic };++ add({ ...main, kind: 'overview', parent: '' });++ for (const p of list) if (p !== main && pages.length < max) add(p);++ return {++ title: oneLine(output && output.title, 120) || oneLine(topic, 120),++ summary: clip(output && output.summary, 1000),++ pages,++ };++}++++/** The sources a page cited (indices in any `citations` array of the writer's output); all of them when it cited none. */++function citedIndices(output, count) {++ const set = new Set();++ const walk = (v) => {++ if (Array.isArray(v)) return v.forEach(walk);++ if (!v || typeof v !== 'object') return;++ for (const [k, x] of Object.entries(v)) {++ if (k === 'citations' && Array.isArray(x)) x.forEach((i) => { if (Number.isInteger(i) && i >= 0 && i < count) set.add(i); });++ else walk(x);++ }++ };++ walk(output);++ return set.size ? [...set].sort((a, b) => a - b) : [...Array(count).keys()];++}++++/** A picture's credit line: who made it or where it is from, and its licence. */++function creditOf(img) {++ let site = null;++ try { site = img.source_url ? new URL(img.source_url).hostname.replace(/^www\./, '') : null; } catch { /* no site */ }++ const who = oneLine(img.author, 80) || site || oneLine(img.provider, 40);++ return [who ? `Picture: ${who}` : 'Picture', img.license ? oneLine(img.license, 80) : null].filter(Boolean).join(', ');++}++++/** The Markdown of one page from the writer's output: lead, sections with their pictures, and the main page's contents. */++function composeBody(out, { placed = new Map(), contents = null }) {++ const parts = [];++ const lead = String(out.lead || '').trim();++ if (lead) parts.push(lead);++ const used = new Set();++ for (const sec of Array.isArray(out.sections) ? out.sections : []) {++ const heading = oneLine(sec.heading, 120).replace(/^#+\s*/, '');++ const body = String(sec.body || '').trim().replace(/^#{1,2}\s/gm, '### ');++ if (!heading || !body) continue;++ const key = heading.toLowerCase();++ const pic = placed.get(key);++ parts.push(`## ${heading}${pic && !used.has(key) ? `\n\n[[media:${pic.mediaId}|${pic.caption.replace(/[[\]|]/g, '')}]]` : ''}\n\n${body}`);++ if (pic) used.add(key);++ }++ // Pictures for headings the writer did not use go under the lead.++ const loose = [...placed.entries()].filter(([k]) => k !== 'lead' && !used.has(k)).map(([, p]) => `[[media:${p.mediaId}|${p.caption.replace(/[[\]|]/g, '')}]]`);++ if (loose.length) parts.splice(lead ? 1 : 0, 0, loose.join('\n\n'));++ if (contents) parts.push(contents);++ return parts.join('\n\n').replace(/\n{3,}/g, '\n\n').trim();++}++++/** The main page's "In this wiki": every other planned page, grouped by kind (links turn blue as pages are written). */++function contentsOf(plan) {++ const rest = plan.pages.slice(1);++ if (!rest.length) return null;++ const groups = new Map();++ for (const p of rest) { const k = GROUP[p.kind] ? p.kind : 'other'; if (!groups.has(k)) groups.set(k, []); groups.get(k).push(p); }++ const blocks = GROUP_ORDER.filter((k) => groups.has(k)).map((k) => `### ${GROUP[k]}\n\n${groups.get(k).map((p) => `- [[${p.title}]]`).join('\n')}`);++ return `## In this wiki\n\n${blocks.join('\n\n')}`;++}++++function createVibes({ db, svc, platform, config, now = () => Date.now(), log = console, fetchImg = fetchImage }) {++ const cfg = config.vibe || {};++ const pagesFor = (size) => (cfg.pages && cfg.pages[size]) || { quick: 6, standard: 12, big: 24 }[size] || 12;++ const owner = `wiki-${process.pid}-${crypto.randomBytes(3).toString('hex')}`;++ const q = {++ insert: db.prepare(`INSERT INTO wiki_vibes (id, space_id, topic, lang, size, status, requested_by, progress, created_at, updated_at)++ VALUES (@id, @space_id, @topic, @lang, @size, 'queued', @requested_by, '{}'::jsonb, @now, @now)`),++ get: db.prepare('SELECT * FROM wiki_vibes WHERE id = ?'),++ bySpace: db.prepare('SELECT * FROM wiki_vibes WHERE space_id = ? ORDER BY created_at DESC LIMIT 1'),++ mine: db.prepare('SELECT * FROM wiki_vibes WHERE requested_by = ? ORDER BY created_at DESC LIMIT ?'),++ countMine: db.prepare('SELECT COUNT(*)::int AS n FROM wiki_vibes WHERE requested_by = ? AND created_at > ?'),++ countAll: db.prepare('SELECT COUNT(*)::int AS n FROM wiki_vibes WHERE created_at > ?'),++ recentDone: db.prepare(`SELECT v.* FROM wiki_vibes v JOIN wiki_spaces s ON s.id = v.space_id++ WHERE v.status = 'done' AND s.deleted_at IS NULL AND s.visibility = 'public' ORDER BY v.finished_at DESC LIMIT ?`),++ claim: db.prepare(`UPDATE wiki_vibes SET lease_owner = @owner, lease_until = @until, updated_at = @now++ WHERE id = (SELECT id FROM wiki_vibes WHERE status IN ('queued','researching','planning','writing')++ AND (retry_at IS NULL OR retry_at <= @now) AND (lease_until IS NULL OR lease_until < @now)++ ORDER BY created_at LIMIT 1 FOR UPDATE SKIP LOCKED)++ RETURNING *`),++ renew: db.prepare('UPDATE wiki_vibes SET lease_until = ? WHERE id = ? AND lease_owner = ?'),++ save: db.prepare(`UPDATE wiki_vibes SET status = @status, plan = @plan::jsonb, progress = @progress::jsonb, error = @error, attempts = @attempts,++ retry_at = @retry_at, lease_owner = @lease_owner, lease_until = @lease_until, updated_at = @now, finished_at = @finished_at WHERE id = @id`),++ };++++ function requirePerson(actor) {++ if (!actor || actor.kind !== 'user' || !/^usr_/.test(String(actor.subject || ''))) throw new VibeError(401, 'vibe.sign_in', 'Sign in with your OpenVibe account to vibe a wiki');++ return actor.subject;++ }++++ async function freeSlug(base) {++ const root = base || `wiki-${crypto.randomBytes(3).toString('hex')}`;++ for (const s of [root, `${root}-wiki`, ...[2, 3, 4, 5, 6].map((n) => `${root}-${n}`), `${root}-${crypto.randomBytes(2).toString('hex')}`]) {++ if (!content.SPACE_SLUG_RE.test(s)) continue;++ if (!await svc.findSpace(s)) return s;++ }++ throw new VibeError(409, 'vibe.slug_taken', 'Every address for this topic is taken; try naming it differently');++ }++++ /** Start one: checks, the person's new wiki, the job. → { vibe, space } */++ async function start({ topic, size = 'standard', visibility = 'public', lang = 'en' } = {}, actor) {++ const who = requirePerson(actor);++ const name = oneLine(topic, 120);++ if (name.replace(/[^\p{L}\p{N}]/gu, '').length < 2) throw new VibeError(422, 'vibe.invalid_topic', 'Name a topic: a show, a game, a place, a person, an idea');++ if (!SIZES.includes(size)) throw new VibeError(422, 'vibe.invalid_size', 'size is quick, standard or big');++ if (!['public', 'private'].includes(visibility)) throw new VibeError(422, 'vibe.invalid_visibility', 'visibility is public or private');++ const language = /^[a-z]{2,3}$/.test(lang || '') ? lang : 'en';++ if (!platform.ai.configured || !platform.media.configured) throw new VibeError(503, 'vibe.unavailable', 'Generating wikis needs OpenVibe.AI and OpenVibe.Media, and one of them is not configured here');++ const since = now() - DAY_MS;++ if ((await q.countMine.get(who, since)).n >= (cfg.perPersonDay || 3)) throw new VibeError(429, 'vibe.daily_limit', `You can vibe ${cfg.perPersonDay || 3} wikis a day; try again tomorrow`, { retryAfter: 3600 });++ if ((await q.countAll.get(since)).n >= (cfg.perDay || 30)) throw new VibeError(429, 'vibe.busy', 'Today\'s wikis are all being written; try again later', { retryAfter: 3600 });++ const slug = await freeSlug(spaceSlugFor(name));++ const space = await svc.createSpace({ slug, name, description: `A wiki about ${name}, being researched and written by AI.`, kind: 'user', visibility }, actor);++ const id = `vib_${ulid(now())}`;++ await q.insert.run({ id, space_id: space.id, topic: name, lang: language, size, requested_by: who, now: now() });++ if (timer) setImmediate(() => { tick().catch((err) => log.warn(`[Vibe] ${err && err.message}`)); }); // a running worker starts it now++ return { vibe: await q.get.get(id), space };++ }++++ /** The public shape of a job (what /vibe/:id and the API show). */++ function present(v, space) {++ const progress = v.progress || {};++ const pages = (v.plan && v.plan.pages) || [];++ const state = progress.pages || {};++ const list = pages.map((p) => ({ title: p.title, kind: p.kind, status: (state[p.slug] && state[p.slug].status) || 'waiting', url: state[p.slug] && state[p.slug].status === 'done' ? `/w/${encodeURIComponent(space.slug)}/${encodeURIComponent(p.slug)}` : null, images: (state[p.slug] && state[p.slug].images) || 0, sources: (state[p.slug] && state[p.slug].sources) || 0 }));++ return {++ id: v.id, topic: v.topic, size: v.size, status: v.status,++ space: { slug: space.slug, name: space.name, url: `/s/${encodeURIComponent(space.slug)}`, visibility: space.visibility },++ research: progress.research ? { sources: progress.research.sources.length, images: progress.research.images.length, from: sourceNames(progress.research.from), topic_wiki: progress.research.topic_wiki || null } : null,++ pages: list,++ done: list.filter((p) => p.status === 'done').length,++ total: list.length,++ error: v.status === 'failed' ? v.error : null,++ waiting_until: v.retry_at && v.retry_at > now() ? new Date(Number(v.retry_at)).toISOString() : null,++ created_at: new Date(Number(v.created_at)).toISOString(),++ finished_at: v.finished_at ? new Date(Number(v.finished_at)).toISOString() : null,++ main_page: list[0] && list[0].url ? list[0].url : null,++ };++ }++++ async function get(id, actor) {++ const v = /^vib_[0-9A-HJKMNP-TV-Z]{26}$/.test(String(id || '')) ? await q.get.get(id) : null;++ if (!v) return null;++ const space = await svc.spaceById(v.space_id);++ if (!space || space.deleted_at) return null;++ await svc.access.loadRoles(actor || { kind: 'anonymous' }, { fresh: true });++ if (!svc.access.canReadSpace(space, actor || { kind: 'anonymous' })) return null;++ return present(v, space);++ }++++ async function mine(actor, limit = 20) {++ const who = requirePerson(actor);++ const rows = await q.mine.all(who, limit);++ const out = [];++ for (const v of rows) { const space = await svc.spaceById(v.space_id); if (space && !space.deleted_at) out.push(present(v, space)); }++ return out;++ }++++ async function recent(limit = 12) {++ const rows = await q.recentDone.all(limit);++ const out = [];++ for (const v of rows) { const space = await svc.spaceById(v.space_id); if (space && !space.deleted_at) out.push(present(v, space)); }++ return out;++ }++++ // ── The worker ──────────────────────────────────────────++ let running = false;++ let timer = null;++++ async function save(v, patch = {}) {++ Object.assign(v, patch);++ await q.save.run({++ id: v.id, status: v.status, plan: JSON.stringify(v.plan || null), progress: JSON.stringify(v.progress || {}), error: v.error || null,++ attempts: v.attempts || 0, retry_at: v.retry_at || null, lease_owner: v.lease_owner || null, lease_until: v.lease_until || null,++ now: now(), finished_at: v.finished_at || null,++ });++ }++++ async function tick() {++ if (running) return false;++ running = true;++ try {++ const v = await q.claim.get({ owner, until: now() + LEASE_MS, now: now() });++ if (!v) return false;++ await runJob(v);++ return true;++ } finally { running = false; }++ }++++ function startWorker() {++ if (timer) return;++ const loop = () => { tick().then((worked) => { timer = setTimeout(loop, worked ? 200 : (cfg.intervalMs || 5000)); timer.unref && timer.unref(); }, (err) => { log.warn(`[Vibe] worker: ${err && err.message}`); timer = setTimeout(loop, cfg.intervalMs || 5000); timer.unref && timer.unref(); }); };++ timer = setTimeout(loop, 1000);++ timer.unref && timer.unref();++ }++ function stopWorker() { if (timer) clearTimeout(timer); timer = null; }++++ const sourceForAi = (s, chars) => ({ ...s, content: clip(s.content, chars), snippet: s.snippet ? clip(s.snippet, 400) : undefined });++++ async function runJob(v) {++ const actor = { kind: 'user', subject: v.requested_by, staff: false };++ const space = await svc.spaceById(v.space_id);++ if (!space || space.deleted_at) { await save(v, { status: 'cancelled', error: 'The wiki was deleted', lease_owner: null, lease_until: null, finished_at: now() }); return; }++ v.progress = v.progress || {};++ const ai = (workflow, input, key) => platform.ai.run(workflow, input, { subject: v.requested_by, idempotencyKey: `vibe:${v.id}:${key}`, totalMs: 300000 });++ try {++ // 1. Research the topic.++ if (!v.progress.research) {++ await save(v, { status: 'researching', error: null });++ const r = await ai('research.gather', { query: v.topic, topic: v.topic, lang: v.lang, max_sources: 14, max_images: 16, content_chars: 4500 }, 'research');++ const out = r.output || {};++ if (!out.sources || !out.sources.length) throw new VibeError(422, 'vibe.nothing_found', `Nothing could be found about "${v.topic}". Try its full name or another spelling.`);++ v.progress.research = {++ sources: out.sources, images: out.images || [], facts: out.facts || [], topic_wiki: out.topic_wiki || null, run: r.runId,++ from: Object.entries(out.providers || {}).filter(([, p]) => p && p.ok && p.count).map(([k]) => k),++ };++ await save(v);++ }++ const research = v.progress.research;++ // 2. Plan the pages.++ if (!v.plan) {++ await save(v, { status: 'planning' });++ const max = pagesFor(v.size);++ const r = await ai('wiki.generate_space', { topic: v.topic, max_pages: max, sources: research.sources.map((s) => sourceForAi(s, 3500)) }, 'plan');++ const plan = normalizePlan(r.output, v.topic, max);++ plan.run = r.runId;++ v.plan = plan;++ v.progress.pages = {};++ await save(v);++ await svc.updateSpace(space.id, { name: plan.title.slice(0, 120), description: clip(plan.summary || `A wiki about ${v.topic}.`, 1000) }, actor).catch((err) => log.warn(`[Vibe] ${v.id} space details: ${err && err.message}`));++ }++ // 3. Write the pages, a few at a time; the main page first so the rest hang under it.++ await save(v, { status: 'writing' });++ const titles = v.plan.pages.map((p) => p.title);++ const todo = v.plan.pages.filter((p) => !['done', 'failed'].includes((v.progress.pages[p.slug] || {}).status));++ const first = todo[0] && todo[0].slug === v.plan.pages[0].slug ? [todo.shift()] : [];++ for (const p of first) await writeOne(v, space, actor, p, titles, ai);++ const width = cfg.concurrency || 2;++ let next = 0;++ await Promise.all(Array.from({ length: Math.min(width, todo.length) }, async () => {++ while (next < todo.length) { const p = todo[next++]; await writeOne(v, space, actor, p, titles, ai); }++ }));++ // One more try for pages that hit a passing failure.++ for (const p of v.plan.pages.filter((x) => (v.progress.pages[x.slug] || {}).status === 'waiting')) await writeOne(v, space, actor, p, titles, ai);++ const states = Object.values(v.progress.pages);++ const written = states.filter((s) => s.status === 'done').length;++ if (!written) throw new VibeError(502, 'vibe.nothing_written', 'No page could be written from what was found');++ await save(v, { status: 'done', error: null, lease_owner: null, lease_until: null, finished_at: now() });++ await svc.updateSpace(space.id, { description: clip(`${v.plan.summary || `A wiki about ${v.topic}.`}`, 1000) }, actor).catch(() => {});++ log.log(`[Vibe] ${v.id} "${v.topic}": ${written} of ${v.plan.pages.length} pages`);++ } catch (err) {++ if (err && err.status === 429) {++ const wait = Math.max(30, Math.min(Number(err.retryAfter) || 300, 6 * 3600));++ await save(v, { retry_at: now() + wait * 1000, error: `Waiting for OpenVibe.AI: ${err.message}`, lease_owner: null, lease_until: null });++ return;++ }++ const attempts = (v.attempts || 0) + 1;++ const final = attempts >= 3 || (err && err.status && err.status < 500 && err.status !== 408);++ log.warn(`[Vibe] ${v.id} "${v.topic}" ${final ? 'failed' : 'will retry'}: ${err && err.message}`);++ await save(v, {++ attempts, error: oneLine(err && err.message, 500) || 'failed',++ ...(final ? { status: 'failed', finished_at: now() } : { retry_at: now() + attempts * 60000 }),++ lease_owner: null, lease_until: null,++ });++ }++ }++++ async function writeOne(v, space, actor, p, titles, ai) {++ const state = v.progress.pages[p.slug] || {};++ v.progress.pages[p.slug] = { ...state, status: 'writing' };++ await save(v);++ await q.renew.run(now() + LEASE_MS, v.id, owner);++ try {++ const isMain = p.slug === v.plan.pages[0].slug;++ const existing = await svc.findPage(space.id, p.slug);++ if (existing && existing.state !== 'deleted') { v.progress.pages[p.slug] = { status: 'done', sources: 0, images: 0 }; await save(v); return; }++ // The page's own research (the main page reuses the topic's).++ let research = v.progress.research;++ if (!isMain) {++ const r = await ai('research.gather', { query: p.query || `${p.title} ${v.topic}`, topic: v.topic, focus: p.title, lang: v.lang, max_sources: 8, max_images: 10, content_chars: 4000 }, `research:${p.slug}`);++ research = r.output || { sources: [], images: [] };++ // The topic's best source helps every page: its facts and names.++ const lead = v.progress.research.sources[0];++ if (lead && !research.sources.some((s) => s.url === lead.url) && research.sources.length < 10) research = { ...research, sources: [...research.sources, lead] };++ }++ const sources = (research.sources || []).slice(0, 10).map((s) => sourceForAi(s, isMain ? 4000 : 3500));++ if (!sources.length) throw new VibeError(422, 'vibe.page_no_sources', 'nothing was found about it');++ const pictures = (research.images || []).slice(0, 12);++ const r = await ai('wiki.generate_page', {++ title: p.title, topic: v.plan.title || v.topic, space: v.plan.title || v.topic, outline: p.outline, pages: titles,++ images: pictures.map((im) => ({ title: oneLine(im.title, 300) || 'Picture', description: oneLine([im.description, im.license ? `(${im.license})` : ''].filter(Boolean).join(' '), 400) })),++ sources,++ }, `page:${p.slug}`);++ const out = r.output || {};++ // Pictures: fetched, stored in Media for the person, placed by heading.++ const placed = new Map();++ const usedPictures = [];++ for (const pick of (Array.isArray(out.images) ? out.images : []).slice(0, MAX_IMAGES_PER_PAGE)) {++ const img = Number.isInteger(pick.index) ? pictures[pick.index] : null;++ if (!img || usedPictures.includes(img)) continue;++ const key = /^lead$/i.test(String(pick.section || '').trim()) || !pick.section ? 'lead' : oneLine(pick.section, 120).replace(/^#+\s*/, '').toLowerCase();++ if (placed.has(key)) continue;++ try {++ const got = await fetchImg(img.url, { timeoutMs: 15000 });++ const ext = { 'image/jpeg': 'jpg', 'image/png': 'png', 'image/gif': 'gif', 'image/webp': 'webp' }[got.mime];++ const mediaId = await platform.media.upload({++ bytes: got.bytes, mime: got.mime, filename: `${content.pageSlug(img.title || p.title).slice(0, 60) || 'picture'}.${ext}`, owner: v.requested_by,++ metadata: { title: oneLine(img.title, 200), source_url: img.source_url || img.url, license: img.license || null, license_url: img.license_url || null, author: img.author || null, free: Boolean(img.free), origin: 'wiki.vibe', vibe: v.id },++ });++ const caption = clip(`${oneLine(pick.caption, 220) || oneLine(img.title, 120)} (${creditOf(img)})`, 300);++ placed.set(key, { mediaId, caption, alt: oneLine(pick.caption || img.title, 200), img });++ usedPictures.push(img);++ } catch (err) {++ log.warn(`[Vibe] ${v.id} picture for "${p.title}" skipped: ${err && err.message}`);++ }++ }++ const lead = placed.get('lead');++ const body = composeBody(out, { placed: new Map([...placed].filter(([k]) => k !== 'lead')), contents: isMain ? contentsOf(v.plan) : null });++ if (!body || body.length < 200) throw new VibeError(422, 'vibe.page_too_thin', 'the sources did not say enough for a page');++ const infobox = [];++ if (lead) infobox.push({ label: 'Picture', type: 'media', value: lead.mediaId });++ const keys = new Set(lead ? ['picture'] : []);++ for (const row of Array.isArray(out.infobox) ? out.infobox : []) {++ const label = oneLine(row.label, 60).replace(/\|/g, '/');++ const value = oneLine(row.value, 300);++ const key = content.keyFromLabel(label);++ if (!label || !value || !key || keys.has(key)) continue;++ keys.add(key);++ infobox.push({ label, type: 'text', value });++ if (infobox.length >= 14) break;++ }++ const cited = citedIndices(out, sources.length);++ const citations = cited.map((i) => sources[i]).filter((s) => s && s.url).map((s) => ({++ url: s.url, title: oneLine(s.title, 300) || s.url, retrievedAt: s.retrieved_at || new Date(now()).toISOString(),++ licenseNote: (s.provenance && s.provenance.license) || null,++ }));++ for (const pic of placed.values()) {++ const img = pic.img;++ if (!img.source_url && !img.url) continue;++ citations.push({ url: img.source_url || img.url, title: `Picture: ${oneLine(img.title, 200) || p.title}`, retrievedAt: new Date(now()).toISOString(), licenseNote: oneLine([img.license, img.author].filter(Boolean).join(', '), 300) || null });++ }++ const from = [...new Set(sources.map((s) => (s.provenance && s.provenance.provider) || s.source_type))];++ const bundle = importer.validateBundle({++ pages: [{++ title: p.title, body, summary: clip(out.summary, 290) || null,++ parent: isMain ? null : v.plan.pages[0].title,++ infobox, citations: citations.slice(0, 50),++ message: clip(`Written by AI from ${citations.length} source${citations.length === 1 ? '' : 's'}`, 290),++ }],++ publish: true, on_existing: 'skip', ai_assisted: true,++ source: clip(`OpenVibe.AI research across ${sourceNames(from).join(', ')}`, 190),++ });++ const made = await svc.importPages(space.id, bundle, actor);++ const page = made.created[0] && made.created[0].page;++ if (page) {++ for (const pic of placed.values()) {++ await svc.attachMedia(page.id, { mediaId: pic.mediaId, alt: pic.alt, caption: pic.caption, role: pic === lead ? 'cover' : 'inline' }, actor, { describe: platform.media.describe })++ .catch((err) => log.warn(`[Vibe] ${v.id} attach to "${p.title}": ${err && err.message}`));++ }++ }++ v.progress.pages[p.slug] = { status: 'done', sources: citations.length, images: placed.size, run: r.runId };++ } catch (err) {++ if (err && err.status === 429) { v.progress.pages[p.slug] = { ...state, status: 'waiting' }; await save(v); throw err; }++ const tries = (state.tries || 0) + 1;++ log.warn(`[Vibe] ${v.id} page "${p.title}": ${err && err.message}`);++ v.progress.pages[p.slug] = { status: tries >= 2 || (err && err.status && err.status < 500) ? 'failed' : 'waiting', tries, error: oneLine(err && err.message, 200) };++ }++ await save(v);++ }++++ return { start, get, mine, recent, tick, startWorker, stopWorker, SIZES, ACTIVE, pagesFor };++}++++module.exports = { createVibes, VibeError, normalizePlan, composeBody, contentsOf, citedIndices, creditOf, spaceSlugFor, SIZES };@@@ -40,7 +40,7 @@ const quiet = process.env.WIKI_TEST_LOG ? console : { log() {}, warn() {}, error* A running Wiki on a database of its own (PGlite by default; WIKI_TEST_STORE=pg: the PostgreSQL + PgBouncer* containers). opts.db: boot on an existing handle (a restart keeps the data); h.stop() closes what boot opened.*/--async function boot({ env = {}, fetch: fetchImpl, db: givenDb = null, now, workers = false, tokens, rateLimits = false, limitsNow = null, log = quiet, indexnow = undefined, accountSend = null, telemetrySink = null } = {}) {++async function boot({ env = {}, fetch: fetchImpl, db: givenDb = null, now, workers = false, tokens, rateLimits = false, limitsNow = null, log = quiet, indexnow = undefined, accountSend = null, telemetrySink = null, imageFetch = null } = {}) {const owned = givenDb ? null : await testDb();const db = givenDb || owned.db;const config = load({@@@ -52,7 +52,7 @@ async function boot({ env = {}, fetch: fetchImpl, db: givenDb = null, now, workeconfig, db, publicKey, log, listen: true, workers, rateLimits, now, limitsNow,fetchImpl: fetchImpl || (async (url) => { throw new Error(`unexpected outbound fetch ${url}`); }),tokens: tokens || { getToken: async () => 'stub-token', authHeaders: async () => ({ Authorization: 'Bearer stub-token' }), invalidate() {} },-- indexnow, accountSend, telemetrySink,++ indexnow, accountSend, telemetrySink, imageFetch,});const base = `http://127.0.0.1:${h.server.address().port}`;const stop = async ({ keepDb = false } = {}) => { await h.stop(); if (owned && !keepDb) await owned.close(); };@@@ -62,8 +62,9 @@ const INTERNAL = ['server/auth/keys.js': 'Network JWKS (configured)','server/auth/session.js': 'Network OAuth token and revoke (configured)','server/http/frame-groups.js': 'Network /api/chrome, the Frame\'s site groups (configured: WIKI_FRAME_URL or the internal Network)',-- 'server/integrations/platform.js': 'Community, Sources, Media, VIP (configured)',++ 'server/integrations/platform.js': 'Community, Sources, Media, VIP, AI (configured)','server/integrations/projects-module.js': 'Network user modules (configured)',++ 'server/integrations/image-fetch.js': 'pictures a generated wiki uses (research results): https only, openvibe-shared/egress safeLookup on every hop, a byte cap, magic bytes checked',};const root = path.join(__dirname, '..');const found = [];nnew file mode 100644@@@ -0,0 +1,196 @@++'use strict';++/**++ * "Vibe a wiki" end to end against stand-ins for OpenVibe.AI and OpenVibe.Media: a person names a topic → a new wiki++ * they own → research, plan, pages written with their sources and pictures (stored in Media for the person, placed in++ * the text and the infobox, credited as citations) → published with the AI disclosure and noindex. AI's quota (429)++ * makes the job wait and resume; the daily limit refuses; a signed-out visitor is asked to sign in. Also: the++ * [[media:…]] syntax and the picture fetcher's refusals.++ */++const assert = require('assert');++const H = require('./helpers');++const content = require('../server/wiki/content');++const { checkUrl, sniff } = require('../server/integrations/image-fetch');++const { normalizePlan, spaceSlugFor, contentsOf } = require('../server/wiki/vibe');++++const MED = (n) => `med_01J8Z6Q3KX0000000000000${String(n).padStart(3, '0')}`;++const PNG = Buffer.from('89504e470d0a1a0a0000000d4948445200000001000000010806000000', 'hex');++const LONG = 'Kim Possible is a teenager who saves the world between classes, with her best friend and a naked mole-rat. '.repeat(6);++++// Unit checks first: the syntax, the fetcher's rules, the plan's shape.++{++ const src = 'Intro.\n\n[[media:' + MED(1) + '|A caption]]\n\nSee [[Ron Stoppable]] and [[media:' + MED(9) + ']].';++ assert.deepStrictEqual(content.extractLinks(src, 'kp').map((l) => l.slug), ['ron-stoppable'], 'a picture is not a page link');++ assert.deepStrictEqual(content.mediaRefs(src), [MED(1), MED(9)]);++ const out = content.renderContent(src, { currentSpace: 'kp', resolve: () => ({ href: '/w/kp/ron-stoppable', exists: true }), media: (id, cap) => (id === MED(1) ? `<figure data-id="${id}">${cap}</figure>` : null) });++ assert.ok(out.includes(`<figure data-id="${MED(1)}">A caption</figure>`), out);++ assert.ok(!/<p>\s*<figure/.test(out), 'a picture on its own line is not wrapped in a paragraph');++ assert.ok(!out.includes(MED(9)), 'a picture the page does not have shows nothing');++ for (const bad of ['http://x.test/a.png', 'https://127.0.0.1/a.png', 'https://user:[email protected]/a.png', 'file:///etc/passwd', 'nonsense']) assert.throws(() => checkUrl(bad), bad);++ assert.strictEqual(sniff(PNG), 'image/png');++ assert.strictEqual(sniff(Buffer.from('<svg xmlns="http://www.w3.org/2000/svg"></svg>')), null, 'an SVG is never taken for a picture');++ assert.strictEqual(spaceSlugFor('Pokémon Red & Blue!'), 'pokemon-red-blue');++ assert.strictEqual(spaceSlugFor('東京'), null);++ const plan = normalizePlan({ title: 'KP', summary: 's', pages: [{ title: 'Ron', kind: 'character', outline: ['A'] }, { title: 'Kim Possible', kind: 'overview' }, { title: 'ron', kind: 'person' }, { title: 'Bad [[x]]|', kind: 'weird' }] }, 'Kim Possible', 3);++ assert.deepStrictEqual(plan.pages.map((p) => [p.title, p.kind]), [['Kim Possible', 'overview'], ['Ron', 'character'], ['Bad x', 'other']], 'main page first, duplicates out, kinds known, link syntax stripped');++ assert.match(contentsOf(plan), /### Characters\n\n- \[\[Ron\]\]/);++}++++(async () => {++ const A = H.subject();++ const calls = [];++ let quota = 1; // the first write of "Ron Stoppable" meets a full quota++ let objects = 0;++ const json = (status, body, headers = {}) => new Response(JSON.stringify(body), { status, headers: { 'Content-Type': 'application/json', ...headers } });++ const research = (focus) => ({++ query: focus || 'Kim Possible', topic: 'Kim Possible', focus: focus || null,++ sources: [++ { source_type: 'wikipedia', url: 'https://en.wikipedia.org/wiki/Kim_Possible', title: 'Kim Possible — Wikipedia', content: LONG, retrieved_at: '2026-10-25T10:00:00.000Z', provenance: { provider: 'wikipedia', license: 'CC BY-SA 4.0 (Wikipedia)' } },++ { source_type: 'fandom', url: `https://kimpossible.fandom.com/wiki/${focus ? 'Ron_Stoppable' : 'Kim_Possible'}`, title: 'Kim Possible Wiki', content: LONG, retrieved_at: '2026-10-25T10:00:00.000Z', provenance: { provider: 'fandom', license: 'CC BY-SA 3.0 (Fandom)' } },++ ],++ images: [{ url: 'https://img.test/kim.png', title: focus || 'Kim Possible', description: 'from the show', license: 'CC BY-SA 4.0', author: 'Jane Doe', free: true, source_url: 'https://commons.wikimedia.org/wiki/File:Kim.png', provider: 'commons' }],++ facts: [], topic_wiki: { name: 'Kim Possible Wiki', url: 'https://kimpossible.fandom.com' }, providers: { wikipedia: { ok: true, count: 1 }, fandom: { ok: true, count: 1 }, deepseek: { ok: false, skipped: 'not configured' } }, usage: {},++ });++ const page = (title) => ({++ title, summary: `${title} is part of Kim Possible.`, lead: `**${title}** is part of the show. ${title === 'Kim Possible' ? 'Her friend is [[Ron Stoppable]].' : 'He is friends with [[Kim Possible]].'} ${LONG}`,++ sections: [{ heading: 'Premise', body: LONG, citations: [0] }, { heading: 'Legacy', body: LONG, citations: [1] }],++ infobox: [{ label: 'Created by', value: 'Bob Schooley and Mark McCorkle', citations: [0] }, { label: 'Network', value: 'Disney Channel', citations: [0] }],++ images: [{ index: 0, caption: `${title} on screen`, section: 'lead' }, { index: 7, caption: 'out of range', section: 'Premise' }],++ citations: [0, 1], gaps: [],++ });++ const fetchStub = async (url, init = {}) => {++ const u = new URL(url);++ calls.push({ host: u.host, path: u.pathname, method: init.method || 'GET', headers: init.headers || {}, body: typeof init.body === 'string' ? init.body : null });++ if (u.host === 'ai.internal') {++ if (u.pathname !== '/api/v1/runs' || init.method !== 'POST') return json(404, {});++ const b = JSON.parse(init.body);++ assert.deepStrictEqual(b.on_behalf_of, { type: 'user', id: A }, 'AI runs on behalf of the person');++ const id = `run_${calls.length}`;++ if (b.workflow === 'research.gather') return json(201, { run: { id, status: 'succeeded', output: research(b.input.focus) } });++ if (b.workflow === 'wiki.generate_space') {++ assert.ok(b.input.sources.length >= 1 && b.input.max_pages === 6);++ return json(201, { run: { id, status: 'succeeded', output: { title: 'Kim Possible', summary: 'An animated series about a teen hero.', pages: [++ { title: 'Kim Possible', kind: 'overview', parent: '', outline: ['Premise', 'Legacy'], query: 'Kim Possible', citations: [0] },++ { title: 'Ron Stoppable', kind: 'character', parent: 'Kim Possible', outline: ['Premise'], query: 'Ron Stoppable Kim Possible', citations: [1] },++ { title: 'Kim Possible', kind: 'other', parent: '', outline: [], query: '', citations: [] },++ ], citations: [0], gaps: [] } } });++ }++ if (b.workflow === 'wiki.generate_page') {++ assert.ok(b.input.pages.includes('Ron Stoppable') && b.input.images.length === 1);++ if (b.input.title === 'Ron Stoppable' && quota-- > 0) return json(429, { code: 'quota.exceeded', detail: 'daily budget used' }, { 'Retry-After': '120' });++ return json(201, { run: { id, status: 'succeeded', output: page(b.input.title) } });++ }++ return json(404, {});++ }++ if (u.host === 'media.internal') {++ const m = /^\/api\/v2\/wiki\/objects(?:\/([^/]+))?(?:\/(content|complete))?$/.exec(u.pathname);++ if (!m) return json(404, {});++ if (!m[1] && init.method === 'POST') {++ const b = JSON.parse(init.body);++ assert.strictEqual(b.visibility, 'public');++ assert.strictEqual(b.kind, 'asset');++ assert.strictEqual(init.headers['X-OV-Subject'], A, 'the picture belongs to the person');++ assert.strictEqual(b.metadata.license, 'CC BY-SA 4.0');++ const id = MED(++objects);++ return json(201, { id, upload: { method: 'PUT', url: `https://openvibe.media/api/v2/wiki/objects/${id}/content?token=t1`, token: 't1', complete_url: `https://openvibe.media/api/v2/wiki/objects/${id}/complete` } });++ }++ if (m[2] === 'content') { assert.strictEqual(init.method, 'PUT'); return json(200, { ok: true }); }++ if (m[2] === 'complete') return json(200, { object: { id: m[1], lifecycle_status: 'ready' } });++ return json(200, { id: m[1], visibility: 'public', owner: { subject: A, app: 'wiki' }, lifecycle_status: 'ready' });++ }++ return json(404, { error: 'stub: no route' });++ };++ const fetched = [];++ const h = await H.boot({++ fetch: fetchStub, imageFetch: async (url) => { fetched.push(url); return { bytes: PNG, mime: 'image/png', url }; },++ env: { OV_MEDIA_INTERNAL_URL: 'http://media.internal', OV_AI_INTERNAL_URL: 'http://ai.internal', WIKI_VIBE_PER_PERSON_DAY: '2', WIKI_VIBE_PAGES_QUICK: '6' },++ });++ const tok = H.userToken({ subject: A });++ try {++ // Signed out: asked to sign in; the form is on /vibe and the home.++ let r = await H.req(h, 'POST', '/vibe', { form: { topic: 'Kim Possible' } });++ assert.strictEqual(r.status, 401);++ r = await H.req(h, 'GET', '/vibe');++ assert.strictEqual(r.status, 200);++ assert.match(r.text, /Vibe a wiki/);++ assert.match(r.text, /Sign in to vibe/);++ r = await H.req(h, 'GET', '/');++ assert.match(r.text, /Any topic\./);++ assert.match(r.text, /href="\/vibe"/);++++ // Start one.++ r = await H.req(h, 'POST', '/api/v1/vibes', { token: tok, body: { topic: ' Kim Possible ', size: 'quick' } });++ assert.strictEqual(r.status, 202, r.text);++ const id = r.json.vibe.id;++ assert.strictEqual(r.json.vibe.status, 'queued');++ assert.strictEqual(r.json.space.slug, 'kim-possible');++ assert.strictEqual(r.json.space.kind, 'user');++ assert.match(r.json.progress_url, new RegExp(`/vibe/${id}$`));++ r = await H.req(h, 'POST', '/api/v1/vibes', { token: tok, body: { topic: 'x' } });++ assert.strictEqual(r.status, 422);++ r = await H.req(h, 'POST', '/api/v1/vibes', { token: tok, body: { topic: 'Kim Possible', size: 'enormous' } });++ assert.strictEqual(r.status, 422);++++ // Research, plan, the main page; Ron meets a full quota → the job waits for Retry-After.++ assert.strictEqual(await h.vibes.tick(), true);++ let v = (await H.req(h, 'GET', `/api/v1/vibes/${id}`, { token: tok })).json.vibe;++ assert.strictEqual(v.status, 'writing');++ assert.ok(v.waiting_until, 'waiting for AI\'s quota');++ assert.deepStrictEqual(v.pages.map((p) => [p.title, p.status]), [['Kim Possible', 'done'], ['Ron Stoppable', 'waiting']]);++ assert.strictEqual(await h.vibes.tick(), false, 'nothing is due while it waits');++ await h.db.prepare('UPDATE wiki_vibes SET retry_at = 1 WHERE id = ?').run(id);++ assert.strictEqual(await h.vibes.tick(), true);++ v = (await H.req(h, 'GET', `/api/v1/vibes/${id}`, { token: tok })).json.vibe;++ assert.strictEqual(v.status, 'done', JSON.stringify(v));++ assert.strictEqual(v.done, 2);++ assert.strictEqual(v.main_page, '/w/kim-possible/kim-possible');++ assert.deepStrictEqual(v.research.from.sort(), ['Fandom', 'Wikipedia']);++ assert.strictEqual(fetched.length, 2, 'one picture per page; an index out of range is ignored');++++ // The wiki: named and described by the plan, the person owns it.++ r = await H.req(h, 'GET', '/api/v1/spaces/kim-possible', { token: tok });++ assert.strictEqual(r.json.space.name, 'Kim Possible');++ assert.strictEqual(r.json.space.description, 'An animated series about a teen hero.');++ assert.deepStrictEqual(r.json.pages.map((p) => [p.slug, p.children.map((c) => c.slug)]), [['kim-possible', ['ron-stoppable']]], 'pages hang under the main page');++++ // The main page as a reader sees it.++ r = await H.req(h, 'GET', '/w/kim-possible/kim-possible');++ assert.strictEqual(r.status, 200);++ assert.match(r.text, /written with AI assistance/i);++ assert.match(r.text, /<meta name="robots" content="noindex/, 'noindex until a person reviews it');++ assert.match(r.text, new RegExp(`class="wk-ib-pic"[\\s\\S]*?<img src="https://openvibe.media/o/${MED(1)}"`), 'the lead picture in the infobox');++ assert.match(r.text, /Kim Possible on screen \(Picture: Jane Doe, CC BY-SA 4.0\)/, 'captioned with its credit');++ assert.match(r.text, /<th scope="row">Created by<\/th><td>Bob Schooley and Mark McCorkle<\/td>/);++ assert.match(r.text, /<h2[^>]*>In this wiki<\/h2>/);++ assert.match(r.text, /<a class="ov-wikilink" href="\/w\/kim-possible\/ron-stoppable">Ron Stoppable<\/a>/, 'a blue link to the page written after it');++ assert.match(r.text, /href="https:\/\/en.wikipedia.org\/wiki\/Kim_Possible"[^>]*>Kim Possible — Wikipedia<\/a>, retrieved/);++ assert.match(r.text, /Picture: Kim Possible<\/a>, retrieved[\s\S]*?\(CC BY-SA 4.0, Jane Doe\)/, 'the picture cited with its licence');++ assert.ok(!/<section class="wk-media">/.test(r.text), 'no second copy of a picture already shown');++ const att = await h.db.prepare('SELECT o.media_id, o.attached_by, o.media_visibility FROM wiki_attachment_origins o JOIN wiki_pages p ON p.id = o.page_id WHERE p.slug = ?').all('kim-possible');++ assert.deepStrictEqual(att, [{ media_id: MED(1), attached_by: A, media_visibility: 'public' }]);++++ // The progress page and its JSON.++ r = await H.req(h, 'GET', `/vibe/${id}`);++ assert.strictEqual(r.status, 200);++ assert.match(r.text, /is ready/);++ assert.match(r.text, /href="\/w\/kim-possible\/kim-possible">Open the wiki/);++ r = await H.req(h, 'GET', `/vibe/${id}.json`);++ assert.strictEqual(r.json.status, 'done');++ assert.strictEqual((await H.req(h, 'GET', '/vibe/vib_01J8Z6Q3KX00000000000000000.json')).status, 404);++ r = await H.req(h, 'GET', '/');++ assert.match(r.text, /Wikis people vibed/);++++ // The daily limit (2 here): the second start is allowed, the third refused.++ r = await H.req(h, 'POST', '/api/v1/vibes', { token: tok, body: { topic: 'Ron Stoppable', size: 'quick', visibility: 'private' } });++ assert.strictEqual(r.status, 202, r.text);++ r = await H.req(h, 'POST', '/api/v1/vibes', { token: tok, body: { topic: 'Rufus' } });++ assert.strictEqual(r.status, 429);++ assert.strictEqual(r.json.code, 'vibe.daily_limit');++ // A private generated wiki's progress is not shown to others.++ const second = (await H.req(h, 'GET', '/api/v1/vibes', { token: tok })).json.vibes.find((x) => x.topic === 'Ron Stoppable');++ assert.strictEqual((await H.req(h, 'GET', `/vibe/${second.id}`)).status, 404);++ assert.strictEqual((await H.req(h, 'GET', `/vibe/${second.id}.json`, { token: H.userToken() })).status, 404);++ console.log('vibe: all checks passed');++ } finally {++ await h.stop();++ }++})().catch((err) => { console.error(err); process.exit(1); });
@@ -61,6 +61,15 @@ OV_OAUTH_CLIENT_SECRET=
@@ -61,6 +61,15 @@ OV_OAUTH_CLIENT_SECRET=
#OV_MEDIA_INTERNAL_URL=http://127.0.0.1:4100
#OV_MEDIA_INTERNAL_URL=http://127.0.0.1:4100
#OV_MEDIA_URL=https://openvibe.media
#OV_MEDIA_URL=https://openvibe.media
#WIKI_MEDIA_APP=wiki
#WIKI_MEDIA_APP=wiki
+# OpenVibe.AI — "vibe a wiki" (research.gather, wiki.generate_space, wiki.generate_page). Unset: /vibe answers that it is
+# not available. Limits per person and per day, page counts by size, pages written at a time.
+#OV_AI_INTERNAL_URL=http://127.0.0.1:4700
+#WIKI_VIBE_PER_PERSON_DAY=3
+#WIKI_VIBE_PER_DAY=30
+#WIKI_VIBE_PAGES_QUICK=6
+#WIKI_VIBE_PAGES_STANDARD=12
+#WIKI_VIBE_PAGES_BIG=24
+#WIKI_VIBE_CONCURRENCY=2
#WIKI_MEDIA_VERIFY_INTERVAL_MS=21600000
#WIKI_MEDIA_VERIFY_INTERVAL_MS=21600000
# Scheduled publication worker.
# Scheduled publication worker.
@@ -66,7 +66,8 @@ and, with the workers on (production), flushes it every minute and runs the rete
@@ -66,7 +66,8 @@ and, with the workers on (production), flushes it every minute and runs the rete
- **Source items** — OpenVibe.Sources owns them; a citation stores the item id plus the URL, title,
- **Source items** — OpenVibe.Sources owns them; a citation stores the item id plus the URL, title,
retrieval time and license read from the item's provenance.
retrieval time and license read from the item's provenance.
- **The network index** — OpenVibe.Search owns it; Wiki sends `wiki.index_document.*` events.
- **The network index** — OpenVibe.Search owns it; Wiki sends `wiki.index_document.*` events.
-- **Publication truth by AI** — never. There are no AI provider calls in Wiki.
+- **Publication truth by AI** — never. There are no AI provider calls in Wiki: generated wikis run OpenVibe.AI's
+ registered workflows, and their pages carry the AI disclosure and stay noindex until a person reviews them.
- **Identity** — OpenVibe.Network (subjects `usr_…`, SSO, service principals).
- **Identity** — OpenVibe.Network (subjects `usr_…`, SSO, service principals).
## What works
## What works
@@ -161,6 +162,26 @@ and, with the workers on (production), flushes it every minute and runs the rete
@@ -161,6 +162,26 @@ and, with the workers on (production), flushes it every minute and runs the rete
the accountable person and where the text comes from; `ai_assisted: true` keeps every page noindex
the accountable person and where the text comes from; `ai_assisted: true` keeps every page noindex
until a person reviews it. Existing slugs stop the import (409) unless `on_existing: "skip"`.
until a person reviews it. Existing slugs stop the import (409) unless `on_existing: "skip"`.
Twenty imports per hour per address. Format: [server/wiki/import.js](server/wiki/import.js).
Twenty imports per hour per address. Format: [server/wiki/import.js](server/wiki/import.js).
+- **Vibe a wiki** (`/vibe`, `POST /api/v1/vibes`; [server/wiki/vibe.js](server/wiki/vibe.js)): a signed-in person names
+ any topic ("Kim Possible", "ancient Rome") and gets a whole wiki about it, in a new user space they own:
+ 1. *Research*: OpenVibe.AI `research.gather` (MegaSource: Wikipedia, the topic's Fandom wiki, Wikidata, DeepSeek web
+ search, Commons, Openverse) for the topic.
+ 2. *Plan*: `wiki.generate_space` names the wiki and plans its pages (the main page first; quick 6, standard 12, big 24).
+ 3. *Write*: per page, its own research (the page's subject inside the topic) and `wiki.generate_page`. The pictures it
+ picks are fetched under openvibe-shared/egress ([server/integrations/image-fetch.js](server/integrations/image-fetch.js):
+ https, public addresses, 8 MB, real JPEG/PNG/GIF/WebP bytes), stored in OpenVibe.Media for the person (public,
+ with source, licence and author in the object's metadata), placed in the text (`[[media:med_…|caption]]`) or the
+ infobox (a `media` row), attached, and cited with their licence. The page is imported like any AI-assisted page:
+ published with the disclosure, noindex until a person reviews it; the main page lists every planned page
+ ("In this wiki"), and the rest hang under it.
+
+ The job is a `wiki_vibes` row (migration `0004_vibes.sql`) a worker takes under a lease; progress is saved after every
+ step, so a restart resumes and pages already made are skipped. AI's quota or queue (429) waits for its Retry-After; a
+ page the sources cannot support is left as a red link. `/vibe/:id` shows it live (and refreshes itself without
+ JavaScript); `/vibe/:id.json` and `GET /api/v1/vibes/:id` give the same progress. Limits: `WIKI_VIBE_PER_PERSON_DAY`
+ (3) and `WIKI_VIBE_PER_DAY` (30); `WIKI_VIBE_PAGES_QUICK|STANDARD|BIG`, `WIKI_VIBE_CONCURRENCY` (2 pages at a time).
+ Needs `OV_AI_INTERNAL_URL` and Media; Network grants `wiki` `ai.run.create`/`ai.run.read` (`wiki.*`, `research.*`)
+ and `media.object.upload` (namespace `wiki`). Tests: `test/vibe.test.js`.
- **Seed**: `npm run seed` imports the official "OpenVibe" space (`seeds/openvibe.json`): 42 pages
- **Seed**: `npm run seed` imports the official "OpenVibe" space (`seeds/openvibe.json`): 42 pages
about the network's repositories, summarised from their README/STATUS files at pinned commits,
about the network's repositories, summarised from their README/STATUS files at pinned commits,
each cited with a GitHub permalink and retrieval time, authorship recorded as `imported` with
each cited with a GitHub permalink and retrieval time, authorship recorded as `imported` with
@@ -185,7 +206,7 @@ match; network search is OpenVibe.Search).
@@ -185,7 +206,7 @@ match; network search is OpenVibe.Search).
## Routes
## Routes
-Pages (SSR, useful without JavaScript): `/`, `/recent`, `/search?q=`, `/new-space`, `/s/:space`,
+Pages (SSR, useful without JavaScript): `/`, `/vibe`, `/vibe/:id` (`.json`), `/recent`, `/search?q=`, `/new-space`, `/s/:space`,
`/s/:space/new`, `/s/:space/settings`, `/s/:space/import`, `/s/:space/proposals`, `/w/:space/:slug` (`?rev=N`),
`/s/:space/new`, `/s/:space/settings`, `/s/:space/import`, `/s/:space/proposals`, `/w/:space/:slug` (`?rev=N`),
`/w/:space/:slug.json`, `/w/:space/:slug/history`, `/w/:space/:slug/sources` (`?rev=N`), `/w/:space/:slug/diff/:a/:b` (`?mode=line`),
`/w/:space/:slug.json`, `/w/:space/:slug/history`, `/w/:space/:slug/sources` (`?rev=N`), `/w/:space/:slug/diff/:a/:b` (`?mode=line`),
`/w/:space/:slug/edit`, `/w/:space/:slug/revert?to=N`, `/w/:space/:slug/settings`,
`/w/:space/:slug/edit`, `/w/:space/:slug/revert?to=N`, `/w/:space/:slug/settings`,
@@ -302,7 +323,8 @@ Production: `/opt/openvibe.wiki`, env `/etc/openvibe/wiki.env`, unit
@@ -302,7 +323,8 @@ Production: `/opt/openvibe.wiki`, env `/etc/openvibe/wiki.env`, unit
OpenVibe.Sources, OpenVibe.Media, OpenVibe.VIP (VIP spaces and pages), OpenVibe.Search (consumer of
OpenVibe.Sources, OpenVibe.Media, OpenVibe.VIP (VIP spaces and pages), OpenVibe.Search (consumer of
the index events). All but the Network key are optional at runtime and degrade to explicit failure
the index events). All but the Network key are optional at runtime and degrade to explicit failure
states.
states.
-- OpenVibe.AI for proposals (not wired to Wiki yet: the proposal API is the seam).
+- OpenVibe.AI: generated wikis (`OV_AI_INTERNAL_URL`; research.gather, wiki.generate_space, wiki.generate_page). AI
+ proposals still arrive through the proposal API.
- Account export and deletion (ADR-033): Network grants `events.subscription.manage` (openvibe.events) for the two
- Account export and deletion (ADR-033): Network grants `events.subscription.manage` (openvibe.events) for the two
subscriptions created at boot, then, last and once the release is live, `network.account.export.contribute` and
subscriptions created at boot, then, last and once the release is live, `network.account.export.contribute` and
`network.account.deletion.confirm` (openvibe.network).
`network.account.deletion.confirm` (openvibe.network).
new file mode 100644
new file mode 100644
@@ -0,0 +1,27 @@
@@ -0,0 +1,27 @@
+-- phase: expand
+-- "Vibe a wiki" (server/wiki/vibe.js): one row per generated wiki. A worker takes a queued or stalled job under a lease,
+-- researches the topic through OpenVibe.AI (research.gather), plans the pages (wiki.generate_space), writes each one
+-- (wiki.generate_page) with its pictures copied into OpenVibe.Media, and records progress here as it goes, so a restart
+-- resumes where it stopped (pages already made are skipped).
+CREATE TABLE IF NOT EXISTS wiki_vibes (
+ id text PRIMARY KEY,
+ space_id text NOT NULL REFERENCES wiki_spaces (id) ON DELETE CASCADE,
+ topic text NOT NULL,
+ lang text NOT NULL DEFAULT 'en',
+ size text NOT NULL DEFAULT 'standard' CHECK (size IN ('quick', 'standard', 'big')),
+ status text NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'researching', 'planning', 'writing', 'done', 'failed', 'cancelled')),
+ requested_by text NOT NULL,
+ plan jsonb,
+ progress jsonb NOT NULL DEFAULT '{}'::jsonb,
+ error text,
+ attempts integer NOT NULL DEFAULT 0,
+ lease_owner text,
+ lease_until bigint,
+ retry_at bigint,
+ created_at bigint NOT NULL,
+ updated_at bigint NOT NULL,
+ finished_at bigint
+);
+CREATE INDEX IF NOT EXISTS wiki_vibes_due_idx ON wiki_vibes (status, retry_at) WHERE status IN ('queued', 'researching', 'planning', 'writing');
+CREATE INDEX IF NOT EXISTS wiki_vibes_requested_idx ON wiki_vibes (requested_by, created_at DESC);
+CREATE INDEX IF NOT EXISTS wiki_vibes_space_idx ON wiki_vibes (space_id);
new file mode 100644
new file mode 100644
@@ -0,0 +1,48 @@
@@ -0,0 +1,48 @@
+/* Vibe a wiki (/vibe, /vibe/:id): the big form, sizes, the steps, the progress of each page. The home's small form
+ reuses its search box (wiki.css); the cards of generated wikis are showcase cards. */
+.wk-vibe-form { display: grid; gap: 10px; margin-top: 8px; }
+.wk-vibe-label { font-size: 14px; font-weight: 600; }
+.wk-vibe-row { display: flex; gap: 8px; }
+.wk-vibe-row input { flex: 1; min-width: 0; padding: 10px 12px; font-size: 1.05rem; border-radius: 10px; }
+.wk-vibe-row button { padding: 10px 18px; border-radius: 10px; white-space: nowrap; }
+.wk-vibe-row input:disabled { opacity: .6; }
+@media (max-width: 560px) { .wk-vibe-row { flex-direction: column; } .wk-vibe-row button { width: 100%; } }
+.wk-vibe-sizes { display: flex; flex-wrap: wrap; gap: 8px; margin: 0; padding: 0; border: 0; }
+.wk-vibe-sizes legend { width: 100%; margin-bottom: 6px; font-size: 14px; font-weight: 600; }
+.wk-vibe-sizes label { position: relative; flex: 1 1 140px; cursor: pointer; }
+.wk-vibe-sizes input { position: absolute; opacity: 0; inset: 0; }
+.wk-vibe-sizes span { display: block; height: 100%; padding: 10px 12px; border: 1px solid var(--wk-line); border-radius: 12px; background: var(--wk-panel); transition: border-color .15s, background .15s; }
+.wk-vibe-sizes small { display: block; color: var(--wk-muted); font-size: 13px; }
+.wk-vibe-sizes input:checked + span { border-color: var(--wk-accent); background: color-mix(in srgb, var(--wk-accent) 10%, var(--wk-panel)); }
+.wk-vibe-sizes input:focus-visible + span { outline: 2px solid var(--wk-accent); outline-offset: 2px; }
+.wk-vibe h1 span { background: linear-gradient(90deg, var(--wk-accent), color-mix(in srgb, var(--wk-accent) 40%, #f472b6)); -webkit-background-clip: text; background-clip: text; color: transparent; }
+.wk-vibe-lede { max-width: 720px; font-size: 1.08rem; color: var(--wk-muted); }
+.wk-vibe-how { list-style: none; counter-reset: wk-how; display: grid; gap: 10px; grid-template-columns: repeat(auto-fit, minmax(min(100%, 200px), 1fr)); margin: 22px 0 10px; padding: 0; }
+.wk-vibe-how li { counter-increment: wk-how; padding: 12px 14px; border: 1px solid var(--wk-line); border-radius: 12px; background: var(--wk-panel); }
+.wk-vibe-how li::before { content: counter(wk-how); display: inline-grid; place-items: center; width: 24px; height: 24px; margin-right: 8px; border-radius: 50%; background: var(--wk-accent); color: var(--on-accent-strong, #0a0f1c); font-size: 13px; font-weight: 800; }
+.wk-vibe-how b { font-size: 1rem; }
+.wk-vibe-how span { display: block; margin-top: 4px; font-size: 14px; color: var(--wk-muted); }
+.wk-vibe-mine { list-style: none; margin: 0; padding: 0; }
+.wk-vibe-mine li { display: flex; flex-wrap: wrap; gap: 4px 10px; align-items: baseline; padding: 8px 0; border-bottom: 1px solid var(--wk-line); }
+.wk-vp-steps { list-style: none; display: flex; flex-wrap: wrap; gap: 6px; margin: 4px 0 14px; padding: 0; }
+.wk-vp-steps li { padding: 5px 12px; border: 1px solid var(--wk-line); border-radius: 999px; font-size: 14px; color: var(--wk-muted); }
+.wk-vp-steps .wk-vp-step-done { border-color: color-mix(in srgb, var(--wk-ok, #34d399) 60%, transparent); color: var(--wk-text); }
+.wk-vp-steps .wk-vp-step-done::before { content: "✓ "; color: var(--wk-ok, #34d399); }
+.wk-vp-steps .wk-vp-step-now { border-color: var(--wk-accent); color: var(--wk-text); background: color-mix(in srgb, var(--wk-accent) 12%, transparent); animation: wk-pulse 1.6s ease-in-out infinite; }
+.wk-vp-status { font-weight: 600; }
+.wk-vp-read { color: var(--wk-muted); }
+.wk-vp-open { padding: 10px 20px; border-radius: 10px; font-size: 1.05rem; }
+.wk-vp-pages { list-style: none; margin: 16px 0 0; padding: 0; display: grid; gap: 8px; grid-template-columns: repeat(auto-fill, minmax(min(100%, 230px), 1fr)); }
+.wk-vp-pages li { display: flex; flex-direction: column; gap: 2px; padding: 10px 12px; border: 1px solid var(--wk-line); border-radius: 12px; background: var(--wk-panel); transition: border-color .2s; }
+.wk-vp-pages li > span { color: var(--wk-muted); }
+.wk-vp-pages small { font-size: 12.5px; color: var(--wk-muted); }
+.wk-vp-pages .wk-vp-done { border-color: color-mix(in srgb, var(--wk-ok, #34d399) 45%, var(--wk-line)); }
+.wk-vp-pages .wk-vp-done a { font-weight: 700; }
+.wk-vp-pages .wk-vp-writing { border-color: var(--wk-accent); }
+.wk-vp-pages .wk-vp-writing small::before { content: ""; display: inline-block; width: 8px; height: 8px; margin-right: 6px; border-radius: 50%; background: var(--wk-accent); animation: wk-pulse 1s ease-in-out infinite; }
+.wk-vp-pages .wk-vp-failed { opacity: .65; }
+.wk-vp-wait { display: flex; gap: 6px; padding: 18px 0; }
+.wk-vp-dot { width: 10px; height: 10px; border-radius: 50%; background: var(--wk-accent); animation: wk-pulse 1.2s ease-in-out infinite; }
+.wk-vp-dot:nth-child(2) { animation-delay: .2s; }
+.wk-vp-dot:nth-child(3) { animation-delay: .4s; }
+@keyframes wk-pulse { 0%, 100% { opacity: 1; } 50% { opacity: .35; } }
@@ -145,3 +145,19 @@ main .wk-h-group h3 { display: flex; flex-wrap: wrap; align-items: baseline; gap
@@ -145,3 +145,19 @@ main .wk-h-group h3 { display: flex; flex-wrap: wrap; align-items: baseline; gap
.wk-h-group .wk-h-host { margin-top: 2px; font-size: 12px; }
.wk-h-group .wk-h-host { margin-top: 2px; font-size: 12px; }
.wk-h-stats dd { font-size: 1.3rem; }
.wk-h-stats dd { font-size: 1.3rem; }
}
}
+
+/* Infobox labels never break inside a word ("Reposi|tory"); a picture row spans the box. Pictures in the text sit
+ beside the words, full width on phones. */
+.wk-infobox th { width: 36%; overflow-wrap: normal; word-break: normal; }
+.wk-infobox td { overflow-wrap: anywhere; }
+.wk-ib-pic td { padding: 0 !important; }
+.wk-ib-pic .ov-media { margin: 0; }
+.wk-ib-pic img { max-height: 420px; object-fit: contain; background: var(--wk-bg); }
+.ov-media img { display: block; width: 100%; height: auto; border-radius: 8px; }
+.ov-media figcaption { padding: 6px 4px; font-size: 13px; line-height: 1.45; color: var(--wk-muted); }
+.wk-content .ov-media { float: right; clear: right; width: min(300px, 46%); margin: 4px 0 12px 18px; }
+.wk-content h2 { clear: both; }
+.wk-media .ov-media { display: inline-block; vertical-align: top; width: min(240px, 100%); margin: 0 12px 12px 0; }
+@media (max-width: 640px) { .wk-content .ov-media { float: none; width: 100%; margin: 8px 0 12px; } }
+.wk-vibe-panel .wk-h-stats { margin-top: 14px; }
+.wk-vibe-ideas { margin: 6px 0 0; font-size: 13.5px; color: var(--wk-muted); }
@@ -23,7 +23,7 @@ const { createActorLimits } = require('./http/actor-limits');
@@ -23,7 +23,7 @@ const { createActorLimits } = require('./http/actor-limits');
const PUBLIC_DIR = path.join(__dirname, '..', 'public');
const PUBLIC_DIR = path.join(__dirname, '..', 'public');
const VERSION = require('../package.json').version;
const VERSION = require('../package.json').version;
-function createApp({ config, telemetry = null, svc, viewers: baseViewers, platform, keys, db, valkey = null, log = console, rateLimits = true, fetchImpl = globalThis.fetch, limitsNow = null, indexnow = null, accountData = null, accountSend = null }) {
+function createApp({ config, telemetry = null, svc, viewers: baseViewers, platform, keys, db, valkey = null, log = console, rateLimits = true, fetchImpl = globalThis.fetch, limitsNow = null, indexnow = null, accountData = null, accountSend = null, vibes = null }) {
const app = express();
const app = express();
// Every resolved actor carries its wiki roles (read once per request), so the synchronous access checks work.
// Every resolved actor carries its wiki roles (read once per request), so the synchronous access checks work.
const viewers = { ...baseViewers, resolve: async (req, opts) => svc.loadRoles(await baseViewers.resolve(req, opts)) };
const viewers = { ...baseViewers, resolve: async (req, opts) => svc.loadRoles(await baseViewers.resolve(req, opts)) };
@@ -111,7 +111,7 @@ function createApp({ config, telemetry = null, svc, viewers: baseViewers, platfo
@@ -111,7 +111,7 @@ function createApp({ config, telemetry = null, svc, viewers: baseViewers, platfo
// Per-actor limits (http/actor-limits.js) on /api/v1 and the editing forms, counted once each router
// Per-actor limits (http/actor-limits.js) on /api/v1 and the editing forms, counted once each router
// resolved req.actor; the per-address limits here stay. limitsNow: the limiter's clock (tests).
// resolved req.actor; the per-address limits here stay. limitsNow: the limiter's clock (tests).
const limits = createActorLimits({ config, now: limitsNow || (() => Date.now()), registry: metrics.registry, log, enabled: rateLimits, valkey });
const limits = createActorLimits({ config, now: limitsNow || (() => Date.now()), registry: metrics.registry, log, enabled: rateLimits, valkey });
- app.use('/api/v1', createApi({ svc, viewers, platform, config, log, limits }));
+ app.use('/api/v1', createApi({ svc, viewers, platform, config, log, limits, vibes }));
app.use('/api', (req, res) => require('openvibe-contracts').http.sendProblem(res, 404, 'route.not_found', { detail: 'Not found' }));
app.use('/api', (req, res) => require('openvibe-contracts').http.sendProblem(res, 404, 'route.not_found', { detail: 'Not found' }));
// This site's own pinned copy of the OpenVibe Frame's browser files (openvibe-shared/serve).
// This site's own pinned copy of the OpenVibe Frame's browser files (openvibe-shared/serve).
@@ -133,7 +133,7 @@ function createApp({ config, telemetry = null, svc, viewers: baseViewers, platfo
@@ -133,7 +133,7 @@ function createApp({ config, telemetry = null, svc, viewers: baseViewers, platfo
app.post(['/new-space', '/s/*', '/w/*', '/proposals/*'], limiter(10 * 60000, 120));
app.post(['/new-space', '/s/*', '/w/*', '/proposals/*'], limiter(10 * 60000, 120));
// Search scans the text of every published page (LIKE): a person's pace, not a crawler's.
// Search scans the text of every published page (LIKE): a person's pace, not a crawler's.
app.get('/search', limiter(60000, 60));
app.get('/search', limiter(60000, 60));
- const pages = createPages({ svc, viewers, platform, config, log, limits });
+ const pages = createPages({ svc, viewers, platform, config, log, limits, vibes });
app.use(pages.router);
app.use(pages.router);
app.use((req, res) => pages.errorPage(req, res, 404, 'Page not found', 'Nothing lives at that address.'));
app.use((req, res) => pages.errorPage(req, res, 404, 'Page not found', 'Nothing lives at that address.'));
@@ -71,6 +71,15 @@ function load(env = process.env) {
@@ -71,6 +71,15 @@ function load(env = process.env) {
mediaPublicUrl: trim(env.OV_MEDIA_URL || 'https://openvibe.media'),
mediaPublicUrl: trim(env.OV_MEDIA_URL || 'https://openvibe.media'),
// Media namespace (app) Wiki's attachments live in, checked with media.object.read.
// Media namespace (app) Wiki's attachments live in, checked with media.object.read.
mediaApp: env.WIKI_MEDIA_APP || 'wiki',
mediaApp: env.WIKI_MEDIA_APP || 'wiki',
+ // "Vibe a wiki" (server/wiki/vibe.js): OpenVibe.AI researches and writes; these bound how many and how big.
+ aiInternalUrl: trim(env.OV_AI_INTERNAL_URL || ''),
+ vibe: {
+ perPersonDay: int(env.WIKI_VIBE_PER_PERSON_DAY, 3),
+ perDay: int(env.WIKI_VIBE_PER_DAY, 30),
+ concurrency: Math.max(1, int(env.WIKI_VIBE_CONCURRENCY, 2)),
+ intervalMs: int(env.WIKI_VIBE_INTERVAL_MS, 5000),
+ pages: { quick: int(env.WIKI_VIBE_PAGES_QUICK, 6), standard: int(env.WIKI_VIBE_PAGES_STANDARD, 12), big: int(env.WIKI_VIBE_PAGES_BIG, 24) },
+ },
mediaVerifyIntervalMs: int(env.WIKI_MEDIA_VERIFY_INTERVAL_MS, 6 * 60 * 60 * 1000),
mediaVerifyIntervalMs: int(env.WIKI_MEDIA_VERIFY_INTERVAL_MS, 6 * 60 * 60 * 1000),
// OpenVibe.VIP: who may read a VIP space or page (server/integrations/vip.js, WS-K task 8).
// OpenVibe.VIP: who may read a VIP space or page (server/integrations/vip.js, WS-K task 8).
@@ -56,6 +56,8 @@ function serviceItself(req) {
@@ -56,6 +56,8 @@ function serviceItself(req) {
const BUDGETS = {
const BUDGETS = {
// A space is a new tree with its own roles: people start a few.
// A space is a new tree with its own roles: people start a few.
'wiki.space.create': { minute: 10, hour: 60 },
'wiki.space.create': { minute: 10, hour: 60 },
+ // A generated wiki is a new space and minutes of research and writing (the daily caps are in server/wiki/vibe.js).
+ 'wiki.vibe.create': { minute: 3, hour: 10 },
// Space settings, roles, and moves, visibility, media and deletes of a page: a person saves a form
// Space settings, roles, and moves, visibility, media and deletes of a page: a person saves a form
// now and then.
// now and then.
'wiki.space.update': { minute: 30, hour: 300 },
'wiki.space.update': { minute: 30, hour: 300 },
@@ -34,6 +34,10 @@
@@ -34,6 +34,10 @@
* GET /proposals/:id wiki.page.read
* GET /proposals/:id wiki.page.read
* POST /proposals/:id/review wiki.revision.publish { decision: approved|rejected, note?, publish? } a person
* POST /proposals/:id/review wiki.revision.publish { decision: approved|rejected, note?, publish? } a person
* GET /search?q=&space=&limit= wiki.search.query
* GET /search?q=&space=&limit= wiki.search.query
+ * POST /vibes wiki.space.create { topic, size?: quick|standard|big, visibility?: public|private, lang? } a person: a whole
+ * generated wiki (server/wiki/vibe.js) → 202 { vibe }, then poll
+ * GET /vibes the caller's generated wikis (a person)
+ * GET /vibes/:vibe wiki.page.read progress: status, pages written, the wiki's address
*/
*/
const express = require('express');
const express = require('express');
const contracts = require('openvibe-contracts');
const contracts = require('openvibe-contracts');
@@ -78,7 +82,7 @@ function serializeTree(nodes, svc, space) {
@@ -78,7 +82,7 @@ function serializeTree(nodes, svc, space) {
function int(v) { const n = Number(v); return Number.isInteger(n) ? n : undefined; }
function int(v) { const n = Number(v); return Number.isInteger(n) ? n : undefined; }
-function createApi({ svc, viewers, platform, config, log = console, limits }) {
+function createApi({ svc, viewers, platform, config, log = console, limits, vibes = null }) {
const router = express.Router();
const router = express.Router();
const ownOrigin = new URL(config.baseUrl).origin;
const ownOrigin = new URL(config.baseUrl).origin;
router.use(http.middleware());
router.use(http.middleware());
@@ -132,6 +136,20 @@ function createApi({ svc, viewers, platform, config, log = console, limits }) {
@@ -132,6 +136,20 @@ function createApi({ svc, viewers, platform, config, log = console, limits }) {
};
};
const pageOut = async (page) => serializePage(page, svc, await svc.spaceById(page.space_id));
const pageOut = async (page) => serializePage(page, svc, await svc.spaceById(page.space_id));
+ // Generated wikis ("vibe a wiki")
+ const vibesOn = () => { if (!vibes) throw new svc.WikiError(503, 'vibe.unavailable', 'Generated wikis are not available here'); return vibes; };
+ router.post('/vibes', guard('wiki.space.create'), B('wiki.vibe.create'), R(async (req) => {
+ const b = req.body || {};
+ const { vibe, space } = await vibesOn().start({ topic: b.topic, size: b.size || 'standard', visibility: b.visibility || 'public', lang: b.lang || 'en' }, req.actor);
+ return { vibe: await vibesOn().get(vibe.id, req.actor), space: serializeSpace(space), progress_url: `${config.baseUrl}/vibe/${vibe.id}` };
+ }, 202));
+ router.get('/vibes', R(async (req) => ({ vibes: await vibesOn().mine(req.actor) })));
+ router.get('/vibes/:vibe', guard('wiki.page.read'), R(async (req) => {
+ const v = await vibesOn().get(req.params.vibe, req.actor);
+ if (!v) throw new svc.WikiError(404, 'vibe.not_found', 'No such generated wiki');
+ return { vibe: v };
+ }));
+
// Spaces
// Spaces
router.get('/spaces', R(async (req) => ({ spaces: (await svc.listSpaces(req.actor)).map(serializeSpace) })));
router.get('/spaces', R(async (req) => ({ spaces: (await svc.listSpaces(req.actor)).map(serializeSpace) })));
router.post('/spaces', guard('wiki.space.create'), B('wiki.space.create'), R(async (req) => ({ space: serializeSpace(await svc.createSpace(req.body || {}, req.actor)) }), 201));
router.post('/spaces', guard('wiki.space.create'), B('wiki.space.create'), R(async (req) => ({ space: serializeSpace(await svc.createSpace(req.body || {}, req.actor)) }), 201));
@@ -4,6 +4,8 @@
@@ -4,6 +4,8 @@
* form posts; the shared navbar is progressive enhancement.
* form posts; the shared navbar is progressive enhancement.
*
*
* GET / spaces + recently published
* GET / spaces + recently published
+ * GET|POST /vibe "vibe a wiki": any topic → a generated wiki (server/wiki/vibe.js)
+ * GET /vibe/:id, /vibe/:id.json its progress (live with JavaScript; refreshes itself without)
* GET /recent, /search?q=
* GET /recent, /search?q=
* GET|POST /new-space
* GET|POST /new-space
* GET /s/:space the page tree
* GET /s/:space the page tree
@@ -31,7 +33,7 @@ const ovServe = require('openvibe-shared/serve');
@@ -31,7 +33,7 @@ const ovServe = require('openvibe-shared/serve');
const cache = require('openvibe-shared/cache-policy');
const cache = require('openvibe-shared/cache-policy');
const seo = require('openvibe-publishing/seo');
const seo = require('openvibe-publishing/seo');
const ssr = require('openvibe-publishing/ssr');
const ssr = require('openvibe-publishing/ssr');
-const { renderPage, AI_SUMMARY } = require('../render/layout');
+const { renderPage, AI_SUMMARY, asset } = require('../render/layout');
const views = require('../render/views');
const views = require('../render/views');
const frame = require('openvibe-shared/frame');
const frame = require('openvibe-shared/frame');
const showcase = require('openvibe-shared/showcase');
const showcase = require('openvibe-shared/showcase');
@@ -43,7 +45,7 @@ const { createFrameGroups } = require('./frame-groups');
@@ -43,7 +45,7 @@ const { createFrameGroups } = require('./frame-groups');
// What the home says in search results and share cards (≤ 160 characters).
// What the home says in search results and share cards (≤ 160 characters).
const HOME_DESCRIPTION = 'Read the OpenVibe wiki or start your own: every page keeps its full revision history, lists the sources behind it and has a discussion thread.';
const HOME_DESCRIPTION = 'Read the OpenVibe wiki or start your own: every page keeps its full revision history, lists the sources behind it and has a discussion thread.';
-function createPages({ svc, viewers, platform, config, log = console, limits, frameGroups = createFrameGroups({ url: config.frameUrl }) }) {
+function createPages({ svc, viewers, platform, config, log = console, limits, vibes = null, frameGroups = createFrameGroups({ url: config.frameUrl }) }) {
const router = express.Router();
const router = express.Router();
const form = express.urlencoded({ extended: false, limit: '600kb' });
const form = express.urlencoded({ extended: false, limit: '600kb' });
router.use(actorMiddleware(viewers, { services: false }));
router.use(actorMiddleware(viewers, { services: false }));
@@ -126,19 +128,56 @@ function createPages({ svc, viewers, platform, config, log = console, limits, fr
@@ -126,19 +128,56 @@ function createPages({ svc, viewers, platform, config, log = console, limits, fr
// ── Home, recent, search ─────────────────────────────────
// ── Home, recent, search ─────────────────────────────────
router.get('/', wrap(async (req, res) => {
router.get('/', wrap(async (req, res) => {
if (req.actor.subject) await svc.access.prepareVip(req.actor, (await svc.vipSpaces()).map((space) => ({ space }))); // VIP spaces the viewer may see
if (req.actor.subject) await svc.access.prepareVip(req.actor, (await svc.vipSpaces()).map((space) => ({ space }))); // VIP spaces the viewer may see
- const [spaces, recent, stats, linked, groups] = await Promise.all([svc.listSpaces(req.actor), svc.recentChanges(8), svc.homeStats(), svc.mostLinked(6), frameGroups.get()]);
+ const [spaces, recent, stats, linked, groups, generated] = await Promise.all([svc.listSpaces(req.actor), svc.recentChanges(8), svc.homeStats(), svc.mostLinked(6), frameGroups.get(), vibes ? vibes.recent(6).catch(() => []) : []]);
// The official wikis' shelves: what is inside them, on the front page (only public spaces: the home is cached).
// The official wikis' shelves: what is inside them, on the front page (only public spaces: the home is cached).
const shelves = new Map(await Promise.all(spaces.filter((s) => s.kind === 'official' && s.visibility === 'public').slice(0, 3)
const shelves = new Map(await Promise.all(spaces.filter((s) => s.kind === 'official' && s.visibility === 'public').slice(0, 3)
.map(async (s) => [s.id, await svc.shelf(s, req.actor, 60)])));
.map(async (s) => [s.id, await svc.shelf(s, req.actor, 60)])));
// The site's one-line AI summary: the shell turns it into the ai-summary meta and the home's
// The site's one-line AI summary: the shell turns it into the ai-summary meta and the home's
// WebPage JSON-LD (openvibe-publishing/layout v1.3.0 forwards `summary`).
// WebPage JSON-LD (openvibe-publishing/layout v1.3.0 forwards `summary`).
- send(req, res, 200, views.home({ spaces, recent, shelves, stats, linked, groups, actor: req.actor }) + frame.shipped({ service: 'wiki', title: 'Recently shipped on OpenVibe.Wiki' }), {
+ send(req, res, 200, views.home({ spaces, recent, shelves, stats, linked, groups, generated, actor: req.actor }) + frame.shipped({ service: 'wiki', title: 'Recently shipped on OpenVibe.Wiki' }), {
robots: 'index, follow', cache: 'public', active: 'home', path: '/', styles: [showcase.STYLESHEET],
robots: 'index, follow', cache: 'public', active: 'home', path: '/', styles: [showcase.STYLESHEET],
summary: AI_SUMMARY, description: HOME_DESCRIPTION,
summary: AI_SUMMARY, description: HOME_DESCRIPTION,
jsonLd: views.homeJsonLd({ shelves, spaces: spaces.filter((s) => s.kind === 'official' && s.visibility === 'public'), baseUrl: config.baseUrl }),
jsonLd: views.homeJsonLd({ shelves, spaces: spaces.filter((s) => s.kind === 'official' && s.visibility === 'public'), baseUrl: config.baseUrl }),
});
});
}));
}));
// What shipped on OpenVibe.Wiki: the shared update log every OpenVibe site has.
// What shipped on OpenVibe.Wiki: the shared update log every OpenVibe site has.
+ // ── Vibe a wiki ──────────────────────────────────────────
+ const VIBE_STYLES = [showcase.STYLESHEET];
+ const VIBE_HEAD = `<link rel="stylesheet" href="${asset('css/vibe.css')}">`;
+ const VIBE_DESCRIPTION = 'Name any topic and OpenVibe researches it across the web, then writes you a whole wiki: pages, pictures and the sources behind every claim.';
+ const vibeCtx = async (req) => ({
+ mine: vibes && req.actor.subject ? await vibes.mine(req.actor, 10).catch(() => []) : [],
+ recent: vibes ? await vibes.recent(12).catch(() => []) : [],
+ perPersonDay: (config.vibe && config.vibe.perPersonDay) || 3,
+ });
+ router.get('/vibe', wrap(async (req, res) => {
+ const topic = typeof req.query.topic === 'string' ? req.query.topic.slice(0, 120) : '';
+ send(req, res, 200, views.vibePage({ actor: req.actor, topic, ...await vibeCtx(req) }), {
+ title: 'Vibe a wiki: any topic, a whole wiki with sources', description: VIBE_DESCRIPTION, robots: topic ? 'noindex, follow' : 'index, follow',
+ cache: topic ? null : 'public', path: '/vibe', active: 'vibe', styles: VIBE_STYLES, head: VIBE_HEAD,
+ });
+ }));
+ router.post('/vibe', B('wiki.vibe.create'), form, wrap(async (req, res) => {
+ if (!req.actor.subject) return needSignIn(req, res, 'Sign in with your OpenVibe account to vibe a wiki.');
+ const b = req.body || {};
+ try {
+ if (!vibes) throw Object.assign(new Error('Generated wikis are not available here'), { status: 503 });
+ const { vibe } = await vibes.start({ topic: b.topic, size: b.size || 'standard', visibility: b.visibility || 'public' }, req.actor);
+ res.redirect(303, `/vibe/${vibe.id}`);
+ } catch (err) {
+ if (!err || !err.status || err.status >= 500 && err.status !== 503) throw err;
+ send(req, res, err.status, views.vibePage({ actor: req.actor, topic: String(b.topic || '').slice(0, 120), error: err.message, ...await vibeCtx(req) }), { title: 'Vibe a wiki', robots: 'noindex, nofollow', path: '/vibe', active: 'vibe', styles: VIBE_STYLES, head: VIBE_HEAD });
+ }
+ }));
+ router.get('/vibe/:vibe', wrap(async (req, res) => {
+ const json = /\.json$/.test(req.params.vibe);
+ const id = json ? req.params.vibe.slice(0, -5) : req.params.vibe;
+ const v = vibes ? await vibes.get(id, req.actor) : null;
+ if (json) return res.status(v ? 200 : 404).set('Cache-Control', cache.htmlHeaders({ private: true })).json(v || { error: 'not_found' });
+ if (!v) return notFound(req, res);
+ send(req, res, 200, views.vibeProgress({ vibe: v }), { title: v.status === 'done' ? `A wiki about ${v.topic}` : `Vibing a wiki about ${v.topic}`, robots: 'noindex, nofollow', path: `/vibe/${v.id}`, active: 'vibe', styles: VIBE_STYLES, head: VIBE_HEAD });
+ }));
+
router.get('/updates', (req, res) => send(req, res, 200, frame.updatesBody({ service: 'wiki', siteName: 'OpenVibe.Wiki' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>`, { title: 'What shipped on OpenVibe.Wiki', robots: 'index, follow', cache: 'public', path: '/updates' }));
router.get('/updates', (req, res) => send(req, res, 200, frame.updatesBody({ service: 'wiki', siteName: 'OpenVibe.Wiki' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>`, { title: 'What shipped on OpenVibe.Wiki', robots: 'index, follow', cache: 'public', path: '/updates' }));
router.get('/recent', wrap(async (req, res) => send(req, res, 200, views.recentPage({ items: await svc.recentChanges(100) }), { title: 'Recent changes', robots: 'noindex, follow', cache: 'public', active: 'recent' })));
router.get('/recent', wrap(async (req, res) => send(req, res, 200, views.recentPage({ items: await svc.recentChanges(100) }), { title: 'Recent changes', robots: 'noindex, follow', cache: 'public', active: 'recent' })));
router.get('/search', wrap(async (req, res) => {
router.get('/search', wrap(async (req, res) => {
@@ -15,6 +15,7 @@ const { load } = require('./config');
@@ -15,6 +15,7 @@ const { load } = require('./config');
const { openDb, migrate, createStores } = require('./db');
const { openDb, migrate, createStores } = require('./db');
const { createPlatform } = require('./integrations/platform');
const { createPlatform } = require('./integrations/platform');
const { createWikiService } = require('./wiki/service');
const { createWikiService } = require('./wiki/service');
+const { createVibes } = require('./wiki/vibe');
const { createKeyStore } = require('./auth/keys');
const { createKeyStore } = require('./auth/keys');
const { createViewerResolver } = require('./auth/viewer');
const { createViewerResolver } = require('./auth/viewer');
const { createApp } = require('./app');
const { createApp } = require('./app');
@@ -44,7 +45,7 @@ function createLifecycle({ server, db, valkey = null, platform, timers = [], tel
@@ -44,7 +45,7 @@ function createLifecycle({ server, db, valkey = null, platform, timers = [], tel
});
});
}
}
-async function start({ config, db: givenDb = null, now = () => Date.now(), fetchImpl = globalThis.fetch, tokens = null, publicKey = null, log = console, telemetrySink = null, listen = true, workers = listen, rateLimits = true, limitsNow = null, indexnow: givenIndexNow = null, signals = false, exit = () => {}, accountSend: givenSend = null } = {}) {
+async function start({ config, db: givenDb = null, now = () => Date.now(), fetchImpl = globalThis.fetch, tokens = null, imageFetch = null, publicKey = null, log = console, telemetrySink = null, listen = true, workers = listen, rateLimits = true, limitsNow = null, indexnow: givenIndexNow = null, signals = false, exit = () => {}, accountSend: givenSend = null } = {}) {
config = config || load();
config = config || load();
// PostgreSQL (ADR-035): migrate on the owner's direct connection, then serve on the pooled runtime role.
// PostgreSQL (ADR-035): migrate on the owner's direct connection, then serve on the pooled runtime role.
// Tests hand in a migrated database of their own (test/db-helper.js).
// Tests hand in a migrated database of their own (test/db-helper.js).
@@ -70,6 +71,8 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch
@@ -70,6 +71,8 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch
const valkey = config.valkey.url ? require('openvibe-sdk/valkey').createValkey({ url: config.valkey.url, prefix: config.valkey.prefix, log }) : null;
const valkey = config.valkey.url ? require('openvibe-sdk/valkey').createValkey({ url: config.valkey.url, prefix: config.valkey.prefix, log }) : null;
const platform = createPlatform({ config, db, fetchImpl, tokens, now, log });
const platform = createPlatform({ config, db, fetchImpl, tokens, now, log });
const svc = createWikiService({ db, stores, outbox: platform.outbox, config, community: platform.community, vip: platform.vip, now, log, indexnow });
const svc = createWikiService({ db, stores, outbox: platform.outbox, config, community: platform.community, vip: platform.vip, now, log, indexnow });
+ // "Vibe a wiki" (server/wiki/vibe.js): generated wikis, researched and written through OpenVibe.AI.
+ const vibes = createVibes({ db, svc, platform, config, now, log, ...(imageFetch ? { fetchImg: imageFetch } : {}) });
const reconciled = await svc.reconcileIndex();
const reconciled = await svc.reconcileIndex();
if (reconciled.sent) log.log(`[Wiki] re-sent ${reconciled.sent} Search document(s) whose indexability changed`);
if (reconciled.sent) log.log(`[Wiki] re-sent ${reconciled.sent} Search document(s) whose indexability changed`);
const keys = createKeyStore({ config, fetchImpl, log, publicKey });
const keys = createKeyStore({ config, fetchImpl, log, publicKey });
@@ -86,7 +89,7 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch
@@ -86,7 +89,7 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch
// on, a one-minute tick writes each interval (the collector has no timer of its own) and the table keeps 14 days.
// on, a one-minute tick writes each interval (the collector has no timer of its own) and the table keeps 14 days.
const telemetry = createHttpTelemetry({ service: 'wiki', sink: telemetrySink || createPgTelemetrySink({ db, log }), skipPrefixes: ['/shared'], intervalMs: 60_000 });
const telemetry = createHttpTelemetry({ service: 'wiki', sink: telemetrySink || createPgTelemetrySink({ db, log }), skipPrefixes: ['/shared'], intervalMs: 60_000 });
let retention = null;
let retention = null;
- const app = createApp({ config, telemetry, svc, viewers, platform, keys, db, valkey, log, rateLimits, fetchImpl, limitsNow, indexnow, accountData, accountSend });
+ const app = createApp({ config, telemetry, svc, viewers, platform, keys, db, valkey, log, rateLimits, fetchImpl, limitsNow, indexnow, accountData, accountSend, vibes });
const timers = [];
const timers = [];
if (workers) {
if (workers) {
@@ -105,6 +108,7 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch
@@ -105,6 +108,7 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch
timers.push(setInterval(() => svc.verifyAllMedia(platform.media.resolve).catch((err) => log.warn(`[Wiki] media check: ${err.message}`)), config.mediaVerifyIntervalMs));
timers.push(setInterval(() => svc.verifyAllMedia(platform.media.resolve).catch((err) => log.warn(`[Wiki] media check: ${err.message}`)), config.mediaVerifyIntervalMs));
}
}
if (platform.eventsConfigured) platform.outbox.start();
if (platform.eventsConfigured) platform.outbox.start();
+ if (platform.ai.configured && platform.media.configured) vibes.startWorker();
timers.push(setInterval(() => platform.outbox.refreshCounts().catch(() => {}), 30 * 1000));
timers.push(setInterval(() => platform.outbox.refreshCounts().catch(() => {}), 30 * 1000));
// wiki.projects on Network (Contracts 0.41.0): people whose spaces or roles changed, every minute.
// wiki.projects on Network (Contracts 0.41.0): people whose spaces or roles changed, every minute.
const projects = require('./integrations/projects-module').createProjectsModule({ db, config, tokens: platform.tokenClient, fetchImpl, now, log });
const projects = require('./integrations/projects-module').createProjectsModule({ db, config, tokens: platform.tokenClient, fetchImpl, now, log });
@@ -126,12 +130,12 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch
@@ -126,12 +130,12 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch
});
});
}
}
- const lifecycle = createLifecycle({ server, db, telemetry, retention, valkey, platform, timers, closeDb: !givenDb, signals, exit, log, extra: [() => { if (subscriptions) subscriptions.stop(); }] });
+ const lifecycle = createLifecycle({ server, db, telemetry, retention, valkey, platform, timers, closeDb: !givenDb, signals, exit, log, extra: [() => { if (subscriptions) subscriptions.stop(); }, () => vibes.stopWorker()] });
async function stop() {
async function stop() {
return lifecycle.stop('stop');
return lifecycle.stop('stop');
}
}
- return { app, db, telemetry, svc, stores, platform, keys, viewers, server, config, indexnow, timers, lifecycle, stop };
+ return { app, db, telemetry, svc, stores, platform, keys, viewers, server, config, indexnow, timers, lifecycle, stop, vibes };
}
}
module.exports = { start, createLifecycle };
module.exports = { start, createLifecycle };
new file mode 100644
new file mode 100644
@@ -0,0 +1,72 @@
@@ -0,0 +1,72 @@
+'use strict';
+/**
+ * Pictures from the web for generated wikis, fetched under openvibe-shared/egress (the network's one rule for addresses
+ * someone else chose): https only, no credentials or IP literals in the URL, the connection only to public addresses
+ * (egress.safeLookup checks every DNS answer inside the connection's own lookup, so an answer cannot point it inside the
+ * network), at most three redirects each judged the same way, a byte cap, a timeout, and bytes that really are a JPEG,
+ * PNG, GIF or WebP whatever the server says.
+ */
+const net = require('net');
+const https = require('https');
+const egress = require('openvibe-shared/egress');
+
+const MAX_BYTES = 8 * 1024 * 1024;
+const USER_AGENT = 'OpenVibeWiki/1.0 (+https://openvibe.wiki; [email protected])';
+const { isPublicAddress } = egress;
+
+/** The picture type the first bytes show, or null. */
+function sniff(buf) {
+ if (buf.length >= 3 && buf[0] === 0xff && buf[1] === 0xd8 && buf[2] === 0xff) return 'image/jpeg';
+ if (buf.length >= 8 && buf.slice(0, 8).equals(Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]))) return 'image/png';
+ if (buf.length >= 6 && /^GIF8[79]a$/.test(buf.slice(0, 6).toString('latin1'))) return 'image/gif';
+ if (buf.length >= 12 && buf.slice(0, 4).toString('latin1') === 'RIFF' && buf.slice(8, 12).toString('latin1') === 'WEBP') return 'image/webp';
+ return null;
+}
+
+function checkUrl(raw) {
+ let u;
+ try { u = new URL(String(raw)); } catch { throw new Error('not a URL'); }
+ if (u.protocol !== 'https:') throw new Error('only https pictures are fetched');
+ if (u.username || u.password) throw new Error('credentials in a URL are refused');
+ if (net.isIP(u.hostname.replace(/^\[|\]$/g, ''))) throw new Error('IP-literal hosts are refused');
+ if (egress.isInternalName(u.hostname)) throw new Error(`${u.hostname} is not a public host name`);
+ return u;
+}
+
+/** GET a picture → { bytes, mime, url } (url: where it was read after redirects). */
+function fetchImage(raw, { timeoutMs = 15000, maxBytes = MAX_BYTES, signal = null, redirects = 3, lookup = egress.safeLookup } = {}) {
+ return new Promise((resolve, reject) => {
+ let u;
+ try { u = checkUrl(raw); } catch (err) { return reject(err); }
+ const req = https.request(u, { method: 'GET', lookup, headers: { 'User-Agent': USER_AGENT, Accept: 'image/webp,image/png,image/jpeg,image/gif;q=0.9,*/*;q=0.1' }, timeout: timeoutMs });
+ const onAbort = () => { req.destroy(new Error('aborted')); };
+ if (signal) { if (signal.aborted) return onAbort(); signal.addEventListener('abort', onAbort, { once: true }); }
+ const done = (fn, v) => { if (signal) signal.removeEventListener('abort', onAbort); fn(v); };
+ req.on('timeout', () => req.destroy(new Error('timed out')));
+ req.on('error', (err) => done(reject, err));
+ req.on('response', (res) => {
+ if ([301, 302, 303, 307, 308].includes(res.statusCode) && res.headers.location) {
+ res.resume();
+ if (redirects <= 0) return done(reject, new Error('too many redirects'));
+ const next = new URL(res.headers.location, u).toString();
+ return fetchImage(next, { timeoutMs, maxBytes, signal, redirects: redirects - 1, lookup }).then((v) => done(resolve, v), (e) => done(reject, e));
+ }
+ if (res.statusCode !== 200) { res.resume(); return done(reject, new Error(`${u.hostname} answered ${res.statusCode}`)); }
+ const declared = Number(res.headers['content-length'] || 0);
+ if (declared > maxBytes) { res.destroy(); return done(reject, new Error('picture too large')); }
+ const chunks = [];
+ let n = 0;
+ res.on('data', (c) => { n += c.length; if (n > maxBytes) { res.destroy(); req.destroy(); done(reject, new Error('picture too large')); } else chunks.push(c); });
+ res.on('error', (err) => done(reject, err));
+ res.on('end', () => {
+ const bytes = Buffer.concat(chunks);
+ const mime = sniff(bytes);
+ if (!mime) return done(reject, new Error('not a JPEG, PNG, GIF or WebP picture'));
+ done(resolve, { bytes, mime, url: u.toString() });
+ });
+ });
+ req.end();
+ });
+}
+
+module.exports = { fetchImage, sniff, isPublicAddress, checkUrl, MAX_BYTES };
@@ -1,7 +1,8 @@
@@ -1,7 +1,8 @@
'use strict';
'use strict';
/**
/**
* Clients for the services Wiki composes (roadmap §29): Events (outbox relay), Community
* Clients for the services Wiki composes (roadmap §29): Events (outbox relay), Community
- * (discussion threads), Sources (citation items) and Media (attachment checks). Each call uses a
+ * (discussion threads), Sources (citation items), Media (attachment checks, and pictures a generated wiki
+ * uploads) and AI (research and writing for "vibe a wiki"). Each call uses a
* Network client-credentials token for svc:wiki, minted per audience by the SDK token client.
* Network client-credentials token for svc:wiki, minted per audience by the SDK token client.
* Hiding the thread of a page that stopped being public needs community.comment.moderate.
* Hiding the thread of a page that stopped being public needs community.comment.moderate.
*
*
@@ -12,6 +13,7 @@ const { createClient } = require('openvibe-sdk/core');
@@ -12,6 +13,7 @@ const { createClient } = require('openvibe-sdk/core');
const { createServiceTokenClient } = require('openvibe-sdk/auth');
const { createServiceTokenClient } = require('openvibe-sdk/auth');
const { createEventsClient, createPgOutbox } = require('openvibe-sdk/events');
const { createEventsClient, createPgOutbox } = require('openvibe-sdk/events');
const { createDiscussionClient } = require('openvibe-publishing/discussion');
const { createDiscussionClient } = require('openvibe-publishing/discussion');
+const crypto = require('crypto');
class IntegrationError extends Error {
class IntegrationError extends Error {
constructor(status, code, message) { super(message); this.status = status; this.code = code; }
constructor(status, code, message) { super(message); this.status = status; this.code = code; }
@@ -180,11 +182,83 @@ function createPlatform({ config, db, fetchImpl = globalThis.fetch, tokens = nul
@@ -180,11 +182,83 @@ function createPlatform({ config, db, fetchImpl = globalThis.fetch, tokens = nul
},
},
};
};
+ /**
+ * A picture into Media for a page (media.object.upload, namespace WIKI_MEDIA_APP): init, bytes, complete (v2 objects),
+ * owned by the person it is uploaded for (X-OV-Subject), public, with where it came from in its metadata.
+ * → the object id (med_…). Throws IntegrationError on any refusal.
+ */
+ media.upload = async function upload({ bytes, mime, filename, owner = null, metadata = {}, timeoutMs = 30000 }) {
+ if (!media.configured) throw new IntegrationError(503, 'media.unavailable', 'OpenVibe.Media is not configured (OV_MEDIA_INTERNAL_URL)');
+ const base = `${config.mediaInternalUrl}/api/v2/${encodeURIComponent(config.mediaApp)}/objects`;
+ const auth = await forAudience('openvibe.media').authHeaders();
+ const subject = owner && SUBJECT_RE.test(owner) ? { 'X-OV-Subject': owner } : {};
+ const call = async (url, init) => {
+ let res;
+ try { res = await fetchImpl(url, { ...init, signal: AbortSignal.timeout(timeoutMs) }); } catch (err) {
+ throw new IntegrationError(503, 'media.unavailable', `Media did not answer: ${err && err.name === 'TimeoutError' ? 'timeout' : err && err.message}`);
+ }
+ const body = await res.json().catch(() => null);
+ if (res.status === 401) forAudience('openvibe.media').invalidate();
+ if (res.status < 200 || res.status > 299) throw new IntegrationError(res.status === 413 || res.status === 415 || res.status === 422 ? 422 : 503, 'media.upload_refused', `Media answered ${res.status}${body && (body.code || body.title) ? ` (${body.code || body.title})` : ''}`);
+ return body || {};
+ };
+ const init = await call(base, {
+ method: 'POST', headers: { ...auth, ...subject, 'Content-Type': 'application/json', Accept: 'application/json' },
+ body: JSON.stringify({ kind: 'asset', visibility: 'public', mime_type: mime, size_bytes: bytes.length, filename: String(filename || 'picture').slice(0, 120), content_hash: crypto.createHash('sha256').update(bytes).digest('hex'), metadata }),
+ });
+ const id = init.id || (init.object && init.object.id);
+ const up = init.upload || {};
+ if (!id || !up.url) throw new IntegrationError(503, 'media.upload_refused', 'Media did not give an upload address');
+ // The upload and complete addresses may name Media's public origin; Wiki reaches it on the internal one.
+ const internal = (u) => { const x = new URL(u, config.mediaInternalUrl); return `${config.mediaInternalUrl}${x.pathname}${x.search}`; };
+ await call(internal(up.url), { method: 'PUT', headers: { ...auth, ...subject, 'Content-Type': mime, ...(up.token ? { 'X-Upload-Token': up.token } : {}) }, body: bytes });
+ await call(internal(up.complete_url || `${base}/${encodeURIComponent(id)}/complete`), { method: 'POST', headers: { ...auth, ...subject, 'Content-Type': 'application/json', Accept: 'application/json', ...(up.token ? { 'X-Upload-Token': up.token } : {}) }, body: '{}' });
+ return id;
+ };
+
+ // ── AI: research and writing for generated wikis (ai.run.create / ai.run.read, namespaces research.* and wiki.*) ──
+ const ai = {
+ configured: !!(tokenClient && config.aiInternalUrl),
+ /**
+ * Run a workflow and wait for it: → the run's output. Long runs are polled. Throws IntegrationError with the run's
+ * own code when it failed, 429 with retryAfter (seconds) when AI's quota or queue is full.
+ */
+ async run(workflow, input, { subject = null, idempotencyKey = null, waitMs = 55000, totalMs = 300000, signal = null } = {}) {
+ if (!ai.configured) throw new IntegrationError(503, 'ai.unavailable', 'OpenVibe.AI is not configured (OV_AI_INTERNAL_URL)');
+ const headers = async () => ({ ...(await forAudience('openvibe.ai').authHeaders()), 'Content-Type': 'application/json', Accept: 'application/json', ...(idempotencyKey ? { 'Idempotency-Key': idempotencyKey } : {}) });
+ const call = async (url, init) => {
+ let res;
+ try { res = await fetchImpl(url, { ...init, signal: signal ? AbortSignal.any([signal, AbortSignal.timeout(waitMs + 15000)]) : AbortSignal.timeout(waitMs + 15000) }); } catch (err) {
+ throw new IntegrationError(503, 'ai.unavailable', `OpenVibe.AI did not answer: ${err && err.name === 'TimeoutError' ? 'timeout' : err && err.message}`);
+ }
+ const body = await res.json().catch(() => null);
+ if (res.status === 401) forAudience('openvibe.ai').invalidate();
+ if (res.status === 429) throw Object.assign(new IntegrationError(429, (body && body.code) || 'ai.busy', (body && (body.detail || body.title)) || 'OpenVibe.AI is busy'), { retryAfter: Number(res.headers.get('retry-after')) || 60 });
+ if (res.status >= 400) throw new IntegrationError(res.status >= 500 ? 503 : res.status, (body && body.code) || 'ai.refused', (body && (body.detail || body.title)) || `OpenVibe.AI answered ${res.status}`);
+ return body && body.run;
+ };
+ const body = { workflow, input, ...(subject ? { on_behalf_of: { type: 'user', id: subject }, attribution: { service: 'wiki', type: 'user', id: subject } } : {}) };
+ let run = await call(`${config.aiInternalUrl}/api/v1/runs?wait=${waitMs}`, { method: 'POST', headers: await headers(), body: JSON.stringify(body) });
+ const until = Date.now() + totalMs;
+ while (run && !['succeeded', 'cached', 'failed', 'cancelled'].includes(run.status)) {
+ if (Date.now() > until) throw new IntegrationError(504, 'ai.timeout', `${workflow} did not finish in ${Math.round(totalMs / 1000)} s`);
+ await new Promise((r) => setTimeout(r, 3000));
+ run = await call(`${config.aiInternalUrl}/api/v1/runs/${encodeURIComponent(run.id)}`, { method: 'GET', headers: await headers() });
+ }
+ if (!run) throw new IntegrationError(503, 'ai.unavailable', 'OpenVibe.AI answered without a run');
+ if (run.status === 'failed' || run.status === 'cancelled') {
+ const e = run.error || {};
+ throw new IntegrationError(e.code === 'quota.exceeded' || e.code === 'govern.exceeded' ? 429 : 502, e.code || 'ai.run_failed', e.detail || e.message || `${workflow} ${run.status}`);
+ }
+ return { output: run.output, runId: run.id, synthetic: Boolean(run.synthetic), provenance: run.provenance || null };
+ },
+ };
+
// ── VIP: VIP spaces and pages (WS-K task 8) ──
// ── VIP: VIP spaces and pages (WS-K task 8) ──
const vip = require('./vip').createVip({ config, tokenClient, fetchImpl, now, log });
const vip = require('./vip').createVip({ config, tokenClient, fetchImpl, now, log });
return {
return {
- tokenClient, sdk, events, outbox, community, sources, media, vip,
+ tokenClient, sdk, events, outbox, community, sources, media, ai, vip,
eventsConfigured: !!(tokenClient && config.eventsUrl),
eventsConfigured: !!(tokenClient && config.eventsUrl),
};
};
}
}
@@ -50,6 +50,7 @@ function navConfig(o, config) {
@@ -50,6 +50,7 @@ function navConfig(o, config) {
apiBase: NETWORK_URL,
apiBase: NETWORK_URL,
links: [
links: [
{ label: 'Wikis', href: '/', active: o.active === 'home' },
{ label: 'Wikis', href: '/', active: o.active === 'home' },
+ { label: 'Vibe a wiki', href: '/vibe', active: o.active === 'vibe' },
{ label: 'Recent changes', href: '/recent', active: o.active === 'recent' },
{ label: 'Recent changes', href: '/recent', active: o.active === 'recent' },
{ label: 'Search', href: '/search', active: o.active === 'search' },
{ label: 'Search', href: '/search', active: o.active === 'search' },
],
],
@@ -6,7 +6,7 @@
@@ -6,7 +6,7 @@
const ssr = require('openvibe-publishing/ssr');
const ssr = require('openvibe-publishing/ssr');
const showcase = require('openvibe-shared/showcase');
const showcase = require('openvibe-shared/showcase');
const { figureHtml } = require('openvibe-publishing/media');
const { figureHtml } = require('openvibe-publishing/media');
-const { infoboxToText } = require('../wiki/content');
+const { infoboxToText, mediaRefs } = require('../wiki/content');
const { html, raw } = ssr;
const { html, raw } = ssr;
const t = (v) => raw(ssr.timeTag(v));
const t = (v) => raw(ssr.timeTag(v));
@@ -64,7 +64,7 @@ const FAQ = [
@@ -64,7 +64,7 @@ const FAQ = [
{ q: 'How do I follow changes?', a: 'The whole site and each public wiki have Atom and JSON feeds, and signed-in readers can watch a page. Every public page is also served as JSON: add .json to its address.' },
{ q: 'How do I follow changes?', a: 'The whole site and each public wiki have Atom and JSON feeds, and signed-in readers can watch a page. Every public page is also served as JSON: add .json to its address.' },
];
];
-function home({ spaces, recent, shelves = new Map(), stats = null, linked = [], groups = null, actor }) {
+function home({ spaces, recent, shelves = new Map(), stats = null, linked = [], groups = null, generated = [], actor }) {
const official = spaces.filter((s) => s.kind === 'official');
const official = spaces.filter((s) => s.kind === 'official');
const user = spaces.filter((s) => s.kind === 'user');
const user = spaces.filter((s) => s.kind === 'user');
const signedIn = !!(actor && actor.subject);
const signedIn = !!(actor && actor.subject);
@@ -112,8 +112,9 @@ ${body}<p class="wk-more"><a href="/s/${e(s.slug)}">Every page of ${s.name}, as
@@ -112,8 +112,9 @@ ${body}<p class="wk-more"><a href="/s/${e(s.slug)}">Every page of ${s.name}, as
stats.links ? ['Links between pages', stats.links] : null,
stats.links ? ['Links between pages', stats.links] : null,
].filter(Boolean) : [];
].filter(Boolean) : [];
const firstOfficial = shelved[0];
const firstOfficial = shelved[0];
- const aside = stats && stats.pages ? html`<div class="wk-h-panel"><h2 class="wk-h-panel-h">The wiki today</h2><dl class="wk-h-stats">${tiles.map(([k, v]) => html`<div><dt>${k}</dt><dd>${v.toLocaleString('en-US')}</dd></div>`)}</dl>
-<form class="wk-h-search" action="/search" method="get" role="search"><label for="wk-hq">Search every public page</label><span><input id="wk-hq" name="q" type="search" placeholder="Search pages"><button type="submit">Search</button></span></form></div>` : '';
+ const aside = html`<div class="wk-h-panel wk-vibe-panel"><h2 class="wk-h-panel-h">Vibe a wiki</h2>${vibeForm({ actor, compact: true })}
+${stats && stats.pages ? html`<dl class="wk-h-stats">${tiles.slice(0, 4).map(([k, v]) => html`<div><dt>${k}</dt><dd>${v.toLocaleString('en-US')}</dd></div>`)}</dl>` : ''}
+<form class="wk-h-search" action="/search" method="get" role="search"><label for="wk-hq">Search every public page</label><span><input id="wk-hq" name="q" type="search" placeholder="Search pages"><button type="submit">Search</button></span></form></div>`;
const steps = showcase.steps({
const steps = showcase.steps({
title: 'How a page comes together',
title: 'How a page comes together',
items: [
items: [
@@ -133,11 +134,13 @@ ${body}<p class="wk-more"><a href="/s/${e(s.slug)}">Every page of ${s.name}, as
@@ -133,11 +134,13 @@ ${body}<p class="wk-more"><a href="/s/${e(s.slug)}">Every page of ${s.name}, as
].filter(Boolean);
].filter(Boolean);
return html`${raw(showcase.hero({
return html`${raw(showcase.hero({
eyebrow: 'OpenVibe.Wiki',
eyebrow: 'OpenVibe.Wiki',
- title: 'Knowledge,', accent: 'with sources',
- lede: 'Read the OpenVibe wiki or start one of your own. Every page keeps its full revision history, lists the sources behind it and has a discussion thread.',
- actions: [start, firstOfficial ? { label: `Read the ${firstOfficial.name} wiki`, href: `#inside-${firstOfficial.slug}` } : { label: 'Browse the wikis', href: '#spaces' }],
- aside: aside ? { html: String(aside) } : undefined,
+ title: 'Any topic.', accent: 'A whole wiki.',
+ lede: 'Name a show, a game, a city, a band or an idea: OpenVibe researches it across the web and writes you a wiki with pages, pictures and sources. Or write one by hand. Every page keeps its full history and cites what it rests on.',
+ actions: [{ label: 'Vibe a wiki', href: '/vibe', primary: true }, { ...start, primary: false }],
+ aside: { html: String(aside) },
}))}
}))}
+${generated && generated.length ? html`<section class="sc-sec" aria-labelledby="wk-gen-h"><h2 id="wk-gen-h">Wikis people vibed</h2><p class="sc-lede">Each one researched and written by AI from the sources it lists, with its pictures credited. Open one, or make your own.</p>
+${raw(vibeCards(generated))}<p class="wk-more"><a href="/vibe">Vibe a wiki</a></p></section>` : ''}
<div id="spaces"></div>
<div id="spaces"></div>
${inside}
${inside}
${linked.length ? html`<section class="sc-sec" aria-labelledby="wk-linked-h"><h2 id="wk-linked-h">Most linked pages</h2><p class="sc-lede">The pages other pages point to most often: a good place to start reading.</p>
${linked.length ? html`<section class="sc-sec" aria-labelledby="wk-linked-h"><h2 id="wk-linked-h">Most linked pages</h2><p class="sc-lede">The pages other pages point to most often: a good place to start reading.</p>
@@ -163,6 +166,86 @@ ${raw(steps)}
@@ -163,6 +166,86 @@ ${raw(steps)}
${raw(showcase.cta({ title: 'Start a wiki of your own', text: 'A team handbook, a game guide, a project\'s docs: write it with history and sources from the first page.', actions: [start] }))}`;
${raw(showcase.cta({ title: 'Start a wiki of your own', text: 'A team handbook, a game guide, a project\'s docs: write it with history and sources from the first page.', actions: [start] }))}`;
}
}
+// ── "Vibe a wiki" (server/wiki/vibe.js) ─────────────────────
+const VIBE_SIZES = [['quick', 'Quick', 'about 6 pages'], ['standard', 'Standard', 'about 12 pages'], ['big', 'Big', 'about 24 pages']];
+const VIBE_IDEAS = ['Kim Possible', 'The Legend of Zelda', 'Studio Ghibli', 'Ancient Rome', 'Octopuses', 'Tokyo', 'Jazz', 'Minecraft'];
+const VIBE_STATUS = { queued: 'Waiting to start', researching: 'Researching', planning: 'Planning the pages', writing: 'Writing pages', done: 'Done', failed: 'Stopped', cancelled: 'Cancelled' };
+
+/** The form: a topic, a size and who can read it. The home's small one is a search-style box (wiki.css .wk-h-search). */
+function vibeForm({ actor, topic = '', compact = false, size = 'standard', visibility = 'public' } = {}) {
+ const signedIn = !!(actor && actor.subject);
+ const off = signedIn ? '' : ' disabled';
+ const id = compact ? 'wk-vibe-topic-c' : 'wk-vibe-topic';
+ const input = html`<input id="${id}" name="topic" type="text" required minlength="2" maxlength="120" placeholder="Kim Possible, ancient Rome, octopuses…" value="${topic}" autocomplete="off" enterkeyhint="go"${raw(off)}>`;
+ const button = html`<button type="submit">${signedIn ? 'Vibe it' : 'Sign in to vibe'}</button>`;
+ const ideas = html`<p class="wk-vibe-ideas">${compact ? 'Try ' : 'Ideas: '}${VIBE_IDEAS.slice(0, compact ? 4 : 8).map((t, i) => html`${i ? ', ' : ''}<a href="/vibe?topic=${e(t)}">${t}</a>`)}</p>`;
+ const attrs = signedIn ? { action: '/vibe', method: 'post' } : { action: '/auth/login', method: 'get' };
+ const next = signedIn ? '' : html`<input type="hidden" name="next" value="/vibe">`;
+ if (compact) return html`<form class="wk-h-search wk-vibe-form" action="${attrs.action}" method="${attrs.method}">${next}<label for="${id}">Name any topic: OpenVibe researches it and writes the wiki</label><span>${input}${button}</span>${ideas}</form>`;
+ const radio = (name, v, checked, label, hint) => html`<label><input type="radio" name="${name}" value="${v}"${checked ? ' checked' : ''}${raw(off)}><span><b>${label}</b><small>${hint}</small></span></label>`;
+ return html`<form class="wk-vibe-form" action="${attrs.action}" method="${attrs.method}">${next}
+<label for="${id}" class="wk-vibe-label">Any topic</label><span class="wk-vibe-row">${input}${button}</span>
+<fieldset class="wk-vibe-sizes"><legend>How big</legend>${VIBE_SIZES.map(([v, label, hint]) => radio('size', v, v === size, label, hint))}</fieldset>
+<fieldset class="wk-vibe-sizes"><legend>Who can read it</legend>${radio('visibility', 'public', visibility !== 'private', 'Everyone', 'a public wiki')}${radio('visibility', 'private', visibility === 'private', 'Just me', 'add people later')}</fieldset>
+${ideas}</form>`;
+}
+
+function vibeCards(list) {
+ return html`<div class="sc-grid">${list.map((v) => html`<a class="sc-card wk-h-card" href="${v.main_page || v.space.url}">${ringTile('wiki')}<span><b>${v.space.name}</b><small>${plural(v.done, 'page')}${v.research && v.research.sources ? ` · ${plural(v.research.sources, 'source')} read` : ''}</small></span></a>`)}</div>`;
+}
+
+function vibePage({ actor, topic = '', mine = [], recent = [], error = null, perPersonDay = 3 }) {
+ return html`${crumbs([{ name: 'Wiki', url: '/' }, { name: 'Vibe a wiki' }])}
+<section class="wk-vibe"><h1>Vibe a wiki</h1>
+<p class="wk-vibe-lede">Name anything. OpenVibe reads about it across Wikipedia, its fan wiki, Wikidata, the web and open picture libraries, plans the pages a reader would want, and writes each one with its pictures and the sources it rests on. It takes a few minutes; you can watch it happen.</p>
+${error ? notice('error', error) : ''}
+${vibeForm({ actor, topic })}
+<ol class="wk-vibe-how"><li><b>Research</b><span>Pages and pictures about your topic, each kept with its link, licence and when it was read.</span></li><li><b>Plan</b><span>A main page, then the people, places, works and ideas the sources cover.</span></li><li><b>Write</b><span>Every page cites its sources and credits its pictures. Then the wiki is yours to edit.</span></li></ol>
+<p class="wk-muted">The text is written by AI and says so on every page; search engines are asked to wait until a person has checked each page. You can vibe ${String(perPersonDay)} wikis a day.</p>
+</section>
+${mine.length ? html`<section class="sc-sec" aria-labelledby="wk-mine-h"><h2 id="wk-mine-h">Your wikis</h2><ul class="wk-vibe-mine">${mine.map((v) => html`<li><a href="/vibe/${v.id}">${v.space.name}</a> <span class="wk-tag">${VIBE_STATUS[v.status] || v.status}</span> <span class="wk-muted">${v.done} of ${v.total || '…'} pages</span></li>`)}</ul></section>` : ''}
+${recent.length ? html`<section class="sc-sec" aria-labelledby="wk-vrecent-h"><h2 id="wk-vrecent-h">Wikis people vibed</h2>${raw(vibeCards(recent))}</section>` : ''}`;
+}
+
+/** The progress page: the steps, what was read, every planned page as it is written. Live with JavaScript, refreshing without. */
+function vibeProgress({ vibe: v }) {
+ const steps = [['researching', 'Research'], ['planning', 'Plan'], ['writing', 'Write'], ['done', 'Done']];
+ const at = Math.max(0, steps.findIndex(([k]) => k === v.status));
+ const active = !['done', 'failed', 'cancelled'].includes(v.status);
+ const state = (i) => (v.status === 'done' || i < at ? 'done' : i === at && active ? 'now' : 'next');
+ const pageRow = (p) => html`<li class="wk-vp-${p.status}" data-title="${p.title}">${p.url ? html`<a href="${p.url}">${p.title}</a>` : html`<span>${p.title}</span>`}<small>${p.status === 'done' ? `${p.sources} sources${p.images ? ` · ${p.images} picture${p.images === 1 ? '' : 's'}` : ''}` : p.status === 'writing' ? 'writing…' : p.status === 'failed' ? 'not enough found' : 'waiting'}</small></li>`;
+ return html`${crumbs([{ name: 'Wiki', url: '/' }, { name: 'Vibe a wiki', url: '/vibe' }, { name: v.topic }])}
+<section class="wk-vibe wk-vp" id="wk-vp" data-vibe="${v.id}" data-active="${active ? '1' : '0'}">
+<h1>${v.status === 'done' ? html`Your wiki about <span>${v.topic}</span> is ready` : html`Vibing a wiki about <span>${v.topic}</span>`}</h1>
+<ol class="wk-vp-steps">${steps.map(([, label], i) => html`<li class="wk-vp-step-${state(i)}">${label}</li>`)}</ol>
+<p class="wk-vp-status" role="status" aria-live="polite">${VIBE_STATUS[v.status] || v.status}${v.total ? ` · ${v.done} of ${v.total} pages` : ''}${v.waiting_until ? ' · waiting for the AI service, it will carry on by itself' : ''}</p>
+${v.error && v.status === 'failed' ? notice('error', v.error) : ''}
+${v.research ? html`<p class="wk-vp-read">Read ${plural(v.research.sources, 'source')} and found ${plural(v.research.images, 'picture')}${v.research.from && v.research.from.length ? html` from ${v.research.from.join(', ')}` : ''}${v.research.topic_wiki ? html`, including <a href="${v.research.topic_wiki.url}" rel="nofollow noopener">${v.research.topic_wiki.name}</a>` : ''}.</p>` : ''}
+${v.status === 'done' && v.main_page ? html`<p><a class="wk-button wk-vp-open" href="${v.main_page}">Open the wiki</a> <a href="${v.space.url}">See every page</a></p>` : ''}
+${v.pages.length ? html`<ol class="wk-vp-pages">${v.pages.map(pageRow)}</ol>` : html`<div class="wk-vp-wait"><span class="wk-vp-dot"></span><span class="wk-vp-dot"></span><span class="wk-vp-dot"></span></div>`}
+<noscript>${active ? raw('<meta http-equiv="refresh" content="8">') : ''}<p class="wk-muted">This page refreshes itself every few seconds while the wiki is written.</p></noscript>
+</section>
+<script>
+(function () {
+ var root = document.getElementById('wk-vp');
+ if (!root || root.getAttribute('data-active') !== '1') return;
+ var id = root.getAttribute('data-vibe'), last = '';
+ function poll() {
+ if (!document.body.contains(root)) return;
+ window.fetch('/vibe/' + id + '.json', { credentials: 'same-origin', cache: 'no-store' }).then(function (r) { return r.ok ? r.json() : null; }).then(function (v) {
+ if (!v) return setTimeout(poll, 5000);
+ var sig = v.status + ':' + v.done + ':' + v.pages.map(function (p) { return p.status; }).join(',');
+ if (sig !== last && last) { location.replace(location.pathname); return; }
+ last = sig;
+ if (['done', 'failed', 'cancelled'].indexOf(v.status) === -1) setTimeout(poll, 2500);
+ else location.replace(location.pathname);
+ }).catch(function () { setTimeout(poll, 5000); });
+ }
+ setTimeout(poll, 2500);
+})();
+</script>`;
+}
+
/** The home's structured data: the site (with its search), the official pages it lists, and the questions it answers. */
/** The home's structured data: the site (with its search), the official pages it lists, and the questions it answers. */
function homeJsonLd({ shelves = new Map(), spaces = [], baseUrl }) {
function homeJsonLd({ shelves = new Map(), spaces = [], baseUrl }) {
const SITE = String(baseUrl).replace(/\/+$/, '');
const SITE = String(baseUrl).replace(/\/+$/, '');
@@ -196,8 +279,12 @@ ${space.visibility === 'public' ? html`<p class="wk-muted">Follow this wiki: <a
@@ -196,8 +279,12 @@ ${space.visibility === 'public' ? html`<p class="wk-muted">Follow this wiki: <a
</section>`;
</section>`;
}
}
-function infoboxHtml(entries, resolve, rel = 'nofollow ugc noopener') {
+function infoboxHtml(entries, resolve, rel = 'nofollow ugc noopener', { media = null } = {}) {
if (!entries.length) return '';
if (!entries.length) return '';
+ // A picture row (type media) whose object the page has attached shows the picture across the box, with its caption.
+ const picture = (x) => (x.type === 'media' && typeof media === 'function' ? media(x.value) : null);
+ const pics = entries.map(picture);
+ const rows = entries.map((x, i) => (pics[i] ? html`<tr class="wk-ib-pic" data-key="${x.key}"><td colspan="2">${raw(pics[i])}</td></tr>` : null));
const cell = (x) => {
const cell = (x) => {
switch (x.type) {
switch (x.type) {
case 'url': return html`<a href="${x.value}" rel="${rel}">${x.value.replace(/^https?:\/\//, '')}</a>`;
case 'url': return html`<a href="${x.value}" rel="${rel}">${x.value.replace(/^https?:\/\//, '')}</a>`;
@@ -209,7 +296,7 @@ function infoboxHtml(entries, resolve, rel = 'nofollow ugc noopener') {
@@ -209,7 +296,7 @@ function infoboxHtml(entries, resolve, rel = 'nofollow ugc noopener') {
default: return x.value;
default: return x.value;
}
}
};
};
- return html`<aside class="wk-infobox" aria-label="Infobox"><table><tbody>${entries.map((x) => html`<tr data-key="${x.key}" data-type="${x.type}"><th scope="row">${x.label}</th><td>${cell(x)}</td></tr>`)}</tbody></table></aside>`;
+ return html`<aside class="wk-infobox" aria-label="Infobox"><table><tbody>${entries.map((x, i) => rows[i] || html`<tr data-key="${x.key}" data-type="${x.type}"><th scope="row">${x.label}</th><td>${cell(x)}</td></tr>`)}</tbody></table></aside>`;
}
}
function citationsHtml(cites, inspectHref, rel = 'nofollow ugc noopener') {
function citationsHtml(cites, inspectHref, rel = 'nofollow ugc noopener') {
@@ -250,7 +337,8 @@ const UNAVAILABLE = {
@@ -250,7 +337,8 @@ const UNAVAILABLE = {
not_found: 'This media is no longer available.',
not_found: 'This media is no longer available.',
};
};
-function mediaHtml(attachments, urlFor) {
+function mediaHtml(attachments, urlFor, shown = new Set()) {
+ attachments = attachments.filter((a) => !shown.has(a.mediaId));
if (!attachments.length) return '';
if (!attachments.length) return '';
return html`<section class="wk-media"><h2>Media</h2>${attachments.map((a) => raw(figureHtml(a, { urlFor, unavailableText: UNAVAILABLE[a.brokenReason] || UNAVAILABLE.not_found })))}</section>`;
return html`<section class="wk-media"><h2>Media</h2>${attachments.map((a) => raw(figureHtml(a, { urlFor, unavailableText: UNAVAILABLE[a.brokenReason] || UNAVAILABLE.not_found })))}</section>`;
}
}
@@ -288,9 +376,9 @@ function articlePage(v, { html: contentHtml, discussion, actor, mediaUrl, resolv
@@ -288,9 +376,9 @@ function articlePage(v, { html: contentHtml, discussion, actor, mediaUrl, resolv
<article class="wk-article" data-page-id="${page.id}" data-revision="${String(rev.number)}">
<article class="wk-article" data-page-id="${page.id}" data-revision="${String(rev.number)}">
<header><h1>${rev.fields.title || page.title}</h1><nav class="wk-actions" aria-label="Page actions">${actions.map((a, i) => html`${i ? ' · ' : ''}${a}`)}</nav></header>
<header><h1>${rev.fields.title || page.title}</h1><nav class="wk-actions" aria-label="Page actions">${actions.map((a, i) => html`${i ? ' · ' : ''}${a}`)}</nav></header>
${banners}
${banners}
-${raw(infoboxHtml(v.infobox, resolve, outRel(space)))}
+${raw(infoboxHtml(v.infobox, resolve, outRel(space), { media: (id) => { const a = v.attachments.find((x) => x.mediaId === id); return a && mediaUrl ? figureHtml(a, { urlFor: mediaUrl, unavailableText: UNAVAILABLE[a.brokenReason] || UNAVAILABLE.not_found }) : null; } }))}
<div class="wk-content">${raw(contentHtml)}</div>
<div class="wk-content">${raw(contentHtml)}</div>
-${raw(mediaHtml(v.attachments, mediaUrl))}
+${raw(mediaHtml(v.attachments, mediaUrl, new Set([...mediaRefs(v.revision.content), ...v.infobox.filter((x) => x.type === 'media').map((x) => x.value)])))}
${raw(citationsHtml(v.citations, `${base}/sources${v.isPublishedRevision ? '' : `?rev=${rev.number}`}`, outRel(space)))}
${raw(citationsHtml(v.citations, `${base}/sources${v.isPublishedRevision ? '' : `?rev=${rev.number}`}`, outRel(space)))}
${v.children.length ? html`<section><h2>Subpages</h2><ul>${v.children.map((c) => html`<li><a href="${wpath(space, c)}">${c.title}</a></li>`)}</ul></section>` : ''}
${v.children.length ? html`<section><h2>Subpages</h2><ul>${v.children.map((c) => html`<li><a href="${wpath(space, c)}">${c.title}</a></li>`)}</ul></section>` : ''}
${v.backlinks.length ? html`<section><h2>What links here</h2><ul>${v.backlinks.map((b) => html`<li><a href="${wpath(b.space, b.page)}">${b.page.title}</a>${b.space.id !== space.id ? html` <span class="wk-muted">(${b.space.name})</span>` : ''}</li>`)}</ul></section>` : ''}
${v.backlinks.length ? html`<section><h2>What links here</h2><ul>${v.backlinks.map((b) => html`<li><a href="${wpath(b.space, b.page)}">${b.page.title}</a>${b.space.id !== space.id ? html` <span class="wk-muted">(${b.space.name})</span>` : ''}</li>`)}</ul></section>` : ''}
@@ -519,6 +607,7 @@ function signInPage({ next, message }) {
@@ -519,6 +607,7 @@ function signInPage({ next, message }) {
}
}
module.exports = {
module.exports = {
+ vibePage, vibeProgress, vibeForm,
vipGateBody,
vipGateBody,
home, homeJsonLd, spacePage, articlePage, historyPage, sourcesPage, diffPage, editPage, formValuesFromRevision, newSpacePage,
home, homeJsonLd, spacePage, articlePage, historyPage, sourcesPage, diffPage, editPage, formValuesFromRevision, newSpacePage,
spaceSettingsPage, importPage, pageSettingsPage, revertPage, confirmPublishPage, proposalsPage, searchPage, recentPage,
spaceSettingsPage, importPage, pageSettingsPage, revertPage, confirmPublishPage, proposalsPage, searchPage, recentPage,
@@ -29,6 +29,7 @@ const TABLES = [
@@ -29,6 +29,7 @@ const TABLES = [
{ table: 'wiki_attachment_origins', subject: 'media_owner', file: null, erase: anonymize },
{ table: 'wiki_attachment_origins', subject: 'media_owner', file: null, erase: anonymize },
{ table: 'wiki_page_revision_purges', subject: 'purged_by', file: null, erase: anonymize },
{ table: 'wiki_page_revision_purges', subject: 'purged_by', file: null, erase: anonymize },
{ table: 'wiki_citation_purges', subject: 'purged_by', file: null, erase: anonymize },
{ table: 'wiki_citation_purges', subject: 'purged_by', file: null, erase: anonymize },
+ { table: 'wiki_vibes', subject: 'requested_by', file: 'generated-wikis.json', columns: ['id', 'space_id', 'topic', 'size', 'status', 'created_at', 'finished_at'], order: 'created_at' },
{ table: 'wiki_page_reviews', subject: 'reviewer', file: 'reviews.json', columns: ['entity_id', 'revision', 'decision', 'note', 'reviewed_at'], order: 'reviewed_at', erase: { keep: 'a person\'s approval is what lets reviewed text stay published' } },
{ table: 'wiki_page_reviews', subject: 'reviewer', file: 'reviews.json', columns: ['entity_id', 'revision', 'decision', 'note', 'reviewed_at'], order: 'reviewed_at', erase: { keep: 'a person\'s approval is what lets reviewed text stay published' } },
];
];
@@ -7,6 +7,8 @@
@@ -7,6 +7,8 @@
* [[Title|label]] with its own label
* [[Title|label]] with its own label
* [[space-slug:Title]] a page in another space (the prefix must look like a space slug:
* [[space-slug:Title]] a page in another space (the prefix must look like a space slug:
* lowercase letters, digits and dashes; anything else is part of the title)
* lowercase letters, digits and dashes; anything else is part of the title)
+ * [[media:med_…|caption]] a picture attached to the page, shown where it stands (on a line of its own it is a
+ * figure); one the page does not have, or that Media no longer shares, shows nothing
* Markdown itself goes through openvibe-publishing/ssr (escaped first, a fixed tag set, no raw HTML).
* Markdown itself goes through openvibe-publishing/ssr (escaped first, a fixed tag set, no raw HTML).
*/
*/
const ssr = require('openvibe-publishing/ssr');
const ssr = require('openvibe-publishing/ssr');
@@ -17,6 +19,7 @@ const SPACE_PREFIX_RE = /^([a-z0-9][a-z0-9-]{0,62}):(.+)$/;
@@ -17,6 +19,7 @@ const SPACE_PREFIX_RE = /^([a-z0-9][a-z0-9-]{0,62}):(.+)$/;
const SPACE_SLUG_RE = /^[a-z0-9][a-z0-9-]{1,62}$/;
const SPACE_SLUG_RE = /^[a-z0-9][a-z0-9-]{1,62}$/;
const PAGE_SLUG_RE = /^[\p{Ll}\p{Lo}\p{N}][\p{Ll}\p{Lo}\p{N}-]{0,79}$/u;
const PAGE_SLUG_RE = /^[\p{Ll}\p{Lo}\p{N}][\p{Ll}\p{Lo}\p{N}-]{0,79}$/u;
const MAX_LINKS = 500;
const MAX_LINKS = 500;
+const MEDIA_TOKEN_RE = /^\s*media:(med_[0-9A-HJKMNP-TV-Z]{26})\s*$/;
class ContentError extends Error {
class ContentError extends Error {
constructor(code, message) { super(message); this.status = 422; this.code = code; }
constructor(code, message) { super(message); this.status = 422; this.code = code; }
@@ -49,11 +52,12 @@ function mapOutsideCode(source, fn) {
@@ -49,11 +52,12 @@ function mapOutsideCode(source, fn) {
}).join('\n');
}).join('\n');
}
}
-/** Every distinct [[link]] in the content: [{ space, slug, title, label }]. */
+/** Every distinct [[link]] in the content: [{ space, slug, title, label }]. Pictures ([[media:…]]) are not links. */
function extractLinks(source, currentSpace) {
function extractLinks(source, currentSpace) {
const seen = new Map();
const seen = new Map();
mapOutsideCode(source, (text) => {
mapOutsideCode(source, (text) => {
text.replace(LINK_RE, (_m, target, label) => {
text.replace(LINK_RE, (_m, target, label) => {
+ if (MEDIA_TOKEN_RE.test(target)) return '';
const t = parseTarget(target, currentSpace);
const t = parseTarget(target, currentSpace);
if (t && seen.size < MAX_LINKS) {
if (t && seen.size < MAX_LINKS) {
const key = `${t.space}\u0000${t.slug}`;
const key = `${t.space}\u0000${t.slug}`;
@@ -66,13 +70,32 @@ function extractLinks(source, currentSpace) {
@@ -66,13 +70,32 @@ function extractLinks(source, currentSpace) {
return [...seen.values()];
return [...seen.values()];
}
}
+/** Media object ids placed in the content with [[media:…]], in order of first appearance. */
+function mediaRefs(source) {
+ const ids = [];
+ mapOutsideCode(source, (text) => {
+ text.replace(LINK_RE, (_m, target) => { const m = MEDIA_TOKEN_RE.exec(target); if (m && !ids.includes(m[1])) ids.push(m[1]); return ''; });
+ return text;
+ });
+ return ids;
+}
+
/**
/**
* Markdown + [[links]] → HTML. resolve({ space, slug }) → { href, exists } (exists false = red link;
* Markdown + [[links]] → HTML. resolve({ space, slug }) → { href, exists } (exists false = red link;
- * href null = no link at all, e.g. a reader who cannot create the page).
+ * href null = no link at all, e.g. a reader who cannot create the page). media(mediaId, caption) → the figure's HTML,
+ * or null when the page has no such picture to show (then nothing is shown).
*/
*/
-function renderContent(source, { currentSpace, resolve, rel = 'noopener' } = {}) {
+function renderContent(source, { currentSpace, resolve, rel = 'noopener', media = null } = {}) {
const held = [];
const held = [];
+ const figures = new Set();
const withTokens = mapOutsideCode(source, (text) => text.replace(LINK_RE, (m, target, label) => {
const withTokens = mapOutsideCode(source, (text) => text.replace(LINK_RE, (m, target, label) => {
+ const pic = MEDIA_TOKEN_RE.exec(target);
+ if (pic) {
+ const figure = typeof media === 'function' ? media(pic[1], label ? label.trim() : null) : null;
+ held.push(figure || '');
+ figures.add(held.length - 1);
+ return `\u0001WL${held.length - 1}\u0001`;
+ }
const t = parseTarget(target, currentSpace);
const t = parseTarget(target, currentSpace);
if (!t) return m;
if (!t) return m;
const r = resolve(t) || { href: null, exists: false };
const r = resolve(t) || { href: null, exists: false };
@@ -84,7 +107,9 @@ function renderContent(source, { currentSpace, resolve, rel = 'noopener' } = {})
@@ -84,7 +107,9 @@ function renderContent(source, { currentSpace, resolve, rel = 'noopener' } = {})
held.push(htmlLink);
held.push(htmlLink);
return `\u0001WL${held.length - 1}\u0001`;
return `\u0001WL${held.length - 1}\u0001`;
}));
}));
- const html = ssr.renderMarkdown(withTokens, { rel, headingShift: 0 });
+ const html = ssr.renderMarkdown(withTokens, { rel, headingShift: 0 })
+ // A picture on a line of its own is a figure, not a paragraph.
+ .replace(/<p>\s*\u0001WL(\d+)\u0001\s*<\/p>/g, (m, n) => (figures.has(Number(n)) ? held[Number(n)] : m));
return html.replace(/\u0001WL(\d+)\u0001/g, (_m, n) => held[Number(n)] || '');
return html.replace(/\u0001WL(\d+)\u0001/g, (_m, n) => held[Number(n)] || '');
}
}
@@ -192,7 +217,7 @@ function infoboxToText(entries) {
@@ -192,7 +217,7 @@ function infoboxToText(entries) {
}
}
module.exports = {
module.exports = {
- ContentError, pageSlug, parseTarget, extractLinks, renderContent, mapOutsideCode,
+ ContentError, pageSlug, parseTarget, extractLinks, renderContent, mapOutsideCode, mediaRefs,
normalizeInfobox, parseInfoboxText, infoboxToText, keyFromLabel,
normalizeInfobox, parseInfoboxText, infoboxToText, keyFromLabel,
TYPES, SPACE_SLUG_RE, PAGE_SLUG_RE, MEDIA_RE,
TYPES, SPACE_SLUG_RE, PAGE_SLUG_RE, MEDIA_RE,
};
};
@@ -32,7 +32,7 @@ function firstSentence(text) {
@@ -32,7 +32,7 @@ function firstSentence(text) {
if (m) return m[1];
if (m) return m[1];
return t.length <= 160 ? t : `${t.slice(0, 160).replace(/\s+\S*$/, '')}…`;
return t.length <= 160 ? t : `${t.slice(0, 160).replace(/\s+\S*$/, '')}…`;
}
}
-const { isMediaId } = require('openvibe-publishing/media');
+const { isMediaId, figureHtml } = require('openvibe-publishing/media');
const { ulid } = require('openvibe-contracts').ids;
const { ulid } = require('openvibe-contracts').ids;
const content = require('./content');
const content = require('./content');
const importer = require('./import');
const importer = require('./import');
@@ -1108,11 +1108,21 @@ function createWikiService({ db, stores, outbox, config, community = null, vip =
@@ -1108,11 +1108,21 @@ function createWikiService({ db, stores, outbox, config, community = null, vip =
* are edited by staff, who vouch for their links. Every [[link]] is resolved first (the
* are edited by staff, who vouch for their links. Every [[link]] is resolved first (the
* renderer itself is synchronous).
* renderer itself is synchronous).
*/
*/
- async renderRevision(space, rev, actor) {
+ /**
+ * A revision's HTML. With attachments and mediaUrl, [[media:…]] shows the page's own pictures where they stand
+ * (an attachment Media no longer shares shows its placeholder; an id the page does not have shows nothing).
+ */
+ async renderRevision(space, rev, actor, { attachments = null, mediaUrl = null } = {}) {
const targets = content.extractLinks(rev.content, space.slug);
const targets = content.extractLinks(rev.content, space.slug);
const resolved = new Map();
const resolved = new Map();
for (const t of targets) resolved.set(`${t.space}\u0000${t.slug}`, await svc.resolveLink(t, actor));
for (const t of targets) resolved.set(`${t.space}\u0000${t.slug}`, await svc.resolveLink(t, actor));
- return content.renderContent(rev.content, { currentSpace: space.slug, resolve: (t) => resolved.get(`${t.space}\u0000${t.slug}`) || { href: null, exists: false }, rel: svc.linkRel(space) });
+ const byMedia = new Map((attachments || []).map((a) => [a.mediaId, a]));
+ const media = mediaUrl ? (id, caption) => {
+ const a = byMedia.get(id);
+ if (!a) return null;
+ return figureHtml({ ...a, caption: caption || a.caption }, { urlFor: mediaUrl, unavailableText: 'This picture is no longer available.' });
+ } : null;
+ return content.renderContent(rev.content, { currentSpace: space.slug, resolve: (t) => resolved.get(`${t.space}\u0000${t.slug}`) || { href: null, exists: false }, rel: svc.linkRel(space), media });
},
},
linkRel(space) { return space && space.kind === 'official' ? 'noopener' : 'nofollow ugc noopener'; },
linkRel(space) { return space && space.kind === 'official' ? 'noopener' : 'nofollow ugc noopener'; },
new file mode 100644
new file mode 100644
@@ -0,0 +1,452 @@
@@ -0,0 +1,452 @@
+'use strict';
+/**
+ * "Vibe a wiki": a person names any topic and gets a whole wiki about it, researched from the web, written page by page
+ * with sources and pictures, in a new wiki they own.
+ *
+ * 1. research OpenVibe.AI research.gather (MegaSource: Wikipedia, the topic's Fandom wiki, Wikidata, DeepSeek web
+ * search, Commons, Openverse) for the topic as a whole
+ * 2. plan wiki.generate_space: the wiki's name, summary and pages (main page first), from that research
+ * 3. write for each page, its own research (the page's subject inside the topic) and wiki.generate_page; the
+ * pictures it picks are fetched safely (integrations/image-fetch.js), stored in OpenVibe.Media for the
+ * person (public, with where they came from), placed in the text and attached; the page is imported
+ * like any AI-assisted page (published with the disclosure, noindex until a person reviews it)
+ *
+ * The wiki's pages say the text was written with AI and cite every source they rest on, pictures included (licence and
+ * credit). Nothing is invented: a page the sources cannot support is not written (it stays a red link to create).
+ *
+ * The job is one wiki_vibes row a worker takes under a lease. Progress is saved after every step, so a restart or a
+ * refusal resumes where it stopped (pages already made are skipped); AI's quota or queue being full (429) waits for its
+ * Retry-After. Limits: WIKI_VIBE_PER_PERSON_DAY and WIKI_VIBE_PER_DAY.
+ */
+const crypto = require('crypto');
+const { ulid } = require('openvibe-contracts').ids;
+const content = require('./content');
+const importer = require('./import');
+const { fetchImage } = require('../integrations/image-fetch');
+
+const SIZES = ['quick', 'standard', 'big'];
+const ACTIVE = ['queued', 'researching', 'planning', 'writing'];
+const LEASE_MS = 10 * 60 * 1000;
+const DAY_MS = 24 * 60 * 60 * 1000;
+const MAX_IMAGES_PER_PAGE = 3;
+// How the main page groups the rest, by what the plan says each page is about.
+const GROUP = { character: 'Characters', person: 'People', episode: 'Episodes', season: 'Seasons', work: 'Works', place: 'Places', group: 'Groups and organizations',
+ object: 'Things', concept: 'Ideas and themes', event: 'Events', production: 'Behind the scenes', reception: 'Reception', list: 'Lists', other: 'More' };
+// What research.gather's providers are called where people read them.
+const SOURCE_NAMES = { wikipedia: 'Wikipedia', fandom: 'Fandom', wikidata: 'Wikidata', deepseek: 'the web', web: 'the web', commons: 'Wikimedia Commons', openverse: 'Openverse', wikipedia_images: 'Wikipedia', wikidata_image: 'Wikidata' };
+const sourceNames = (list) => [...new Set((list || []).map((k) => SOURCE_NAMES[k] || k))];
+const GROUP_ORDER = ['character', 'person', 'work', 'season', 'episode', 'place', 'group', 'object', 'concept', 'event', 'production', 'reception', 'list', 'other'];
+
+class VibeError extends Error {
+ constructor(status, code, message, extra) { super(message); this.status = status; this.code = code; if (extra) Object.assign(this, extra); }
+}
+
+const oneLine = (s, n) => String(s == null ? '' : s).replace(/\s+/g, ' ').trim().slice(0, n);
+const clip = (s, n) => { const t = String(s || '').trim(); return t.length <= n ? t : `${t.slice(0, n).replace(/\s+\S*$/, '')}…`; };
+
+/** An ASCII space slug for a topic ("Pokémon Red" → "pokemon-red"), or null when nothing Latin is left. */
+function spaceSlugFor(topic) {
+ const s = String(topic || '').normalize('NFKD').replace(/[̀-ͯ]/g, '').toLowerCase()
+ .replace(/['’]/g, '').replace(/[^a-z0-9]+/g, '-').replace(/^-+|-+$/g, '').slice(0, 50).replace(/-+$/, '');
+ return s.length >= 2 ? s : null;
+}
+
+/** The plan as the job keeps it: unique titles (by slug), the main page first, at most `max` pages. */
+function normalizePlan(output, topic, max) {
+ const pages = [];
+ const seen = new Set();
+ const add = (p) => {
+ const title = oneLine(p && p.title, 120).replace(/[[\]|]/g, '').trim();
+ if (!title) return;
+ let slug;
+ try { slug = content.pageSlug(title); } catch { return; }
+ if (seen.has(slug)) return;
+ seen.add(slug);
+ pages.push({
+ title, slug,
+ kind: GROUP[p.kind] || p.kind === 'overview' ? p.kind : 'other',
+ parent: oneLine(p.parent, 120),
+ outline: (Array.isArray(p.outline) ? p.outline : []).map((h) => oneLine(h, 80)).filter(Boolean).slice(0, 8),
+ query: oneLine(p.query, 120),
+ status: 'waiting',
+ });
+ };
+ const list = Array.isArray(output && output.pages) ? output.pages : [];
+ const main = list.find((p) => p && p.kind === 'overview') || list[0] || { title: topic };
+ add({ ...main, kind: 'overview', parent: '' });
+ for (const p of list) if (p !== main && pages.length < max) add(p);
+ return {
+ title: oneLine(output && output.title, 120) || oneLine(topic, 120),
+ summary: clip(output && output.summary, 1000),
+ pages,
+ };
+}
+
+/** The sources a page cited (indices in any `citations` array of the writer's output); all of them when it cited none. */
+function citedIndices(output, count) {
+ const set = new Set();
+ const walk = (v) => {
+ if (Array.isArray(v)) return v.forEach(walk);
+ if (!v || typeof v !== 'object') return;
+ for (const [k, x] of Object.entries(v)) {
+ if (k === 'citations' && Array.isArray(x)) x.forEach((i) => { if (Number.isInteger(i) && i >= 0 && i < count) set.add(i); });
+ else walk(x);
+ }
+ };
+ walk(output);
+ return set.size ? [...set].sort((a, b) => a - b) : [...Array(count).keys()];
+}
+
+/** A picture's credit line: who made it or where it is from, and its licence. */
+function creditOf(img) {
+ let site = null;
+ try { site = img.source_url ? new URL(img.source_url).hostname.replace(/^www\./, '') : null; } catch { /* no site */ }
+ const who = oneLine(img.author, 80) || site || oneLine(img.provider, 40);
+ return [who ? `Picture: ${who}` : 'Picture', img.license ? oneLine(img.license, 80) : null].filter(Boolean).join(', ');
+}
+
+/** The Markdown of one page from the writer's output: lead, sections with their pictures, and the main page's contents. */
+function composeBody(out, { placed = new Map(), contents = null }) {
+ const parts = [];
+ const lead = String(out.lead || '').trim();
+ if (lead) parts.push(lead);
+ const used = new Set();
+ for (const sec of Array.isArray(out.sections) ? out.sections : []) {
+ const heading = oneLine(sec.heading, 120).replace(/^#+\s*/, '');
+ const body = String(sec.body || '').trim().replace(/^#{1,2}\s/gm, '### ');
+ if (!heading || !body) continue;
+ const key = heading.toLowerCase();
+ const pic = placed.get(key);
+ parts.push(`## ${heading}${pic && !used.has(key) ? `\n\n[[media:${pic.mediaId}|${pic.caption.replace(/[[\]|]/g, '')}]]` : ''}\n\n${body}`);
+ if (pic) used.add(key);
+ }
+ // Pictures for headings the writer did not use go under the lead.
+ const loose = [...placed.entries()].filter(([k]) => k !== 'lead' && !used.has(k)).map(([, p]) => `[[media:${p.mediaId}|${p.caption.replace(/[[\]|]/g, '')}]]`);
+ if (loose.length) parts.splice(lead ? 1 : 0, 0, loose.join('\n\n'));
+ if (contents) parts.push(contents);
+ return parts.join('\n\n').replace(/\n{3,}/g, '\n\n').trim();
+}
+
+/** The main page's "In this wiki": every other planned page, grouped by kind (links turn blue as pages are written). */
+function contentsOf(plan) {
+ const rest = plan.pages.slice(1);
+ if (!rest.length) return null;
+ const groups = new Map();
+ for (const p of rest) { const k = GROUP[p.kind] ? p.kind : 'other'; if (!groups.has(k)) groups.set(k, []); groups.get(k).push(p); }
+ const blocks = GROUP_ORDER.filter((k) => groups.has(k)).map((k) => `### ${GROUP[k]}\n\n${groups.get(k).map((p) => `- [[${p.title}]]`).join('\n')}`);
+ return `## In this wiki\n\n${blocks.join('\n\n')}`;
+}
+
+function createVibes({ db, svc, platform, config, now = () => Date.now(), log = console, fetchImg = fetchImage }) {
+ const cfg = config.vibe || {};
+ const pagesFor = (size) => (cfg.pages && cfg.pages[size]) || { quick: 6, standard: 12, big: 24 }[size] || 12;
+ const owner = `wiki-${process.pid}-${crypto.randomBytes(3).toString('hex')}`;
+ const q = {
+ insert: db.prepare(`INSERT INTO wiki_vibes (id, space_id, topic, lang, size, status, requested_by, progress, created_at, updated_at)
+ VALUES (@id, @space_id, @topic, @lang, @size, 'queued', @requested_by, '{}'::jsonb, @now, @now)`),
+ get: db.prepare('SELECT * FROM wiki_vibes WHERE id = ?'),
+ bySpace: db.prepare('SELECT * FROM wiki_vibes WHERE space_id = ? ORDER BY created_at DESC LIMIT 1'),
+ mine: db.prepare('SELECT * FROM wiki_vibes WHERE requested_by = ? ORDER BY created_at DESC LIMIT ?'),
+ countMine: db.prepare('SELECT COUNT(*)::int AS n FROM wiki_vibes WHERE requested_by = ? AND created_at > ?'),
+ countAll: db.prepare('SELECT COUNT(*)::int AS n FROM wiki_vibes WHERE created_at > ?'),
+ recentDone: db.prepare(`SELECT v.* FROM wiki_vibes v JOIN wiki_spaces s ON s.id = v.space_id
+ WHERE v.status = 'done' AND s.deleted_at IS NULL AND s.visibility = 'public' ORDER BY v.finished_at DESC LIMIT ?`),
+ claim: db.prepare(`UPDATE wiki_vibes SET lease_owner = @owner, lease_until = @until, updated_at = @now
+ WHERE id = (SELECT id FROM wiki_vibes WHERE status IN ('queued','researching','planning','writing')
+ AND (retry_at IS NULL OR retry_at <= @now) AND (lease_until IS NULL OR lease_until < @now)
+ ORDER BY created_at LIMIT 1 FOR UPDATE SKIP LOCKED)
+ RETURNING *`),
+ renew: db.prepare('UPDATE wiki_vibes SET lease_until = ? WHERE id = ? AND lease_owner = ?'),
+ save: db.prepare(`UPDATE wiki_vibes SET status = @status, plan = @plan::jsonb, progress = @progress::jsonb, error = @error, attempts = @attempts,
+ retry_at = @retry_at, lease_owner = @lease_owner, lease_until = @lease_until, updated_at = @now, finished_at = @finished_at WHERE id = @id`),
+ };
+
+ function requirePerson(actor) {
+ if (!actor || actor.kind !== 'user' || !/^usr_/.test(String(actor.subject || ''))) throw new VibeError(401, 'vibe.sign_in', 'Sign in with your OpenVibe account to vibe a wiki');
+ return actor.subject;
+ }
+
+ async function freeSlug(base) {
+ const root = base || `wiki-${crypto.randomBytes(3).toString('hex')}`;
+ for (const s of [root, `${root}-wiki`, ...[2, 3, 4, 5, 6].map((n) => `${root}-${n}`), `${root}-${crypto.randomBytes(2).toString('hex')}`]) {
+ if (!content.SPACE_SLUG_RE.test(s)) continue;
+ if (!await svc.findSpace(s)) return s;
+ }
+ throw new VibeError(409, 'vibe.slug_taken', 'Every address for this topic is taken; try naming it differently');
+ }
+
+ /** Start one: checks, the person's new wiki, the job. → { vibe, space } */
+ async function start({ topic, size = 'standard', visibility = 'public', lang = 'en' } = {}, actor) {
+ const who = requirePerson(actor);
+ const name = oneLine(topic, 120);
+ if (name.replace(/[^\p{L}\p{N}]/gu, '').length < 2) throw new VibeError(422, 'vibe.invalid_topic', 'Name a topic: a show, a game, a place, a person, an idea');
+ if (!SIZES.includes(size)) throw new VibeError(422, 'vibe.invalid_size', 'size is quick, standard or big');
+ if (!['public', 'private'].includes(visibility)) throw new VibeError(422, 'vibe.invalid_visibility', 'visibility is public or private');
+ const language = /^[a-z]{2,3}$/.test(lang || '') ? lang : 'en';
+ if (!platform.ai.configured || !platform.media.configured) throw new VibeError(503, 'vibe.unavailable', 'Generating wikis needs OpenVibe.AI and OpenVibe.Media, and one of them is not configured here');
+ const since = now() - DAY_MS;
+ if ((await q.countMine.get(who, since)).n >= (cfg.perPersonDay || 3)) throw new VibeError(429, 'vibe.daily_limit', `You can vibe ${cfg.perPersonDay || 3} wikis a day; try again tomorrow`, { retryAfter: 3600 });
+ if ((await q.countAll.get(since)).n >= (cfg.perDay || 30)) throw new VibeError(429, 'vibe.busy', 'Today\'s wikis are all being written; try again later', { retryAfter: 3600 });
+ const slug = await freeSlug(spaceSlugFor(name));
+ const space = await svc.createSpace({ slug, name, description: `A wiki about ${name}, being researched and written by AI.`, kind: 'user', visibility }, actor);
+ const id = `vib_${ulid(now())}`;
+ await q.insert.run({ id, space_id: space.id, topic: name, lang: language, size, requested_by: who, now: now() });
+ if (timer) setImmediate(() => { tick().catch((err) => log.warn(`[Vibe] ${err && err.message}`)); }); // a running worker starts it now
+ return { vibe: await q.get.get(id), space };
+ }
+
+ /** The public shape of a job (what /vibe/:id and the API show). */
+ function present(v, space) {
+ const progress = v.progress || {};
+ const pages = (v.plan && v.plan.pages) || [];
+ const state = progress.pages || {};
+ const list = pages.map((p) => ({ title: p.title, kind: p.kind, status: (state[p.slug] && state[p.slug].status) || 'waiting', url: state[p.slug] && state[p.slug].status === 'done' ? `/w/${encodeURIComponent(space.slug)}/${encodeURIComponent(p.slug)}` : null, images: (state[p.slug] && state[p.slug].images) || 0, sources: (state[p.slug] && state[p.slug].sources) || 0 }));
+ return {
+ id: v.id, topic: v.topic, size: v.size, status: v.status,
+ space: { slug: space.slug, name: space.name, url: `/s/${encodeURIComponent(space.slug)}`, visibility: space.visibility },
+ research: progress.research ? { sources: progress.research.sources.length, images: progress.research.images.length, from: sourceNames(progress.research.from), topic_wiki: progress.research.topic_wiki || null } : null,
+ pages: list,
+ done: list.filter((p) => p.status === 'done').length,
+ total: list.length,
+ error: v.status === 'failed' ? v.error : null,
+ waiting_until: v.retry_at && v.retry_at > now() ? new Date(Number(v.retry_at)).toISOString() : null,
+ created_at: new Date(Number(v.created_at)).toISOString(),
+ finished_at: v.finished_at ? new Date(Number(v.finished_at)).toISOString() : null,
+ main_page: list[0] && list[0].url ? list[0].url : null,
+ };
+ }
+
+ async function get(id, actor) {
+ const v = /^vib_[0-9A-HJKMNP-TV-Z]{26}$/.test(String(id || '')) ? await q.get.get(id) : null;
+ if (!v) return null;
+ const space = await svc.spaceById(v.space_id);
+ if (!space || space.deleted_at) return null;
+ await svc.access.loadRoles(actor || { kind: 'anonymous' }, { fresh: true });
+ if (!svc.access.canReadSpace(space, actor || { kind: 'anonymous' })) return null;
+ return present(v, space);
+ }
+
+ async function mine(actor, limit = 20) {
+ const who = requirePerson(actor);
+ const rows = await q.mine.all(who, limit);
+ const out = [];
+ for (const v of rows) { const space = await svc.spaceById(v.space_id); if (space && !space.deleted_at) out.push(present(v, space)); }
+ return out;
+ }
+
+ async function recent(limit = 12) {
+ const rows = await q.recentDone.all(limit);
+ const out = [];
+ for (const v of rows) { const space = await svc.spaceById(v.space_id); if (space && !space.deleted_at) out.push(present(v, space)); }
+ return out;
+ }
+
+ // ── The worker ──────────────────────────────────────────
+ let running = false;
+ let timer = null;
+
+ async function save(v, patch = {}) {
+ Object.assign(v, patch);
+ await q.save.run({
+ id: v.id, status: v.status, plan: JSON.stringify(v.plan || null), progress: JSON.stringify(v.progress || {}), error: v.error || null,
+ attempts: v.attempts || 0, retry_at: v.retry_at || null, lease_owner: v.lease_owner || null, lease_until: v.lease_until || null,
+ now: now(), finished_at: v.finished_at || null,
+ });
+ }
+
+ async function tick() {
+ if (running) return false;
+ running = true;
+ try {
+ const v = await q.claim.get({ owner, until: now() + LEASE_MS, now: now() });
+ if (!v) return false;
+ await runJob(v);
+ return true;
+ } finally { running = false; }
+ }
+
+ function startWorker() {
+ if (timer) return;
+ const loop = () => { tick().then((worked) => { timer = setTimeout(loop, worked ? 200 : (cfg.intervalMs || 5000)); timer.unref && timer.unref(); }, (err) => { log.warn(`[Vibe] worker: ${err && err.message}`); timer = setTimeout(loop, cfg.intervalMs || 5000); timer.unref && timer.unref(); }); };
+ timer = setTimeout(loop, 1000);
+ timer.unref && timer.unref();
+ }
+ function stopWorker() { if (timer) clearTimeout(timer); timer = null; }
+
+ const sourceForAi = (s, chars) => ({ ...s, content: clip(s.content, chars), snippet: s.snippet ? clip(s.snippet, 400) : undefined });
+
+ async function runJob(v) {
+ const actor = { kind: 'user', subject: v.requested_by, staff: false };
+ const space = await svc.spaceById(v.space_id);
+ if (!space || space.deleted_at) { await save(v, { status: 'cancelled', error: 'The wiki was deleted', lease_owner: null, lease_until: null, finished_at: now() }); return; }
+ v.progress = v.progress || {};
+ const ai = (workflow, input, key) => platform.ai.run(workflow, input, { subject: v.requested_by, idempotencyKey: `vibe:${v.id}:${key}`, totalMs: 300000 });
+ try {
+ // 1. Research the topic.
+ if (!v.progress.research) {
+ await save(v, { status: 'researching', error: null });
+ const r = await ai('research.gather', { query: v.topic, topic: v.topic, lang: v.lang, max_sources: 14, max_images: 16, content_chars: 4500 }, 'research');
+ const out = r.output || {};
+ if (!out.sources || !out.sources.length) throw new VibeError(422, 'vibe.nothing_found', `Nothing could be found about "${v.topic}". Try its full name or another spelling.`);
+ v.progress.research = {
+ sources: out.sources, images: out.images || [], facts: out.facts || [], topic_wiki: out.topic_wiki || null, run: r.runId,
+ from: Object.entries(out.providers || {}).filter(([, p]) => p && p.ok && p.count).map(([k]) => k),
+ };
+ await save(v);
+ }
+ const research = v.progress.research;
+ // 2. Plan the pages.
+ if (!v.plan) {
+ await save(v, { status: 'planning' });
+ const max = pagesFor(v.size);
+ const r = await ai('wiki.generate_space', { topic: v.topic, max_pages: max, sources: research.sources.map((s) => sourceForAi(s, 3500)) }, 'plan');
+ const plan = normalizePlan(r.output, v.topic, max);
+ plan.run = r.runId;
+ v.plan = plan;
+ v.progress.pages = {};
+ await save(v);
+ await svc.updateSpace(space.id, { name: plan.title.slice(0, 120), description: clip(plan.summary || `A wiki about ${v.topic}.`, 1000) }, actor).catch((err) => log.warn(`[Vibe] ${v.id} space details: ${err && err.message}`));
+ }
+ // 3. Write the pages, a few at a time; the main page first so the rest hang under it.
+ await save(v, { status: 'writing' });
+ const titles = v.plan.pages.map((p) => p.title);
+ const todo = v.plan.pages.filter((p) => !['done', 'failed'].includes((v.progress.pages[p.slug] || {}).status));
+ const first = todo[0] && todo[0].slug === v.plan.pages[0].slug ? [todo.shift()] : [];
+ for (const p of first) await writeOne(v, space, actor, p, titles, ai);
+ const width = cfg.concurrency || 2;
+ let next = 0;
+ await Promise.all(Array.from({ length: Math.min(width, todo.length) }, async () => {
+ while (next < todo.length) { const p = todo[next++]; await writeOne(v, space, actor, p, titles, ai); }
+ }));
+ // One more try for pages that hit a passing failure.
+ for (const p of v.plan.pages.filter((x) => (v.progress.pages[x.slug] || {}).status === 'waiting')) await writeOne(v, space, actor, p, titles, ai);
+ const states = Object.values(v.progress.pages);
+ const written = states.filter((s) => s.status === 'done').length;
+ if (!written) throw new VibeError(502, 'vibe.nothing_written', 'No page could be written from what was found');
+ await save(v, { status: 'done', error: null, lease_owner: null, lease_until: null, finished_at: now() });
+ await svc.updateSpace(space.id, { description: clip(`${v.plan.summary || `A wiki about ${v.topic}.`}`, 1000) }, actor).catch(() => {});
+ log.log(`[Vibe] ${v.id} "${v.topic}": ${written} of ${v.plan.pages.length} pages`);
+ } catch (err) {
+ if (err && err.status === 429) {
+ const wait = Math.max(30, Math.min(Number(err.retryAfter) || 300, 6 * 3600));
+ await save(v, { retry_at: now() + wait * 1000, error: `Waiting for OpenVibe.AI: ${err.message}`, lease_owner: null, lease_until: null });
+ return;
+ }
+ const attempts = (v.attempts || 0) + 1;
+ const final = attempts >= 3 || (err && err.status && err.status < 500 && err.status !== 408);
+ log.warn(`[Vibe] ${v.id} "${v.topic}" ${final ? 'failed' : 'will retry'}: ${err && err.message}`);
+ await save(v, {
+ attempts, error: oneLine(err && err.message, 500) || 'failed',
+ ...(final ? { status: 'failed', finished_at: now() } : { retry_at: now() + attempts * 60000 }),
+ lease_owner: null, lease_until: null,
+ });
+ }
+ }
+
+ async function writeOne(v, space, actor, p, titles, ai) {
+ const state = v.progress.pages[p.slug] || {};
+ v.progress.pages[p.slug] = { ...state, status: 'writing' };
+ await save(v);
+ await q.renew.run(now() + LEASE_MS, v.id, owner);
+ try {
+ const isMain = p.slug === v.plan.pages[0].slug;
+ const existing = await svc.findPage(space.id, p.slug);
+ if (existing && existing.state !== 'deleted') { v.progress.pages[p.slug] = { status: 'done', sources: 0, images: 0 }; await save(v); return; }
+ // The page's own research (the main page reuses the topic's).
+ let research = v.progress.research;
+ if (!isMain) {
+ const r = await ai('research.gather', { query: p.query || `${p.title} ${v.topic}`, topic: v.topic, focus: p.title, lang: v.lang, max_sources: 8, max_images: 10, content_chars: 4000 }, `research:${p.slug}`);
+ research = r.output || { sources: [], images: [] };
+ // The topic's best source helps every page: its facts and names.
+ const lead = v.progress.research.sources[0];
+ if (lead && !research.sources.some((s) => s.url === lead.url) && research.sources.length < 10) research = { ...research, sources: [...research.sources, lead] };
+ }
+ const sources = (research.sources || []).slice(0, 10).map((s) => sourceForAi(s, isMain ? 4000 : 3500));
+ if (!sources.length) throw new VibeError(422, 'vibe.page_no_sources', 'nothing was found about it');
+ const pictures = (research.images || []).slice(0, 12);
+ const r = await ai('wiki.generate_page', {
+ title: p.title, topic: v.plan.title || v.topic, space: v.plan.title || v.topic, outline: p.outline, pages: titles,
+ images: pictures.map((im) => ({ title: oneLine(im.title, 300) || 'Picture', description: oneLine([im.description, im.license ? `(${im.license})` : ''].filter(Boolean).join(' '), 400) })),
+ sources,
+ }, `page:${p.slug}`);
+ const out = r.output || {};
+ // Pictures: fetched, stored in Media for the person, placed by heading.
+ const placed = new Map();
+ const usedPictures = [];
+ for (const pick of (Array.isArray(out.images) ? out.images : []).slice(0, MAX_IMAGES_PER_PAGE)) {
+ const img = Number.isInteger(pick.index) ? pictures[pick.index] : null;
+ if (!img || usedPictures.includes(img)) continue;
+ const key = /^lead$/i.test(String(pick.section || '').trim()) || !pick.section ? 'lead' : oneLine(pick.section, 120).replace(/^#+\s*/, '').toLowerCase();
+ if (placed.has(key)) continue;
+ try {
+ const got = await fetchImg(img.url, { timeoutMs: 15000 });
+ const ext = { 'image/jpeg': 'jpg', 'image/png': 'png', 'image/gif': 'gif', 'image/webp': 'webp' }[got.mime];
+ const mediaId = await platform.media.upload({
+ bytes: got.bytes, mime: got.mime, filename: `${content.pageSlug(img.title || p.title).slice(0, 60) || 'picture'}.${ext}`, owner: v.requested_by,
+ metadata: { title: oneLine(img.title, 200), source_url: img.source_url || img.url, license: img.license || null, license_url: img.license_url || null, author: img.author || null, free: Boolean(img.free), origin: 'wiki.vibe', vibe: v.id },
+ });
+ const caption = clip(`${oneLine(pick.caption, 220) || oneLine(img.title, 120)} (${creditOf(img)})`, 300);
+ placed.set(key, { mediaId, caption, alt: oneLine(pick.caption || img.title, 200), img });
+ usedPictures.push(img);
+ } catch (err) {
+ log.warn(`[Vibe] ${v.id} picture for "${p.title}" skipped: ${err && err.message}`);
+ }
+ }
+ const lead = placed.get('lead');
+ const body = composeBody(out, { placed: new Map([...placed].filter(([k]) => k !== 'lead')), contents: isMain ? contentsOf(v.plan) : null });
+ if (!body || body.length < 200) throw new VibeError(422, 'vibe.page_too_thin', 'the sources did not say enough for a page');
+ const infobox = [];
+ if (lead) infobox.push({ label: 'Picture', type: 'media', value: lead.mediaId });
+ const keys = new Set(lead ? ['picture'] : []);
+ for (const row of Array.isArray(out.infobox) ? out.infobox : []) {
+ const label = oneLine(row.label, 60).replace(/\|/g, '/');
+ const value = oneLine(row.value, 300);
+ const key = content.keyFromLabel(label);
+ if (!label || !value || !key || keys.has(key)) continue;
+ keys.add(key);
+ infobox.push({ label, type: 'text', value });
+ if (infobox.length >= 14) break;
+ }
+ const cited = citedIndices(out, sources.length);
+ const citations = cited.map((i) => sources[i]).filter((s) => s && s.url).map((s) => ({
+ url: s.url, title: oneLine(s.title, 300) || s.url, retrievedAt: s.retrieved_at || new Date(now()).toISOString(),
+ licenseNote: (s.provenance && s.provenance.license) || null,
+ }));
+ for (const pic of placed.values()) {
+ const img = pic.img;
+ if (!img.source_url && !img.url) continue;
+ citations.push({ url: img.source_url || img.url, title: `Picture: ${oneLine(img.title, 200) || p.title}`, retrievedAt: new Date(now()).toISOString(), licenseNote: oneLine([img.license, img.author].filter(Boolean).join(', '), 300) || null });
+ }
+ const from = [...new Set(sources.map((s) => (s.provenance && s.provenance.provider) || s.source_type))];
+ const bundle = importer.validateBundle({
+ pages: [{
+ title: p.title, body, summary: clip(out.summary, 290) || null,
+ parent: isMain ? null : v.plan.pages[0].title,
+ infobox, citations: citations.slice(0, 50),
+ message: clip(`Written by AI from ${citations.length} source${citations.length === 1 ? '' : 's'}`, 290),
+ }],
+ publish: true, on_existing: 'skip', ai_assisted: true,
+ source: clip(`OpenVibe.AI research across ${sourceNames(from).join(', ')}`, 190),
+ });
+ const made = await svc.importPages(space.id, bundle, actor);
+ const page = made.created[0] && made.created[0].page;
+ if (page) {
+ for (const pic of placed.values()) {
+ await svc.attachMedia(page.id, { mediaId: pic.mediaId, alt: pic.alt, caption: pic.caption, role: pic === lead ? 'cover' : 'inline' }, actor, { describe: platform.media.describe })
+ .catch((err) => log.warn(`[Vibe] ${v.id} attach to "${p.title}": ${err && err.message}`));
+ }
+ }
+ v.progress.pages[p.slug] = { status: 'done', sources: citations.length, images: placed.size, run: r.runId };
+ } catch (err) {
+ if (err && err.status === 429) { v.progress.pages[p.slug] = { ...state, status: 'waiting' }; await save(v); throw err; }
+ const tries = (state.tries || 0) + 1;
+ log.warn(`[Vibe] ${v.id} page "${p.title}": ${err && err.message}`);
+ v.progress.pages[p.slug] = { status: tries >= 2 || (err && err.status && err.status < 500) ? 'failed' : 'waiting', tries, error: oneLine(err && err.message, 200) };
+ }
+ await save(v);
+ }
+
+ return { start, get, mine, recent, tick, startWorker, stopWorker, SIZES, ACTIVE, pagesFor };
+}
+
+module.exports = { createVibes, VibeError, normalizePlan, composeBody, contentsOf, citedIndices, creditOf, spaceSlugFor, SIZES };
@@ -40,7 +40,7 @@ const quiet = process.env.WIKI_TEST_LOG ? console : { log() {}, warn() {}, error
@@ -40,7 +40,7 @@ const quiet = process.env.WIKI_TEST_LOG ? console : { log() {}, warn() {}, error
* A running Wiki on a database of its own (PGlite by default; WIKI_TEST_STORE=pg: the PostgreSQL + PgBouncer
* A running Wiki on a database of its own (PGlite by default; WIKI_TEST_STORE=pg: the PostgreSQL + PgBouncer
* containers). opts.db: boot on an existing handle (a restart keeps the data); h.stop() closes what boot opened.
* containers). opts.db: boot on an existing handle (a restart keeps the data); h.stop() closes what boot opened.
*/
*/
-async function boot({ env = {}, fetch: fetchImpl, db: givenDb = null, now, workers = false, tokens, rateLimits = false, limitsNow = null, log = quiet, indexnow = undefined, accountSend = null, telemetrySink = null } = {}) {
+async function boot({ env = {}, fetch: fetchImpl, db: givenDb = null, now, workers = false, tokens, rateLimits = false, limitsNow = null, log = quiet, indexnow = undefined, accountSend = null, telemetrySink = null, imageFetch = null } = {}) {
const owned = givenDb ? null : await testDb();
const owned = givenDb ? null : await testDb();
const db = givenDb || owned.db;
const db = givenDb || owned.db;
const config = load({
const config = load({
@@ -52,7 +52,7 @@ async function boot({ env = {}, fetch: fetchImpl, db: givenDb = null, now, worke
@@ -52,7 +52,7 @@ async function boot({ env = {}, fetch: fetchImpl, db: givenDb = null, now, worke
config, db, publicKey, log, listen: true, workers, rateLimits, now, limitsNow,
config, db, publicKey, log, listen: true, workers, rateLimits, now, limitsNow,
fetchImpl: fetchImpl || (async (url) => { throw new Error(`unexpected outbound fetch ${url}`); }),
fetchImpl: fetchImpl || (async (url) => { throw new Error(`unexpected outbound fetch ${url}`); }),
tokens: tokens || { getToken: async () => 'stub-token', authHeaders: async () => ({ Authorization: 'Bearer stub-token' }), invalidate() {} },
tokens: tokens || { getToken: async () => 'stub-token', authHeaders: async () => ({ Authorization: 'Bearer stub-token' }), invalidate() {} },
- indexnow, accountSend, telemetrySink,
+ indexnow, accountSend, telemetrySink, imageFetch,
});
});
const base = `http://127.0.0.1:${h.server.address().port}`;
const base = `http://127.0.0.1:${h.server.address().port}`;
const stop = async ({ keepDb = false } = {}) => { await h.stop(); if (owned && !keepDb) await owned.close(); };
const stop = async ({ keepDb = false } = {}) => { await h.stop(); if (owned && !keepDb) await owned.close(); };
@@ -62,8 +62,9 @@ const INTERNAL = [
@@ -62,8 +62,9 @@ const INTERNAL = [
'server/auth/keys.js': 'Network JWKS (configured)',
'server/auth/keys.js': 'Network JWKS (configured)',
'server/auth/session.js': 'Network OAuth token and revoke (configured)',
'server/auth/session.js': 'Network OAuth token and revoke (configured)',
'server/http/frame-groups.js': 'Network /api/chrome, the Frame\'s site groups (configured: WIKI_FRAME_URL or the internal Network)',
'server/http/frame-groups.js': 'Network /api/chrome, the Frame\'s site groups (configured: WIKI_FRAME_URL or the internal Network)',
- 'server/integrations/platform.js': 'Community, Sources, Media, VIP (configured)',
+ 'server/integrations/platform.js': 'Community, Sources, Media, VIP, AI (configured)',
'server/integrations/projects-module.js': 'Network user modules (configured)',
'server/integrations/projects-module.js': 'Network user modules (configured)',
+ 'server/integrations/image-fetch.js': 'pictures a generated wiki uses (research results): https only, openvibe-shared/egress safeLookup on every hop, a byte cap, magic bytes checked',
};
};
const root = path.join(__dirname, '..');
const root = path.join(__dirname, '..');
const found = [];
const found = [];
new file mode 100644
new file mode 100644
@@ -0,0 +1,196 @@
@@ -0,0 +1,196 @@
+'use strict';
+/**
+ * "Vibe a wiki" end to end against stand-ins for OpenVibe.AI and OpenVibe.Media: a person names a topic → a new wiki
+ * they own → research, plan, pages written with their sources and pictures (stored in Media for the person, placed in
+ * the text and the infobox, credited as citations) → published with the AI disclosure and noindex. AI's quota (429)
+ * makes the job wait and resume; the daily limit refuses; a signed-out visitor is asked to sign in. Also: the
+ * [[media:…]] syntax and the picture fetcher's refusals.
+ */
+const assert = require('assert');
+const H = require('./helpers');
+const content = require('../server/wiki/content');
+const { checkUrl, sniff } = require('../server/integrations/image-fetch');
+const { normalizePlan, spaceSlugFor, contentsOf } = require('../server/wiki/vibe');
+
+const MED = (n) => `med_01J8Z6Q3KX0000000000000${String(n).padStart(3, '0')}`;
+const PNG = Buffer.from('89504e470d0a1a0a0000000d4948445200000001000000010806000000', 'hex');
+const LONG = 'Kim Possible is a teenager who saves the world between classes, with her best friend and a naked mole-rat. '.repeat(6);
+
+// Unit checks first: the syntax, the fetcher's rules, the plan's shape.
+{
+ const src = 'Intro.\n\n[[media:' + MED(1) + '|A caption]]\n\nSee [[Ron Stoppable]] and [[media:' + MED(9) + ']].';
+ assert.deepStrictEqual(content.extractLinks(src, 'kp').map((l) => l.slug), ['ron-stoppable'], 'a picture is not a page link');
+ assert.deepStrictEqual(content.mediaRefs(src), [MED(1), MED(9)]);
+ const out = content.renderContent(src, { currentSpace: 'kp', resolve: () => ({ href: '/w/kp/ron-stoppable', exists: true }), media: (id, cap) => (id === MED(1) ? `<figure data-id="${id}">${cap}</figure>` : null) });
+ assert.ok(out.includes(`<figure data-id="${MED(1)}">A caption</figure>`), out);
+ assert.ok(!/<p>\s*<figure/.test(out), 'a picture on its own line is not wrapped in a paragraph');
+ assert.ok(!out.includes(MED(9)), 'a picture the page does not have shows nothing');
+ for (const bad of ['http://x.test/a.png', 'https://127.0.0.1/a.png', 'https://user:[email protected]/a.png', 'file:///etc/passwd', 'nonsense']) assert.throws(() => checkUrl(bad), bad);
+ assert.strictEqual(sniff(PNG), 'image/png');
+ assert.strictEqual(sniff(Buffer.from('<svg xmlns="http://www.w3.org/2000/svg"></svg>')), null, 'an SVG is never taken for a picture');
+ assert.strictEqual(spaceSlugFor('Pokémon Red & Blue!'), 'pokemon-red-blue');
+ assert.strictEqual(spaceSlugFor('東京'), null);
+ const plan = normalizePlan({ title: 'KP', summary: 's', pages: [{ title: 'Ron', kind: 'character', outline: ['A'] }, { title: 'Kim Possible', kind: 'overview' }, { title: 'ron', kind: 'person' }, { title: 'Bad [[x]]|', kind: 'weird' }] }, 'Kim Possible', 3);
+ assert.deepStrictEqual(plan.pages.map((p) => [p.title, p.kind]), [['Kim Possible', 'overview'], ['Ron', 'character'], ['Bad x', 'other']], 'main page first, duplicates out, kinds known, link syntax stripped');
+ assert.match(contentsOf(plan), /### Characters\n\n- \[\[Ron\]\]/);
+}
+
+(async () => {
+ const A = H.subject();
+ const calls = [];
+ let quota = 1; // the first write of "Ron Stoppable" meets a full quota
+ let objects = 0;
+ const json = (status, body, headers = {}) => new Response(JSON.stringify(body), { status, headers: { 'Content-Type': 'application/json', ...headers } });
+ const research = (focus) => ({
+ query: focus || 'Kim Possible', topic: 'Kim Possible', focus: focus || null,
+ sources: [
+ { source_type: 'wikipedia', url: 'https://en.wikipedia.org/wiki/Kim_Possible', title: 'Kim Possible — Wikipedia', content: LONG, retrieved_at: '2026-10-25T10:00:00.000Z', provenance: { provider: 'wikipedia', license: 'CC BY-SA 4.0 (Wikipedia)' } },
+ { source_type: 'fandom', url: `https://kimpossible.fandom.com/wiki/${focus ? 'Ron_Stoppable' : 'Kim_Possible'}`, title: 'Kim Possible Wiki', content: LONG, retrieved_at: '2026-10-25T10:00:00.000Z', provenance: { provider: 'fandom', license: 'CC BY-SA 3.0 (Fandom)' } },
+ ],
+ images: [{ url: 'https://img.test/kim.png', title: focus || 'Kim Possible', description: 'from the show', license: 'CC BY-SA 4.0', author: 'Jane Doe', free: true, source_url: 'https://commons.wikimedia.org/wiki/File:Kim.png', provider: 'commons' }],
+ facts: [], topic_wiki: { name: 'Kim Possible Wiki', url: 'https://kimpossible.fandom.com' }, providers: { wikipedia: { ok: true, count: 1 }, fandom: { ok: true, count: 1 }, deepseek: { ok: false, skipped: 'not configured' } }, usage: {},
+ });
+ const page = (title) => ({
+ title, summary: `${title} is part of Kim Possible.`, lead: `**${title}** is part of the show. ${title === 'Kim Possible' ? 'Her friend is [[Ron Stoppable]].' : 'He is friends with [[Kim Possible]].'} ${LONG}`,
+ sections: [{ heading: 'Premise', body: LONG, citations: [0] }, { heading: 'Legacy', body: LONG, citations: [1] }],
+ infobox: [{ label: 'Created by', value: 'Bob Schooley and Mark McCorkle', citations: [0] }, { label: 'Network', value: 'Disney Channel', citations: [0] }],
+ images: [{ index: 0, caption: `${title} on screen`, section: 'lead' }, { index: 7, caption: 'out of range', section: 'Premise' }],
+ citations: [0, 1], gaps: [],
+ });
+ const fetchStub = async (url, init = {}) => {
+ const u = new URL(url);
+ calls.push({ host: u.host, path: u.pathname, method: init.method || 'GET', headers: init.headers || {}, body: typeof init.body === 'string' ? init.body : null });
+ if (u.host === 'ai.internal') {
+ if (u.pathname !== '/api/v1/runs' || init.method !== 'POST') return json(404, {});
+ const b = JSON.parse(init.body);
+ assert.deepStrictEqual(b.on_behalf_of, { type: 'user', id: A }, 'AI runs on behalf of the person');
+ const id = `run_${calls.length}`;
+ if (b.workflow === 'research.gather') return json(201, { run: { id, status: 'succeeded', output: research(b.input.focus) } });
+ if (b.workflow === 'wiki.generate_space') {
+ assert.ok(b.input.sources.length >= 1 && b.input.max_pages === 6);
+ return json(201, { run: { id, status: 'succeeded', output: { title: 'Kim Possible', summary: 'An animated series about a teen hero.', pages: [
+ { title: 'Kim Possible', kind: 'overview', parent: '', outline: ['Premise', 'Legacy'], query: 'Kim Possible', citations: [0] },
+ { title: 'Ron Stoppable', kind: 'character', parent: 'Kim Possible', outline: ['Premise'], query: 'Ron Stoppable Kim Possible', citations: [1] },
+ { title: 'Kim Possible', kind: 'other', parent: '', outline: [], query: '', citations: [] },
+ ], citations: [0], gaps: [] } } });
+ }
+ if (b.workflow === 'wiki.generate_page') {
+ assert.ok(b.input.pages.includes('Ron Stoppable') && b.input.images.length === 1);
+ if (b.input.title === 'Ron Stoppable' && quota-- > 0) return json(429, { code: 'quota.exceeded', detail: 'daily budget used' }, { 'Retry-After': '120' });
+ return json(201, { run: { id, status: 'succeeded', output: page(b.input.title) } });
+ }
+ return json(404, {});
+ }
+ if (u.host === 'media.internal') {
+ const m = /^\/api\/v2\/wiki\/objects(?:\/([^/]+))?(?:\/(content|complete))?$/.exec(u.pathname);
+ if (!m) return json(404, {});
+ if (!m[1] && init.method === 'POST') {
+ const b = JSON.parse(init.body);
+ assert.strictEqual(b.visibility, 'public');
+ assert.strictEqual(b.kind, 'asset');
+ assert.strictEqual(init.headers['X-OV-Subject'], A, 'the picture belongs to the person');
+ assert.strictEqual(b.metadata.license, 'CC BY-SA 4.0');
+ const id = MED(++objects);
+ return json(201, { id, upload: { method: 'PUT', url: `https://openvibe.media/api/v2/wiki/objects/${id}/content?token=t1`, token: 't1', complete_url: `https://openvibe.media/api/v2/wiki/objects/${id}/complete` } });
+ }
+ if (m[2] === 'content') { assert.strictEqual(init.method, 'PUT'); return json(200, { ok: true }); }
+ if (m[2] === 'complete') return json(200, { object: { id: m[1], lifecycle_status: 'ready' } });
+ return json(200, { id: m[1], visibility: 'public', owner: { subject: A, app: 'wiki' }, lifecycle_status: 'ready' });
+ }
+ return json(404, { error: 'stub: no route' });
+ };
+ const fetched = [];
+ const h = await H.boot({
+ fetch: fetchStub, imageFetch: async (url) => { fetched.push(url); return { bytes: PNG, mime: 'image/png', url }; },
+ env: { OV_MEDIA_INTERNAL_URL: 'http://media.internal', OV_AI_INTERNAL_URL: 'http://ai.internal', WIKI_VIBE_PER_PERSON_DAY: '2', WIKI_VIBE_PAGES_QUICK: '6' },
+ });
+ const tok = H.userToken({ subject: A });
+ try {
+ // Signed out: asked to sign in; the form is on /vibe and the home.
+ let r = await H.req(h, 'POST', '/vibe', { form: { topic: 'Kim Possible' } });
+ assert.strictEqual(r.status, 401);
+ r = await H.req(h, 'GET', '/vibe');
+ assert.strictEqual(r.status, 200);
+ assert.match(r.text, /Vibe a wiki/);
+ assert.match(r.text, /Sign in to vibe/);
+ r = await H.req(h, 'GET', '/');
+ assert.match(r.text, /Any topic\./);
+ assert.match(r.text, /href="\/vibe"/);
+
+ // Start one.
+ r = await H.req(h, 'POST', '/api/v1/vibes', { token: tok, body: { topic: ' Kim Possible ', size: 'quick' } });
+ assert.strictEqual(r.status, 202, r.text);
+ const id = r.json.vibe.id;
+ assert.strictEqual(r.json.vibe.status, 'queued');
+ assert.strictEqual(r.json.space.slug, 'kim-possible');
+ assert.strictEqual(r.json.space.kind, 'user');
+ assert.match(r.json.progress_url, new RegExp(`/vibe/${id}$`));
+ r = await H.req(h, 'POST', '/api/v1/vibes', { token: tok, body: { topic: 'x' } });
+ assert.strictEqual(r.status, 422);
+ r = await H.req(h, 'POST', '/api/v1/vibes', { token: tok, body: { topic: 'Kim Possible', size: 'enormous' } });
+ assert.strictEqual(r.status, 422);
+
+ // Research, plan, the main page; Ron meets a full quota → the job waits for Retry-After.
+ assert.strictEqual(await h.vibes.tick(), true);
+ let v = (await H.req(h, 'GET', `/api/v1/vibes/${id}`, { token: tok })).json.vibe;
+ assert.strictEqual(v.status, 'writing');
+ assert.ok(v.waiting_until, 'waiting for AI\'s quota');
+ assert.deepStrictEqual(v.pages.map((p) => [p.title, p.status]), [['Kim Possible', 'done'], ['Ron Stoppable', 'waiting']]);
+ assert.strictEqual(await h.vibes.tick(), false, 'nothing is due while it waits');
+ await h.db.prepare('UPDATE wiki_vibes SET retry_at = 1 WHERE id = ?').run(id);
+ assert.strictEqual(await h.vibes.tick(), true);
+ v = (await H.req(h, 'GET', `/api/v1/vibes/${id}`, { token: tok })).json.vibe;
+ assert.strictEqual(v.status, 'done', JSON.stringify(v));
+ assert.strictEqual(v.done, 2);
+ assert.strictEqual(v.main_page, '/w/kim-possible/kim-possible');
+ assert.deepStrictEqual(v.research.from.sort(), ['Fandom', 'Wikipedia']);
+ assert.strictEqual(fetched.length, 2, 'one picture per page; an index out of range is ignored');
+
+ // The wiki: named and described by the plan, the person owns it.
+ r = await H.req(h, 'GET', '/api/v1/spaces/kim-possible', { token: tok });
+ assert.strictEqual(r.json.space.name, 'Kim Possible');
+ assert.strictEqual(r.json.space.description, 'An animated series about a teen hero.');
+ assert.deepStrictEqual(r.json.pages.map((p) => [p.slug, p.children.map((c) => c.slug)]), [['kim-possible', ['ron-stoppable']]], 'pages hang under the main page');
+
+ // The main page as a reader sees it.
+ r = await H.req(h, 'GET', '/w/kim-possible/kim-possible');
+ assert.strictEqual(r.status, 200);
+ assert.match(r.text, /written with AI assistance/i);
+ assert.match(r.text, /<meta name="robots" content="noindex/, 'noindex until a person reviews it');
+ assert.match(r.text, new RegExp(`class="wk-ib-pic"[\\s\\S]*?<img src="https://openvibe.media/o/${MED(1)}"`), 'the lead picture in the infobox');
+ assert.match(r.text, /Kim Possible on screen \(Picture: Jane Doe, CC BY-SA 4.0\)/, 'captioned with its credit');
+ assert.match(r.text, /<th scope="row">Created by<\/th><td>Bob Schooley and Mark McCorkle<\/td>/);
+ assert.match(r.text, /<h2[^>]*>In this wiki<\/h2>/);
+ assert.match(r.text, /<a class="ov-wikilink" href="\/w\/kim-possible\/ron-stoppable">Ron Stoppable<\/a>/, 'a blue link to the page written after it');
+ assert.match(r.text, /href="https:\/\/en.wikipedia.org\/wiki\/Kim_Possible"[^>]*>Kim Possible — Wikipedia<\/a>, retrieved/);
+ assert.match(r.text, /Picture: Kim Possible<\/a>, retrieved[\s\S]*?\(CC BY-SA 4.0, Jane Doe\)/, 'the picture cited with its licence');
+ assert.ok(!/<section class="wk-media">/.test(r.text), 'no second copy of a picture already shown');
+ const att = await h.db.prepare('SELECT o.media_id, o.attached_by, o.media_visibility FROM wiki_attachment_origins o JOIN wiki_pages p ON p.id = o.page_id WHERE p.slug = ?').all('kim-possible');
+ assert.deepStrictEqual(att, [{ media_id: MED(1), attached_by: A, media_visibility: 'public' }]);
+
+ // The progress page and its JSON.
+ r = await H.req(h, 'GET', `/vibe/${id}`);
+ assert.strictEqual(r.status, 200);
+ assert.match(r.text, /is ready/);
+ assert.match(r.text, /href="\/w\/kim-possible\/kim-possible">Open the wiki/);
+ r = await H.req(h, 'GET', `/vibe/${id}.json`);
+ assert.strictEqual(r.json.status, 'done');
+ assert.strictEqual((await H.req(h, 'GET', '/vibe/vib_01J8Z6Q3KX00000000000000000.json')).status, 404);
+ r = await H.req(h, 'GET', '/');
+ assert.match(r.text, /Wikis people vibed/);
+
+ // The daily limit (2 here): the second start is allowed, the third refused.
+ r = await H.req(h, 'POST', '/api/v1/vibes', { token: tok, body: { topic: 'Ron Stoppable', size: 'quick', visibility: 'private' } });
+ assert.strictEqual(r.status, 202, r.text);
+ r = await H.req(h, 'POST', '/api/v1/vibes', { token: tok, body: { topic: 'Rufus' } });
+ assert.strictEqual(r.status, 429);
+ assert.strictEqual(r.json.code, 'vibe.daily_limit');
+ // A private generated wiki's progress is not shown to others.
+ const second = (await H.req(h, 'GET', '/api/v1/vibes', { token: tok })).json.vibes.find((x) => x.topic === 'Ron Stoppable');
+ assert.strictEqual((await H.req(h, 'GET', `/vibe/${second.id}`)).status, 404);
+ assert.strictEqual((await H.req(h, 'GET', `/vibe/${second.id}.json`, { token: H.userToken() })).status, 404);
+ console.log('vibe: all checks passed');
+ } finally {
+ await h.stop();
+ }
+})().catch((err) => { console.error(err); process.exit(1); });