openvibers/openvibe.exchange Commits

Exchange: list the network's own free work and refresh it (#29)

OP OpenVibers committed 13 days ago · d87ebb1058 · 16 files, +880 −23

* Exchange: list the network's own free work and refresh it

Seed official, network-owned listings for Design's built-in themes and their token files, Live's OBS scenes and alerts overlay, and public Games worlds. Each keeps its real home URL and a source id, so the periodic refresh updates in place, retires what a source no longer lists, and never invents a person or touches someone's listing.

* Exchange: fold token files into their theme listing

A theme is one listing with its token exports as downloads (files: format, label, URL) instead of one listing per format. Migration 0003 adds the files column; the refresh deletes the old design.tokens rows it made, and does it idempotently. Listings (including the home's fresh list) are now interleaved by kind, newest first within each kind, so one kind cannot fill a page.

Changes

ddiff --git a/.env.example b/.env.example
iindex 8c45202..0f88a77 100644
---- a/.env.example
++++ b/.env.example
@@@ -35,6 +35,14 @@ OV_OAUTH_CLIENT_SECRET=
#EXCHANGE_SANDBOX_TTL_MINUTES=60
#EXCHANGE_SANDBOX_PER_DAY=20
++# The first-party catalogue: Design's built-in themes and token files, Live's OBS scenes and alerts, and Games'
++# public worlds. The job runs at boot and every interval; EXCHANGE_FIRST_PARTY=0 turns it off everywhere.
++#EXCHANGE_FIRST_PARTY_INTERVAL_MS=21600000
++#EXCHANGE_FIRST_PARTY_TIMEOUT_MS=8000
++#OV_DESIGN_URL=https://openvibe.design
++#OV_LIVE_URL=https://openvibe.live
++#OV_GAMES_URL=https://openvibe.games
++
# OpenVibe.Events: account export/deletion deliveries to POST /internal/events. The secret signs a delivery
# (comma-separated for rotation, 32+ characters each; unset makes the route answer 503).
EXCHANGE_EVENTS_SECRET=
ddiff --git a/README.md b/README.md
iindex d39ca2d..c7124a7 100644
---- a/README.md
++++ b/README.md
@@@ -90,7 +90,7 @@ JWKS. A write made with the session cookie must come from `https://openvibe.exch
| Route | What it does |
|---|---|
--| `GET /listings?limit=&cursor=&mode=&kind=&q=` | public listings, newest first; opaque keyset cursor; `Cache-Control: public, max-age=60` |
++| `GET /listings?limit=&cursor=&mode=&kind=&q=` | public listings, interleaved by kind, newest first inside each kind; opaque keyset cursor; `Cache-Control: public, max-age=60` |
| `GET /listings/:slug` | one listing with passport, manifest, licence, tests and counters; a draft/hidden/closed one is 404 unless it is yours or you are staff |
| `GET /listings/:slug/passport` | the exact passport, manifest, licence, version and tests an installer accepts |
| `GET /listings/:slug/tests` | the conformance tests for the version the listing names |
@@@ -159,6 +159,13 @@ each response's cacheability.
`npm run seed:proof` creates the first proof through the same store functions as the API: a CC0-1.0 Website template
listed, tried in the sandbox, installed with its passport and revoked. It is idempotent.
++`npm run seed:first-party` (and the boot job, every `EXCHANGE_FIRST_PARTY_INTERVAL_MS`) lists the free things the
++network makes: Design's built-in themes, each with its token files as downloads on one listing, the OBS scenes and
++alerts overlay on Live, and public Games worlds. Each carries its real home URL and is marked official with no person
++as owner; `source`/`source_id` is the idempotency key, so a restart or a changed source updates the same row. A source
++that stops answering never empties the catalogue; one that no longer lists an item retires it. The list is
++interleaved by kind, so a page cannot be filled by one kind.
++
## Grants it will need
v1 is written so each of these is a single call site; none is used today. They are external grants on other
ddiff --git a/migrations/0002_first_party.sql b/migrations/0002_first_party.sql
nnew file mode 100644
iindex 0000000..8e0d39a
---- /dev/null
++++ b/migrations/0002_first_party.sql
@@@ -0,0 +1,23 @@
++-- phase: expand
++-- OpenVibe.Exchange: first-party listings of the free things the network makes (Design themes and token files, the
++-- Live OBS scenes and alerts, public Games worlds). They are listed by the network, not by a person: owner is NULL,
++-- official is true, and a source/source_id pair makes the periodic refresh idempotent. home_url is where the thing
++-- really lives, so a listing is never a dead end.
++--
++-- The official flag may only be set on a network-owned row. A person's row is never official, which is what stops a
++-- seeded listing from wearing a maker's name.
++
++ALTER TABLE exchange_listings ADD COLUMN home_url text;
++ALTER TABLE exchange_listings ADD COLUMN official boolean NOT NULL DEFAULT false;
++ALTER TABLE exchange_listings ADD COLUMN source text COLLATE "C";
++ALTER TABLE exchange_listings ADD COLUMN source_id text COLLATE "C";
++
++ALTER TABLE exchange_listings ADD CONSTRAINT exchange_listings_home_url
++ CHECK (home_url IS NULL OR home_url ~ '^https://');
++ALTER TABLE exchange_listings ADD CONSTRAINT exchange_listings_official
++ CHECK (NOT official OR owner IS NULL);
++
++CREATE UNIQUE INDEX exchange_listings_source_key
++ ON exchange_listings (source, source_id)
++ WHERE source IS NOT NULL AND source_id IS NOT NULL;
++CREATE INDEX exchange_listings_by_source ON exchange_listings (source, source_id);
ddiff --git a/migrations/0003_listing_files.sql b/migrations/0003_listing_files.sql
nnew file mode 100644
iindex 0000000..343196e
---- /dev/null
++++ b/migrations/0003_listing_files.sql
@@@ -0,0 +1,9 @@
++-- phase: expand
++-- A theme is one listing with its token exports as downloads, not one listing per export. files is an array of
++-- { format, label, url }, public metadata: the format id, the human label and the https file itself. Existing rows
++-- get the empty array, and a user's listing may carry its own downloads through the same field.
++
++ALTER TABLE exchange_listings ADD COLUMN files jsonb NOT NULL DEFAULT '[]'::jsonb;
++
++ALTER TABLE exchange_listings ADD CONSTRAINT exchange_listings_files
++ CHECK (jsonb_typeof(files) = 'array');
ddiff --git a/package.json b/package.json
iindex a78ee1b..5e910b4 100644
---- a/package.json
++++ b/package.json
@@@ -8,6 +8,7 @@
"start": "node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
"seed:proof": "node scripts/seed-proof.js",
++ "seed:first-party": "node scripts/seed-first-party.js",
"test": "node test/run.js",
"test:pg": "EXCHANGE_TEST_STORE=pg node test/run.js"
},
ddiff --git a/public/css/exchange.css b/public/css/exchange.css
iindex 9ff2e14..ddc0456 100644
---- a/public/css/exchange.css
++++ b/public/css/exchange.css
@@@ -58,7 +58,7 @@ html, body { margin: 0; background: var(--bg-primary, #0a0f1c); color: var(--tex
.ex-filters a { padding: 6px 12px; border-radius: 999px; border: 1px solid var(--border, #1f2d47); font-size: .86rem; font-weight: 700; text-decoration: none; color: var(--text-secondary, #96a7c2); }
.ex-filters a[aria-current] { background: var(--room-fill); color: var(--room-on); border-color: transparent; }
.ex-inline { margin: 0; display: inline; }
--.ex-ok, .ex-status-active, .ex-verified { color: var(--success-text, #22c55e); font-weight: 800; }
++.ex-ok, .ex-status-active, .ex-verified, .ex-official { color: var(--success-text, #22c55e); font-weight: 800; }
.ex-status-revoked, .ex-unverified { color: var(--text-muted, #7386a3); font-weight: 800; }
.ex-status, .ex-kind, .ex-mode, .ex-license, .ex-version { display: inline-flex; align-items: center; padding: 1px 9px; border-radius: 999px; font-size: .75rem; font-weight: 800; background: color-mix(in srgb, var(--text-muted, #7386a3) 14%, transparent); color: var(--text-secondary, #96a7c2); }
.ex-kind, .ex-mode { background: color-mix(in srgb, var(--room-accent) 16%, transparent); color: var(--room-strong); }
@@@ -107,6 +107,12 @@ html.ov-motion .ex-motif-passport { animation: ex-stamp .5s cubic-bezier(.2,.9,.
.ex-none { color: var(--text-muted, #7386a3); font-style: italic; }
.ex-manifest { margin: 0; padding: 14px; border-radius: 12px; background: var(--bg-primary, #0a0f1c); border: 1px solid var(--border, #1f2d47); font: 500 .84rem/1.5 ui-monospace, SFMono-Regular, Menlo, monospace; white-space: pre-wrap; overflow-wrap: anywhere; max-height: 360px; overflow: auto; }
++.ex-file-list { list-style: none; margin: 0; padding: 0; display: grid; grid-template-columns: repeat(auto-fit, minmax(min(100%, 220px), 1fr)); gap: 8px; }
++.ex-file-list li { min-width: 0; }
++.ex-file-list a { display: flex; flex-wrap: wrap; align-items: baseline; justify-content: space-between; gap: 4px 12px; padding: 11px 13px; border-radius: 12px; background: var(--bg-card, #0f1729); border: 1px solid var(--border, #1f2d47); text-decoration: none; }
++.ex-file-list b { font-weight: 700; overflow-wrap: anywhere; }
++.ex-file-list span { font: 600 .76rem/1 ui-monospace, SFMono-Regular, Menlo, monospace; color: var(--text-muted, #7386a3); }
++
/* Checks and conformance tests */
.ex-checks, .ex-tests { list-style: none; margin: 0; padding: 0; display: grid; gap: 6px; }
.ex-check, .ex-test { display: flex; flex-wrap: wrap; align-items: baseline; gap: 4px 10px; padding: 9px 12px; border-radius: 10px; background: var(--bg-card, #0f1729); border: 1px solid var(--border, #1f2d47); font-size: .88rem; }
ddiff --git a/scripts/seed-first-party.js b/scripts/seed-first-party.js
nnew file mode 100644
iindex 0000000..c6ee47a
---- /dev/null
++++ b/scripts/seed-first-party.js
@@@ -0,0 +1,44 @@
++#!/usr/bin/env node
++'use strict';
++
++/**
++ * Seed or refresh the first-party catalogue now (the service also runs it at boot and every
++ * EXCHANGE_FIRST_PARTY_INTERVAL_MS):
++ *
++ * node scripts/seed-first-party.js [design,obs,games]
++ *
++ * It uses the same store functions and the same sources as the job, prints counts only, and never creates a person's
++ * listing. Re-running is safe: source/source_id is the idempotency key.
++ */
++const { load } = require('../server/config');
++const { openStore } = require('../server/db');
++const { createStore } = require('../server/store');
++const { createSearchIndex } = require('../server/search');
++const { createFirstParty, SOURCES } = require('../server/jobs/first-party');
++
++(async () => {
++ const config = load(process.env);
++ if (config.drill) throw new Error('refusing to seed under EXCHANGE_DRILL');
++ const s = await openStore(config);
++ try {
++ const search = createSearchIndex({ config, s });
++ const entries = createStore(s, { search, config });
++ const job = createFirstParty({ config, s, entries, now: s.now });
++ const asked = process.argv.slice(2).flatMap((x) => x.split(',')).filter(Boolean);
++ const sources = asked.length ? asked : SOURCES;
++ const out = await job.runOnce({ sources });
++ const lines = [];
++ for (const [source, result] of Object.entries(out.sources || {})) {
++ if (result && result.themes) {
++ lines.push(`[seed] ${source}.themes: ${result.themes.ok ? 'ok' : 'failed'} imported=${result.themes.imported || 0} changed=${result.themes.changed || 0} retired=${result.themes.retired || 0} removed_token_listings=${result.removed_tokens || 0}`);
++ } else {
++ lines.push(`[seed] ${source}: ${result && result.ok ? 'ok' : 'failed'} imported=${(result && result.imported) || 0} changed=${(result && result.changed) || 0} retired=${(result && result.retired) || 0}${result && result.error ? ` error=${result.error}` : ''}`);
++ }
++ }
++ for (const line of lines) console.log(line);
++ const ok = Object.values(out.sources || {}).every((r) => !r || r.ok !== false) && Object.values(out.sources || {}).every((r) => !r || r.themes === undefined || r.themes.ok);
++ process.exitCode = ok ? 0 : 1;
++ } finally {
++ await s.close();
++ }
++})().catch((err) => { console.error(`[seed] ${err && err.message ? err.message : err}`); process.exitCode = 1; });
ddiff --git a/server/app.js b/server/app.js
iindex 2e66782..641c81b 100644
---- a/server/app.js
++++ b/server/app.js
@@@ -11,7 +11,9 @@
* /api/health, /api/ready, /release.json, /metrics (loopback only)
* /internal/events account export/deletion (loopback only)
*
-- * There are no jobs in v1: sandbox trials run on request and expire by timestamp. The Search relay is not wired.
++ * Sandbox trials run on request and expire by timestamp; the Search relay is not wired. The only background job is
++ * the first-party catalogue refresh (server/jobs/first-party.js), started by server/index.js and off under
++ * EXCHANGE_DRILL.
*/
const path = require('path');
const express = require('express');
@@@ -35,6 +37,7 @@ const { raw } = require('./render/html');
const accountDataLib = require('./identity/account-data');
const { createStore } = require('./store');
const { createSearchIndex } = require('./search');
++const { createFirstParty } = require('./jobs/first-party');
const { createNetworkSender } = require('openvibe-sdk/account-data');
const PUBLIC_DIR = path.join(__dirname, '..', 'public');
@@@ -58,6 +61,7 @@ async function createApp(opts = {}) {
const search = createSearchIndex({ config, s });
const entries = createStore(s, { search, config });
++ const firstParty = createFirstParty({ config, s, entries, fetchImpl, log, now: s.now });
// Account export and deletion (ADR-033). The sender posts to Network's internal routes with this service's own
// client-credentials token; a test injects a stand-in through opts.accountSend.
@@@ -66,7 +70,7 @@ async function createApp(opts = {}) {
? createNetworkSender({ networkInternalUrl: config.networkInternalUrl, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, fetch: fetchImpl })
: async () => { throw new Error('OV_OAUTH_CLIENT_SECRET is not set: Exchange cannot answer account events'); });
-- const ctx = { config, s, keys, sso, principal, search, entries, accountData, accountSend, log };
++ const ctx = { config, s, keys, sso, principal, search, entries, firstParty, accountData, accountSend, log };
const app = express();
app.disable('x-powered-by');
ddiff --git a/server/config.js b/server/config.js
iindex d4bcae8..2e66ac3 100644
---- a/server/config.js
++++ b/server/config.js
@@@ -67,6 +67,17 @@ function load(env = process.env) {
ttlMinutes: Math.min(240, Math.max(5, int(env.EXCHANGE_SANDBOX_TTL_MINUTES, 60))),
perDay: Math.min(100, Math.max(1, int(env.EXCHANGE_SANDBOX_PER_DAY, 20))),
},
++
++ // The first-party catalogue (server/jobs/first-party.js): Design's built-in themes and token files, Live's OBS
++ // scenes and alerts, and Games' public worlds. Off under EXCHANGE_DRILL and when EXCHANGE_FIRST_PARTY=0.
++ firstParty: {
++ enabled: env.EXCHANGE_FIRST_PARTY !== '0' && env.EXCHANGE_FIRST_PARTY !== 'false',
++ intervalMs: Math.max(60_000, int(env.EXCHANGE_FIRST_PARTY_INTERVAL_MS, 6 * 60 * 60_000)),
++ timeoutMs: Math.max(500, int(env.EXCHANGE_FIRST_PARTY_TIMEOUT_MS, 8000)),
++ designUrl: trim(env.OV_DESIGN_URL || 'https://openvibe.design'),
++ liveUrl: trim(env.OV_LIVE_URL || 'https://openvibe.live'),
++ gamesUrl: trim(env.OV_GAMES_URL || 'https://openvibe.games'),
++ },
};
}
ddiff --git a/server/http/api.js b/server/http/api.js
iindex a6e84fb..4467557 100644
---- a/server/http/api.js
++++ b/server/http/api.js
@@@ -3,7 +3,7 @@
/**
* /api/v1 — OpenVibe.Exchange's JSON API.
*
-- * GET /listings?limit=&cursor=&mode=&kind=&q= public listings, newest first, cursor-paged
++ * GET /listings?limit=&cursor=&mode=&kind=&q= public listings, interleaved by kind, newest first inside each kind; cursor-paged
* GET /listings/:slug one listing with its passport, tests and licence
* GET /listings/:slug/passport the exact passport and manifest an installer accepts
* GET /listings/:slug/tests the conformance tests for this version
ddiff --git a/server/index.js b/server/index.js
iindex 239cdc7..abe99f5 100644
---- a/server/index.js
++++ b/server/index.js
@@@ -19,6 +19,7 @@ function createLifecycle({ server, ctx, exit, signals, timers = [], extra = [] }
close: [
() => { for (const t of timers) clearInterval(t); },
() => ctx.keys.client.stop(),
++ () => ctx.firstParty.stop(),
async () => { clearInterval(ctx.telemetryTick); if (ctx.retention) await ctx.retention.stop(); if (ctx.telemetry) await ctx.telemetry.stop(); },
() => ctx.s.close(),
...extra,
@@@ -44,6 +45,7 @@ async function start() {
ctx.telemetryTick = setInterval(() => { ctx.telemetry.flush().catch(() => {}); }, 60_000);
ctx.telemetryTick.unref();
ctx.keys.client.start();
++ ctx.firstParty.start();
// The two account topics (export_requested, deleted) at OpenVibe.Events: idempotent; off without the URL, the
// delivery secret and the client secret. The consumer itself is mounted in server/app.js.
ddiff --git a/server/jobs/first-party.js b/server/jobs/first-party.js
nnew file mode 100644
iindex 0000000..f4622dc
---- /dev/null
++++ b/server/jobs/first-party.js
@@@ -0,0 +1,298 @@
++'use strict';
++
++/**
++ * OpenVibe.Exchange's first-party catalogue. The network makes real, free things and this job lists them where people
++ * can find them: Design's built-in themes and their token files, the OBS scenes and alerts overlay on Live, and the
++ * public creator worlds on Games. Each listing keeps a home_url to where the thing really lives, is owned by the
++ * network (owner NULL) and is marked official, with the real maker named when the source says who made it.
++ *
++ * There are no invented listings and no invented people. A source that stops answering is not allowed to empty the
++ * catalogue; a source that answers with items and no longer lists one retires that item. source/source_id is the
++ * idempotency key, so a restart or an overlapping run updates the same row.
++ *
++ * Sources that were checked and do not exist yet (so nothing is listed):
++ * - Design icon kits: Design publishes themes and token exports; it has no icon-kit API.
++ * - Codes agent templates: Codes publishes a harness/agent catalog, not templates.
++ * When either gains one, add a reader here and a source name to SOURCES.
++ */
++const SOURCES = Object.freeze(['design', 'obs', 'games']);
++const DESIGN_LICENSE = 'AGPL-3.0-only';
++const DESIGN_LICENSE_URL = 'https://github.com/OpenVibers/OpenVibe.Design/blob/main/LICENSE';
++const LIVE_LICENSE_URL = 'https://github.com/OpenVibers/OpenVibe.Live/blob/main/LICENSE';
++const MAX_TITLE = 120;
++const MAX_SUMMARY = 600;
++const MAX_DESCRIPTION = 4000;
++
++const plain = (v, max) => String(v == null ? '' : v).replace(/\s+/g, ' ').trim().slice(0, max);
++const base = (url) => String(url || '').replace(/\/+$/, '');
++
++function createFirstParty({ config, s, entries, fetchImpl = globalThis.fetch, log = console, now = s.now }) {
++ const stats = { last: null, running: false, started: false };
++ let timer = null;
++ let timeout = null;
++
++ async function getJson(url, { fetch = fetchImpl } = {}) {
++ const res = await fetch(url, {
++ headers: { Accept: 'application/json', 'User-Agent': 'OpenVibe.Exchange (+https://openvibe.exchange)' },
++ signal: AbortSignal.timeout(config.firstParty.timeoutMs),
++ });
++ if (!res.ok) {
++ const err = new Error(`${new URL(url).host} answered ${res.status}`);
++ err.status = res.status;
++ throw err;
++ }
++ const body = await res.json().catch(() => null);
++ if (!body || typeof body !== 'object') throw new Error(`${new URL(url).host} answered without JSON`);
++ return body;
++ }
++
++ function contact(origin) {
++ return { reads: [], writes: [], spends: [], contacts: [origin] };
++ }
++
++ function record({ source, sourceId, creator, body, homeUrl, runtime }) {
++ const title = plain(body.title, MAX_TITLE);
++ const summary = plain(body.summary, MAX_SUMMARY);
++ if (!source || !sourceId || title.length < 3 || summary.length < 3 || !/^https:\/\//.test(homeUrl || '')) return null;
++ return {
++ source,
++ sourceId,
++ creator: plain(creator, 80) || 'OpenVibe',
++ body: {
++ title,
++ kind: body.kind,
++ mode: 'free',
++ summary,
++ description: plain(body.description, MAX_DESCRIPTION) || null,
++ license: body.license,
++ license_url: body.license_url || null,
++ home_url: homeUrl,
++ version: '1.0.0',
++ manifest: { name: title, entry: homeUrl.slice(0, 200), runtime: runtime || null },
++ passport: body.passport || contact(new URL(homeUrl).origin),
++ files: Array.isArray(body.files) ? body.files : [],
++ },
++ };
++ }
++
++ async function designAll(fetch) {
++ const themesBody = await getJson(`${base(config.firstParty.designUrl)}/api/themes`, { fetch });
++ const all = Array.isArray(themesBody.themes) ? themesBody.themes : [];
++ // First-party means the built-in themes. A community theme belongs to the person who made it.
++ const themes = all.filter((t) => t && t.slug && (t.is_builtin === 1 || t.is_builtin === true));
++
++ // One export list for every theme: the theme's own listing carries the files, not one listing per file.
++ let targets = [];
++ let targetsOk = false;
++ try {
++ const targetsBody = await getJson(`${base(config.firstParty.designUrl)}/api/v1/tokens`, { fetch });
++ targets = (Array.isArray(targetsBody.targets) ? targetsBody.targets : [])
++ .filter((t) => t && typeof t.id === 'string' && t.id)
++ .map((t) => ({ id: plain(t.id, 40), label: plain(t.label || t.id, 80), lang: plain(t.lang, 40) }));
++ targetsOk = true;
++ } catch (err) {
++ log.warn(`[OpenVibe.Exchange] Design token exports: ${(err && err.message) || err}`);
++ }
++ // A token endpoint that is briefly down must not wipe the downloads already on a theme listing.
++ const priorFiles = new Map();
++ if (!targetsOk) {
++ const rows = await s.db.many("SELECT source_id, files FROM exchange_listings WHERE source = 'design.theme'");
++ for (const row of rows) priorFiles.set(row.source_id, entries.wireListing(row).files);
++ }
++
++ const themeRecords = [];
++ for (const theme of themes) {
++ const slug = plain(theme.slug, 80);
++ const name = plain(theme.name || theme.slug, 90);
++ const home = `${base(config.firstParty.designUrl)}/studio?theme=${encodeURIComponent(slug)}`;
++ const files = targetsOk ? targets.map((target) => ({
++ format: target.id,
++ label: target.label,
++ url: `${base(config.firstParty.designUrl)}/api/v1/tokens/${encodeURIComponent(slug)}?format=${encodeURIComponent(target.id)}`,
++ })) : (priorFiles.get(slug) || []);
++ const rec = record({
++ source: 'design.theme',
++ sourceId: slug,
++ creator: 'OpenVibe',
++ homeUrl: home,
++ runtime: 'web',
++ body: {
++ title: `${name} theme`,
++ kind: 'theme',
++ summary: plain(theme.description, MAX_SUMMARY) || `A free ${theme.mode === 'light' ? 'light' : 'dark'} theme from OpenVibe.Design.`,
++ description: [plain(theme.description, 1000), `Open it in the studio to wear it on every OpenVibe site: ${home}`].filter(Boolean).join('\n\n'),
++ license: DESIGN_LICENSE,
++ license_url: DESIGN_LICENSE_URL,
++ passport: contact('https://openvibe.design'),
++ files,
++ },
++ });
++ if (rec) themeRecords.push(rec);
++ }
++ return { themeRecords };
++ }
++
++ function obsRecords() {
++ const live = base(config.firstParty.liveUrl);
++ const scenes = [
++ ['starting', 'Starting soon scene for OBS', 'A full-screen Starting soon scene in your theme, with the mark\'s intro and an optional countdown.'],
++ ['brb', 'Be right back scene for OBS', 'A Be right back scene in your theme, with the mark breathing while you are away.'],
++ ['ending', 'Thanks for watching scene for OBS', 'A Thanks for watching scene in your theme, with an optional next-stream line.'],
++ ];
++ const out = [];
++ for (const [kind, title, description] of scenes) {
++ const home = `${live}/obs/scene/${kind}/openvibe`;
++ const rec = record({
++ source: 'live.obs',
++ sourceId: `scene:${kind}`,
++ creator: 'OpenVibe',
++ homeUrl: home,
++ runtime: 'html',
++ body: {
++ title, kind: 'template',
++ summary: 'A free OBS browser-source scene from OpenVibe.Live: add it as a source, size it 1920 × 1080, and it wears your theme.',
++ description: `${description}\n\nAdd the URL as an OBS browser source at 1920 × 1080. Replace the channel in the URL with your own Live username.\n\n${home}`,
++ license: DESIGN_LICENSE,
++ license_url: LIVE_LICENSE_URL,
++ passport: contact(live),
++ },
++ });
++ if (rec) out.push(rec);
++ }
++ const alerts = `${live}/obs/alerts/openvibe?demo=1`;
++ const alert = record({
++ source: 'live.obs',
++ sourceId: 'alerts',
++ creator: 'OpenVibe',
++ homeUrl: alerts,
++ runtime: 'html',
++ body: {
++ title: 'Alerts overlay for OBS',
++ kind: 'template',
++ summary: 'A free alerts overlay from OpenVibe.Live: follows, tips, subs, raids and goals spring in, glint and leave, in your theme.',
++ description: `Add the URL as an OBS browser source. The demo query lines the alerts up so you can style the source; drop it for the real feed, and replace the channel with your own Live username.\n\n${alerts}`,
++ license: DESIGN_LICENSE,
++ license_url: LIVE_LICENSE_URL,
++ passport: contact(live),
++ },
++ });
++ if (alert) out.push(alert);
++ return out;
++ }
++
++ async function gamesRecords(fetch) {
++ const out = [];
++ const seen = new Set();
++ for (let page = 1; page <= 10; page++) {
++ const body = await getJson(`${base(config.firstParty.gamesUrl)}/api/worlds?page=${page}`, { fetch });
++ const worlds = Array.isArray(body.worlds) ? body.worlds : [];
++ for (const world of worlds) {
++ if (!world || !world.id || seen.has(String(world.id))) continue;
++ seen.add(String(world.id));
++ const home = `${base(config.firstParty.gamesUrl)}${String(world.play_url || `/play?world=${encodeURIComponent(world.id)}`)}`;
++ const creator = plain(world.owner, 80) || 'a maker';
++ const rec = record({
++ source: 'games.world',
++ sourceId: plain(world.id, 64),
++ creator,
++ homeUrl: home,
++ runtime: 'game',
++ body: {
++ title: plain(world.title, MAX_TITLE) || 'A creator world on OpenVibe.Games',
++ kind: 'game_asset',
++ summary: plain(world.summary, MAX_SUMMARY) || 'A free creator world published on OpenVibe.Games.',
++ description: `A creator world on OpenVibe.Games${world.owner ? ` by ${plain(world.owner, 80)}` : ''}. Free to play in the browser.\n\n${home}`,
++ // The world is free to play; the map and its assets stay the maker's. This is a custom SPDX
++ // licence reference, not a claim that the maker released them under AGPL.
++ license: 'LicenseRef-OpenVibe-Games',
++ license_url: null,
++ passport: contact('https://openvibe.games'),
++ },
++ });
++ if (rec) out.push(rec);
++ }
++ if (!body.more) break;
++ }
++ return out;
++ }
++
++ async function apply(source, records) {
++ const seen = [];
++ let changed = 0;
++ await s.tx(async (t) => {
++ for (const rec of records) {
++ const out = await entries.upsertFirstParty(rec, t);
++ if (!out.ok) {
++ log.warn(`[OpenVibe.Exchange] ${source} ${rec.sourceId}: ${out.message || 'refused'}`);
++ continue;
++ }
++ seen.push(rec.sourceId);
++ if (!out.unchanged) changed++;
++ }
++ });
++ // An empty answer is treated as a quiet source, not as "delete the catalogue". Only a read that listed
++ // something may retire what it no longer lists.
++ const retired = seen.length ? (await entries.retireFirstParty({ source, seen, now: now() })).retired : 0;
++ return { ok: true, imported: seen.length, changed, retired };
++ }
++
++ async function runOnce({ sources = SOURCES, fetchImpl: fetch = fetchImpl } = {}) {
++ if (stats.running) return { busy: true, at: new Date(now()).toISOString(), sources: {} };
++ stats.running = true;
++ const out = {};
++ try {
++ for (const source of sources) {
++ try {
++ if (source === 'design') {
++ const { themeRecords } = await designAll(fetch);
++ // The old job made one listing per token file. They were never meant to be separate
++ // listings, so remove exactly those official rows (a person's row is never touched).
++ const removed = await entries.removeFirstPartySource('design.tokens');
++ out.design = {
++ themes: await apply('design.theme', themeRecords),
++ removed_tokens: removed.removed,
++ };
++ } else if (source === 'obs') {
++ out.obs = await apply('live.obs', obsRecords());
++ } else if (source === 'games') {
++ out.games = await apply('games.world', await gamesRecords(fetch));
++ }
++ } catch (err) {
++ out[source] = { ok: false, imported: 0, changed: 0, retired: 0, error: (err && err.message) || String(err) };
++ log.warn(`[OpenVibe.Exchange] first-party ${source}: ${(err && err.message) || err}`);
++ }
++ }
++ stats.last = { at: new Date(now()).toISOString(), sources: out };
++ return stats.last;
++ } finally {
++ stats.running = false;
++ }
++ }
++
++ function start() {
++ if (!config.firstParty.enabled || config.drill) return false;
++ if (stats.started) return true;
++ stats.started = true;
++ const tick = async () => {
++ try { await runOnce(); } catch (err) { log.warn('[OpenVibe.Exchange] first-party refresh:', (err && err.message) || err); }
++ };
++ timeout = setTimeout(tick, 15_000);
++ if (timeout.unref) timeout.unref();
++ timer = setInterval(tick, config.firstParty.intervalMs);
++ if (timer.unref) timer.unref();
++ log.log(`[OpenVibe.Exchange] first-party refresh every ${Math.round(config.firstParty.intervalMs / 60_000)} min: ${SOURCES.join(', ')}`);
++ return true;
++ }
++
++ function stop() {
++ if (timeout) clearTimeout(timeout);
++ if (timer) clearInterval(timer);
++ timeout = null;
++ timer = null;
++ stats.started = false;
++ }
++
++ return { runOnce, start, stop, stats, sources: SOURCES, _designAll: designAll, _obsRecords: obsRecords, _gamesRecords: gamesRecords };
++}
++
++module.exports = { createFirstParty, SOURCES };
ddiff --git a/server/search.js b/server/search.js
iindex 7860cbd..9c15eb2 100644
---- a/server/search.js
++++ b/server/search.js
@@@ -36,6 +36,7 @@ function documentOf(listing, { baseUrl = 'https://openvibe.exchange' } = {}) {
writes.length ? `Writes: ${writes.join(', ')}` : '',
contacts.length ? `Contacts: ${contacts.join(', ')}` : '',
`Licence: ${listing.license} · version ${listing.version} · ${listing.mode}`,
++ listing.home_url ? `Home: ${listing.home_url}` : '',
].filter(Boolean).join('\n').slice(0, 48_000);
return {
owner: OWNER,
@@@ -53,6 +54,7 @@ function documentOf(listing, { baseUrl = 'https://openvibe.exchange' } = {}) {
mode: listing.mode,
license: listing.license,
creator: listing.creator,
++ official: Boolean(listing.official),
can_read: reads,
can_write: writes,
can_contact: contacts,
ddiff --git a/server/store.js b/server/store.js
iindex ada8cd6..3177483 100644
---- a/server/store.js
++++ b/server/store.js
@@@ -56,6 +56,15 @@ function wireManifest(v) {
return { name: m.name || '', entry: m.entry || '', runtime: m.runtime || null };
}
++/** The downloads a listing carries: { format, label, url } only, in the order the maker listed them. */
++function wireFiles(v) {
++ const list = json(v, []);
++ if (!Array.isArray(list)) return [];
++ return list
++ .filter((f) => f && typeof f === 'object' && typeof f.format === 'string' && typeof f.url === 'string')
++ .map((f) => ({ format: String(f.format).slice(0, 40), label: String(f.label || f.format).slice(0, 80), url: String(f.url).slice(0, 300) }));
++}
++
function wireTest(row) {
if (!row) return null;
return { id: Number(row.id), name: row.name, status: row.status, detail: row.detail == null ? null : row.detail, ran_at: iso(row.ran_at) };
@@@ -80,6 +89,9 @@ function wireListing(row, { tests = null, includeOwner = false, summaryTests = f
revision: Number(row.revision) || 1,
passport: wirePassport(row.passport),
manifest: wireManifest(row.manifest),
++ files: wireFiles(row.files),
++ home_url: row.home_url == null ? null : row.home_url,
++ official: Boolean(row.official),
listed_at: iso(row.listed_at),
closed_at: iso(row.closed_at),
closed_reason: row.closed_reason == null ? null : row.closed_reason,
@@@ -204,10 +216,34 @@ function normalizeManifest(input, version) {
return { ok: true, value: { name, entry, runtime: runtime || null, version } };
}
++/** A listing's downloads: at most 20 { format, label, url } entries with unique formats. */
++function normalizeFiles(input) {
++ if (input == null) return { ok: true, value: [] };
++ if (!Array.isArray(input)) return refusal(400, 'exchange.invalid', 'files must be an array of downloads', 'files');
++ if (input.length > 20) return refusal(400, 'exchange.invalid', 'a listing may carry at most 20 download files', 'files');
++ const out = [];
++ const seen = new Set();
++ for (const entry of input) {
++ if (!entry || typeof entry !== 'object' || Array.isArray(entry)) return refusal(400, 'exchange.invalid', 'each file must be { format, label, url }', 'files');
++ const unknown = Object.keys(entry).filter((k) => !['format', 'label', 'url'].includes(k));
++ if (unknown.length) return refusal(400, 'exchange.invalid', `Unknown file field: ${unknown[0]}`, 'files');
++ const format = cleanText(entry.format, 40);
++ const label = cleanText(entry.label, 80);
++ const url = cleanText(entry.url, 300);
++ if (!/^[a-z0-9][a-z0-9._-]{0,39}$/.test(format)) return refusal(400, 'exchange.invalid', 'file.format must be a lower-case id such as css or tokens.json', 'files');
++ if (label.length < 1 || label.length > 80) return refusal(400, 'exchange.invalid', 'file.label must be 1 to 80 characters', 'files');
++ if (!/^https:\/\/[^\s"'<>]{1,300}$/.test(url)) return refusal(400, 'exchange.invalid', 'file.url must be an https URL', 'files');
++ if (seen.has(format)) return refusal(400, 'exchange.invalid', `Two files share the format ${format}`, 'files');
++ seen.add(format);
++ out.push({ format, label, url });
++ }
++ return { ok: true, value: out };
++}
++
/** Validate a new listing body. Returns { ok, value } or a refusal; the page form and the API share it. */
function normalizeListing(body) {
const b = body && typeof body === 'object' ? body : {};
-- const allowed = ['title', 'kind', 'mode', 'summary', 'description', 'license', 'license_url', 'version', 'wants', 'manifest', 'passport'];
++ const allowed = ['title', 'kind', 'mode', 'summary', 'description', 'license', 'license_url', 'version', 'wants', 'manifest', 'passport', 'home_url', 'files'];
const unknown = Object.keys(b).filter((k) => !allowed.includes(k));
if (unknown.length) return refusal(400, 'exchange.invalid', `Unknown field: ${unknown[0]}`, unknown[0]);
const title = cleanText(b.title, 120);
@@@ -221,6 +257,8 @@ function normalizeListing(body) {
if (!LICENSE_RE.test(license)) return refusal(400, 'exchange.invalid', 'license must be an SPDX identifier such as CC0-1.0, MIT or AGPL-3.0-only', 'license');
const licenseUrl = cleanText(b.license_url, 300);
if (licenseUrl && !/^https:\/\/[^\s"'<>]{1,300}$/.test(licenseUrl)) return refusal(400, 'exchange.invalid', 'license_url must be an https URL', 'license_url');
++ const homeUrl = cleanText(b.home_url, 300);
++ if (homeUrl && !/^https:\/\/[^\s"'<>]{1,300}$/.test(homeUrl)) return refusal(400, 'exchange.invalid', 'home_url must be an https URL', 'home_url');
const version = cleanText(b.version, 32);
if (!VERSION_RE.test(version)) return refusal(400, 'exchange.invalid', 'version must be a semantic version such as 1.0.0', 'version');
const wants = cleanText(b.wants, 300);
@@@ -230,17 +268,42 @@ function normalizeListing(body) {
if (!manifest.ok) return manifest;
const passport = normalizePassport(b.passport);
if (!passport.ok) return passport;
++ const files = normalizeFiles(b.files);
++ if (!files.ok) return files;
const hit = refusals(title, summary, description, wants);
if (hit) return hit;
return {
ok: true,
value: {
title, kind: b.kind, mode: b.mode, summary, description: description || null, license,
-- license_url: licenseUrl || null, version, wants: wants || null, manifest: manifest.value, passport: passport.value,
++ license_url: licenseUrl || null, home_url: homeUrl || null, files: files.value, version, wants: wants || null,
++ manifest: manifest.value, passport: passport.value,
},
};
}
++/** Does a first-party row still say the same thing as the source? */
++function firstPartyChanged(row, v, creator) {
++ const manifest = wireManifest(row.manifest);
++ const passport = wirePassport(row.passport);
++ return row.title !== v.title
++ || row.kind !== v.kind
++ || row.mode !== v.mode
++ || (row.summary || '') !== (v.summary || '')
++ || (row.description || '') !== (v.description || '')
++ || (row.wants || '') !== (v.wants || '')
++ || row.license !== v.license
++ || (row.license_url || '') !== (v.license_url || '')
++ || (row.home_url || '') !== (v.home_url || '')
++ || row.version !== v.version
++ || (row.creator || '') !== creator
++ || manifest.name !== v.manifest.name
++ || manifest.entry !== v.manifest.entry
++ || (manifest.runtime || '') !== (v.manifest.runtime || '')
++ || JSON.stringify(passport) !== JSON.stringify(v.passport)
++ || JSON.stringify(wireFiles(row.files)) !== JSON.stringify(v.files);
++}
++
/** The four conformance tests run for every version. A listing is not public until every one passed. */
function checksFor(v) {
return [
@@@ -311,12 +374,12 @@ function createStore(s, { search, config }) {
const slug = await uniqueSlug(slugify(v.title), tx);
const creatorName = cleanText(creator, 80) || 'a maker';
const row = await tx.maybe(
-- `INSERT INTO exchange_listings (id, slug, title, kind, mode, summary, description, wants, license, license_url, version,
++ `INSERT INTO exchange_listings (id, slug, title, kind, mode, summary, description, wants, license, license_url, home_url, files, version,
creator, owner, status, manifest, passport, revision, created_at, updated_at)
-- VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, 'draft', $14::jsonb, $15::jsonb, 1, now(), now())
++ VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12::jsonb, $13, $14, $15, 'draft', $16::jsonb, $17::jsonb, 1, now(), now())
RETURNING *`,
-- [id, slug, v.title, v.kind, v.mode, v.summary, v.description, v.wants, v.license, v.license_url, v.version,
-- creatorName, owner, JSON.stringify(v.manifest), JSON.stringify(v.passport)],
++ [id, slug, v.title, v.kind, v.mode, v.summary, v.description, v.wants, v.license, v.license_url, v.home_url,
++ JSON.stringify(v.files), v.version, creatorName, owner, JSON.stringify(v.manifest), JSON.stringify(v.passport)],
);
const checks = checksFor(v);
for (const c of checks) {
@@@ -330,7 +393,10 @@ function createStore(s, { search, config }) {
return t ? await run(t) : await s.tx(run);
}
-- /** Public listings, newest first. cursor is an opaque (listed_at, id) keyset position. */
++ /**
++ * Public listings, interleaved by kind so one kind cannot fill a page, newest first inside each kind. The cursor
++ * is an opaque (kind_rank, listed_at, id) keyset position from the interleaved order.
++ */
async function list({ limit = 20, cursor = null, mode = null, kind = null, q: text = null } = {}) {
const n = Math.min(50, Math.max(1, Number(limit) || 20));
const args = [];
@@@ -346,18 +412,26 @@ function createStore(s, { search, config }) {
if (cursor) {
try {
const decoded = JSON.parse(Buffer.from(String(cursor), 'base64url').toString('utf8'));
-- if (typeof decoded.t === 'string' && typeof decoded.i === 'string' && !Number.isNaN(Date.parse(decoded.t))) pos = decoded;
++ if (Number.isInteger(decoded.r) && decoded.r > 0 && typeof decoded.t === 'string' && typeof decoded.i === 'string' && !Number.isNaN(Date.parse(decoded.t))) pos = decoded;
} catch { /* refused below */ }
if (!pos) return { error: 'cursor' };
-- args.push(pos.t, pos.i);
-- where += ` AND (listed_at, id) < ($${args.length - 1}::timestamptz, $${args.length})`;
++ }
++ let after = '';
++ if (pos) {
++ args.push(pos.r, pos.t, pos.i);
++ after = ` WHERE (kind_rank > $${args.length - 2} OR (kind_rank = $${args.length - 2} AND (listed_at, id) < ($${args.length - 1}::timestamptz, $${args.length})))`;
}
args.push(n + 1);
const rows = await db.many(
-- `SELECT l.*,
-- (SELECT COUNT(*)::int FROM exchange_listing_tests t WHERE t.listing_id = l.id AND t.version = l.version AND t.status = 'passed') AS test_count,
-- (SELECT COUNT(*)::int FROM exchange_listing_tests t WHERE t.listing_id = l.id AND t.version = l.version AND t.status = 'failed') AS test_failed
-- FROM exchange_listings l WHERE ${where} ORDER BY listed_at DESC, id DESC LIMIT $${args.length}`,
++ `WITH ranked AS (
++ SELECT l.*,
++ ROW_NUMBER() OVER (PARTITION BY l.kind ORDER BY l.listed_at DESC, l.id DESC) AS kind_rank,
++ (SELECT COUNT(*)::int FROM exchange_listing_tests t WHERE t.listing_id = l.id AND t.version = l.version AND t.status = 'passed') AS test_count,
++ (SELECT COUNT(*)::int FROM exchange_listing_tests t WHERE t.listing_id = l.id AND t.version = l.version AND t.status = 'failed') AS test_failed
++ FROM exchange_listings l WHERE ${where}
++ )
++ SELECT * FROM ranked${after}
++ ORDER BY kind_rank ASC, listed_at DESC, id DESC LIMIT $${args.length}`,
args,
);
const hasMore = rows.length > n;
@@@ -365,7 +439,9 @@ function createStore(s, { search, config }) {
const last = page[page.length - 1];
return {
listings: page.map((r) => wireListing(r, { summaryTests: true })),
-- next_cursor: hasMore && last ? Buffer.from(JSON.stringify({ t: iso(last.listed_at), i: last.id })).toString('base64url') : null,
++ next_cursor: hasMore && last
++ ? Buffer.from(JSON.stringify({ r: Number(last.kind_rank), t: iso(last.listed_at), i: last.id })).toString('base64url')
++ : null,
};
}
@@@ -619,6 +695,124 @@ function createStore(s, { search, config }) {
return await createListing({ owner, creator, body });
}
++ /**
++ * One first-party listing, refreshed in place. It is owned by the network (owner NULL) and marked official, so it
++ * can never stand in for a person; source/source_id is its idempotency key. A person's row never enters this
++ * path, and a staff-hidden row is left hidden rather than resurrected.
++ */
++ async function upsertFirstParty({ source, sourceId, body, creator = 'OpenVibe' }, t = null) {
++ const src = cleanText(source, 80);
++ const sid = cleanText(sourceId, 160);
++ if (!src || !sid) return refusal(400, 'exchange.invalid', 'a first-party listing needs a source and a source id');
++ const checked = normalizeListing(body);
++ if (!checked.ok) return checked;
++ const v = checked.value;
++ const run = async (tx) => {
++ const nowIso = new Date(s.now()).toISOString();
++ const creatorName = cleanText(creator, 80) || 'OpenVibe';
++ let row = await tx.maybe('SELECT * FROM exchange_listings WHERE source = $1 AND source_id = $2', [src, sid]);
++ if (!row) {
++ const id = s.newId('exl');
++ const slug = await uniqueSlug(slugify(v.title), tx);
++ row = await tx.maybe(
++ `INSERT INTO exchange_listings (id, slug, title, kind, mode, summary, description, wants, license, license_url, home_url, files, version,
++ creator, owner, status, manifest, passport, revision, official, source, source_id, created_at, updated_at)
++ VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12::jsonb, $13, $14, NULL, 'draft', $15::jsonb, $16::jsonb, 1, true, $17, $18, $19::timestamptz, $19::timestamptz)
++ RETURNING *`,
++ [id, slug, v.title, v.kind, v.mode, v.summary, v.description, v.wants, v.license, v.license_url, v.home_url,
++ JSON.stringify(v.files), v.version, creatorName, JSON.stringify(v.manifest), JSON.stringify(v.passport), src, sid, nowIso],
++ );
++ await ensureTests(tx, row.id, v);
++ const listed = await tx.maybe(
++ `UPDATE exchange_listings SET status = 'listed', listed_at = $2::timestamptz, revision = revision + 1, updated_at = $2::timestamptz
++ WHERE id = $1 RETURNING *`,
++ [row.id, nowIso],
++ );
++ await listingEvent(tx, row.id, null, 'listed', 'first-party seed', 'service:exchange', nowIso);
++ await search.sync(tx, wireListing(listed));
++ return { ok: true, created: true, listing: wireListing(listed, { tests: await testsFor(listed.id, listed.version, tx), includeOwner: true }) };
++ }
++ if (row.status === 'hidden') {
++ return { ok: true, created: false, unchanged: true, skipped: 'hidden', listing: wireListing(row, { tests: await testsFor(row.id, row.version, tx), includeOwner: true }) };
++ }
++ if (row.status === 'listed' && !firstPartyChanged(row, v, creatorName)) {
++ return { ok: true, created: false, unchanged: true, listing: wireListing(row, { tests: await testsFor(row.id, row.version, tx), includeOwner: true }) };
++ }
++ const updated = await tx.maybe(
++ `UPDATE exchange_listings SET title = $2, kind = $3, mode = $4, summary = $5, description = $6, wants = $7,
++ license = $8, license_url = $9, home_url = $10, files = $11::jsonb, version = $12, creator = $13,
++ manifest = $14::jsonb, passport = $15::jsonb, official = true, status = 'listed',
++ listed_at = COALESCE(listed_at, $16::timestamptz), closed_at = NULL, closed_reason = NULL,
++ revision = revision + 1, updated_at = $16::timestamptz
++ WHERE id = $1 RETURNING *`,
++ [row.id, v.title, v.kind, v.mode, v.summary, v.description, v.wants, v.license, v.license_url, v.home_url,
++ JSON.stringify(v.files), v.version, creatorName, JSON.stringify(v.manifest), JSON.stringify(v.passport), nowIso],
++ );
++ await ensureTests(tx, row.id, v);
++ await listingEvent(tx, row.id, row.status, 'listed', 'first-party refresh', 'service:exchange', nowIso);
++ await search.sync(tx, wireListing(updated));
++ return { ok: true, created: false, unchanged: false, listing: wireListing(updated, { tests: await testsFor(updated.id, updated.version, tx), includeOwner: true }) };
++ };
++ return t ? await run(t) : await s.tx(run);
++ }
++
++ /** Retire first-party listings a successful read no longer lists. A hidden row is already out of the catalogue. */
++ async function retireFirstParty({ source, seen = [], now = null }) {
++ const src = cleanText(source, 80);
++ if (!src) return { retired: 0 };
++ const nowIso = new Date(now == null ? s.now() : now).toISOString();
++ return await s.tx(async (tx) => {
++ const rows = await tx.many(
++ `SELECT * FROM exchange_listings
++ WHERE source = $1 AND official = true AND status = 'listed' AND NOT (source_id = ANY($2::text[]))`,
++ [src, seen],
++ );
++ for (const row of rows) {
++ const updated = await tx.maybe(
++ `UPDATE exchange_listings SET status = 'closed', closed_reason = 'withdrawn', closed_at = $2::timestamptz,
++ revision = revision + 1, updated_at = $2::timestamptz WHERE id = $1 RETURNING *`,
++ [row.id, nowIso],
++ );
++ await listingEvent(tx, row.id, row.status, 'closed', 'first-party refresh: the source no longer lists it', 'service:exchange', nowIso);
++ await search.sync(tx, wireListing(updated));
++ }
++ return { retired: rows.length };
++ });
++ }
++
++ /**
++ * Remove every official listing a retired source created. The old one-per-token-file rows are deleted, with a
++ * Search tombstone first, because they were never meant to be separate listings; a person's row is never touched.
++ */
++ async function removeFirstPartySource(source) {
++ const src = cleanText(source, 80);
++ if (!src) return { removed: 0 };
++ return await s.tx(async (tx) => {
++ const rows = await tx.many('SELECT id, revision FROM exchange_listings WHERE source = $1 AND official = true', [src]);
++ for (const row of rows) await search.remove(tx, row.id, Number(row.revision) + 1);
++ await tx.exec('DELETE FROM exchange_listings WHERE source = $1 AND official = true', [src]);
++ return { removed: rows.length };
++ });
++ }
++
++ async function ensureTests(tx, listingId, v) {
++ for (const c of checksFor(v)) {
++ await tx.exec(
++ `INSERT INTO exchange_listing_tests (listing_id, version, name, status, detail, ran_at)
++ VALUES ($1, $2, $3, $4, $5, now())
++ ON CONFLICT (listing_id, version, name) DO UPDATE SET status = EXCLUDED.status, detail = EXCLUDED.detail, ran_at = now()`,
++ [listingId, v.version, c.name, c.status, c.detail],
++ );
++ }
++ }
++
++ async function listingEvent(tx, listingId, from, to, reason, changedBy, at) {
++ await tx.exec(
++ 'INSERT INTO exchange_listing_events (listing_id, from_status, to_status, reason, changed_by, changed_at) VALUES ($1, $2, $3, $4, $5, $6::timestamptz)',
++ [listingId, from, to, reason, changedBy, at],
++ );
++ }
++
return {
wireListing, wireTrial, wireInstall, wireOffer, wirePassport, wireTest,
normalizeListing, normalizePassport, normalizeManifest, checksFor,
@@@ -628,7 +822,7 @@ function createStore(s, { search, config }) {
sandbox, trialById,
hasPassedTrial, activeInstall, install, installsFor, revoke,
offer, offersFor, offersForListing, decideOffer,
-- seedListing,
++ seedListing, upsertFirstParty, retireFirstParty, removeFirstPartySource,
KINDS, MODES, STATUSES, CLOSED_REASONS, OFFER_KINDS, OFFER_STATUSES, SAFETY,
};
}
ddiff --git a/server/views/index.js b/server/views/index.js
iindex a0a0e23..2163e5f 100644
---- a/server/views/index.js
++++ b/server/views/index.js
@@@ -234,7 +234,7 @@ ${offerList(offers, { owner: true })}
<p class="ex-kicker">${kindLabel(l.kind)} · ${modeBadge(l.mode)}${statusBadge(l.status)}${staff ? ' · <span class="ex-staff">staff view</span>' : ''}</p>
<h1>${esc(l.title)}</h1>
<p class="ex-lede">${esc(l.summary)}</p>
--<p class="ex-byline">by <span class="ex-creator">${esc(l.creator)}</span> · version <span class="ex-version">${esc(l.version)}</span> · <span class="ex-license">${esc(l.license)}</span>${l.license_url ? ` · <a href="${esc(l.license_url)}" rel="nofollow noopener">the licence</a>` : ''}</p>
++<p class="ex-byline">by <span class="ex-creator">${esc(l.creator)}</span> · version <span class="ex-version">${esc(l.version)}</span> · <span class="ex-license">${esc(l.license)}</span>${l.license_url ? ` · <a href="${esc(l.license_url)}" rel="nofollow noopener">the licence</a>` : ''}${l.official ? ' · <span class="ex-official">listed by OpenVibe</span>' : ''}</p>
</header>
${notice ? `<p class="ex-note" role="status">${esc(notice)}</p>` : ''}
@@@ -250,9 +250,16 @@ ${l.mode === 'swap' && l.wants ? `<p class="ex-wants"><b>Wants in return:</b> ${
<div><dt>Mode</dt><dd>${esc(MODE_LABEL[l.mode] || l.mode)}</dd></div>
<div><dt>Listed</dt><dd>${l.listed_at ? time(l.listed_at) : 'not yet'}</dd></div>
<div><dt>Revision</dt><dd>${Number(l.revision) || 1}</dd></div>
++${l.home_url ? `<div><dt>Home</dt><dd><a href="${esc(l.home_url)}" rel="noopener">${esc(l.home_url)}</a></dd></div>` : ''}
</dl>
</section>
++${l.files && l.files.length ? `<section class="ex-files" id="ex-files" aria-labelledby="ex-files-h">
++<div class="ex-section-head"><h2 id="ex-files-h">Downloads</h2><span class="ex-count">${l.files.length} files</span></div>
++<p class="ex-fine">Every file is generated from the same tokens, so pick the platform you need; the format is on each file.</p>
++<ul class="ex-file-list">${l.files.map((f) => `<li><a href="${esc(f.url)}" rel="noopener" download><b>${esc(f.label)}</b><span>${esc(f.format)}</span></a></li>`).join('')}</ul>
++</section>` : ''}
++
${passport(l.passport)}
${testsBlock(tests)}
${sandbox}
@@@ -396,7 +403,7 @@ function developersPage() {
<header class="ex-page-h"><p class="ex-kicker">Developers</p><h1>The Exchange API</h1>
<p class="ex-lede">JSON under <code>/api/v1</code>, with <code>application/problem+json</code> errors. A person is the <code>exchange_at</code> session cookie or a Network user token as a Bearer.</p></header>
<section><h2>Read</h2><dl class="ex-eps">
--${ep('GET', '/api/v1/listings?limit=&cursor=&mode=&kind=&q=', 'Public listings, newest first, cursor-paged (at most 50).')}
++${ep('GET', '/api/v1/listings?limit=&cursor=&mode=&kind=&q=', 'Public listings, interleaved by kind, newest first inside each kind; cursor-paged (at most 50).')}
${ep('GET', '/api/v1/listings/:slug', 'One listing with its passport, manifest, licence, tests and counters. A draft/hidden/closed one is 404 unless it is yours or you are staff.')}
${ep('GET', '/api/v1/listings/:slug/passport', 'The exact passport and manifest an installer accepts.')}
${ep('GET', '/api/v1/listings/:slug/tests', 'The conformance tests for the version the listing names.')}
ddiff --git a/test/first-party.test.js b/test/first-party.test.js
nnew file mode 100644
iindex 0000000..e2ba4c0
---- /dev/null
++++ b/test/first-party.test.js
@@@ -0,0 +1,241 @@
++'use strict';
++/**
++ * The first-party catalogue: Design's built-in themes and their token files, Live's OBS scenes and alerts, and Games'
++ * public worlds. The job is idempotent, keeps each thing's real home, marks it official without inventing a person,
++ * retires what a source no longer lists, and never touches a person's own listing.
++ */
++const assert = require('assert');
++const { boot, check, done } = require('./helpers/boot');
++const fixtures = require('./helpers/fixtures');
++
++const THEMES = {
++ themes: [
++ {
++ id: 'vibe', name: 'Vibe', slug: 'vibe', mode: 'dark',
++ description: 'The default OpenVibe theme.', is_builtin: 1,
++ variables: { '--bg-primary': '#0a0f1c' }, owner_email: '[email protected]', api_key: 'should_never_be_listed',
++ },
++ { id: 'paper', name: 'Paper', slug: 'paper', mode: 'light', description: 'A quiet light theme.', is_builtin: true, variables: {} },
++ { id: 'made-by-a-person', name: 'Mine', slug: 'mine', mode: 'dark', description: 'A community theme.', is_builtin: 0, variables: {} },
++ ],
++};
++
++const TARGETS = {
++ targets: [
++ { id: 'css', label: 'CSS variables', ext: 'css', lang: 'css' },
++ { id: 'json', label: 'Design tokens', ext: 'tokens.json', lang: 'json' },
++ ],
++};
++
++const WORLDS = {
++ worlds: [
++ { id: 'wabc123def0', title: 'Floating islands', summary: 'A world of bridges in the sky.', owner: 'maker', plays: 12, online: 0, play_url: '/play?world=wabc123def0' },
++ ],
++ page: 1,
++ more: false,
++};
++
++function fakeFetch({ themes = THEMES, targets = TARGETS, worlds = WORLDS, failGames = false, failTargets = false } = {}) {
++ return async (url) => {
++ const u = new URL(url);
++ if (u.hostname === 'openvibe.design' && u.pathname === '/api/themes') return { ok: true, status: 200, json: async () => themes };
++ if (u.hostname === 'openvibe.design' && u.pathname === '/api/v1/tokens') {
++ if (failTargets) return { ok: false, status: 503, json: async () => ({}) };
++ return { ok: true, status: 200, json: async () => targets };
++ }
++ if (u.hostname === 'openvibe.games' && u.pathname === '/api/worlds') {
++ if (failGames) throw new Error('Games is restarting');
++ return { ok: true, status: 200, json: async () => worlds };
++ }
++ return { ok: false, status: 404, json: async () => ({}) };
++ };
++}
++
++(async () => {
++ const t = await boot();
++ const alice = t.network.addUser('alice');
++ try {
++ const fetch = fakeFetch();
++ let themeId = null;
++ let revision = null;
++
++ await check('a first pass lists each theme once, with its token files, plus the OBS templates and a public world', async () => {
++ const out = await t.ctx.firstParty.runOnce({ sources: ['design', 'obs', 'games'], fetchImpl: fetch });
++ assert.strictEqual(out.sources.design.themes.ok, true, JSON.stringify(out.sources.design));
++ assert.strictEqual(out.sources.design.themes.imported, 2, 'only the two built-in themes');
++ assert.strictEqual(out.sources.design.removed_tokens, 0);
++ assert.strictEqual(out.sources.obs.imported, 4, 'three scenes and the alerts overlay');
++ assert.strictEqual(out.sources.games.imported, 1);
++
++ const list = await t.get('/api/v1/listings?limit=50');
++ assert.strictEqual(list.status, 200, list.text);
++ assert.strictEqual(list.json().count, 7);
++ const rows = list.json().listings;
++ for (const row of rows) {
++ assert.strictEqual(row.official, true, `${row.slug} is official`);
++ assert.match(row.home_url, /^https:\/\//, `${row.slug} keeps a real home`);
++ assert.strictEqual(row.status, 'listed');
++ assert.strictEqual(row.mode, 'free');
++ assert.ok(!('owner' in row), 'the public list never carries an owner');
++ }
++
++ const theme = rows.find((r) => r.slug === 'vibe-theme');
++ assert.ok(theme, 'the Vibe theme is listed');
++ themeId = theme.id;
++ revision = theme.revision;
++ assert.strictEqual(theme.kind, 'theme');
++ assert.strictEqual(theme.creator, 'OpenVibe');
++ assert.strictEqual(theme.home_url, 'https://openvibe.design/studio?theme=vibe');
++ assert.strictEqual(theme.manifest.entry, theme.home_url);
++ assert.strictEqual(theme.license, 'AGPL-3.0-only');
++ assert.deepStrictEqual(theme.files, [
++ { format: 'css', label: 'CSS variables', url: 'https://openvibe.design/api/v1/tokens/vibe?format=css' },
++ { format: 'json', label: 'Design tokens', url: 'https://openvibe.design/api/v1/tokens/vibe?format=json' },
++ ], 'the theme carries its token files');
++ assert.ok(!rows.some((r) => r.home_url.includes('?format=')), 'no token file is a listing of its own');
++ assert.strictEqual(await t.ctx.s.db.value("SELECT COUNT(*)::int AS n FROM exchange_listings WHERE source = 'design.tokens'"), 0);
++
++ const world = rows.find((r) => r.slug === 'floating-islands');
++ assert.ok(world, 'the public world is listed');
++ assert.strictEqual(world.kind, 'game_asset');
++ assert.strictEqual(world.creator, 'maker', 'the real maker is credited on a network listing');
++ assert.strictEqual(world.home_url, 'https://openvibe.games/play?world=wabc123def0');
++
++ assert.ok(rows.some((r) => r.home_url === 'https://openvibe.live/obs/scene/starting/openvibe'));
++ assert.ok(rows.some((r) => r.home_url === 'https://openvibe.live/obs/scene/brb/openvibe'));
++ assert.ok(rows.some((r) => r.home_url === 'https://openvibe.live/obs/scene/ending/openvibe'));
++ assert.ok(rows.some((r) => r.home_url === 'https://openvibe.live/obs/alerts/openvibe?demo=1'));
++ assert.ok(!rows.some((r) => r.home_url.includes('/mine')), 'a community theme is not relisted as first-party');
++ });
++
++ await check('a second pass changes nothing and creates no duplicate', async () => {
++ const before = await t.ctx.s.db.value('SELECT COUNT(*)::int AS n FROM exchange_listings');
++ const out = await t.ctx.firstParty.runOnce({ sources: ['design', 'obs', 'games'], fetchImpl: fetch });
++ assert.strictEqual(out.sources.design.themes.imported, 2);
++ assert.strictEqual(out.sources.design.removed_tokens, 0);
++ assert.strictEqual(out.sources.games.imported, 1);
++ assert.strictEqual(await t.ctx.s.db.value('SELECT COUNT(*)::int AS n FROM exchange_listings'), before);
++ const dupes = await t.ctx.s.db.many('SELECT source, source_id, COUNT(*)::int AS n FROM exchange_listings WHERE source IS NOT NULL GROUP BY 1, 2 HAVING COUNT(*) > 1');
++ assert.deepStrictEqual(dupes, []);
++ const theme = await t.ctx.s.db.maybe('SELECT * FROM exchange_listings WHERE id = $1', [themeId]);
++ assert.strictEqual(theme.revision, revision, 'an unchanged row is untouched');
++ });
++
++ await check('the old one-listing-per-token-file rows are removed, idempotently', async () => {
++ const legacy = await t.ctx.entries.upsertFirstParty({
++ source: 'design.tokens',
++ sourceId: 'vibe:css',
++ creator: 'OpenVibe',
++ body: {
++ title: 'Vibe tokens — CSS variables', kind: 'template', mode: 'free',
++ summary: 'An old per-file listing.', license: 'AGPL-3.0-only', version: '1.0.0',
++ home_url: 'https://openvibe.design/api/v1/tokens/vibe?format=css',
++ manifest: { name: 'Vibe CSS', entry: 'https://openvibe.design/api/v1/tokens/vibe?format=css', runtime: 'css' },
++ passport: { reads: [], writes: [], spends: [], contacts: ['https://openvibe.design'] },
++ },
++ });
++ assert.strictEqual(legacy.ok, true, JSON.stringify(legacy));
++ const out = await t.ctx.firstParty.runOnce({ sources: ['design'], fetchImpl: fetch });
++ assert.strictEqual(out.sources.design.removed_tokens, 1);
++ assert.strictEqual(await t.ctx.s.db.value("SELECT COUNT(*)::int AS n FROM exchange_listings WHERE source = 'design.tokens'"), 0);
++ const again = await t.ctx.firstParty.runOnce({ sources: ['design'], fetchImpl: fetch });
++ assert.strictEqual(again.sources.design.removed_tokens, 0, 'a second run removes nothing');
++ });
++
++ await check('a token endpoint that is down leaves the theme downloads as they were', async () => {
++ const before = await t.ctx.s.db.maybe('SELECT * FROM exchange_listings WHERE id = $1', [themeId]);
++ const out = await t.ctx.firstParty.runOnce({ sources: ['design'], fetchImpl: fakeFetch({ failTargets: true }) });
++ assert.strictEqual(out.sources.design.themes.ok, true);
++ const after = await t.ctx.s.db.maybe('SELECT * FROM exchange_listings WHERE id = $1', [themeId]);
++ assert.deepStrictEqual(after.files, before.files);
++ assert.strictEqual(after.revision, before.revision, 'no needless rewrite');
++ });
++
++ await check('a changed theme updates the same listing and bumps its revision', async () => {
++ const themes = JSON.parse(JSON.stringify(THEMES));
++ themes.themes[0].description = 'The default OpenVibe theme, now with a new blurb.';
++ const out = await t.ctx.firstParty.runOnce({ sources: ['design'], fetchImpl: fakeFetch({ themes }) });
++ assert.strictEqual(out.sources.design.themes.changed, 1);
++ const theme = await t.ctx.s.db.maybe('SELECT * FROM exchange_listings WHERE id = $1', [themeId]);
++ assert.strictEqual(theme.summary, 'The default OpenVibe theme, now with a new blurb.');
++ assert.strictEqual(Number(theme.revision), Number(revision) + 1);
++ assert.strictEqual(await t.ctx.s.db.value('SELECT COUNT(*)::int AS n FROM exchange_listings WHERE source = $1', ['design.theme']), 2);
++ });
++
++ await check('a theme the source no longer lists is retired', async () => {
++ const themes = { themes: [THEMES.themes[1]] }; // Paper only
++ const out = await t.ctx.firstParty.runOnce({ sources: ['design'], fetchImpl: fakeFetch({ themes }) });
++ assert.strictEqual(out.sources.design.themes.retired, 1);
++ const oldTheme = await t.ctx.s.db.maybe('SELECT * FROM exchange_listings WHERE id = $1', [themeId]);
++ assert.strictEqual(oldTheme.status, 'closed');
++ assert.strictEqual(oldTheme.closed_reason, 'withdrawn');
++ const list = await t.get('/api/v1/listings?limit=50');
++ assert.ok(!list.json().listings.some((r) => r.id === themeId));
++ });
++
++ await check('a retired item that comes back is relisted under the same row', async () => {
++ const out = await t.ctx.firstParty.runOnce({ sources: ['design'], fetchImpl: fakeFetch() });
++ assert.strictEqual(out.sources.design.themes.imported, 2);
++ const back = await t.ctx.s.db.maybe('SELECT * FROM exchange_listings WHERE id = $1', [themeId]);
++ assert.strictEqual(back.status, 'listed');
++ assert.strictEqual(back.id, themeId, 'same row, same home, no duplicate');
++ assert.strictEqual(back.closed_at, null);
++ });
++
++ await check('a source that fails retires nothing', async () => {
++ const before = await t.ctx.s.db.value("SELECT COUNT(*)::int AS n FROM exchange_listings WHERE source = 'games.world' AND status = 'listed'");
++ const out = await t.ctx.firstParty.runOnce({ sources: ['games'], fetchImpl: fakeFetch({ failGames: true }) });
++ assert.strictEqual(out.sources.games.ok, false);
++ assert.match(out.sources.games.error, /restarting/);
++ assert.strictEqual(await t.ctx.s.db.value("SELECT COUNT(*)::int AS n FROM exchange_listings WHERE source = 'games.world' AND status = 'listed'"), before);
++ });
++
++ await check('the first page is interleaved by kind, and the theme page shows its downloads', async () => {
++ const page = await t.get('/api/v1/listings?limit=4');
++ assert.strictEqual(page.status, 200, page.text);
++ const kinds = new Set(page.json().listings.map((r) => r.kind));
++ assert.ok(kinds.size >= 2, `one kind must not fill the first page: ${[...kinds].join(', ')}`);
++
++ const html = await t.get('/listings/vibe-theme');
++ assert.strictEqual(html.status, 200, html.text);
++ assert.match(html.text, /Downloads/);
++ assert.match(html.text, /https:\/\/openvibe\.design\/api\/v1\/tokens\/vibe\?format=css/);
++ assert.match(html.text, /https:\/\/openvibe\.design\/api\/v1\/tokens\/vibe\?format=json/);
++ });
++
++ await check('a person\'s listing is never made official, and the official rows have no owner', async () => {
++ const mine = await fixtures.listed(t, alice, { title: 'Alice hand-made theme', kind: 'theme' });
++ await t.ctx.firstParty.runOnce({ sources: ['design', 'obs', 'games'], fetchImpl: fetch });
++ const after = await t.get(`/api/v1/listings/${mine.slug}`, { as: alice });
++ assert.strictEqual(after.status, 200, after.text);
++ assert.strictEqual(after.json().official, false);
++ assert.strictEqual(after.json().owner, alice.subject);
++ assert.strictEqual(await t.ctx.s.db.value('SELECT COUNT(*)::int AS n FROM exchange_listings WHERE official AND owner IS NOT NULL'), 0);
++ assert.strictEqual(await t.ctx.s.db.value('SELECT COUNT(*)::int AS n FROM exchange_listings WHERE official AND source IS NULL'), 0);
++ });
++
++ await check('nothing private from a source is stored or served', async () => {
++ const row = await t.ctx.s.db.maybe('SELECT * FROM exchange_listings WHERE id = $1', [themeId]);
++ const dump = JSON.stringify(row) + (await t.get(`/api/v1/listings/${row.slug}`)).text;
++ assert.ok(!/designer@example\.test/.test(dump), 'no source email');
++ assert.ok(!/should_never_be_listed/.test(dump), 'no source credential');
++ assert.ok(!/owner_email/.test(dump));
++ });
++
++ await check('the listing page links to the real home and says who listed it', async () => {
++ const paper = await t.ctx.s.db.maybe("SELECT * FROM exchange_listings WHERE slug = 'paper-theme'");
++ assert.ok(paper, 'Paper is still listed');
++ const page = await t.get(`/listings/${paper.slug}`);
++ assert.strictEqual(page.status, 200);
++ assert.match(page.text, /https:\/\/openvibe\.design\/studio\?theme=paper/);
++ assert.match(page.text, /listed by OpenVibe/);
++ });
++
++ await check('the job refuses to start under drill', async () => {
++ assert.strictEqual(t.ctx.firstParty.start(), false);
++ });
++ } finally {
++ await t.close();
++ }
++ done();
++})();