openvibers/openvibe.deals Commits

Deals: crowdfunding discovery — one search across Kickstarter, Indiegogo, GoFundMe, Open Collective and more, plus Open Collective campaigns from Sources (#62)

OP OpenVibers committed 29 days ago · 2f49efdf27 · 11 files, +379 −7

Changes

ddiff --git a/README.md b/README.md
iindex 08c67fa..234cd39 100644
---- a/README.md
++++ b/README.md
@@@ -40,7 +40,7 @@ The nine charter tables live in Deals' own PostgreSQL database (`ov_deals` on th
Companion tables (not charter tables): `deal_stores` (store records by domain — the plan's "store
records" obligation; a deviation from the nine-table list, see the final section), `watch_notifications`
(the (watch, observation) uniqueness), `moderation_log`, `rate_events`, `import_state`,
--`subject_projections`, the SDK's `event_outbox` / `idempotency_receipts`, and openvibe-publishing's
++`subject_projections`, `deal_campaigns` (crowdfunding discovery), the SDK's `event_outbox` / `idempotency_receipts`, and openvibe-publishing's
`deal_offer_discussion_refs` and `deal_index_revisions`.
Telemetry (openvibe-sdk 0.46, `platform.telemetry-sample@1`): requests are aggregated by route template, method and status class,
@@@ -67,6 +67,7 @@ gauges to `telemetry_samples` every minute and on stop, and deletes rows older t
| `GET /p/:slug` | a product: **price comparison** across its offers, each with its own latest observation and time (`.json` too) |
| `GET /s/:domain` | a store's deals |
| `GET /search?q=` | search (noindex), with "save this search" / "watch these words" |
++| `GET /crowdfunding?q=&sort=` | crowdfunding discovery: one search opens Kickstarter, Indiegogo, Crowd Supply, Ulule, GoFundMe, Open Collective, Patreon, Wefunder and StartEngine at the words (deep links, nothing read from them), and the campaigns Sources collected (Open Collective) by contributors, newest or most received, amounts "as of" their fetch. Indexable without words; a search is noindex. `/crowdfunding.json` is the same as data (CORS *). |
| `GET/POST /submit` | submit a deal (signed in) |
| `GET/POST /watches` | watches and saved searches; `POST /watches/:id/delete` |
| `GET /mod` | moderators: open flags (incl. vote rings), text waiting for review, possible duplicates; `POST /mod/offers/:slug/(merge\|unmerge\|disable\|enable\|review\|expire)`, `POST /mod/flags/:id/(resolve\|dismiss)` |
@@@ -243,6 +244,12 @@ when Sources fetched them again. A removed item (takedown, licence) removes its
offer with no source left is disabled. Imported text is `review_state = pending` → `noindex` until a
moderator records a review (a `usr_` subject; a service cannot review).
++The crowdfunding feed (`category=crowdfunding`, server/domain/campaigns.js) is pulled through the same chassis with its
++own cursor (`sources.crowdfunding.after` in `deals_ingest_cursor`) into `deal_campaigns` (migration
++`0005_crowdfunding.sql`), never into offers: name, summary, type, first tag, currency, the total received and yearly
++budget in cents and the contributor count, exactly as stated, with the time Sources fetched them. A removed item is
++deleted. Each campaign links to its page on its platform (Open Collective: `https://opencollective.com/<slug>`).
++
### Per-source policy: verbatim links, attribution and browser extensions
Deals normalises an imported offer's link by default: the fragment and tracking/affiliate parameters
ddiff --git a/migrations/0005_crowdfunding.sql b/migrations/0005_crowdfunding.sql
nnew file mode 100644
iindex 0000000..0e9a78f
---- /dev/null
++++ b/migrations/0005_crowdfunding.sql
@@@ -0,0 +1,26 @@
++-- phase: expand
++-- Crowdfunding discovery (server/domain/campaigns.js): the campaigns OpenVibe.Sources fetched in its `crowdfunding`
++-- category (Open Collective's collectives, projects and funds), kept as Sources stated them. Amounts are observations:
++-- observed_at is when Sources fetched them. A removed item is deleted here.
++CREATE TABLE deal_campaigns (
++ item_id text COLLATE "C" PRIMARY KEY,
++ source_key text COLLATE "C" NOT NULL,
++ platform text NOT NULL,
++ url text NOT NULL,
++ name text NOT NULL,
++ summary text,
++ kind text,
++ tag text,
++ currency text,
++ raised_cents bigint,
++ budget_cents bigint,
++ backers integer,
++ website text,
++ started_at text,
++ observed_at bigint NOT NULL,
++ first_seen_at bigint NOT NULL,
++ updated_at bigint NOT NULL
++);
++CREATE INDEX deal_campaigns_backers ON deal_campaigns (backers DESC NULLS LAST, item_id);
++CREATE INDEX deal_campaigns_started ON deal_campaigns (started_at DESC NULLS LAST, item_id);
++CREATE INDEX deal_campaigns_raised ON deal_campaigns (raised_cents DESC NULLS LAST, item_id);
ddiff --git a/public/css/deals.css b/public/css/deals.css
iindex 91a1ba9..30c340f 100644
---- a/public/css/deals.css
++++ b/public/css/deals.css
@@@ -87,3 +87,13 @@ code { overflow-wrap: anywhere; }
.page { max-width: 1080px; }
.account-bar { max-width: 1080px; }
.page .sc-hero, .page .sc-sec, .page .sc-cta { padding-left: 0; padding-right: 0; }
++/* Crowdfunding (server/http/crowdfunding.js) */
++.cf-search { display: flex; flex-wrap: wrap; gap: 8px; margin: 14px 0 8px; }
++.cf-search input { flex: 1 1 260px; width: auto; }
++.cf-search button { margin: 0; }
++.cf-groups, .cf-grid { display: grid; gap: 12px; grid-template-columns: repeat(auto-fill, minmax(260px, 1fr)); }
++.cf-grid { list-style: none; padding: 0; }
++.cf-group, .cf-campaign { margin: 0; }
++.cf-group h3, .cf-campaign h3 { margin: 0 0 4px; font-size: 1.05rem; }
++.cf-links { list-style: none; padding: 0; margin: 8px 0 0; display: flex; flex-wrap: wrap; gap: 8px; }
++.cf-links .button { margin: 0; padding: 6px 12px; }
ddiff --git a/server/app.js b/server/app.js
iindex bb39d40..c477d81 100644
---- a/server/app.js
++++ b/server/app.js
@@@ -90,6 +90,10 @@ async function createApp(opts = {}) {
const votes = createVotes({ config, store, reads, hotness, limits, flags, people, outbox });
const listings = createListings({ store });
const importer = createImporter({ config, store, reads, catalog, offers, indexing, sources, log });
++ // Crowdfunding discovery (domain/campaigns.js): Sources' crowdfunding items through the same chassis, its own cursor.
++ const campaigns = require('./domain/campaigns').createCampaigns({
++ store, log, sources: config.sources.enabled ? createSourcesClient({ config: { ...config, sources: { ...config.sources, category: 'crowdfunding' } }, fetchImpl }) : null,
++ });
const sso = opts.sso || createSsoClient({
site: 'deals', baseUrl: config.baseUrl,
clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret,
@@@ -100,7 +104,7 @@ async function createApp(opts = {}) {
const auth = opts.auth || sso;
const jwksUrl = `${config.networkInternalUrl}/api/.well-known/jwks`;
const viewers = createViewerResolver({ auth, config, people });
-- const worker = createWorker({ config, store, offers, hotness, indexing, listings, importer, limits, log });
++ const worker = createWorker({ config, store, offers, hotness, indexing, listings, importer, campaigns, limits, log });
// Account export and deletion (ADR-033, domain/account-data.js), pushed to Network's internal routes with this service's
// own client-credentials token; a test injects a stand-in through opts.accountSend.
@@@ -109,7 +113,7 @@ async function createApp(opts = {}) {
? createNetworkSender({ networkInternalUrl: config.networkInternalUrl, clientId: config.oauth.clientId, clientSecret: config.oauth.clientSecret, fetch: fetchImpl })
: null);
-- const ctx = { config, store, outbox, people, community, sources, reads, catalog, publication, indexing, hotness, watches, limits, access, offers, flags, votes, listings, importer, auth, viewers, worker, indexnow, accountData, accountSend };
++ const ctx = { config, store, outbox, people, community, sources, reads, catalog, publication, indexing, hotness, watches, limits, access, offers, flags, votes, listings, importer, campaigns, auth, viewers, worker, indexnow, accountData, accountSend };
const app = express();
app.disable('x-powered-by');
@@@ -194,6 +198,7 @@ async function createApp(opts = {}) {
const pages = createPages(ctx);
const formLimit = rateLimit({ windowMs: 60_000, max: 60, standardHeaders: true, legacyHeaders: false });
app.use(['/submit', '/watches', '/mod', '/d/:slug/:action'], (req, res, next) => (req.method === 'POST' ? formLimit(req, res, next) : next()));
++ app.use(require('./http/crowdfunding').createCrowdfundingRoutes(ctx));
app.use(pages.router);
app.use((req, res) => pages.notFound(req, res));
ddiff --git a/server/domain/campaigns.js b/server/domain/campaigns.js
nnew file mode 100644
iindex 0000000..9313829
---- /dev/null
++++ b/server/domain/campaigns.js
@@@ -0,0 +1,119 @@
++'use strict';
++/**
++ * Crowdfunding discovery (the owner's ask: one place to find campaigns across Kickstarter, Indiegogo and the rest).
++ *
++ * PLATFORMS every platform's own search, opened at the person's words (deep links: nothing is fetched from them)
++ * pull() the campaigns OpenVibe.Sources fetched in its `crowdfunding` category (Open Collective's public API),
++ * by change cursor (openvibe-publishing/ingest, cursor sources.crowdfunding.after); a removed item is
++ * deleted, so a takedown at the source reaches the page on the next pull
++ * list() those campaigns, by most contributors, newest or most received, optionally matching words
++ *
++ * Amounts and counts are exactly what the source stated, kept with the time Sources fetched them (never "current").
++ */
++const { createChangeCursor, pullChanges } = require('openvibe-publishing/ingest');
++
++const CURSOR = 'sources.crowdfunding.after';
++const clean = (v, n) => (v == null ? null : String(v).replace(/\s+/g, ' ').trim().slice(0, n) || null);
++const intOrNull = (v) => (Number.isFinite(Number(v)) && v !== null && v !== '' ? Math.round(Number(v)) : null);
++
++/** Where a source's campaigns live: its platform's name and the page of one campaign. */
++const SOURCE_PLATFORMS = {
++ 'opencollective-accounts': { name: 'Open Collective', page: (f) => (f.slug ? `https://opencollective.com/${encodeURIComponent(f.slug)}` : null) },
++};
++
++/**
++ * Each platform's search, checked in a browser on 2026-10-15 (a search URL that drops the words is left out).
++ * `browse` is the page without words. group: what kind of money it is.
++ */
++const PLATFORMS = [
++ { id: 'kickstarter', name: 'Kickstarter', group: 'projects', browse: 'https://www.kickstarter.com/discover', search: (q) => `https://www.kickstarter.com/discover/advanced?term=${encodeURIComponent(q)}&sort=magic` },
++ { id: 'indiegogo', name: 'Indiegogo', group: 'projects', browse: 'https://www.indiegogo.com/explore/all', search: (q) => `https://www.indiegogo.com/en/projects/search?q=${encodeURIComponent(q)}` },
++ { id: 'crowdsupply', name: 'Crowd Supply', group: 'projects', browse: 'https://www.crowdsupply.com/browse', search: (q) => `https://www.crowdsupply.com/search?q=${encodeURIComponent(q)}` },
++ { id: 'ulule', name: 'Ulule', group: 'projects', browse: 'https://www.ulule.com/discover/', search: (q) => `https://www.ulule.com/discover/?q=${encodeURIComponent(q)}` },
++ { id: 'gofundme', name: 'GoFundMe', group: 'causes', browse: 'https://www.gofundme.com/discover', search: (q) => `https://www.gofundme.com/s?q=${encodeURIComponent(q)}` },
++ { id: 'opencollective', name: 'Open Collective', group: 'communities', browse: 'https://opencollective.com/discover', search: (q) => `https://opencollective.com/search?q=${encodeURIComponent(q)}` },
++ { id: 'patreon', name: 'Patreon', group: 'creators', browse: 'https://www.patreon.com/explore', search: (q) => `https://www.patreon.com/explore/search?query=${encodeURIComponent(q)}` },
++ { id: 'wefunder', name: 'Wefunder', group: 'invest', browse: 'https://wefunder.com/explore', search: (q) => `https://wefunder.com/explore?query=${encodeURIComponent(q)}` },
++ { id: 'startengine', name: 'StartEngine', group: 'invest', browse: 'https://www.startengine.com/explore', search: (q) => `https://www.startengine.com/explore?search=${encodeURIComponent(q)}` },
++];
++const GROUPS = [
++ { id: 'projects', title: 'Products and creative projects', note: 'Back a project for a reward: gadgets, games, films, books.' },
++ { id: 'causes', title: 'Causes and people', note: 'Donations for a person, a cause or an emergency.' },
++ { id: 'communities', title: 'Open source and communities', note: 'Collectives that spend in the open.' },
++ { id: 'creators', title: 'Creators', note: 'Monthly memberships for people who make things.' },
++ { id: 'invest', title: 'Invest in startups', note: 'Equity crowdfunding is investing: you can lose all of it. Nothing here is advice.' },
++];
++
++/** The platforms grouped, each with the link for these words (or its browse page). */
++function platformLinks(q) {
++ const words = clean(q, 120);
++ return GROUPS.map((g) => ({ ...g, platforms: PLATFORMS.filter((p) => p.group === g.id).map((p) => ({ id: p.id, name: p.name, url: words ? p.search(words) : p.browse })) }));
++}
++
++const SORTS = {
++ backers: 'backers DESC NULLS LAST, item_id',
++ new: 'started_at DESC NULLS LAST, item_id',
++ raised: 'raised_cents DESC NULLS LAST, item_id',
++};
++
++function createCampaigns({ store, sources, log = console }) {
++ const { db } = store;
++ const cursor = createChangeCursor(db, { prefix: 'deals', now: store.now });
++
++ async function apply(item, t) {
++ const platform = SOURCE_PLATFORMS[item.source_key];
++ if (!platform) return 'hold';
++ if (item.removed) {
++ await t.query('DELETE FROM deal_campaigns WHERE item_id = $1', [item.id]);
++ return 'removed';
++ }
++ const f = item.fields || {};
++ const url = item.canonical_url || platform.page(f);
++ const name = clean(item.title, 200);
++ if (!url || !name) return 'hold';
++ const at = store.now();
++ const observed = Date.parse((item.provenance && item.provenance.retrieved_at) || '') || at;
++ await t.query(`INSERT INTO deal_campaigns (item_id, source_key, platform, url, name, summary, kind, tag, currency, raised_cents, budget_cents, backers, website,
++ started_at, observed_at, first_seen_at, updated_at)
++ VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $16)
++ ON CONFLICT (item_id) DO UPDATE SET url = excluded.url, name = excluded.name, summary = excluded.summary, kind = excluded.kind,
++ tag = excluded.tag, currency = excluded.currency, raised_cents = excluded.raised_cents, budget_cents = excluded.budget_cents,
++ backers = excluded.backers, website = excluded.website, started_at = excluded.started_at, observed_at = excluded.observed_at,
++ updated_at = excluded.updated_at`,
++ [item.id, item.source_key, platform.name, url, name, clean(item.summary, 600), clean(f.account_type, 30), clean(f.tag, 60),
++ /^[A-Z]{3}$/.test(String(f.raised_currency || f.currency || '')) ? String(f.raised_currency || f.currency) : null,
++ intOrNull(f.raised_cents), intOrNull(f.yearly_budget_cents), intOrNull(f.backers),
++ /^https?:\/\//i.test(String(f.website || '')) ? String(f.website).slice(0, 500) : null, item.published_at || null, observed, at]);
++ return 'applied';
++ }
++
++ function where(q) {
++ const words = clean(q, 120);
++ if (!words) return { sql: '', params: [] };
++ const like = `%${words.replace(/[\\%_]/g, (c) => `\\${c}`)}%`;
++ return { sql: 'WHERE name ILIKE $1 OR summary ILIKE $1 OR tag ILIKE $1', params: [like] };
++ }
++
++ return {
++ PLATFORMS, GROUPS, platformLinks,
++ async pull({ pageSize = 200, maxPages = 10 } = {}) {
++ if (!sources || !sources.enabled) return { skipped: 'sources not configured' };
++ try {
++ return await pullChanges({ db, cursor, source: sources, apply, name: CURSOR, pageSize, maxPages });
++ } catch (err) {
++ log.warn('[Deals] crowdfunding pull failed:', err.message);
++ return { error: err.message };
++ }
++ },
++ async list({ q = null, sort = 'backers', limit = 24, offset = 0 } = {}) {
++ const w = where(q);
++ const order = SORTS[sort] || SORTS.backers;
++ const n = w.params.length;
++ const rows = await db.many(`SELECT * FROM deal_campaigns ${w.sql} ORDER BY ${order} LIMIT $${n + 1} OFFSET $${n + 2}`, [...w.params, limit, offset]);
++ const total = Number(await db.value(`SELECT count(*) FROM deal_campaigns ${w.sql}`, w.params)) || 0;
++ return { total, campaigns: rows.map((r) => ({ ...r, raised_cents: r.raised_cents == null ? null : Number(r.raised_cents), budget_cents: r.budget_cents == null ? null : Number(r.budget_cents), observed_at: Number(r.observed_at) })) };
++ },
++ };
++}
++
++module.exports = { createCampaigns, platformLinks, PLATFORMS, GROUPS, SORTS };
ddiff --git a/server/http/crowdfunding.js b/server/http/crowdfunding.js
nnew file mode 100644
iindex 0000000..81eaba1
---- /dev/null
++++ b/server/http/crowdfunding.js
@@@ -0,0 +1,120 @@
++'use strict';
++/**
++ * Crowdfunding discovery on openvibe.deals (server/domain/campaigns.js):
++ *
++ * GET /crowdfunding?q=&sort= every platform's search at once for your words, and the campaigns OpenVibe.Sources
++ * collected (Open Collective), by contributors, newest or most received
++ * GET /crowdfunding.json?q=&sort= the same as data (CORS *)
++ *
++ * The page without words is indexable (the platform list is ours); a search is noindex. Amounts are "as of" when
++ * Sources fetched them. Campaign links leave with rel="nofollow noopener".
++ */
++const express = require('express');
++const seo = require('openvibe-publishing/seo');
++const cache = require('openvibe-shared/cache-policy');
++const ssr = require('openvibe-publishing/ssr');
++const { renderPage } = require('../render/layout');
++
++const { html, raw, paginationHtml } = ssr;
++const PER_PAGE = 24;
++const SORT_LABEL = { backers: 'Most contributors', new: 'Newest', raised: 'Most received' };
++
++const pad = (n) => String(n).padStart(2, '0');
++const day = (ms) => { const d = new Date(ms); return `${d.getUTCFullYear()}-${pad(d.getUTCMonth() + 1)}-${pad(d.getUTCDate())}`; };
++function money(cents, currency) {
++ if (cents == null || !currency) return null;
++ try { return new Intl.NumberFormat('en-US', { style: 'currency', currency, maximumFractionDigits: 0 }).format(cents / 100); } catch { return null; }
++}
++const count = (n) => Number(n).toLocaleString('en-US');
++const KIND = { COLLECTIVE: 'Collective', PROJECT: 'Project', FUND: 'Fund' };
++
++function campaignCard(c) {
++ const raised = money(c.raised_cents, c.currency);
++ const budget = c.budget_cents ? money(c.budget_cents, c.currency) : null;
++ const facts = [raised ? `${raised} received to date` : null, c.backers ? `${count(c.backers)} contributor${c.backers === 1 ? '' : 's'}` : null, budget ? `${budget} a year` : null].filter(Boolean);
++ return html`<li class="card cf-campaign">
++<h3><a href="${c.url}" rel="nofollow noopener">${c.name}</a></h3>
++<p class="meta">${c.platform}${c.kind && KIND[c.kind] ? html` · ${KIND[c.kind]}` : ''}${c.tag ? html` · #${c.tag}` : ''}</p>
++${c.summary ? html`<p>${c.summary.length > 220 ? `${c.summary.slice(0, 217)}…` : c.summary}</p>` : ''}
++${facts.length ? html`<p class="meta">${facts.join(' · ')} <span class="cf-asof">(as of ${day(c.observed_at)})</span></p>` : ''}
++</li>`;
++}
++
++/** /crowdfunding with these words, sort and page (defaults left out). */
++function pageUrl({ q, sort, page }) {
++ const s = new URLSearchParams({ ...(q ? { q } : {}), ...(sort && sort !== 'backers' ? { sort } : {}), ...(page > 1 ? { page: String(page) } : {}) }).toString();
++ return s ? `/crowdfunding?${s}` : '/crowdfunding';
++}
++
++function body({ q, sort, groups, list, pager }) {
++ const tab = (key) => (key === sort ? html`<span aria-current="page">${SORT_LABEL[key]}</span>` : html`<a href="${pageUrl({ q, sort: key })}#campaigns">${SORT_LABEL[key]}</a>`);
++ return String(html`<header class="cf-head"><h1>Crowdfunding, every platform at once</h1>
++<p class="lede">Type what you are looking for once and open it on Kickstarter, Indiegogo, GoFundMe, Open Collective and the rest, then see the open-source and community campaigns OpenVibe tracks itself.</p>
++<form class="search cf-search" action="/crowdfunding" method="get" role="search"><label class="sr" for="cf-q">What are you looking for?</label><input id="cf-q" name="q" type="search" value="${q || ''}" placeholder="e.g. solar charger, board game, open source" maxlength="120"><button type="submit">Search everywhere</button></form></header>
++<section aria-labelledby="h-platforms"><h2 id="h-platforms">${q ? html`“${q}” on every platform` : 'The platforms'}</h2>
++<div class="cf-groups">${groups.map((g) => html`<div class="card cf-group"><h3>${g.title}</h3><p class="meta">${g.note}</p><ul class="cf-links">${g.platforms.map((p) => html`<li><a class="button" href="${p.url}" rel="nofollow noopener" target="_blank">${p.name}</a></li>`)}</ul></div>`)}</div>
++<p class="meta">Each link opens that site's own ${q ? 'search' : 'listings'}; OpenVibe reads nothing from them.</p></section>
++<section aria-labelledby="h-campaigns" id="campaigns"><h2 id="h-campaigns">${q ? html`Campaigns OpenVibe tracks matching “${q}”` : 'Campaigns OpenVibe tracks'}</h2>
++<p class="meta">From Open Collective's public data, refreshed every six hours. Amounts are what it stated when fetched.</p>
++<nav class="tabs" aria-label="Sort">${['backers', 'new', 'raised'].map((k) => html`${tab(k)} `)}</nav>
++${list.campaigns.length ? html`<ol class="cf-grid">${list.campaigns.map(campaignCard)}</ol>` : html`<p class="empty">${q ? 'None of the campaigns OpenVibe tracks match those words; try the platforms above.' : 'No campaigns collected yet.'}</p>`}
++${raw(paginationHtml(pager))}</section>`);
++}
++
++function createCrowdfundingRoutes(ctx) {
++ const { config, store, campaigns, viewers } = ctx;
++ const router = express.Router();
++ router.use(['/crowdfunding', '/crowdfunding.json'], viewers.middleware({ services: false }));
++ const wrap = (fn) => (req, res, next) => Promise.resolve(fn(req, res, next)).catch(next);
++
++ function params(req) {
++ const q = String(req.query.q || '').replace(/\s+/g, ' ').trim().slice(0, 120) || null;
++ const sort = SORT_LABEL[req.query.sort] ? String(req.query.sort) : 'backers';
++ const n = parseInt(req.query.page, 10);
++ return { q, sort, page: Number.isInteger(n) && n > 0 ? Math.min(n, 1000) : 1 };
++ }
++
++ router.get('/crowdfunding.json', wrap(async (req, res) => {
++ const { q, sort, page } = params(req);
++ const list = await campaigns.list({ q, sort, limit: PER_PAGE, offset: (page - 1) * PER_PAGE });
++ res.set('Cache-Control', cache.htmlHeaders({ maxAge: 300 }));
++ res.set('Access-Control-Allow-Origin', '*');
++ res.set('X-Robots-Tag', 'noindex');
++ res.json({
++ q, sort, page, total: list.total,
++ platforms: campaigns.platformLinks(q).flatMap((g) => g.platforms.map((p) => ({ ...p, group: g.id }))),
++ campaigns: list.campaigns.map((c) => ({
++ name: c.name, url: c.url, platform: c.platform, kind: c.kind, tag: c.tag, summary: c.summary, currency: c.currency,
++ raised_cents: c.raised_cents, yearly_budget_cents: c.budget_cents, contributors: c.backers, started_at: c.started_at,
++ observed_at: new Date(c.observed_at).toISOString(),
++ })),
++ });
++ }));
++
++ router.get('/crowdfunding', wrap(async (req, res) => {
++ const { q, sort, page } = params(req);
++ const list = await campaigns.list({ q, sort, limit: PER_PAGE, offset: (page - 1) * PER_PAGE });
++ const pager = ssr.paginate({ page, perPage: PER_PAGE, total: list.total, href: (p) => pageUrl({ q, sort, page: p }) });
++ const indexable = !q && sort === 'backers' && pager.page === 1;
++ const decision = seo.evaluate({
++ state: indexable ? 'published' : 'draft', visibility: indexable ? 'public' : 'private',
++ canonicalUrl: seo.canonicalUrl(config.baseUrl, '/crowdfunding'), wordCount: 0,
++ }, { policy: { minWords: 0 }, now: store.now() });
++ res.vary('Cookie');
++ res.vary('Authorization');
++ res.set('Cache-Control', req.viewer.kind === 'anonymous' ? cache.htmlHeaders({ maxAge: 60 }) : cache.htmlHeaders({ private: true }));
++ if (decision.robots !== 'index, follow') res.set('X-Robots-Tag', decision.robots);
++ res.status(200).type('html').send(renderPage({
++ title: q ? `“${q}” on every crowdfunding site` : 'Crowdfunding, every platform at once',
++ description: 'Search Kickstarter, Indiegogo, GoFundMe, Open Collective, Crowd Supply, Patreon and more at once, and browse open-source and community campaigns with what they have raised.',
++ decision, canonical: seo.canonicalUrl(config.baseUrl, '/crowdfunding'),
++ prev: pager.prev ? pager.prev.href : null, next: pager.next ? pager.next.href : null,
++ body: body({ q, sort, groups: campaigns.platformLinks(q), list, pager }),
++ viewer: req.viewer, config, path: req.originalUrl,
++ }));
++ }));
++
++ return router;
++}
++
++module.exports = { createCrowdfundingRoutes };
ddiff --git a/server/http/discovery.js b/server/http/discovery.js
iindex 11a0c30..ad91daf 100644
---- a/server/http/discovery.js
++++ b/server/http/discovery.js
@@@ -61,7 +61,7 @@ function createDiscoveryRoutes({ config, store, publication, listings }) {
details: 'A price on this site is an observation: it has a source and an observed_at time, and it is only claimed "as of" that time. Offers whose latest observation is older than the freshness window are marked stale and are noindex; expired offers are noindex; a price nobody stated is shown as "not stated" and omitted from structured data (never 0). Every deal page has a JSON twin at <deal URL>.json with the full observation history, sources, votes and indexability reasons. Imported and AI-assisted text is noindex until a person reviews it.',
sections: [
{ title: 'Start here', links: [
-- { title: 'Hot deals', url: abs('/') }, { title: 'New deals', url: abs('/new') }, { title: 'Sitemap', url: abs('/sitemap.xml') },
++ { title: 'Hot deals', url: abs('/') }, { title: 'New deals', url: abs('/new') }, { title: 'Crowdfunding', url: abs('/crowdfunding'), note: 'one search across Kickstarter, Indiegogo, GoFundMe, Open Collective and more, and open-source campaigns with what they raised (/crowdfunding.json)' }, { title: 'Sitemap', url: abs('/sitemap.xml') },
] },
{ title: 'Feeds', links: [{ title: 'RSS', url: abs('/feed.xml') }, { title: 'Atom', url: abs('/atom.xml') }, { title: 'JSON Feed', url: abs('/feed.json') }] },
{ title: 'Data', links: [
ddiff --git a/server/render/layout.js b/server/render/layout.js
iindex 2a28f2b..0c793e5 100644
---- a/server/render/layout.js
++++ b/server/render/layout.js
@@@ -23,6 +23,7 @@ const PUBLIC_DIR = path.join(__dirname, '..', '..', 'public');
const NAV_LINKS = [
{ label: 'Hot', href: '/' },
{ label: 'New', href: '/new' },
++ { label: 'Crowdfunding', href: '/crowdfunding' },
{ label: 'Submit', href: '/submit', icon: 'fa-plus' },
{ label: 'Watches', href: '/watches', icon: 'fa-bell' },
];
ddiff --git a/server/worker.js b/server/worker.js
iindex 6491d63..abc178e 100644
---- a/server/worker.js
++++ b/server/worker.js
@@@ -8,12 +8,13 @@
* 3. freshness re-derive every offer's Search document; a price that went stale since the
* last tick changes the document (freshness facet, noindex) and only then is
* an event emitted (the sequencer makes unchanged documents no-ops)
-- * 4. Sources pull the deals change feed, then re-confirm imported offers going stale
++ * 4. Sources pull the deals change feed, then re-confirm imported offers going stale, then the
++ * crowdfunding feed (domain/campaigns.js)
* 5. housekeeping prune old rate-limit windows
*
* Every step is idempotent, so a tick re-run after a crash changes nothing that already happened.
*/
--function createWorker({ config, store, offers, hotness, indexing, listings, importer, limits, log = console }) {
++function createWorker({ config, store, offers, hotness, indexing, listings, importer, campaigns = null, limits, log = console }) {
let timer = null;
let running = false;
let lastError = null;
@@@ -34,6 +35,7 @@ function createWorker({ config, store, offers, hotness, indexing, listings, impo
summary.reindexed = reindexed;
summary.import = await importer.pull();
summary.refresh = await importer.refresh();
++ if (campaigns) summary.campaigns = await campaigns.pull();
summary.pruned = await limits.prune();
lastError = null;
} catch (err) {
ddiff --git a/test/crowdfunding.test.js b/test/crowdfunding.test.js
nnew file mode 100644
iindex 0000000..30780ac
---- /dev/null
++++ b/test/crowdfunding.test.js
@@@ -0,0 +1,82 @@
++'use strict';
++/**
++ * Crowdfunding discovery (server/domain/campaigns.js, server/http/crowdfunding.js):
++ * - one search opens every platform's own search with the words (and their browse pages without words);
++ * - the campaigns come from Sources' crowdfunding category through their own cursor (the deals importer never sees
++ * them), amounts as stated with the fetch time; a removed item disappears;
++ * - sorting, matching words, the JSON twin; the page without words is indexable, a search is not.
++ */
++const assert = require('assert');
++const { boot, check, done, robotsOf } = require('./helpers/boot');
++const { sourceItem } = require('./helpers/mocks');
++const { platformLinks } = require('../server/domain/campaigns');
++
++const campaign = (id, { slug, name, raised, backers, created, tag = null, type = 'COLLECTIVE', summary = null, removed = null }, at) => ({
++ ...sourceItem({ id, kind: 'campaign', title: name, url: null, sourceKey: 'opencollective-accounts', retrievedAt: at, removed, summary,
++ fields: { slug, account_type: type, currency: 'USD', raised_cents: raised, raised_currency: 'USD', yearly_budget_cents: raised ? Math.round(raised / 4) : 0, backers, image_url: null, website: null, tag } }),
++ category: 'crowdfunding', identity: slug, canonical_url: null, published_at: created,
++});
++
++(async () => {
++ await check('every platform\'s own search for the words; browse pages without words', () => {
++ const groups = platformLinks('solar charger');
++ const all = Object.fromEntries(groups.flatMap((g) => g.platforms.map((p) => [p.id, p.url])));
++ assert.strictEqual(all.kickstarter, 'https://www.kickstarter.com/discover/advanced?term=solar%20charger&sort=magic');
++ assert.strictEqual(all.indiegogo, 'https://www.indiegogo.com/en/projects/search?q=solar%20charger');
++ assert.strictEqual(all.gofundme, 'https://www.gofundme.com/s?q=solar%20charger');
++ assert.strictEqual(all.opencollective, 'https://opencollective.com/search?q=solar%20charger');
++ assert.deepStrictEqual(groups.map((g) => g.id), ['projects', 'causes', 'communities', 'creators', 'invest']);
++ assert.ok(groups.find((g) => g.id === 'invest').note.includes('you can lose all of it'));
++ assert.strictEqual(Object.fromEntries(platformLinks(null).flatMap((g) => g.platforms.map((p) => [p.id, p.url]))).kickstarter, 'https://www.kickstarter.com/discover');
++ });
++
++ const t = await boot();
++ const at = t.clock.now();
++ t.sources.put(campaign('itm_01', { slug: 'logseq', name: 'Logseq', raised: 95659957, backers: 12182, created: '2020-09-22T10:40:25.537Z', tag: 'knowledge-base', summary: 'A privacy-first platform for knowledge.' }, at));
++ t.sources.put(campaign('itm_02', { slug: 'tiny-solar', name: 'Tiny Solar Charger', raised: 120000, backers: 14, created: '2026-10-14T08:00:00Z', type: 'PROJECT', summary: 'An open solar charger.' }, at));
++ t.sources.put(campaign('itm_03', { slug: 'mid', name: 'Mid Collective', raised: 5000000, backers: 300, created: '2024-01-01T00:00:00Z' }, at));
++
++ await check('the crowdfunding feed lands in its own table through its own cursor; deals never see it', async () => {
++ const r = await t.ctx.campaigns.pull();
++ assert.strictEqual(r.applied, 3, JSON.stringify(r));
++ assert.ok(t.sources.calls.some((u) => u.includes('category=crowdfunding')));
++ const offers = await t.ctx.store.db.value('SELECT count(*) FROM deal_offers');
++ assert.strictEqual(Number(offers), 0, 'no offers made from campaigns');
++ const { campaigns } = await t.ctx.campaigns.list();
++ assert.deepStrictEqual(campaigns.map((c) => c.name), ['Logseq', 'Mid Collective', 'Tiny Solar Charger'], 'most contributors first');
++ assert.strictEqual(campaigns[0].url, 'https://opencollective.com/logseq');
++ assert.deepStrictEqual([campaigns[0].raised_cents, campaigns[0].backers, campaigns[0].currency, campaigns[0].observed_at], [95659957, 12182, 'USD', at]);
++ assert.deepStrictEqual((await t.ctx.campaigns.list({ sort: 'new' })).campaigns.map((c) => c.name)[0], 'Tiny Solar Charger');
++ assert.deepStrictEqual((await t.ctx.campaigns.list({ sort: 'raised' })).campaigns.map((c) => c.name)[0], 'Logseq');
++ assert.deepStrictEqual((await t.ctx.campaigns.list({ q: 'solar' })).campaigns.map((c) => c.name), ['Tiny Solar Charger']);
++ assert.deepStrictEqual((await t.ctx.campaigns.list({ q: '100%_' })).campaigns, [], 'LIKE wildcards are matched literally');
++ });
++
++ await check('the page: every platform at once, the campaigns with their amounts as of the fetch, sorting', async () => {
++ const r = await t.get('/crowdfunding');
++ assert.strictEqual(r.status, 200);
++ assert.ok(r.text.includes('Crowdfunding, every platform at once'));
++ assert.ok(r.text.includes('href="https://www.kickstarter.com/discover" rel="nofollow noopener" target="_blank"'));
++ assert.ok(r.text.includes('$956,600 received to date · 12,182 contributors · $239,150 a year'), r.text.slice(r.text.indexOf('cf-campaign'), r.text.indexOf('cf-campaign') + 600));
++ assert.ok(r.text.includes('href="https://opencollective.com/logseq" rel="nofollow noopener"'));
++ assert.ok(r.text.includes('Open Collective · Collective · #knowledge-base'));
++ assert.match(robotsOf(r.text), /^index/);
++ const s = await t.get('/crowdfunding?q=solar+charger&sort=new');
++ assert.ok(s.text.includes('href="https://www.kickstarter.com/discover/advanced?term=solar%20charger&amp;sort=magic"'));
++ assert.ok(s.text.includes('Tiny Solar Charger') && !s.text.includes('>Logseq<'));
++ assert.match(robotsOf(s.text), /noindex/);
++ assert.ok(s.text.includes('href="/crowdfunding?q=solar+charger#campaigns">Most contributors</a>'));
++ });
++
++ await check('the JSON twin, and a removed item leaves', async () => {
++ const j = (await t.get('/crowdfunding.json?q=logseq')).json();
++ assert.deepStrictEqual([j.total, j.campaigns[0].name, j.campaigns[0].contributors, j.campaigns[0].observed_at], [1, 'Logseq', 12182, new Date(at).toISOString()]);
++ assert.ok(j.platforms.some((p) => p.id === 'patreon' && p.url === 'https://www.patreon.com/explore/search?query=logseq'));
++ t.sources.put(campaign('itm_01', { slug: 'logseq', name: 'Logseq', removed: { at: new Date(at).toISOString(), reason: 'takedown' } }, at));
++ await t.ctx.campaigns.pull();
++ assert.strictEqual((await t.ctx.campaigns.list()).total, 2);
++ });
++
++ await t.close();
++ done();
++})();
ddiff --git a/test/perf-budget.test.js b/test/perf-budget.test.js
iindex 3237149..4f92051 100644
---- a/test/perf-budget.test.js
++++ b/test/perf-budget.test.js
@@@ -19,7 +19,7 @@ const BUDGETS = {
jsBrotliKB: 60, // 56.3 (49.9 before web-runtime.js)
cssFiles: 2, // 2 (deals.css, openvibe-shared showcase.css)
cssRawKB: 21, // 19.0 with the deal cards
-- cssBrotliKB: 4.6, // 4.1 (1.5 before showcase.css)
++ cssBrotliKB: 5, // 4.7 with the crowdfunding rules (2026-10-15); 4.1 before (1.5 before showcase.css)
externalFiles: 1, // 0
};