openvibers/openvibe.meme README.md
README.md
| 1 | # OpenVibe.Meme |
| 2 | |
| 3 | > The open meme database, rebuilt with AI. |
| 4 | |
| 5 | **Live site:** https://openvibe.meme · **API docs:** https://openvibe.services · **For AI:** https://openvibe.meme/llms.txt |
| 6 | **Run it:** `npm ci && npm run dev` (Node 22 or newer) · **Tests:** `npm test` |
| 7 | |
| 8 | **Status:** v1 written and tested (see [STATUS.json](STATUS.json)). |
| 9 | **Domain:** `openvibe.meme` · **Port:** 5120 · **Service id:** `meme` · **Env prefix:** `MEME` |
| 10 | **License:** AGPL-3.0-only (same as every OpenVibe service). |
| 11 | |
| 12 | OpenVibe.Meme is an open encyclopedia of internet memes and a maker attached to each entry. Pages explain the format, |
| 13 | cite its origin and spread, connect its variations and related memes, and link to licensed templates. Members can |
| 14 | make editable captions and submit examples for moderation. Known memes may show an OpenVibe.Live appearance when a |
| 15 | transcript names that meme; OpenVibe-only catchphrase pages redirect to the homepage. Every public entry is also a |
| 16 | Search Fabric document. |
| 17 | |
| 18 | ## Purpose |
| 19 | |
| 20 | - **Entries, cited.** Every entry has one origin citation and any number of later uses; the quote is verbatim, |
| 21 | the URL is canonical (OpenVibe.Video's `?t=` redirect is followed, e.g. `/v/6026?t=864` → `/v/6025?t=1541`), |
| 22 | and `quote_hash` is its sha256. A summary is stored as written (L24 has no looks filter); only an empty summary |
| 23 | falls back to the title. Verbatim quotes are never rewritten. |
| 24 | - **Live references stay attached to known memes.** `server/importer.js` reads public Chat/Live history and |
| 25 | transcripts, then attaches a dated use citation only when the text names an existing public internet-meme entry. |
| 26 | It never creates a meme from a stream catchphrase; the Chat global timeline is a paraphrase and is not used as a source. |
| 27 | - **A judge writes the entry.** `server/ai/judge.js` asks OpenVibe.AI to score each candidate from its own |
| 28 | citations — with the chat lines and transcript lines around each quote — and to write the entry from that material |
| 29 | only: a title in natural case, a short summary, where it started, how people use it and two or three example uses. |
| 30 | The score and the reason are stored with the entry. |
| 31 | - **Only what clears the bar is published.** Score 0.75 or more, used at least three times across at least two days |
| 32 | or two channels, nothing denied, nothing about anyone's looks, the entry complete and a live origin citation. |
| 33 | Everything else stays hidden for staff, with its score, on `GET /api/v1/staff/candidates`. Every decision is a |
| 34 | status event. |
| 35 | - **The encyclopedia.** The well-known internet memes, seeded from Wikipedia's lists (CC BY-SA) and Wikidata (CC0) |
| 36 | with a Commons picture only when its licence allows it, each an entry of kind `classic` with its origin, its |
| 37 | spread, its sources and its attribution. |
| 38 | - **The citation is checked again.** A daily job reads every public citation URL; a 404/410 marks it `gone`, a |
| 39 | 401/403 marks it `hidden`, and an entry whose origin is gone is hidden (revision +1, Search tombstone). |
| 40 | - **Meme of the week.** People vote once per ISO week; a second vote changes the choice, never adds a ballot. |
| 41 | - **The encyclopedia (M2).** An entry has a `scope` (`openvibe` for legacy OpenVibe-only phrases, which no longer |
| 42 | have public meme pages, `internet` for the encyclopedia), an origin with a public citation, spread points (date, place, URL, count), named variations, |
| 43 | Media examples, meaning and usage, L1 labels (`nudity`, `language`, `graphic`, `18+`, `flashing`, `spoilers`), |
| 44 | related memes and a moderated revision history. A signed-in person proposes an entry or an edit; it stays |
| 45 | pending, and staff approve or reject it. Every public entry is still one Search Fabric document, now with |
| 46 | `scope` and `labels` facets and the origin as provenance. |
| 47 | - **Trends (M3).** Reddit's official API (OAuth2 client credentials, the four permitted subreddits, no scraping) |
| 48 | records a score sample per post; `/api/v1/trends?window=day|week` ranks posts by upvote velocity, lists the |
| 49 | templates a repeated perceptual hash reveals, and the public entries used in the same window. The importer is |
| 50 | off when `MEME_REDDIT_CLIENT_ID`/`MEME_REDDIT_CLIENT_SECRET` are missing. |
| 51 | - **Analyse (M4).** `POST /api/v1/analyze { url | media_id }` sends the picture to OpenVibe.AI's vision call and |
| 52 | one structured extraction. It returns the template, the matched entries, origin guesses with a confidence, |
| 53 | meaning and usage, L1 labels, and for a Reddit URL its spread and the comments' tone. The daily AI budget is |
| 54 | enforced before any call. |
| 55 | - **Deep dives (M5).** `POST /api/v1/research { topic }` starts a job that gathers sources with OpenVibe.AI's |
| 56 | `research.gather` and writes a long read in which every paragraph cites a numbered source. A read that cites |
| 57 | nothing is failed, not published. `GET /api/v1/research/:id` polls it, and the author may save it as a public |
| 58 | deep-dive entry. |
| 59 | - **The generator.** Pick a template, type 1–4 lines of at most 120 characters each and preview them as HTML text |
| 60 | over the image; a caption is stored as text lines, never as a rendered image. |
| 61 | - **The generator (M6).** `POST /api/v1/generate { template_slug | topic, lines? }` researches how the format is |
| 62 | used right now, writes captions in that voice, and renders a real PNG server-side with the template's own text |
| 63 | boxes, bundled DejaVu fonts, outline and wrap. The PNG goes to OpenVibe.Media and the caption keeps its labels |
| 64 | and Media id. The older caption page still previews text in HTML over the template. |
| 65 | |
| 66 | ## Owns |
| 67 | |
| 68 | | Table | Holds | Migration | |
| 69 | |---|---|---| |
| 70 | | `meme_entries` | `mem_` + ULID, unique slug (`/m/<slug>`), title, normalised `title_key` (lowercase, quotes/punctuation stripped, whitespace collapsed; unique per kind for public rows), kind `joke\|catchphrase\|bit\|meme\|saga\|classic\|format\|event\|person\|phrase\|deep_dive`, scope `openvibe\|internet`, AI summary, meaning/usage, L1 `labels`, status + `status_since`, first/last seen, visibility `public\|unlisted\|hidden`, `submitted_by` for a proposal, `revision` (+1 on every change), timestamps, the picture and its licence, the attribution, the Wikidata id | `migrations/0001_initial.sql`, `0002_judge_and_encyclopedia.sql`, `0003_encyclopedia.sql`, `0009_entry_title_key.sql` | |
| 71 | | `meme_citations` | entry, role `origin\|use\|example`, source service/type/id, stream/VOD/clip/message ids, canonical VOD `offset_seconds`, verbatim `quote_text`, `quote_hash`, canonical `url`, `observed_at`, `source_status` `live\|hidden\|gone`; the open web is a source too (`wikipedia`, `wikidata`, `commons`) | `0001_initial.sql`, `0002_judge_and_encyclopedia.sql` | |
| 72 | | `meme_judgements` | one row per entry: the score, the reason, the title/summary/origin/spread/examples the AI wrote, whether the text names looks or carries a denied word, the model and the run | `0002_judge_and_encyclopedia.sql` | |
| 73 | | `meme_status_events` | every status change: from, to, reason, who, when; a judge decision adds `decision`, `from_visibility`, `to_visibility` and `score` | `0001_initial.sql`, `0002_judge_and_encyclopedia.sql` | |
| 74 | | `meme_origins` | an internet entry's public origin: source `web\|reddit\|media\|openvibe`, URL, title, publisher, author, published/retrieved time, quote | `0003_encyclopedia.sql` | |
| 75 | | `meme_spread_points` | entry, date, place, optional URL and count, note — one sighting per (entry, date, place, URL) | `0003_encyclopedia.sql` | |
| 76 | | `meme_variations` | entry, name, description, optional `med_` Media id, source URL | `0003_encyclopedia.sql` | |
| 77 | | `meme_examples` | entry, `med_` Media id, caption, source URL — reposted pictures keep their source link | `0003_encyclopedia.sql` | |
| 78 | | `meme_entry_related` | directed related pairs; the API returns both sides' public entries | `0003_encyclopedia.sql` | |
| 79 | | `meme_entry_revisions` | entry, revision, author, `change` jsonb, field-by-field `diff`, moderation state `pending\|approved\|rejected`, reviewer and note | `0003_encyclopedia.sql` | |
| 80 | | `meme_entry_redirects` | a merged entry's old slug/id → the kept entry; both answer 301, and a redirect wins over a hidden row that reused the slug | `0009_entry_title_key.sql`, `0011_merge_hidden_duplicates.sql` | |
| 81 | | `meme_votes` | (`week` `YYYY-Www`, `subject`) → entry; one vote a week | `0001_initial.sql` | |
| 82 | | `meme_templates` | id, name, `image_url` (openvibe.media, a local data-free placeholder or a Wikimedia Commons file), `box_count`, source, license, credit, source page, attribution; three placeholders seeded | `0001_initial.sql`, `0002_judge_and_encyclopedia.sql` | |
| 83 | | `meme_captions` | `mcp_` + ULID, subject, template, `lines` jsonb (1–4 plain-text lines), visibility, created | `0001_initial.sql` | |
| 84 | | `meme_reports` | `mrp_` + ULID, entry, subject, kind `takedown\|correction`, reason, note, status | `0001_initial.sql` | |
| 85 | | `meme_reddit_posts` | one Reddit post: id `t3_…`, subreddit, title, permalink, source URL, author, score, comments, over-18, dHash, Media id, created/last seen | `0004_trends.sql` | |
| 86 | | `meme_reddit_samples` | one score observation per post per run; velocity is computed from these, never from one score | `0004_trends.sql` | |
| 87 | | `meme_reddit_templates` | a dHash cluster repeated across posts: label, representative picture, Media id, post count, first/last seen | `0004_trends.sql` | |
| 88 | | `meme_reddit_template_posts` | which posts belong to a template and their hash distance | `0004_trends.sql` | |
| 89 | | `meme_ai_usage` | per UTC day and kind (`analyze`/`research`/`generate`): AI calls reserved and the cost the run stated | `0005_analyze.sql` | |
| 90 | | `meme_analyses` | `anl_` + ULID, subject (or null for a public analysis), input URL/Media id, status, the finished result jsonb and its cost | `0005_analyze.sql` | |
| 91 | | `meme_research` | `rsc_` + ULID, subject, topic, status, title/summary, the Markdown read, numbered sources/facts/citations, cost, error, saved entry slug, times | `0006_research.sql` | |
| 92 | | `meme_deep_dives` | entry, research job, the full Markdown read and its numbered sources/citations | `0006_research.sql` | |
| 93 | | `meme_research_usage` | per UTC day: research calls reserved and the cost the runs stated | `0006_research.sql` | |
| 94 | | `meme_templates` | id, name, `image_url` (openvibe.media or a local data-free placeholder), `box_count`, source, license, `layout` jsonb (text boxes, font, size, colour, outline, align, wrap); three placeholders seeded | `0001_initial.sql`, `0007_generate.sql` | |
| 95 | | `meme_captions` | `mcp_` + ULID, subject, template, `lines` jsonb (1–4 plain-text lines), visibility, created, L1 `labels`, rendered `media_id` | `0001_initial.sql`, `0007_generate.sql` | |
| 96 | | `meme_reports` | `mrp_` + ULID, entry, subject, kind `takedown\|correction`, reason, note, status | `0001_initial.sql` | |
| 97 | | `meme_generate_usage` | per UTC day: generation calls reserved and the cost the runs stated | `0007_generate.sql` | |
| 98 | | `meme_outbox` | Today's outbox shape: fixed `evt_` id, topic, JSON payload, attempts, last error, claim/backoff lease — the Search document queue | `0001_initial.sql` | |
| 99 | | `account_data_events` | openvibe-sdk/account-data's export/deletion receipts | `0001_initial.sql` | |
| 100 | | `telemetry_samples` | aggregated request telemetry, pruned after 14 days | `0001_initial.sql` | |
| 101 | |
| 102 | ## Does not own |
| 103 | |
| 104 | - **Chat, Live and the VODs** — OpenVibe.Chat and OpenVibe.Live. Meme reads their public history, Rundown and |
| 105 | transcripts. It never reads a private VOD, and it never cites VOD **5709** or clip **483** (the fixed deny list). |
| 106 | The one service grant Meme holds is OpenVibe.AI's `ai.run.create`, used by the judge and the encyclopedia. |
| 107 | - **Identity, roles and accounts** — OpenVibe.Network. Meme verifies session tokens against its JWKS; a staff |
| 108 | route asks for `global_mod` or above. |
| 109 | - **Search** — OpenVibe.Search. Meme produces `search.index-document@1` documents in its own outbox, and |
| 110 | `openvibe-publishing/search-relay` delivers them to Search's owner API with Meme's `search.document.write` grant |
| 111 | (`MEME_SEARCH_URL`; unset, they wait). |
| 112 | - **Events, the host and the database** — OpenVibe.Events, OpenVibe.Host. |
| 113 | |
| 114 | ## Depends on |
| 115 | |
| 116 | Environment variable names are in [.env.example](.env.example); no value is committed. |
| 117 | |
| 118 | - **OpenVibe.Network** — sign-in (OAuth client `meme`, PKCE S256, cookies `meme_at`/`meme_rt`/`meme_oauth`), JWKS, |
| 119 | token exchange/refresh/revoke, account export/deletion, staff roles: `OV_NETWORK_URL`, |
| 120 | `OV_NETWORK_INTERNAL_URL`, `OV_NETWORK_ISSUER`, `OV_OAUTH_CLIENT_ID`, `OV_OAUTH_CLIENT_SECRET`, |
| 121 | `OV_OAUTH_REDIRECT_URI`, `OV_SESSION_AUDIENCE`, `MEME_AUDIENCE`. |
| 122 | - **OpenVibe.Chat / Live / Video** — the public reads: `OV_CHAT_URL`, `OV_LIVE_URL`, `OV_VIDEO_URL`, |
| 123 | `MEME_UPSTREAM_TIMEOUT_MS`. |
| 124 | - **OpenVibe.AI** — the judge and the encyclopedia's writing pass: `OV_AI_INTERNAL_URL` (default |
| 125 | `http://127.0.0.1:4700`), `MEME_AI_AUDIENCE` (`openvibe.ai`), `MEME_AI_SCOPE` (`ai.run.create`), |
| 126 | `MEME_AI_WAIT_MS`, `MEME_AI_TIMEOUT_MS`, `MEME_AI_POLL_MS`, `MEME_AI_MAX_TOKENS`. Meme calls `ai.generate` as |
| 127 | the service `meme` with a Network client-credentials token; Network grants `ai.run.create` for the `ai.generate` |
| 128 | namespace. Meme never names a provider — AI's routing decides, and the run reports what answered. |
| 129 | - **Wikipedia, Wikidata and Wikimedia Commons** — the encyclopedia: `MEME_WIKIPEDIA_URL`, `MEME_WIKIDATA_URL`, |
| 130 | `MEME_COMMONS_URL`, `MEME_WIKI_USER_AGENT`, `MEME_WIKI_TIMEOUT_MS`, `MEME_WIKI_MIN_INTERVAL_MS`, |
| 131 | `MEME_ENCYCLOPEDIA_LIMIT`, `MEME_ENCYCLOPEDIA_LISTS`, `MEME_ENCYCLOPEDIA_CATEGORIES`, |
| 132 | `MEME_ENCYCLOPEDIA_MIN_SCORE`, `MEME_TEMPLATES_LIMIT`. |
| 133 | - **OpenVibe.Events** — account deliveries: `MEME_EVENTS_SECRET`, `MEME_EVENTS_URL`, `MEME_EVENTS_ENDPOINT`. |
| 134 | - **PostgreSQL** — `DATABASE_URL` (through PgBouncer) and `DATABASE_DIRECT_URL` (migrations); embedded PGlite in |
| 135 | `MEME_PGLITE_DIR` for development. |
| 136 | - **Valkey** — optional `VALKEY_URL`, `VALKEY_PREFIX` (readiness lists it; the limits in v1 count in-process). |
| 137 | - **Packages** — `openvibe-contracts` v0.229.0, `openvibe-sdk` v0.74.0, `openvibe-shared` v4.179.0, the same |
| 138 | tags OpenVibe.Today pins. |
| 139 | |
| 140 | ## API |
| 141 | |
| 142 | Everything is JSON under `/api/v1`; errors are `application/problem+json` with a `code`. A person is the `meme_at` |
| 143 | session cookie or `Authorization: Bearer <Network user token>`, verified offline with the Network JWKS. A write |
| 144 | made with the session cookie must come from `https://openvibe.meme` (403 `meme.cross_site`). A signed-in account |
| 145 | also gets per-actor minute/hour limits (`MEME_LIMITS_MINUTE`, `MEME_LIMITS_HOUR`). |
| 146 | |
| 147 | | Route | What it does | |
| 148 | |---|---| |
| 149 | | `GET /entries?scope=&q=&status=&label=&kind=&sort=&window=&channel=&limit=&cursor=` | public entries; `scope=openvibe\|internet`; `q` matches title/summary/meaning/usage/slug; `label` is one L1 label; `sort=top&window=day\|week\|month` ranks by live uses (a cursor only on `newest`); `channel=<stream_id>` is Live's per-channel Memes block; `Cache-Control: public, max-age=60` | |
| 150 | | `GET /memes?scope=internet&q=&sort=top&window=week` | internet-only meme-library alias; defaults to `scope=internet` | |
| 151 | | `GET /memes/:slug` | one public internet meme with origin, spread, variations, examples, related memes, and revisions | |
| 152 | | `GET /entries/:slug` | one entry with `origin` (a citation, or the web origin), `web_origin`, citations, `timeline`, `examples`, `spread`, `variations`, `related` and approved `revisions`, and the judge's `notes` (where it started, how people use it, the examples); a hidden proposal is 404 unless its author or staff | |
| 153 | | `GET /entries/:slug/revisions` | the approved revision history; staff also see pending and rejected ones | |
| 154 | | `GET /entries/:slug/series?days=30` | uses per UTC day, zero-filled, 1–365 days | |
| 155 | | `POST /entries` | signed in, same-origin: a proposal (`title`, `kind`, `scope`, `summary`, `meaning`, `usage`, `labels`, `origin`, `spread`, `variations`, `examples`, `related`) — born hidden with revision 1 pending (202); staff proposals are approved and public (201) | |
| 156 | | `POST /entries/:slug/revisions` | signed in: a proposed edit, stored as a revision with a field-by-field `diff`; a person's waits for review (202), staff's applies at once (201) | |
| 157 | | `POST /entries/:slug/revisions/:revision/review` | staff: `{ state: "approved"\|"rejected", note? }`; approving copies the change onto the entry, bumps its revision and re-indexes it | |
| 158 | | `GET /moderation/revisions?limit=` | staff: the pending revision queue with each entry's slug and title | |
| 159 | | `GET /trending?limit=` | public entries, `rising` first, then `peaked`, `classic`, `dead`; last use newest first | |
| 160 | | `GET /trends?window=day\|week&limit=` | rising Reddit posts with upvote velocity (from score samples), new template clusters and the public entries used in the window; `Cache-Control: public, max-age=120` | |
| 161 | | `POST /research` | signed in; `{ topic }` starts a deep-dive job (202 + `Location`); 503 without AI, 429 `meme.ai_budget`, 400 for a topic outside 2–300 characters | |
| 162 | | `GET /research/:id` | the job: queued/running/failed is visible to its author and staff; succeeded is public. Carries the source pack, facts, citations and the Markdown read | |
| 163 | | `POST /research/:id/entry` | the author or staff save a succeeded read as a public deep-dive entry (201, 200 when already saved) | |
| 164 | | `GET /entries/:slug/deep-dive` | the saved deep dive's full Markdown, numbered sources and citations | |
| 165 | | `GET /week` | this ISO week's candidates (public entries with a live use that week) and their vote counts | |
| 166 | | `POST /week/vote` | signed in, same-origin; `{ entry_id }`; upsert per (`week`, subject) — one vote a week | |
| 167 | | `GET /templates` | the caption templates with their caption counts and `layout` | |
| 168 | | `POST /caption-ideas` | signed out or signed in; `{ template_id, topic }` returns short ideas using OpenVibe.AI's cheapest route by default (8 requests/minute/address) | |
| 169 | | `POST /templates/uploads` | signed in; `{ name, image }` accepts PNG/JPEG/WebP up to 6 MB and sends it to Media in the pending review queue | |
| 170 | | `GET /staff/templates/uploads`, `POST /staff/templates/:id/review` | staff-only template moderation queue and approval/rejection | |
| 171 | | `POST /render` | signed in; `{ template_id, lines, format?, editor? }` returns PNG, GIF, or MP4 from the local renderer without an AI call | |
| 172 | | `GET /captions/:id/render?format=png\|gif\|mp4` | render a public saved caption in the requested format; private captions require the owner or staff | |
| 173 | | `POST /generate` | signed in; `{ template_slug \| topic, lines? }`; researches current uses, writes captions, renders the template's layout as a PNG, stores it in Media and saves the caption. 201 returns `media_id`, `media_url`, `captions`, `template`, `labels`, `caption`; 503 `meme.ai_unavailable`/`meme.media_unavailable`; 429 `meme.ai_budget` | |
| 174 | | `POST /captions` | signed in; `{ template_id, lines, visibility?, editor? }`, 1–4 lines × 120 chars, blocked words refused; safe editor geometry is saved with the text | |
| 175 | | `GET /captions/:id` | one caption; a public read never returns the author's subject | |
| 176 | | `POST /entries/:slug/report` | signed in; `{ kind: "takedown"\|"correction", reason, note? }`, stored for staff | |
| 177 | | `GET /reports` | staff: the open reports | |
| 178 | | `GET /staff/candidates` | staff: the hidden candidates with the judge's score, its reason, the uses, the days and the channels | |
| 179 | | `GET /staff/candidates/:slug` | staff: one candidate — its judgement, its citations and its decision history | |
| 180 | | `POST /entries/:slug/publish` | staff: hidden → public; `409 meme.origin_gone` when the origin is not live | |
| 181 | | `POST /entries/:slug/hide` | staff: public → hidden; Search tombstone | |
| 182 | | `PATCH /entries/:slug` | staff: `{ title, kind, status, reason? }`; a status change writes `meme_status_events` and the edit is kept as an approved revision | |
| 183 | |
| 184 | Validation refusals are `400 { code: "meme.invalid", message }`; a blocked caption is |
| 185 | `400 { code: "meme.blocked_words" }`; an edit that changes nothing is `409 meme.no_change` and a revision |
| 186 | already reviewed is `409 meme.revision_reviewed`. Other codes: `meme.sign_in_required`, `meme.staff_required`, |
| 187 | `meme.cross_site`, `meme.not_found`, `meme.template_not_found`, `meme.origin_gone`, `meme.rate_limited`. |
| 188 | |
| 189 | ## The judge |
| 190 | |
| 191 | `server/jobs/judge.js` runs every 30 minutes (`MEME_JUDGE_INTERVAL_MS`), off under `MEME_DRILL`; tests call |
| 192 | `runOnce()`. |
| 193 | |
| 194 | For every candidate nobody has judged yet it: |
| 195 | |
| 196 | 1. reads the entry's live citations and opens each one up — the chat lines around a chat message, the transcript |
| 197 | lines around a VOD second (`server/ai/context.js`), because a joke only makes sense with its neighbours; |
| 198 | 2. asks OpenVibe.AI to score it from 0 to 1 on whether it is a real meme, catchphrase or inside joke rather than |
| 199 | everyday words that recur, and to write the entry from that material only; |
| 200 | 3. stores the score, the reason and the written entry in `meme_judgements`; |
| 201 | 4. applies the publish rule (`server/ai/gate.js`): score ≥ `MEME_JUDGE_MIN_SCORE` (0.75), at least |
| 202 | `MEME_JUDGE_MIN_USES` (3) uses across at least `MEME_JUDGE_MIN_DAYS` (2) days or `MEME_JUDGE_MIN_CHANNELS` (2) |
| 203 | channels, nothing denied, nothing that names looks, the entry complete, and a live origin citation; |
| 204 | 5. publishes what cleared the bar and leaves the rest hidden, logging both as a status event with the score and |
| 205 | the reasons. |
| 206 | |
| 207 | The AI's text is never taken at its word: a title that names looks is dropped (the entry keeps the name it had), |
| 208 | every other field is repaired word by word with `noLooks`, and a field that still names looks — or that carries a |
| 209 | word from the deny list — keeps the entry hidden. |
| 210 | |
| 211 | `node scripts/rejudge.js` runs one round over every hidden candidate and prints what was published and why; |
| 212 | `--dry-run` judges and reports without publishing, `--slug <slug>` judges only those. |
| 213 | |
| 214 | ## The encyclopedia |
| 215 | |
| 216 | `node scripts/seed-classics.js` seeds the well-known internet memes; `node scripts/seed-templates.js` seeds the |
| 217 | caption templates. Neither starts the server. |
| 218 | |
| 219 | The classics are read from Wikipedia's lists of Internet phenomena and memes (CC BY-SA) and from the categories |
| 220 | behind them, and every entry is finished from Wikidata (CC0): the Q id, the inception date and the picture. A |
| 221 | Commons picture is used only when its licence allows it (public domain, CC0, CC), and each entry keeps its credit, |
| 222 | its licence, its file page and the attribution of the text it was written from. The same AI pass writes the entry |
| 223 | — the title in natural case, the summary, where it started, how it spread, two or three example uses — and the |
| 224 | same publish rule decides what becomes public: an entry the AI scored under `MEME_ENCYCLOPEDIA_MIN_SCORE` (0.75), |
| 225 | or whose text names looks or carries a denied word, stays hidden for staff with its score. |
| 226 | |
| 227 | Nothing is scraped from a site whose terms forbid it: Wikipedia, Wikidata and Wikimedia Commons only, each with |
| 228 | its User-Agent, paced per host. |
| 229 | |
| 230 | ## Trends (M3) |
| 231 | |
| 232 | `server/reddit.js` reads `oauth.reddit.com` with this app's own OAuth2 client-credentials token and a User-Agent |
| 233 | that names it. The official API is the only Reddit path: no page, login or captcha wall is touched. Each run |
| 234 | reads `/r/<sub>/top?t=day` for the subreddits in `MEME_REDDIT_SUBREDDITS` (default r/memes, r/dankmemes, |
| 235 | r/MemeEconomy and r/wholesomememes), upserts the posts, appends a score sample, and dHashes the picture with |
| 236 | `sharp` (9×8 grayscale, 64-bit). Equal-enough hashes are grouped; a group of at least |
| 237 | `MEME_REDDIT_MIN_TEMPLATE_POSTS` distinct posts becomes a `meme_reddit_templates` row, and the picture is |
| 238 | uploaded to OpenVibe.Media when `OV_MEDIA_INTERNAL_URL` and the Network client secret are set. Without the Reddit |
| 239 | credentials the importer is off, and the read still serves whatever was recorded before. |
| 240 | |
| 241 | `server/source-registration.js` creates the registry row in OpenVibe.Sources (`meme-reddit`, disabled, with its |
| 242 | terms and robots notes) once when `MEME_SOURCES_REGISTER=1` and `OV_SOURCES_INTERNAL_URL` are set; it never |
| 243 | overwrites an existing row. The row documents where the data comes from; Meme's importer owns the OAuth exchange. |
| 244 | ## Analyse (M4) |
| 245 | |
| 246 | `POST /api/v1/analyze` accepts exactly one of `url` (a public http(s) URL) or `media_id` (a `med_…` object). For a |
| 247 | Reddit URL it reads the post and its top comments through the official API (`server/reddit-read.js`); any other URL |
| 248 | is treated as the picture, and a Media id is resolved to `https://openvibe.media/o/<id>`. |
| 249 | |
| 250 | It then makes two OpenVibe.AI runs: `pics.describe_image` (vision: alt, caption, tags) and `ai.extract` (one |
| 251 | structured answer: template, meaning, usage, labels, origin guesses with confidence, similar formats, comment |
| 252 | tone). Meme matches the answer against its own templates and public entries and never lets the model write an |
| 253 | entry. `MEME_AI_DAILY_CALLS` (200) and `MEME_AI_DAILY_BUDGET_USD` (5) cap the day; the calls are reserved before |
| 254 | the first run and the run's stated cost is recorded in `meme_ai_usage`. Reddit credentials are the same |
| 255 | `MEME_REDDIT_CLIENT_ID`/`MEME_REDDIT_CLIENT_SECRET` as the trends importer. |
| 256 | ## Deep dives (M5) |
| 257 | |
| 258 | `POST /api/v1/research` requires a signed-in person and a topic. The job runs `research.gather` (Wikipedia, the |
| 259 | topic's Fandom wiki, Wikidata, DeepSeek search, Commons/Openverse, as OpenVibe.AI implements it) and then |
| 260 | `ai.generate` with the numbered evidence pack. The prompt requires a 700–1400 word Markdown read whose every |
| 261 | paragraph ends with its source numbers. `groundedIssues()` checks that every body paragraph of at least ten words |
| 262 | has a `[n]` and that every `n` names a gathered source; a read that fails is stored `failed` with |
| 263 | `meme.research_ungrounded`, never published. The read is public once succeeded. Saving it creates a public entry |
| 264 | and `meme_deep_dives` keeps its full text for `GET /entries/:slug/deep-dive`. |
| 265 | |
| 266 | Two calls per job are reserved against `MEME_RESEARCH_DAILY_CALLS` (40) and `MEME_RESEARCH_DAILY_BUDGET_USD` (5); |
| 267 | the calls and the runs' stated cost go in `meme_research_usage`. The job is in-process: a restart loses a queued |
| 268 | job (it stays `running` for the operator to see; there is no retry button yet). |
| 269 | ## The generator (M6) |
| 270 | |
| 271 | `POST /api/v1/generate` takes a template (`template_slug`, which is the id, slug or name) or a topic. It first runs |
| 272 | `research.gather` on how the format is actually captioned right now, then `ai.extract` for the lines and the L1 |
| 273 | labels; with `lines` supplied the person's lines are used verbatim and only the labels come from the model. When a |
| 274 | topic alone is given, the model picks one of the configured templates. |
| 275 | |
| 276 | `server/render-png.js` loads the template's `layout` (or a vertical fallback for a template without one), wraps |
| 277 | each line to its box, shrinks the font until it fits, converts the text to outlines with opentype.js (bundled |
| 278 | DejaVu Sans and DejaVu Sans Bold under `server/assets/fonts`, Bitstream Vera licence included), strokes the |
| 279 | outline and fills the text, then composites the SVG over the template artwork with sharp. The PNG is uploaded to |
| 280 | OpenVibe.Media as a public object; without Media the route answers 503 rather than returning an unstored picture. |
| 281 | The saved caption carries the labels and the `med_…` id. Two calls per request are reserved against |
| 282 | `MEME_GENERATE_DAILY_CALLS` (60) and `MEME_GENERATE_DAILY_BUDGET_USD` (5) in `meme_generate_usage`. |
| 283 | |
| 284 | ## The importer |
| 285 | |
| 286 | `server/importer.js` runs every 30 minutes (`MEME_IMPORTER_INTERVAL_MS`), off under `MEME_DRILL`; tests call |
| 287 | `runOnce()`. It reads public sources only: |
| 288 | |
| 289 | 1. **Chat global history** — verbatim lines with ids and timestamps, each a usable citation. |
| 290 | 2. **Rundown topics and segments** — every quote is verbatim speech with `vod_id` and a VOD second, with `said_at`. |
| 291 | 3. **VOD transcripts** — a segment is used only when the public VOD row gives a date; source part ids are |
| 292 | canonicalised through OpenVibe.Video before a citation is stored. |
| 293 | |
| 294 | The linker matches a transcript against public internet-meme titles, attaches matched moments as `use` citations, |
| 295 | and extends the entry's first/last-use range. It does not create entries or rewrite the researched origin. The |
| 296 | source URL remains canonical, uses naming the deny list are dropped, and repeated reads are deduplicated by the |
| 297 | citation store. `MEME_IMPORTER_MAX_KNOWN_MEMES` limits how many public entries are checked per run. |
| 298 | |
| 299 | ## Citations and re-verification |
| 300 | |
| 301 | `server/reverify.js` runs daily (`MEME_REVERIFY_INTERVAL_MS`), off under `MEME_DRILL`. Each citation is read again: |
| 302 | |
| 303 | - a Chat line is looked up in `GET /api/chat/global/history?before=<timestamp+1m>&limit=500`; a missing id (or a |
| 304 | deleted message) is `gone`/`hidden`; |
| 305 | - every other citation's canonical URL is fetched; 404/410 → `gone`, 401/403 → `hidden`, 2xx/3xx → `live`; |
| 306 | - a redirect to a joined VOD updates `vod_id`, `offset_seconds` and `url`; |
| 307 | - an entry whose origin is no longer `live` is hidden (`revision` +1) and a Search tombstone is queued. |
| 308 | |
| 309 | A transient 5xx is recorded and changes nothing: a hiccup must not erase a real citation. |
| 310 | |
| 311 | ## Search |
| 312 | |
| 313 | `server/search.js` builds one `search.index-document@1` for each public entry: owner `meme`, type `entry`, id |
| 314 | `mem_…`, revision = the entry's `revision`, canonical page `${BASE_URL}/m/<slug>`, title/summary/body (the |
| 315 | verbatim quotes as plain text), facets `kind`/`status`/`source`, authorship `ai_assisted`, and one provenance |
| 316 | reference per live citation. Publishing or editing queues `meme.index_document.upserted`; hiding, deleting or |
| 317 | losing the origin queues `meme.index_document.deleted` (a tombstone wins over an older revision). Rows live in |
| 318 | `meme_outbox` in the same transaction as the entry change; `openvibe-publishing/search-relay` (v1.5.0) sends them to |
| 319 | Search's owner API a few seconds after the change and every minute, oldest first, superseding a document's older rows. The `search-feed` recipe (Work, Help, Rent, Quest) is followed: a document from one |
| 320 | row, a sequencer revision in the row, a tombstone when the page is no longer public. |
| 321 | |
| 322 | ## Pages |
| 323 | |
| 324 | All server-rendered through the OpenVibe Frame (`openvibe-shared/shell`), readable without JavaScript, with semantic |
| 325 | HTML and stable `mm-` ids/classes. Public pages are `public, max-age=60` with `Vary: Cookie`; a page drawn for a |
| 326 | signed-in person is `private, no-store`. |
| 327 | |
| 328 | | Page | What it is | |
| 329 | |---|---| |
| 330 | | `/` | trending, meme of the week, the generator teaser | |
| 331 | | `/browse` | searchable public encyclopedia with collection, type, and usage filters (`/search` alias) | |
| 332 | | `/memes` | permanent redirect to the internet-meme collection | |
| 333 | | `/m/:slug` | one entry: origin with its citation link, the use timeline, examples, and a report form when signed in | |
| 334 | | `/week` | the ballot (`POST /week/vote`) | |
| 335 | | `/make` | phone-first editor: searchable templates, movable/resizable text, crop, type/outline style, stickers, AI ideas, and PNG/GIF/MP4 export; saves captions as text and editor state | |
| 336 | | `/c/:id` | one saved caption, native share/copy, Community draft, Pics/Chat links, and downloadable exports | |
| 337 | | `/developers` | the API, the Search document, the importer's sources | |
| 338 | | 404 | the not-found page | |
| 339 | |
| 340 | The pages remain server-rendered and readable without JavaScript; `public/js/meme.js` makes the caption preview |
| 341 | update as the user types and adds touch dragging/resizing. Template uploads go to OpenVibe.Media and remain hidden |
| 342 | until a staff member approves them. GIF and MP4 exports are short loops of the rendered image. The templates' images |
| 343 | are local data-free SVG placeholders (or a licensed Commons / member `openvibe.media` object). `robots.txt`, `sitemap.xml`, `llms.txt` and `llms-full.txt` list |
| 344 | the public pages and every public entry. |
| 345 | |
| 346 | ## The first proof |
| 347 | |
| 348 | `scripts/seed-proof.js` reads the live public APIs and creates **one hidden entry**: |
| 349 | |
| 350 | - slug `cozmo-offline`, title `Cozmo is offline`, kind `joke`, status `classic`, visibility `hidden`; |
| 351 | - origin: Chat message **86957** (2026-10-28 00:11:53, “your Cozmo ain’t online yet”); |
| 352 | - use: VOD **6025** at **1541 s** — the canonical target of `/v/6026?t=864` (source part 6026 at 864.08 s), quote |
| 353 | “but yeah and Cosmo is broken now…”; |
| 354 | - summary AI-written and stored as written (no looks filter); `first_seen_at`/`last_seen_at` from the two citations. |
| 355 | |
| 356 | It never publishes. A re-run leaves the existing entry and adds only a citation that is not already there. VOD |
| 357 | 5709 and clip 483 are refused before anything is written. |
| 358 | |
| 359 | ## Run and test |
| 360 | |
| 361 | ```sh |
| 362 | npm install |
| 363 | npm test # PGlite; every test file in its own process, no network needed |
| 364 | npm run test:pg # the same through PostgreSQL + PgBouncer (openvibe-sdk scripts/test-services.sh up) |
| 365 | OV_ALLOW_SUITE=1 npm test |
| 366 | npm start # PORT 5120, loopback; DATABASE_URL or an embedded PGlite in development |
| 367 | node scripts/seed-proof.js # the first proof, hidden, from the live public APIs |
| 368 | node scripts/rejudge.js # one judge round over every hidden candidate; prints what was published |
| 369 | node scripts/seed-classics.js # the encyclopedia, from Wikipedia and Wikidata |
| 370 | node scripts/seed-templates.js # the caption templates, from Wikimedia Commons |
| 371 | ``` |
| 372 | |
| 373 | The tests boot the app with a mock Network, an OpenVibe.AI stand-in, a Chat/Live/Video fixture server, and a |
| 374 | temporary database: the migration, the importer's candidate and its canonical redirect, the deny list, L24 text |
| 375 | (no looks filter), the one-vote-a-week rule, the caption limits, staff publish/patch/hide, the Search document (validated against |
| 376 | `search.index-document@1`), the citation 404 → gone → hidden path, sign-in, the pages and cache headers, and the |
| 377 | account export/deletion over votes, captions and reports. The judge, the publish rule, the encyclopedia's |
| 378 | attribution and the templates' licences have their own files: `test/judge.test.js`, `test/encyclopedia.test.js` |
| 379 | and `test/templates.test.js`. |
| 380 | |
| 381 | ## Deploy |
| 382 | |
| 383 | `deploy/systemd/openvibe-meme.service` (127.0.0.1:5120, `/etc/openvibe/meme.env`) and |
| 384 | `deploy/nginx/openvibe.meme.conf` (TLS, www → apex, `/auth/me` on the API zone, `/internal` and `/metrics` 404, a |
| 385 | forms zone for the POST routes). Not deployed by this change. |
| 386 | |
| 387 | ## License |
| 388 | |
| 389 | AGPL-3.0-only. See [LICENSE](LICENSE). |
| 390 | |
| 391 | <!-- versions:start --> |
| 392 | - openvibe-contracts: v0.419.0 |
| 393 | - openvibe-sdk: v0.161.0 |
| 394 | - openvibe-shared: v4.241.0 |
| 395 | - openvibe-publishing: v1.7.0 |
| 396 | <!-- versions:end --> |
| 397 |