openvibers/openvibe.meme Commits

One public entry per normalised title; merge the existing duplicates (#108)

OP OpenVibers committed 6 days ago · 739239954d · 13 files, +559 −44

* One public entry per normalised title; merge existing duplicates

The importer deduped by slug, so two overlapping phrase windows from one running joke became keep-friends-close and close-and-enemies; the judge then renamed both to the same title. Add title_key (lowercase, quotes and punctuation stripped, whitespace collapsed) with a trigger and a partial unique index per kind, and have the importer upsert on it.

Migration 0009 merges existing public duplicates into the oldest entry, moves their citations, votes, reports, judgements, origins, spread, examples, variations, related rows and revision history, records the merge, and keeps the old slugs and ids as 301 redirects.

* Run the 0009 merge test's DDL as the schema owner

The PostgreSQL test store serves the app from a DML-only runtime role; CREATE FUNCTION and CREATE INDEX must use the owner's direct connection. PGlite keeps its single handle.

Changes

ddiff --git a/CHANGELOG.md b/CHANGELOG.md
iindex 97b8b9c..febd494 100644
---- a/CHANGELOG.md
++++ b/CHANGELOG.md
@@@ -2,6 +2,17 @@
What changed in OpenVibe.Meme, newest first.
++## Unreleased
++
++- **One title, one public entry.** `meme_entries.title_key` is the title lowercased with quotes/punctuation stripped
++ and whitespace collapsed; a trigger keeps it current and a partial unique index on `(kind, title_key)` stops two
++ public entries sharing a title. The importer upserts on the key instead of the slug, so a second phrase that the
++ judge titles the same way enriches the entry that exists. Migration 0009 merges the existing public duplicates into
++ the oldest entry, moves their citations, votes, reports, judgements, origins, spread, examples, variations, related
++ rows and revision history, records the merge in the kept entry's history, and turns the old slug and id into 301
++ redirects (`meme_entry_redirects`).
++- **Tests.** Two ingests of one normalised title stay one entry; 0009 moves everything and old addresses 301.
++
## 0.1.0 — 2026-10-28
The first version: the open meme database, rebuilt with AI.
ddiff --git a/README.md b/README.md
iindex 6b46112..6bda0b0 100644
---- a/README.md
++++ b/README.md
@@@ -62,7 +62,7 @@ a chat line, a VOD second, a Rundown segment — a timeline of later uses, a sta
| Table | Holds | Migration |
|---|---|---|
--| `meme_entries` | `mem_` + ULID, unique slug (`/m/<slug>`), title, kind `joke\|catchphrase\|bit\|meme\|saga\|classic\|format\|event\|person\|phrase\|deep_dive`, scope `openvibe\|internet`, AI summary, meaning/usage, L1 `labels`, status + `status_since`, first/last seen, visibility `public\|unlisted\|hidden`, `submitted_by` for a proposal, `revision` (+1 on every change), timestamps, the picture and its licence, the attribution, the Wikidata id | `migrations/0001_initial.sql`, `0002_judge_and_encyclopedia.sql`, `0003_encyclopedia.sql` |
++| `meme_entries` | `mem_` + ULID, unique slug (`/m/<slug>`), title, normalised `title_key` (lowercase, quotes/punctuation stripped, whitespace collapsed; unique per kind for public rows), kind `joke\|catchphrase\|bit\|meme\|saga\|classic\|format\|event\|person\|phrase\|deep_dive`, scope `openvibe\|internet`, AI summary, meaning/usage, L1 `labels`, status + `status_since`, first/last seen, visibility `public\|unlisted\|hidden`, `submitted_by` for a proposal, `revision` (+1 on every change), timestamps, the picture and its licence, the attribution, the Wikidata id | `migrations/0001_initial.sql`, `0002_judge_and_encyclopedia.sql`, `0003_encyclopedia.sql`, `0009_entry_title_key.sql` |
| `meme_citations` | entry, role `origin\|use\|example`, source service/type/id, stream/VOD/clip/message ids, canonical VOD `offset_seconds`, verbatim `quote_text`, `quote_hash`, canonical `url`, `observed_at`, `source_status` `live\|hidden\|gone`; the open web is a source too (`wikipedia`, `wikidata`, `commons`) | `0001_initial.sql`, `0002_judge_and_encyclopedia.sql` |
| `meme_judgements` | one row per entry: the score, the reason, the title/summary/origin/spread/examples the AI wrote, whether the text names looks or carries a denied word, the model and the run | `0002_judge_and_encyclopedia.sql` |
| `meme_status_events` | every status change: from, to, reason, who, when; a judge decision adds `decision`, `from_visibility`, `to_visibility` and `score` | `0001_initial.sql`, `0002_judge_and_encyclopedia.sql` |
@@@ -72,6 +72,7 @@ a chat line, a VOD second, a Rundown segment — a timeline of later uses, a sta
| `meme_examples` | entry, `med_` Media id, caption, source URL — reposted pictures keep their source link | `0003_encyclopedia.sql` |
| `meme_entry_related` | directed related pairs; the API returns both sides' public entries | `0003_encyclopedia.sql` |
| `meme_entry_revisions` | entry, revision, author, `change` jsonb, field-by-field `diff`, moderation state `pending\|approved\|rejected`, reviewer and note | `0003_encyclopedia.sql` |
++| `meme_entry_redirects` | a merged entry's old slug/id → the kept entry; both answer 301 | `0009_entry_title_key.sql` |
| `meme_votes` | (`week` `YYYY-Www`, `subject`) → entry; one vote a week | `0001_initial.sql` |
| `meme_templates` | id, name, `image_url` (openvibe.media, a local data-free placeholder or a Wikimedia Commons file), `box_count`, source, license, credit, source page, attribution; three placeholders seeded | `0001_initial.sql`, `0002_judge_and_encyclopedia.sql` |
| `meme_captions` | `mcp_` + ULID, subject, template, `lines` jsonb (1–4 plain-text lines), visibility, created | `0001_initial.sql` |
@@@ -283,8 +284,10 @@ The saved caption carries the labels and the `med_…` id. Two calls per request
A 3–6 word phrase that starts and ends with a content word, appears in at least `MEME_IMPORTER_MIN_USES` uses on at
least `MEME_IMPORTER_MIN_DAYS` different days, and is not a substring of a stronger phrase becomes one candidate
(at most `MEME_IMPORTER_MAX_CANDIDATES` per run). Its earliest use is `origin`, the rest are `use`, and the entry
--is inserted `hidden`. Re-running enriches the existing slug with new uses and an extended first/last range; it
--never creates a duplicate and never publishes. Uses naming the deny list are dropped before anything is written.
++is inserted `hidden` with a `title_key` (lowercase, quotes and punctuation stripped, whitespace collapsed). A
++re-run, or a second phrase the judge titles the same way, enriches that one entry for its kind; two public entries
++cannot share the key (the partial unique index), and if duplicates are merged the old slug and id answer 301 to the
++kept entry. The importer never publishes. Uses naming the deny list are dropped before anything is written.
## Citations and re-verification
ddiff --git a/migrations/0009_entry_title_key.sql b/migrations/0009_entry_title_key.sql
nnew file mode 100644
iindex 0000000..1099c92
---- /dev/null
++++ b/migrations/0009_entry_title_key.sql
@@@ -0,0 +1,207 @@
++-- phase: expand
++-- One title, one public entry. The importer recognises a joke by a short phrase and the judge later rewrites the
++-- title in natural case; two overlapping phrases from the same recurring line therefore became two entries with
++-- different slugs and then the same title (Trending showed the joke twice, and its votes and citations were split).
++--
++-- title_key is the title lowercased, with quotes and punctuation stripped and whitespace collapsed. The trigger
++-- keeps it right for every writer, old or new. A unique partial index makes it impossible for two public entries
++-- of the same kind to share a title. Before that index is made, the duplicates that already exist are merged into
++-- the oldest entry (its id and slug stay), their citations, votes, reports, judgements, history and related rows
++-- are moved, and the old slug and id become 301 redirects. Nothing here is a secret.
++
++-- The one normalisation the trigger, the backfill and the app's titleKey() share. Quotes are removed (so "don't"
++-- and "dont" are one key); every other punctuation run becomes a space (so a dash or ellipsis reads like a space);
++-- case and whitespace do not matter.
++CREATE FUNCTION meme_title_key(value text) RETURNS text
++LANGUAGE sql IMMUTABLE STRICT PARALLEL SAFE AS $$
++ SELECT btrim(regexp_replace(
++ regexp_replace(
++ translate(lower(value), chr(8217) || chr(8216) || chr(96) || chr(180) || chr(39), ''),
++ '[^[:alnum:]]+', ' ', 'g'),
++ '\s+', ' ', 'g'))
++$$;
++
++ALTER TABLE meme_entries ADD COLUMN title_key text;
++UPDATE meme_entries SET title_key = meme_title_key(title);
++ALTER TABLE meme_entries ALTER COLUMN title_key SET NOT NULL;
++
++CREATE FUNCTION meme_entries_set_title_key() RETURNS trigger
++LANGUAGE plpgsql AS $$
++BEGIN
++ NEW.title_key := meme_title_key(NEW.title);
++ RETURN NEW;
++END
++$$;
++CREATE TRIGGER meme_entries_title_key
++ BEFORE INSERT OR UPDATE OF title ON meme_entries
++ FOR EACH ROW EXECUTE FUNCTION meme_entries_set_title_key();
++
++CREATE INDEX meme_entries_by_title_key ON meme_entries (title_key, kind);
++
++-- An old slug, or an old entry id, answers 301 to the entry it was merged into. The redirect stays when the
++-- duplicate entry row is deleted; the target keeps its own row.
++CREATE TABLE meme_entry_redirects (
++ from_slug text COLLATE "C" PRIMARY KEY,
++ from_entry_id text COLLATE "C" NOT NULL UNIQUE,
++ to_entry_id text COLLATE "C" NOT NULL REFERENCES meme_entries (id) ON DELETE CASCADE,
++ reason text,
++ created_at timestamptz NOT NULL DEFAULT now(),
++ CONSTRAINT meme_entry_redirects_slug CHECK (from_slug ~ '^[a-z0-9]+(?:-[a-z0-9]+)*$' AND length(from_slug) <= 80),
++ CONSTRAINT meme_entry_redirects_id CHECK (from_entry_id ~ '^mem_[0-9A-HJKMNP-TV-Z]{26}$')
++);
++CREATE INDEX meme_entry_redirects_to ON meme_entry_redirects (to_entry_id);
++
++-- The decision column learns 'merge' so the merged entry's history says what happened.
++ALTER TABLE meme_status_events DROP CONSTRAINT meme_status_events_decision;
++ALTER TABLE meme_status_events ADD CONSTRAINT meme_status_events_decision
++ CHECK (decision IS NULL OR decision IN ('publish', 'keep_hidden', 'judge', 'hide', 'seed', 'merge'));
++
++-- Merge every public duplicate group into its oldest entry. The function is kept: it is the same answer if a
++-- duplicate ever reaches the table through a path that bypassed the index (a restore, or a manual insert).
++CREATE FUNCTION meme_merge_public_duplicates() RETURNS integer
++LANGUAGE plpgsql AS $$
++DECLARE
++ dup record;
++ keeper_revision bigint;
++ merged_ids text[] := '{}';
++ moved integer := 0;
++BEGIN
++ FOR dup IN
++ SELECT d.id, d.slug, d.kind, d.revision, d.first_seen_at, d.last_seen_at,
++ k.id AS keeper_id, k.slug AS keeper_slug, k.status AS keeper_status
++ FROM (
++ SELECT e.*,
++ row_number() OVER (PARTITION BY e.kind, e.title_key ORDER BY e.created_at ASC, e.id ASC) AS rn,
++ first_value(e.id) OVER (PARTITION BY e.kind, e.title_key ORDER BY e.created_at ASC, e.id ASC) AS keeper_id
++ FROM meme_entries e
++ WHERE e.visibility = 'public'
++ ) d
++ JOIN meme_entries k ON k.id = d.keeper_id
++ WHERE d.rn > 1
++ ORDER BY d.kind, d.title_key, d.created_at, d.id
++ LOOP
++ -- Citations: the keeper's own copy of a moment wins; a second origin becomes a use.
++ DELETE FROM meme_citations c
++ WHERE c.entry_id = dup.id
++ AND EXISTS (
++ SELECT 1 FROM meme_citations k
++ WHERE k.entry_id = dup.keeper_id
++ AND k.source_service = c.source_service AND k.source_type = c.source_type
++ AND k.source_id = c.source_id
++ AND COALESCE(k.offset_seconds, -1) = COALESCE(c.offset_seconds, -1)
++ );
++ IF EXISTS (SELECT 1 FROM meme_citations WHERE entry_id = dup.keeper_id AND role = 'origin') THEN
++ UPDATE meme_citations SET role = 'use' WHERE entry_id = dup.id AND role = 'origin';
++ END IF;
++ UPDATE meme_citations SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++
++ -- The week's votes are a (week, subject) key: move the ones the keeper has no ballot for.
++ DELETE FROM meme_votes v
++ WHERE v.entry_id = dup.id
++ AND EXISTS (SELECT 1 FROM meme_votes k WHERE k.entry_id = dup.keeper_id AND k.week = v.week AND k.subject = v.subject);
++ UPDATE meme_votes SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++
++ UPDATE meme_status_events SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++ UPDATE meme_reports SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++
++ -- One judgement, origin and deep dive per entry: the keeper's own row wins.
++ IF EXISTS (SELECT 1 FROM meme_judgements WHERE entry_id = dup.keeper_id) THEN
++ DELETE FROM meme_judgements WHERE entry_id = dup.id;
++ ELSE
++ UPDATE meme_judgements SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++ END IF;
++ IF EXISTS (SELECT 1 FROM meme_origins WHERE entry_id = dup.keeper_id) THEN
++ DELETE FROM meme_origins WHERE entry_id = dup.id;
++ ELSE
++ UPDATE meme_origins SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++ END IF;
++ IF EXISTS (SELECT 1 FROM meme_deep_dives WHERE entry_id = dup.keeper_id) THEN
++ DELETE FROM meme_deep_dives WHERE entry_id = dup.id;
++ ELSE
++ UPDATE meme_deep_dives SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++ END IF;
++
++ DELETE FROM meme_spread_points s
++ WHERE s.entry_id = dup.id
++ AND EXISTS (
++ SELECT 1 FROM meme_spread_points k
++ WHERE k.entry_id = dup.keeper_id AND k.observed_at = s.observed_at
++ AND k.place = s.place AND k.url IS NOT DISTINCT FROM s.url
++ );
++ UPDATE meme_spread_points SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++
++ UPDATE meme_variations SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++ UPDATE meme_examples SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++
++ -- Related memes: a relation from one duplicate to the other becomes a self-reference, so it goes; a
++ -- relation the keeper already has wins; the rest are rewritten to the keeper.
++ DELETE FROM meme_entry_related
++ WHERE (entry_id = dup.id AND related_id = dup.keeper_id)
++ OR (entry_id = dup.keeper_id AND related_id = dup.id);
++ DELETE FROM meme_entry_related r
++ WHERE (r.entry_id = dup.id
++ AND EXISTS (SELECT 1 FROM meme_entry_related k WHERE k.entry_id = dup.keeper_id AND k.related_id = r.related_id))
++ OR (r.related_id = dup.id
++ AND EXISTS (SELECT 1 FROM meme_entry_related k WHERE k.entry_id = r.entry_id AND k.related_id = dup.keeper_id));
++ UPDATE meme_entry_related SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++ UPDATE meme_entry_related SET related_id = dup.keeper_id WHERE related_id = dup.id;
++ DELETE FROM meme_entry_related WHERE entry_id = related_id;
++
++ -- Wiki history: keep the keeper's revision numbers; the duplicate's colliding ones go, the rest move over.
++ DELETE FROM meme_entry_revisions v
++ WHERE v.entry_id = dup.id
++ AND EXISTS (SELECT 1 FROM meme_entry_revisions k WHERE k.entry_id = dup.keeper_id AND k.revision = v.revision);
++ UPDATE meme_entry_revisions SET entry_id = dup.keeper_id WHERE entry_id = dup.id;
++
++ -- Search: a pending document for the merged id must not be delivered after the delete; queue the tombstone.
++ DELETE FROM meme_outbox WHERE published_at IS NULL AND payload->>'id' = dup.id;
++ INSERT INTO meme_outbox (event_id, topic, payload, created_at, next_attempt_at)
++ VALUES ('evt_' || substr(dup.id, 5), 'meme.index_document.deleted',
++ jsonb_build_object('owner', 'meme', 'type', 'entry', 'id', dup.id,
++ 'revision', dup.revision + 1, 'deleted', true),
++ now(), now())
++ ON CONFLICT (event_id) DO NOTHING;
++
++ -- The kept entry takes the merged range, one revision and one history row per merge. The new revision is
++ -- above the highest history row either entry brought, not just above the keeper's old counter.
++ SELECT GREATEST(k.revision, COALESCE((SELECT MAX(r.revision) FROM meme_entry_revisions r WHERE r.entry_id = k.id), 0)) + 1
++ INTO keeper_revision
++ FROM meme_entries k WHERE k.id = dup.keeper_id;
++
++ UPDATE meme_entries
++ SET revision = keeper_revision,
++ first_seen_at = LEAST(first_seen_at, dup.first_seen_at),
++ last_seen_at = GREATEST(last_seen_at, dup.last_seen_at),
++ updated_at = now()
++ WHERE id = dup.keeper_id;
++
++ INSERT INTO meme_entry_revisions (entry_id, revision, author, base_revision, change, diff, moderation_state, note, created_at, reviewed_at, reviewed_by)
++ VALUES (dup.keeper_id, keeper_revision, 'service:meme:migration', keeper_revision - 1,
++ jsonb_build_object('merge', jsonb_build_object('id', dup.id, 'slug', dup.slug)),
++ jsonb_build_array(jsonb_build_object('field', 'merged', 'from', dup.slug, 'to', dup.keeper_slug)),
++ 'approved', 'duplicate merged by migration 0009', now(), now(), 'service:meme:migration');
++
++ INSERT INTO meme_status_events (entry_id, from_status, to_status, reason, changed_by, decision, from_visibility, to_visibility)
++ VALUES (dup.keeper_id, dup.keeper_status, dup.keeper_status,
++ 'merged duplicate ' || dup.id || ' (/' || dup.slug || ') into /' || dup.keeper_slug,
++ 'service:meme:migration', 'merge', 'public', 'public');
++
++ INSERT INTO meme_entry_redirects (from_slug, from_entry_id, to_entry_id, reason)
++ VALUES (dup.slug, dup.id, dup.keeper_id, 'duplicate merged by migration 0009')
++ ON CONFLICT (from_slug) DO NOTHING;
++
++ merged_ids := merged_ids || dup.id;
++ moved := moved + 1;
++ END LOOP;
++
++ IF cardinality(merged_ids) > 0 THEN
++ DELETE FROM meme_entries WHERE id = ANY(merged_ids);
++ END IF;
++ RETURN moved;
++END
++$$;
++
++SELECT meme_merge_public_duplicates();
++
++-- From here on a second public entry with the same normalised title and kind cannot exist.
++CREATE UNIQUE INDEX meme_entries_public_title_key ON meme_entries (kind, title_key) WHERE visibility = 'public';
ddiff --git a/scripts/seed-proof.js b/scripts/seed-proof.js
iindex 183c975..e5a2c07 100644
---- a/scripts/seed-proof.js
++++ b/scripts/seed-proof.js
@@@ -99,18 +99,15 @@ async function runSeed({ config, s, upstreams, log = console } = {}) {
const last = citations.reduce((m, c) => (Date.parse(c.observed_at) > Date.parse(m) ? c.observed_at : m), citations[0].observed_at);
const summary = cleanSummary(TITLE, 'Cozmo, the robot, is not online: a chat line and a stream moment about its Wi-Fi. The robot is offline, again.');
-- const existing = await entries.bySlug(SLUG);
-- if (existing) {
-- const enriched = await entries.enrich(existing.id, { first_seen_at: first, last_seen_at: last, cites: citations });
-- const added = enriched && enriched.added ? enriched.added : 0;
-- log.log(`[seed-proof] ${SLUG} already exists (${existing.id}, ${existing.visibility}); ${added} new citation(s) added — left as it is`);
-- return { created: false, entry: existing, citations, added };
-- }
--
-- const out = await entries.createCandidate({
++ const out = await entries.upsertCandidate({
slug: SLUG, title: TITLE, kind: 'joke', summary, status: 'classic',
first_seen_at: first, last_seen_at: last, visibility: 'hidden', citations,
});
++ if (!out.created) {
++ const added = out.added ? out.added : 0;
++ log.log(`[seed-proof] ${SLUG} already exists (${out.entry.id}, ${out.entry.visibility}); ${added} new citation(s) added — left as it is`);
++ return { created: false, entry: out.entry, citations, added };
++ }
log.log(`[seed-proof] created ${out.entry.id} /${SLUG} (${out.entry.visibility}, ${out.entry.status})`);
log.log(`[seed-proof] origin: chat message ${msg.id} at ${msg.timestamp} — "${short(msg.message, 90)}"`);
log.log(`[seed-proof] use: VOD ${canon.vod_id} @${canon.offset_seconds}s (source part ${SOURCE_VOD} @${SOURCE_SEC}s) — "${short(seg.text, 90)}"`);
ddiff --git a/server/http/api.js b/server/http/api.js
iindex d7bb73a..3b15567 100644
---- a/server/http/api.js
++++ b/server/http/api.js
@@@ -67,6 +67,18 @@ function createApi(ctx) {
const problem = (res, status, code, detail) => http.sendProblem(res, status, code, { detail, ctx: res.req && res.req.ov });
const invalid = (res, detail) => problem(res, 400, 'meme.invalid', detail);
++ /** A merged slug or id answers 301 to the kept entry; null when there is no entry and no redirect. */
++ async function bySlugOrRedirect(value, req, res, { suffix = '' } = {}) {
++ const row = await entries.bySlug(value);
++ if (row) return row;
++ const redirect = await entries.redirectFor(value);
++ if (!redirect) return null;
++ const target = await entries.byId(redirect.to_entry_id);
++ if (!target || (target.visibility === 'hidden' && !isStaff(req.principal))) return null;
++ res.redirect(301, `/api/v1/entries/${encodeURIComponent(target.slug)}${suffix}`);
++ return undefined;
++ }
++
function person(req, res) {
const p = req.principal;
if (p.kind === 'anonymous') { problem(res, 401, 'meme.sign_in_required', 'Sign in at openvibe.meme, or send a person\'s Network token as a Bearer.'); return null; }
@@@ -125,7 +137,8 @@ function createApi(ctx) {
});
r.get('/entries/:slug/series', async (req, res) => {
-- const row = await entries.bySlug(String(req.params.slug || ''));
++ const row = await bySlugOrRedirect(String(req.params.slug || ''), req, res, { suffix: '/series' });
++ if (row === undefined) return;
if (!row || (row.visibility === 'hidden' && !isStaff(req.principal))) return problem(res, 404, 'meme.not_found', 'No such meme.');
const days = req.query.days == null || req.query.days === '' ? 30 : Number.parseInt(req.query.days, 10);
if (!Number.isFinite(days) || days < 1 || days > 365) return invalid(res, 'days must be a whole number from 1 to 365');
@@@ -137,7 +150,8 @@ function createApi(ctx) {
// GET /entries/:slug — the origin, citations, spread, variations, Media examples, related memes and the
// approved revision history. A pending proposal is visible to its author and staff only.
r.get('/entries/:slug', async (req, res) => {
-- const row = await entries.bySlug(String(req.params.slug || ''));
++ const row = await bySlugOrRedirect(String(req.params.slug || ''), req, res);
++ if (row === undefined) return;
const staff = isStaff(req.principal);
const mine = Boolean(row && row.submitted_by && req.principal.kind === 'user' && req.principal.subject === row.submitted_by);
if (!row || (row.visibility === 'hidden' && !staff && !mine)) return problem(res, 404, 'meme.not_found', 'No such meme.');
@@@ -156,7 +170,8 @@ function createApi(ctx) {
// GET /entries/:slug/revisions — approved history for anyone; pending/rejected too for staff.
r.get('/entries/:slug/revisions', async (req, res) => {
-- const row = await entries.bySlug(String(req.params.slug || ''));
++ const row = await bySlugOrRedirect(String(req.params.slug || ''), req, res, { suffix: '/revisions' });
++ if (row === undefined) return;
const staff = isStaff(req.principal);
const mine = Boolean(row && row.submitted_by && req.principal.kind === 'user' && req.principal.subject === row.submitted_by);
if (!row || (row.visibility === 'hidden' && !staff && !mine)) return problem(res, 404, 'meme.not_found', 'No such meme.');
@@@ -274,7 +289,8 @@ function createApi(ctx) {
// The full text of a saved deep dive, for the entry page.
r.get('/entries/:slug/deep-dive', async (req, res) => {
-- const entry = await entries.bySlug(String(req.params.slug || ''));
++ const entry = await bySlugOrRedirect(String(req.params.slug || ''), req, res, { suffix: '/deep-dive' });
++ if (entry === undefined) return;
if (!entry || entry.visibility !== 'public') return problem(res, 404, 'meme.not_found', 'No such deep dive.');
const dive = await research.deepDiveFor(entry.id);
if (!dive) return problem(res, 404, 'meme.not_found', 'No deep dive for that entry.');
ddiff --git a/server/http/pages.js b/server/http/pages.js
iindex 178404f..3d23e9a 100644
---- a/server/http/pages.js
++++ b/server/http/pages.js
@@@ -73,8 +73,17 @@ function createPageRoutes(ctx) {
async function entryPage(req, res, { problem = null } = {}) {
const slug = String(req.params.slug || '');
-- const row = await entries.bySlug(slug);
const staff = staffViewer(req);
++ const row = await entries.bySlug(slug);
++ if (!row) {
++ // A merged entry's old slug (or id) answers 301 to the entry it was merged into, so an old link lands
++ // on the live page instead of a 404.
++ const redirect = await entries.redirectFor(slug);
++ if (redirect) {
++ const target = await entries.byId(redirect.to_entry_id);
++ if (target && (target.visibility === 'public' || staff)) return res.redirect(301, `/m/${encodeURIComponent(target.slug)}`);
++ }
++ }
if (!row || (row.visibility === 'hidden' && !staff)) return notFound(req, res);
const citations = await entries.citations(row.id);
const e = entries.wireEntry(row);
ddiff --git a/server/importer.js b/server/importer.js
iindex 217442e..82b236e 100644
---- a/server/importer.js
++++ b/server/importer.js
@@@ -237,18 +237,15 @@ function createImporter({ config, s, entries, upstreams, fetchImpl = globalThis.
const lastSeen = cites.reduce((m, c) => (Date.parse(c.observed_at) > Date.parse(m) ? c.observed_at : m), cites[0].observed_at);
const title = displayTitle(candidate.phrase);
const summary = cleanSummary(title, `"${title}" recurs in ${cites.length} moments across ${new Set(cites.map((c) => utcDay(c.observed_at))).size} days on OpenVibe, from ${firstSeen.slice(0, 10)} to ${lastSeen.slice(0, 10)}.${hints.includes(candidate.phrase) ? ' Chat\'s timeline names it too.' : ''}`);
-- const existing = await entries.bySlug(slugify(title));
-- if (existing) {
-- const r = await entries.enrich(existing.id, { first_seen_at: firstSeen, last_seen_at: lastSeen, cites });
-- if (r && r.changed) enriched++;
-- } else {
-- await entries.createCandidate({
-- slug: slugify(title), title, kind: candidate.phrase.split(' ').length <= 4 ? 'catchphrase' : 'meme',
-- summary, status: statusFor(firstSeen, lastSeen, s.now()), first_seen_at: firstSeen, last_seen_at: lastSeen,
-- visibility: 'hidden', citations: cites,
-- });
-- created++;
-- }
++ // One entry per normalised title and kind: a phrase the table already has is enriched, whatever
++ // slug an earlier ingest gave it. A different slug never means a different joke.
++ const r = await entries.upsertCandidate({
++ slug: slugify(title), title, kind: candidate.phrase.split(' ').length <= 4 ? 'catchphrase' : 'meme',
++ summary, status: statusFor(firstSeen, lastSeen, s.now()), first_seen_at: firstSeen, last_seen_at: lastSeen,
++ visibility: 'hidden', citations: cites,
++ });
++ if (r.created) created++;
++ else if (r.changed) enriched++;
} catch (err) {
errors.push(`candidate ${candidate.phrase}: ${err.message}`);
log.warn(`[OpenVibe.Meme] importer candidate failed: ${err.message}`);
ddiff --git a/server/store.js b/server/store.js
iindex ce0f47a..b027864 100644
---- a/server/store.js
++++ b/server/store.js
@@@ -33,6 +33,17 @@ const nOrNull = (v) => (v == null || v === '' ? null : (Number.isFinite(Number(v
const sha256 = (s) => crypto.createHash('sha256').update(String(s), 'utf8').digest('hex');
const refusal = (status, code, message, field) => ({ ok: false, status, code, message, ...(field ? { field } : {}) });
++/** The same normalisation as migration 0009's meme_title_key(): lowercase, quotes removed, other punctuation as a
++ * space, whitespace collapsed. Two titles that differ only in case, quotes or punctuation have one key. */
++function titleKey(title) {
++ return String(title == null ? '' : title)
++ .toLowerCase()
++ .replace(/['’‘`´]/g, '')
++ .replace(/[^\p{L}\p{N}]+/gu, ' ')
++ .replace(/\s+/g, ' ')
++ .trim();
++}
++
function wireEntry(row) {
if (!row) return null;
return {
@@@ -494,6 +505,32 @@ function createStore(s, { search }) {
async function byId(id, t) {
return await q(t).maybe('SELECT * FROM meme_entries WHERE id = $1', [String(id || '')]);
}
++ /** The entry a merged slug or id answers 301 to, or null. */
++ async function redirectFor(value, t) {
++ return await q(t).maybe('SELECT * FROM meme_entry_redirects WHERE from_slug = $1 OR from_entry_id = $1 LIMIT 1', [String(value || '')]);
++ }
++ /** The entry whose title has this key. A public entry wins over a hidden candidate. */
++ async function byTitleKey(title, kind = null, t) {
++ const key = titleKey(title);
++ if (!key) return null;
++ const args = [key];
++ let where = 'title_key = $1';
++ if (kind) { args.push(kind); where += ' AND kind = $2'; }
++ return await q(t).maybe(
++ `SELECT * FROM meme_entries WHERE ${where}
++ ORDER BY CASE visibility WHEN 'public' THEN 0 WHEN 'hidden' THEN 1 ELSE 2 END, created_at ASC, id ASC LIMIT 1`,
++ args,
++ );
++ }
++ /** A different public entry already using this title and kind, if any. */
++ async function publicTitleConflict(kind, title, id, t) {
++ const key = titleKey(title);
++ if (!key) return null;
++ return await q(t).maybe(
++ "SELECT id, slug FROM meme_entries WHERE visibility = 'public' AND kind = $1 AND title_key = $2 AND id <> $3 LIMIT 1",
++ [kind, key, String(id || '')],
++ );
++ }
async function citations(entryId, t) {
return (await q(t).many('SELECT * FROM meme_citations WHERE entry_id = $1 ORDER BY observed_at ASC, id ASC', [entryId])).map(wireCitation);
}
@@@ -721,6 +758,30 @@ function createStore(s, { search }) {
return await createEntry({ ...opts, visibility: opts.visibility || 'hidden' }, t);
}
++ /**
++ * The importer's write: one entry per normalised title and kind. A second ingest of the same phrase adds its
++ * citations to the entry that is already there instead of inserting a row with a different slug. The advisory
++ * lock serialises two ingest processes that both miss the lookup; the entry row itself is not unique while
++ * hidden, so the lock is what stops a duplicate candidate.
++ */
++ async function upsertCandidate(opts, t = null) {
++ const run = async (tx) => {
++ const cleanTitle = String(opts.title || '').replace(/\s+/g, ' ').trim().slice(0, 160) || 'Untitled meme';
++ const cleanKind = KINDS.includes(opts.kind) ? opts.kind : 'meme';
++ await q(tx).exec('SELECT pg_advisory_xact_lock(hashtext($1)::bigint)', [`meme:title:${cleanKind}:${titleKey(cleanTitle)}`]);
++ const existing = await byTitleKey(cleanTitle, cleanKind, tx);
++ if (existing) {
++ const enriched = await enrich(existing.id, {
++ first_seen_at: opts.first_seen_at, last_seen_at: opts.last_seen_at, cites: opts.citations || [],
++ }, tx);
++ return { ...enriched, created: false };
++ }
++ const created = await createEntry({ ...opts, title: cleanTitle, kind: cleanKind }, tx);
++ return { ...created, created: true };
++ };
++ return t ? await run(t) : await s.tx(run);
++ }
++
/** Insert citations in the caller's transaction; the sha256 is computed from the verbatim quote. */
async function addCitations(entryId, cites = [], t = null) {
const run = async (tx) => {
@@@ -749,25 +810,26 @@ function createStore(s, { search }) {
}
/** Add uses to an existing candidate, extend its range and bump its revision only when something changed. */
-- async function enrich(entryId, { first_seen_at, last_seen_at, cites = [] }) {
-- return await s.tx(async (t) => {
-- const row = await byId(entryId, t);
++ async function enrich(entryId, { first_seen_at, last_seen_at, cites = [] }, t = null) {
++ const run = async (tx) => {
++ const row = await byId(entryId, tx);
if (!row) return null;
const first = iso(first_seen_at) || iso(row.first_seen_at);
const last = iso(last_seen_at) || iso(row.last_seen_at);
-- const added = await addCitations(entryId, cites, t);
++ const added = await addCitations(entryId, cites, tx);
const earlier = Date.parse(first) < Date.parse(iso(row.first_seen_at));
const later = Date.parse(last) > Date.parse(iso(row.last_seen_at));
if (!added.length && !earlier && !later) return { entry: wireEntry(row), added: 0, changed: false };
-- const updated = await t.maybe(
++ const updated = await tx.maybe(
`UPDATE meme_entries SET first_seen_at = LEAST(first_seen_at, $2::timestamptz), last_seen_at = GREATEST(last_seen_at, $3::timestamptz),
revision = revision + 1, updated_at = now() WHERE id = $1 RETURNING *`,
[entryId, first, last],
);
-- const all = await liveCitations(entryId, t);
-- if (updated.visibility === 'public') await search.sync(t, wireEntry(updated), all);
++ const all = await liveCitations(entryId, tx);
++ if (updated.visibility === 'public') await search.sync(tx, wireEntry(updated), all);
return { entry: wireEntry(updated), added: added.length, changed: true };
-- });
++ };
++ return t ? await run(t) : await s.tx(run);
}
/** A citation changed (a redirect, or a use went away) while the entry stayed public: a new revision and document. */
@@@ -1031,6 +1093,8 @@ function createStore(s, { search }) {
const webOrigin = await t.maybe('SELECT entry_id FROM meme_origins WHERE entry_id = $1', [row.id]);
if (!citationOrigin && !webOrigin) return refusal(409, 'meme.origin_gone', 'The origin citation is missing or gone; this entry cannot be published.');
if (row.visibility === 'public') return { ok: true, entry: wireEntry(row), unchanged: true };
++ const conflict = await publicTitleConflict(row.kind, row.title, row.id, t);
++ if (conflict) return refusal(409, 'meme.duplicate_title', `Another public entry already has this title and kind (/${conflict.slug}).`);
const updated = await t.maybe('UPDATE meme_entries SET visibility = $2, revision = revision + 1, updated_at = now() WHERE id = $1 RETURNING *', [row.id, 'public']);
const all = await liveCitations(row.id, t);
await search.sync(t, wireEntry(updated), all);
@@@ -1056,6 +1120,11 @@ function createStore(s, { search }) {
return await s.tx(async (t) => {
const row = await bySlug(slug, t);
if (!row) return refusal(404, 'meme.not_found', 'No such meme.');
++ if (row.visibility === 'public' && (v.title !== undefined || v.kind !== undefined)) {
++ const conflict = await publicTitleConflict(v.kind === undefined ? row.kind : v.kind,
++ v.title === undefined ? row.title : v.title, row.id, t);
++ if (conflict) return refusal(409, 'meme.duplicate_title', `Another public entry already has this title and kind (/${conflict.slug}).`);
++ }
const sets = [];
const args = [row.id];
for (const key of ['title', 'kind', 'status']) {
@@@ -1196,6 +1265,10 @@ function createStore(s, { search }) {
const sets = [];
const args = [entryId];
const cleanTitle = String(title == null ? '' : title).replace(/\s+/g, ' ').trim().slice(0, 160);
++ if (row.visibility === 'public' && cleanTitle && cleanTitle !== row.title) {
++ const conflict = await publicTitleConflict(row.kind, cleanTitle, row.id, t);
++ if (conflict) return { entry: wireEntry(row), changed: false, conflict: 'meme.duplicate_title' };
++ }
if (cleanTitle && cleanTitle !== row.title) { args.push(cleanTitle); sets.push(`title = $${args.length}`); }
const cleanSummary = summary == null ? null : String(summary).replace(/\s+/g, ' ').trim().slice(0, 600);
if (cleanSummary && cleanSummary !== row.summary) { args.push(cleanSummary); sets.push(`summary = $${args.length}`); }
@@@ -1315,9 +1388,9 @@ function createStore(s, { search }) {
return {
wireEntry, wireCitation, wireOrigin, wireSpread, wireVariation, wireExample, wireRelated, wireRevision,
wireTemplate, wireCaption, wireReport, wireJudgement,
-- bySlug, byId, citations, liveCitations, originOf, spreadOf, variationsOf, examplesOf, relatedOf, revisionsOf, detail,
++ bySlug, byId, byTitleKey, redirectFor, citations, liveCitations, originOf, spreadOf, variationsOf, examplesOf, relatedOf, revisionsOf, detail,
list, trending, week, series, showcase, uniqueSlug,
-- createCandidate, createEntry, addCitations, enrich, reindex, publish, hide, patch,
++ createCandidate, createEntry, upsertCandidate, addCitations, enrich, reindex, publish, hide, patch,
createProposal, addRevision, reviewRevision, pendingRevisions,
templates, templateById, addTemplate, createCaption, captionById, captionCountForTemplate,
castVote, createReport, listOpenReports,
@@@ -1338,7 +1411,7 @@ function slugify(title) {
}
module.exports = {
-- createStore, slugify, patchFields, entryFields, changeDiff, captionLines, windowStart,
++ createStore, slugify, titleKey, patchFields, entryFields, changeDiff, captionLines, windowStart,
wireEntry, wireCitation, wireOrigin, wireSpread, wireVariation, wireExample, wireRelated, wireRevision,
wireTemplate, wireCaption, wireReport, SLUG_RE, MEDIA_ID_RE,
KINDS, STATUSES, VISIBILITIES, SCOPES, LABELS, MODERATION_STATES, BLOCKED_WORDS,
ddiff --git a/test/helpers/boot.js b/test/helpers/boot.js
iindex 63693af..f635114 100644
---- a/test/helpers/boot.js
++++ b/test/helpers/boot.js
@@@ -45,7 +45,7 @@ async function boot(opts = {}) {
const config = configLib.load(env);
const { createStore } = require('../../server/db');
-- const testdb = await require('./db').testDb();
++ const testdb = await require('./db').testDb(opts.migrations ? { migrations: opts.migrations } : {});
const store = createStore(testdb.db, { now: opts.now });
const built = await createApp({ config, store, log, telemetrySink: opts.telemetrySink, accountSend: opts.accountSend,
media: opts.media, reddit: opts.reddit, ai: opts.ai, aiClient: opts.aiClient, aiBudget: opts.aiBudget,
@@@ -81,7 +81,7 @@ async function boot(opts = {}) {
}
const t = {
-- base, network, upstreams, ai, config, ctx: built.ctx, get, signIn, dbDump,
++ base, network, upstreams, ai, config, ctx: built.ctx, get, signIn, dbDump, testdb,
logs: () => captured.join('\n'),
async close() {
built.ctx.importer.stop();
ddiff --git a/test/helpers/db.js b/test/helpers/db.js
iindex 289a17b..c680ccb 100644
---- a/test/helpers/db.js
++++ b/test/helpers/db.js
@@@ -6,6 +6,7 @@
const { createTestDb, pgAvailable } = require('openvibe-sdk/testing');
const { MIGRATIONS } = require('../../server/db');
--const testDb = ({ store = process.env.MEME_TEST_STORE || 'pglite', max = 4 } = {}) => createTestDb({ migrations: MIGRATIONS, store, service: 'meme', max });
++const testDb = ({ store = process.env.MEME_TEST_STORE || 'pglite', max = 4, migrations = MIGRATIONS } = {}) =>
++ createTestDb({ migrations, store, service: 'meme', max });
module.exports = { testDb, pgAvailable };
ddiff --git a/test/importer.test.js b/test/importer.test.js
iindex 3b46066..ffa140c 100644
---- a/test/importer.test.js
++++ b/test/importer.test.js
@@@ -51,6 +51,30 @@ const { boot, check, done } = require('./helpers/boot');
assert.strictEqual(again.created, 0);
assert.strictEqual(Number(await t.ctx.s.db.value("SELECT COUNT(*)::int AS n FROM meme_entries WHERE slug = 'robot-is-offline'")), 1);
});
++
++ await check('two ingests of the same phrase make one entry, whatever the case, spaces or punctuation', async () => {
++ const before = await t.ctx.s.db.maybe("SELECT * FROM meme_entries WHERE slug = 'robot-is-offline'");
++ const cite = (id) => ({
++ source_service: 'chat', source_type: 'chat_message', source_id: String(id), chat_message_id: id,
++ quote_text: 'robot is offline', observed_at: '2026-10-24T12:00:00.000Z', url: 'https://openvibe.chat/',
++ });
++ const first = await t.ctx.entries.upsertCandidate({
++ slug: 'robot-is-offline-two', title: ' ROBOT is offline! ', kind: 'catchphrase',
++ visibility: 'hidden', first_seen_at: '2026-10-24T12:00:00.000Z', last_seen_at: '2026-10-24T12:00:00.000Z',
++ citations: [cite(9999)],
++ });
++ const second = await t.ctx.entries.upsertCandidate({
++ slug: 'robot-is-offline-three', title: 'robot is offline', kind: 'catchphrase',
++ visibility: 'hidden', first_seen_at: '2026-10-25T12:00:00.000Z', last_seen_at: '2026-10-25T12:00:00.000Z',
++ citations: [cite(10000)],
++ });
++ assert.strictEqual(first.created, false);
++ assert.strictEqual(second.created, false);
++ assert.strictEqual(first.entry.id, before.id);
++ assert.strictEqual(second.entry.id, before.id);
++ assert.strictEqual(Number(await t.ctx.s.db.value("SELECT COUNT(*)::int AS n FROM meme_entries WHERE title_key = 'robot is offline'")), 1);
++ assert.strictEqual(Number(await t.ctx.s.db.value("SELECT COUNT(*)::int AS n FROM meme_citations WHERE source_id IN ('9999', '10000')")), 2);
++ });
} finally {
await t.close();
}
ddiff --git a/test/merge-duplicates.test.js b/test/merge-duplicates.test.js
nnew file mode 100644
iindex 0000000..114b773
---- /dev/null
++++ b/test/merge-duplicates.test.js
@@@ -0,0 +1,146 @@
++'use strict';
++/**
++ * Migration 0009: a public entry's title is unique for its kind, and the duplicates that already exist are merged
++ * into the oldest entry. This test boots the schema as it stood before 0009 (migrations 0001–0008), creates the
++ * two-entry shape production had, applies 0009, and checks that everything moved, that the old address answers
++ * 301, and that a second public entry with the same normalised title is refused.
++ */
++const assert = require('assert');
++const fs = require('fs');
++const os = require('os');
++const path = require('path');
++const { ids } = require('openvibe-contracts');
++const { createDb } = require('openvibe-sdk/db');
++const { boot, check, done } = require('./helpers/boot');
++const { MIGRATIONS } = require('../server/db');
++
++const ulid = () => ids.ulid();
++
++(async () => {
++ const beforeDir = fs.mkdtempSync(path.join(os.tmpdir(), 'meme-0009-'));
++ for (const f of fs.readdirSync(MIGRATIONS)) {
++ if (/^000[1-8]_/.test(f)) fs.copyFileSync(path.join(MIGRATIONS, f), path.join(beforeDir, f));
++ }
++ const t = await boot({ migrations: beforeDir });
++ try {
++ const db = t.ctx.s.db;
++ const quiet = { log() {}, warn() {}, error() {}, info() {} };
++ // DDL is the owner's job: on the PostgreSQL test store the app's runtime role may not create functions or
++ // indexes. PGlite has one role, so its handle can apply the migration directly.
++ const applyMigration = async (sql) => {
++ if (!t.testdb.directUrl) return await db.query(sql);
++ const owner = createDb({ url: t.testdb.directUrl, service: 'meme-merge-owner', max: 1, log: quiet });
++ try { await owner.query(sql); } finally { await owner.close(); }
++ };
++ const now = new Date().toISOString();
++ const cite = (o) => ({ source_service: 'live', source_type: 'rundown_segment', url: 'https://openvibe.video/v/1?t=1', quote_text: 'keep friends close', observed_at: '2026-10-14T13:16:25.000Z', ...o });
++
++ const keeper = await t.ctx.entries.createCandidate({
++ slug: 'keep-friends-close', title: 'Keep friends close', kind: 'catchphrase', visibility: 'public',
++ first_seen_at: '2026-10-14T13:16:25.000Z', last_seen_at: '2026-10-19T13:25:56.000Z',
++ citations: [
++ cite({ role: 'origin', source_id: '1', quote_text: 'Keep friends close.', observed_at: '2026-10-14T13:16:25.000Z' }),
++ cite({ role: 'use', source_id: '2', quote_text: 'Keep friends close and enemies guessing.' }),
++ ],
++ });
++ const dup = await t.ctx.entries.createCandidate({
++ slug: 'close-and-enemies', title: 'Keep friends close and enemies guessing', kind: 'catchphrase', visibility: 'public',
++ first_seen_at: '2026-10-19T13:25:56.000Z', last_seen_at: '2026-10-19T13:25:56.000Z',
++ citations: [
++ cite({ role: 'origin', source_id: '3', quote_text: 'Keep friends close and enemies guessing.', observed_at: '2026-10-19T13:25:56.000Z' }),
++ cite({ role: 'use', source_id: '4', quote_text: 'Keep friends close and enemies guessing.' }),
++ // The keeper already has this moment: the merge must drop it, not violate the citation key.
++ cite({ role: 'use', source_id: '2', quote_text: 'Keep friends close and enemies guessing.' }),
++ ],
++ });
++ const third = await t.ctx.entries.createCandidate({
++ slug: 'third-meme', title: 'Third meme', kind: 'joke', visibility: 'public',
++ first_seen_at: now, last_seen_at: now, citations: [cite({ source_id: '5', quote_text: 'third' })],
++ });
++
++ // Give the keeper a different title so both rows can exist before 0009, like the production pair: two
++ // different phrases (and slugs) that the judge later rewrote into the same words.
++ await db.exec("UPDATE meme_entries SET title = 'Keep friends close and enemies guessing' WHERE id = $1", [keeper.entry.id]);
++ assert.strictEqual(Number(await db.value("SELECT COUNT(*)::int AS n FROM meme_entries WHERE visibility = 'public'")), 3);
++
++ const subjectA = `usr_${ulid()}`;
++ const subjectB = `usr_${ulid()}`;
++ await db.exec("INSERT INTO meme_votes (week, entry_id, subject) VALUES ('2026-W45', $1, $2)", [keeper.entry.id, subjectA]);
++ await db.exec("INSERT INTO meme_votes (week, entry_id, subject) VALUES ('2026-W45', $1, $2)", [dup.entry.id, subjectB]);
++ await db.exec("INSERT INTO meme_reports (id, entry_id, subject, kind, reason) VALUES ($1, $2, $3, 'correction', 'the quote is wrong')",
++ [`mrp_${ulid()}`, dup.entry.id, subjectA]);
++ await db.exec("INSERT INTO meme_status_events (entry_id, from_status, to_status, reason, changed_by, decision) VALUES ($1, 'peaked', 'peaked', 'judged', 'service:meme:judge', 'judge')", [dup.entry.id]);
++ await db.exec("INSERT INTO meme_judgements (entry_id, score, reason, title) VALUES ($1, 0.750, 'duplicate judgement', 'Keep friends close and enemies guessing')", [dup.entry.id]);
++ await db.exec("INSERT INTO meme_origins (entry_id, source, url, title) VALUES ($1, 'web', 'https://example.org/dup', 'dup origin')", [dup.entry.id]);
++ await db.exec("INSERT INTO meme_spread_points (entry_id, observed_at, place, url, count) VALUES ($1, now(), 'reddit', 'https://example.org/spread', 12)", [dup.entry.id]);
++ await db.exec("INSERT INTO meme_variations (entry_id, name) VALUES ($1, 'a variation')", [dup.entry.id]);
++ await db.exec("INSERT INTO meme_examples (entry_id, media_id) VALUES ($1, $2)", [dup.entry.id, `med_${ulid()}`]);
++ await db.exec("INSERT INTO meme_entry_related (entry_id, related_id) VALUES ($1, $2)", [dup.entry.id, keeper.entry.id]);
++ await db.exec("INSERT INTO meme_entry_related (entry_id, related_id) VALUES ($1, $2)", [dup.entry.id, third.entry.id]);
++ await db.exec("INSERT INTO meme_entry_related (entry_id, related_id) VALUES ($1, $2)", [keeper.entry.id, third.entry.id]);
++ await db.exec("INSERT INTO meme_entry_revisions (entry_id, revision, author, change, diff, moderation_state) VALUES ($1, 2, 'service:meme', '{}'::jsonb, '[]'::jsonb, 'approved')", [keeper.entry.id]);
++ await db.exec("INSERT INTO meme_entry_revisions (entry_id, revision, author, change, diff, moderation_state) VALUES ($1, 2, 'service:meme', '{}'::jsonb, '[]'::jsonb, 'approved')", [dup.entry.id]);
++ await db.exec("INSERT INTO meme_entry_revisions (entry_id, revision, author, change, diff, moderation_state) VALUES ($1, 3, 'service:meme', '{}'::jsonb, '[]'::jsonb, 'approved')", [dup.entry.id]);
++ await db.exec('UPDATE meme_entries SET revision = 3 WHERE id = $1', [dup.entry.id]);
++ await db.exec('UPDATE meme_entries SET revision = 2 WHERE id = $1', [keeper.entry.id]);
++
++ await check('0009 merges the public duplicates into the oldest id and moves everything to it', async () => {
++ const sql = fs.readFileSync(path.join(MIGRATIONS, '0009_entry_title_key.sql'), 'utf8');
++ await applyMigration(sql);
++
++ const rows = await db.many("SELECT id, slug, title, kind, revision FROM meme_entries WHERE visibility = 'public' ORDER BY slug");
++ assert.strictEqual(rows.length, 2, 'one public entry per title: ' + JSON.stringify(rows));
++ const kept = rows.find((r) => r.id === keeper.entry.id);
++ assert.ok(kept, 'the oldest id was kept');
++ assert.strictEqual(kept.slug, 'keep-friends-close');
++ assert.strictEqual(Number(await db.value('SELECT COUNT(*)::int AS n FROM meme_entries WHERE id = $1', [dup.entry.id])), 0);
++
++ const cites = await db.many('SELECT role, source_id FROM meme_citations WHERE entry_id = $1 ORDER BY id', [kept.id]);
++ assert.strictEqual(cites.length, 4, 'four distinct moments: ' + JSON.stringify(cites));
++ assert.strictEqual(cites.filter((c) => c.role === 'origin').length, 1, 'the keeper keeps one origin');
++ assert.strictEqual(cites.filter((c) => c.source_id === '2').length, 1, 'the shared moment was not copied twice');
++
++ assert.strictEqual(Number(await db.value('SELECT COUNT(*)::int AS n FROM meme_votes WHERE entry_id = $1', [kept.id])), 2);
++ assert.strictEqual(Number(await db.value('SELECT COUNT(*)::int AS n FROM meme_reports WHERE entry_id = $1', [kept.id])), 1);
++ assert.strictEqual(Number(await db.value('SELECT COUNT(*)::int AS n FROM meme_spread_points WHERE entry_id = $1', [kept.id])), 1);
++ assert.strictEqual(Number(await db.value('SELECT COUNT(*)::int AS n FROM meme_entry_related WHERE entry_id = $1', [kept.id])), 1);
++ assert.strictEqual(Number(await db.value('SELECT COUNT(*)::int AS n FROM meme_judgements WHERE entry_id = $1', [kept.id])), 1);
++ assert.strictEqual(Number(await db.value('SELECT COUNT(*)::int AS n FROM meme_origins WHERE entry_id = $1', [kept.id])), 1);
++ assert.strictEqual(Number(await db.value('SELECT COUNT(*)::int AS n FROM meme_entry_revisions WHERE entry_id = $1', [kept.id])), 4);
++ assert.strictEqual(Number(await db.value("SELECT COUNT(*)::int AS n FROM meme_status_events WHERE entry_id = $1 AND decision = 'merge'", [kept.id])), 1);
++ assert.strictEqual(Number(await db.value('SELECT COUNT(*)::int AS n FROM meme_entry_redirects WHERE to_entry_id = $1', [kept.id])), 1);
++
++ const redirect = await db.maybe('SELECT * FROM meme_entry_redirects WHERE from_slug = $1', ['close-and-enemies']);
++ assert.strictEqual(redirect.from_entry_id, dup.entry.id);
++ assert.strictEqual(Number(await db.value('SELECT COUNT(*)::int AS n FROM meme_outbox WHERE topic = $1 AND payload->>\'id\' = $2', ['meme.index_document.deleted', dup.entry.id])), 1);
++ });
++
++ await check('the old slug and the old id both answer 301 to the kept entry', async () => {
++ const page = await t.get('/m/close-and-enemies');
++ assert.strictEqual(page.status, 301);
++ assert.strictEqual(page.headers.get('location'), '/m/keep-friends-close');
++ const byId = await t.get(`/m/${dup.entry.id}`);
++ assert.strictEqual(byId.status, 301);
++ assert.strictEqual(byId.headers.get('location'), '/m/keep-friends-close');
++ const api = await t.get('/api/v1/entries/close-and-enemies');
++ assert.strictEqual(api.status, 301);
++ assert.strictEqual(api.headers.get('location'), '/api/v1/entries/keep-friends-close');
++ });
++
++ await check('a second public entry with the same title and kind is refused, and Trending lists it once', async () => {
++ const at = new Date().toISOString();
++ await assert.rejects(db.exec(
++ `INSERT INTO meme_entries (id, slug, title, kind, visibility, first_seen_at, last_seen_at)
++ VALUES ($1, 'another-keep-friends', 'KEEP friends close, and enemies guessing!', 'catchphrase', 'public', $2, $2)`,
++ [`mem_${ulid()}`, at],
++ ), /meme_entries_public_title_key|duplicate key|unique/i);
++ const trending = await t.ctx.entries.trending({ limit: 50 });
++ const keys = trending.map((e) => `${e.kind}:${e.title.toLowerCase()}`);
++ assert.strictEqual(new Set(keys).size, keys.length, 'no repeated title in Trending: ' + JSON.stringify(keys));
++ });
++ } finally {
++ await t.close();
++ fs.rmSync(beforeDir, { recursive: true, force: true });
++ }
++ done();
++})();
ddiff --git a/test/migration.test.js b/test/migration.test.js
iindex c1b9d1d..ccb9303 100644
---- a/test/migration.test.js
++++ b/test/migration.test.js
@@@ -12,7 +12,7 @@ const { boot, check, done } = require('./helpers/boot');
try {
await check('0001_initial.sql applies and creates the Meme tables', async () => {
const names = (await t.ctx.s.db.many("SELECT table_name AS name FROM information_schema.tables WHERE table_schema = current_schema() ORDER BY table_name")).map((r) => r.name);
-- for (const table of ['meme_entries', 'meme_citations', 'meme_status_events', 'meme_votes', 'meme_templates', 'meme_captions', 'meme_reports', 'meme_outbox', 'account_data_events', 'telemetry_samples']) {
++ for (const table of ['meme_entries', 'meme_citations', 'meme_status_events', 'meme_votes', 'meme_templates', 'meme_captions', 'meme_reports', 'meme_entry_redirects', 'meme_outbox', 'account_data_events', 'telemetry_samples']) {
assert.ok(names.includes(table), `missing ${table}`);
}
});
@@@ -55,6 +55,37 @@ const { boot, check, done } = require('./helpers/boot');
await db.exec("INSERT INTO meme_votes (week, entry_id, subject) VALUES ('2026-W44', $1, $2)", [eid, subject]);
await assert.rejects(db.exec("INSERT INTO meme_votes (week, entry_id, subject) VALUES ('2026-W44', $1, $2)", [eid, subject]), /duplicate key|unique/i);
});
++
++ await check('0009 makes one public entry per normalised title and kind', async () => {
++ const db = t.ctx.s.db;
++ const at = new Date().toISOString();
++ const id = () => `mem_${ids.ulid()}`;
++ await db.exec(`INSERT INTO meme_entries (id, slug, title, kind, visibility, first_seen_at, last_seen_at) VALUES ($1, 'same-title-a', 'Same Title', 'joke', 'public', $2, $2)`, [id(), at]);
++ await assert.rejects(db.exec(
++ `INSERT INTO meme_entries (id, slug, title, kind, visibility, first_seen_at, last_seen_at) VALUES ($1, 'same-title-b', 'same—title!', 'joke', 'public', $2, $2)`,
++ [id(), at],
++ ), /meme_entries_public_title_key|duplicate key|unique/i);
++ // The index is partial: a hidden candidate may still sit beside the public entry.
++ await db.exec(`INSERT INTO meme_entries (id, slug, title, kind, visibility, first_seen_at, last_seen_at) VALUES ($1, 'same-title-c', 'same title', 'joke', 'hidden', $2, $2)`, [id(), at]);
++ await db.exec(`UPDATE meme_entries SET title = 'Renamed & cleaned' WHERE slug = 'same-title-a'`);
++ assert.strictEqual(await db.value("SELECT title_key FROM meme_entries WHERE slug = 'same-title-a'"), 'renamed cleaned');
++ });
++
++ await check('publishing a second entry with the same title is a 409, not a database error', async () => {
++ await t.ctx.entries.createCandidate({
++ slug: 'same-title-hidden', title: 'Renamed — Cleaned!', kind: 'joke', visibility: 'hidden',
++ citations: [{
++ role: 'origin', source_service: 'chat', source_type: 'chat_message', source_id: '70001',
++ chat_message_id: 70001, quote_text: 'the same title', url: 'https://openvibe.chat/',
++ observed_at: '2026-10-20T10:00:00.000Z',
++ }],
++ });
++ const out = await t.ctx.entries.publish('same-title-hidden');
++ assert.strictEqual(out.ok, false);
++ assert.strictEqual(out.status, 409);
++ assert.strictEqual(out.code, 'meme.duplicate_title');
++ assert.strictEqual(await t.ctx.s.db.value("SELECT visibility FROM meme_entries WHERE slug = 'same-title-hidden'"), 'hidden');
++ });
} finally {
await t.close();
}