openvibers/openvibe.community Commits
Discord relay both ways, as an Events worker, with an external message map (roadmap WS-J tasks 5 and 6): replies follow new threads to Discord, edits PATCH and deletes (or gating a thread members-only) DELETE the Discord messages through relay_message_map (Discord message <-> thread/post per mapping, unique both ways; creates are sent with ?wait=true to learn the message id, and a mapping can post into a Discord thread via thread_id). Creates are queued by an Events worker that reads Community's own community.thread.* / community.post.* from OpenVibe.Events' pull API with a cursor kept in relay_cursors (moved in the same transaction as the deliveries; first start begins at the head; dedupe keys make replays a no-op); without EVENTS_URL / the client secret, or with DISCORD_RELAY_EVENTS=off, the forum queues them as before. Inbound: a Discord gateway client on Node 22's built-in WebSocket (IDENTIFY/RESUME, heartbeats with zombie detection, RECONNECT, INVALID_SESSION, fatal close codes; intents GUILD_MESSAGES | MESSAGE_CONTENT) behind DISCORD_RELAY_INBOUND=on and DISCORD_BOT_TOKEN turns replies to relayed messages in mapped channels into posts with origin discord (no author subject, the Discord name in posts.relay_author, a Discord badge), rate- and size-limited with mentions defused; edits and deletes on Discord follow through the map. Loop prevention both ways (origin discord never goes out; webhook, bot and system messages never come in). Dead letters: failed deliveries keep their error; staff retry or drop them, and inbound failures (ids and reason, never text) are listed and dismissed at /api/v1/relay (new: status, drop, inbound). Off by default and inert without the owner's webhooks, mappings and bot token; owner steps in docs/discord-relay.md. Schema: relay_deliveries rebuilt once (post_id, action, source, event_id, statuses dropped/skipped), relay_mappings gets discord_channel_id/discord_thread_id/inbound, posts gets relay_author, new relay_message_map, relay_cursors, relay_inbound_failures. Tests: relay.test.js, relay-events.test.js (fake Events pull server, the whole outbox -> Events -> worker -> Discord path), relay-inbound.test.js (fake gateway), shutdown.test.js
Changes
@@@ -394,9 +394,9 @@ by the API, the `members_only` box on the new-thread form, or the button on the- **Listings** keep a gated thread's title with `members_only` (never a body); gated spaces are listedwith their join link.- **Never public**: gated spaces and threads stay out of Pulse (not recorded, removed when gated later,-- and filtered at read time), the Discord relay (not queued; pending deliveries dropped; re-checked at-- send), the sitemap, the RSS feeds (a gated space's feed is 404) and JSON-LD; a member's view of a-- gated thread is `noindex` as well.++ and filtered at read time), the Discord relay (not queued; when gated later, what was waiting is++ skipped and what was sent is deleted on Discord; re-checked at send), the sitemap, the RSS feeds++ (a gated space's feed is 404) and JSON-LD; a member's view of a gated thread is `noindex` as well.- **Convergence**: answers are cached per viewer and resource (`server/vip/`, VIP's `createVipCache`):a "yes" at most `VIP_CACHE_TTL_MS` (30 s), a "no" 10 s, a failure 2 s. Community has no Events inbox,so **once VIP stops granting** (it applied Billing's `billing.entitlement.changed` and emitted@@@ -414,27 +414,37 @@ a score.## Discord relay--Outbound only, and off unless `DISCORD_RELAY_ENABLED=true`. When a thread is created in a--public space that has an enabled mapping, Community posts "New thread in s/<space> by <name>"--with the title, an excerpt and a link to the thread to that mapping's Discord webhook.----- A mapping (`relay_mappings`) names the **environment variable** that holds the webhook URL-- (`webhook_url_ref`, e.g. `DISCORD_WEBHOOK_FEEDBACK`); the URL itself never enters the-- database or any API response. Put the variable in `/etc/openvibe/community.env`. Only-- allow-listed names can be mapped or sent to: `DISCORD_WEBHOOK_*`, or exactly the names in-- `DISCORD_RELAY_WEBHOOK_VARS` when that is set — staff cannot point the relay at the URL in any-- other variable (an internal service's base URL).--- One delivery per (thread, mapping) — the dedupe key in `relay_deliveries`. Network errors,-- timeouts, 5xx, 429 (its `retry_after` honoured) and an unset variable are retried with-- exponential backoff (`DISCORD_RELAY_BACKOFF_MS` · 2^(attempt−1), at most an hour) up to-- `DISCORD_RELAY_MAX_ATTEMPTS`; other 4xx fail at once. Mentions are disabled.--- Loop prevention: a thread whose origin is `discord` is never relayed out (checked when-- queueing and when sending). Members/staff spaces are never relayed.--- Staff (admin/global_mod browsers, or services with `community.comment.moderate`):-- `GET /api/v1/relay/deliveries?status=failed` shows what failed and why,-- `POST /api/v1/relay/deliveries/:id/retry` queues one again,-- `GET|POST /api/v1/relay/mappings` and `PUT /api/v1/relay/mappings/:id { enabled }` manage-- mappings (`POST { space: 'feedback', webhook_url_ref: 'DISCORD_WEBHOOK_FEEDBACK' }`).++Off unless `DISCORD_RELAY_ENABLED=true`, and inert without the owner's webhooks, mappings and bot++token. Setup, owner steps, limits and operations: [docs/discord-relay.md](docs/discord-relay.md).++++- **Out**: a new thread in a mapped public space is announced through the mapping's Discord++ webhook ("New thread in s/<space> by <name>", title, excerpt, link); its replies follow into the++ same channel (or the Discord thread the mapping names); edits PATCH and deletes (or gating a++ thread members-only) DELETE the Discord messages through the **external message map**++ (`relay_message_map`, unique both ways; WS-J task 5). Creates are queued by the relay's++ **Events worker** from Community's own `community.thread.*` / `community.post.*` read back from++ OpenVibe.Events' pull API with a stored cursor (`relay_cursors`; WS-J task 6), or by the forum++ itself when the worker is off (no `EVENTS_URL` / client secret, or `DISCORD_RELAY_EVENTS=off`).++ Dedupe keys in `relay_deliveries` mean nothing is queued or posted twice.++- **In** (`DISCORD_RELAY_INBOUND=on`, `DISCORD_BOT_TOKEN`, `inbound: true` on the mapping): a++ Discord gateway client (Node 22's WebSocket; intents GUILD_MESSAGES and MESSAGE_CONTENT) turns++ replies to relayed messages in mapped channels into posts with origin `discord`, attributed to++ the Discord name only (never to anyone on the site), size- and rate-limited, mentions defused;++ edits and deletes on Discord follow through the map.++- **Loop prevention**: origin `discord` never goes out (checked when queueing and sending);++ webhook, bot and system messages never come in. Members/staff spaces and VIP members-only++ content never leave the site.++- **Secrets**: a mapping names the environment variable holding the webhook URL++ (`webhook_url_ref`, allow-listed `DISCORD_WEBHOOK_*` or exactly `DISCORD_RELAY_WEBHOOK_VARS`);++ the URL and the bot token never enter the database or any API response.++- **Retries and the dead letter**: network errors, timeouts, 5xx, 429 and an unset variable are++ retried with exponential backoff up to `DISCORD_RELAY_MAX_ATTEMPTS`, then the delivery is++ `failed` (the dead letter); other 4xx fail at once. Mentions are disabled on every message.++- **Staff** (admin/global_mod browsers, or services with `community.comment.moderate`):++ `GET /api/v1/relay/status` (queue, Events worker cursor/lag/errors/gaps, gateway state),++ `GET /api/v1/relay/deliveries?status=failed`, `POST /api/v1/relay/deliveries/:id/retry|drop`,++ `GET /api/v1/relay/inbound` and `POST /api/v1/relay/inbound/:id/dismiss` (inbound failures),++ `GET|POST /api/v1/relay/mappings`, `PUT /api/v1/relay/mappings/:id`.## Pulse@@@ -517,9 +527,13 @@ Copy `.env.example` to `.env` (production: `/etc/openvibe/community.env`, mode 0| `PASTES_AUTHORITY` | `live` | `live` = proxy to Live; `community` = this site's database is the authority || `COMMUNITY_DB_PATH` | `./data/community.db` | SQLite file (the systemd unit sets `/var/lib/openvibe-community/community.db`) || `API_CORS_ORIGINS` | Live, Media, Network, Tools, Games origins | Browser origins that may call `/api/v1/comments` and `/api/v1/pulse` with a Bearer JWT |--| `DISCORD_RELAY_ENABLED` | off | `true` turns the outbound Discord relay on |--| `DISCORD_RELAY_POLL_MS` / `DISCORD_RELAY_BACKOFF_MS` / `DISCORD_RELAY_MAX_ATTEMPTS` | `30000` / `30000` / `6` | Relay worker cadence, first retry delay, attempts before `failed` |++| `DISCORD_RELAY_ENABLED` | off | `true` turns the Discord relay on ([docs/discord-relay.md](docs/discord-relay.md)) |++| `DISCORD_RELAY_POLL_MS` / `DISCORD_RELAY_BACKOFF_MS` / `DISCORD_RELAY_MAX_ATTEMPTS` | `30000` / `30000` / `6` | Relay sender cadence, first retry delay, attempts before `failed` (the dead letter) || *(any name)* e.g. `DISCORD_WEBHOOK_FEEDBACK` | — | A Discord webhook URL, named by a relay mapping's `webhook_url_ref` |++| `DISCORD_RELAY_EVENTS` / `DISCORD_RELAY_EVENTS_POLL_MS` | on / `5000` | The relay's Events worker (needs `EVENTS_URL` and the client secret); `off` leaves creates to the forum |++| `DISCORD_RELAY_INBOUND` / `DISCORD_BOT_TOKEN` | off / — | Replies from Discord through the gateway (a bot with the MESSAGE CONTENT intent) |++| `DISCORD_GATEWAY_URL` | `wss://gateway.discord.gg/?v=10&encoding=json` | The Discord gateway |++| `DISCORD_RELAY_INBOUND_PER_MINUTE` / `DISCORD_RELAY_INBOUND_MAX_CHARS` | `6` / `4000` | Inbound limits per Discord author per mapping, and per message || `VIEW_HASH_SECRET` | derived from the client secret | Salt for hashed visitor ids in view counts || `COOKIE_SECURE` | `true` in production | Set `false` for plain-http local dev || `OV_VIP_INTERNAL_URL` | `http://127.0.0.1:4620` | OpenVibe.VIP's API (members-only spaces and threads) |@@@ -588,7 +602,8 @@ server/forum/ spaces/threads/posts: store, service, api (/api/v1/spaces, /posts), routes (pages)vip/ OpenVibe.VIP gate for members-only spaces/threads (index.js) + the vendored clientpulse/ Pulse read model: store, service (hooks + ingest), api (/api/v1/pulse)-- relay/ Discord relay: discord.js (queue, worker, backoff), api (/api/v1/relay)++ relay/ Discord relay: discord.js (queue, sender, backoff, message map), events-worker.js (creates++ from Events), discord-gateway.js + inbound.js (replies from Discord), api (/api/v1/relay)http/v1.js /api/v1 helpers: problem errors, capability guards, cursors, CORSidentity/capabilities.js capability checks incl. the proposed ids; discussion staff/moderatorsidentity/authors.js author display from subject_projection; the AI label@@@ -609,12 +624,13 @@ scripts/import-pastes.js Media paste bundle importerscripts/import-live-comments.js Live's VOD/clip comments → Community threads (dry run by default)test/ run.js + *.test.js (mock Live, Network and Media with a real RS256 key)docs/capabilities-proposal/ Wave 5 capability manifests (released in openvibe-contracts v0.7.0)++docs/discord-relay.md the Discord relay: how it works, owner steps, staff API, limits```## What is next--Spaces per streamer, game and project (with membership), inbound Discord relay (threads from--Discord arrive with origin `discord` and are never relayed back), visibility changes for++Spaces per streamer, game and project (with membership), the Discord relay's production round++trip (owner steps in [docs/discord-relay.md](docs/discord-relay.md)), visibility changes forcomment threads arriving through Events, moving paste comments onto the typed commentthreads, and submissions (clips, art, ideas, reports for community review).nnew file mode 100644@@@ -0,0 +1,164 @@++# Discord relay++++Community threads on Discord, and replies from Discord back as posts (roadmap WS-J tasks 5 and 6).++Everything is **off by default** and stays inert until the owner provides webhooks, a mapping and,++for the inbound half, a bot token.++++## What it does++++**Out** (`DISCORD_RELAY_ENABLED=true` plus a mapping):++++- A new thread in a mapped **public** space is announced through the mapping's Discord webhook:++ "New thread in s/<space> by <name>", the title, an excerpt of the opening post and the link.++- Each reply to a thread that went to Discord follows it into the same channel (or into the++ Discord thread the mapping names), with who replied, an excerpt and the link to the post.++- Editing the opening post edits the thread's Discord message; editing a reply edits its message.++- Deleting a reply or a thread (by its author or a moderator), or making a thread or space++ members-only (VIP), deletes the Discord messages the relay had sent for it. Anything still++ waiting to be sent is skipped.++- Mentions are disabled on every message (`allowed_mentions.parse = []`); names and titles are++ escaped, so nothing relayed can ping anyone.++++**In** (`DISCORD_RELAY_INBOUND=on`, `DISCORD_BOT_TOKEN`, and `inbound: true` on the mapping):++++- A reply on Discord to a relayed message (Discord's Reply), or any message in a Discord thread++ started from a relayed message, becomes a post in that Community thread. Other messages in++ the channel are ignored.++- The post has origin `discord`, belongs to nobody on the site (no author subject) and shows the++ Discord display name with a "Discord" badge. It is never relayed back out.++- Edits on Discord edit the post (a new revision, `edited_by` `discord`); deletes on Discord++ delete it. Deleting the relay's own announcement on Discord only marks it gone; the thread stays.++- The thread must still be public, open (not locked) and not members-only; otherwise the++ message is refused and the reason is recorded for staff.++- Limits: `DISCORD_RELAY_INBOUND_PER_MINUTE` (6) messages a minute per Discord author per++ mapping, ten times that per mapping; `DISCORD_RELAY_INBOUND_MAX_CHARS` (4000) per message,++ longer text is cut. User, role and channel mentions, custom emoji and timestamps become plain++ text, `@everyone`/`@here` are defused, attachments are noted ("2 attachments on Discord") but++ never fetched. The rest is Markdown, which the forum's safe renderer shows (no raw HTML).++++## How it works++++| Piece | File | Notes |++| --- | --- | --- |++| Queue and sender | `server/relay/discord.js` | `relay_deliveries`: one row per dedupe key, actions `create`, `edit`, `delete`. Creates are sent with `?wait=true`, so Discord answers with the message and its id goes into the map. |++| External message map (task 5) | `relay_message_map` | Every Discord message the relay knows, both ways (`out`: a thread or reply it sent; `in`: a reply that came from Discord). Unique both ways: a Discord message is one local object, and a local object is one message per mapping. Edits and deletes on either side find their counterpart here. |++| Events worker (task 6) | `server/relay/events-worker.js` | Reads Community's own `community.thread.*` and `community.post.*` from OpenVibe.Events (`GET /api/v1/events`, capability `events.event.read`) and queues the creates. Its cursor (`relay_cursors`) moves in the same transaction as the deliveries a page queued; a replay queues nothing twice (dedupe keys), and nothing already in the map is sent twice. The first start begins at Events' head: older threads are never announced. |++| Gateway | `server/relay/discord-gateway.js` | Discord gateway v10 over Node 22's built-in WebSocket: IDENTIFY/RESUME, heartbeats (a missing ACK means a zombied connection: resume), RECONNECT, INVALID_SESSION, backoff. Intents `GUILD_MESSAGES` and `MESSAGE_CONTENT`. |++| Inbound | `server/relay/inbound.js` | `MESSAGE_CREATE`, `MESSAGE_UPDATE`, `MESSAGE_DELETE`, `MESSAGE_DELETE_BULK` for mapped channels only. |++| Staff API | `server/relay/api.js` | `/api/v1/relay/*`, below. |++++When the Events worker is on, it is the one path for creates (the forum's own calls queue none).++Without `EVENTS_URL` and `OV_OAUTH_CLIENT_SECRET`, or with `DISCORD_RELAY_EVENTS=off`, the forum++queues creates itself, as the first relay did. Edits and deletes are always queued by the forum:++Contracts has no `community.thread`/`community.post` update or delete events yet.++++**Loop prevention**, both halves: a thread or post with origin `discord` is never queued or sent++out (checked when queueing and again when sending); inbound ignores webhook messages (the++relay's own among them), bots, the bot itself and system messages, and never takes a message++that is already in the map.++++**Retries and the dead letter**: network errors, timeouts, 5xx, 429 (its `retry_after`++honoured; the rest of that webhook's queue waits for the next pass) and an unset webhook variable++are retried with exponential backoff (`DISCORD_RELAY_BACKOFF_MS` · 2^(attempt−1), at most an hour)++up to `DISCORD_RELAY_MAX_ATTEMPTS`. Then the delivery is `failed`: the dead letter, kept with its++error, that staff retry or drop. Other 4xx fail at once. A delete that finds the message already++gone on Discord counts as delivered; an edit of a message that is gone is skipped.++++**Secrets never live in the database**: a mapping names the environment variable holding the++webhook URL (`webhook_url_ref`), and only allow-listed names (`DISCORD_WEBHOOK_*`, or exactly++`DISCORD_RELAY_WEBHOOK_VARS`) can be mapped. No API response carries a webhook URL or the bot token.++++## Staff API++++Staff only: an admin/global_mod browser session, or a service holding `community.comment.moderate`.++++| Route | What |++| --- | --- |++| `GET /api/v1/relay/status` | The queue by status (`failed` = dead letters), the Events worker (cursor, latest seq, lag, last error, last gap), the gateway (state, reconnects, last error) and open inbound failures |++| `GET /api/v1/relay/deliveries?status=&action=&limit=` | Deliveries; `status` pending, delivered, failed, dropped or skipped; `action` create, edit or delete |++| `POST /api/v1/relay/deliveries/:id/retry` | Queue it again now, with a fresh attempt budget |++| `POST /api/v1/relay/deliveries/:id/drop` | Give up on a pending or failed one (kept as `dropped`) |++| `GET /api/v1/relay/inbound?all=1` | Messages from Discord that did not become (or change) a post, and why: ids only, never the text |++| `POST /api/v1/relay/inbound/:id/dismiss` | Mark one seen |++| `GET /api/v1/relay/mappings` | Mappings: space, webhook variable (and whether it is set), Discord channel and thread ids, inbound |++| `POST /api/v1/relay/mappings` | `{ space, webhook_url_ref, enabled?, discord_channel_id?, discord_thread_id?, inbound? }` |++| `PUT /api/v1/relay/mappings/:id` | `{ enabled?, inbound?, discord_channel_id?, discord_thread_id? }` (`null` clears an id) |++++`GET /api/ready` shows the same summary under `discord_relay` while the relay is on.++++## Owner steps++++1. **Webhook per channel** (outbound). In Discord: the channel's settings → Integrations →++ Webhooks → New Webhook, name it (for example "OpenVibe.Community"), Copy Webhook URL. You need++ Manage Webhooks on that channel to do this; the bot does not.++2. **The bot** (inbound only). At <https://discord.com/developers/applications>: New Application →++ Bot → Reset Token (this is `DISCORD_BOT_TOKEN`). Under Privileged Gateway Intents turn on++ **Message Content Intent** (Presence and Server Members stay off). Invite it: OAuth2 → URL++ Generator → scope `bot`, permissions **View Channels** and **Read Message History** (permission++ integer `66560`). It sends nothing itself (the webhook does), so leave Send Messages and Manage++ Webhooks off. It must be able to see each mapped channel, and threads in it (public threads only).++3. **Environment** in `/etc/openvibe/community.env` (0600):++++ ```sh++ DISCORD_RELAY_ENABLED=true++ DISCORD_WEBHOOK_GENERAL=https://discord.com/api/webhooks/<id>/<token> # one per channel, any DISCORD_WEBHOOK_* name++ # inbound++ DISCORD_RELAY_INBOUND=on++ DISCORD_BOT_TOKEN=<the bot token>++ ```++++ `EVENTS_URL` and `OV_OAUTH_CLIENT_SECRET` are already set for the outbox, so the Events worker++ starts by itself. It needs the Network grant `community` → `events.event.read` on audience++ `openvibe.events` (in the Network's principal seed; check it is provisioned: a missing grant++ shows as a 403 in the worker's `last_error`).++ Then `sudo systemctl restart openvibe-community`. The boot log says++ `[Relay] Discord relay on: creates queued by the Events worker; events worker on; inbound on`.++4. **Map a space**, signed in as an admin on openvibe.community, from the browser console:++++ ```js++ await (await fetch('/api/v1/relay/mappings', { method: 'POST', headers: { 'content-type': 'application/json' },++ body: JSON.stringify({ space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL', inbound: true }) })).json();++ ```++++ `discord_channel_id` is learned from Discord's answer to the first message; set it yourself++ (Discord: Settings → Advanced → Developer Mode, then right-click the channel → Copy Channel ID)++ if you want replies accepted before anything was sent. To post into one Discord thread instead++ of the channel, add `discord_thread_id`. Only public spaces are ever relayed.++5. **One production round trip**: start a thread in the mapped space → it appears on Discord;++ reply to it on Discord (Reply, or in a thread on the message) → the reply appears in the++ thread with a Discord badge; edit and delete it on Discord → the post follows; reply and edit++ on the site → Discord follows; delete the thread → its messages leave Discord. Watch++ `GET /api/v1/relay/status` (no dead letters, lag 0, gateway `ready`).++++**Off again**: unset `DISCORD_RELAY_ENABLED` and restart (nothing is sent or read; the tables stay),++or per mapping `PUT /api/v1/relay/mappings/:id { "enabled": false }` (deletes still go out) or++`{ "inbound": false }`. Unset `DISCORD_RELAY_INBOUND` to close the gateway only.++++## Configuration++++| Variable | Default | What |++| --- | --- | --- |++| `DISCORD_RELAY_ENABLED` | off | The relay at all |++| `DISCORD_WEBHOOK_*` | — | Webhook URLs, named by mappings |++| `DISCORD_RELAY_WEBHOOK_VARS` | — | Exact variable names mappings may use (instead of `DISCORD_WEBHOOK_*`) |++| `DISCORD_RELAY_POLL_MS` / `DISCORD_RELAY_BACKOFF_MS` / `DISCORD_RELAY_MAX_ATTEMPTS` | `30000` / `30000` / `6` | Sender cadence, first retry delay, attempts before the dead letter |++| `DISCORD_RELAY_EVENTS` | on | `off` leaves creates to the forum instead of the Events worker |++| `DISCORD_RELAY_EVENTS_POLL_MS` | `5000` | How often the worker reads Events |++| `EVENTS_URL`, `OV_OAUTH_CLIENT_SECRET` | — | Events' address and Community's client secret (shared with the outbox) |++| `DISCORD_RELAY_INBOUND` | off | The gateway |++| `DISCORD_BOT_TOKEN` | — | The bot's token (inbound only) |++| `DISCORD_GATEWAY_URL` | `wss://gateway.discord.gg/?v=10&encoding=json` | The gateway |++| `DISCORD_RELAY_INBOUND_PER_MINUTE` | `6` | Messages a minute per Discord author per mapping (10× per mapping) |++| `DISCORD_RELAY_INBOUND_MAX_CHARS` | `4000` | Longest message taken in; longer is cut |++++## Known limits++++- Messages sent on Discord while Community is restarting (or the gateway is down longer than a++ resume allows) are not replayed: a new session starts from now.++- Private Discord threads, attachments (either way), reactions and Discord-side thread titles are++ not relayed.++- Threads created before the Events worker's first start are never announced, and replies go++ out only for threads that went out through that mapping.++- Edits made by the Roadmap sync (`docs/roadmap/public.json`) are not propagated to Discord.++- A retention gap in Events (the worker fell further behind than Events keeps) skips the creates++ in it; the gap is logged and shown in the status.@@@ -169,6 +169,7 @@ a.badge:hover { border-color: var(--accent); text-decoration: none; color: var(-.thread-meta { display: flex; flex-wrap: wrap; align-items: center; gap: 4px 6px; margin: 4px 0 0; font-size: .82rem; color: var(--text-secondary); }.pulse-source { flex: none; margin-top: 2px; }.badge-ai { border-color: var(--info); color: var(--info); }++.badge-relay { border-color: var(--accent-light); color: var(--accent-light); }.badge-vis { font-size: .7rem; vertical-align: middle; }/* Categories, requests and the roadmap (forum pages) */.badge-status, .badge-category { font-size: .7rem; vertical-align: middle; }@@@ -55,6 +55,9 @@ const { createForumRoutes } = require('./forum/routes');const { createPulse } = require('./pulse/service');const { createPulseApi } = require('./pulse/api');const { createDiscordRelay } = require('./relay/discord');++const { createRelayEventsWorker } = require('./relay/events-worker');++const { createDiscordGateway } = require('./relay/discord-gateway');++const { createDiscordInbound } = require('./relay/inbound');const { createRelayApi } = require('./relay/api');const { pulsePage } = require('./render/pulse');const { extensionFor } = require('./render/highlight');@@@ -124,6 +127,24 @@ function createApp(opts = {}) {webhookVars: config.discordRelay.webhookVars,...(opts.relayOptions || {}),});++ // Its Events worker (creates from community.thread.* / community.post.*) and the inbound gateway;++ // both off unless configured (relay/events-worker.js, relay/discord-gateway.js, relay/inbound.js).++ const relayInbound = relay.enabled ? createDiscordInbound({ db, perMinute: config.discordRelay.inboundPerMinute, maxChars: config.discordRelay.inboundMaxChars, ...(opts.inboundOptions || {}) }) : null;++ if (!opts.relay) {++ relay.attach({++ worker: createRelayEventsWorker({++ db, relay, enabled: config.discordRelay.events, eventsUrl: config.discordRelay.eventsUrl, clientSecret: config.oauth.clientSecret,++ tokenUrl: `${config.networkInternalUrl}/oauth/token`, clientId: config.oauth.clientId, pollMs: config.discordRelay.eventsPollMs, fetchImpl: opts.fetchImpl,++ ...(opts.relayWorkerOptions || {}),++ }),++ gateway: relayInbound && config.discordRelay.inbound ? createDiscordGateway({++ token: config.discordRelay.botToken, url: config.discordRelay.gatewayUrl,++ onDispatch: (type, data, ctx) => relayInbound.handle(type, data, ctx),++ ...(opts.gatewayOptions || {}),++ }) : null,++ inbound: relayInbound,++ });++ }// OpenVibe.VIP: members-only spaces and threads (fails closed without a client secret or VIP).const vip = opts.vip || createVipGate({ config, ...(opts.vipOptions || {}) });// Images on posts go to OpenVibe.Media's Object API as med_ objects (media/objects.js).@@@ -132,7 +153,7 @@ function createApp(opts = {}) {const community = config.pastesAuthority === 'community';const comments = createCommentService({ db, network, pastesLocal: community, limits: opts.commentLimits });seo.useForum(forum);-- Object.assign(app.locals, { db, network, pulse, relay, vip, forum, comments });++ Object.assign(app.locals, { db, network, pulse, relay, relayInbound, vip, forum, comments });if (opts.startRelay !== false) relay.start();// ── Paste authority ──────────────────────────────────────@@@ -189,7 +210,7 @@ function createApp(opts = {}) {app.use('/api/v1/spaces', createSpacesApi({ forum, viewers }));app.use('/api/v1/posts', createPostsApi({ forum, viewers }));app.use('/api/v1/space-groups', createGroupsApi({ forum, viewers }));-- app.use('/api/v1/relay', createRelayApi({ relay, db, viewers }));++ app.use('/api/v1/relay', createRelayApi({ relay, db, viewers, inbound: relayInbound }));app.get('/api/health', (_req, res) => res.json({ status: 'ok', service: 'openvibe-community', version: VERSION }));// GET /release.json (ADR-016) and POST /release-metrics: open tabs' update reports (a same-origin@@@ -76,7 +76,10 @@ module.exports = {unavailableTtlMs: parseInt(process.env.VIP_CACHE_UNAVAILABLE_TTL_MS, 10) || 2_000,},-- // Discord relay (outbound: new threads → a Discord webhook per mapped space). Off by default.++ // Discord relay (server/relay; docs/discord-relay.md). Off by default, and inert without an owner's++ // webhook variables, mappings and (for inbound) bot token.++ // out threads and replies in mapped public spaces → the mapping's Discord webhook; edits and deletes follow++ // in replies on Discord → posts (the gateway; DISCORD_RELAY_INBOUND=on and DISCORD_BOT_TOKEN)// Webhook URLs live in environment variables named by relay_mappings.webhook_url_ref.discordRelay: {enabled: /^(1|true|yes|on)$/i.test(process.env.DISCORD_RELAY_ENABLED || ''),@@@ -85,5 +88,16 @@ module.exports = {maxAttempts: parseInt(process.env.DISCORD_RELAY_MAX_ATTEMPTS, 10) || 6,// The only variables a mapping may name (comma-separated exact names); unset = DISCORD_WEBHOOK_*.webhookVars: (process.env.DISCORD_RELAY_WEBHOOK_VARS || '').split(',').map((s) => s.trim()).filter(Boolean),++ // The Events worker queues creates from community.thread.* / community.post.* (needs EVENTS_URL and++ // OV_OAUTH_CLIENT_SECRET, capability events.event.read); 'off' leaves them to the forum.++ events: !/^(0|false|no|off)$/i.test(process.env.DISCORD_RELAY_EVENTS || ''),++ eventsUrl: (process.env.EVENTS_URL || '').replace(/\/+$/, '') || null,++ eventsPollMs: parseInt(process.env.DISCORD_RELAY_EVENTS_POLL_MS, 10) || 5000,++ // Inbound through the Discord gateway (a bot in the server with the MESSAGE CONTENT intent).++ inbound: /^(1|true|yes|on)$/i.test(process.env.DISCORD_RELAY_INBOUND || ''),++ botToken: process.env.DISCORD_BOT_TOKEN || '',++ gatewayUrl: process.env.DISCORD_GATEWAY_URL || 'wss://gateway.discord.gg/?v=10&encoding=json',++ inboundPerMinute: parseInt(process.env.DISCORD_RELAY_INBOUND_PER_MINUTE, 10) || 6,++ inboundMaxChars: parseInt(process.env.DISCORD_RELAY_INBOUND_MAX_CHARS, 10) || 4000,},};@@@ -16,6 +16,26 @@ const fs = require('fs');const path = require('path');const Database = require('better-sqlite3');++// relay_deliveries' columns: the table in SCHEMA, and migrate()'s one-time rebuild of the older shape.++const RELAY_DELIVERIES_COLUMNS = `++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ thread_id INTEGER NOT NULL REFERENCES threads(id) ON DELETE CASCADE,++ post_id INTEGER REFERENCES posts(id) ON DELETE CASCADE,++ mapping_id INTEGER NOT NULL REFERENCES relay_mappings(id) ON DELETE CASCADE,++ action TEXT NOT NULL DEFAULT 'create' CHECK(action IN ('create', 'edit', 'delete')),++ dedupe_key TEXT UNIQUE NOT NULL,++ source TEXT NOT NULL DEFAULT 'direct' CHECK(source IN ('direct', 'events')),++ event_id TEXT,++ status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'delivered', 'failed', 'dropped', 'skipped')),++ attempts INTEGER NOT NULL DEFAULT 0,++ next_attempt_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,++ last_status INTEGER,++ last_error TEXT,++ delivered_at DATETIME,++ created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,++ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP++`;++const SCHEMA = `CREATE TABLE IF NOT EXISTS pastes (id INTEGER PRIMARY KEY AUTOINCREMENT,@@@ -242,9 +262,6 @@ CREATE TABLE IF NOT EXISTS thread_votes (PRIMARY KEY (thread_id, subject_id));-- Images attached to forum posts (WS-J task 2): the bytes are a Media object (med_…, tenant community,-- owned by the person); an upload waits here with no post until the thread or reply naming it is saved.CREATE TABLE IF NOT EXISTS attachments (@@@ -299,6 +316,11 @@ CREATE TABLE IF NOT EXISTS categories (UNIQUE (space_id, slug));++-- ── Discord relay (server/relay; DISCORD_RELAY_ENABLED) ──────++-- webhook_url_ref is the NAME of an environment variable holding the webhook URL: secrets never++-- live in the database. migrate() adds discord_channel_id (the channel the webhook posts into;++-- learned from Discord's answer when unset), discord_thread_id (post into this Discord thread) and++-- inbound (replies from Discord come back as posts).CREATE TABLE IF NOT EXISTS relay_mappings (id INTEGER PRIMARY KEY AUTOINCREMENT,space_id INTEGER NOT NULL REFERENCES spaces(id) ON DELETE CASCADE,@@@ -309,23 +331,63 @@ CREATE TABLE IF NOT EXISTS relay_mappings (UNIQUE (space_id, direction, webhook_url_ref));--CREATE TABLE IF NOT EXISTS relay_deliveries (++-- One outbound action per dedupe key (WS-J task 6): send a thread's or a reply's Discord message++-- (create), edit it or delete it. post_id NULL: the thread's own message. source: queued by the++-- Events worker from community.thread.* / community.post.* (event_id), or directly by the forum.++-- status pending → delivered | failed (the dead letter: retries used up, or refused; staff retry or++-- drop it) | dropped (by staff) | skipped (nothing left to do, e.g. deleted before it was sent).++CREATE TABLE IF NOT EXISTS relay_deliveries (${RELAY_DELIVERIES_COLUMNS});++CREATE INDEX IF NOT EXISTS idx_relay_deliveries_due ON relay_deliveries(status, next_attempt_at);++++-- External message map (WS-J task 5): every Discord message the relay knows, both ways.++-- out a thread (local_type 'thread': title and opening post) or a reply sent through the mapping's webhook++-- in a reply written on Discord, now a post with origin 'discord' (local_type 'post')++-- external_channel_id is where the message is (a Discord thread's id for one in a thread);++-- external_thread_id is set when the relay sent it into a Discord thread (edits and deletes name it).++-- Unique both ways: a Discord message is one local object; a local object is one message per mapping.++-- Edits and deletes on either side find their counterpart here.++CREATE TABLE IF NOT EXISTS relay_message_map (id INTEGER PRIMARY KEY AUTOINCREMENT,-- thread_id INTEGER NOT NULL REFERENCES threads(id) ON DELETE CASCADE,++ platform TEXT NOT NULL DEFAULT 'discord' CHECK(platform IN ('discord')),mapping_id INTEGER NOT NULL REFERENCES relay_mappings(id) ON DELETE CASCADE,-- dedupe_key TEXT UNIQUE NOT NULL,-- status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'delivered', 'failed')),-- attempts INTEGER NOT NULL DEFAULT 0,-- next_attempt_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,-- last_status INTEGER,-- last_error TEXT,-- delivered_at DATETIME,++ direction TEXT NOT NULL CHECK(direction IN ('out', 'in')),++ local_type TEXT NOT NULL CHECK(local_type IN ('thread', 'post')),++ local_id INTEGER NOT NULL,++ thread_id INTEGER NOT NULL REFERENCES threads(id) ON DELETE CASCADE,++ external_channel_id TEXT NOT NULL,++ external_thread_id TEXT,++ external_message_id TEXT NOT NULL,++ external_webhook_id TEXT,++ external_deleted_at DATETIME,created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,-- UNIQUE (thread_id, mapping_id)++ UNIQUE (platform, external_message_id),++ UNIQUE (platform, mapping_id, local_type, local_id)++);++CREATE INDEX IF NOT EXISTS idx_relay_message_map_thread ON relay_message_map(thread_id);++++-- The relay's Events worker's place in OpenVibe.Events (server/relay/events-worker.js): the last seq++-- it handled, moved in the same transaction as the deliveries that page queued.++CREATE TABLE IF NOT EXISTS relay_cursors (++ name TEXT PRIMARY KEY,++ cursor INTEGER NOT NULL,++ latest_seq INTEGER,++ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP++);++++-- Messages from Discord that did not become (or change) a post, for staff (GET /api/v1/relay/inbound).++-- Ids and the reason only, never the text.++CREATE TABLE IF NOT EXISTS relay_inbound_failures (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ mapping_id INTEGER REFERENCES relay_mappings(id) ON DELETE SET NULL,++ event TEXT NOT NULL,++ external_channel_id TEXT,++ external_message_id TEXT,++ external_author_id TEXT,++ error TEXT NOT NULL,++ dismissed_at DATETIME,++ created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP);--CREATE INDEX IF NOT EXISTS idx_relay_deliveries_due ON relay_deliveries(status, next_attempt_at);-- ── Pulse (server/pulse): public activity across the network, with provenance ──-- Only public things enter; AI items carry origin 'ai' and never an actor.@@@ -463,6 +525,28 @@ function migrate(db) {['roadmap', 'features', 'Features', 'New things on sites that are already open.', 2],['roadmap', 'platform', 'Under the hood', 'Accounts, safety, reliability and the shared systems every site uses.', 3],]) seedCategory.run(slug, name, description, position, space);++ // The Discord relay both ways (WS-J tasks 5 and 6; server/relay): where a mapping's webhook posts and++ // whether replies come back, the Discord name on posts from Discord, and relay_deliveries rebuilt once++ // for replies, edits and deletes (the older table allowed one row per (thread, mapping) and no actions).++ const mappingCols = new Set(db.prepare('PRAGMA table_info(relay_mappings)').all().map((x) => x.name));++ if (!mappingCols.has('discord_channel_id')) db.exec('ALTER TABLE relay_mappings ADD COLUMN discord_channel_id TEXT');++ if (!mappingCols.has('discord_thread_id')) db.exec('ALTER TABLE relay_mappings ADD COLUMN discord_thread_id TEXT');++ if (!mappingCols.has('inbound')) db.exec('ALTER TABLE relay_mappings ADD COLUMN inbound INTEGER NOT NULL DEFAULT 0');++ const postCols = new Set(db.prepare('PRAGMA table_info(posts)').all().map((x) => x.name));++ if (!postCols.has('relay_author')) db.exec('ALTER TABLE posts ADD COLUMN relay_author TEXT');++ const deliveryCols = new Set(db.prepare('PRAGMA table_info(relay_deliveries)').all().map((x) => x.name));++ if (!deliveryCols.has('action')) {++ db.transaction(() => {++ db.exec(`CREATE TABLE relay_deliveries_v2 (${RELAY_DELIVERIES_COLUMNS});++ INSERT INTO relay_deliveries_v2 (id, thread_id, mapping_id, dedupe_key, status, attempts, next_attempt_at, last_status, last_error, delivered_at, created_at, updated_at)++ SELECT id, thread_id, mapping_id, dedupe_key, status, attempts, next_attempt_at, last_status, last_error, delivered_at, created_at, updated_at FROM relay_deliveries;++ DROP TABLE relay_deliveries;++ ALTER TABLE relay_deliveries_v2 RENAME TO relay_deliveries;++ CREATE INDEX IF NOT EXISTS idx_relay_deliveries_due ON relay_deliveries(status, next_attempt_at);`);++ })();++ }++ db.exec(`CREATE INDEX IF NOT EXISTS idx_relay_deliveries_thread ON relay_deliveries(thread_id, post_id);++ CREATE INDEX IF NOT EXISTS idx_relay_mappings_channel ON relay_mappings(discord_channel_id) WHERE discord_channel_id IS NOT NULL;`);}/** cth_ + 22 base64url characters (16 random bytes). */@@@ -15,8 +15,10 @@* own posts; moderators (identity/capabilities.js discussionModerator) any, and pin/lock.* Edits keep every revision in post_versions.*-- * Side effects of a new thread/post: a Pulse item (public spaces) and, for threads, a Discord-- * relay delivery per mapping (relay/discord.js; off unless DISCORD_RELAY_ENABLED).++ * Side effects of a new thread/post: a Pulse item (public spaces) and a Discord relay delivery per++ * mapping (relay/discord.js; off unless DISCORD_RELAY_ENABLED; while its Events worker runs, it++ * queues creates from community.thread/post.created instead). Edits, deletes and gating queue the++ * matching Discord edit or delete for whatever the relay had sent.** Members-only (OpenVibe.VIP): a space or a single thread can be for one creator's VIP members* (members_only_owner = the creator's usr_ subject). Reading its threads and posts, starting a@@@ -282,7 +284,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, viattachments: deleted ? [] : ((attachments && attachments.get(p.id)) || []),pastes: deleted ? [] : ((pastes && pastes.get(p.id)) || []),id: p.id, thread_id: p.thread_id, is_opening: !!p.is_opening, origin: p.origin,-- author: deleted ? null : authors.author(p.author_subject, p.origin, projections),++ author: deleted ? null : authors.author(p.author_subject, p.origin, projections, p.relay_author),body_markdown: deleted ? null : p.body_markdown,body_html: deleted ? null : renderMarkdown(p.body_markdown),revision: p.revision, deleted,@@@ -311,14 +313,14 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi};}-- /** Newly gated threads leave Pulse and the Discord relay queue at once (reads re-check as well). */++ /**++ * Newly gated threads leave Pulse at once, and Discord: what the relay had not sent yet is skipped and++ * what it had sent is deleted there (reads and sends re-check as well).++ */function hideGated(threadIds) {if (!threadIds.length) return;if (pulse) hook(() => { for (const id of threadIds) { pulse.threadGone(id); for (const p of db.prepare('SELECT id FROM posts WHERE thread_id = ?').all(id)) pulse.postGone(p.id); } });-- hook(() => {-- const del = db.prepare("DELETE FROM relay_deliveries WHERE thread_id = ? AND status = 'pending'");-- for (const id of threadIds) del.run(id);-- });++ if (relay) hook(() => relay.hideThreads(threadIds, 'made members-only'));}function removeThread(v, thread) {@@@ -329,6 +331,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, viif (!mine) events.moderationAction(v, 'thread.deleted', { type: 'thread', id: String(thread.id), owner_subject: thread.author_subject || null });})();if (pulse) hook(() => { pulse.threadGone(thread.id); for (const p of db.prepare('SELECT id FROM posts WHERE thread_id = ?').all(thread.id)) pulse.postGone(p.id); });++ if (relay) hook(() => relay.enqueueDelete(thread.id, null, mine ? 'deleted by its author' : 'deleted by a moderator'));return { ok: true, id: thread.id, deleted: 'thread' };}@@@ -586,7 +589,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, viconst { post, thread, space } = postFor(v, postId);await requireMembership(v, space, thread);const projections = await authors.projectionsFor([post.author_subject]);-- const a = authors.author(post.author_subject, post.origin, projections);++ const a = authors.author(post.author_subject, post.origin, projections, post.relay_author);const name = a ? (a.username ? `@${a.username}` : a.display_name || 'Someone') : 'Someone';const quoted = String(post.body_markdown || '').split('\n').slice(0, 40).map((l) => `> ${l}`).join('\n');return { markdown: `**${name}** wrote:\n${quoted}\n\n` };@@@ -749,6 +752,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, viattach(images, post.id);attachPastes(pasteRows, post.id);if (pulse) hook(() => pulse.postCreated(post, thread, space));++ if (relay) hook(() => relay.enqueuePost(post, thread, space));const projections = await authors.projectionsFor([post.author_subject]);// Where the new post lands: its page in the thread (posts are numbered in id order).const position = db.prepare('SELECT COUNT(*) AS c FROM posts WHERE thread_id = ? AND id <= ?').get(thread.id, post.id).c;@@@ -764,6 +768,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, viif (!mine && !moderator(v)) fail(403, 'post.not_yours', 'Only the author or a moderator edits a post');if (thread.locked && !moderator(v)) fail(403, 'thread.locked', 'This thread is locked');const next = store.editPost(db, post.id, cleanBody(body.body != null ? body.body : body.body_markdown), v.subject || v.service || null);++ if (relay && next && next.revision !== post.revision) hook(() => relay.enqueueEdit(next));const projections = await authors.projectionsFor([next.author_subject]);return { post: shapePost(next, v, projections) };},@@@ -787,6 +792,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, viif (!(person(v) && post.author_subject === v.subject)) events.moderationAction(v, 'post.deleted', { type: 'post', id: String(post.id), owner_subject: post.author_subject || null }, { details: { thread: String(thread.id) } });})();if (pulse) hook(() => pulse.postGone(post.id));++ if (relay) hook(() => relay.enqueueDelete(thread.id, post.id, person(v) && post.author_subject === v.subject ? 'deleted by its author' : 'deleted by a moderator'));return { ok: true, id: post.id };},@@@ -239,11 +239,14 @@ function getPost(db, id) {return db.prepare('SELECT * FROM posts WHERE id = ?').get(id) || null;}--/** A reply; bumps the thread's reply_count and last activity. */--function addPost(db, { thread_id, author_subject = null, origin = 'user', body_markdown }) {++/**++ * A reply; bumps the thread's reply_count and last activity. relay_author: the name a reply written++ * on Discord carries (origin 'discord', no author subject; server/relay/inbound.js).++ */++function addPost(db, { thread_id, author_subject = null, origin = 'user', body_markdown, relay_author = null }) {return db.transaction(() => {-- const info = db.prepare('INSERT INTO posts (thread_id, author_subject, origin, body_markdown) VALUES (?, ?, ?, ?)')-- .run(thread_id, author_subject, origin, body_markdown);++ const info = db.prepare('INSERT INTO posts (thread_id, author_subject, origin, body_markdown, relay_author) VALUES (?, ?, ?, ?, ?)')++ .run(thread_id, author_subject, origin, body_markdown, relay_author);db.prepare('UPDATE threads SET reply_count = reply_count + 1, last_activity_at = CURRENT_TIMESTAMP WHERE id = ?').run(thread_id);const post = getPost(db, info.lastInsertRowid);const thread = getThread(db, thread_id);@@@ -20,10 +20,12 @@ function createAuthors({ db, network = null }) {/*** { subject, username, display_name, avatar_url, profile_color } for a person, a fixed AI-- * label for AI output, null for anonymous writes.++ * label for AI output, the Discord name (relayName; is_relay) for a reply written on Discord,++ * null for anonymous writes.*/-- function author(subject, origin, projections) {++ function author(subject, origin, projections, relayName = null) {if (origin === 'ai') return { subject: null, username: null, display_name: AI_DISPLAY_NAME, avatar_url: null, profile_color: null, is_ai: true };++ if (origin === 'discord') return { subject: null, username: null, display_name: relayName || 'Discord', avatar_url: null, profile_color: null, is_relay: true, relay: 'discord' };if (origin === 'system' && !subject) return { subject: null, username: null, display_name: 'OpenVibe', avatar_url: null, profile_color: null, is_system: true };if (!subject) return null;const p = projections.get(subject);@@@ -8,6 +8,11 @@ const config = require('./config');const { createApp } = require('./app');const app = createApp();++if (config.discordRelay.enabled) {++ const rs = app.locals.relay.status();++ const part = (x) => (x.enabled ? 'on' : `off (${x.reason})`);++ console.log(`[Relay] Discord relay on: creates queued by the ${rs.creates_from === 'events' ? 'Events worker' : 'forum'}; events worker ${part(rs.events_worker)}; inbound ${part(rs.inbound)}`);++}// Community → OpenVibe.Events (server/events.js): off unless EVENTS_URL and the client secret are set.try { require('./events').init(require('./db').getDb()); } catch (err) { console.warn('[Events] not started:', err.message); }// The Roadmap space follows docs/roadmap/public.json (server/forum/roadmap.js).@@@ -39,8 +44,9 @@ try { searchDocs = require('./search/documents').createSearchDocuments({ db: req// SIGTERM: the Pulse subscription retries, the community.profile scan and the search-document scans// stop (nothing new starts); the server stops taking connections, closes idle keep-alive ones (it keeps// them 65 s otherwise) and lets requests in flight finish (4 s at most); then the profile scan in--// progress, the Discord relay's drain and the events outbox's send finish (unsent rows stay in their--// tables for the next start), community.db closes, and the process exits 0, within the manifest's 5 s.++// progress, the Discord relay's drain and its Events worker's page (the gateway closes at once) and the++// events outbox's send finish (unsent rows stay in their tables for the next start), community.db++// closes, and the process exits 0, within the manifest's 5 s.const { within } = require('./graceful');let profilesDone = null;require('./graceful').gracefulStop({@@@ -56,7 +62,13 @@ require('./graceful').gracefulStop({() => within(1500, require('./events').stop()),() => {const ev = require('./events').status();-- console.log(`[Community] stopped: subscriptions ${subscriptions ? 'stopped' : 'off'}, profile scan ${profiles && profiles.enabled ? 'stopped' : 'off'}, search scans ${searchDocs ? 'stopped' : 'off'}, relay ${config.discordRelay.enabled ? 'stopped' : 'off'}, outbox ${ev.enabled ? `stopped (${ev.pending} pending)` : 'off'}`);++ let relay = 'off';++ if (config.discordRelay.enabled) {++ let rs = null;++ try { rs = app.locals.relay.status(); } catch { /* the database is gone */ }++ relay = rs ? `stopped (events worker ${rs.events_worker.enabled ? 'stopped' : 'off'}, gateway ${rs.inbound.enabled ? 'stopped' : 'off'})` : 'stopped';++ }++ console.log(`[Community] stopped: subscriptions ${subscriptions ? 'stopped' : 'off'}, profile scan ${profiles && profiles.enabled ? 'stopped' : 'off'}, search scans ${searchDocs ? 'stopped' : 'off'}, relay ${relay}, outbox ${ev.enabled ? `stopped (${ev.pending} pending)` : 'off'}`);require('./db').closeDb();},],@@@ -57,9 +57,10 @@ function createCommunityReadiness({ db, auth, config, relay = null, release = nudetails: (body) => {const out = { pastes_authority: config.pastesAuthority };if (relay && relay.enabled && body.checks.db.status === 'ok') {-- const q = {};-- for (const r of db.prepare('SELECT status, COUNT(*) AS n FROM relay_deliveries GROUP BY status').all()) q[r.status] = r.n;-- out.discord_relay = { enabled: true, deliveries: q };++ // The queue by status (failed = dead letters), and whether the Events worker and the inbound gateway run.++ const st = relay.status();++ const brief = (x) => (x.enabled ? { enabled: true, state: x.state || (x.running ? 'running' : 'stopped'), last_error: x.last_error || null, ...(x.lag != null ? { lag: x.lag } : {}) } : { enabled: false, reason: x.reason });++ out.discord_relay = { enabled: true, deliveries: st.deliveries, creates_from: st.creates_from, events_worker: brief(st.events_worker), inbound: brief(st.inbound), inbound_failures: st.inbound_failures };}return out;},@@@ -4,21 +4,29 @@* /api/v1/relay — Discord relay administration, staff only (an admin/global_mod browser, or a* service holding community.comment.moderate: acting as itself, or vouching with X-OV-Staff).*-- * GET /deliveries?status=pending|delivered|failed&limit= what was sent, what is failing and why-- * POST /deliveries/:id/retry queue a failed delivery again-- * GET /mappings space → webhook variable name-- * POST /mappings { space, webhook_url_ref, enabled? } map a space (the URL stays in the env; allow-listed names only)-- * PUT /mappings/:id { enabled }++ * GET /status the queue by status, the Events worker (cursor, lag,++ * last error, gaps) and the inbound gateway's state++ * GET /deliveries?status=pending|delivered|failed|dropped|skipped&action=create|edit|delete&limit=++ * what was sent, what is failing and why (failed = the dead letters)++ * POST /deliveries/:id/retry queue a delivery again, with a fresh attempt budget++ * POST /deliveries/:id/drop give up on a waiting or failed one (kept as 'dropped')++ * GET /inbound?all=1&limit= messages from Discord that did not become posts, and why++ * POST /inbound/:id/dismiss mark one seen++ * GET /mappings space → webhook variable name, Discord channel/thread, inbound++ * POST /mappings { space, webhook_url_ref, enabled?, discord_channel_id?, discord_thread_id?, inbound? }++ * map a space (the URL stays in the env; allow-listed names only)++ * PUT /mappings/:id { enabled?, inbound?, discord_channel_id?, discord_thread_id? }*-- * Responses never contain a webhook URL, only the variable name and whether it is set.++ * Responses never contain a webhook URL or the bot token, only variable names and whether they are set.*/const express = require('express');const contracts = require('openvibe-contracts');const { run, jsonBody, fail, intIn } = require('../http/v1');const { discussionModerator } = require('../identity/capabilities');const forumStore = require('../forum/store');++const { SNOWFLAKE, DELIVERY_STATUSES } = require('./discord');--function createRelayApi({ relay, db, viewers }) {++function createRelayApi({ relay, db, viewers, inbound = null }) {const router = express.Router();router.use(contracts.http.middleware());router.use(viewers.middleware());@@@ -31,15 +39,38 @@ function createRelayApi({ relay, db, viewers }) {});const idOf = (req) => (/^\d{1,15}$/.test(req.params.id) ? Number(req.params.id) : fail(404, 'route.not_found', 'Not found'));++ /** Discord ids in a mapping body: digits, or null/'' to clear; undefined when absent. */++ function discordIds(b) {++ const out = {};++ for (const k of ['discord_channel_id', 'discord_thread_id']) {++ if (b[k] === undefined) continue;++ if (b[k] !== null && b[k] !== '' && !SNOWFLAKE.test(String(b[k]))) fail(400, 'relay.invalid_discord_id', `${k} is a Discord id (15 to 21 digits), or null`);++ out[k] = b[k] === '' ? null : b[k];++ }++ return out;++ }++ router.get('/status', run(() => relay.status()));router.get('/deliveries', run((req) => {-- const status = ['pending', 'delivered', 'failed'].includes(req.query.status) ? req.query.status : null;-- return { enabled: relay.enabled, deliveries: relay.listDeliveries({ status, limit: intIn(req.query.limit, 50, 1, 200) }) };++ const status = DELIVERY_STATUSES.includes(req.query.status) ? req.query.status : null;++ const action = ['create', 'edit', 'delete'].includes(req.query.action) ? req.query.action : null;++ return { enabled: relay.enabled, deliveries: relay.listDeliveries({ status, action, limit: intIn(req.query.limit, 50, 1, 200) }) };}));router.post('/deliveries/:id/retry', run((req) => {if (!relay.retry(idOf(req))) fail(404, 'relay.delivery_not_found', 'No such delivery waiting to be sent');return { ok: true };}));++ router.post('/deliveries/:id/drop', run((req) => {++ if (!relay.drop(idOf(req))) fail(404, 'relay.delivery_not_found', 'No such pending or failed delivery');++ return { ok: true };++ }));++ router.get('/inbound', run((req) => ({++ enabled: !!inbound, failures: inbound ? inbound.listFailures({ all: req.query.all === '1', limit: intIn(req.query.limit, 50, 1, 200) }) : [],++ })));++ router.post('/inbound/:id/dismiss', run((req) => {++ if (!inbound || !inbound.dismiss(idOf(req))) fail(404, 'relay.inbound_not_found', 'No such inbound failure to dismiss');++ return { ok: true };++ }));router.get('/mappings', run(() => ({ enabled: relay.enabled, mappings: relay.listMappings() })));router.post('/mappings', jsonBody, run((req) => {const b = req.body || {};@@@ -48,10 +79,15 @@ function createRelayApi({ relay, db, viewers }) {const ref = String(b.webhook_url_ref || '');if (!relay.ENV_NAME.test(ref)) fail(400, 'relay.invalid_ref', 'webhook_url_ref is the NAME of an environment variable (A-Z, 0-9, _), never the URL');if (!relay.refAllowed(ref)) fail(400, 'relay.ref_not_allowed', 'webhook_url_ref must be an allowed webhook variable (DISCORD_RELAY_WEBHOOK_VARS, else DISCORD_WEBHOOK_*)');-- return { mapping: relay.addMapping({ space_id: space.id, webhook_url_ref: ref, enabled: b.enabled !== false }) };++ const ids = discordIds(b);++ return { mapping: relay.addMapping({ space_id: space.id, webhook_url_ref: ref, enabled: b.enabled !== false, ...ids, inbound: b.inbound === undefined ? undefined : !!b.inbound }) };}, 201));router.put('/mappings/:id', jsonBody, run((req) => {-- const m = relay.setMappingEnabled(idOf(req), !!(req.body || {}).enabled);++ const b = req.body || {};++ const fields = { ...discordIds(b) };++ if (b.enabled !== undefined) fields.enabled = !!b.enabled;++ if (b.inbound !== undefined) fields.inbound = !!b.inbound;++ const m = relay.updateMapping(idOf(req), fields);if (!m) fail(404, 'relay.mapping_not_found', 'No such mapping');return { mapping: m };}));nnew file mode 100644@@@ -0,0 +1,239 @@++'use strict';++++/**++ * A minimal Discord gateway client (API v10, JSON, no compression) for the relay's inbound side++ * (roadmap WS-J task 6; DISCORD_RELAY_INBOUND=on with DISCORD_BOT_TOKEN).++ *++ * One session: HELLO → IDENTIFY (or RESUME on resume_gateway_url after a drop); heartbeats at the++ * interval Discord asks for, the first one jittered, and a heartbeat that was not ACKed by the next++ * one means a zombied connection (closed, then resumed); op 1 asks for a heartbeat now; op 7++ * RECONNECT resumes; op 9 INVALID_SESSION identifies again (or resumes when Discord says it may).++ * Reconnects back off exponentially (minBackoffMs · 2^n, at most maxBackoffMs), reset once READY or++ * RESUMED. Close codes that reconnecting cannot heal (4004 a wrong token; 4010–4014 shard, version++ * or intents) stop it, with the reason shown to staff; 4007 and 4009 start a fresh session.++ *++ * Intents: GUILD_MESSAGES | MESSAGE_CONTENT (privileged: turn it on for the bot in the Developer++ * Portal). MESSAGE_CREATE / MESSAGE_UPDATE / MESSAGE_DELETE / MESSAGE_DELETE_BULK go to onDispatch++ * (relay/inbound.js); everything else is dropped here. The WebSocket is the client built into++ * Node 22 (no dependency); without one the gateway stays off and says so. The token never leaves++ * the IDENTIFY/RESUME payloads and is never logged.++ */++const INTENTS = { GUILD_MESSAGES: 1 << 9, MESSAGE_CONTENT: 1 << 15 };++const DEFAULT_URL = 'wss://gateway.discord.gg/?v=10&encoding=json';++const FATAL = {++ 4004: 'authentication failed: DISCORD_BOT_TOKEN is wrong',++ 4010: 'invalid shard',++ 4011: 'sharding required (the bot is in too many servers for one connection)',++ 4012: 'invalid gateway API version',++ 4013: 'invalid intents',++ 4014: 'disallowed intents: turn on the MESSAGE CONTENT intent for the bot in the Discord Developer Portal',++};++const FRESH = new Set([4007, 4009]);++const DISPATCH = new Set(['MESSAGE_CREATE', 'MESSAGE_UPDATE', 'MESSAGE_DELETE', 'MESSAGE_DELETE_BULK']);++const OPEN = 1;++++/** The gateway URL with the version and encoding this client speaks (a resume URL comes without them). */++function withQuery(u) {++ try {++ const url = new URL(u);++ url.searchParams.set('v', '10');++ url.searchParams.set('encoding', 'json');++ return url.toString();++ } catch { return u; }++}++++function createDiscordGateway({ token = '', url = DEFAULT_URL, WebSocketImpl = globalThis.WebSocket, onDispatch = () => {}, intents = INTENTS.GUILD_MESSAGES | INTENTS.MESSAGE_CONTENT, minBackoffMs = 1000, maxBackoffMs = 60_000, helloTimeoutMs = 30_000, invalidSessionMs = 1000, jitter = Math.random, now = () => Date.now() } = {}) {++ let reason = null;++ if (!token) reason = 'DISCORD_BOT_TOKEN is not set';++ else if (typeof WebSocketImpl !== 'function') reason = 'this Node has no built-in WebSocket client (Node 22 or later)';++ const on = !reason;++ const state = { status: on ? 'idle' : 'off', session_id: null, seq: null, resume_url: null, bot_user_id: null, last_error: null, last_reconnect: null, ready_at: null, connects: 0, reconnects: 0, dispatched: 0 };++ let ws = null;++ let firstBeat = null;++ let beats = null;++ let reconnectTimer = null;++ let helloTimer = null;++ let acked = true;++ let failures = 0;++ let stopped = true;++++ const iso = () => new Date(now()).toISOString();++ function send(obj) {++ if (ws && ws.readyState === OPEN) ws.send(JSON.stringify(obj));++ }++ function clearHeartbeat() {++ if (helloTimer) clearTimeout(helloTimer);++ helloTimer = null;++ if (firstBeat) clearTimeout(firstBeat);++ if (beats) clearInterval(beats);++ firstBeat = null;++ beats = null;++ }++ function beat() {++ acked = false;++ send({ op: 1, d: state.seq });++ }++ function heartbeatEvery(ms) {++ clearHeartbeat();++ acked = true;++ const interval = Math.max(Number(ms) || 41_250, 10);++ firstBeat = setTimeout(() => {++ firstBeat = null;++ beat();++ beats = setInterval(() => {++ if (!acked) return reconnect('no heartbeat ACK (zombied connection)');++ beat();++ }, interval);++ if (beats.unref) beats.unref();++ }, Math.floor(interval * jitter()));++ if (firstBeat.unref) firstBeat.unref();++ }++++ function connect() {++ reconnectTimer = null;++ if (stopped) return;++ const resuming = !!(state.session_id && state.resume_url);++ state.status = resuming ? 'resuming' : 'connecting';++ state.connects++;++ let sock;++ try {++ sock = new WebSocketImpl(resuming ? withQuery(state.resume_url) : url);++ } catch (err) {++ state.last_error = `cannot open the gateway: ${err.message}`;++ return scheduleReconnect();++ }++ ws = sock;++ helloTimer = setTimeout(() => { helloTimer = null; if (ws === sock) reconnect('no HELLO from the gateway'); }, helloTimeoutMs);++ if (helloTimer.unref) helloTimer.unref();++ sock.addEventListener('message', (ev) => { if (ws === sock) onMessage(ev.data); });++ sock.addEventListener('close', (ev) => { if (ws === sock) onClose(ev.code, ev.reason); });++ sock.addEventListener('error', () => { /* a close event follows */ });++ }++++ function onMessage(raw) {++ let p;++ try { p = JSON.parse(typeof raw === 'string' ? raw : Buffer.from(raw).toString('utf8')); } catch { return; }++ if (!p || typeof p !== 'object') return;++ switch (p.op) {++ case 10: // HELLO++ heartbeatEvery(p.d && p.d.heartbeat_interval); // also clears the HELLO timeout++ if (state.session_id && state.seq != null) send({ op: 6, d: { token, session_id: state.session_id, seq: state.seq } });++ else send({ op: 2, d: { token, intents, properties: { os: process.platform, browser: 'openvibe-community', device: 'openvibe-community' } } });++ break;++ case 11: // HEARTBEAT_ACK++ acked = true;++ break;++ case 1: // HEARTBEAT (asked for one now)++ beat();++ break;++ case 7: // RECONNECT++ reconnect('Discord asked to reconnect');++ break;++ case 9: // INVALID_SESSION (d: may resume)++ if (!p.d) { state.session_id = null; state.seq = null; state.resume_url = null; }++ reconnect('invalid session', invalidSessionMs + Math.floor(jitter() * 4 * invalidSessionMs)); // Discord: wait 1 to 5 s++ break;++ case 0: // DISPATCH++ if (p.s != null) state.seq = p.s;++ dispatch(p.t, p.d);++ break;++ default:++ break;++ }++ }++++ function dispatch(t, d) {++ if (t === 'READY') {++ state.session_id = d && d.session_id ? String(d.session_id) : null;++ state.resume_url = d && d.resume_gateway_url ? String(d.resume_gateway_url) : null;++ state.bot_user_id = d && d.user && d.user.id ? String(d.user.id) : null;++ state.status = 'ready';++ state.ready_at = iso();++ state.last_error = null;++ failures = 0;++ return;++ }++ if (t === 'RESUMED') {++ state.status = 'ready';++ state.last_error = null;++ failures = 0;++ return;++ }++ if (!DISPATCH.has(t)) return;++ state.dispatched++;++ try {++ const r = onDispatch(t, d || {}, { botUserId: state.bot_user_id });++ if (r && typeof r.catch === 'function') r.catch((err) => console.warn(`[Relay] inbound ${t} failed:`, err.message));++ } catch (err) {++ console.warn(`[Relay] inbound ${t} failed:`, err.message);++ }++ }++++ function onClose(code, why) {++ clearHeartbeat();++ ws = null;++ if (stopped) { state.status = 'stopped'; return; }++ if (FATAL[code]) {++ state.status = 'failed';++ state.last_error = `Discord closed the gateway (${code}): ${FATAL[code]}`;++ console.warn(`[Relay] ${state.last_error}; inbound stays off until a restart`);++ return;++ }++ if (FRESH.has(code)) { state.session_id = null; state.seq = null; state.resume_url = null; }++ state.last_error = `gateway closed (${code}${why ? `: ${String(why).slice(0, 200)}` : ''})`;++ state.last_reconnect = { reason: state.last_error, at: iso() };++ scheduleReconnect();++ }++++ /** Drop this connection (keeping the session, so the next one resumes) and connect again. */++ function reconnect(why, delayMs = null) {++ const sock = ws;++ ws = null;++ clearHeartbeat();++ state.last_error = why;++ state.last_reconnect = { reason: why, at: iso() };++ if (sock) { try { sock.close(4000, why.slice(0, 120)); } catch { /* already closed */ } }++ scheduleReconnect(delayMs);++ }++++ function scheduleReconnect(delayMs = null) {++ if (stopped || reconnectTimer) return;++ failures++;++ state.reconnects++;++ state.status = 'waiting';++ const delay = delayMs != null ? delayMs : Math.min(minBackoffMs * Math.pow(2, Math.max(failures - 1, 0)), maxBackoffMs);++ reconnectTimer = setTimeout(connect, delay);++ if (reconnectTimer.unref) reconnectTimer.unref();++ }++++ function start() {++ if (!on || !stopped) return;++ stopped = false;++ failures = 0;++ connect();++ }++ /** Close the connection (1000) and cancel every timer. */++ function stop() {++ stopped = true;++ clearHeartbeat();++ if (reconnectTimer) clearTimeout(reconnectTimer);++ reconnectTimer = null;++ const sock = ws;++ ws = null;++ if (sock) { try { sock.close(1000, 'stopping'); } catch { /* already closed */ } }++ if (on) state.status = 'stopped';++ return Promise.resolve();++ }++++ function status() {++ if (!on) return { enabled: false, reason };++ return {++ enabled: true, state: state.status, session: !!state.session_id, bot_user_id: state.bot_user_id,++ ready_at: state.ready_at, connects: state.connects, reconnects: state.reconnects, last_reconnect: state.last_reconnect, events_handled: state.dispatched, last_error: state.last_error,++ };++ }++++ return { enabled: on, reason, start, stop, status, _state: state };++}++++module.exports = { createDiscordGateway, INTENTS, DEFAULT_URL, FATAL };@@@ -1,11 +1,23 @@'use strict';/**-- * Discord relay, outbound only (DISCORD_RELAY_ENABLED; off by default).++ * Discord relay (DISCORD_RELAY_ENABLED; off by default), roadmap WS-J tasks 5 and 6.*-- * When a thread is created in a public space that has an enabled relay_mappings row, one-- * relay_deliveries row is created per (thread, mapping) — its dedupe key — and a worker posts a-- * message with source attribution and a link to the mapping's Discord webhook.++ * Out: a new thread in a public space with an enabled relay_mappings row is announced through the++ * mapping's Discord webhook; its replies follow (into the same channel, or the Discord thread the++ * mapping names); edits and deletes follow them through the external message map (relay_message_map).++ * In: with the gateway on, replies written on Discord come back as posts (server/relay/inbound.js).++ *++ * The queue is relay_deliveries, one row per dedupe key:++ * create thread:<id>:mapping:<m> | post:<id>:mapping:<m> queued by the Events worker from++ * community.thread.created / community.post.created (server/relay/events-worker.js), or by the++ * forum itself while that worker is off++ * edit edit:<thread|post>:<id>:mapping:<m>:r<revision> an edited opening post or reply that is on Discord++ * delete delete:<thread|post>:<id>:mapping:<m> deleted, moderated away, or gated members-only++ * A create is POSTed with ?wait=true, so Discord answers with the message and its id goes into the map;++ * an edit or delete PATCHes or DELETEs /messages/<id> of the same webhook (?thread_id= when it was sent++ * into a Discord thread). Nothing already in the map is created twice, and a reply waits for its++ * thread's own message (a reply to a thread that is not on Discord through that mapping is skipped).** - Secrets never live in the database: a mapping stores webhook_url_ref, the NAME of an* environment variable (e.g. DISCORD_WEBHOOK_FEEDBACK); the URL is read from process.env at@@@ -13,13 +25,17 @@* DISCORD_WEBHOOK_*. Staff can therefore never point the relay at the URL in some other* variable (an internal service's base URL), nor learn which other variables are set; a* mapping outside the list is refused when made and never sent.-- * - Loop prevention: a thread whose origin is 'discord' (a future inbound relay) is never-- * relayed out, checked both when queueing and when sending.++ * - Visibility: public spaces only, never members-only (VIP) spaces or threads, checked when++ * queueing and again when sending. Gating a thread later deletes what the relay had sent.++ * - Loop prevention: a thread or post whose origin is 'discord' is never relayed out, checked both++ * when queueing and when sending (inbound.js adds the other half).* - Retries: network errors, timeouts, 5xx, 429 and a missing webhook variable are retried with* exponential backoff (baseMs · 2^(attempt−1), capped at an hour; a 429's retry_after is-- * honoured) up to maxAttempts, then the delivery is 'failed'. Other 4xx answers fail at once.-- * - Mentions are disabled (allowed_mentions.parse = []), so a title can never ping anyone.-- * Failures are visible to staff at GET /api/v1/relay/deliveries.++ * honoured, and the rest of that webhook's queue waits for the next pass) up to maxAttempts.++ * Then the delivery is 'failed': the dead letter, with its error, that staff retry or drop.++ * Other 4xx fail at once. A delete that finds the message already gone counts as delivered.++ * - Mentions are disabled (allowed_mentions.parse = []), so nothing relayed can ping anyone.++ * Staff see and act on all of it at /api/v1/relay (relay/api.js).*/const pasteStore = require('../pastes/store');const forumStore = require('../forum/store');@@@ -29,8 +45,12 @@ const { AI_DISPLAY_NAME } = require('../identity/authors');const ENV_NAME = /^[A-Z][A-Z0-9_]{1,63}$/;const WEBHOOK_VAR = /^DISCORD_WEBHOOK_[A-Z0-9_]+$/;++const SNOWFLAKE = /^\d{15,21}$/;const MAX_BACKOFF_MS = 60 * 60_000;const SEND_TIMEOUT_MS = 10_000;++const POSTS_PER_PAGE = 50; // forum/service.js: a reply's link names its page++const UNKNOWN_MESSAGE = 10008; // Discord's error code: the message is gone++const DELIVERY_STATUSES = ['pending', 'delivered', 'failed', 'dropped', 'skipped'];/** Discord markdown in names/titles shown as plain text. */const escapeDiscord = (s) => String(s || '').replace(/([\\*_~`|>#[\]])/g, '\\$1').replace(/@/g, '@\u200b');@@@ -44,38 +64,145 @@ function createDiscordRelay({ db, config = {}, env = process.env, fetchImpl = gllet timer = null;let draining = null;let stopped = false;++ // While the Events worker runs it queues every create from the event stream; the forum's direct++ // calls then queue nothing (the dedupe keys would make them harmless anyway).++ let createsFromEvents = false;++ const parts = { worker: null, gateway: null, inbound: null };++++ const keys = {++ create: (type, id, m) => `${type}:${id}:mapping:${m}`,++ edit: (type, id, m, rev) => `edit:${type}:${id}:mapping:${m}:r${rev}`,++ remove: (type, id, m) => `delete:${type}:${id}:mapping:${m}`,++ };++ /** Kept for callers of the first relay: the create key of a thread. */++ function deliveryKey(threadId, mappingId) { return keys.create('thread', threadId, mappingId); }++++ const insertStmt = db.prepare(`INSERT INTO relay_deliveries (thread_id, post_id, mapping_id, action, dedupe_key, source, event_id, next_attempt_at)++ VALUES (@thread_id, @post_id, @mapping_id, @action, @key, @source, @event_id, @at) ON CONFLICT(dedupe_key) DO NOTHING`);++ function insert(row) {++ return insertStmt.run({ post_id: null, source: 'direct', event_id: null, ...row, at: sqlTime(now()) }).changes;++ }++ const outMappings = (spaceId) => db.prepare("SELECT * FROM relay_mappings WHERE space_id = ? AND direction = 'out' AND enabled = 1 ORDER BY id").all(spaceId);++ const mapRow = (mappingId, type, id) => db.prepare("SELECT * FROM relay_message_map WHERE platform = 'discord' AND mapping_id = ? AND local_type = ? AND local_id = ?").get(mappingId, type, id) || null;-- function deliveryKey(threadId, mappingId) { return `thread:${threadId}:mapping:${mappingId}`; }++ /** Why this thread (and a post of it) may not leave the site, or null. */++ function refusal(thread, space, post = null) {++ if (thread.origin === 'discord' || (post && post.origin === 'discord')) return `loop prevention: ${post && post.origin === 'discord' ? 'post' : 'thread'} came from Discord`;++ if (space.visibility !== 'public' || space.members_only_owner || thread.members_only_owner) return 'the thread is no longer public (members-only or a restricted space)';++ return null;++ }++ /** Queue-time filter: the same rules, quietly. */++ const mayLeave = (thread, space, post = null) => !!(thread && space && !thread.deleted_at && !(post && post.deleted_at) && !refusal(thread, space, post));-- /** Queue a new thread for every enabled out-mapping of its space. → number of deliveries queued */-- function enqueueThread(thread, space) {++ // ── queueing ─────────────────────────────────────────────++ /**++ * A new thread, for every enabled out-mapping of its space. source 'events' (the Events worker,++ * eventId) or 'direct' (the forum; ignored while the worker queues creates). → number queued++ */++ function enqueueThread(thread, space, { source = 'direct', eventId = null } = {}) {if (!enabled || !thread || !space) return 0;-- if (thread.origin === 'discord') return 0; // loop prevention-- if (space.visibility !== 'public') return 0; // members/staff spaces never leave the site-- if (space.members_only_owner || thread.members_only_owner) return 0; // nor do VIP members-only ones-- const mappings = db.prepare("SELECT * FROM relay_mappings WHERE space_id = ? AND direction = 'out' AND enabled = 1").all(space.id);++ if (source === 'direct' && createsFromEvents) return 0;++ if (!mayLeave(thread, space)) return 0;++ let queued = 0;++ for (const m of outMappings(space.id)) {++ queued += insert({ thread_id: thread.id, mapping_id: m.id, action: 'create', key: keys.create('thread', thread.id, m.id), source, event_id: eventId });++ }++ if (queued) kick();++ return queued;++ }++++ /**++ * A new reply: to every mapping its thread went to (sent, or still waiting to be). The opening post++ * travels with its thread. → number queued++ */++ function enqueuePost(post, thread, space, { source = 'direct', eventId = null } = {}) {++ if (!enabled || !post || !thread || !space || post.is_opening) return 0;++ if (source === 'direct' && createsFromEvents) return 0;++ if (!mayLeave(thread, space, post)) return 0;let queued = 0;-- const insert = db.prepare(`INSERT INTO relay_deliveries (thread_id, mapping_id, dedupe_key, next_attempt_at) VALUES (?, ?, ?, ?)-- ON CONFLICT(dedupe_key) DO NOTHING`);-- for (const m of mappings) queued += insert.run(thread.id, m.id, deliveryKey(thread.id, m.id), sqlTime(now())).changes;++ for (const m of outMappings(space.id)) {++ const parent = db.prepare("SELECT status FROM relay_deliveries WHERE dedupe_key = ? AND status IN ('pending', 'delivered')").get(keys.create('thread', thread.id, m.id));++ if (!parent && !mapRow(m.id, 'thread', thread.id)) continue;++ queued += insert({ thread_id: thread.id, post_id: post.id, mapping_id: m.id, action: 'create', key: keys.create('post', post.id, m.id), source, event_id: eventId });++ }if (queued) kick();return queued;}++ /**++ * An edited post: the Discord message showing it (the thread's, for an opening post) is edited.++ * Only what is already on Discord; a create still waiting carries the new text anyway.++ */++ function enqueueEdit(post) {++ if (!enabled || !post) return 0;++ const type = post.is_opening ? 'thread' : 'post';++ const id = post.is_opening ? post.thread_id : post.id;++ const rows = db.prepare("SELECT * FROM relay_message_map WHERE platform = 'discord' AND direction = 'out' AND local_type = ? AND local_id = ? AND external_deleted_at IS NULL").all(type, id);++ let queued = 0;++ for (const r of rows) {++ queued += insert({ thread_id: post.thread_id, post_id: post.is_opening ? null : post.id, mapping_id: r.mapping_id, action: 'edit', key: keys.edit(type, id, r.mapping_id, post.revision) });++ }++ if (queued) kick();++ return queued;++ }++++ /**++ * A thread (postId null: with every reply of it) or one reply left the public site — deleted,++ * moderated away or gated members-only: what is still waiting is skipped and what Discord shows is++ * deleted. Queued even while the relay is off, so it happens once it is on again.++ */++ function enqueueDelete(threadId, postId = null, reason = 'deleted') {++ const note = `skipped: ${reason} before it was sent`;++ return db.transaction(() => {++ if (postId == null) {++ db.prepare("UPDATE relay_deliveries SET status = 'skipped', last_error = ?, updated_at = CURRENT_TIMESTAMP WHERE thread_id = ? AND status = 'pending' AND action <> 'delete'").run(note, threadId);++ } else {++ db.prepare("UPDATE relay_deliveries SET status = 'skipped', last_error = ?, updated_at = CURRENT_TIMESTAMP WHERE post_id = ? AND status = 'pending' AND action <> 'delete'").run(note, postId);++ }++ const rows = postId == null++ ? db.prepare("SELECT * FROM relay_message_map WHERE thread_id = ? AND direction = 'out' AND external_deleted_at IS NULL").all(threadId)++ : db.prepare("SELECT * FROM relay_message_map WHERE local_type = 'post' AND local_id = ? AND direction = 'out' AND external_deleted_at IS NULL").all(postId);++ let queued = 0;++ for (const r of rows) {++ queued += insert({ thread_id: r.thread_id, post_id: r.local_type === 'post' ? r.local_id : null, mapping_id: r.mapping_id, action: 'delete', key: keys.remove(r.local_type, r.local_id, r.mapping_id) });++ }++ if (queued && enabled) kick();++ return queued;++ })();++ }++ /** Threads gated members-only (or otherwise no longer public): take them off Discord. */++ function hideThreads(threadIds, reason = 'made members-only') {++ let n = 0;++ for (const id of threadIds) n += enqueueDelete(id, null, reason);++ return n;++ }++function kick() { setImmediate(() => { drain().catch((err) => console.warn('[Relay] drain failed:', err.message)); }); }++ // ── messages ─────────────────────────────────────────────++ function authorName(subject, origin) {++ if (origin === 'ai') return AI_DISPLAY_NAME;++ if (origin === 'system' && !subject) return 'OpenVibe';++ if (subject) {++ const p = pasteStore.getProjections(db, [subject]).get(subject);++ if (p) return p.display_name || p.username || 'someone';++ }++ return 'someone';++ }++ const threadUrl = (thread, space) => `${base}/s/${space.slug}/t/${thread.slug}`;++ function postUrl(post, thread, space) {++ const position = db.prepare('SELECT COUNT(*) AS c FROM posts WHERE thread_id = ? AND id <= ?').get(thread.id, post.id).c;++ const page = Math.max(Math.ceil(position / POSTS_PER_PAGE), 1);++ return `${threadUrl(thread, space)}${page > 1 ? `?page=${page}` : ''}#post-${post.id}`;++ }++++ /** The thread's message: who started it, the title, an excerpt of the opening post, the link. */function message(thread, space) {const opening = db.prepare('SELECT body_markdown FROM posts WHERE thread_id = ? AND is_opening = 1').get(thread.id);-- let who = 'someone';-- if (thread.origin === 'ai') who = AI_DISPLAY_NAME;-- else if (thread.author_subject) {-- const p = pasteStore.getProjections(db, [thread.author_subject]).get(thread.author_subject);-- if (p) who = p.display_name || p.username || who;-- }-- const url = `${base}/s/${space.slug}/t/${thread.slug}`;++ const url = threadUrl(thread, space);return {username: 'OpenVibe.Community',-- content: `New thread in **s/${escapeDiscord(space.slug)}** by ${escapeDiscord(who)}: <${url}>`,++ content: `New thread in **s/${escapeDiscord(space.slug)}** by ${escapeDiscord(authorName(thread.author_subject, thread.origin))}: <${url}>`,allowed_mentions: { parse: [] },embeds: [{title: String(thread.title).slice(0, 256),@@@ -87,16 +214,36 @@ function createDiscordRelay({ db, config = {}, env = process.env, fetchImpl = gl};}++ /** A reply's message: who replied to which thread, an excerpt, the link to the post. */++ function postMessage(post, thread, space) {++ const url = postUrl(post, thread, space);++ const title = String(thread.title).slice(0, 200);++ return {++ username: 'OpenVibe.Community',++ content: `${escapeDiscord(authorName(post.author_subject, post.origin))} replied to **${escapeDiscord(title)}** in **s/${escapeDiscord(space.slug)}**: <${url}>`,++ allowed_mentions: { parse: [] },++ embeds: [{++ title: `Re: ${title}`.slice(0, 256),++ url,++ description: markdownToText(post.body_markdown, 500),++ footer: { text: `OpenVibe.Community · s/${space.slug}` },++ timestamp: isoTime(post.created_at) || undefined,++ }],++ };++ }++++ // ── sending ──────────────────────────────────────────────function backoff(attempts, retryAfterSec) {const exp = Math.min(baseMs * Math.pow(2, Math.max(attempts - 1, 0)), MAX_BACKOFF_MS);return Math.max(exp, retryAfterSec ? retryAfterSec * 1000 : 0);}-- function record(d, { ok, status = null, error = null, retry = false, retryAfter = 0 }) {++ /** One send happened: delivered, retried later, or failed (the dead letter). */++ function record(d, { ok, status = null, error = null, retry = false, retryAfter = 0, note = null }) {const attempts = d.attempts + 1;if (ok) {-- db.prepare("UPDATE relay_deliveries SET status = 'delivered', attempts = ?, last_status = ?, last_error = NULL, delivered_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP WHERE id = ?")-- .run(attempts, status, d.id);++ db.prepare("UPDATE relay_deliveries SET status = 'delivered', attempts = ?, last_status = ?, last_error = ?, delivered_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP WHERE id = ?")++ .run(attempts, status, note, d.id);return 'delivered';}const giveUp = !retry || attempts >= maxAttempts;@@@ -104,97 +251,256 @@ function createDiscordRelay({ db, config = {}, env = process.env, fetchImpl = gl.run(giveUp ? 'failed' : 'pending', attempts, status, String(error || '').slice(0, 500), sqlTime(now() + (giveUp ? 0 : backoff(attempts, retryAfter))), d.id);return giveUp ? 'failed' : 'retry';}++ /** Nothing was sent: already done (delivered) or nothing left to do (skipped). */++ function settle(d, status, reason) {++ db.prepare(`UPDATE relay_deliveries SET status = ?, last_error = ?, updated_at = CURRENT_TIMESTAMP,++ delivered_at = CASE WHEN ? = 'delivered' THEN CURRENT_TIMESTAMP ELSE delivered_at END WHERE id = ?`).run(status, reason, status, d.id);++ return status === 'delivered' ? 'delivered' : 'skipped';++ }++ /** Not yet (a reply waiting for its thread's message): later, without using an attempt. */++ function defer(d, untilSql, reason) {++ const until = Math.max(Date.parse(`${String(untilSql || '').replace(' ', 'T')}Z`) || 0, now() + 1000);++ db.prepare('UPDATE relay_deliveries SET next_attempt_at = ?, last_error = ?, updated_at = CURRENT_TIMESTAMP WHERE id = ?').run(sqlTime(until), reason, d.id);++ return 'retry';++ }async function send(d) {++ const m = db.prepare('SELECT * FROM relay_mappings WHERE id = ?').get(d.mapping_id);const thread = db.prepare('SELECT * FROM threads WHERE id = ?').get(d.thread_id);const space = thread ? forumStore.getSpaceById(db, thread.space_id) : null;-- if (!thread || thread.deleted_at || !space) return record(d, { ok: false, error: 'thread is gone' });-- if (thread.origin === 'discord') return record(d, { ok: false, error: 'loop prevention: thread came from Discord' });-- if (space.visibility !== 'public' || space.members_only_owner || thread.members_only_owner) return record(d, { ok: false, error: 'the thread is no longer public (members-only or a restricted space)' });-- if (!refAllowed(d.webhook_url_ref)) return record(d, { ok: false, error: 'webhook_url_ref is not an allowed webhook variable' });-- const url = env[d.webhook_url_ref];-- if (!url) return record(d, { ok: false, retry: true, error: `webhook URL variable ${d.webhook_url_ref} is not set` });-- if (!/^https?:\/\//i.test(url)) return record(d, { ok: false, error: `${d.webhook_url_ref} is not an http(s) URL` });++ const post = d.post_id ? forumStore.getPost(db, d.post_id) : null;++ const type = d.post_id ? 'post' : 'thread';++ const localId = d.post_id || d.thread_id;++ const gone = !thread || thread.deleted_at || !space || (d.post_id && (!post || post.deleted_at));++ const mapped = mapRow(d.mapping_id, type, localId);++++ if (d.action === 'create') {++ if (gone) return settle(d, 'skipped', `the ${type} was deleted before it was sent`);++ const why = refusal(thread, space, post);++ if (why) return record(d, { ok: false, error: why });++ if (mapped) return settle(d, 'delivered', 'already on Discord');++ if (type === 'post' && !mapRow(d.mapping_id, 'thread', thread.id)) {++ const parent = db.prepare('SELECT status, next_attempt_at FROM relay_deliveries WHERE dedupe_key = ?').get(keys.create('thread', thread.id, d.mapping_id));++ if (parent && parent.status === 'pending') return defer(d, parent.next_attempt_at, "waiting for the thread's own message");++ return settle(d, 'skipped', 'the thread is not on Discord through this mapping');++ }++ } else {++ if (!mapped || mapped.direction !== 'out') return settle(d, 'skipped', 'not on Discord');++ if (mapped.external_deleted_at) return settle(d, 'skipped', 'already deleted on Discord');++ if (d.action === 'edit') {++ if (gone) return settle(d, 'skipped', `the ${type} was deleted`);++ const why = refusal(thread, space, post);++ if (why) return settle(d, 'skipped', why);++ }++ }++++ if (!m || !refAllowed(m.webhook_url_ref)) return record(d, { ok: false, error: 'webhook_url_ref is not an allowed webhook variable' });++ const raw = env[m.webhook_url_ref];++ if (!raw) return record(d, { ok: false, retry: true, error: `webhook URL variable ${m.webhook_url_ref} is not set` });++ let url = null;++ try { url = new URL(raw); } catch { /* below */ }++ if (!url || !/^https?:$/.test(url.protocol)) return record(d, { ok: false, error: `${m.webhook_url_ref} is not an http(s) URL` });++++ let method = 'POST';++ let body = null;++ let threadTarget = null;++ if (d.action === 'create') {++ threadTarget = (SNOWFLAKE.test(m.discord_thread_id || '') ? m.discord_thread_id : null) || url.searchParams.get('thread_id') || null;++ url.searchParams.set('wait', 'true');++ if (threadTarget) url.searchParams.set('thread_id', threadTarget);++ body = type === 'thread' ? message(thread, space) : postMessage(post, thread, space);++ } else {++ method = d.action === 'edit' ? 'PATCH' : 'DELETE';++ url.pathname = `${url.pathname.replace(/\/+$/, '')}/messages/${mapped.external_message_id}`;++ url.search = '';++ if (mapped.external_thread_id) url.searchParams.set('thread_id', mapped.external_thread_id);++ if (d.action === 'edit') body = type === 'thread' ? message(thread, space) : postMessage(post, thread, space);++ }++let res;try {-- res = await fetchImpl(url, {-- method: 'POST',-- headers: { 'Content-Type': 'application/json', 'User-Agent': 'OpenVibe.Community relay' },-- body: JSON.stringify(message(thread, space)),++ res = await fetchImpl(url.toString(), {++ method,++ headers: { ...(body ? { 'Content-Type': 'application/json' } : {}), 'User-Agent': 'OpenVibe.Community relay' },++ body: body ? JSON.stringify(body) : undefined,signal: AbortSignal.timeout(SEND_TIMEOUT_MS),});} catch (err) {return record(d, { ok: false, retry: true, error: err.name === 'TimeoutError' ? 'timed out' : err.message });}-- if (res.ok) return record(d, { ok: true, status: res.status });-- let detail = '';++ let answer = null;++ if (res.status !== 204) { try { answer = await res.json(); } catch { /* no JSON */ } }++ if (res.ok) {++ if (d.action === 'create') return created(d, res.status, answer, { type, localId, threadTarget, mapping: m });++ if (d.action === 'delete') db.prepare('UPDATE relay_message_map SET external_deleted_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP WHERE id = ?').run(mapped.id);++ else db.prepare('UPDATE relay_message_map SET updated_at = CURRENT_TIMESTAMP WHERE id = ?').run(mapped.id);++ return record(d, { ok: true, status: res.status });++ }++ // The message itself is gone on Discord (someone deleted it there): a delete is done, an edit has nothing to edit.++ if (res.status === 404 && d.action !== 'create' && answer && Number(answer.code) === UNKNOWN_MESSAGE) {++ db.prepare('UPDATE relay_message_map SET external_deleted_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP WHERE id = ?').run(mapped.id);++ return d.action === 'delete' ? record(d, { ok: true, status: 404, note: 'the message was already gone on Discord' }) : settle(d, 'skipped', 'the message is gone on Discord');++ }++ const detail = answer && (answer.message || answer.error) ? String(answer.message || answer.error) : '';let retryAfter = Number(res.headers.get('retry-after')) || 0;-- try { const j = await res.json(); detail = j && (j.message || j.error) ? String(j.message || j.error) : ''; if (j && j.retry_after) retryAfter = Math.max(retryAfter, Number(j.retry_after) || 0); } catch { /* not JSON */ }++ if (answer && answer.retry_after) retryAfter = Math.max(retryAfter, Number(answer.retry_after) || 0);const retry = res.status === 429 || res.status === 408 || res.status >= 500;-- return record(d, { ok: false, retry, retryAfter, status: res.status, error: `Discord answered ${res.status}${detail ? `: ${detail}` : ''}` });++ const outcome = record(d, { ok: false, retry, retryAfter, status: res.status, error: `Discord answered ${res.status}${detail ? `: ${detail}` : ''}` });++ return res.status === 429 ? { outcome, rateLimited: true } : outcome;++ }++++ /** A create was answered: map the message (both ways unique) and learn the mapping's channel. */++ function created(d, status, msg, { type, localId, threadTarget, mapping }) {++ const id = msg && SNOWFLAKE.test(String(msg.id || '')) ? String(msg.id) : null;++ const channel = msg && SNOWFLAKE.test(String(msg.channel_id || '')) ? String(msg.channel_id) : null;++ if (!id || !channel) return record(d, { ok: true, status, note: 'Discord did not answer with the message, so edits and deletes cannot follow it' });++ return db.transaction(() => {++ db.prepare(`INSERT INTO relay_message_map (platform, mapping_id, direction, local_type, local_id, thread_id, external_channel_id, external_thread_id, external_message_id, external_webhook_id)++ VALUES ('discord', ?, 'out', ?, ?, ?, ?, ?, ?, ?) ON CONFLICT DO NOTHING`)++ .run(mapping.id, type, localId, d.thread_id, channel, threadTarget ? channel : null, id, msg.webhook_id ? String(msg.webhook_id) : null);++ if (!threadTarget) db.prepare('UPDATE relay_mappings SET discord_channel_id = ? WHERE id = ? AND discord_channel_id IS NULL').run(channel, mapping.id);++ return record(d, { ok: true, status });++ })();}-- /** Send every due delivery once. One drain at a time. → { delivered, retry, failed } */++ /**++ * Send every due delivery once, oldest first. One drain at a time. A 429 leaves the rest of that++ * webhook's queue for the next pass. → { delivered, retry, failed, skipped }++ */function drain() {if (draining) return draining;-- if (stopped) return Promise.resolve({ delivered: 0, retry: 0, failed: 0 });++ if (stopped) return Promise.resolve({ delivered: 0, retry: 0, failed: 0, skipped: 0 });draining = (async () => {-- const summary = { delivered: 0, retry: 0, failed: 0 };++ const summary = { delivered: 0, retry: 0, failed: 0, skipped: 0 };if (!enabled) return summary;-- const due = db.prepare(`SELECT d.*, m.webhook_url_ref FROM relay_deliveries d JOIN relay_mappings m ON m.id = d.mapping_id-- WHERE d.status = 'pending' AND m.enabled = 1 AND d.next_attempt_at <= ? ORDER BY d.id ASC LIMIT 50`).all(sqlTime(now()));-- for (const d of due) summary[await send(d)]++;++ // Deletes go out even for a disabled mapping: what left the site leaves Discord too.++ const due = db.prepare(`SELECT d.* FROM relay_deliveries d JOIN relay_mappings m ON m.id = d.mapping_id++ WHERE d.status = 'pending' AND (m.enabled = 1 OR d.action = 'delete') AND d.next_attempt_at <= ?++ ORDER BY d.id ASC LIMIT 50`).all(sqlTime(now()));++ const paused = new Set();++ for (const d of due) {++ if (stopped) break;++ if (paused.has(d.mapping_id)) continue;++ const r = await send(d);++ if (r && typeof r === 'object') { paused.add(d.mapping_id); summary[r.outcome]++; } else summary[r]++;++ }return summary;})().finally(() => { draining = null; });return draining;}++ // ── lifecycle ────────────────────────────────────────────++ /** The Events worker, the gateway and the inbound handler (app.js), started and stopped with the relay. */++ function attach({ worker = null, gateway = null, inbound = null } = {}) { Object.assign(parts, { worker, gateway, inbound }); }++ /** The Events worker says it queues creates (true) or no longer does. */++ function useEventsForCreates(on) { createsFromEvents = !!on; }++function start() {if (!enabled || timer) return;stopped = false;timer = setInterval(() => { drain().catch((err) => console.warn('[Relay] drain failed:', err.message)); }, pollMs);if (timer.unref) timer.unref();kick();++ if (parts.worker) parts.worker.start();++ if (parts.gateway) parts.gateway.start();++ }++ /** Graceful stop: no further drains or reads; resolves when the drain and page in progress finished (the rest stay queued). */++ function stop() {++ stopped = true;++ if (timer) clearInterval(timer);++ timer = null;++ const waits = [draining || Promise.resolve()];++ if (parts.worker) waits.push(parts.worker.stop());++ if (parts.gateway) waits.push(parts.gateway.stop());++ return Promise.all(waits).then(() => undefined);}-- /** Graceful stop: no further drains; resolves when the drain in progress has finished (the rest stay pending). */-- function stop() { stopped = true; if (timer) clearInterval(timer); timer = null; return draining || Promise.resolve(); }// ── admin (staff) ────────────────────────────────────────function shapeMapping(m) {-- return { id: m.id, space: m.space_slug, direction: m.direction, webhook_url_ref: m.webhook_url_ref, webhook_configured: refAllowed(m.webhook_url_ref) && !!env[m.webhook_url_ref], enabled: !!m.enabled, created_at: isoTime(m.created_at) };-- }-- function listMappings() {-- return db.prepare('SELECT m.*, s.slug AS space_slug FROM relay_mappings m JOIN spaces s ON s.id = m.space_id ORDER BY m.id').all().map(shapeMapping);++ return {++ id: m.id, space: m.space_slug, direction: m.direction, webhook_url_ref: m.webhook_url_ref, webhook_configured: refAllowed(m.webhook_url_ref) && !!env[m.webhook_url_ref],++ enabled: !!m.enabled, discord_channel_id: m.discord_channel_id || null, discord_thread_id: m.discord_thread_id || null, inbound: !!m.inbound, created_at: isoTime(m.created_at),++ };}-- function addMapping({ space_id, webhook_url_ref, enabled: on = true }) {++ const mappingQuery = 'SELECT m.*, s.slug AS space_slug FROM relay_mappings m JOIN spaces s ON s.id = m.space_id';++ function listMappings() { return db.prepare(`${mappingQuery} ORDER BY m.id`).all().map(shapeMapping); }++ const snowflakeOrNull = (v, name) => {++ if (v === undefined) return undefined;++ if (v === null || v === '') return null;++ if (!SNOWFLAKE.test(String(v))) throw new Error(`${name} is a Discord id (digits)`);++ return String(v);++ };++ /** { space_id, webhook_url_ref, enabled?, discord_channel_id?, discord_thread_id?, inbound? } → mapping (made, or updated) */++ function addMapping({ space_id, webhook_url_ref, enabled: on = true, discord_channel_id, discord_thread_id, inbound }) {if (!refAllowed(webhook_url_ref)) throw new Error(`${webhook_url_ref} is not an allowed webhook variable`);-- db.prepare(`INSERT INTO relay_mappings (space_id, direction, webhook_url_ref, enabled) VALUES (?, 'out', ?, ?)-- ON CONFLICT(space_id, direction, webhook_url_ref) DO UPDATE SET enabled = excluded.enabled`).run(space_id, webhook_url_ref, on ? 1 : 0);-- return shapeMapping(db.prepare("SELECT m.*, s.slug AS space_slug FROM relay_mappings m JOIN spaces s ON s.id = m.space_id WHERE m.space_id = ? AND m.direction = 'out' AND m.webhook_url_ref = ?").get(space_id, webhook_url_ref));++ const channel = snowflakeOrNull(discord_channel_id, 'discord_channel_id');++ const thread = snowflakeOrNull(discord_thread_id, 'discord_thread_id');++ db.prepare(`INSERT INTO relay_mappings (space_id, direction, webhook_url_ref, enabled, discord_channel_id, discord_thread_id, inbound) VALUES (?, 'out', ?, ?, ?, ?, ?)++ ON CONFLICT(space_id, direction, webhook_url_ref) DO UPDATE SET enabled = excluded.enabled`).run(space_id, webhook_url_ref, on ? 1 : 0, channel || null, thread || null, inbound ? 1 : 0);++ const row = db.prepare(`${mappingQuery} WHERE m.space_id = ? AND m.direction = 'out' AND m.webhook_url_ref = ?`).get(space_id, webhook_url_ref);++ return updateMapping(row.id, { discord_channel_id: channel, discord_thread_id: thread, inbound });}-- function setMappingEnabled(id, on) {-- if (!db.prepare('UPDATE relay_mappings SET enabled = ? WHERE id = ?').run(on ? 1 : 0, id).changes) return null;-- return shapeMapping(db.prepare('SELECT m.*, s.slug AS space_slug FROM relay_mappings m JOIN spaces s ON s.id = m.space_id WHERE m.id = ?').get(id));++ /** { enabled?, inbound?, discord_channel_id?, discord_thread_id? } → mapping, or null when there is no such mapping */++ function updateMapping(id, fields = {}) {++ const sets = [];++ const params = [];++ if (fields.enabled !== undefined) { sets.push('enabled = ?'); params.push(fields.enabled ? 1 : 0); }++ if (fields.inbound !== undefined) { sets.push('inbound = ?'); params.push(fields.inbound ? 1 : 0); }++ for (const k of ['discord_channel_id', 'discord_thread_id']) {++ const v = snowflakeOrNull(fields[k], k);++ if (v !== undefined) { sets.push(`${k} = ?`); params.push(v); }++ }++ if (sets.length && !db.prepare(`UPDATE relay_mappings SET ${sets.join(', ')} WHERE id = ?`).run(...params, id).changes) return null;++ const row = db.prepare(`${mappingQuery} WHERE m.id = ?`).get(id);++ return row ? shapeMapping(row) : null;}-- function listDeliveries({ status = null, limit = 50 } = {}) {++ function setMappingEnabled(id, on) { return updateMapping(id, { enabled: on }); }++++ function listDeliveries({ status = null, action = null, limit = 50 } = {}) {return db.prepare(`SELECT d.*, m.webhook_url_ref, t.title AS thread_title, t.slug AS thread_slug, s.slug AS space_slugFROM relay_deliveries d JOIN relay_mappings m ON m.id = d.mapping_id JOIN threads t ON t.id = d.thread_id JOIN spaces s ON s.id = t.space_id-- WHERE (? IS NULL OR d.status = ?) ORDER BY d.id DESC LIMIT ?`).all(status, status, limit)++ WHERE (? IS NULL OR d.status = ?) AND (? IS NULL OR d.action = ?) ORDER BY d.id DESC LIMIT ?`).all(status, status, action, action, limit).map((d) => ({-- id: d.id, dedupe_key: d.dedupe_key, status: d.status, attempts: d.attempts, last_status: d.last_status, last_error: d.last_error,-- next_attempt_at: d.status === 'pending' ? isoTime(d.next_attempt_at) : null, delivered_at: isoTime(d.delivered_at), created_at: isoTime(d.created_at),++ id: d.id, dedupe_key: d.dedupe_key, action: d.action, source: d.source, event_id: d.event_id || null,++ status: d.status, attempts: d.attempts, last_status: d.last_status, last_error: d.last_error,++ next_attempt_at: d.status === 'pending' ? isoTime(d.next_attempt_at) : null, delivered_at: isoTime(d.delivered_at), created_at: isoTime(d.created_at), updated_at: isoTime(d.updated_at),mapping: { id: d.mapping_id, webhook_url_ref: d.webhook_url_ref },thread: { id: d.thread_id, title: d.thread_title, url: `/s/${d.space_slug}/t/${d.thread_slug}` },++ post: d.post_id ? { id: d.post_id } : null,}));}-- /** Put a failed (or pending) delivery back in the queue now, with a fresh attempt budget. */++ /** Put a delivery that was not delivered back in the queue now, with a fresh attempt budget. */function retry(id) {const n = db.prepare("UPDATE relay_deliveries SET status = 'pending', attempts = 0, next_attempt_at = ?, updated_at = CURRENT_TIMESTAMP WHERE id = ? AND status <> 'delivered'").run(sqlTime(now()), id).changes;if (n) kick();return n;}++ /** Staff give up on a waiting or dead delivery: 'dropped', kept for the record. */++ function drop(id) {++ return db.prepare("UPDATE relay_deliveries SET status = 'dropped', updated_at = CURRENT_TIMESTAMP WHERE id = ? AND status IN ('pending', 'failed')").run(id).changes;++ }++++ /** What staff (and /api/ready) see: the queue by status, and the Events worker's and the gateway's state. */++ function status() {++ const deliveries = Object.fromEntries(DELIVERY_STATUSES.map((s) => [s, 0]));++ for (const r of db.prepare('SELECT status, COUNT(*) AS n FROM relay_deliveries GROUP BY status').all()) deliveries[r.status] = r.n;++ const off = (reason) => ({ enabled: false, reason });++ return {++ enabled,++ creates_from: createsFromEvents ? 'events' : 'forum',++ deliveries,++ dead_letters: deliveries.failed,++ messages_mapped: db.prepare('SELECT COUNT(*) AS n FROM relay_message_map WHERE external_deleted_at IS NULL').get().n,++ events_worker: parts.worker ? parts.worker.status() : off(enabled ? 'not configured' : 'the relay is off'),++ inbound: parts.gateway ? { ...parts.gateway.status(), ...(parts.inbound ? { handled: parts.inbound.stats() } : {}) } : off(enabled ? 'DISCORD_RELAY_INBOUND is off' : 'the relay is off'),++ inbound_failures: db.prepare('SELECT COUNT(*) AS n FROM relay_inbound_failures WHERE dismissed_at IS NULL').get().n,++ };++ }-- return { enabled, enqueueThread, drain, start, stop, listMappings, addMapping, setMappingEnabled, listDeliveries, retry, ENV_NAME, refAllowed, message };++ return {++ enabled, enqueueThread, enqueuePost, enqueueEdit, enqueueDelete, hideThreads, drain, start, stop, attach, useEventsForCreates, status,++ listMappings, addMapping, updateMapping, setMappingEnabled, listDeliveries, retry, drop, deliveryKey, ENV_NAME, refAllowed, message, postMessage,++ };}--module.exports = { createDiscordRelay, escapeDiscord, ENV_NAME, WEBHOOK_VAR };++module.exports = { createDiscordRelay, escapeDiscord, ENV_NAME, WEBHOOK_VAR, SNOWFLAKE, DELIVERY_STATUSES };nnew file mode 100644@@@ -0,0 +1,164 @@++'use strict';++++/**++ * The Discord relay's Events worker (roadmap WS-J task 6): reads Community's own++ * community.thread.* and community.post.* from OpenVibe.Events' pull API (GET /api/v1/events,++ * capability events.event.read, Community's service token on audience openvibe.events) and queues++ * the relay's creates from them (relay/discord.js).++ *++ * - Cursor: the relay_cursors row 'discord-relay:events', moved in the same transaction as the++ * deliveries its page queued. A crash replays at most the page in progress, and the dedupe keys++ * make that replay queue nothing twice.++ * - First start (no cursor yet): it begins at Events' head. Threads from before are never announced.++ * - Events carry ids, never content: the worker reads the thread or post as it is now and the relay's++ * rules decide (public spaces only, never members-only, never origin 'discord'). Redacted events++ * and events from any other source are ignored.++ * - A gap (retention pruned events the worker had not read yet) is logged and shown to staff; the++ * creates in it are not relayed.++ * - While it runs, the forum's own create calls queue nothing (the worker is the one path). Off++ * without EVENTS_URL and OV_OAUTH_CLIENT_SECRET, or with DISCORD_RELAY_EVENTS=off: then the forum++ * queues creates itself, as the first relay did. Edits and deletes are queued by the forum in both++ * cases (Contracts has no community.thread/post update or delete events yet).++ */++const forumStore = require('../forum/store');++++const TOPICS = ['community.thread.*', 'community.post.*'];++const CURSOR = 'discord-relay:events';++const MAX_PAGES_PER_TICK = 20;++++/**++ * events: an openvibe-sdk events client (tests); otherwise one is made from eventsUrl, clientSecret and++ * tokenUrl (the Network's /oauth/token, client credentials of clientId).++ */++function createRelayEventsWorker({ db, relay, events = null, eventsUrl = null, clientSecret = null, tokenUrl = null, clientId = 'community', fetchImpl, enabled = true, pollMs = 5000, pageLimit = 100, now = () => Date.now() } = {}) {++ let reason = null;++ if (!relay || !relay.enabled) reason = 'the Discord relay is off (DISCORD_RELAY_ENABLED)';++ else if (!enabled) reason = 'DISCORD_RELAY_EVENTS=off';++ else if (!events && (!eventsUrl || !clientSecret || !tokenUrl)) reason = 'EVENTS_URL and OV_OAUTH_CLIENT_SECRET are not both set';++ let client = events;++ if (!reason && !client) {++ const { createClient } = require('openvibe-sdk/core');++ const { createServiceTokenClient } = require('openvibe-sdk/auth');++ const { createEventsClient } = require('openvibe-sdk/events');++ const tokens = createServiceTokenClient({ tokenUrl, clientId, clientSecret, fetch: fetchImpl });++ const core = createClient({ baseUrls: { events: String(eventsUrl).replace(/\/+$/, '') }, tokenProvider: tokens, fetch: fetchImpl, retries: 0 });++ client = createEventsClient(core, { source: 'community' });++ }++ const on = !reason;++ const stats = { seen: 0, queued: 0, pages: 0, last_error: null, last_error_at: null, last_ok_at: null, last_gap: null, started_at_seq: null };++ let timer = null;++ let running = null;++ let stopped = false;++++ const iso = () => new Date(now()).toISOString();++ const readCursor = () => db.prepare('SELECT cursor, latest_seq FROM relay_cursors WHERE name = ?').get(CURSOR) || null;++ function writeCursor(cursor, latest) {++ db.prepare(`INSERT INTO relay_cursors (name, cursor, latest_seq, updated_at) VALUES (?, ?, ?, CURRENT_TIMESTAMP)++ ON CONFLICT(name) DO UPDATE SET cursor = excluded.cursor, latest_seq = excluded.latest_seq, updated_at = CURRENT_TIMESTAMP`).run(CURSOR, cursor, latest == null ? null : latest);++ }++++ /** One event → deliveries queued (0 when it is not for the relay). */++ function handle(ev) {++ stats.seen++;++ if (!ev || ev.source !== 'community' || !ev.payload || ev.payload.redacted) return 0;++ const p = ev.payload;++ if (p.visibility && p.visibility !== 'public') return 0;++ if (ev.event_type === 'community.thread.created') {++ const thread = db.prepare('SELECT * FROM threads WHERE id = ?').get(Number(p.thread_id));++ const space = thread ? forumStore.getSpaceById(db, thread.space_id) : null;++ return relay.enqueueThread(thread, space, { source: 'events', eventId: ev.event_id || null });++ }++ if (ev.event_type === 'community.post.created') {++ const post = forumStore.getPost(db, Number(p.post_id));++ const thread = post ? db.prepare('SELECT * FROM threads WHERE id = ?').get(post.thread_id) : null;++ const space = thread ? forumStore.getSpaceById(db, thread.space_id) : null;++ return relay.enqueuePost(post, thread, space, { source: 'events', eventId: ev.event_id || null });++ }++ return 0;++ }++++ /** Read everything new (at most MAX_PAGES_PER_TICK pages). One tick at a time. */++ function tick() {++ if (!on) return Promise.resolve({ queued: 0 });++ if (running) return running;++ running = (async () => {++ let queuedNow = 0;++ try {++ let cur = readCursor();++ if (!cur) {++ const head = await client.pull({ topic: TOPICS, afterSeq: 0, limit: 1 });++ const latest = Number(head.latest_seq) || 0;++ writeCursor(latest, latest);++ stats.started_at_seq = latest;++ console.log(`[Relay] Events worker starts at seq ${latest} (earlier threads are not announced)`);++ cur = { cursor: latest, latest_seq: latest };++ }++ let cursor = Number(cur.cursor) || 0;++ for (let pages = 0; pages < MAX_PAGES_PER_TICK && !stopped; pages++) {++ const page = await client.pull({ topic: TOPICS, afterSeq: cursor, limit: pageLimit });++ if (stopped) break;++ if (page.gap) {++ stats.last_gap = { from_seq: page.gap.from_seq, to_seq: page.gap.to_seq, at: iso() };++ console.warn(`[Relay] Events gap: seq ${page.gap.from_seq}..${page.gap.to_seq} were pruned before the relay read them; threads and replies in it are not relayed`);++ }++ const next = Number(page.next_after_seq);++ const latest = Number(page.latest_seq);++ const moved = Number.isFinite(next) && next > cursor ? next : cursor;++ queuedNow += db.transaction(() => {++ let n = 0;++ for (const item of page.events || []) n += handle(item && item.event);++ writeCursor(moved, Number.isFinite(latest) ? latest : null);++ return n;++ })();++ stats.pages++;++ if (moved === cursor || !Number.isFinite(latest) || moved >= latest) break;++ cursor = moved;++ }++ stats.queued += queuedNow;++ stats.last_ok_at = iso();++ if (stats.last_error) console.log('[Relay] Events worker reading again');++ stats.last_error = null;++ stats.last_error_at = null;++ } catch (err) {++ const m = err && err.message ? err.message : String(err);++ if (m !== stats.last_error) console.warn('[Relay] Events worker cannot read (will retry):', m);++ stats.last_error = m;++ stats.last_error_at = iso();++ }++ return { queued: queuedNow };++ })().finally(() => { running = null; });++ return running;++ }++++ function start() {++ if (!on || timer) return;++ stopped = false;++ relay.useEventsForCreates(true);++ timer = setInterval(() => { tick(); }, pollMs);++ if (timer.unref) timer.unref();++ tick();++ }++ /** Graceful stop: no further reads; resolves when the page in progress has been handled. */++ function stop() {++ stopped = true;++ if (timer) clearInterval(timer);++ timer = null;++ return running ? running.then(() => undefined) : Promise.resolve();++ }++++ function status() {++ if (!on) return { enabled: false, reason };++ const cur = readCursor();++ return {++ enabled: true, running: !!timer, topics: TOPICS,++ cursor: cur ? cur.cursor : null, latest_seq: cur ? cur.latest_seq : null,++ lag: cur && cur.latest_seq != null ? Math.max(cur.latest_seq - cur.cursor, 0) : null,++ events_seen: stats.seen, deliveries_queued: stats.queued, started_at_seq: stats.started_at_seq,++ last_ok_at: stats.last_ok_at, last_error: stats.last_error, last_error_at: stats.last_error_at, last_gap: stats.last_gap,++ };++ }++++ return { enabled: on, reason, tick, start, stop, status, TOPICS, CURSOR };++}++++module.exports = { createRelayEventsWorker, TOPICS, CURSOR };nnew file mode 100644@@@ -0,0 +1,199 @@++'use strict';++++/**++ * Replies from Discord (roadmap WS-J task 6, inbound; DISCORD_RELAY_INBOUND=on with DISCORD_BOT_TOKEN).++ * The gateway (relay/discord-gateway.js) hands message events here.++ *++ * What becomes a post: only messages in a mapped channel (a mapping with inbound on: its++ * discord_channel_id or discord_thread_id, or a Discord thread started from one of its messages), and++ * only replies to something in the external message map:++ * - a Discord reply (message_reference) to a mapped message in the same channel, or++ * - any message in a Discord thread started from a mapped message (that thread's id is the message's).++ * Everything else in the channel is ignored. The post goes into the mapped message's local thread with++ * origin 'discord', no author subject (never attributed to anyone on the site; the Discord display name++ * is kept in posts.relay_author and shown with a "Discord" badge), and the map gets its direction 'in'++ * row. Edits on Discord edit the post (a new revision, edited_by 'discord'); deletes on Discord delete++ * it. A delete of a message the relay sent only marks it gone there (the thread stays).++ *++ * Loop prevention: webhook messages (the relay's own among them), bots, the bot itself and system++ * messages are never taken in; a message already in the map is never taken twice (and the map's unique++ * key refuses it anyway); posts with origin 'discord' are never relayed out (relay/discord.js).++ * Limits: perMinute messages per Discord author per mapping and 10× that per mapping (a rolling++ * minute), maxChars per message (longer text is cut). Mentions, roles, channels, custom emoji and++ * timestamps become plain text and @everyone/@here are defused; the rest is Markdown, which the++ * forum's safe renderer shows (render/markdown.js: no raw HTML, safe links only). Attachments are++ * noted, never fetched. The thread must still be public, open (not locked) and not members-only.++ * What could not be applied goes to relay_inbound_failures: ids and the reason, never the text.++ */++const forumStore = require('../forum/store');++const { SNOWFLAKE } = require('./discord');++++const KEEP_FAILURES = 1000;++const MESSAGE_TYPES = new Set([0, 19]); // DEFAULT, REPLY++++/** A Discord display name as plain text: no control or format characters, one line, at most 32. */++function cleanName(s) {++ const t = String(s == null ? '' : s).replace(/[\u0000-\u001F\u007F-\u009F\u200b-\u200F\u202A-\u202E\u2060-\u2064\uFEFF]/g, '').replace(/\s+/g, ' ').trim();++ return t.slice(0, 32);++}++++/** Discord message text → forum Markdown (see the header). '' when there is nothing to show. */++function toMarkdown(d, maxChars = 4000) {++ const names = new Map((Array.isArray(d.mentions) ? d.mentions : []).map((u) => [String(u && u.id), cleanName(u && (u.global_name || u.username))]));++ let text = (typeof d.content === 'string' ? d.content : '')++ .replace(/<@!?(\d{15,21})>/g, (_, id) => `@${names.get(id) || 'someone'}`)++ .replace(/<@&\d{15,21}>/g, '@role')++ .replace(/<#\d{15,21}>/g, '#channel')++ .replace(/<\/([\w-]{1,32}(?: [\w-]{1,32}){0,2}):\d{15,21}>/g, '/$1')++ .replace(/<a?:(\w{1,32}):\d{15,21}>/g, ':$1:')++ .replace(/<t:(-?\d{1,13})(?::[tTdDfFR])?>/g, (_, sec) => {++ const t = new Date(Number(sec) * 1000);++ return Number.isNaN(t.getTime()) ? '' : `${t.toISOString().slice(0, 16).replace('T', ' ')} UTC`;++ })++ .replace(/@(everyone|here)/g, '@\u200b$1')++ .replace(/\r\n?/g, '\n')++ .replace(/[\u0000-\u0008\u000B\u000C\u000E-\u001F\u007F]/g, '')++ .trim();++ if (text.length > maxChars) text = `${text.slice(0, Math.max(maxChars - 1, 0)).trimEnd()}…`;++ const files = Array.isArray(d.attachments) ? d.attachments.length : 0;++ if (files) text += `${text ? '\n\n' : ''}_(${files} attachment${files === 1 ? '' : 's'} on Discord)_`;++ return text;++}++++function createDiscordInbound({ db, perMinute = 6, maxChars = 4000, now = () => Date.now() } = {}) {++ const windows = new Map(); // key → timestamps in the last minute++ const counts = { applied: 0, ignored: 0, failed: 0 };++++ function allow(key, limit) {++ const t = now();++ const list = (windows.get(key) || []).filter((x) => t - x < 60_000);++ if (list.length >= limit) { windows.set(key, list); return false; }++ list.push(t);++ windows.set(key, list);++ if (windows.size > 5000) for (const [k, l] of windows) if (!l.length || t - l[l.length - 1] >= 60_000) windows.delete(k);++ return true;++ }++++ const mapByMessage = (id) => db.prepare("SELECT * FROM relay_message_map WHERE platform = 'discord' AND external_message_id = ?").get(String(id)) || null;++ const mappingById = (id) => db.prepare('SELECT * FROM relay_mappings WHERE id = ?').get(id) || null;++++ function ignore(reason) { counts.ignored++; return { status: 'ignored', reason }; }++ function failure(event, d, mappingId, error) {++ db.prepare(`INSERT INTO relay_inbound_failures (mapping_id, event, external_channel_id, external_message_id, external_author_id, error)++ VALUES (?, ?, ?, ?, ?, ?)`).run(mappingId || null, event, d.channel_id ? String(d.channel_id) : null, d.id ? String(d.id) : null, d.author && d.author.id ? String(d.author.id) : null, String(error).slice(0, 300));++ db.prepare('DELETE FROM relay_inbound_failures WHERE id <= (SELECT MAX(id) FROM relay_inbound_failures) - ?').run(KEEP_FAILURES);++ counts.failed++;++ return { status: 'failed', error };++ }++ /** Is the local thread still somewhere a reply from Discord may land? → null or why not */++ function closed(thread) {++ if (!thread || thread.deleted_at) return 'the thread was deleted';++ const space = forumStore.getSpaceById(db, thread.space_id);++ if (!space || space.visibility !== 'public' || space.members_only_owner || thread.members_only_owner) return 'the thread is not public';++ if (thread.locked) return 'the thread is locked';++ return null;++ }++++ function create(d, botUserId) {++ if (d.webhook_id) return ignore('a webhook message (the relay\'s own included)');++ const author = d.author || {};++ if (!author.id || author.bot || author.system || (botUserId && String(author.id) === String(botUserId))) return ignore('a bot or system message');++ if (d.type !== undefined && !MESSAGE_TYPES.has(Number(d.type))) return ignore('not a plain message or a reply');++ if (!SNOWFLAKE.test(String(d.id || '')) || !SNOWFLAKE.test(String(d.channel_id || ''))) return ignore('malformed ids');++ if (mapByMessage(d.id)) return ignore('already relayed');++ const channel = String(d.channel_id);++ // A Discord thread started from a mapped message, else a reply to one in the same channel.++ let anchor = mapByMessage(channel);++ // (message_reference type 1 is a forward, not a reply: never taken in.)++ if (!anchor && d.message_reference && d.message_reference.message_id && !Number(d.message_reference.type || 0)) {++ const ref = mapByMessage(String(d.message_reference.message_id));++ if (ref && ref.external_channel_id === channel) anchor = ref;++ }++ if (!anchor) return ignore('not a reply to a relayed message');++ const mapping = mappingById(anchor.mapping_id);++ if (!mapping || !mapping.enabled || !mapping.inbound) return ignore('inbound is off for this mapping');++ const mapped = [mapping.discord_channel_id, mapping.discord_thread_id].filter(Boolean);++ const inMapped = mapped.includes(channel) || (channel === anchor.external_message_id && mapped.includes(anchor.external_channel_id));++ if (!inMapped) return ignore('not a mapped channel');++ const thread = db.prepare('SELECT * FROM threads WHERE id = ?').get(anchor.thread_id);++ const why = closed(thread);++ if (why) return failure('MESSAGE_CREATE', d, mapping.id, why);++ if (!allow(`a:${mapping.id}:${author.id}`, perMinute) || !allow(`m:${mapping.id}`, perMinute * 10)) return failure('MESSAGE_CREATE', d, mapping.id, 'rate limited');++ const body = toMarkdown(d, maxChars);++ if (!body) return failure('MESSAGE_CREATE', d, mapping.id, 'the message has no text: is the MESSAGE CONTENT intent on for the bot?');++ const name = cleanName((d.member && d.member.nick) || author.global_name || author.username) || 'Discord';++ try {++ const post = db.transaction(() => {++ const p = forumStore.addPost(db, { thread_id: thread.id, author_subject: null, origin: 'discord', body_markdown: body, relay_author: name });++ db.prepare(`INSERT INTO relay_message_map (platform, mapping_id, direction, local_type, local_id, thread_id, external_channel_id, external_message_id)++ VALUES ('discord', ?, 'in', 'post', ?, ?, ?, ?)`).run(mapping.id, p.id, thread.id, channel, String(d.id));++ return p;++ })();++ counts.applied++;++ return { status: 'applied', post_id: post.id };++ } catch (err) {++ if (/UNIQUE/.test(err.message)) return ignore('already relayed');++ return failure('MESSAGE_CREATE', d, mapping.id, `could not save the post: ${err.message}`);++ }++ }++++ function update(d) {++ if (!d.id) return ignore('malformed ids');++ const row = mapByMessage(d.id);++ if (!row) return ignore('not a relayed message');++ if (row.direction !== 'in') return ignore('the relay\'s own message');++ if (typeof d.content !== 'string') return ignore('no text change');++ const mapping = mappingById(row.mapping_id);++ if (!mapping || !mapping.enabled || !mapping.inbound) return ignore('inbound is off for this mapping');++ const post = forumStore.getPost(db, row.local_id);++ if (!post || post.deleted_at) return failure('MESSAGE_UPDATE', d, mapping.id, 'the post was deleted');++ const why = closed(db.prepare('SELECT * FROM threads WHERE id = ?').get(post.thread_id));++ if (why) return failure('MESSAGE_UPDATE', d, mapping.id, why);++ const body = toMarkdown(d, maxChars);++ if (!body) return failure('MESSAGE_UPDATE', d, mapping.id, 'the edit has no text');++ forumStore.editPost(db, post.id, body, 'discord');++ db.prepare('UPDATE relay_message_map SET updated_at = CURRENT_TIMESTAMP WHERE id = ?').run(row.id);++ counts.applied++;++ return { status: 'applied', post_id: post.id };++ }++++ /** A delete on Discord. Applied whatever the mapping's switches: taking something down never waits. */++ function remove(id) {++ const row = mapByMessage(id);++ if (!row) return ignore('not a relayed message');++ db.transaction(() => {++ db.prepare('UPDATE relay_message_map SET external_deleted_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP WHERE id = ? AND external_deleted_at IS NULL').run(row.id);++ if (row.direction === 'in') forumStore.softDeletePost(db, row.local_id);++ })();++ counts.applied++;++ return { status: 'applied', post_id: row.direction === 'in' ? row.local_id : null, note: row.direction === 'out' ? 'the relay\'s message was deleted on Discord; the thread stays' : undefined };++ }++++ /** One gateway dispatch (type, data, { botUserId }) → result (an array for a bulk delete). */++ function handle(type, d, { botUserId = null } = {}) {++ if (!d || typeof d !== 'object') return ignore('no data');++ if (type === 'MESSAGE_CREATE') return create(d, botUserId);++ if (type === 'MESSAGE_UPDATE') return update(d);++ if (type === 'MESSAGE_DELETE') return remove(String(d.id || ''));++ if (type === 'MESSAGE_DELETE_BULK') return (Array.isArray(d.ids) ? d.ids : []).map((id) => remove(String(id)));++ return ignore('not a message event');++ }++++ // ── staff ────────────────────────────────────────────────++ function listFailures({ all = false, limit = 50 } = {}) {++ return db.prepare(`SELECT f.*, s.slug AS space_slug FROM relay_inbound_failures f LEFT JOIN relay_mappings m ON m.id = f.mapping_id LEFT JOIN spaces s ON s.id = m.space_id++ WHERE (? = 1 OR f.dismissed_at IS NULL) ORDER BY f.id DESC LIMIT ?`).all(all ? 1 : 0, limit)++ .map((f) => ({++ id: f.id, event: f.event, error: f.error, mapping: f.mapping_id ? { id: f.mapping_id, space: f.space_slug || null } : null,++ discord: { channel_id: f.external_channel_id, message_id: f.external_message_id, author_id: f.external_author_id },++ dismissed: !!f.dismissed_at, created_at: f.created_at ? new Date(`${String(f.created_at).replace(' ', 'T')}Z`).toISOString() : null,++ }));++ }++ function dismiss(id) {++ return db.prepare('UPDATE relay_inbound_failures SET dismissed_at = CURRENT_TIMESTAMP WHERE id = ? AND dismissed_at IS NULL').run(id).changes;++ }++++ return { handle, listFailures, dismiss, stats: () => ({ ...counts }) };++}++++module.exports = { createDiscordInbound, toMarkdown, cleanName };@@@ -159,7 +159,7 @@ function authorPanel(p) {return `<aside class="postbit-author">${pic ? `<img class="postbit-avatar" src="${esc(pic)}" alt="" width="64" height="64" loading="lazy">` : `<span class="postbit-avatar postbit-letter" aria-hidden="true">${initial}</span>`}<span class="postbit-name">${a.username ? `<a href="https://openvibe.live/@${encodeURIComponent(a.username)}" rel="noopener">${esc(name)}</a>` : esc(name)}</span>-- ${a.is_ai ? '<span class="badge badge-ai" title="Written by AI, not by a person">AI</span>' : a.is_system ? '<span class="badge">OpenVibe</span>' : ''}++ ${a.is_ai ? '<span class="badge badge-ai" title="Written by AI, not by a person">AI</span>' : a.is_system ? '<span class="badge">OpenVibe</span>' : a.is_relay ? '<span class="badge badge-relay" title="Written on Discord and relayed here">Discord</span>' : ''}${st ? `<dl class="postbit-stats"><div><dt>Posts</dt><dd>${num(st.posts)}</dd></div>${monthYear(st.first_post_at) ? `<div><dt>Since</dt><dd>${esc(monthYear(st.first_post_at))}</dd></div>` : ''}</dl>${(st.ratings || []).length ? `<p class="postbit-ratings" aria-label="Ratings received">${st.ratings.map((r) => `<span title="${esc(r.label)}">${r.emoji} ${num(r.count)}</span>`).join(' ')}</p>` : ''}` : ''}</aside>`;@@@ -23,6 +23,7 @@ function who(a) {if (!a) return '<span class="author"><span class="avatar avatar-letter" aria-hidden="true">?</span><span>Anonymous</span></span>';if (a.is_ai) return `<span class="author">${authorHtml({ display_name: a.display_name }, { link: false })}</span> <span class="badge badge-ai" title="Written by AI, not by a person">AI</span>`;if (a.is_system) return `<span class="author">${authorHtml({ display_name: a.display_name }, { link: false })}</span>`;++ if (a.is_relay) return `<span class="author">${authorHtml({ display_name: a.display_name }, { link: false })}</span> <span class="badge badge-relay" title="Written on Discord and relayed here">Discord</span>`;return authorHtml(a);}nnew file mode 100644@@@ -0,0 +1,97 @@++'use strict';++/**++ * Stand-ins for Discord in the relay tests.++ *++ * startWebhooks(): an HTTP server speaking the webhook API the relay uses:++ * POST /api/webhooks/:id/:token?wait=true[&thread_id=] → 200 the message { id, channel_id, webhook_id } (204 without wait)++ * PATCH /api/webhooks/:id/:token/messages/:mid[?thread_id] → 200 the message++ * DELETE /api/webhooks/:id/:token/messages/:mid[?thread_id] → 204++ * an unknown or deleted message → 404 { code: 10008, message: 'Unknown Message' }++ * Message ids count up from 1100000000000000000; a webhook's channel is 2000000000000000000 + its id,++ * or the thread_id it was sent to. `plan` queues canned answers ({ status, body, headers } or++ * { hang: true }) that take precedence; `hits` records every request.++ *++ * createGateway(): a fake WebSocket class (gw.WebSocket) the gateway client is given, with the++ * server side scripted by the test: gw.sockets, gw.push(sock, payload), gw.drop(sock, code), and++ * gw.onOpen / gw.onSend hooks. Sockets open on the next tick.++ */++const http = require('http');++++function startWebhooks() {++ const hits = [];++ const plan = [];++ const messages = new Map();++ let next = 1100000000000000000n;++ const server = http.createServer((req, res) => {++ let raw = '';++ req.on('data', (c) => { raw += c; });++ req.on('end', () => {++ const u = new URL(req.url, 'http://discord.test');++ const body = raw ? JSON.parse(raw) : null;++ hits.push({ method: req.method, path: u.pathname + u.search, pathname: u.pathname, query: Object.fromEntries(u.searchParams), body });++ const json = (status, obj) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(obj === undefined ? '' : JSON.stringify(obj)); };++ const canned = plan.shift();++ if (canned) {++ if (canned.hang) return undefined;++ res.writeHead(canned.status, { 'Content-Type': 'application/json', ...(canned.headers || {}) });++ return res.end(canned.body ? JSON.stringify(canned.body) : '');++ }++ const m = u.pathname.match(/^\/api\/webhooks\/(\d+)\/([^/]+)(?:\/messages\/(\d+))?$/);++ if (!m) return json(404, { message: 'Unknown Webhook', code: 10015 });++ const [, hookId, , mid] = m;++ if (req.method === 'POST' && !mid) {++ const id = String(next++);++ const channelId = u.searchParams.get('thread_id') || String(2000000000000000000n + BigInt(hookId));++ messages.set(id, { id, channel_id: channelId, webhook_id: hookId, body, deleted: false });++ if (u.searchParams.get('wait') !== 'true') { res.writeHead(204); return res.end(); }++ return json(200, { id, channel_id: channelId, webhook_id: hookId, content: body && body.content });++ }++ const msg = mid ? messages.get(mid) : null;++ if (!msg || msg.deleted) return json(404, { message: 'Unknown Message', code: 10008 });++ if (req.method === 'PATCH') { msg.body = body; return json(200, { id: msg.id, channel_id: msg.channel_id, webhook_id: msg.webhook_id, content: body && body.content }); }++ if (req.method === 'DELETE') { msg.deleted = true; res.writeHead(204); return res.end(); }++ return json(405, { message: '405: Method Not Allowed', code: 0 });++ });++ });++ return new Promise((resolve) => server.listen(0, '127.0.0.1', () => resolve({++ url: `http://127.0.0.1:${server.address().port}`, hits, plan, messages,++ close: () => new Promise((r) => server.close(r)),++ })));++}++++function createGateway() {++ const gw = { sockets: [], onOpen: null, onSend: null };++ gw.WebSocket = class FakeWebSocket extends EventTarget {++ constructor(url) {++ super();++ this.url = url;++ this.readyState = 0;++ this.sent = [];++ this.closedWith = null;++ gw.sockets.push(this);++ setImmediate(() => {++ if (this.readyState !== 0) return;++ this.readyState = 1;++ this.dispatchEvent(new Event('open'));++ if (gw.onOpen) gw.onOpen(this);++ });++ }++ send(data) {++ const p = JSON.parse(data);++ this.sent.push(p);++ if (gw.onSend) gw.onSend(this, p);++ }++ close(code = 1000, reason = '') {++ if (this.readyState === 3) return;++ this.readyState = 3;++ this.closedWith = code;++ setImmediate(() => this.dispatchEvent(Object.assign(new Event('close'), { code, reason })));++ }++ };++ gw.push = (sock, payload) => sock.dispatchEvent(Object.assign(new Event('message'), { data: JSON.stringify(payload) }));++ gw.drop = (sock, code, reason = '') => { sock.readyState = 3; sock.dispatchEvent(Object.assign(new Event('close'), { code, reason })); };++ gw.last = () => gw.sockets[gw.sockets.length - 1];++ return gw;++}++++module.exports = { startWebhooks, createGateway };nnew file mode 100644@@@ -0,0 +1,74 @@++'use strict';++/**++ * A stand-in for OpenVibe.Events in the relay tests: the publish route Community's outbox uses and++ * the pull route the relay's Events worker reads.++ *++ * POST /api/v1/events one envelope or { events: [...] } → { event_id, seq, duplicate } | { results }++ * GET /api/v1/events?topic=a.*,b.*&after_seq=&limit= → { events: [{ seq, event }], next_after_seq, latest_seq, gap? }++ *++ * Topic patterns follow Events (`*` is one or more whole segments). Every call needs a Bearer token.++ * `failNext` answers that many calls with 503; `prunedThrough` makes seq ≤ it look pruned (a gap).++ */++const http = require('http');++++function matcher(pattern) {++ return new RegExp(`^${pattern.split('.').map((s) => (s === '*' ? '[a-z0-9_]+(?:\\.[a-z0-9_]+)*' : s)).join('\\.')}$`);++}++++function startEvents() {++ const log = []; // { seq, event }++ const calls = [];++ const state = { failNext: 0, prunedThrough: 0 };++ const server = http.createServer((req, res) => {++ let raw = '';++ req.on('data', (c) => { raw += c; });++ req.on('end', () => {++ const u = new URL(req.url, 'http://events.test');++ calls.push({ method: req.method, path: u.pathname + u.search, auth: req.headers.authorization || null });++ const json = (status, obj) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(obj)); };++ if (!/^Bearer \S+/.test(req.headers.authorization || '')) return json(401, { type: 'about:blank', code: 'auth.required', status: 401 });++ if (state.failNext > 0) { state.failNext--; return json(503, { type: 'about:blank', code: 'events.unavailable', status: 503, detail: 'down for a test' }); }++ if (u.pathname === '/api/v1/events' && req.method === 'POST') {++ const body = JSON.parse(raw || '{}');++ const batch = Array.isArray(body.events);++ const results = (batch ? body.events : [body]).map((env) => {++ const known = log.find((x) => x.event.event_id === env.event_id);++ if (known) return { event_id: env.event_id, seq: known.seq, duplicate: true };++ const seq = log.length + 1;++ log.push({ seq, event: env });++ return { event_id: env.event_id, seq, duplicate: false };++ });++ return json(201, batch ? { results } : results[0]);++ }++ if (u.pathname === '/api/v1/events' && req.method === 'GET') {++ const patterns = String(u.searchParams.get('topic') || '*').split(',').map(matcher);++ const after = Number(u.searchParams.get('after_seq') || 0);++ const limit = Number(u.searchParams.get('limit') || 100);++ const out = {};++ let from = after;++ if (state.prunedThrough && after < state.prunedThrough) { out.gap = { from_seq: after + 1, to_seq: state.prunedThrough }; from = state.prunedThrough; }++ const rows = [];++ let cursor = from;++ for (const item of log) {++ if (item.seq <= from) continue;++ cursor = item.seq;++ if (patterns.some((re) => re.test(item.event.event_type))) rows.push(item);++ if (rows.length >= limit) break;++ }++ out.events = rows;++ out.next_after_seq = cursor;++ out.latest_seq = log.length;++ return json(200, out);++ }++ return json(404, { type: 'about:blank', code: 'route.not_found', status: 404 });++ });++ });++ /** Put an event straight into the log (as if some producer had published it). → seq */++ function add(event) { log.push({ seq: log.length + 1, event }); return log.length; }++ return new Promise((resolve) => server.listen(0, '127.0.0.1', () => resolve({++ url: `http://127.0.0.1:${server.address().port}`, log, calls, state, add,++ close: () => new Promise((r) => server.close(r)),++ })));++}++++module.exports = { startEvents };nnew file mode 100644@@@ -0,0 +1,207 @@++'use strict';++/**++ * The Discord relay as an Events worker (relay/events-worker.js): creates come from Community's own++ * community.thread.created / community.post.created read back from OpenVibe.Events' pull API with a++ * stored cursor. First start begins at the head (history is never announced); new events queue++ * deliveries (source 'events', the event id) that are sent once; a replay queues nothing twice; the++ * cursor survives a restart; members-only, Discord-origin, redacted and foreign events are ignored;++ * Events being down or a retention gap shows in the status; while the worker runs the forum's own++ * calls queue no creates. Last, the whole path in the app: an API write → Community's outbox → a fake++ * Events → the worker → a fake Discord webhook.++ */++const assert = require('assert');++const { ids } = require('openvibe-contracts');++const { createClient } = require('openvibe-sdk/core');++const { createEventsClient } = require('openvibe-sdk/events');++const { boot, check, done } = require('./helpers/app');++const { startWebhooks } = require('./helpers/fake-discord');++const { startEvents } = require('./helpers/fake-events');++const { openDb } = require('../server/db');++const forumStore = require('../server/forum/store');++const { createDiscordRelay } = require('../server/relay/discord');++const { createRelayEventsWorker, CURSOR } = require('../server/relay/events-worker');++++let evn = 0;++function envelope(type, payload, extra = {}) {++ evn++;++ return { event_id: `evt_TEST${String(evn).padStart(22, '0')}`, event_type: type, source: 'community', version: 1, timestamp: new Date().toISOString(), payload, ...extra };++}++const threadEvent = (t, visibility = 'public') => envelope('community.thread.created', { thread_id: t.id, space: 'general', visibility, author: null, url: null });++const postEvent = (p, t, visibility = 'public') => envelope('community.post.created', { post_id: p.id, thread_id: t.id, space: 'general', visibility, author: null, url: null });++++(async () => {++ const hook = await startWebhooks();++ const events = await startEvents();++ const db = openDb(':memory:');++ const general = forumStore.getSpace(db, 'general');++ const env = { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/1/general` };++ const relay = createDiscordRelay({ db, config: { baseUrl: 'https://openvibe.community' }, env, enabled: true, baseMs: 1000, maxAttempts: 3 });++ relay.addMapping({ space_id: general.id, webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' });++ const client = createEventsClient(createClient({ baseUrls: { events: events.url }, getToken: async () => 'test-token', retries: 0 }), { source: 'community' });++ const worker = createRelayEventsWorker({ db, relay, events: client, pollMs: 3_600_000, pageLimit: 2 });++ relay.attach({ worker });++ const alex = ids.newId('user');++ const newThread = (title, extra = {}) => forumStore.createThread(db, { space_id: general.id, title, author_subject: alex, body_markdown: `${title} body`, ...extra }).thread;++ const deliveries = () => db.prepare('SELECT * FROM relay_deliveries ORDER BY id').all();++ const settle = () => new Promise((r) => setImmediate(r));++++ await check('off without Events or the client secret, or with DISCORD_RELAY_EVENTS=off; the forum queues creates then', async () => {++ const a = createRelayEventsWorker({ db, relay, eventsUrl: null, clientSecret: 'x', tokenUrl: 'http://127.0.0.1:9/oauth/token' });++ assert.deepStrictEqual([a.enabled, a.status().reason], [false, 'EVENTS_URL and OV_OAUTH_CLIENT_SECRET are not both set']);++ const b = createRelayEventsWorker({ db, relay, eventsUrl: events.url, clientSecret: '', tokenUrl: 'http://127.0.0.1:9/oauth/token' });++ assert.strictEqual(b.enabled, false);++ const c = createRelayEventsWorker({ db, relay, events: client, enabled: false });++ assert.deepStrictEqual([c.enabled, c.status().reason], [false, 'DISCORD_RELAY_EVENTS=off']);++ const d = createRelayEventsWorker({ db, relay: createDiscordRelay({ db, env }), events: client });++ assert.match(d.status().reason, /relay is off/);++ c.start();++ assert.strictEqual(relay.status().creates_from, 'forum');++ assert.deepStrictEqual(await c.tick(), { queued: 0 });++ });++++ let before;++ await check('first start begins at the head: threads from before are never announced', async () => {++ const old1 = newThread('Older one');++ const old2 = newThread('Older two');++ events.add(threadEvent(old1));++ events.add(threadEvent(old2));++ events.add(envelope('community.paste.created', { paste_id: 'x' }));++ relay.start(); // starts the worker too (its interval is an hour here: ticks are driven by the test)++ await worker.tick();++ assert.strictEqual(relay.status().creates_from, 'events');++ const st = worker.status();++ assert.deepStrictEqual([st.enabled, st.cursor, st.latest_seq, st.started_at_seq, st.lag], [true, 3, 3, 3, 0]);++ assert.strictEqual(deliveries().length, 0);++ assert.deepStrictEqual(db.prepare('SELECT name, cursor FROM relay_cursors').all(), [{ name: CURSOR, cursor: 3 }]);++ before = events.calls.length;++ });++++ let t1;++ let p1;++ await check('new events queue creates (source events, the event id) that are sent once; the forum\'s own calls queue none', async () => {++ t1 = newThread('From the stream');++ assert.strictEqual(relay.enqueueThread(t1, general), 0, 'the forum queues no creates while the worker runs');++ p1 = forumStore.addPost(db, { thread_id: t1.id, author_subject: alex, body_markdown: 'a reply via events' });++ assert.strictEqual(relay.enqueuePost(p1, t1, general), 0);++ const e1 = threadEvent(t1);++ const e2 = postEvent(p1, t1);++ events.add(e1);++ events.add(envelope('community.comment.created', { comment_id: 1 })); // not on the relay's topics: never returned++ events.add(e2);++ assert.deepStrictEqual(await worker.tick(), { queued: 2 });++ const rows = deliveries();++ assert.deepStrictEqual(rows.map((d) => [d.action, d.source, d.event_id, d.post_id]), [['create', 'events', e1.event_id, null], ['create', 'events', e2.event_id, p1.id]]);++ assert.ok(events.calls.slice(before).every((c) => /topic=community\.thread\.\*%2Ccommunity\.post\.\*/.test(c.path) && c.auth === 'Bearer test-token'), JSON.stringify(events.calls.slice(before)));++ await settle(); await relay.drain(); await relay.drain();++ assert.deepStrictEqual(hook.hits.map((h) => h.method), ['POST', 'POST']);++ assert.match(hook.hits[1].body.content, /replied to \*\*From the stream\*\*/);++ assert.deepStrictEqual([worker.status().cursor, worker.status().lag], [6, 0]);++ });++++ await check('a replay (cursor moved back) queues nothing twice and posts nothing twice', async () => {++ db.prepare('UPDATE relay_cursors SET cursor = 3 WHERE name = ?').run(CURSOR);++ assert.deepStrictEqual(await worker.tick(), { queued: 0 });++ await relay.drain();++ assert.strictEqual(hook.hits.length, 2);++ assert.strictEqual(worker.status().cursor, 6);++ });++++ await check('the cursor survives a restart: a new worker carries on from it (no second "start at head")', async () => {++ const t2 = newThread('While restarting');++ events.add(threadEvent(t2));++ const again = createRelayEventsWorker({ db, relay, events: client, pollMs: 3_600_000 });++ assert.deepStrictEqual(await again.tick(), { queued: 1 });++ assert.strictEqual(again.status().started_at_seq, null);++ assert.strictEqual(deliveries().pop().thread_id, t2.id);++ });++++ await check('members-only, Discord-origin, redacted and foreign events queue nothing', async () => {++ const n = deliveries().length;++ const gated = newThread('Gated', { members_only_owner: alex });++ const fromDiscord = forumStore.addPost(db, { thread_id: t1.id, origin: 'discord', body_markdown: 'hi from discord', relay_author: 'Kim' });++ const t3 = newThread('Looks public');++ events.add(threadEvent(gated, 'members'));++ events.add(threadEvent(gated)); // even a (wrongly) public-looking event: the row decides++ events.add(postEvent(fromDiscord, t1));++ events.add({ ...threadEvent(t3), source: 'live' });++ events.add({ ...threadEvent(t3), payload: { redacted: true } });++ assert.deepStrictEqual(await worker.tick(), { queued: 0 });++ assert.strictEqual(deliveries().length, n);++ });++++ await check('Events down: the error shows, the cursor stays, and it catches up once Events answers', async () => {++ const t4 = newThread('After the outage');++ events.add(threadEvent(t4));++ const cursor = worker.status().cursor;++ events.state.failNext = 1;++ assert.deepStrictEqual(await worker.tick(), { queued: 0 });++ const st = worker.status();++ assert.ok(st.last_error, 'the error is shown');++ assert.strictEqual(st.cursor, cursor);++ assert.strictEqual(st.lag, st.latest_seq - cursor);++ assert.deepStrictEqual(await worker.tick(), { queued: 1 });++ assert.strictEqual(worker.status().last_error, null);++ });++++ await check('a retention gap is logged and shown to staff; reading carries on after it', async () => {++ db.prepare('UPDATE relay_cursors SET cursor = 1 WHERE name = ?').run(CURSOR);++ events.state.prunedThrough = 5;++ await worker.tick();++ assert.deepStrictEqual([worker.status().last_gap.from_seq, worker.status().last_gap.to_seq], [2, 5]);++ assert.strictEqual(worker.status().cursor, events.log.length);++ events.state.prunedThrough = 0;++ assert.strictEqual(relay.status().events_worker.last_gap.to_seq, 5);++ });++++ await check('the whole path in the app: API write → outbox → Events → worker → Discord, and back out on edit and delete', async () => {++ const hook2 = await startWebhooks();++ const events2 = await startEvents();++ process.env.EVENTS_URL = events2.url;++ process.env.DISCORD_RELAY_EVENTS_POLL_MS = '40';++ try {++ const t = await boot({++ authority: 'community',++ appOpts: { relayOptions: { enabled: true, env: { DISCORD_WEBHOOK_GENERAL: `${hook2.url}/api/webhooks/7/app` } }, forumLimits: { threads: { cooldownSec: 0 }, posts: { cooldownSec: 0 } } },++ });++ const outbox = require('../server/events').init(t.db, { eventsUrl: events2.url, intervalMs: 40 });++ assert.ok(outbox, 'the outbox publishes to the fake Events');++ const net = t.network;++ const samU = net.addUser({ network_user_id: 9, username: 'sam', display_name: 'Sam' });++ const samJwt = net.sign({ id: 9, subject_id: samU.subject_id, username: 'sam', role: 'user' });++ const adminJwt = net.sign({ id: 1, subject_id: ids.newId('user'), username: 'boss', role: 'admin' });++ const call = (p, { method = 'GET', cookie, json } = {}) => t.get(p, { method, headers: json !== undefined ? { 'content-type': 'application/json' } : {}, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });++ const waitFor = async (fn, what) => { for (let i = 0; i < 200; i++) { if (fn()) return; await new Promise((r) => setTimeout(r, 20)); } assert.fail(`timed out waiting for ${what}`); };++ assert.strictEqual((await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' } })).status, 201);++ await waitFor(() => t.app.locals.relay.status().events_worker.cursor != null, 'the worker\'s first read');++ const st = (await call('/api/v1/relay/status', { cookie: adminJwt })).json();++ assert.deepStrictEqual([st.creates_from, st.events_worker.enabled], ['events', true]);++ const th = await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'Round trip', body: 'through Events' } });++ assert.strictEqual(th.status, 201, th.text);++ await waitFor(() => hook2.hits.length >= 1, 'the thread on Discord');++ assert.strictEqual(hook2.hits[0].path, '/api/webhooks/7/app?wait=true');++ assert.ok(events2.log.some((x) => x.event.event_type === 'community.thread.created'), 'it went through Events');++ const d = t.db.prepare('SELECT * FROM relay_deliveries').get();++ assert.deepStrictEqual([d.source, /^evt_/.test(d.event_id)], ['events', true]);++ const reply = await call(`/api/v1/spaces/general/threads/${th.json().thread.slug}/posts`, { method: 'POST', cookie: samJwt, json: { body: 'and a reply' } });++ await waitFor(() => hook2.hits.length >= 2, 'the reply on Discord');++ assert.strictEqual((await call(`/api/v1/posts/${reply.json().post.id}`, { method: 'PUT', cookie: samJwt, json: { body: 'and a reply, edited' } })).status, 200);++ await waitFor(() => hook2.hits.length >= 3, 'the edit on Discord');++ assert.strictEqual((await call(`/api/v1/spaces/general/threads/${th.json().thread.slug}`, { method: 'DELETE', cookie: samJwt })).status, 200);++ await waitFor(() => hook2.hits.length >= 5, 'the deletes on Discord');++ assert.deepStrictEqual(hook2.hits.map((h) => h.method), ['POST', 'POST', 'PATCH', 'DELETE', 'DELETE']);++ await new Promise((r) => setTimeout(r, 200));++ assert.strictEqual(hook2.hits.length, 5, 'nothing was sent twice');++ await t.app.locals.relay.stop();++ await require('../server/events').stop();++ await t.close();++ } finally {++ delete process.env.EVENTS_URL; delete process.env.DISCORD_RELAY_EVENTS_POLL_MS;++ await hook2.close(); await events2.close();++ }++ });++++ await relay.stop();++ await hook.close();++ await events.close();++ done();++})();nnew file mode 100644@@@ -0,0 +1,337 @@++'use strict';++/**++ * Discord relay, inbound (relay/discord-gateway.js + relay/inbound.js), against a fake gateway:++ * HELLO → IDENTIFY (token, intents GUILD_MESSAGES | MESSAGE_CONTENT), heartbeats with the last seq,++ * RECONNECT and zombied connections resume, INVALID_SESSION identifies again, fatal close codes stop++ * it with a reason, a missing HELLO reconnects, stop() closes. Then what becomes a post: replies to++ * relayed messages (or messages in a Discord thread started from one) in mapped channels only, with++ * origin 'discord', no author subject and the Discord name; mentions defused; size and rate limits;++ * edits and deletes through the map; webhook/bot/own/system messages, unmapped channels and++ * non-replies ignored; closed threads and empty messages recorded as failures; nothing from Discord++ * ever relayed back out. Last, the app: status and failures at /api/v1/relay (staff, no token), and++ * the post on the thread page with its Discord badge.++ */++const assert = require('assert');++const { ids } = require('openvibe-contracts');++const { boot, check, done } = require('./helpers/app');++const { startWebhooks, createGateway } = require('./helpers/fake-discord');++const { openDb } = require('../server/db');++const forumStore = require('../server/forum/store');++const { createDiscordRelay } = require('../server/relay/discord');++const { createDiscordGateway, DEFAULT_URL, INTENTS } = require('../server/relay/discord-gateway');++const { createDiscordInbound, toMarkdown } = require('../server/relay/inbound');++const { createForumService } = require('../server/forum/service');++++const sleep = (ms) => new Promise((r) => setTimeout(r, ms));++const BOT = '4000000000000000001';++const KIM = '4000000000000000002';++const LEE = '4000000000000000003';++let snow = 5000000000000000000n;++const nextId = () => String(snow++);++++/** A scripted gateway: HELLO on open, READY on IDENTIFY, RESUMED on RESUME, ACK on heartbeat (unless paused). */++function scriptedGateway() {++ const gw = createGateway();++ gw.ack = true;++ gw.seq = 0;++ gw.onOpen = (sock) => gw.push(sock, { op: 10, d: { heartbeat_interval: 30 } });++ gw.onSend = (sock, p) => {++ if (p.op === 2) gw.push(sock, { op: 0, t: 'READY', s: ++gw.seq, d: { session_id: `sess-${gw.sockets.length}`, resume_gateway_url: 'wss://resume.discord.test', user: { id: BOT, bot: true } } });++ if (p.op === 6) gw.push(sock, { op: 0, t: 'RESUMED', s: ++gw.seq, d: {} });++ if (p.op === 1 && gw.ack) gw.push(sock, { op: 11 });++ };++ gw.dispatch = (t, d) => gw.push(gw.last(), { op: 0, t, s: ++gw.seq, d });++ return gw;++}++const waitFor = async (fn, what, ms = 2000) => { const end = Date.now() + ms; while (Date.now() < end) { if (fn()) return; await sleep(5); } assert.fail(`timed out waiting for ${what}`); };++const sent = (gw, op) => gw.sockets.flatMap((s) => s.sent).filter((p) => p.op === op);++++(async () => {++ // ── the gateway client ───────────────────────────────────++ const opts = { token: 'test-bot-token', minBackoffMs: 5, maxBackoffMs: 40, helloTimeoutMs: 150, invalidSessionMs: 5, jitter: () => 0 };++++ await check('off without a token or a WebSocket client, and says why', async () => {++ const a = createDiscordGateway({ token: '' });++ assert.deepStrictEqual([a.enabled, a.status()], [false, { enabled: false, reason: 'DISCORD_BOT_TOKEN is not set' }]);++ const b = createDiscordGateway({ token: 'x', WebSocketImpl: null });++ assert.match(b.status().reason, /no built-in WebSocket/);++ b.start();++ assert.strictEqual(typeof globalThis.WebSocket, 'function', 'Node 22 has the client the gateway uses');++ });++++ await check('HELLO → IDENTIFY with the token and intents; heartbeats carry the last seq and are ACKed; READY is ready', async () => {++ const gw = scriptedGateway();++ const seen = [];++ const g = createDiscordGateway({ ...opts, WebSocketImpl: gw.WebSocket, onDispatch: (t, d) => { seen.push([t, d.id]); } });++ g.start();++ await waitFor(() => g.status().state === 'ready', 'READY');++ assert.strictEqual(gw.sockets[0].url, DEFAULT_URL);++ const [identify] = sent(gw, 2);++ assert.deepStrictEqual([identify.d.token, identify.d.intents], ['test-bot-token', INTENTS.GUILD_MESSAGES | INTENTS.MESSAGE_CONTENT]);++ assert.strictEqual(INTENTS.GUILD_MESSAGES | INTENTS.MESSAGE_CONTENT, 33280);++ await waitFor(() => sent(gw, 1).length >= 2, 'two heartbeats');++ assert.strictEqual(sent(gw, 1).pop().d, 1, 'the last seq');++ gw.dispatch('MESSAGE_CREATE', { id: '1' });++ gw.dispatch('GUILD_CREATE', { id: '2' }); // not a message event: dropped here++ gw.dispatch('MESSAGE_DELETE', { id: '3' });++ assert.deepStrictEqual(seen, [['MESSAGE_CREATE', '1'], ['MESSAGE_DELETE', '3']]);++ const st = g.status();++ assert.deepStrictEqual([st.state, st.session, st.bot_user_id, st.events_handled], ['ready', true, BOT, 2]);++ assert.ok(!JSON.stringify(st).includes('test-bot-token'), 'the token never shows');++ await g.stop();++ assert.strictEqual(gw.sockets[0].closedWith, 1000);++ assert.strictEqual(g.status().state, 'stopped');++ await sleep(60);++ assert.strictEqual(gw.sockets.length, 1, 'no reconnect after stop()');++ });++++ await check('RECONNECT, a zombied connection and an abnormal close all RESUME on the resume URL with the session and seq', async () => {++ const gw = scriptedGateway();++ const g = createDiscordGateway({ ...opts, WebSocketImpl: gw.WebSocket });++ g.start();++ await waitFor(() => g.status().state === 'ready', 'READY');++ gw.dispatch('MESSAGE_CREATE', { id: '10' });++ gw.push(gw.sockets[0], { op: 7, d: null });++ await waitFor(() => gw.sockets.length === 2 && sent(gw, 6).length === 1, 'a resume');++ assert.strictEqual(gw.sockets[0].closedWith, 4000, 'closed with a non-1000 code, so the session stays resumable');++ assert.strictEqual(gw.sockets[1].url, 'wss://resume.discord.test/?v=10&encoding=json');++ assert.deepStrictEqual(sent(gw, 6)[0].d, { token: 'test-bot-token', session_id: 'sess-1', seq: 2 });++ await waitFor(() => g.status().state === 'ready', 'RESUMED');++ // No ACK: the next heartbeat finds the last one unanswered and reconnects.++ gw.ack = false;++ await waitFor(() => gw.sockets.length === 3, 'a reconnect after a missing ACK', 1000);++ gw.ack = true;++ await waitFor(() => sent(gw, 6).length === 2, 'a second resume');++ assert.match(g.status().last_reconnect.reason, /no heartbeat ACK \(zombied connection\)/);++ // An abnormal close (1006) resumes too.++ await waitFor(() => g.status().state === 'ready', 'ready again');++ gw.drop(gw.last(), 1006);++ await waitFor(() => sent(gw, 6).length === 3, 'a third resume');++ assert.strictEqual(sent(gw, 2).length, 1, 'identified only once');++ await g.stop();++ });++++ await check('INVALID_SESSION (not resumable) and close codes 4007/4009 identify afresh; no HELLO reconnects', async () => {++ const gw = scriptedGateway();++ const g = createDiscordGateway({ ...opts, WebSocketImpl: gw.WebSocket });++ g.start();++ await waitFor(() => g.status().state === 'ready', 'READY');++ gw.push(gw.last(), { op: 9, d: false });++ await waitFor(() => sent(gw, 2).length === 2, 'a second IDENTIFY');++ assert.strictEqual(gw.last().url, DEFAULT_URL, 'a fresh session starts on the main URL');++ await waitFor(() => g.status().state === 'ready', 'READY again');++ gw.drop(gw.last(), 4009);++ await waitFor(() => sent(gw, 2).length === 3, 'IDENTIFY after 4009');++ await waitFor(() => g.status().state === 'ready', 'READY after 4009');++ // A gateway that never says HELLO.++ gw.onOpen = null;++ gw.drop(gw.last(), 1001);++ await waitFor(() => gw.sockets.length >= 5, 'a reconnect after the HELLO timeout', 1500);++ assert.match(g.status().last_error, /no HELLO/);++ await g.stop();++ });++++ await check('fatal close codes stop it with the reason (4004 a wrong token, 4014 the MESSAGE CONTENT intent)', async () => {++ for (const [code, re] of [[4004, /DISCORD_BOT_TOKEN is wrong/], [4014, /MESSAGE CONTENT intent/]]) {++ const gw = scriptedGateway();++ const g = createDiscordGateway({ ...opts, WebSocketImpl: gw.WebSocket });++ g.start();++ await waitFor(() => g.status().state === 'ready', 'READY');++ gw.drop(gw.last(), code);++ await sleep(80);++ assert.strictEqual(gw.sockets.length, 1, `no reconnect after ${code}`);++ assert.strictEqual(g.status().state, 'failed');++ assert.match(g.status().last_error, re);++ await g.stop();++ }++ });++++ // ── inbound: what becomes a post ────────────────────────++ const hook = await startWebhooks();++ const db = openDb(':memory:');++ const general = forumStore.getSpace(db, 'general');++ const relay = createDiscordRelay({ db, config: { baseUrl: 'https://openvibe.community' }, env: { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/1/general` }, enabled: true });++ relay.addMapping({ space_id: general.id, webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL', inbound: true });++ const CHANNEL = '2000000000000000001'; // the fake webhook 1's channel (learned on the first send)++ const inbound = createDiscordInbound({ db, perMinute: 3, maxChars: 200 });++ const forum = createForumService({ db, relay, limits: { threads: { cooldownSec: 0, perMinute: 100 }, posts: { cooldownSec: 0, perMinute: 100 }, threadsPerDay: 0 } });++ const sam = ids.newId('user');++ const samV = { kind: 'user', subject: sam, staff: false };++ const settle = async () => { await new Promise((r) => setImmediate(r)); await relay.drain(); await relay.drain(); };++ const { thread } = await forum.createThread(samV, 'general', { title: 'Talk to Discord', body: 'Say hi over there' });++ await settle();++ const threadMsg = db.prepare("SELECT external_message_id FROM relay_message_map WHERE local_type = 'thread' AND local_id = ?").get(thread.id).external_message_id;++ const failures = () => db.prepare('SELECT * FROM relay_inbound_failures ORDER BY id').all();++ const msg = (extra = {}) => ({ id: nextId(), channel_id: CHANNEL, guild_id: '6000000000000000001', type: 19, content: 'hello from Discord', author: { id: KIM, username: 'kim', global_name: 'Kim' }, message_reference: { message_id: threadMsg, channel_id: CHANNEL }, mentions: [], attachments: [], ...extra });++ const postsOf = (tid) => db.prepare('SELECT * FROM posts WHERE thread_id = ? ORDER BY id').all(tid);++++ await check('a Discord reply to a relayed message becomes a post: origin discord, nobody on the site, the Discord name, mapped', async () => {++ const m = msg({ content: 'Hi <@4000000000000000003> and <@&4000000000000000009> in <#4000000000000000010> @everyone <:wave:4000000000000000011> <t:1790000000:R>', mentions: [{ id: LEE, username: 'lee', global_name: 'Lee' }], member: { nick: 'Kim (mod)' } });++ const r = inbound.handle('MESSAGE_CREATE', m, { botUserId: BOT });++ assert.strictEqual(r.status, 'applied', JSON.stringify(r));++ const p = forumStore.getPost(db, r.post_id);++ assert.deepStrictEqual([p.thread_id, p.origin, p.author_subject, p.relay_author], [thread.id, 'discord', null, 'Kim (mod)']);++ assert.strictEqual(p.body_markdown, 'Hi @Lee and @role in #channel @\u200beveryone :wave: 2026-09-21 14:13 UTC');++ const row = db.prepare('SELECT * FROM relay_message_map WHERE external_message_id = ?').get(m.id);++ assert.deepStrictEqual([row.direction, row.local_type, row.local_id, row.thread_id, row.external_channel_id], ['in', 'post', p.id, thread.id, CHANNEL]);++ const shown = (await forum.getThread(samV, 'general', thread.slug, {})).posts.find((x) => x.id === p.id);++ assert.deepStrictEqual([shown.author.display_name, shown.author.is_relay, shown.author.subject, shown.origin], ['Kim (mod)', true, null, 'discord']);++ // The same message again (a gateway replay) is not taken twice.++ assert.deepStrictEqual(inbound.handle('MESSAGE_CREATE', m, { botUserId: BOT }), { status: 'ignored', reason: 'already relayed' });++ });++++ await check('a message in a Discord thread started from a relayed message becomes a post in that thread', async () => {++ const r = inbound.handle('MESSAGE_CREATE', msg({ channel_id: threadMsg, type: 0, message_reference: undefined, content: 'in the Discord thread' }));++ assert.strictEqual(r.status, 'applied');++ assert.strictEqual(forumStore.getPost(db, r.post_id).thread_id, thread.id);++ // A reply to that Discord-origin post (mapped 'in') lands in the same thread too.++ const inId = db.prepare("SELECT external_message_id FROM relay_message_map WHERE local_id = ? AND direction = 'in'").get(r.post_id).external_message_id;++ const r2 = inbound.handle('MESSAGE_CREATE', msg({ channel_id: threadMsg, message_reference: { message_id: inId }, content: 'reply to a reply', author: { id: LEE, username: 'lee' } }));++ assert.strictEqual(forumStore.getPost(db, r2.post_id).relay_author, 'lee');++ });++++ await check('everything else is ignored: webhooks (the relay\'s own), bots, the bot itself, system messages, non-replies, other channels, inbound off', async () => {++ const n = postsOf(thread.id).length;++ const cases = [++ [msg({ webhook_id: '1' }), /webhook message/],++ [msg({ author: { id: LEE, username: 'lee', bot: true } }), /bot or system/],++ [msg({ author: { id: BOT, username: 'relaybot' } }), /bot or system/],++ [msg({ type: 7 }), /not a plain message/],++ [msg({ message_reference: undefined, type: 0 }), /not a reply to a relayed message/],++ [msg({ message_reference: { message_id: '9999999999999999999' } }), /not a reply to a relayed message/],++ [msg({ channel_id: '7000000000000000001' }), /not a reply to a relayed message/],++ [msg({ type: 0, content: '', message_reference: { type: 1, message_id: threadMsg, channel_id: CHANNEL } }), /not a reply to a relayed message/], // a forward++ ];++ for (const [m, re] of cases) {++ const r = inbound.handle('MESSAGE_CREATE', m, { botUserId: BOT });++ assert.strictEqual(r.status, 'ignored', JSON.stringify(m));++ assert.match(r.reason, re);++ }++ relay.updateMapping(1, { inbound: false });++ assert.match(inbound.handle('MESSAGE_CREATE', msg()).reason, /inbound is off/);++ relay.updateMapping(1, { inbound: true, discord_channel_id: '7000000000000000002' });++ assert.match(inbound.handle('MESSAGE_CREATE', msg()).reason, /not a mapped channel/);++ relay.updateMapping(1, { discord_channel_id: CHANNEL });++ assert.strictEqual(postsOf(thread.id).length, n);++ assert.strictEqual(failures().length, 0, 'ignoring is not a failure');++ });++++ await check('limits: a long message is cut, a fourth message a minute is refused, an empty one explains the intent; attachments are noted', async () => {++ const long = inbound.handle('MESSAGE_CREATE', msg({ author: { id: '4000000000000000020', username: 'long' }, content: 'x'.repeat(500) }));++ const body = forumStore.getPost(db, long.post_id).body_markdown;++ assert.strictEqual(body.length, 200);++ assert.ok(body.endsWith('…'));++ const talker = { id: '4000000000000000021', username: 'talker' };++ for (let i = 0; i < 3; i++) assert.strictEqual(inbound.handle('MESSAGE_CREATE', msg({ author: talker, content: `m${i}` })).status, 'applied');++ const r = inbound.handle('MESSAGE_CREATE', msg({ author: talker, content: 'one too many' }));++ assert.deepStrictEqual([r.status, r.error], ['failed', 'rate limited']);++ const empty = inbound.handle('MESSAGE_CREATE', msg({ author: { id: '4000000000000000022', username: 'quiet' }, content: '' }));++ assert.match(empty.error, /MESSAGE CONTENT intent/);++ const pic = inbound.handle('MESSAGE_CREATE', msg({ author: { id: '4000000000000000023', username: 'pics' }, content: '', attachments: [{ id: '1' }, { id: '2' }] }));++ assert.strictEqual(forumStore.getPost(db, pic.post_id).body_markdown, '_(2 attachments on Discord)_');++ const f = failures();++ assert.deepStrictEqual(f.map((x) => x.error.slice(0, 12)), ['rate limited', 'the message ']);++ assert.deepStrictEqual([f[0].event, f[0].external_author_id, f[0].mapping_id], ['MESSAGE_CREATE', talker.id, 1]);++ assert.ok(!JSON.stringify(f).includes('one too many'), 'failures never keep the text');++ assert.strictEqual(toMarkdown({ content: ' <@123456789012345678> ' }), '@someone');++ });++++ await check('closed threads refuse replies from Discord and say why: locked, members-only, deleted', async () => {++ const t = (await forum.createThread(samV, 'general', { title: 'Soon closed', body: 'x' })).thread;++ await settle();++ const anchor = db.prepare("SELECT external_message_id FROM relay_message_map WHERE local_type = 'thread' AND local_id = ?").get(t.id).external_message_id;++ const to = (extra = {}) => msg({ author: { id: '4000000000000000030', username: 'late' }, message_reference: { message_id: anchor }, ...extra });++ forumStore.setThreadFlags(db, t.id, { locked: true });++ assert.strictEqual(inbound.handle('MESSAGE_CREATE', to()).error, 'the thread is locked');++ forumStore.setThreadFlags(db, t.id, { locked: false });++ forumStore.setThreadMembersOnly(db, t.id, sam);++ assert.strictEqual(inbound.handle('MESSAGE_CREATE', to()).error, 'the thread is not public');++ forumStore.setThreadMembersOnly(db, t.id, null);++ forumStore.softDeleteThread(db, t.id);++ assert.strictEqual(inbound.handle('MESSAGE_CREATE', to()).error, 'the thread was deleted');++ assert.strictEqual(postsOf(t.id).length, 1, 'only the opening post');++ });++++ await check('edits and deletes on Discord follow through the map; a delete of the relay\'s own message leaves the thread', async () => {++ const m = msg({ author: { id: '4000000000000000040', username: 'editor' }, content: 'first words' });++ const { post_id: pid } = inbound.handle('MESSAGE_CREATE', m);++ const r = inbound.handle('MESSAGE_UPDATE', { id: m.id, channel_id: CHANNEL, content: 'second words <@4000000000000000003>', mentions: [{ id: LEE, username: 'lee' }], edited_timestamp: new Date().toISOString() });++ assert.strictEqual(r.status, 'applied');++ const p = forumStore.getPost(db, pid);++ assert.deepStrictEqual([p.body_markdown, p.revision], ['second words @lee', 2]);++ assert.deepStrictEqual(forumStore.listPostVersions(db, pid).map((v) => v.edited_by), [null, 'discord']);++ assert.match(inbound.handle('MESSAGE_UPDATE', { id: m.id, channel_id: CHANNEL, embeds: [] }).reason, /no text change/);++ assert.match(inbound.handle('MESSAGE_UPDATE', { id: threadMsg, channel_id: CHANNEL, content: 'edited on Discord?' }).reason, /relay's own/);++ const other = inbound.handle('MESSAGE_CREATE', msg({ author: { id: '4000000000000000041', username: 'bulk' }, content: 'bulk me' }));++ const otherId = db.prepare("SELECT external_message_id FROM relay_message_map WHERE local_id = ? AND direction = 'in'").get(other.post_id).external_message_id;++ assert.strictEqual(inbound.handle('MESSAGE_DELETE', { id: m.id, channel_id: CHANNEL }).status, 'applied');++ assert.ok(forumStore.getPost(db, pid).deleted_at, 'the post is deleted');++ const bulk = inbound.handle('MESSAGE_DELETE_BULK', { ids: [otherId, '1234567890123456789'], channel_id: CHANNEL });++ assert.deepStrictEqual(bulk.map((x) => x.status), ['applied', 'ignored']);++ assert.ok(forumStore.getPost(db, other.post_id).deleted_at);++ // A moderator on Discord removed the relay's announcement: the map knows, the thread stays, no edit goes there again.++ assert.match(inbound.handle('MESSAGE_DELETE', { id: threadMsg, channel_id: CHANNEL }).note, /thread stays/);++ assert.ok(forumStore.getThread(db, thread.id), 'the thread is still here');++ assert.ok(db.prepare('SELECT external_deleted_at FROM relay_message_map WHERE external_message_id = ?').get(threadMsg).external_deleted_at);++ assert.strictEqual(relay.enqueueEdit(forumStore.getPost(db, db.prepare('SELECT id FROM posts WHERE thread_id = ? AND is_opening = 1').get(thread.id).id)), 0);++ });++++ await check('loop prevention: nothing from Discord is ever relayed back out', async () => {++ await settle();++ const before = hook.hits.length;++ const fromDiscord = postsOf(thread.id).filter((p) => p.origin === 'discord');++ assert.ok(fromDiscord.length >= 5);++ for (const p of fromDiscord) assert.strictEqual(relay.enqueuePost(p, forumStore.getThread(db, thread.id), general), 0);++ assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM relay_deliveries WHERE post_id IN (SELECT id FROM posts WHERE origin = \'discord\')').get().n, 0);++ await settle();++ assert.strictEqual(hook.hits.length, before, 'no webhook call for anything that came from Discord');++ });++++ await check('the app: the gateway starts with the relay; status and failures for staff only, without the token; the post shows with a Discord badge', async () => {++ const gw = scriptedGateway();++ process.env.DISCORD_RELAY_INBOUND = 'on';++ process.env.DISCORD_BOT_TOKEN = 'app-bot-token-not-real';++ let t;++ try {++ t = await boot({++ authority: 'community',++ appOpts: { relayOptions: { enabled: true, env: { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/5/app` } }, gatewayOptions: { WebSocketImpl: gw.WebSocket, minBackoffMs: 5, jitter: () => 0 }, forumLimits: { threads: { cooldownSec: 0 } } },++ });++ } finally { delete process.env.DISCORD_RELAY_INBOUND; delete process.env.DISCORD_BOT_TOKEN; }++ const net = t.network;++ const samU = net.addUser({ network_user_id: 9, username: 'sam', display_name: 'Sam' });++ const samJwt = net.sign({ id: 9, subject_id: samU.subject_id, username: 'sam', role: 'user' });++ const adminJwt = net.sign({ id: 1, subject_id: ids.newId('user'), username: 'boss', role: 'admin' });++ const call = (p, { method = 'GET', cookie, json } = {}) => t.get(p, { method, headers: json !== undefined ? { 'content-type': 'application/json' } : {}, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });++ await waitFor(() => t.app.locals.relay.status().inbound.state === 'ready', 'the app\'s gateway READY');++ assert.strictEqual(sent(gw, 2)[0].d.token, 'app-bot-token-not-real');++ const m = await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL', inbound: true } });++ assert.strictEqual(m.json().mapping.inbound, true);++ const th = await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'Ask Discord', body: 'what do you think?' } });++ await waitFor(() => t.db.prepare("SELECT 1 FROM relay_message_map WHERE local_type = 'thread'").get(), 'the thread on Discord');++ const anchor = t.db.prepare("SELECT external_message_id, external_channel_id FROM relay_message_map WHERE local_type = 'thread'").get();++ gw.dispatch('MESSAGE_CREATE', { id: nextId(), channel_id: anchor.external_channel_id, type: 19, content: 'Looks **great**', author: { id: KIM, username: 'kim', global_name: 'Kim' }, message_reference: { message_id: anchor.external_message_id } });++ const locked = nextId();++ const page = await call(`/s/general/t/${th.json().thread.slug}`);++ assert.strictEqual(page.status, 200);++ assert.ok(page.text.includes('badge-relay') && page.text.includes('Kim') && page.text.includes('<strong>great</strong>'), 'the reply shows with its Discord badge');++ t.db.prepare('UPDATE threads SET locked = 1').run();++ gw.dispatch('MESSAGE_CREATE', { id: locked, channel_id: anchor.external_channel_id, type: 19, content: 'too late', author: { id: KIM, username: 'kim' }, message_reference: { message_id: anchor.external_message_id } });++ assert.strictEqual((await call('/api/v1/relay/inbound', { cookie: samJwt })).status, 403);++ const inb = await call('/api/v1/relay/inbound', { cookie: adminJwt });++ assert.strictEqual(inb.status, 200);++ assert.deepStrictEqual(inb.json().failures.map((f) => [f.error, f.discord.message_id, f.mapping.space]), [['the thread is locked', locked, 'general']]);++ assert.strictEqual((await call(`/api/v1/relay/inbound/${inb.json().failures[0].id}/dismiss`, { method: 'POST', cookie: adminJwt })).status, 200);++ assert.strictEqual((await call('/api/v1/relay/inbound', { cookie: adminJwt })).json().failures.length, 0);++ assert.strictEqual((await call('/api/v1/relay/inbound?all=1', { cookie: adminJwt })).json().failures.length, 1);++ const st = await call('/api/v1/relay/status', { cookie: adminJwt });++ assert.deepStrictEqual([st.json().inbound.enabled, st.json().inbound.state, st.json().inbound.handled.applied, st.json().inbound_failures], [true, 'ready', 1, 0]);++ assert.ok(!(st.text + inb.text + (await call('/api/ready')).text).includes('app-bot-token-not-real'), 'the token never shows');++ await t.app.locals.relay.stop();++ assert.strictEqual(gw.last().closedWith, 1000);++ await t.close();++ });++++ await hook.close();++ done();++})();@@@ -1,41 +1,31 @@'use strict';/**-- * Discord relay (outbound): off unless DISCORD_RELAY_ENABLED; one delivery per (thread, mapping);-- * the webhook URL comes from the environment variable a mapping names; retries with backoff on-- * 5xx/429/network errors, immediate failure on other 4xx; loop prevention for Discord-origin-- * threads; members/staff spaces never relayed; the staff admin endpoint. A stub webhook server-- * stands in for Discord; the relay's clock is injected so backoff is tested without waiting.++ * Discord relay, outbound (relay/discord.js): off unless DISCORD_RELAY_ENABLED; one delivery per++ * dedupe key; the webhook URL comes from the environment variable a mapping names; creates are sent++ * with ?wait=true and the answer's message id goes into the external message map (unique both ways);++ * replies follow their thread (into the mapping's Discord thread when it names one); edits PATCH and++ * deletes DELETE through the map; retries with backoff on 5xx/429/network errors, then the dead letter++ * ('failed') that staff retry or drop; loop prevention for Discord-origin threads and posts; members,++ * staff and VIP members-only content never relayed (and deleted when gated later); the staff API;++ * the one-time rebuild of an older relay_deliveries table. A fake Discord stands in for the webhooks;++ * the relay's clock is injected so backoff is tested without waiting. The forum queues creates++ * directly here (no Events worker: see relay-events.test.js).*/const assert = require('assert');--const http = require('http');++const fs = require('fs');++const os = require('os');++const path = require('path');const { ids } = require('openvibe-contracts');const { boot, check, done } = require('./helpers/app');++const { startWebhooks } = require('./helpers/fake-discord');const { openDb } = require('../server/db');const forumStore = require('../server/forum/store');const { createDiscordRelay } = require('../server/relay/discord');++const { createForumService } = require('../server/forum/service');const { sqlTime } = require('../server/http/v1');--function stubWebhook() {-- const hits = [];-- const plan = []; // queued responses: { status, body, headers }-- const server = http.createServer((req, res) => {-- let raw = '';-- req.on('data', (c) => { raw += c; });-- req.on('end', () => {-- hits.push({ path: req.url, body: raw ? JSON.parse(raw) : null });-- const next = plan.shift() || { status: 204 };-- if (next.hang) return; // never answers-- res.writeHead(next.status, { 'Content-Type': 'application/json', ...(next.headers || {}) });-- res.end(next.body ? JSON.stringify(next.body) : '');-- });-- });-- return new Promise((resolve) => server.listen(0, '127.0.0.1', () => resolve({-- url: `http://127.0.0.1:${server.address().port}`, hits, plan, close: () => new Promise((r) => server.close(r)),-- })));--}--(async () => {-- const hook = await stubWebhook();++ const hook = await startWebhooks();const db = openDb(':memory:');const general = forumStore.getSpace(db, 'general');const feedback = forumStore.getSpace(db, 'feedback');@@@ -49,17 +39,23 @@ function stubWebhook() {relay.addMapping({ space_id: feedback.id, webhook_url_ref: 'DISCORD_WEBHOOK_FEEDBACK' });const newThread = (space, title, extra = {}) => forumStore.createThread(db, { space_id: space.id, title, author_subject: alex, body_markdown: '**Hello** there, [link](https://x.y)', ...extra }).thread;const deliveries = () => db.prepare('SELECT * FROM relay_deliveries ORDER BY id').all();++ const mapOf = (type, id) => db.prepare('SELECT * FROM relay_message_map WHERE local_type = ? AND local_id = ? ORDER BY id').all(type, id);const settle = () => new Promise((r) => setImmediate(r));++ const EMPTY = { delivered: 0, retry: 0, failed: 0, skipped: 0 };await check('off by default: nothing is queued or sent', async () => {const off = createDiscordRelay({ db, env, now });assert.strictEqual(off.enabled, false);-- assert.strictEqual(off.enqueueThread(newThread(general, 'Quiet thread'), general), 0);-- assert.deepStrictEqual(await off.drain(), { delivered: 0, retry: 0, failed: 0 });++ const th = newThread(general, 'Quiet thread');++ assert.strictEqual(off.enqueueThread(th, general), 0);++ assert.strictEqual(off.enqueuePost(forumStore.addPost(db, { thread_id: th.id, author_subject: alex, body_markdown: 'shh' }), th, general), 0);++ assert.deepStrictEqual(await off.drain(), EMPTY);assert.strictEqual(deliveries().length, 0);++ const st = off.status();++ assert.deepStrictEqual([st.enabled, st.events_worker.enabled, st.inbound.enabled], [false, false, false]);});-- await check('a new thread in a mapped space is posted once, with attribution, a link, and no mentions', async () => {++ await check('a new thread is posted once with ?wait=true, attribution, a link and no mentions; its message id is mapped', async () => {const th = newThread(general, 'Relay me <please>');assert.strictEqual(relay.enqueueThread(th, general), 1);assert.strictEqual(relay.enqueueThread(th, general), 0, 'dedupe key per (thread, mapping)');@@@ -67,7 +63,8 @@ function stubWebhook() {await relay.drain();assert.strictEqual(hook.hits.length, 1);const msg = hook.hits[0];-- assert.strictEqual(msg.path, '/api/webhooks/1/general');++ assert.strictEqual(msg.method, 'POST');++ assert.strictEqual(msg.path, '/api/webhooks/1/general?wait=true');assert.deepStrictEqual(msg.body.allowed_mentions, { parse: [] });assert.match(msg.body.content, /^New thread in \*\*s\/general\*\* by Alex @\u200beveryone: <https:\/\/openvibe\.community\/s\/general\/t\/relay-me-please>$/);assert.strictEqual(msg.body.embeds[0].title, 'Relay me <please>');@@@ -75,9 +72,30 @@ function stubWebhook() {assert.strictEqual(msg.body.embeds[0].description, 'Hello there, link');assert.strictEqual(msg.body.embeds[0].footer.text, 'OpenVibe.Community · s/general');const d = deliveries().find((x) => x.thread_id === th.id);-- assert.deepStrictEqual([d.status, d.attempts, d.last_status, d.dedupe_key], ['delivered', 1, 204, `thread:${th.id}:mapping:1`]);++ assert.deepStrictEqual([d.status, d.attempts, d.last_status, d.dedupe_key, d.action, d.source], ['delivered', 1, 200, `thread:${th.id}:mapping:1`, 'create', 'direct']);++ const [row] = mapOf('thread', th.id);++ assert.deepStrictEqual([row.platform, row.direction, row.mapping_id, row.thread_id, row.external_channel_id, row.external_thread_id, row.external_webhook_id],++ ['discord', 'out', 1, th.id, '2000000000000000001', null, '1']);++ assert.ok(hook.messages.has(row.external_message_id));++ assert.strictEqual(db.prepare('SELECT discord_channel_id FROM relay_mappings WHERE id = 1').get().discord_channel_id, '2000000000000000001', 'the mapping learned its channel');await relay.drain();assert.strictEqual(hook.hits.length, 1, 'a delivered thread is never sent again');++ // Even a create queued again under another key is not sent twice: the map already has it.++ db.prepare("INSERT INTO relay_deliveries (thread_id, mapping_id, dedupe_key, next_attempt_at) VALUES (?, 1, 'replay-of-the-same', ?)").run(th.id, sqlTime(clock));++ await relay.drain();++ assert.strictEqual(hook.hits.length, 1);++ assert.deepStrictEqual([deliveries().pop().status, deliveries().pop().last_error], ['delivered', 'already on Discord']);++ });++++ await check('the external message map is unique both ways', async () => {++ const th = newThread(general, 'Unique map');++ const put = (mapping, type, local, external) => db.prepare(`INSERT INTO relay_message_map (mapping_id, direction, local_type, local_id, thread_id, external_channel_id, external_message_id)++ VALUES (?, 'out', ?, ?, ?, '2000000000000000001', ?)`).run(mapping, type, local, th.id, external);++ put(1, 'thread', th.id, '1900000000000000001');++ assert.throws(() => put(2, 'post', 999, '1900000000000000001'), /UNIQUE/, 'one Discord message is one local object');++ assert.throws(() => put(1, 'thread', th.id, '1900000000000000002'), /UNIQUE/, 'one local object is one message per mapping');++ put(2, 'thread', th.id, '1900000000000000003'); // the same thread through another mapping is fine++ db.prepare('DELETE FROM relay_message_map WHERE thread_id = ?').run(th.id);});await check('retries with exponential backoff on 5xx, then delivers', async () => {@@@ -104,14 +122,20 @@ function stubWebhook() {assert.strictEqual(hook.hits.length, 3);});-- await check('429 honours retry_after; exhausted attempts end as failed; other 4xx fail at once', async () => {++ await check('429 honours retry_after and holds that webhook for the pass; used-up attempts end as failed; other 4xx fail at once', async () => {const th = newThread(general, 'Rate limited');++ const th2 = newThread(general, 'Behind it');hook.plan.push({ status: 429, body: { message: 'You are being rate limited.', retry_after: 300 } });relay.enqueueThread(th, general);++ relay.enqueueThread(th2, general);++ const before = hook.hits.length;await settle(); await relay.drain();++ assert.strictEqual(hook.hits.length, before + 1, 'the next delivery to the same webhook waits for the next pass');let d = deliveries().find((x) => x.thread_id === th.id);assert.strictEqual(d.status, 'pending');assert.strictEqual(Date.parse(`${d.next_attempt_at.replace(' ', 'T')}Z`) - clock, 300_000);++ await relay.drain();++ assert.strictEqual(deliveries().find((x) => x.thread_id === th2.id).status, 'delivered');clock += 300_000;hook.plan.push({ status: 503 }, { status: 503 });await relay.drain();@@@ -121,7 +145,7 @@ function stubWebhook() {assert.deepStrictEqual([d.status, d.attempts, d.last_status], ['failed', 3, 503]);const gone = newThread(general, 'Webhook deleted');-- hook.plan.push({ status: 404, body: { message: 'Unknown Webhook' } });++ hook.plan.push({ status: 404, body: { message: 'Unknown Webhook', code: 10015 } });relay.enqueueThread(gone, general);await settle(); await relay.drain();d = deliveries().find((x) => x.thread_id === gone.id);@@@ -148,11 +172,11 @@ function stubWebhook() {await relay.drain();d = deliveries().find((x) => x.thread_id === th.id);assert.strictEqual(d.status, 'delivered');-- const everything = JSON.stringify(db.prepare('SELECT * FROM relay_mappings').all()) + JSON.stringify(deliveries());++ const everything = JSON.stringify(db.prepare('SELECT * FROM relay_mappings').all()) + JSON.stringify(deliveries()) + JSON.stringify(db.prepare('SELECT * FROM relay_message_map').all());assert.ok(!everything.includes('/api/webhooks/'), 'no webhook URL stored anywhere');});-- await check('loop prevention: Discord-origin threads are never relayed; members/staff spaces never leave the site', async () => {++ await check('loop prevention: Discord-origin threads and posts are never relayed; members/staff spaces never leave the site', async () => {const before = hook.hits.length;const fromDiscord = newThread(general, 'Came from Discord', { origin: 'discord' });assert.strictEqual(relay.enqueueThread(fromDiscord, general), 0);@@@ -161,14 +185,23 @@ function stubWebhook() {await relay.drain();const d = deliveries().find((x) => x.thread_id === fromDiscord.id);assert.deepStrictEqual([d.status, d.last_error], ['failed', 'loop prevention: thread came from Discord']);++ // A reply that came from Discord, in a thread that is on Discord, stays here too.++ const th = newThread(general, 'Mixed replies');++ relay.enqueueThread(th, general);++ await settle(); await relay.drain();++ const inbound = forumStore.addPost(db, { thread_id: th.id, origin: 'discord', body_markdown: 'from discord', relay_author: 'Kim' });++ assert.strictEqual(relay.enqueuePost(inbound, th, general), 0);++ db.prepare('INSERT INTO relay_deliveries (thread_id, post_id, mapping_id, dedupe_key, next_attempt_at) VALUES (?, ?, 1, ?, ?)').run(th.id, inbound.id, `post:${inbound.id}:mapping:1`, sqlTime(clock));++ await relay.drain();++ assert.deepStrictEqual([deliveries().pop().status, deliveries().pop().last_error], ['failed', 'loop prevention: post came from Discord']);db.prepare("INSERT INTO spaces (slug, name, visibility) VALUES ('insiders', 'Insiders', 'members')").run();const ins = forumStore.getSpace(db, 'insiders');relay.addMapping({ space_id: ins.id, webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' });assert.strictEqual(relay.enqueueThread(newThread(ins, 'Members only'), ins), 0);-- assert.strictEqual(hook.hits.length, before);++ assert.strictEqual(hook.hits.length, before + 1, 'only the public thread went out');});-- await check('disabled mappings queue nothing; retry() puts a failed delivery back with a fresh budget', async () => {++ await check('disabled mappings queue nothing; retry() puts a failed delivery back with a fresh budget; drop() keeps it as dropped', async () => {relay.setMappingEnabled(1, false);assert.strictEqual(relay.enqueueThread(newThread(general, 'Mapping off'), general), 0);relay.setMappingEnabled(1, true);@@@ -177,64 +210,285 @@ function stubWebhook() {await settle(); await relay.drain();assert.strictEqual(deliveries().find((x) => x.id === failed.id).status, 'delivered');assert.strictEqual(relay.retry(failed.id), 0, 'delivered ones stay delivered');++ const dead = deliveries().find((x) => x.status === 'failed' && x.last_status === 503);++ assert.strictEqual(relay.drop(dead.id), 1);++ assert.strictEqual(deliveries().find((x) => x.id === dead.id).status, 'dropped');++ assert.strictEqual(relay.drop(dead.id), 0);++ assert.strictEqual(relay.status().deliveries.dropped, 1);++ });++++ // ── replies, edits and deletes through the forum service ─────────────────────────++ const forum = createForumService({ db, relay, limits: { threads: { cooldownSec: 0, perMinute: 100 }, posts: { cooldownSec: 0, perMinute: 100 }, threadsPerDay: 0 } });++ const sam = ids.newId('user');++ require('../server/pastes/store').upsertProjection(db, { subject_id: sam, username: 'sam', display_name: 'Sam' });++ const samV = { kind: 'user', subject: sam, staff: false };++ const alexV = { kind: 'user', subject: alex, staff: false };++ const modV = { kind: 'user', subject: ids.newId('user'), staff: true };++ const drainAll = async () => { await settle(); for (let i = 0; i < 5; i++) await relay.drain(); };++++ await check('replies follow their thread: one message per reply, linked to the post, mapped; a reply waits for its thread\'s message', async () => {++ hook.hits.length = 0;++ const { thread } = await forum.createThread(samV, 'general', { title: 'Replies go too', body: 'Opening words' });++ const { post } = await forum.reply(alexV, 'general', thread.slug, { body: 'First **reply**' });++ await drainAll();++ assert.deepStrictEqual(hook.hits.map((h) => h.method), ['POST', 'POST']);++ const r = hook.hits[1];++ assert.strictEqual(r.path, '/api/webhooks/1/general?wait=true');++ assert.match(r.body.content, /^Alex @\u200beveryone replied to \*\*Replies go too\*\* in \*\*s\/general\*\*: <https:\/\/openvibe\.community\/s\/general\/t\/replies-go-too#post-\d+>$/);++ assert.strictEqual(r.body.embeds[0].description, 'First reply');++ assert.deepStrictEqual(r.body.allowed_mentions, { parse: [] });++ const [row] = mapOf('post', post.id);++ assert.deepStrictEqual([row.direction, row.thread_id], ['out', thread.id]);++ // The thread's own message failing for now: the reply waits, without using an attempt.++ hook.hits.length = 0;++ hook.plan.push({ status: 500 });++ const t2 = await forum.createThread(samV, 'general', { title: 'Thread first', body: 'x' });++ const p2 = (await forum.reply(alexV, 'general', t2.thread.slug, { body: 'waits' })).post;++ await drainAll();++ const reply = deliveries().find((x) => x.post_id === p2.id);++ assert.deepStrictEqual([reply.status, reply.attempts, reply.last_error], ['pending', 0, "waiting for the thread's own message"]);++ assert.strictEqual(hook.hits.length, 1, 'only the thread was tried');++ clock += 60_000;++ await drainAll();++ assert.deepStrictEqual(hook.hits.map((h) => h.body.content.startsWith('New thread') ? 'thread' : 'reply'), ['thread', 'thread', 'reply'], 'the thread, then its reply');++ assert.strictEqual(deliveries().find((x) => x.post_id === p2.id).status, 'delivered');++ // A thread that was never relayed (made before the mapping): its replies are not either.++ const old = newThread(general, 'Before the mapping');++ assert.strictEqual(relay.enqueuePost(forumStore.addPost(db, { thread_id: old.id, author_subject: sam, body_markdown: 'late' }), old, general), 0);});-- await check('app wiring: a thread created through the API is relayed; the admin endpoint is staff-only and URL-free', async () => {++ await check('a mapping that names a Discord thread posts into it (thread_id) and edits and deletes there', async () => {++ hook.hits.length = 0;++ relay.updateMapping(2, { discord_thread_id: '3000000000000000002' });++ const { thread } = await forum.createThread(samV, 'feedback', { title: 'Into a Discord thread', body: 'hi' });++ const { post } = await forum.reply(samV, 'feedback', thread.slug, { body: 'reply in thread' });++ await drainAll();++ assert.deepStrictEqual(hook.hits.map((h) => h.path), ['/api/webhooks/2/feedback?wait=true&thread_id=3000000000000000002', '/api/webhooks/2/feedback?wait=true&thread_id=3000000000000000002'], JSON.stringify(hook.hits.map((h) => h.path)));++ const [row] = mapOf('post', post.id);++ assert.deepStrictEqual([row.external_channel_id, row.external_thread_id], ['3000000000000000002', '3000000000000000002']);++ await forum.editPost(samV, post.id, { body: 'reply in thread, edited' });++ await drainAll();++ await forum.deletePost(samV, post.id);++ await drainAll();++ assert.deepStrictEqual(hook.hits.slice(2).map((h) => `${h.method} ${h.path}`), [++ `PATCH /api/webhooks/2/feedback/messages/${row.external_message_id}?thread_id=3000000000000000002`,++ `DELETE /api/webhooks/2/feedback/messages/${row.external_message_id}?thread_id=3000000000000000002`,++ ]);++ assert.strictEqual(db.prepare('SELECT discord_channel_id FROM relay_mappings WHERE id = 2').get().discord_channel_id, '2000000000000000002', 'the channel learned before stays');++ relay.updateMapping(2, { discord_thread_id: null });++ });++++ await check('edits follow through the map: a reply\'s message and, for the opening post, the thread\'s message are PATCHed', async () => {++ hook.hits.length = 0;++ const { thread, post: opening } = await forum.createThread(samV, 'general', { title: 'Edit me', body: 'Version one' });++ const { post } = await forum.reply(samV, 'general', thread.slug, { body: 'Reply one' });++ await drainAll();++ const threadMsg = mapOf('thread', thread.id)[0].external_message_id;++ const postMsg = mapOf('post', post.id)[0].external_message_id;++ await forum.editPost(samV, post.id, { body: 'Reply **two**' });++ await forum.editPost(samV, opening.id, { body: 'Version two' });++ await forum.editPost(samV, opening.id, { body: 'Version two' }); // no change: no revision, nothing queued++ await drainAll();++ const edits = hook.hits.filter((h) => h.method === 'PATCH');++ assert.deepStrictEqual(edits.map((h) => h.pathname), [`/api/webhooks/1/general/messages/${postMsg}`, `/api/webhooks/1/general/messages/${threadMsg}`]);++ assert.strictEqual(edits[0].body.embeds[0].description, 'Reply two');++ assert.strictEqual(edits[1].body.embeds[0].description, 'Version two');++ assert.deepStrictEqual(edits[1].body.allowed_mentions, { parse: [] });++ assert.strictEqual(hook.messages.get(threadMsg).body.embeds[0].description, 'Version two');++ const keys = deliveries().filter((d) => d.action === 'edit').map((d) => d.dedupe_key);++ assert.ok(keys.includes(`edit:post:${post.id}:mapping:1:r2`) && keys.includes(`edit:thread:${thread.id}:mapping:1:r2`), keys.join());++ // Someone deleted the reply's message on Discord: the next edit finds nothing and says so.++ hook.messages.get(postMsg).deleted = true;++ await forum.editPost(samV, post.id, { body: 'Reply three' });++ await drainAll();++ const last = deliveries().pop();++ assert.deepStrictEqual([last.action, last.status, last.last_error], ['edit', 'skipped', 'the message is gone on Discord']);++ assert.ok(mapOf('post', post.id)[0].external_deleted_at, 'the map knows it is gone');++ const queued = deliveries().length;++ await forum.editPost(samV, post.id, { body: 'Reply four' });++ assert.strictEqual(deliveries().length, queued, 'nothing more is queued for a message that is gone');++ // An edit right before a delete is skipped: only the delete goes out.++ const { post: p2 } = await forum.reply(samV, 'general', thread.slug, { body: 'short-lived' });++ await drainAll();++ hook.hits.length = 0;++ await forum.editPost(samV, p2.id, { body: 'short-lived, edited' });++ await forum.deletePost(samV, p2.id);++ await drainAll();++ assert.deepStrictEqual(hook.hits.map((h) => h.method), ['DELETE']);++ assert.strictEqual(deliveries().find((d) => d.post_id === p2.id && d.action === 'edit').status, 'skipped');++ });++++ await check('deletes follow through the map: a reply, and a thread with every reply of it; what was not sent yet is skipped', async () => {++ hook.hits.length = 0;++ const { thread } = await forum.createThread(samV, 'general', { title: 'Delete me', body: 'soon gone' });++ const a = (await forum.reply(samV, 'general', thread.slug, { body: 'a' })).post;++ const b = (await forum.reply(alexV, 'general', thread.slug, { body: 'b' })).post;++ await drainAll();++ const msgs = { thread: mapOf('thread', thread.id)[0].external_message_id, a: mapOf('post', a.id)[0].external_message_id, b: mapOf('post', b.id)[0].external_message_id };++ await forum.deletePost(modV, a.id); // moderated away++ await drainAll();++ assert.deepStrictEqual(hook.hits.filter((h) => h.method === 'DELETE').map((h) => h.pathname), [`/api/webhooks/1/general/messages/${msgs.a}`]);++ const c = (await forum.reply(samV, 'general', thread.slug, { body: 'c, never sent' })).post; // queued, not sent yet++ await forum.deleteThread(samV, 'general', thread.slug);++ await drainAll();++ const deleted = hook.hits.filter((h) => h.method === 'DELETE').map((h) => h.pathname).sort();++ assert.deepStrictEqual(deleted, [msgs.thread, msgs.a, msgs.b].map((m) => `/api/webhooks/1/general/messages/${m}`).sort());++ assert.ok(!hook.hits.some((h) => h.method === 'POST' && /c, never sent/.test(JSON.stringify(h.body))), 'the reply queued before the delete never went out');++ assert.deepStrictEqual([deliveries().find((d) => d.post_id === c.id).status], ['skipped']);++ assert.ok(mapOf('thread', thread.id).every((r) => r.external_deleted_at));++ // A delete whose message is already gone on Discord counts as done.++ const t3 = (await forum.createThread(samV, 'general', { title: 'Gone already', body: 'x' })).thread;++ await drainAll();++ hook.messages.get(mapOf('thread', t3.id)[0].external_message_id).deleted = true;++ await forum.deleteThread(samV, 'general', t3.slug);++ await drainAll();++ const del = deliveries().pop();++ assert.deepStrictEqual([del.action, del.status, del.last_status, del.last_error], ['delete', 'delivered', 404, 'the message was already gone on Discord']);++ });++++ await check('members-only (VIP): gated threads and spaces never go out, and gating later deletes what went out', async () => {++ hook.hits.length = 0;++ const { thread } = await forum.createThread(samV, 'general', { title: 'Public for now', body: 'x' });++ await forum.reply(alexV, 'general', thread.slug, { body: 'a public reply' });++ await drainAll();++ const posted = hook.hits.length;++ assert.strictEqual(posted, 2);++ await forum.setThreadMembersOnly(samV, 'general', thread.slug, { members_only: sam });++ await drainAll();++ assert.strictEqual(hook.hits.filter((h) => h.method === 'DELETE').length, 2, 'the thread\'s and the reply\'s messages were deleted');++ // Replies in the gated thread (the creator may still post) never go out.++ const gatedThread = db.prepare('SELECT * FROM threads WHERE id = ?').get(thread.id);++ const reply = forumStore.addPost(db, { thread_id: thread.id, author_subject: sam, body_markdown: 'members only' });++ assert.strictEqual(relay.enqueuePost(reply, gatedThread, general), 0);++ // A members-only thread from the start is never queued, nor is anything in a VIP space.++ const gated = newThread(general, 'VIP from the start', { members_only_owner: sam });++ assert.strictEqual(relay.enqueueThread(gated, general), 0);++ db.prepare('UPDATE spaces SET members_only_owner = ? WHERE id = ?').run(sam, feedback.id);++ const vipSpace = forumStore.getSpace(db, 'feedback');++ assert.strictEqual(relay.enqueueThread(newThread(vipSpace, 'In a VIP space'), vipSpace), 0);++ db.prepare('UPDATE spaces SET members_only_owner = NULL WHERE id = ?').run(feedback.id);++ await drainAll();++ assert.ok(!hook.hits.slice(posted).some((h) => h.method !== 'DELETE'), 'nothing but the two deletes');++ });++++ await check('an older database: relay_deliveries is rebuilt once, rows kept, the new columns in place', async () => {++ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-relay-migrate-'));++ const file = path.join(dir, 'community.db');++ const old = openDb(file);++ const th = forumStore.createThread(old, { space_id: 1, title: 'Old row', body_markdown: 'x' }).thread;++ old.exec(`DROP TABLE relay_deliveries;++ CREATE TABLE relay_deliveries (id INTEGER PRIMARY KEY AUTOINCREMENT, thread_id INTEGER NOT NULL REFERENCES threads(id) ON DELETE CASCADE,++ mapping_id INTEGER NOT NULL REFERENCES relay_mappings(id) ON DELETE CASCADE, dedupe_key TEXT UNIQUE NOT NULL,++ status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'delivered', 'failed')), attempts INTEGER NOT NULL DEFAULT 0,++ next_attempt_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, last_status INTEGER, last_error TEXT, delivered_at DATETIME,++ created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, UNIQUE (thread_id, mapping_id));++ CREATE INDEX idx_relay_deliveries_due ON relay_deliveries(status, next_attempt_at);`);++ old.prepare("INSERT INTO relay_mappings (space_id, webhook_url_ref) VALUES (1, 'DISCORD_WEBHOOK_GENERAL')").run();++ old.prepare("INSERT INTO relay_deliveries (thread_id, mapping_id, dedupe_key, status, attempts, last_status) VALUES (?, 1, ?, 'delivered', 1, 204)").run(th.id, `thread:${th.id}:mapping:1`);++ old.close();++ const reopened = openDb(file);++ const cols = reopened.prepare('PRAGMA table_info(relay_deliveries)').all().map((c) => c.name);++ for (const c of ['post_id', 'action', 'source', 'event_id']) assert.ok(cols.includes(c), c);++ const row = reopened.prepare('SELECT * FROM relay_deliveries').get();++ assert.deepStrictEqual([row.thread_id, row.dedupe_key, row.status, row.action, row.source, row.last_status], [th.id, `thread:${th.id}:mapping:1`, 'delivered', 'create', 'direct', 204]);++ reopened.prepare("INSERT INTO relay_deliveries (thread_id, mapping_id, dedupe_key, status) VALUES (?, 1, 'x', 'dropped')").run(th.id);++ assert.ok(reopened.prepare("SELECT 1 FROM sqlite_master WHERE name = 'idx_relay_deliveries_due'").get());++ const mcols = reopened.prepare('PRAGMA table_info(relay_mappings)').all().map((c) => c.name);++ assert.ok(['discord_channel_id', 'discord_thread_id', 'inbound'].every((c) => mcols.includes(c)));++ reopened.close();++ const again = openDb(file); // idempotent++ assert.strictEqual(again.prepare('SELECT COUNT(*) AS n FROM relay_deliveries').get().n, 2);++ again.close();++ fs.rmSync(dir, { recursive: true, force: true });++ });++++ await check('app wiring: threads, replies, edits and deletes through the API are relayed; the admin endpoints are staff-only and URL-free', async () => {const t = await boot({authority: 'community',-- appOpts: { relayOptions: { enabled: true, env: { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/9/app` } }, forumLimits: { threads: { cooldownSec: 0 } } },++ appOpts: { relayOptions: { enabled: true, env: { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/9/app` } }, forumLimits: { threads: { cooldownSec: 0 }, posts: { cooldownSec: 0 } } },});const net = t.network;-- const sam = net.addUser({ network_user_id: 9, username: 'sam', display_name: 'Sam' });-- const samJwt = net.sign({ id: 9, subject_id: sam.subject_id, username: 'sam', role: 'user' });++ const samU = net.addUser({ network_user_id: 9, username: 'sam', display_name: 'Sam' });++ const samJwt = net.sign({ id: 9, subject_id: samU.subject_id, username: 'sam', role: 'user' });const adminJwt = net.sign({ id: 1, subject_id: ids.newId('user'), username: 'boss', role: 'admin' });-- const call = (path, { method = 'GET', cookie, token, json, headers = {} } = {}) => {++ const call = (p, { method = 'GET', cookie, token, json, headers = {} } = {}) => {const h = { ...headers };if (token) h.authorization = `Bearer ${token}`;if (json !== undefined) h['content-type'] = 'application/json';-- return t.get(path, { method, headers: h, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });++ return t.get(p, { method, headers: h, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });};++ const r = t.app.locals.relay;++ assert.strictEqual(r.status().creates_from, 'forum', 'no EVENTS_URL: the forum queues creates');++ assert.deepStrictEqual([r.status().events_worker.enabled, r.status().inbound.enabled], [false, false]);assert.strictEqual((await call('/api/v1/relay/mappings', { method: 'POST', cookie: samJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' } })).status, 403);const bad = await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'https://discord.com/api/webhooks/secret' } });assert.strictEqual(bad.status, 400);assert.strictEqual(bad.json().code, 'relay.invalid_ref');++ assert.strictEqual((await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL', discord_channel_id: 'nope' } })).json().code, 'relay.invalid_discord_id');const m = await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' } });assert.strictEqual(m.status, 201, m.text);-- assert.deepStrictEqual([m.json().mapping.space, m.json().mapping.webhook_configured, m.json().mapping.enabled], ['general', true, true]);++ assert.deepStrictEqual([m.json().mapping.space, m.json().mapping.webhook_configured, m.json().mapping.enabled, m.json().mapping.inbound, m.json().mapping.discord_channel_id], ['general', true, true, false, null]);++ const put = await call(`/api/v1/relay/mappings/${m.json().mapping.id}`, { method: 'PUT', cookie: adminJwt, json: { inbound: true, discord_channel_id: '2000000000000000009' } });++ assert.deepStrictEqual([put.json().mapping.inbound, put.json().mapping.discord_channel_id, put.json().mapping.enabled], [true, '2000000000000000009', true]);const before = hook.hits.length;++ const waitHits = async (n) => { for (let i = 0; i < 100 && hook.hits.length < n; i++) await new Promise((res) => setTimeout(res, 10)); };const th = await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'Through the app', body: 'hi' } });assert.strictEqual(th.status, 201);-- for (let i = 0; i < 50 && hook.hits.length === before; i++) await new Promise((r) => setTimeout(r, 10));-- assert.strictEqual(hook.hits.length, before + 1);-- assert.strictEqual(hook.hits[hook.hits.length - 1].path, '/api/webhooks/9/app');++ await waitHits(before + 1);++ assert.strictEqual(hook.hits[hook.hits.length - 1].path, '/api/webhooks/9/app?wait=true');++ const reply = await call(`/api/v1/spaces/general/threads/${th.json().thread.slug}/posts`, { method: 'POST', cookie: samJwt, json: { body: 'a reply' } });++ assert.strictEqual(reply.status, 201, reply.text);++ await waitHits(before + 2);++ assert.match(hook.hits[hook.hits.length - 1].body.content, /replied to \*\*Through the app\*\*/);++ assert.strictEqual((await call(`/api/v1/posts/${reply.json().post.id}`, { method: 'PUT', cookie: samJwt, json: { body: 'a reply, edited' } })).status, 200);++ await waitHits(before + 3);++ assert.strictEqual(hook.hits[hook.hits.length - 1].method, 'PATCH');++ assert.strictEqual((await call(`/api/v1/posts/${reply.json().post.id}`, { method: 'DELETE', cookie: samJwt })).status, 200);++ await waitHits(before + 4);++ assert.strictEqual(hook.hits[hook.hits.length - 1].method, 'DELETE');hook.plan.push({ status: 400, body: { message: 'Invalid Form Body' } });await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'This one fails', body: 'hi' } });-- for (let i = 0; i < 50 && hook.hits.length === before + 1; i++) await new Promise((r) => setTimeout(r, 10));-- await t.app.locals.relay.drain();++ await waitHits(before + 5);++ await r.drain();assert.strictEqual((await call('/api/v1/relay/deliveries')).status, 401);assert.strictEqual((await call('/api/v1/relay/deliveries', { cookie: samJwt })).status, 403);++ assert.strictEqual((await call('/api/v1/relay/status', { cookie: samJwt })).status, 403);const failed = await call('/api/v1/relay/deliveries?status=failed', { cookie: adminJwt });assert.strictEqual(failed.status, 200, failed.text);assert.strictEqual(failed.json().enabled, true);assert.strictEqual(failed.json().deliveries.length, 1);-- assert.match(failed.json().deliveries[0].last_error, /400: Invalid Form Body/);-- assert.strictEqual(failed.json().deliveries[0].thread.url, '/s/general/t/this-one-fails');++ const dl = failed.json().deliveries[0];++ assert.match(dl.last_error, /400: Invalid Form Body/);++ assert.deepStrictEqual([dl.thread.url, dl.action, dl.source], ['/s/general/t/this-one-fails', 'create', 'direct']);++ const edits = await call('/api/v1/relay/deliveries?action=edit', { cookie: adminJwt });++ assert.deepStrictEqual(edits.json().deliveries.map((x) => [x.action, x.status, x.post.id]), [['edit', 'delivered', reply.json().post.id]]);++ const st = await call('/api/v1/relay/status', { cookie: adminJwt });++ assert.strictEqual(st.status, 200);++ assert.deepStrictEqual([st.json().dead_letters, st.json().deliveries.failed, st.json().creates_from], [1, 1, 'forum']);++ assert.strictEqual((await call(`/api/v1/relay/deliveries/${dl.id}/drop`, { method: 'POST', cookie: adminJwt })).status, 200);++ assert.strictEqual((await call(`/api/v1/relay/deliveries/${dl.id}/drop`, { method: 'POST', cookie: adminJwt })).status, 404, 'dropped already');++ assert.strictEqual((await call(`/api/v1/relay/deliveries/${dl.id}/retry`, { method: 'POST', cookie: adminJwt })).status, 200, 'a dropped one can be retried');++ await waitHits(before + 6);++ await r.drain();++ assert.strictEqual((await call('/api/v1/relay/deliveries?status=failed', { cookie: adminJwt })).json().deliveries.length, 0);++ const ready = await call('/api/ready');++ assert.deepStrictEqual([ready.json().discord_relay.enabled, ready.json().discord_relay.events_worker.enabled, ready.json().discord_relay.deliveries.delivered > 0], [true, false, true]);const svcAdmin = await call('/api/v1/relay/deliveries', { token: net.signService({ cap: ['community.comment.moderate'] }) });assert.strictEqual(svcAdmin.status, 200);-- assert.ok(!(failed.text + svcAdmin.text + (await call('/api/v1/relay/mappings', { cookie: adminJwt })).text).includes('/api/webhooks/'), 'responses never carry a webhook URL');++ assert.ok(!(failed.text + svcAdmin.text + st.text + (await call('/api/v1/relay/mappings', { cookie: adminJwt })).text).includes('/api/webhooks/'), 'responses never carry a webhook URL');await t.close();});await check('staff can map only allow-listed webhook variables, never the URL in some other env var', async () => {const secretHits = [];const internal = { url: 'http://127.0.0.1:9/internal', hits: secretHits };-- const env = { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/1/ok`, OV_MEDIA_INTERNAL_URL: internal.url, NETWORK_INTERNAL_URL: internal.url, DISCORD_WEBHOOKS_X: internal.url };++ const env2 = { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/1/ok`, OV_MEDIA_INTERNAL_URL: internal.url, NETWORK_INTERNAL_URL: internal.url, DISCORD_WEBHOOKS_X: internal.url };const fetchSpy = (url, o) => { if (String(url).startsWith(internal.url)) secretHits.push(url); return fetch(url, o); };-- const t = await boot({ authority: 'community', appOpts: { relayOptions: { enabled: true, env, fetchImpl: fetchSpy }, forumLimits: { threads: { cooldownSec: 0 } } } });++ const t = await boot({ authority: 'community', appOpts: { relayOptions: { enabled: true, env: env2, fetchImpl: fetchSpy }, forumLimits: { threads: { cooldownSec: 0 } } } });const net = t.network;const adminJwt = net.sign({ id: 1, subject_id: ids.newId('user'), username: 'boss', role: 'admin' });const samJwt = net.sign({ id: 9, subject_id: net.addUser({ network_user_id: 9, username: 'sam' }).subject_id, username: 'sam', role: 'user' });-- const call = (path, { method = 'GET', cookie, token, json } = {}) => t.get(path, { method, headers: { ...(token ? { authorization: `Bearer ${token}` } : {}), ...(json !== undefined ? { 'content-type': 'application/json' } : {}) }, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });++ const call = (p, { method = 'GET', cookie, token, json } = {}) => t.get(p, { method, headers: { ...(token ? { authorization: `Bearer ${token}` } : {}), ...(json !== undefined ? { 'content-type': 'application/json' } : {}) }, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });const map = (ref, who = { cookie: adminJwt }) => call('/api/v1/relay/mappings', { method: 'POST', json: { space: 'general', webhook_url_ref: ref }, ...who });for (const ref of ['OV_MEDIA_INTERNAL_URL', 'NETWORK_INTERNAL_URL', 'DISCORD_WEBHOOKS_X', 'PATH', 'DISCORD_WEBHOOK_']) {const r = await map(ref);@@@ -247,9 +501,9 @@ function stubWebhook() {assert.throws(() => t.app.locals.relay.addMapping({ space_id: 1, webhook_url_ref: 'OV_MEDIA_INTERNAL_URL' }), /not an allowed webhook variable/);// A mapping that predates the allow-list (straight into the table) is never sent to, and// does not reveal whether that variable is set.-- const general = t.db.prepare("SELECT id FROM spaces WHERE slug = 'general'").get();-- t.db.prepare("INSERT INTO relay_mappings (space_id, direction, webhook_url_ref, enabled) VALUES (?, 'out', 'OV_MEDIA_INTERNAL_URL', 1)").run(general.id);-- const listed = (await call('/api/v1/relay/mappings', { cookie: adminJwt })).json().mappings.find((m) => m.webhook_url_ref === 'OV_MEDIA_INTERNAL_URL');++ const gen = t.db.prepare("SELECT id FROM spaces WHERE slug = 'general'").get();++ t.db.prepare("INSERT INTO relay_mappings (space_id, direction, webhook_url_ref, enabled) VALUES (?, 'out', 'OV_MEDIA_INTERNAL_URL', 1)").run(gen.id);++ const listed = (await call('/api/v1/relay/mappings', { cookie: adminJwt })).json().mappings.find((x) => x.webhook_url_ref === 'OV_MEDIA_INTERNAL_URL');assert.strictEqual(listed.webhook_configured, false);assert.strictEqual((await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'Would go inside', body: 'hi' } })).status, 201);await t.app.locals.relay.drain();@@@ -260,11 +514,25 @@ function stubWebhook() {assert.strictEqual((await map('DISCORD_WEBHOOK_GENERAL')).status, 201, 'the conventional names still work');await t.close();// DISCORD_RELAY_WEBHOOK_VARS narrows it to exact names.-- const { createDiscordRelay } = require('../server/relay/discord');-- const strict = createDiscordRelay({ db: t.db, env, webhookVars: ['DISCORD_WEBHOOK_GENERAL'] });++ const strict = createDiscordRelay({ db: t.db, env: env2, webhookVars: ['DISCORD_WEBHOOK_GENERAL'] });assert.strictEqual(strict.refAllowed('DISCORD_WEBHOOK_GENERAL'), true);assert.strictEqual(strict.refAllowed('DISCORD_WEBHOOK_OTHER'), false);-- assert.strictEqual(createDiscordRelay({ db: t.db, env }).refAllowed('DISCORD_WEBHOOK_OTHER'), true);++ assert.strictEqual(createDiscordRelay({ db: t.db, env: env2 }).refAllowed('DISCORD_WEBHOOK_OTHER'), true);++ });++++ await check('disabled by default in the app: no relay, no Events worker, no gateway, whatever else is set', async () => {++ process.env.DISCORD_BOT_TOKEN = 'not-a-real-token';++ process.env.DISCORD_RELAY_INBOUND = 'on';++ process.env.EVENTS_URL = 'http://127.0.0.1:9';++ try {++ const t = await boot({ authority: 'community' });++ const st = t.app.locals.relay.status();++ assert.deepStrictEqual([st.enabled, st.events_worker.enabled, st.inbound.enabled, t.app.locals.relayInbound], [false, false, false, null]);++ assert.match(st.events_worker.reason, /relay is off/);++ await t.close();++ } finally {++ delete process.env.DISCORD_BOT_TOKEN; delete process.env.DISCORD_RELAY_INBOUND; delete process.env.EVENTS_URL;++ }});await hook.close();@@@ -1,7 +1,7 @@'use strict';/*** Graceful stop (roadmap WS-P lifecycle; server/graceful.js), on the real process with the events-- * outbox, the Discord relay, the Pulse subscriptions and the profile and search scans all on (pointed++ * outbox, the Discord relay (with its Events worker and gateway), the Pulse subscriptions and the profile and search scans all on (pointed* at nothing): SIGTERM while a request is in flight and a keep-alive connection sits idle. New* connections are refused, the request is answered (Connection: close), every background worker is* stopped, community.db is closed, and the process exits 0 within the manifest's 5 s; the idle@@@ -27,6 +27,7 @@ const freePort = () => new Promise((resolve) => { const s = net.createServer();...process.env, PORT: String(port), HOST: '127.0.0.1', NODE_ENV: 'test', COMMUNITY_DB_PATH: path.join(dir, 'community.db'),EVENTS_URL: 'http://127.0.0.1:9', OV_NETWORK_INTERNAL_URL: 'http://127.0.0.1:9', OV_NETWORK_URL: 'http://127.0.0.1:9',OV_OAUTH_CLIENT_SECRET: 'test-secret-not-real', COMMUNITY_EVENTS_SECRET: 'e'.repeat(40), DISCORD_RELAY_ENABLED: '1',++ DISCORD_RELAY_INBOUND: 'on', DISCORD_BOT_TOKEN: 'not-a-real-bot-token', DISCORD_GATEWAY_URL: 'ws://127.0.0.1:9/?v=10&encoding=json',},stdio: ['ignore', 'pipe', 'pipe'],});@@@ -43,6 +44,7 @@ const freePort = () => new Promise((resolve) => { const s = net.createServer();}assert.ok(up, `the server did not start:\n${out}`);assert.match(out, /\[Events\] community →/, 'the outbox is on');++ assert.match(out, /\[Relay\] Discord relay on: creates queued by the Events worker; events worker on; inbound on/, 'the relay, its Events worker and the gateway are on');// An idle keep-alive connection (the server keeps them 65 s).const agent = new http.Agent({ keepAlive: true });@@@ -73,7 +75,7 @@ const freePort = () => new Promise((resolve) => { const s = net.createServer();const ms = Date.now() - t0;assert.strictEqual(code, 0, `exit 0 (got ${code} ${signal})\n${out.slice(-2000)}`);assert.ok(ms < 5000, `within the manifest's 5 s (${ms} ms)`);-- assert.match(out, /\[Community\] stopped: subscriptions stopped, profile scan stopped, search scans stopped, relay stopped, outbox stopped \(\d+ pending\)/, out.slice(-2000));++ assert.match(out, /\[Community\] stopped: subscriptions stopped, profile scan stopped, search scans stopped, relay stopped \(events worker stopped, gateway stopped\), outbox stopped \(\d+ pending\)/, out.slice(-2000));assert.match(out, /\[Community\] stopped in \d+ ms/);agent.destroy();console.log(`shutdown: SIGTERM → exit 0 in ${ms} ms, the request in flight answered, every worker stopped`);
@@ -394,9 +394,9 @@ by the API, the `members_only` box on the new-thread form, or the button on the
@@ -394,9 +394,9 @@ by the API, the `members_only` box on the new-thread form, or the button on the
- **Listings** keep a gated thread's title with `members_only` (never a body); gated spaces are listed
- **Listings** keep a gated thread's title with `members_only` (never a body); gated spaces are listed
with their join link.
with their join link.
- **Never public**: gated spaces and threads stay out of Pulse (not recorded, removed when gated later,
- **Never public**: gated spaces and threads stay out of Pulse (not recorded, removed when gated later,
- and filtered at read time), the Discord relay (not queued; pending deliveries dropped; re-checked at
- send), the sitemap, the RSS feeds (a gated space's feed is 404) and JSON-LD; a member's view of a
- gated thread is `noindex` as well.
+ and filtered at read time), the Discord relay (not queued; when gated later, what was waiting is
+ skipped and what was sent is deleted on Discord; re-checked at send), the sitemap, the RSS feeds
+ (a gated space's feed is 404) and JSON-LD; a member's view of a gated thread is `noindex` as well.
- **Convergence**: answers are cached per viewer and resource (`server/vip/`, VIP's `createVipCache`):
- **Convergence**: answers are cached per viewer and resource (`server/vip/`, VIP's `createVipCache`):
a "yes" at most `VIP_CACHE_TTL_MS` (30 s), a "no" 10 s, a failure 2 s. Community has no Events inbox,
a "yes" at most `VIP_CACHE_TTL_MS` (30 s), a "no" 10 s, a failure 2 s. Community has no Events inbox,
so **once VIP stops granting** (it applied Billing's `billing.entitlement.changed` and emitted
so **once VIP stops granting** (it applied Billing's `billing.entitlement.changed` and emitted
@@ -414,27 +414,37 @@ a score.
@@ -414,27 +414,37 @@ a score.
## Discord relay
## Discord relay
-Outbound only, and off unless `DISCORD_RELAY_ENABLED=true`. When a thread is created in a
-public space that has an enabled mapping, Community posts "New thread in s/<space> by <name>"
-with the title, an excerpt and a link to the thread to that mapping's Discord webhook.
-
-- A mapping (`relay_mappings`) names the **environment variable** that holds the webhook URL
- (`webhook_url_ref`, e.g. `DISCORD_WEBHOOK_FEEDBACK`); the URL itself never enters the
- database or any API response. Put the variable in `/etc/openvibe/community.env`. Only
- allow-listed names can be mapped or sent to: `DISCORD_WEBHOOK_*`, or exactly the names in
- `DISCORD_RELAY_WEBHOOK_VARS` when that is set — staff cannot point the relay at the URL in any
- other variable (an internal service's base URL).
-- One delivery per (thread, mapping) — the dedupe key in `relay_deliveries`. Network errors,
- timeouts, 5xx, 429 (its `retry_after` honoured) and an unset variable are retried with
- exponential backoff (`DISCORD_RELAY_BACKOFF_MS` · 2^(attempt−1), at most an hour) up to
- `DISCORD_RELAY_MAX_ATTEMPTS`; other 4xx fail at once. Mentions are disabled.
-- Loop prevention: a thread whose origin is `discord` is never relayed out (checked when
- queueing and when sending). Members/staff spaces are never relayed.
-- Staff (admin/global_mod browsers, or services with `community.comment.moderate`):
- `GET /api/v1/relay/deliveries?status=failed` shows what failed and why,
- `POST /api/v1/relay/deliveries/:id/retry` queues one again,
- `GET|POST /api/v1/relay/mappings` and `PUT /api/v1/relay/mappings/:id { enabled }` manage
- mappings (`POST { space: 'feedback', webhook_url_ref: 'DISCORD_WEBHOOK_FEEDBACK' }`).
+Off unless `DISCORD_RELAY_ENABLED=true`, and inert without the owner's webhooks, mappings and bot
+token. Setup, owner steps, limits and operations: [docs/discord-relay.md](docs/discord-relay.md).
+
+- **Out**: a new thread in a mapped public space is announced through the mapping's Discord
+ webhook ("New thread in s/<space> by <name>", title, excerpt, link); its replies follow into the
+ same channel (or the Discord thread the mapping names); edits PATCH and deletes (or gating a
+ thread members-only) DELETE the Discord messages through the **external message map**
+ (`relay_message_map`, unique both ways; WS-J task 5). Creates are queued by the relay's
+ **Events worker** from Community's own `community.thread.*` / `community.post.*` read back from
+ OpenVibe.Events' pull API with a stored cursor (`relay_cursors`; WS-J task 6), or by the forum
+ itself when the worker is off (no `EVENTS_URL` / client secret, or `DISCORD_RELAY_EVENTS=off`).
+ Dedupe keys in `relay_deliveries` mean nothing is queued or posted twice.
+- **In** (`DISCORD_RELAY_INBOUND=on`, `DISCORD_BOT_TOKEN`, `inbound: true` on the mapping): a
+ Discord gateway client (Node 22's WebSocket; intents GUILD_MESSAGES and MESSAGE_CONTENT) turns
+ replies to relayed messages in mapped channels into posts with origin `discord`, attributed to
+ the Discord name only (never to anyone on the site), size- and rate-limited, mentions defused;
+ edits and deletes on Discord follow through the map.
+- **Loop prevention**: origin `discord` never goes out (checked when queueing and sending);
+ webhook, bot and system messages never come in. Members/staff spaces and VIP members-only
+ content never leave the site.
+- **Secrets**: a mapping names the environment variable holding the webhook URL
+ (`webhook_url_ref`, allow-listed `DISCORD_WEBHOOK_*` or exactly `DISCORD_RELAY_WEBHOOK_VARS`);
+ the URL and the bot token never enter the database or any API response.
+- **Retries and the dead letter**: network errors, timeouts, 5xx, 429 and an unset variable are
+ retried with exponential backoff up to `DISCORD_RELAY_MAX_ATTEMPTS`, then the delivery is
+ `failed` (the dead letter); other 4xx fail at once. Mentions are disabled on every message.
+- **Staff** (admin/global_mod browsers, or services with `community.comment.moderate`):
+ `GET /api/v1/relay/status` (queue, Events worker cursor/lag/errors/gaps, gateway state),
+ `GET /api/v1/relay/deliveries?status=failed`, `POST /api/v1/relay/deliveries/:id/retry|drop`,
+ `GET /api/v1/relay/inbound` and `POST /api/v1/relay/inbound/:id/dismiss` (inbound failures),
+ `GET|POST /api/v1/relay/mappings`, `PUT /api/v1/relay/mappings/:id`.
## Pulse
## Pulse
@@ -517,9 +527,13 @@ Copy `.env.example` to `.env` (production: `/etc/openvibe/community.env`, mode 0
@@ -517,9 +527,13 @@ Copy `.env.example` to `.env` (production: `/etc/openvibe/community.env`, mode 0
| `PASTES_AUTHORITY` | `live` | `live` = proxy to Live; `community` = this site's database is the authority |
| `PASTES_AUTHORITY` | `live` | `live` = proxy to Live; `community` = this site's database is the authority |
| `COMMUNITY_DB_PATH` | `./data/community.db` | SQLite file (the systemd unit sets `/var/lib/openvibe-community/community.db`) |
| `COMMUNITY_DB_PATH` | `./data/community.db` | SQLite file (the systemd unit sets `/var/lib/openvibe-community/community.db`) |
| `API_CORS_ORIGINS` | Live, Media, Network, Tools, Games origins | Browser origins that may call `/api/v1/comments` and `/api/v1/pulse` with a Bearer JWT |
| `API_CORS_ORIGINS` | Live, Media, Network, Tools, Games origins | Browser origins that may call `/api/v1/comments` and `/api/v1/pulse` with a Bearer JWT |
-| `DISCORD_RELAY_ENABLED` | off | `true` turns the outbound Discord relay on |
-| `DISCORD_RELAY_POLL_MS` / `DISCORD_RELAY_BACKOFF_MS` / `DISCORD_RELAY_MAX_ATTEMPTS` | `30000` / `30000` / `6` | Relay worker cadence, first retry delay, attempts before `failed` |
+| `DISCORD_RELAY_ENABLED` | off | `true` turns the Discord relay on ([docs/discord-relay.md](docs/discord-relay.md)) |
+| `DISCORD_RELAY_POLL_MS` / `DISCORD_RELAY_BACKOFF_MS` / `DISCORD_RELAY_MAX_ATTEMPTS` | `30000` / `30000` / `6` | Relay sender cadence, first retry delay, attempts before `failed` (the dead letter) |
| *(any name)* e.g. `DISCORD_WEBHOOK_FEEDBACK` | — | A Discord webhook URL, named by a relay mapping's `webhook_url_ref` |
| *(any name)* e.g. `DISCORD_WEBHOOK_FEEDBACK` | — | A Discord webhook URL, named by a relay mapping's `webhook_url_ref` |
+| `DISCORD_RELAY_EVENTS` / `DISCORD_RELAY_EVENTS_POLL_MS` | on / `5000` | The relay's Events worker (needs `EVENTS_URL` and the client secret); `off` leaves creates to the forum |
+| `DISCORD_RELAY_INBOUND` / `DISCORD_BOT_TOKEN` | off / — | Replies from Discord through the gateway (a bot with the MESSAGE CONTENT intent) |
+| `DISCORD_GATEWAY_URL` | `wss://gateway.discord.gg/?v=10&encoding=json` | The Discord gateway |
+| `DISCORD_RELAY_INBOUND_PER_MINUTE` / `DISCORD_RELAY_INBOUND_MAX_CHARS` | `6` / `4000` | Inbound limits per Discord author per mapping, and per message |
| `VIEW_HASH_SECRET` | derived from the client secret | Salt for hashed visitor ids in view counts |
| `VIEW_HASH_SECRET` | derived from the client secret | Salt for hashed visitor ids in view counts |
| `COOKIE_SECURE` | `true` in production | Set `false` for plain-http local dev |
| `COOKIE_SECURE` | `true` in production | Set `false` for plain-http local dev |
| `OV_VIP_INTERNAL_URL` | `http://127.0.0.1:4620` | OpenVibe.VIP's API (members-only spaces and threads) |
| `OV_VIP_INTERNAL_URL` | `http://127.0.0.1:4620` | OpenVibe.VIP's API (members-only spaces and threads) |
@@ -588,7 +602,8 @@ server/
@@ -588,7 +602,8 @@ server/
forum/ spaces/threads/posts: store, service, api (/api/v1/spaces, /posts), routes (pages)
forum/ spaces/threads/posts: store, service, api (/api/v1/spaces, /posts), routes (pages)
vip/ OpenVibe.VIP gate for members-only spaces/threads (index.js) + the vendored client
vip/ OpenVibe.VIP gate for members-only spaces/threads (index.js) + the vendored client
pulse/ Pulse read model: store, service (hooks + ingest), api (/api/v1/pulse)
pulse/ Pulse read model: store, service (hooks + ingest), api (/api/v1/pulse)
- relay/ Discord relay: discord.js (queue, worker, backoff), api (/api/v1/relay)
+ relay/ Discord relay: discord.js (queue, sender, backoff, message map), events-worker.js (creates
+ from Events), discord-gateway.js + inbound.js (replies from Discord), api (/api/v1/relay)
http/v1.js /api/v1 helpers: problem errors, capability guards, cursors, CORS
http/v1.js /api/v1 helpers: problem errors, capability guards, cursors, CORS
identity/capabilities.js capability checks incl. the proposed ids; discussion staff/moderators
identity/capabilities.js capability checks incl. the proposed ids; discussion staff/moderators
identity/authors.js author display from subject_projection; the AI label
identity/authors.js author display from subject_projection; the AI label
@@ -609,12 +624,13 @@ scripts/import-pastes.js Media paste bundle importer
@@ -609,12 +624,13 @@ scripts/import-pastes.js Media paste bundle importer
scripts/import-live-comments.js Live's VOD/clip comments → Community threads (dry run by default)
scripts/import-live-comments.js Live's VOD/clip comments → Community threads (dry run by default)
test/ run.js + *.test.js (mock Live, Network and Media with a real RS256 key)
test/ run.js + *.test.js (mock Live, Network and Media with a real RS256 key)
docs/capabilities-proposal/ Wave 5 capability manifests (released in openvibe-contracts v0.7.0)
docs/capabilities-proposal/ Wave 5 capability manifests (released in openvibe-contracts v0.7.0)
+docs/discord-relay.md the Discord relay: how it works, owner steps, staff API, limits
```
```
## What is next
## What is next
-Spaces per streamer, game and project (with membership), inbound Discord relay (threads from
-Discord arrive with origin `discord` and are never relayed back), visibility changes for
+Spaces per streamer, game and project (with membership), the Discord relay's production round
+trip (owner steps in [docs/discord-relay.md](docs/discord-relay.md)), visibility changes for
comment threads arriving through Events, moving paste comments onto the typed comment
comment threads arriving through Events, moving paste comments onto the typed comment
threads, and submissions (clips, art, ideas, reports for community review).
threads, and submissions (clips, art, ideas, reports for community review).
new file mode 100644
new file mode 100644
@@ -0,0 +1,164 @@
@@ -0,0 +1,164 @@
+# Discord relay
+
+Community threads on Discord, and replies from Discord back as posts (roadmap WS-J tasks 5 and 6).
+Everything is **off by default** and stays inert until the owner provides webhooks, a mapping and,
+for the inbound half, a bot token.
+
+## What it does
+
+**Out** (`DISCORD_RELAY_ENABLED=true` plus a mapping):
+
+- A new thread in a mapped **public** space is announced through the mapping's Discord webhook:
+ "New thread in s/<space> by <name>", the title, an excerpt of the opening post and the link.
+- Each reply to a thread that went to Discord follows it into the same channel (or into the
+ Discord thread the mapping names), with who replied, an excerpt and the link to the post.
+- Editing the opening post edits the thread's Discord message; editing a reply edits its message.
+- Deleting a reply or a thread (by its author or a moderator), or making a thread or space
+ members-only (VIP), deletes the Discord messages the relay had sent for it. Anything still
+ waiting to be sent is skipped.
+- Mentions are disabled on every message (`allowed_mentions.parse = []`); names and titles are
+ escaped, so nothing relayed can ping anyone.
+
+**In** (`DISCORD_RELAY_INBOUND=on`, `DISCORD_BOT_TOKEN`, and `inbound: true` on the mapping):
+
+- A reply on Discord to a relayed message (Discord's Reply), or any message in a Discord thread
+ started from a relayed message, becomes a post in that Community thread. Other messages in
+ the channel are ignored.
+- The post has origin `discord`, belongs to nobody on the site (no author subject) and shows the
+ Discord display name with a "Discord" badge. It is never relayed back out.
+- Edits on Discord edit the post (a new revision, `edited_by` `discord`); deletes on Discord
+ delete it. Deleting the relay's own announcement on Discord only marks it gone; the thread stays.
+- The thread must still be public, open (not locked) and not members-only; otherwise the
+ message is refused and the reason is recorded for staff.
+- Limits: `DISCORD_RELAY_INBOUND_PER_MINUTE` (6) messages a minute per Discord author per
+ mapping, ten times that per mapping; `DISCORD_RELAY_INBOUND_MAX_CHARS` (4000) per message,
+ longer text is cut. User, role and channel mentions, custom emoji and timestamps become plain
+ text, `@everyone`/`@here` are defused, attachments are noted ("2 attachments on Discord") but
+ never fetched. The rest is Markdown, which the forum's safe renderer shows (no raw HTML).
+
+## How it works
+
+| Piece | File | Notes |
+| --- | --- | --- |
+| Queue and sender | `server/relay/discord.js` | `relay_deliveries`: one row per dedupe key, actions `create`, `edit`, `delete`. Creates are sent with `?wait=true`, so Discord answers with the message and its id goes into the map. |
+| External message map (task 5) | `relay_message_map` | Every Discord message the relay knows, both ways (`out`: a thread or reply it sent; `in`: a reply that came from Discord). Unique both ways: a Discord message is one local object, and a local object is one message per mapping. Edits and deletes on either side find their counterpart here. |
+| Events worker (task 6) | `server/relay/events-worker.js` | Reads Community's own `community.thread.*` and `community.post.*` from OpenVibe.Events (`GET /api/v1/events`, capability `events.event.read`) and queues the creates. Its cursor (`relay_cursors`) moves in the same transaction as the deliveries a page queued; a replay queues nothing twice (dedupe keys), and nothing already in the map is sent twice. The first start begins at Events' head: older threads are never announced. |
+| Gateway | `server/relay/discord-gateway.js` | Discord gateway v10 over Node 22's built-in WebSocket: IDENTIFY/RESUME, heartbeats (a missing ACK means a zombied connection: resume), RECONNECT, INVALID_SESSION, backoff. Intents `GUILD_MESSAGES` and `MESSAGE_CONTENT`. |
+| Inbound | `server/relay/inbound.js` | `MESSAGE_CREATE`, `MESSAGE_UPDATE`, `MESSAGE_DELETE`, `MESSAGE_DELETE_BULK` for mapped channels only. |
+| Staff API | `server/relay/api.js` | `/api/v1/relay/*`, below. |
+
+When the Events worker is on, it is the one path for creates (the forum's own calls queue none).
+Without `EVENTS_URL` and `OV_OAUTH_CLIENT_SECRET`, or with `DISCORD_RELAY_EVENTS=off`, the forum
+queues creates itself, as the first relay did. Edits and deletes are always queued by the forum:
+Contracts has no `community.thread`/`community.post` update or delete events yet.
+
+**Loop prevention**, both halves: a thread or post with origin `discord` is never queued or sent
+out (checked when queueing and again when sending); inbound ignores webhook messages (the
+relay's own among them), bots, the bot itself and system messages, and never takes a message
+that is already in the map.
+
+**Retries and the dead letter**: network errors, timeouts, 5xx, 429 (its `retry_after`
+honoured; the rest of that webhook's queue waits for the next pass) and an unset webhook variable
+are retried with exponential backoff (`DISCORD_RELAY_BACKOFF_MS` · 2^(attempt−1), at most an hour)
+up to `DISCORD_RELAY_MAX_ATTEMPTS`. Then the delivery is `failed`: the dead letter, kept with its
+error, that staff retry or drop. Other 4xx fail at once. A delete that finds the message already
+gone on Discord counts as delivered; an edit of a message that is gone is skipped.
+
+**Secrets never live in the database**: a mapping names the environment variable holding the
+webhook URL (`webhook_url_ref`), and only allow-listed names (`DISCORD_WEBHOOK_*`, or exactly
+`DISCORD_RELAY_WEBHOOK_VARS`) can be mapped. No API response carries a webhook URL or the bot token.
+
+## Staff API
+
+Staff only: an admin/global_mod browser session, or a service holding `community.comment.moderate`.
+
+| Route | What |
+| --- | --- |
+| `GET /api/v1/relay/status` | The queue by status (`failed` = dead letters), the Events worker (cursor, latest seq, lag, last error, last gap), the gateway (state, reconnects, last error) and open inbound failures |
+| `GET /api/v1/relay/deliveries?status=&action=&limit=` | Deliveries; `status` pending, delivered, failed, dropped or skipped; `action` create, edit or delete |
+| `POST /api/v1/relay/deliveries/:id/retry` | Queue it again now, with a fresh attempt budget |
+| `POST /api/v1/relay/deliveries/:id/drop` | Give up on a pending or failed one (kept as `dropped`) |
+| `GET /api/v1/relay/inbound?all=1` | Messages from Discord that did not become (or change) a post, and why: ids only, never the text |
+| `POST /api/v1/relay/inbound/:id/dismiss` | Mark one seen |
+| `GET /api/v1/relay/mappings` | Mappings: space, webhook variable (and whether it is set), Discord channel and thread ids, inbound |
+| `POST /api/v1/relay/mappings` | `{ space, webhook_url_ref, enabled?, discord_channel_id?, discord_thread_id?, inbound? }` |
+| `PUT /api/v1/relay/mappings/:id` | `{ enabled?, inbound?, discord_channel_id?, discord_thread_id? }` (`null` clears an id) |
+
+`GET /api/ready` shows the same summary under `discord_relay` while the relay is on.
+
+## Owner steps
+
+1. **Webhook per channel** (outbound). In Discord: the channel's settings → Integrations →
+ Webhooks → New Webhook, name it (for example "OpenVibe.Community"), Copy Webhook URL. You need
+ Manage Webhooks on that channel to do this; the bot does not.
+2. **The bot** (inbound only). At <https://discord.com/developers/applications>: New Application →
+ Bot → Reset Token (this is `DISCORD_BOT_TOKEN`). Under Privileged Gateway Intents turn on
+ **Message Content Intent** (Presence and Server Members stay off). Invite it: OAuth2 → URL
+ Generator → scope `bot`, permissions **View Channels** and **Read Message History** (permission
+ integer `66560`). It sends nothing itself (the webhook does), so leave Send Messages and Manage
+ Webhooks off. It must be able to see each mapped channel, and threads in it (public threads only).
+3. **Environment** in `/etc/openvibe/community.env` (0600):
+
+ ```sh
+ DISCORD_RELAY_ENABLED=true
+ DISCORD_WEBHOOK_GENERAL=https://discord.com/api/webhooks/<id>/<token> # one per channel, any DISCORD_WEBHOOK_* name
+ # inbound
+ DISCORD_RELAY_INBOUND=on
+ DISCORD_BOT_TOKEN=<the bot token>
+ ```
+
+ `EVENTS_URL` and `OV_OAUTH_CLIENT_SECRET` are already set for the outbox, so the Events worker
+ starts by itself. It needs the Network grant `community` → `events.event.read` on audience
+ `openvibe.events` (in the Network's principal seed; check it is provisioned: a missing grant
+ shows as a 403 in the worker's `last_error`).
+ Then `sudo systemctl restart openvibe-community`. The boot log says
+ `[Relay] Discord relay on: creates queued by the Events worker; events worker on; inbound on`.
+4. **Map a space**, signed in as an admin on openvibe.community, from the browser console:
+
+ ```js
+ await (await fetch('/api/v1/relay/mappings', { method: 'POST', headers: { 'content-type': 'application/json' },
+ body: JSON.stringify({ space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL', inbound: true }) })).json();
+ ```
+
+ `discord_channel_id` is learned from Discord's answer to the first message; set it yourself
+ (Discord: Settings → Advanced → Developer Mode, then right-click the channel → Copy Channel ID)
+ if you want replies accepted before anything was sent. To post into one Discord thread instead
+ of the channel, add `discord_thread_id`. Only public spaces are ever relayed.
+5. **One production round trip**: start a thread in the mapped space → it appears on Discord;
+ reply to it on Discord (Reply, or in a thread on the message) → the reply appears in the
+ thread with a Discord badge; edit and delete it on Discord → the post follows; reply and edit
+ on the site → Discord follows; delete the thread → its messages leave Discord. Watch
+ `GET /api/v1/relay/status` (no dead letters, lag 0, gateway `ready`).
+
+**Off again**: unset `DISCORD_RELAY_ENABLED` and restart (nothing is sent or read; the tables stay),
+or per mapping `PUT /api/v1/relay/mappings/:id { "enabled": false }` (deletes still go out) or
+`{ "inbound": false }`. Unset `DISCORD_RELAY_INBOUND` to close the gateway only.
+
+## Configuration
+
+| Variable | Default | What |
+| --- | --- | --- |
+| `DISCORD_RELAY_ENABLED` | off | The relay at all |
+| `DISCORD_WEBHOOK_*` | — | Webhook URLs, named by mappings |
+| `DISCORD_RELAY_WEBHOOK_VARS` | — | Exact variable names mappings may use (instead of `DISCORD_WEBHOOK_*`) |
+| `DISCORD_RELAY_POLL_MS` / `DISCORD_RELAY_BACKOFF_MS` / `DISCORD_RELAY_MAX_ATTEMPTS` | `30000` / `30000` / `6` | Sender cadence, first retry delay, attempts before the dead letter |
+| `DISCORD_RELAY_EVENTS` | on | `off` leaves creates to the forum instead of the Events worker |
+| `DISCORD_RELAY_EVENTS_POLL_MS` | `5000` | How often the worker reads Events |
+| `EVENTS_URL`, `OV_OAUTH_CLIENT_SECRET` | — | Events' address and Community's client secret (shared with the outbox) |
+| `DISCORD_RELAY_INBOUND` | off | The gateway |
+| `DISCORD_BOT_TOKEN` | — | The bot's token (inbound only) |
+| `DISCORD_GATEWAY_URL` | `wss://gateway.discord.gg/?v=10&encoding=json` | The gateway |
+| `DISCORD_RELAY_INBOUND_PER_MINUTE` | `6` | Messages a minute per Discord author per mapping (10× per mapping) |
+| `DISCORD_RELAY_INBOUND_MAX_CHARS` | `4000` | Longest message taken in; longer is cut |
+
+## Known limits
+
+- Messages sent on Discord while Community is restarting (or the gateway is down longer than a
+ resume allows) are not replayed: a new session starts from now.
+- Private Discord threads, attachments (either way), reactions and Discord-side thread titles are
+ not relayed.
+- Threads created before the Events worker's first start are never announced, and replies go
+ out only for threads that went out through that mapping.
+- Edits made by the Roadmap sync (`docs/roadmap/public.json`) are not propagated to Discord.
+- A retention gap in Events (the worker fell further behind than Events keeps) skips the creates
+ in it; the gap is logged and shown in the status.
@@ -169,6 +169,7 @@ a.badge:hover { border-color: var(--accent); text-decoration: none; color: var(-
@@ -169,6 +169,7 @@ a.badge:hover { border-color: var(--accent); text-decoration: none; color: var(-
.thread-meta { display: flex; flex-wrap: wrap; align-items: center; gap: 4px 6px; margin: 4px 0 0; font-size: .82rem; color: var(--text-secondary); }
.thread-meta { display: flex; flex-wrap: wrap; align-items: center; gap: 4px 6px; margin: 4px 0 0; font-size: .82rem; color: var(--text-secondary); }
.pulse-source { flex: none; margin-top: 2px; }
.pulse-source { flex: none; margin-top: 2px; }
.badge-ai { border-color: var(--info); color: var(--info); }
.badge-ai { border-color: var(--info); color: var(--info); }
+.badge-relay { border-color: var(--accent-light); color: var(--accent-light); }
.badge-vis { font-size: .7rem; vertical-align: middle; }
.badge-vis { font-size: .7rem; vertical-align: middle; }
/* Categories, requests and the roadmap (forum pages) */
/* Categories, requests and the roadmap (forum pages) */
.badge-status, .badge-category { font-size: .7rem; vertical-align: middle; }
.badge-status, .badge-category { font-size: .7rem; vertical-align: middle; }
@@ -55,6 +55,9 @@ const { createForumRoutes } = require('./forum/routes');
@@ -55,6 +55,9 @@ const { createForumRoutes } = require('./forum/routes');
const { createPulse } = require('./pulse/service');
const { createPulse } = require('./pulse/service');
const { createPulseApi } = require('./pulse/api');
const { createPulseApi } = require('./pulse/api');
const { createDiscordRelay } = require('./relay/discord');
const { createDiscordRelay } = require('./relay/discord');
+const { createRelayEventsWorker } = require('./relay/events-worker');
+const { createDiscordGateway } = require('./relay/discord-gateway');
+const { createDiscordInbound } = require('./relay/inbound');
const { createRelayApi } = require('./relay/api');
const { createRelayApi } = require('./relay/api');
const { pulsePage } = require('./render/pulse');
const { pulsePage } = require('./render/pulse');
const { extensionFor } = require('./render/highlight');
const { extensionFor } = require('./render/highlight');
@@ -124,6 +127,24 @@ function createApp(opts = {}) {
@@ -124,6 +127,24 @@ function createApp(opts = {}) {
webhookVars: config.discordRelay.webhookVars,
webhookVars: config.discordRelay.webhookVars,
...(opts.relayOptions || {}),
...(opts.relayOptions || {}),
});
});
+ // Its Events worker (creates from community.thread.* / community.post.*) and the inbound gateway;
+ // both off unless configured (relay/events-worker.js, relay/discord-gateway.js, relay/inbound.js).
+ const relayInbound = relay.enabled ? createDiscordInbound({ db, perMinute: config.discordRelay.inboundPerMinute, maxChars: config.discordRelay.inboundMaxChars, ...(opts.inboundOptions || {}) }) : null;
+ if (!opts.relay) {
+ relay.attach({
+ worker: createRelayEventsWorker({
+ db, relay, enabled: config.discordRelay.events, eventsUrl: config.discordRelay.eventsUrl, clientSecret: config.oauth.clientSecret,
+ tokenUrl: `${config.networkInternalUrl}/oauth/token`, clientId: config.oauth.clientId, pollMs: config.discordRelay.eventsPollMs, fetchImpl: opts.fetchImpl,
+ ...(opts.relayWorkerOptions || {}),
+ }),
+ gateway: relayInbound && config.discordRelay.inbound ? createDiscordGateway({
+ token: config.discordRelay.botToken, url: config.discordRelay.gatewayUrl,
+ onDispatch: (type, data, ctx) => relayInbound.handle(type, data, ctx),
+ ...(opts.gatewayOptions || {}),
+ }) : null,
+ inbound: relayInbound,
+ });
+ }
// OpenVibe.VIP: members-only spaces and threads (fails closed without a client secret or VIP).
// OpenVibe.VIP: members-only spaces and threads (fails closed without a client secret or VIP).
const vip = opts.vip || createVipGate({ config, ...(opts.vipOptions || {}) });
const vip = opts.vip || createVipGate({ config, ...(opts.vipOptions || {}) });
// Images on posts go to OpenVibe.Media's Object API as med_ objects (media/objects.js).
// Images on posts go to OpenVibe.Media's Object API as med_ objects (media/objects.js).
@@ -132,7 +153,7 @@ function createApp(opts = {}) {
@@ -132,7 +153,7 @@ function createApp(opts = {}) {
const community = config.pastesAuthority === 'community';
const community = config.pastesAuthority === 'community';
const comments = createCommentService({ db, network, pastesLocal: community, limits: opts.commentLimits });
const comments = createCommentService({ db, network, pastesLocal: community, limits: opts.commentLimits });
seo.useForum(forum);
seo.useForum(forum);
- Object.assign(app.locals, { db, network, pulse, relay, vip, forum, comments });
+ Object.assign(app.locals, { db, network, pulse, relay, relayInbound, vip, forum, comments });
if (opts.startRelay !== false) relay.start();
if (opts.startRelay !== false) relay.start();
// ── Paste authority ──────────────────────────────────────
// ── Paste authority ──────────────────────────────────────
@@ -189,7 +210,7 @@ function createApp(opts = {}) {
@@ -189,7 +210,7 @@ function createApp(opts = {}) {
app.use('/api/v1/spaces', createSpacesApi({ forum, viewers }));
app.use('/api/v1/spaces', createSpacesApi({ forum, viewers }));
app.use('/api/v1/posts', createPostsApi({ forum, viewers }));
app.use('/api/v1/posts', createPostsApi({ forum, viewers }));
app.use('/api/v1/space-groups', createGroupsApi({ forum, viewers }));
app.use('/api/v1/space-groups', createGroupsApi({ forum, viewers }));
- app.use('/api/v1/relay', createRelayApi({ relay, db, viewers }));
+ app.use('/api/v1/relay', createRelayApi({ relay, db, viewers, inbound: relayInbound }));
app.get('/api/health', (_req, res) => res.json({ status: 'ok', service: 'openvibe-community', version: VERSION }));
app.get('/api/health', (_req, res) => res.json({ status: 'ok', service: 'openvibe-community', version: VERSION }));
// GET /release.json (ADR-016) and POST /release-metrics: open tabs' update reports (a same-origin
// GET /release.json (ADR-016) and POST /release-metrics: open tabs' update reports (a same-origin
@@ -76,7 +76,10 @@ module.exports = {
@@ -76,7 +76,10 @@ module.exports = {
unavailableTtlMs: parseInt(process.env.VIP_CACHE_UNAVAILABLE_TTL_MS, 10) || 2_000,
unavailableTtlMs: parseInt(process.env.VIP_CACHE_UNAVAILABLE_TTL_MS, 10) || 2_000,
},
},
- // Discord relay (outbound: new threads → a Discord webhook per mapped space). Off by default.
+ // Discord relay (server/relay; docs/discord-relay.md). Off by default, and inert without an owner's
+ // webhook variables, mappings and (for inbound) bot token.
+ // out threads and replies in mapped public spaces → the mapping's Discord webhook; edits and deletes follow
+ // in replies on Discord → posts (the gateway; DISCORD_RELAY_INBOUND=on and DISCORD_BOT_TOKEN)
// Webhook URLs live in environment variables named by relay_mappings.webhook_url_ref.
// Webhook URLs live in environment variables named by relay_mappings.webhook_url_ref.
discordRelay: {
discordRelay: {
enabled: /^(1|true|yes|on)$/i.test(process.env.DISCORD_RELAY_ENABLED || ''),
enabled: /^(1|true|yes|on)$/i.test(process.env.DISCORD_RELAY_ENABLED || ''),
@@ -85,5 +88,16 @@ module.exports = {
@@ -85,5 +88,16 @@ module.exports = {
maxAttempts: parseInt(process.env.DISCORD_RELAY_MAX_ATTEMPTS, 10) || 6,
maxAttempts: parseInt(process.env.DISCORD_RELAY_MAX_ATTEMPTS, 10) || 6,
// The only variables a mapping may name (comma-separated exact names); unset = DISCORD_WEBHOOK_*.
// The only variables a mapping may name (comma-separated exact names); unset = DISCORD_WEBHOOK_*.
webhookVars: (process.env.DISCORD_RELAY_WEBHOOK_VARS || '').split(',').map((s) => s.trim()).filter(Boolean),
webhookVars: (process.env.DISCORD_RELAY_WEBHOOK_VARS || '').split(',').map((s) => s.trim()).filter(Boolean),
+ // The Events worker queues creates from community.thread.* / community.post.* (needs EVENTS_URL and
+ // OV_OAUTH_CLIENT_SECRET, capability events.event.read); 'off' leaves them to the forum.
+ events: !/^(0|false|no|off)$/i.test(process.env.DISCORD_RELAY_EVENTS || ''),
+ eventsUrl: (process.env.EVENTS_URL || '').replace(/\/+$/, '') || null,
+ eventsPollMs: parseInt(process.env.DISCORD_RELAY_EVENTS_POLL_MS, 10) || 5000,
+ // Inbound through the Discord gateway (a bot in the server with the MESSAGE CONTENT intent).
+ inbound: /^(1|true|yes|on)$/i.test(process.env.DISCORD_RELAY_INBOUND || ''),
+ botToken: process.env.DISCORD_BOT_TOKEN || '',
+ gatewayUrl: process.env.DISCORD_GATEWAY_URL || 'wss://gateway.discord.gg/?v=10&encoding=json',
+ inboundPerMinute: parseInt(process.env.DISCORD_RELAY_INBOUND_PER_MINUTE, 10) || 6,
+ inboundMaxChars: parseInt(process.env.DISCORD_RELAY_INBOUND_MAX_CHARS, 10) || 4000,
},
},
};
};
@@ -16,6 +16,26 @@ const fs = require('fs');
@@ -16,6 +16,26 @@ const fs = require('fs');
const path = require('path');
const path = require('path');
const Database = require('better-sqlite3');
const Database = require('better-sqlite3');
+// relay_deliveries' columns: the table in SCHEMA, and migrate()'s one-time rebuild of the older shape.
+const RELAY_DELIVERIES_COLUMNS = `
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ thread_id INTEGER NOT NULL REFERENCES threads(id) ON DELETE CASCADE,
+ post_id INTEGER REFERENCES posts(id) ON DELETE CASCADE,
+ mapping_id INTEGER NOT NULL REFERENCES relay_mappings(id) ON DELETE CASCADE,
+ action TEXT NOT NULL DEFAULT 'create' CHECK(action IN ('create', 'edit', 'delete')),
+ dedupe_key TEXT UNIQUE NOT NULL,
+ source TEXT NOT NULL DEFAULT 'direct' CHECK(source IN ('direct', 'events')),
+ event_id TEXT,
+ status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'delivered', 'failed', 'dropped', 'skipped')),
+ attempts INTEGER NOT NULL DEFAULT 0,
+ next_attempt_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ last_status INTEGER,
+ last_error TEXT,
+ delivered_at DATETIME,
+ created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
+`;
+
const SCHEMA = `
const SCHEMA = `
CREATE TABLE IF NOT EXISTS pastes (
CREATE TABLE IF NOT EXISTS pastes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
id INTEGER PRIMARY KEY AUTOINCREMENT,
@@ -242,9 +262,6 @@ CREATE TABLE IF NOT EXISTS thread_votes (
@@ -242,9 +262,6 @@ CREATE TABLE IF NOT EXISTS thread_votes (
PRIMARY KEY (thread_id, subject_id)
PRIMARY KEY (thread_id, subject_id)
);
);
-- Images attached to forum posts (WS-J task 2): the bytes are a Media object (med_…, tenant community,
-- Images attached to forum posts (WS-J task 2): the bytes are a Media object (med_…, tenant community,
-- owned by the person); an upload waits here with no post until the thread or reply naming it is saved.
-- owned by the person); an upload waits here with no post until the thread or reply naming it is saved.
CREATE TABLE IF NOT EXISTS attachments (
CREATE TABLE IF NOT EXISTS attachments (
@@ -299,6 +316,11 @@ CREATE TABLE IF NOT EXISTS categories (
@@ -299,6 +316,11 @@ CREATE TABLE IF NOT EXISTS categories (
UNIQUE (space_id, slug)
UNIQUE (space_id, slug)
);
);
+-- ── Discord relay (server/relay; DISCORD_RELAY_ENABLED) ──────
+-- webhook_url_ref is the NAME of an environment variable holding the webhook URL: secrets never
+-- live in the database. migrate() adds discord_channel_id (the channel the webhook posts into;
+-- learned from Discord's answer when unset), discord_thread_id (post into this Discord thread) and
+-- inbound (replies from Discord come back as posts).
CREATE TABLE IF NOT EXISTS relay_mappings (
CREATE TABLE IF NOT EXISTS relay_mappings (
id INTEGER PRIMARY KEY AUTOINCREMENT,
id INTEGER PRIMARY KEY AUTOINCREMENT,
space_id INTEGER NOT NULL REFERENCES spaces(id) ON DELETE CASCADE,
space_id INTEGER NOT NULL REFERENCES spaces(id) ON DELETE CASCADE,
@@ -309,23 +331,63 @@ CREATE TABLE IF NOT EXISTS relay_mappings (
@@ -309,23 +331,63 @@ CREATE TABLE IF NOT EXISTS relay_mappings (
UNIQUE (space_id, direction, webhook_url_ref)
UNIQUE (space_id, direction, webhook_url_ref)
);
);
-CREATE TABLE IF NOT EXISTS relay_deliveries (
+-- One outbound action per dedupe key (WS-J task 6): send a thread's or a reply's Discord message
+-- (create), edit it or delete it. post_id NULL: the thread's own message. source: queued by the
+-- Events worker from community.thread.* / community.post.* (event_id), or directly by the forum.
+-- status pending → delivered | failed (the dead letter: retries used up, or refused; staff retry or
+-- drop it) | dropped (by staff) | skipped (nothing left to do, e.g. deleted before it was sent).
+CREATE TABLE IF NOT EXISTS relay_deliveries (${RELAY_DELIVERIES_COLUMNS});
+CREATE INDEX IF NOT EXISTS idx_relay_deliveries_due ON relay_deliveries(status, next_attempt_at);
+
+-- External message map (WS-J task 5): every Discord message the relay knows, both ways.
+-- out a thread (local_type 'thread': title and opening post) or a reply sent through the mapping's webhook
+-- in a reply written on Discord, now a post with origin 'discord' (local_type 'post')
+-- external_channel_id is where the message is (a Discord thread's id for one in a thread);
+-- external_thread_id is set when the relay sent it into a Discord thread (edits and deletes name it).
+-- Unique both ways: a Discord message is one local object; a local object is one message per mapping.
+-- Edits and deletes on either side find their counterpart here.
+CREATE TABLE IF NOT EXISTS relay_message_map (
id INTEGER PRIMARY KEY AUTOINCREMENT,
id INTEGER PRIMARY KEY AUTOINCREMENT,
- thread_id INTEGER NOT NULL REFERENCES threads(id) ON DELETE CASCADE,
+ platform TEXT NOT NULL DEFAULT 'discord' CHECK(platform IN ('discord')),
mapping_id INTEGER NOT NULL REFERENCES relay_mappings(id) ON DELETE CASCADE,
mapping_id INTEGER NOT NULL REFERENCES relay_mappings(id) ON DELETE CASCADE,
- dedupe_key TEXT UNIQUE NOT NULL,
- status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'delivered', 'failed')),
- attempts INTEGER NOT NULL DEFAULT 0,
- next_attempt_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
- last_status INTEGER,
- last_error TEXT,
- delivered_at DATETIME,
+ direction TEXT NOT NULL CHECK(direction IN ('out', 'in')),
+ local_type TEXT NOT NULL CHECK(local_type IN ('thread', 'post')),
+ local_id INTEGER NOT NULL,
+ thread_id INTEGER NOT NULL REFERENCES threads(id) ON DELETE CASCADE,
+ external_channel_id TEXT NOT NULL,
+ external_thread_id TEXT,
+ external_message_id TEXT NOT NULL,
+ external_webhook_id TEXT,
+ external_deleted_at DATETIME,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP,
- UNIQUE (thread_id, mapping_id)
+ UNIQUE (platform, external_message_id),
+ UNIQUE (platform, mapping_id, local_type, local_id)
+);
+CREATE INDEX IF NOT EXISTS idx_relay_message_map_thread ON relay_message_map(thread_id);
+
+-- The relay's Events worker's place in OpenVibe.Events (server/relay/events-worker.js): the last seq
+-- it handled, moved in the same transaction as the deliveries that page queued.
+CREATE TABLE IF NOT EXISTS relay_cursors (
+ name TEXT PRIMARY KEY,
+ cursor INTEGER NOT NULL,
+ latest_seq INTEGER,
+ updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Messages from Discord that did not become (or change) a post, for staff (GET /api/v1/relay/inbound).
+-- Ids and the reason only, never the text.
+CREATE TABLE IF NOT EXISTS relay_inbound_failures (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ mapping_id INTEGER REFERENCES relay_mappings(id) ON DELETE SET NULL,
+ event TEXT NOT NULL,
+ external_channel_id TEXT,
+ external_message_id TEXT,
+ external_author_id TEXT,
+ error TEXT NOT NULL,
+ dismissed_at DATETIME,
+ created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP
);
);
-CREATE INDEX IF NOT EXISTS idx_relay_deliveries_due ON relay_deliveries(status, next_attempt_at);
-- ── Pulse (server/pulse): public activity across the network, with provenance ──
-- ── Pulse (server/pulse): public activity across the network, with provenance ──
-- Only public things enter; AI items carry origin 'ai' and never an actor.
-- Only public things enter; AI items carry origin 'ai' and never an actor.
@@ -463,6 +525,28 @@ function migrate(db) {
@@ -463,6 +525,28 @@ function migrate(db) {
['roadmap', 'features', 'Features', 'New things on sites that are already open.', 2],
['roadmap', 'features', 'Features', 'New things on sites that are already open.', 2],
['roadmap', 'platform', 'Under the hood', 'Accounts, safety, reliability and the shared systems every site uses.', 3],
['roadmap', 'platform', 'Under the hood', 'Accounts, safety, reliability and the shared systems every site uses.', 3],
]) seedCategory.run(slug, name, description, position, space);
]) seedCategory.run(slug, name, description, position, space);
+ // The Discord relay both ways (WS-J tasks 5 and 6; server/relay): where a mapping's webhook posts and
+ // whether replies come back, the Discord name on posts from Discord, and relay_deliveries rebuilt once
+ // for replies, edits and deletes (the older table allowed one row per (thread, mapping) and no actions).
+ const mappingCols = new Set(db.prepare('PRAGMA table_info(relay_mappings)').all().map((x) => x.name));
+ if (!mappingCols.has('discord_channel_id')) db.exec('ALTER TABLE relay_mappings ADD COLUMN discord_channel_id TEXT');
+ if (!mappingCols.has('discord_thread_id')) db.exec('ALTER TABLE relay_mappings ADD COLUMN discord_thread_id TEXT');
+ if (!mappingCols.has('inbound')) db.exec('ALTER TABLE relay_mappings ADD COLUMN inbound INTEGER NOT NULL DEFAULT 0');
+ const postCols = new Set(db.prepare('PRAGMA table_info(posts)').all().map((x) => x.name));
+ if (!postCols.has('relay_author')) db.exec('ALTER TABLE posts ADD COLUMN relay_author TEXT');
+ const deliveryCols = new Set(db.prepare('PRAGMA table_info(relay_deliveries)').all().map((x) => x.name));
+ if (!deliveryCols.has('action')) {
+ db.transaction(() => {
+ db.exec(`CREATE TABLE relay_deliveries_v2 (${RELAY_DELIVERIES_COLUMNS});
+ INSERT INTO relay_deliveries_v2 (id, thread_id, mapping_id, dedupe_key, status, attempts, next_attempt_at, last_status, last_error, delivered_at, created_at, updated_at)
+ SELECT id, thread_id, mapping_id, dedupe_key, status, attempts, next_attempt_at, last_status, last_error, delivered_at, created_at, updated_at FROM relay_deliveries;
+ DROP TABLE relay_deliveries;
+ ALTER TABLE relay_deliveries_v2 RENAME TO relay_deliveries;
+ CREATE INDEX IF NOT EXISTS idx_relay_deliveries_due ON relay_deliveries(status, next_attempt_at);`);
+ })();
+ }
+ db.exec(`CREATE INDEX IF NOT EXISTS idx_relay_deliveries_thread ON relay_deliveries(thread_id, post_id);
+ CREATE INDEX IF NOT EXISTS idx_relay_mappings_channel ON relay_mappings(discord_channel_id) WHERE discord_channel_id IS NOT NULL;`);
}
}
/** cth_ + 22 base64url characters (16 random bytes). */
/** cth_ + 22 base64url characters (16 random bytes). */
@@ -15,8 +15,10 @@
@@ -15,8 +15,10 @@
* own posts; moderators (identity/capabilities.js discussionModerator) any, and pin/lock.
* own posts; moderators (identity/capabilities.js discussionModerator) any, and pin/lock.
* Edits keep every revision in post_versions.
* Edits keep every revision in post_versions.
*
*
- * Side effects of a new thread/post: a Pulse item (public spaces) and, for threads, a Discord
- * relay delivery per mapping (relay/discord.js; off unless DISCORD_RELAY_ENABLED).
+ * Side effects of a new thread/post: a Pulse item (public spaces) and a Discord relay delivery per
+ * mapping (relay/discord.js; off unless DISCORD_RELAY_ENABLED; while its Events worker runs, it
+ * queues creates from community.thread/post.created instead). Edits, deletes and gating queue the
+ * matching Discord edit or delete for whatever the relay had sent.
*
*
* Members-only (OpenVibe.VIP): a space or a single thread can be for one creator's VIP members
* Members-only (OpenVibe.VIP): a space or a single thread can be for one creator's VIP members
* (members_only_owner = the creator's usr_ subject). Reading its threads and posts, starting a
* (members_only_owner = the creator's usr_ subject). Reading its threads and posts, starting a
@@ -282,7 +284,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
@@ -282,7 +284,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
attachments: deleted ? [] : ((attachments && attachments.get(p.id)) || []),
attachments: deleted ? [] : ((attachments && attachments.get(p.id)) || []),
pastes: deleted ? [] : ((pastes && pastes.get(p.id)) || []),
pastes: deleted ? [] : ((pastes && pastes.get(p.id)) || []),
id: p.id, thread_id: p.thread_id, is_opening: !!p.is_opening, origin: p.origin,
id: p.id, thread_id: p.thread_id, is_opening: !!p.is_opening, origin: p.origin,
- author: deleted ? null : authors.author(p.author_subject, p.origin, projections),
+ author: deleted ? null : authors.author(p.author_subject, p.origin, projections, p.relay_author),
body_markdown: deleted ? null : p.body_markdown,
body_markdown: deleted ? null : p.body_markdown,
body_html: deleted ? null : renderMarkdown(p.body_markdown),
body_html: deleted ? null : renderMarkdown(p.body_markdown),
revision: p.revision, deleted,
revision: p.revision, deleted,
@@ -311,14 +313,14 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
@@ -311,14 +313,14 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
};
};
}
}
- /** Newly gated threads leave Pulse and the Discord relay queue at once (reads re-check as well). */
+ /**
+ * Newly gated threads leave Pulse at once, and Discord: what the relay had not sent yet is skipped and
+ * what it had sent is deleted there (reads and sends re-check as well).
+ */
function hideGated(threadIds) {
function hideGated(threadIds) {
if (!threadIds.length) return;
if (!threadIds.length) return;
if (pulse) hook(() => { for (const id of threadIds) { pulse.threadGone(id); for (const p of db.prepare('SELECT id FROM posts WHERE thread_id = ?').all(id)) pulse.postGone(p.id); } });
if (pulse) hook(() => { for (const id of threadIds) { pulse.threadGone(id); for (const p of db.prepare('SELECT id FROM posts WHERE thread_id = ?').all(id)) pulse.postGone(p.id); } });
- hook(() => {
- const del = db.prepare("DELETE FROM relay_deliveries WHERE thread_id = ? AND status = 'pending'");
- for (const id of threadIds) del.run(id);
- });
+ if (relay) hook(() => relay.hideThreads(threadIds, 'made members-only'));
}
}
function removeThread(v, thread) {
function removeThread(v, thread) {
@@ -329,6 +331,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
@@ -329,6 +331,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
if (!mine) events.moderationAction(v, 'thread.deleted', { type: 'thread', id: String(thread.id), owner_subject: thread.author_subject || null });
if (!mine) events.moderationAction(v, 'thread.deleted', { type: 'thread', id: String(thread.id), owner_subject: thread.author_subject || null });
})();
})();
if (pulse) hook(() => { pulse.threadGone(thread.id); for (const p of db.prepare('SELECT id FROM posts WHERE thread_id = ?').all(thread.id)) pulse.postGone(p.id); });
if (pulse) hook(() => { pulse.threadGone(thread.id); for (const p of db.prepare('SELECT id FROM posts WHERE thread_id = ?').all(thread.id)) pulse.postGone(p.id); });
+ if (relay) hook(() => relay.enqueueDelete(thread.id, null, mine ? 'deleted by its author' : 'deleted by a moderator'));
return { ok: true, id: thread.id, deleted: 'thread' };
return { ok: true, id: thread.id, deleted: 'thread' };
}
}
@@ -586,7 +589,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
@@ -586,7 +589,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
const { post, thread, space } = postFor(v, postId);
const { post, thread, space } = postFor(v, postId);
await requireMembership(v, space, thread);
await requireMembership(v, space, thread);
const projections = await authors.projectionsFor([post.author_subject]);
const projections = await authors.projectionsFor([post.author_subject]);
- const a = authors.author(post.author_subject, post.origin, projections);
+ const a = authors.author(post.author_subject, post.origin, projections, post.relay_author);
const name = a ? (a.username ? `@${a.username}` : a.display_name || 'Someone') : 'Someone';
const name = a ? (a.username ? `@${a.username}` : a.display_name || 'Someone') : 'Someone';
const quoted = String(post.body_markdown || '').split('\n').slice(0, 40).map((l) => `> ${l}`).join('\n');
const quoted = String(post.body_markdown || '').split('\n').slice(0, 40).map((l) => `> ${l}`).join('\n');
return { markdown: `**${name}** wrote:\n${quoted}\n\n` };
return { markdown: `**${name}** wrote:\n${quoted}\n\n` };
@@ -749,6 +752,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
@@ -749,6 +752,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
attach(images, post.id);
attach(images, post.id);
attachPastes(pasteRows, post.id);
attachPastes(pasteRows, post.id);
if (pulse) hook(() => pulse.postCreated(post, thread, space));
if (pulse) hook(() => pulse.postCreated(post, thread, space));
+ if (relay) hook(() => relay.enqueuePost(post, thread, space));
const projections = await authors.projectionsFor([post.author_subject]);
const projections = await authors.projectionsFor([post.author_subject]);
// Where the new post lands: its page in the thread (posts are numbered in id order).
// Where the new post lands: its page in the thread (posts are numbered in id order).
const position = db.prepare('SELECT COUNT(*) AS c FROM posts WHERE thread_id = ? AND id <= ?').get(thread.id, post.id).c;
const position = db.prepare('SELECT COUNT(*) AS c FROM posts WHERE thread_id = ? AND id <= ?').get(thread.id, post.id).c;
@@ -764,6 +768,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
@@ -764,6 +768,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
if (!mine && !moderator(v)) fail(403, 'post.not_yours', 'Only the author or a moderator edits a post');
if (!mine && !moderator(v)) fail(403, 'post.not_yours', 'Only the author or a moderator edits a post');
if (thread.locked && !moderator(v)) fail(403, 'thread.locked', 'This thread is locked');
if (thread.locked && !moderator(v)) fail(403, 'thread.locked', 'This thread is locked');
const next = store.editPost(db, post.id, cleanBody(body.body != null ? body.body : body.body_markdown), v.subject || v.service || null);
const next = store.editPost(db, post.id, cleanBody(body.body != null ? body.body : body.body_markdown), v.subject || v.service || null);
+ if (relay && next && next.revision !== post.revision) hook(() => relay.enqueueEdit(next));
const projections = await authors.projectionsFor([next.author_subject]);
const projections = await authors.projectionsFor([next.author_subject]);
return { post: shapePost(next, v, projections) };
return { post: shapePost(next, v, projections) };
},
},
@@ -787,6 +792,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
@@ -787,6 +792,7 @@ function createForumService({ db, network = null, pulse = null, relay = null, vi
if (!(person(v) && post.author_subject === v.subject)) events.moderationAction(v, 'post.deleted', { type: 'post', id: String(post.id), owner_subject: post.author_subject || null }, { details: { thread: String(thread.id) } });
if (!(person(v) && post.author_subject === v.subject)) events.moderationAction(v, 'post.deleted', { type: 'post', id: String(post.id), owner_subject: post.author_subject || null }, { details: { thread: String(thread.id) } });
})();
})();
if (pulse) hook(() => pulse.postGone(post.id));
if (pulse) hook(() => pulse.postGone(post.id));
+ if (relay) hook(() => relay.enqueueDelete(thread.id, post.id, person(v) && post.author_subject === v.subject ? 'deleted by its author' : 'deleted by a moderator'));
return { ok: true, id: post.id };
return { ok: true, id: post.id };
},
},
@@ -239,11 +239,14 @@ function getPost(db, id) {
@@ -239,11 +239,14 @@ function getPost(db, id) {
return db.prepare('SELECT * FROM posts WHERE id = ?').get(id) || null;
return db.prepare('SELECT * FROM posts WHERE id = ?').get(id) || null;
}
}
-/** A reply; bumps the thread's reply_count and last activity. */
-function addPost(db, { thread_id, author_subject = null, origin = 'user', body_markdown }) {
+/**
+ * A reply; bumps the thread's reply_count and last activity. relay_author: the name a reply written
+ * on Discord carries (origin 'discord', no author subject; server/relay/inbound.js).
+ */
+function addPost(db, { thread_id, author_subject = null, origin = 'user', body_markdown, relay_author = null }) {
return db.transaction(() => {
return db.transaction(() => {
- const info = db.prepare('INSERT INTO posts (thread_id, author_subject, origin, body_markdown) VALUES (?, ?, ?, ?)')
- .run(thread_id, author_subject, origin, body_markdown);
+ const info = db.prepare('INSERT INTO posts (thread_id, author_subject, origin, body_markdown, relay_author) VALUES (?, ?, ?, ?, ?)')
+ .run(thread_id, author_subject, origin, body_markdown, relay_author);
db.prepare('UPDATE threads SET reply_count = reply_count + 1, last_activity_at = CURRENT_TIMESTAMP WHERE id = ?').run(thread_id);
db.prepare('UPDATE threads SET reply_count = reply_count + 1, last_activity_at = CURRENT_TIMESTAMP WHERE id = ?').run(thread_id);
const post = getPost(db, info.lastInsertRowid);
const post = getPost(db, info.lastInsertRowid);
const thread = getThread(db, thread_id);
const thread = getThread(db, thread_id);
@@ -20,10 +20,12 @@ function createAuthors({ db, network = null }) {
@@ -20,10 +20,12 @@ function createAuthors({ db, network = null }) {
/**
/**
* { subject, username, display_name, avatar_url, profile_color } for a person, a fixed AI
* { subject, username, display_name, avatar_url, profile_color } for a person, a fixed AI
- * label for AI output, null for anonymous writes.
+ * label for AI output, the Discord name (relayName; is_relay) for a reply written on Discord,
+ * null for anonymous writes.
*/
*/
- function author(subject, origin, projections) {
+ function author(subject, origin, projections, relayName = null) {
if (origin === 'ai') return { subject: null, username: null, display_name: AI_DISPLAY_NAME, avatar_url: null, profile_color: null, is_ai: true };
if (origin === 'ai') return { subject: null, username: null, display_name: AI_DISPLAY_NAME, avatar_url: null, profile_color: null, is_ai: true };
+ if (origin === 'discord') return { subject: null, username: null, display_name: relayName || 'Discord', avatar_url: null, profile_color: null, is_relay: true, relay: 'discord' };
if (origin === 'system' && !subject) return { subject: null, username: null, display_name: 'OpenVibe', avatar_url: null, profile_color: null, is_system: true };
if (origin === 'system' && !subject) return { subject: null, username: null, display_name: 'OpenVibe', avatar_url: null, profile_color: null, is_system: true };
if (!subject) return null;
if (!subject) return null;
const p = projections.get(subject);
const p = projections.get(subject);
@@ -8,6 +8,11 @@ const config = require('./config');
@@ -8,6 +8,11 @@ const config = require('./config');
const { createApp } = require('./app');
const { createApp } = require('./app');
const app = createApp();
const app = createApp();
+if (config.discordRelay.enabled) {
+ const rs = app.locals.relay.status();
+ const part = (x) => (x.enabled ? 'on' : `off (${x.reason})`);
+ console.log(`[Relay] Discord relay on: creates queued by the ${rs.creates_from === 'events' ? 'Events worker' : 'forum'}; events worker ${part(rs.events_worker)}; inbound ${part(rs.inbound)}`);
+}
// Community → OpenVibe.Events (server/events.js): off unless EVENTS_URL and the client secret are set.
// Community → OpenVibe.Events (server/events.js): off unless EVENTS_URL and the client secret are set.
try { require('./events').init(require('./db').getDb()); } catch (err) { console.warn('[Events] not started:', err.message); }
try { require('./events').init(require('./db').getDb()); } catch (err) { console.warn('[Events] not started:', err.message); }
// The Roadmap space follows docs/roadmap/public.json (server/forum/roadmap.js).
// The Roadmap space follows docs/roadmap/public.json (server/forum/roadmap.js).
@@ -39,8 +44,9 @@ try { searchDocs = require('./search/documents').createSearchDocuments({ db: req
@@ -39,8 +44,9 @@ try { searchDocs = require('./search/documents').createSearchDocuments({ db: req
// SIGTERM: the Pulse subscription retries, the community.profile scan and the search-document scans
// SIGTERM: the Pulse subscription retries, the community.profile scan and the search-document scans
// stop (nothing new starts); the server stops taking connections, closes idle keep-alive ones (it keeps
// stop (nothing new starts); the server stops taking connections, closes idle keep-alive ones (it keeps
// them 65 s otherwise) and lets requests in flight finish (4 s at most); then the profile scan in
// them 65 s otherwise) and lets requests in flight finish (4 s at most); then the profile scan in
-// progress, the Discord relay's drain and the events outbox's send finish (unsent rows stay in their
-// tables for the next start), community.db closes, and the process exits 0, within the manifest's 5 s.
+// progress, the Discord relay's drain and its Events worker's page (the gateway closes at once) and the
+// events outbox's send finish (unsent rows stay in their tables for the next start), community.db
+// closes, and the process exits 0, within the manifest's 5 s.
const { within } = require('./graceful');
const { within } = require('./graceful');
let profilesDone = null;
let profilesDone = null;
require('./graceful').gracefulStop({
require('./graceful').gracefulStop({
@@ -56,7 +62,13 @@ require('./graceful').gracefulStop({
@@ -56,7 +62,13 @@ require('./graceful').gracefulStop({
() => within(1500, require('./events').stop()),
() => within(1500, require('./events').stop()),
() => {
() => {
const ev = require('./events').status();
const ev = require('./events').status();
- console.log(`[Community] stopped: subscriptions ${subscriptions ? 'stopped' : 'off'}, profile scan ${profiles && profiles.enabled ? 'stopped' : 'off'}, search scans ${searchDocs ? 'stopped' : 'off'}, relay ${config.discordRelay.enabled ? 'stopped' : 'off'}, outbox ${ev.enabled ? `stopped (${ev.pending} pending)` : 'off'}`);
+ let relay = 'off';
+ if (config.discordRelay.enabled) {
+ let rs = null;
+ try { rs = app.locals.relay.status(); } catch { /* the database is gone */ }
+ relay = rs ? `stopped (events worker ${rs.events_worker.enabled ? 'stopped' : 'off'}, gateway ${rs.inbound.enabled ? 'stopped' : 'off'})` : 'stopped';
+ }
+ console.log(`[Community] stopped: subscriptions ${subscriptions ? 'stopped' : 'off'}, profile scan ${profiles && profiles.enabled ? 'stopped' : 'off'}, search scans ${searchDocs ? 'stopped' : 'off'}, relay ${relay}, outbox ${ev.enabled ? `stopped (${ev.pending} pending)` : 'off'}`);
require('./db').closeDb();
require('./db').closeDb();
},
},
],
],
@@ -57,9 +57,10 @@ function createCommunityReadiness({ db, auth, config, relay = null, release = nu
@@ -57,9 +57,10 @@ function createCommunityReadiness({ db, auth, config, relay = null, release = nu
details: (body) => {
details: (body) => {
const out = { pastes_authority: config.pastesAuthority };
const out = { pastes_authority: config.pastesAuthority };
if (relay && relay.enabled && body.checks.db.status === 'ok') {
if (relay && relay.enabled && body.checks.db.status === 'ok') {
- const q = {};
- for (const r of db.prepare('SELECT status, COUNT(*) AS n FROM relay_deliveries GROUP BY status').all()) q[r.status] = r.n;
- out.discord_relay = { enabled: true, deliveries: q };
+ // The queue by status (failed = dead letters), and whether the Events worker and the inbound gateway run.
+ const st = relay.status();
+ const brief = (x) => (x.enabled ? { enabled: true, state: x.state || (x.running ? 'running' : 'stopped'), last_error: x.last_error || null, ...(x.lag != null ? { lag: x.lag } : {}) } : { enabled: false, reason: x.reason });
+ out.discord_relay = { enabled: true, deliveries: st.deliveries, creates_from: st.creates_from, events_worker: brief(st.events_worker), inbound: brief(st.inbound), inbound_failures: st.inbound_failures };
}
}
return out;
return out;
},
},
@@ -4,21 +4,29 @@
@@ -4,21 +4,29 @@
* /api/v1/relay — Discord relay administration, staff only (an admin/global_mod browser, or a
* /api/v1/relay — Discord relay administration, staff only (an admin/global_mod browser, or a
* service holding community.comment.moderate: acting as itself, or vouching with X-OV-Staff).
* service holding community.comment.moderate: acting as itself, or vouching with X-OV-Staff).
*
*
- * GET /deliveries?status=pending|delivered|failed&limit= what was sent, what is failing and why
- * POST /deliveries/:id/retry queue a failed delivery again
- * GET /mappings space → webhook variable name
- * POST /mappings { space, webhook_url_ref, enabled? } map a space (the URL stays in the env; allow-listed names only)
- * PUT /mappings/:id { enabled }
+ * GET /status the queue by status, the Events worker (cursor, lag,
+ * last error, gaps) and the inbound gateway's state
+ * GET /deliveries?status=pending|delivered|failed|dropped|skipped&action=create|edit|delete&limit=
+ * what was sent, what is failing and why (failed = the dead letters)
+ * POST /deliveries/:id/retry queue a delivery again, with a fresh attempt budget
+ * POST /deliveries/:id/drop give up on a waiting or failed one (kept as 'dropped')
+ * GET /inbound?all=1&limit= messages from Discord that did not become posts, and why
+ * POST /inbound/:id/dismiss mark one seen
+ * GET /mappings space → webhook variable name, Discord channel/thread, inbound
+ * POST /mappings { space, webhook_url_ref, enabled?, discord_channel_id?, discord_thread_id?, inbound? }
+ * map a space (the URL stays in the env; allow-listed names only)
+ * PUT /mappings/:id { enabled?, inbound?, discord_channel_id?, discord_thread_id? }
*
*
- * Responses never contain a webhook URL, only the variable name and whether it is set.
+ * Responses never contain a webhook URL or the bot token, only variable names and whether they are set.
*/
*/
const express = require('express');
const express = require('express');
const contracts = require('openvibe-contracts');
const contracts = require('openvibe-contracts');
const { run, jsonBody, fail, intIn } = require('../http/v1');
const { run, jsonBody, fail, intIn } = require('../http/v1');
const { discussionModerator } = require('../identity/capabilities');
const { discussionModerator } = require('../identity/capabilities');
const forumStore = require('../forum/store');
const forumStore = require('../forum/store');
+const { SNOWFLAKE, DELIVERY_STATUSES } = require('./discord');
-function createRelayApi({ relay, db, viewers }) {
+function createRelayApi({ relay, db, viewers, inbound = null }) {
const router = express.Router();
const router = express.Router();
router.use(contracts.http.middleware());
router.use(contracts.http.middleware());
router.use(viewers.middleware());
router.use(viewers.middleware());
@@ -31,15 +39,38 @@ function createRelayApi({ relay, db, viewers }) {
@@ -31,15 +39,38 @@ function createRelayApi({ relay, db, viewers }) {
});
});
const idOf = (req) => (/^\d{1,15}$/.test(req.params.id) ? Number(req.params.id) : fail(404, 'route.not_found', 'Not found'));
const idOf = (req) => (/^\d{1,15}$/.test(req.params.id) ? Number(req.params.id) : fail(404, 'route.not_found', 'Not found'));
+ /** Discord ids in a mapping body: digits, or null/'' to clear; undefined when absent. */
+ function discordIds(b) {
+ const out = {};
+ for (const k of ['discord_channel_id', 'discord_thread_id']) {
+ if (b[k] === undefined) continue;
+ if (b[k] !== null && b[k] !== '' && !SNOWFLAKE.test(String(b[k]))) fail(400, 'relay.invalid_discord_id', `${k} is a Discord id (15 to 21 digits), or null`);
+ out[k] = b[k] === '' ? null : b[k];
+ }
+ return out;
+ }
+ router.get('/status', run(() => relay.status()));
router.get('/deliveries', run((req) => {
router.get('/deliveries', run((req) => {
- const status = ['pending', 'delivered', 'failed'].includes(req.query.status) ? req.query.status : null;
- return { enabled: relay.enabled, deliveries: relay.listDeliveries({ status, limit: intIn(req.query.limit, 50, 1, 200) }) };
+ const status = DELIVERY_STATUSES.includes(req.query.status) ? req.query.status : null;
+ const action = ['create', 'edit', 'delete'].includes(req.query.action) ? req.query.action : null;
+ return { enabled: relay.enabled, deliveries: relay.listDeliveries({ status, action, limit: intIn(req.query.limit, 50, 1, 200) }) };
}));
}));
router.post('/deliveries/:id/retry', run((req) => {
router.post('/deliveries/:id/retry', run((req) => {
if (!relay.retry(idOf(req))) fail(404, 'relay.delivery_not_found', 'No such delivery waiting to be sent');
if (!relay.retry(idOf(req))) fail(404, 'relay.delivery_not_found', 'No such delivery waiting to be sent');
return { ok: true };
return { ok: true };
}));
}));
+ router.post('/deliveries/:id/drop', run((req) => {
+ if (!relay.drop(idOf(req))) fail(404, 'relay.delivery_not_found', 'No such pending or failed delivery');
+ return { ok: true };
+ }));
+ router.get('/inbound', run((req) => ({
+ enabled: !!inbound, failures: inbound ? inbound.listFailures({ all: req.query.all === '1', limit: intIn(req.query.limit, 50, 1, 200) }) : [],
+ })));
+ router.post('/inbound/:id/dismiss', run((req) => {
+ if (!inbound || !inbound.dismiss(idOf(req))) fail(404, 'relay.inbound_not_found', 'No such inbound failure to dismiss');
+ return { ok: true };
+ }));
router.get('/mappings', run(() => ({ enabled: relay.enabled, mappings: relay.listMappings() })));
router.get('/mappings', run(() => ({ enabled: relay.enabled, mappings: relay.listMappings() })));
router.post('/mappings', jsonBody, run((req) => {
router.post('/mappings', jsonBody, run((req) => {
const b = req.body || {};
const b = req.body || {};
@@ -48,10 +79,15 @@ function createRelayApi({ relay, db, viewers }) {
@@ -48,10 +79,15 @@ function createRelayApi({ relay, db, viewers }) {
const ref = String(b.webhook_url_ref || '');
const ref = String(b.webhook_url_ref || '');
if (!relay.ENV_NAME.test(ref)) fail(400, 'relay.invalid_ref', 'webhook_url_ref is the NAME of an environment variable (A-Z, 0-9, _), never the URL');
if (!relay.ENV_NAME.test(ref)) fail(400, 'relay.invalid_ref', 'webhook_url_ref is the NAME of an environment variable (A-Z, 0-9, _), never the URL');
if (!relay.refAllowed(ref)) fail(400, 'relay.ref_not_allowed', 'webhook_url_ref must be an allowed webhook variable (DISCORD_RELAY_WEBHOOK_VARS, else DISCORD_WEBHOOK_*)');
if (!relay.refAllowed(ref)) fail(400, 'relay.ref_not_allowed', 'webhook_url_ref must be an allowed webhook variable (DISCORD_RELAY_WEBHOOK_VARS, else DISCORD_WEBHOOK_*)');
- return { mapping: relay.addMapping({ space_id: space.id, webhook_url_ref: ref, enabled: b.enabled !== false }) };
+ const ids = discordIds(b);
+ return { mapping: relay.addMapping({ space_id: space.id, webhook_url_ref: ref, enabled: b.enabled !== false, ...ids, inbound: b.inbound === undefined ? undefined : !!b.inbound }) };
}, 201));
}, 201));
router.put('/mappings/:id', jsonBody, run((req) => {
router.put('/mappings/:id', jsonBody, run((req) => {
- const m = relay.setMappingEnabled(idOf(req), !!(req.body || {}).enabled);
+ const b = req.body || {};
+ const fields = { ...discordIds(b) };
+ if (b.enabled !== undefined) fields.enabled = !!b.enabled;
+ if (b.inbound !== undefined) fields.inbound = !!b.inbound;
+ const m = relay.updateMapping(idOf(req), fields);
if (!m) fail(404, 'relay.mapping_not_found', 'No such mapping');
if (!m) fail(404, 'relay.mapping_not_found', 'No such mapping');
return { mapping: m };
return { mapping: m };
}));
}));
new file mode 100644
new file mode 100644
@@ -0,0 +1,239 @@
@@ -0,0 +1,239 @@
+'use strict';
+
+/**
+ * A minimal Discord gateway client (API v10, JSON, no compression) for the relay's inbound side
+ * (roadmap WS-J task 6; DISCORD_RELAY_INBOUND=on with DISCORD_BOT_TOKEN).
+ *
+ * One session: HELLO → IDENTIFY (or RESUME on resume_gateway_url after a drop); heartbeats at the
+ * interval Discord asks for, the first one jittered, and a heartbeat that was not ACKed by the next
+ * one means a zombied connection (closed, then resumed); op 1 asks for a heartbeat now; op 7
+ * RECONNECT resumes; op 9 INVALID_SESSION identifies again (or resumes when Discord says it may).
+ * Reconnects back off exponentially (minBackoffMs · 2^n, at most maxBackoffMs), reset once READY or
+ * RESUMED. Close codes that reconnecting cannot heal (4004 a wrong token; 4010–4014 shard, version
+ * or intents) stop it, with the reason shown to staff; 4007 and 4009 start a fresh session.
+ *
+ * Intents: GUILD_MESSAGES | MESSAGE_CONTENT (privileged: turn it on for the bot in the Developer
+ * Portal). MESSAGE_CREATE / MESSAGE_UPDATE / MESSAGE_DELETE / MESSAGE_DELETE_BULK go to onDispatch
+ * (relay/inbound.js); everything else is dropped here. The WebSocket is the client built into
+ * Node 22 (no dependency); without one the gateway stays off and says so. The token never leaves
+ * the IDENTIFY/RESUME payloads and is never logged.
+ */
+const INTENTS = { GUILD_MESSAGES: 1 << 9, MESSAGE_CONTENT: 1 << 15 };
+const DEFAULT_URL = 'wss://gateway.discord.gg/?v=10&encoding=json';
+const FATAL = {
+ 4004: 'authentication failed: DISCORD_BOT_TOKEN is wrong',
+ 4010: 'invalid shard',
+ 4011: 'sharding required (the bot is in too many servers for one connection)',
+ 4012: 'invalid gateway API version',
+ 4013: 'invalid intents',
+ 4014: 'disallowed intents: turn on the MESSAGE CONTENT intent for the bot in the Discord Developer Portal',
+};
+const FRESH = new Set([4007, 4009]);
+const DISPATCH = new Set(['MESSAGE_CREATE', 'MESSAGE_UPDATE', 'MESSAGE_DELETE', 'MESSAGE_DELETE_BULK']);
+const OPEN = 1;
+
+/** The gateway URL with the version and encoding this client speaks (a resume URL comes without them). */
+function withQuery(u) {
+ try {
+ const url = new URL(u);
+ url.searchParams.set('v', '10');
+ url.searchParams.set('encoding', 'json');
+ return url.toString();
+ } catch { return u; }
+}
+
+function createDiscordGateway({ token = '', url = DEFAULT_URL, WebSocketImpl = globalThis.WebSocket, onDispatch = () => {}, intents = INTENTS.GUILD_MESSAGES | INTENTS.MESSAGE_CONTENT, minBackoffMs = 1000, maxBackoffMs = 60_000, helloTimeoutMs = 30_000, invalidSessionMs = 1000, jitter = Math.random, now = () => Date.now() } = {}) {
+ let reason = null;
+ if (!token) reason = 'DISCORD_BOT_TOKEN is not set';
+ else if (typeof WebSocketImpl !== 'function') reason = 'this Node has no built-in WebSocket client (Node 22 or later)';
+ const on = !reason;
+ const state = { status: on ? 'idle' : 'off', session_id: null, seq: null, resume_url: null, bot_user_id: null, last_error: null, last_reconnect: null, ready_at: null, connects: 0, reconnects: 0, dispatched: 0 };
+ let ws = null;
+ let firstBeat = null;
+ let beats = null;
+ let reconnectTimer = null;
+ let helloTimer = null;
+ let acked = true;
+ let failures = 0;
+ let stopped = true;
+
+ const iso = () => new Date(now()).toISOString();
+ function send(obj) {
+ if (ws && ws.readyState === OPEN) ws.send(JSON.stringify(obj));
+ }
+ function clearHeartbeat() {
+ if (helloTimer) clearTimeout(helloTimer);
+ helloTimer = null;
+ if (firstBeat) clearTimeout(firstBeat);
+ if (beats) clearInterval(beats);
+ firstBeat = null;
+ beats = null;
+ }
+ function beat() {
+ acked = false;
+ send({ op: 1, d: state.seq });
+ }
+ function heartbeatEvery(ms) {
+ clearHeartbeat();
+ acked = true;
+ const interval = Math.max(Number(ms) || 41_250, 10);
+ firstBeat = setTimeout(() => {
+ firstBeat = null;
+ beat();
+ beats = setInterval(() => {
+ if (!acked) return reconnect('no heartbeat ACK (zombied connection)');
+ beat();
+ }, interval);
+ if (beats.unref) beats.unref();
+ }, Math.floor(interval * jitter()));
+ if (firstBeat.unref) firstBeat.unref();
+ }
+
+ function connect() {
+ reconnectTimer = null;
+ if (stopped) return;
+ const resuming = !!(state.session_id && state.resume_url);
+ state.status = resuming ? 'resuming' : 'connecting';
+ state.connects++;
+ let sock;
+ try {
+ sock = new WebSocketImpl(resuming ? withQuery(state.resume_url) : url);
+ } catch (err) {
+ state.last_error = `cannot open the gateway: ${err.message}`;
+ return scheduleReconnect();
+ }
+ ws = sock;
+ helloTimer = setTimeout(() => { helloTimer = null; if (ws === sock) reconnect('no HELLO from the gateway'); }, helloTimeoutMs);
+ if (helloTimer.unref) helloTimer.unref();
+ sock.addEventListener('message', (ev) => { if (ws === sock) onMessage(ev.data); });
+ sock.addEventListener('close', (ev) => { if (ws === sock) onClose(ev.code, ev.reason); });
+ sock.addEventListener('error', () => { /* a close event follows */ });
+ }
+
+ function onMessage(raw) {
+ let p;
+ try { p = JSON.parse(typeof raw === 'string' ? raw : Buffer.from(raw).toString('utf8')); } catch { return; }
+ if (!p || typeof p !== 'object') return;
+ switch (p.op) {
+ case 10: // HELLO
+ heartbeatEvery(p.d && p.d.heartbeat_interval); // also clears the HELLO timeout
+ if (state.session_id && state.seq != null) send({ op: 6, d: { token, session_id: state.session_id, seq: state.seq } });
+ else send({ op: 2, d: { token, intents, properties: { os: process.platform, browser: 'openvibe-community', device: 'openvibe-community' } } });
+ break;
+ case 11: // HEARTBEAT_ACK
+ acked = true;
+ break;
+ case 1: // HEARTBEAT (asked for one now)
+ beat();
+ break;
+ case 7: // RECONNECT
+ reconnect('Discord asked to reconnect');
+ break;
+ case 9: // INVALID_SESSION (d: may resume)
+ if (!p.d) { state.session_id = null; state.seq = null; state.resume_url = null; }
+ reconnect('invalid session', invalidSessionMs + Math.floor(jitter() * 4 * invalidSessionMs)); // Discord: wait 1 to 5 s
+ break;
+ case 0: // DISPATCH
+ if (p.s != null) state.seq = p.s;
+ dispatch(p.t, p.d);
+ break;
+ default:
+ break;
+ }
+ }
+
+ function dispatch(t, d) {
+ if (t === 'READY') {
+ state.session_id = d && d.session_id ? String(d.session_id) : null;
+ state.resume_url = d && d.resume_gateway_url ? String(d.resume_gateway_url) : null;
+ state.bot_user_id = d && d.user && d.user.id ? String(d.user.id) : null;
+ state.status = 'ready';
+ state.ready_at = iso();
+ state.last_error = null;
+ failures = 0;
+ return;
+ }
+ if (t === 'RESUMED') {
+ state.status = 'ready';
+ state.last_error = null;
+ failures = 0;
+ return;
+ }
+ if (!DISPATCH.has(t)) return;
+ state.dispatched++;
+ try {
+ const r = onDispatch(t, d || {}, { botUserId: state.bot_user_id });
+ if (r && typeof r.catch === 'function') r.catch((err) => console.warn(`[Relay] inbound ${t} failed:`, err.message));
+ } catch (err) {
+ console.warn(`[Relay] inbound ${t} failed:`, err.message);
+ }
+ }
+
+ function onClose(code, why) {
+ clearHeartbeat();
+ ws = null;
+ if (stopped) { state.status = 'stopped'; return; }
+ if (FATAL[code]) {
+ state.status = 'failed';
+ state.last_error = `Discord closed the gateway (${code}): ${FATAL[code]}`;
+ console.warn(`[Relay] ${state.last_error}; inbound stays off until a restart`);
+ return;
+ }
+ if (FRESH.has(code)) { state.session_id = null; state.seq = null; state.resume_url = null; }
+ state.last_error = `gateway closed (${code}${why ? `: ${String(why).slice(0, 200)}` : ''})`;
+ state.last_reconnect = { reason: state.last_error, at: iso() };
+ scheduleReconnect();
+ }
+
+ /** Drop this connection (keeping the session, so the next one resumes) and connect again. */
+ function reconnect(why, delayMs = null) {
+ const sock = ws;
+ ws = null;
+ clearHeartbeat();
+ state.last_error = why;
+ state.last_reconnect = { reason: why, at: iso() };
+ if (sock) { try { sock.close(4000, why.slice(0, 120)); } catch { /* already closed */ } }
+ scheduleReconnect(delayMs);
+ }
+
+ function scheduleReconnect(delayMs = null) {
+ if (stopped || reconnectTimer) return;
+ failures++;
+ state.reconnects++;
+ state.status = 'waiting';
+ const delay = delayMs != null ? delayMs : Math.min(minBackoffMs * Math.pow(2, Math.max(failures - 1, 0)), maxBackoffMs);
+ reconnectTimer = setTimeout(connect, delay);
+ if (reconnectTimer.unref) reconnectTimer.unref();
+ }
+
+ function start() {
+ if (!on || !stopped) return;
+ stopped = false;
+ failures = 0;
+ connect();
+ }
+ /** Close the connection (1000) and cancel every timer. */
+ function stop() {
+ stopped = true;
+ clearHeartbeat();
+ if (reconnectTimer) clearTimeout(reconnectTimer);
+ reconnectTimer = null;
+ const sock = ws;
+ ws = null;
+ if (sock) { try { sock.close(1000, 'stopping'); } catch { /* already closed */ } }
+ if (on) state.status = 'stopped';
+ return Promise.resolve();
+ }
+
+ function status() {
+ if (!on) return { enabled: false, reason };
+ return {
+ enabled: true, state: state.status, session: !!state.session_id, bot_user_id: state.bot_user_id,
+ ready_at: state.ready_at, connects: state.connects, reconnects: state.reconnects, last_reconnect: state.last_reconnect, events_handled: state.dispatched, last_error: state.last_error,
+ };
+ }
+
+ return { enabled: on, reason, start, stop, status, _state: state };
+}
+
+module.exports = { createDiscordGateway, INTENTS, DEFAULT_URL, FATAL };
@@ -1,11 +1,23 @@
@@ -1,11 +1,23 @@
'use strict';
'use strict';
/**
/**
- * Discord relay, outbound only (DISCORD_RELAY_ENABLED; off by default).
+ * Discord relay (DISCORD_RELAY_ENABLED; off by default), roadmap WS-J tasks 5 and 6.
*
*
- * When a thread is created in a public space that has an enabled relay_mappings row, one
- * relay_deliveries row is created per (thread, mapping) — its dedupe key — and a worker posts a
- * message with source attribution and a link to the mapping's Discord webhook.
+ * Out: a new thread in a public space with an enabled relay_mappings row is announced through the
+ * mapping's Discord webhook; its replies follow (into the same channel, or the Discord thread the
+ * mapping names); edits and deletes follow them through the external message map (relay_message_map).
+ * In: with the gateway on, replies written on Discord come back as posts (server/relay/inbound.js).
+ *
+ * The queue is relay_deliveries, one row per dedupe key:
+ * create thread:<id>:mapping:<m> | post:<id>:mapping:<m> queued by the Events worker from
+ * community.thread.created / community.post.created (server/relay/events-worker.js), or by the
+ * forum itself while that worker is off
+ * edit edit:<thread|post>:<id>:mapping:<m>:r<revision> an edited opening post or reply that is on Discord
+ * delete delete:<thread|post>:<id>:mapping:<m> deleted, moderated away, or gated members-only
+ * A create is POSTed with ?wait=true, so Discord answers with the message and its id goes into the map;
+ * an edit or delete PATCHes or DELETEs /messages/<id> of the same webhook (?thread_id= when it was sent
+ * into a Discord thread). Nothing already in the map is created twice, and a reply waits for its
+ * thread's own message (a reply to a thread that is not on Discord through that mapping is skipped).
*
*
* - Secrets never live in the database: a mapping stores webhook_url_ref, the NAME of an
* - Secrets never live in the database: a mapping stores webhook_url_ref, the NAME of an
* environment variable (e.g. DISCORD_WEBHOOK_FEEDBACK); the URL is read from process.env at
* environment variable (e.g. DISCORD_WEBHOOK_FEEDBACK); the URL is read from process.env at
@@ -13,13 +25,17 @@
@@ -13,13 +25,17 @@
* DISCORD_WEBHOOK_*. Staff can therefore never point the relay at the URL in some other
* DISCORD_WEBHOOK_*. Staff can therefore never point the relay at the URL in some other
* variable (an internal service's base URL), nor learn which other variables are set; a
* variable (an internal service's base URL), nor learn which other variables are set; a
* mapping outside the list is refused when made and never sent.
* mapping outside the list is refused when made and never sent.
- * - Loop prevention: a thread whose origin is 'discord' (a future inbound relay) is never
- * relayed out, checked both when queueing and when sending.
+ * - Visibility: public spaces only, never members-only (VIP) spaces or threads, checked when
+ * queueing and again when sending. Gating a thread later deletes what the relay had sent.
+ * - Loop prevention: a thread or post whose origin is 'discord' is never relayed out, checked both
+ * when queueing and when sending (inbound.js adds the other half).
* - Retries: network errors, timeouts, 5xx, 429 and a missing webhook variable are retried with
* - Retries: network errors, timeouts, 5xx, 429 and a missing webhook variable are retried with
* exponential backoff (baseMs · 2^(attempt−1), capped at an hour; a 429's retry_after is
* exponential backoff (baseMs · 2^(attempt−1), capped at an hour; a 429's retry_after is
- * honoured) up to maxAttempts, then the delivery is 'failed'. Other 4xx answers fail at once.
- * - Mentions are disabled (allowed_mentions.parse = []), so a title can never ping anyone.
- * Failures are visible to staff at GET /api/v1/relay/deliveries.
+ * honoured, and the rest of that webhook's queue waits for the next pass) up to maxAttempts.
+ * Then the delivery is 'failed': the dead letter, with its error, that staff retry or drop.
+ * Other 4xx fail at once. A delete that finds the message already gone counts as delivered.
+ * - Mentions are disabled (allowed_mentions.parse = []), so nothing relayed can ping anyone.
+ * Staff see and act on all of it at /api/v1/relay (relay/api.js).
*/
*/
const pasteStore = require('../pastes/store');
const pasteStore = require('../pastes/store');
const forumStore = require('../forum/store');
const forumStore = require('../forum/store');
@@ -29,8 +45,12 @@ const { AI_DISPLAY_NAME } = require('../identity/authors');
@@ -29,8 +45,12 @@ const { AI_DISPLAY_NAME } = require('../identity/authors');
const ENV_NAME = /^[A-Z][A-Z0-9_]{1,63}$/;
const ENV_NAME = /^[A-Z][A-Z0-9_]{1,63}$/;
const WEBHOOK_VAR = /^DISCORD_WEBHOOK_[A-Z0-9_]+$/;
const WEBHOOK_VAR = /^DISCORD_WEBHOOK_[A-Z0-9_]+$/;
+const SNOWFLAKE = /^\d{15,21}$/;
const MAX_BACKOFF_MS = 60 * 60_000;
const MAX_BACKOFF_MS = 60 * 60_000;
const SEND_TIMEOUT_MS = 10_000;
const SEND_TIMEOUT_MS = 10_000;
+const POSTS_PER_PAGE = 50; // forum/service.js: a reply's link names its page
+const UNKNOWN_MESSAGE = 10008; // Discord's error code: the message is gone
+const DELIVERY_STATUSES = ['pending', 'delivered', 'failed', 'dropped', 'skipped'];
/** Discord markdown in names/titles shown as plain text. */
/** Discord markdown in names/titles shown as plain text. */
const escapeDiscord = (s) => String(s || '').replace(/([\\*_~`|>#[\]])/g, '\\$1').replace(/@/g, '@\u200b');
const escapeDiscord = (s) => String(s || '').replace(/([\\*_~`|>#[\]])/g, '\\$1').replace(/@/g, '@\u200b');
@@ -44,38 +64,145 @@ function createDiscordRelay({ db, config = {}, env = process.env, fetchImpl = gl
@@ -44,38 +64,145 @@ function createDiscordRelay({ db, config = {}, env = process.env, fetchImpl = gl
let timer = null;
let timer = null;
let draining = null;
let draining = null;
let stopped = false;
let stopped = false;
+ // While the Events worker runs it queues every create from the event stream; the forum's direct
+ // calls then queue nothing (the dedupe keys would make them harmless anyway).
+ let createsFromEvents = false;
+ const parts = { worker: null, gateway: null, inbound: null };
+
+ const keys = {
+ create: (type, id, m) => `${type}:${id}:mapping:${m}`,
+ edit: (type, id, m, rev) => `edit:${type}:${id}:mapping:${m}:r${rev}`,
+ remove: (type, id, m) => `delete:${type}:${id}:mapping:${m}`,
+ };
+ /** Kept for callers of the first relay: the create key of a thread. */
+ function deliveryKey(threadId, mappingId) { return keys.create('thread', threadId, mappingId); }
+
+ const insertStmt = db.prepare(`INSERT INTO relay_deliveries (thread_id, post_id, mapping_id, action, dedupe_key, source, event_id, next_attempt_at)
+ VALUES (@thread_id, @post_id, @mapping_id, @action, @key, @source, @event_id, @at) ON CONFLICT(dedupe_key) DO NOTHING`);
+ function insert(row) {
+ return insertStmt.run({ post_id: null, source: 'direct', event_id: null, ...row, at: sqlTime(now()) }).changes;
+ }
+ const outMappings = (spaceId) => db.prepare("SELECT * FROM relay_mappings WHERE space_id = ? AND direction = 'out' AND enabled = 1 ORDER BY id").all(spaceId);
+ const mapRow = (mappingId, type, id) => db.prepare("SELECT * FROM relay_message_map WHERE platform = 'discord' AND mapping_id = ? AND local_type = ? AND local_id = ?").get(mappingId, type, id) || null;
- function deliveryKey(threadId, mappingId) { return `thread:${threadId}:mapping:${mappingId}`; }
+ /** Why this thread (and a post of it) may not leave the site, or null. */
+ function refusal(thread, space, post = null) {
+ if (thread.origin === 'discord' || (post && post.origin === 'discord')) return `loop prevention: ${post && post.origin === 'discord' ? 'post' : 'thread'} came from Discord`;
+ if (space.visibility !== 'public' || space.members_only_owner || thread.members_only_owner) return 'the thread is no longer public (members-only or a restricted space)';
+ return null;
+ }
+ /** Queue-time filter: the same rules, quietly. */
+ const mayLeave = (thread, space, post = null) => !!(thread && space && !thread.deleted_at && !(post && post.deleted_at) && !refusal(thread, space, post));
- /** Queue a new thread for every enabled out-mapping of its space. → number of deliveries queued */
- function enqueueThread(thread, space) {
+ // ── queueing ─────────────────────────────────────────────
+ /**
+ * A new thread, for every enabled out-mapping of its space. source 'events' (the Events worker,
+ * eventId) or 'direct' (the forum; ignored while the worker queues creates). → number queued
+ */
+ function enqueueThread(thread, space, { source = 'direct', eventId = null } = {}) {
if (!enabled || !thread || !space) return 0;
if (!enabled || !thread || !space) return 0;
- if (thread.origin === 'discord') return 0; // loop prevention
- if (space.visibility !== 'public') return 0; // members/staff spaces never leave the site
- if (space.members_only_owner || thread.members_only_owner) return 0; // nor do VIP members-only ones
- const mappings = db.prepare("SELECT * FROM relay_mappings WHERE space_id = ? AND direction = 'out' AND enabled = 1").all(space.id);
+ if (source === 'direct' && createsFromEvents) return 0;
+ if (!mayLeave(thread, space)) return 0;
+ let queued = 0;
+ for (const m of outMappings(space.id)) {
+ queued += insert({ thread_id: thread.id, mapping_id: m.id, action: 'create', key: keys.create('thread', thread.id, m.id), source, event_id: eventId });
+ }
+ if (queued) kick();
+ return queued;
+ }
+
+ /**
+ * A new reply: to every mapping its thread went to (sent, or still waiting to be). The opening post
+ * travels with its thread. → number queued
+ */
+ function enqueuePost(post, thread, space, { source = 'direct', eventId = null } = {}) {
+ if (!enabled || !post || !thread || !space || post.is_opening) return 0;
+ if (source === 'direct' && createsFromEvents) return 0;
+ if (!mayLeave(thread, space, post)) return 0;
let queued = 0;
let queued = 0;
- const insert = db.prepare(`INSERT INTO relay_deliveries (thread_id, mapping_id, dedupe_key, next_attempt_at) VALUES (?, ?, ?, ?)
- ON CONFLICT(dedupe_key) DO NOTHING`);
- for (const m of mappings) queued += insert.run(thread.id, m.id, deliveryKey(thread.id, m.id), sqlTime(now())).changes;
+ for (const m of outMappings(space.id)) {
+ const parent = db.prepare("SELECT status FROM relay_deliveries WHERE dedupe_key = ? AND status IN ('pending', 'delivered')").get(keys.create('thread', thread.id, m.id));
+ if (!parent && !mapRow(m.id, 'thread', thread.id)) continue;
+ queued += insert({ thread_id: thread.id, post_id: post.id, mapping_id: m.id, action: 'create', key: keys.create('post', post.id, m.id), source, event_id: eventId });
+ }
if (queued) kick();
if (queued) kick();
return queued;
return queued;
}
}
+ /**
+ * An edited post: the Discord message showing it (the thread's, for an opening post) is edited.
+ * Only what is already on Discord; a create still waiting carries the new text anyway.
+ */
+ function enqueueEdit(post) {
+ if (!enabled || !post) return 0;
+ const type = post.is_opening ? 'thread' : 'post';
+ const id = post.is_opening ? post.thread_id : post.id;
+ const rows = db.prepare("SELECT * FROM relay_message_map WHERE platform = 'discord' AND direction = 'out' AND local_type = ? AND local_id = ? AND external_deleted_at IS NULL").all(type, id);
+ let queued = 0;
+ for (const r of rows) {
+ queued += insert({ thread_id: post.thread_id, post_id: post.is_opening ? null : post.id, mapping_id: r.mapping_id, action: 'edit', key: keys.edit(type, id, r.mapping_id, post.revision) });
+ }
+ if (queued) kick();
+ return queued;
+ }
+
+ /**
+ * A thread (postId null: with every reply of it) or one reply left the public site — deleted,
+ * moderated away or gated members-only: what is still waiting is skipped and what Discord shows is
+ * deleted. Queued even while the relay is off, so it happens once it is on again.
+ */
+ function enqueueDelete(threadId, postId = null, reason = 'deleted') {
+ const note = `skipped: ${reason} before it was sent`;
+ return db.transaction(() => {
+ if (postId == null) {
+ db.prepare("UPDATE relay_deliveries SET status = 'skipped', last_error = ?, updated_at = CURRENT_TIMESTAMP WHERE thread_id = ? AND status = 'pending' AND action <> 'delete'").run(note, threadId);
+ } else {
+ db.prepare("UPDATE relay_deliveries SET status = 'skipped', last_error = ?, updated_at = CURRENT_TIMESTAMP WHERE post_id = ? AND status = 'pending' AND action <> 'delete'").run(note, postId);
+ }
+ const rows = postId == null
+ ? db.prepare("SELECT * FROM relay_message_map WHERE thread_id = ? AND direction = 'out' AND external_deleted_at IS NULL").all(threadId)
+ : db.prepare("SELECT * FROM relay_message_map WHERE local_type = 'post' AND local_id = ? AND direction = 'out' AND external_deleted_at IS NULL").all(postId);
+ let queued = 0;
+ for (const r of rows) {
+ queued += insert({ thread_id: r.thread_id, post_id: r.local_type === 'post' ? r.local_id : null, mapping_id: r.mapping_id, action: 'delete', key: keys.remove(r.local_type, r.local_id, r.mapping_id) });
+ }
+ if (queued && enabled) kick();
+ return queued;
+ })();
+ }
+ /** Threads gated members-only (or otherwise no longer public): take them off Discord. */
+ function hideThreads(threadIds, reason = 'made members-only') {
+ let n = 0;
+ for (const id of threadIds) n += enqueueDelete(id, null, reason);
+ return n;
+ }
+
function kick() { setImmediate(() => { drain().catch((err) => console.warn('[Relay] drain failed:', err.message)); }); }
function kick() { setImmediate(() => { drain().catch((err) => console.warn('[Relay] drain failed:', err.message)); }); }
+ // ── messages ─────────────────────────────────────────────
+ function authorName(subject, origin) {
+ if (origin === 'ai') return AI_DISPLAY_NAME;
+ if (origin === 'system' && !subject) return 'OpenVibe';
+ if (subject) {
+ const p = pasteStore.getProjections(db, [subject]).get(subject);
+ if (p) return p.display_name || p.username || 'someone';
+ }
+ return 'someone';
+ }
+ const threadUrl = (thread, space) => `${base}/s/${space.slug}/t/${thread.slug}`;
+ function postUrl(post, thread, space) {
+ const position = db.prepare('SELECT COUNT(*) AS c FROM posts WHERE thread_id = ? AND id <= ?').get(thread.id, post.id).c;
+ const page = Math.max(Math.ceil(position / POSTS_PER_PAGE), 1);
+ return `${threadUrl(thread, space)}${page > 1 ? `?page=${page}` : ''}#post-${post.id}`;
+ }
+
+ /** The thread's message: who started it, the title, an excerpt of the opening post, the link. */
function message(thread, space) {
function message(thread, space) {
const opening = db.prepare('SELECT body_markdown FROM posts WHERE thread_id = ? AND is_opening = 1').get(thread.id);
const opening = db.prepare('SELECT body_markdown FROM posts WHERE thread_id = ? AND is_opening = 1').get(thread.id);
- let who = 'someone';
- if (thread.origin === 'ai') who = AI_DISPLAY_NAME;
- else if (thread.author_subject) {
- const p = pasteStore.getProjections(db, [thread.author_subject]).get(thread.author_subject);
- if (p) who = p.display_name || p.username || who;
- }
- const url = `${base}/s/${space.slug}/t/${thread.slug}`;
+ const url = threadUrl(thread, space);
return {
return {
username: 'OpenVibe.Community',
username: 'OpenVibe.Community',
- content: `New thread in **s/${escapeDiscord(space.slug)}** by ${escapeDiscord(who)}: <${url}>`,
+ content: `New thread in **s/${escapeDiscord(space.slug)}** by ${escapeDiscord(authorName(thread.author_subject, thread.origin))}: <${url}>`,
allowed_mentions: { parse: [] },
allowed_mentions: { parse: [] },
embeds: [{
embeds: [{
title: String(thread.title).slice(0, 256),
title: String(thread.title).slice(0, 256),
@@ -87,16 +214,36 @@ function createDiscordRelay({ db, config = {}, env = process.env, fetchImpl = gl
@@ -87,16 +214,36 @@ function createDiscordRelay({ db, config = {}, env = process.env, fetchImpl = gl
};
};
}
}
+ /** A reply's message: who replied to which thread, an excerpt, the link to the post. */
+ function postMessage(post, thread, space) {
+ const url = postUrl(post, thread, space);
+ const title = String(thread.title).slice(0, 200);
+ return {
+ username: 'OpenVibe.Community',
+ content: `${escapeDiscord(authorName(post.author_subject, post.origin))} replied to **${escapeDiscord(title)}** in **s/${escapeDiscord(space.slug)}**: <${url}>`,
+ allowed_mentions: { parse: [] },
+ embeds: [{
+ title: `Re: ${title}`.slice(0, 256),
+ url,
+ description: markdownToText(post.body_markdown, 500),
+ footer: { text: `OpenVibe.Community · s/${space.slug}` },
+ timestamp: isoTime(post.created_at) || undefined,
+ }],
+ };
+ }
+
+ // ── sending ──────────────────────────────────────────────
function backoff(attempts, retryAfterSec) {
function backoff(attempts, retryAfterSec) {
const exp = Math.min(baseMs * Math.pow(2, Math.max(attempts - 1, 0)), MAX_BACKOFF_MS);
const exp = Math.min(baseMs * Math.pow(2, Math.max(attempts - 1, 0)), MAX_BACKOFF_MS);
return Math.max(exp, retryAfterSec ? retryAfterSec * 1000 : 0);
return Math.max(exp, retryAfterSec ? retryAfterSec * 1000 : 0);
}
}
- function record(d, { ok, status = null, error = null, retry = false, retryAfter = 0 }) {
+ /** One send happened: delivered, retried later, or failed (the dead letter). */
+ function record(d, { ok, status = null, error = null, retry = false, retryAfter = 0, note = null }) {
const attempts = d.attempts + 1;
const attempts = d.attempts + 1;
if (ok) {
if (ok) {
- db.prepare("UPDATE relay_deliveries SET status = 'delivered', attempts = ?, last_status = ?, last_error = NULL, delivered_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP WHERE id = ?")
- .run(attempts, status, d.id);
+ db.prepare("UPDATE relay_deliveries SET status = 'delivered', attempts = ?, last_status = ?, last_error = ?, delivered_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP WHERE id = ?")
+ .run(attempts, status, note, d.id);
return 'delivered';
return 'delivered';
}
}
const giveUp = !retry || attempts >= maxAttempts;
const giveUp = !retry || attempts >= maxAttempts;
@@ -104,97 +251,256 @@ function createDiscordRelay({ db, config = {}, env = process.env, fetchImpl = gl
@@ -104,97 +251,256 @@ function createDiscordRelay({ db, config = {}, env = process.env, fetchImpl = gl
.run(giveUp ? 'failed' : 'pending', attempts, status, String(error || '').slice(0, 500), sqlTime(now() + (giveUp ? 0 : backoff(attempts, retryAfter))), d.id);
.run(giveUp ? 'failed' : 'pending', attempts, status, String(error || '').slice(0, 500), sqlTime(now() + (giveUp ? 0 : backoff(attempts, retryAfter))), d.id);
return giveUp ? 'failed' : 'retry';
return giveUp ? 'failed' : 'retry';
}
}
+ /** Nothing was sent: already done (delivered) or nothing left to do (skipped). */
+ function settle(d, status, reason) {
+ db.prepare(`UPDATE relay_deliveries SET status = ?, last_error = ?, updated_at = CURRENT_TIMESTAMP,
+ delivered_at = CASE WHEN ? = 'delivered' THEN CURRENT_TIMESTAMP ELSE delivered_at END WHERE id = ?`).run(status, reason, status, d.id);
+ return status === 'delivered' ? 'delivered' : 'skipped';
+ }
+ /** Not yet (a reply waiting for its thread's message): later, without using an attempt. */
+ function defer(d, untilSql, reason) {
+ const until = Math.max(Date.parse(`${String(untilSql || '').replace(' ', 'T')}Z`) || 0, now() + 1000);
+ db.prepare('UPDATE relay_deliveries SET next_attempt_at = ?, last_error = ?, updated_at = CURRENT_TIMESTAMP WHERE id = ?').run(sqlTime(until), reason, d.id);
+ return 'retry';
+ }
async function send(d) {
async function send(d) {
+ const m = db.prepare('SELECT * FROM relay_mappings WHERE id = ?').get(d.mapping_id);
const thread = db.prepare('SELECT * FROM threads WHERE id = ?').get(d.thread_id);
const thread = db.prepare('SELECT * FROM threads WHERE id = ?').get(d.thread_id);
const space = thread ? forumStore.getSpaceById(db, thread.space_id) : null;
const space = thread ? forumStore.getSpaceById(db, thread.space_id) : null;
- if (!thread || thread.deleted_at || !space) return record(d, { ok: false, error: 'thread is gone' });
- if (thread.origin === 'discord') return record(d, { ok: false, error: 'loop prevention: thread came from Discord' });
- if (space.visibility !== 'public' || space.members_only_owner || thread.members_only_owner) return record(d, { ok: false, error: 'the thread is no longer public (members-only or a restricted space)' });
- if (!refAllowed(d.webhook_url_ref)) return record(d, { ok: false, error: 'webhook_url_ref is not an allowed webhook variable' });
- const url = env[d.webhook_url_ref];
- if (!url) return record(d, { ok: false, retry: true, error: `webhook URL variable ${d.webhook_url_ref} is not set` });
- if (!/^https?:\/\//i.test(url)) return record(d, { ok: false, error: `${d.webhook_url_ref} is not an http(s) URL` });
+ const post = d.post_id ? forumStore.getPost(db, d.post_id) : null;
+ const type = d.post_id ? 'post' : 'thread';
+ const localId = d.post_id || d.thread_id;
+ const gone = !thread || thread.deleted_at || !space || (d.post_id && (!post || post.deleted_at));
+ const mapped = mapRow(d.mapping_id, type, localId);
+
+ if (d.action === 'create') {
+ if (gone) return settle(d, 'skipped', `the ${type} was deleted before it was sent`);
+ const why = refusal(thread, space, post);
+ if (why) return record(d, { ok: false, error: why });
+ if (mapped) return settle(d, 'delivered', 'already on Discord');
+ if (type === 'post' && !mapRow(d.mapping_id, 'thread', thread.id)) {
+ const parent = db.prepare('SELECT status, next_attempt_at FROM relay_deliveries WHERE dedupe_key = ?').get(keys.create('thread', thread.id, d.mapping_id));
+ if (parent && parent.status === 'pending') return defer(d, parent.next_attempt_at, "waiting for the thread's own message");
+ return settle(d, 'skipped', 'the thread is not on Discord through this mapping');
+ }
+ } else {
+ if (!mapped || mapped.direction !== 'out') return settle(d, 'skipped', 'not on Discord');
+ if (mapped.external_deleted_at) return settle(d, 'skipped', 'already deleted on Discord');
+ if (d.action === 'edit') {
+ if (gone) return settle(d, 'skipped', `the ${type} was deleted`);
+ const why = refusal(thread, space, post);
+ if (why) return settle(d, 'skipped', why);
+ }
+ }
+
+ if (!m || !refAllowed(m.webhook_url_ref)) return record(d, { ok: false, error: 'webhook_url_ref is not an allowed webhook variable' });
+ const raw = env[m.webhook_url_ref];
+ if (!raw) return record(d, { ok: false, retry: true, error: `webhook URL variable ${m.webhook_url_ref} is not set` });
+ let url = null;
+ try { url = new URL(raw); } catch { /* below */ }
+ if (!url || !/^https?:$/.test(url.protocol)) return record(d, { ok: false, error: `${m.webhook_url_ref} is not an http(s) URL` });
+
+ let method = 'POST';
+ let body = null;
+ let threadTarget = null;
+ if (d.action === 'create') {
+ threadTarget = (SNOWFLAKE.test(m.discord_thread_id || '') ? m.discord_thread_id : null) || url.searchParams.get('thread_id') || null;
+ url.searchParams.set('wait', 'true');
+ if (threadTarget) url.searchParams.set('thread_id', threadTarget);
+ body = type === 'thread' ? message(thread, space) : postMessage(post, thread, space);
+ } else {
+ method = d.action === 'edit' ? 'PATCH' : 'DELETE';
+ url.pathname = `${url.pathname.replace(/\/+$/, '')}/messages/${mapped.external_message_id}`;
+ url.search = '';
+ if (mapped.external_thread_id) url.searchParams.set('thread_id', mapped.external_thread_id);
+ if (d.action === 'edit') body = type === 'thread' ? message(thread, space) : postMessage(post, thread, space);
+ }
+
let res;
let res;
try {
try {
- res = await fetchImpl(url, {
- method: 'POST',
- headers: { 'Content-Type': 'application/json', 'User-Agent': 'OpenVibe.Community relay' },
- body: JSON.stringify(message(thread, space)),
+ res = await fetchImpl(url.toString(), {
+ method,
+ headers: { ...(body ? { 'Content-Type': 'application/json' } : {}), 'User-Agent': 'OpenVibe.Community relay' },
+ body: body ? JSON.stringify(body) : undefined,
signal: AbortSignal.timeout(SEND_TIMEOUT_MS),
signal: AbortSignal.timeout(SEND_TIMEOUT_MS),
});
});
} catch (err) {
} catch (err) {
return record(d, { ok: false, retry: true, error: err.name === 'TimeoutError' ? 'timed out' : err.message });
return record(d, { ok: false, retry: true, error: err.name === 'TimeoutError' ? 'timed out' : err.message });
}
}
- if (res.ok) return record(d, { ok: true, status: res.status });
- let detail = '';
+ let answer = null;
+ if (res.status !== 204) { try { answer = await res.json(); } catch { /* no JSON */ } }
+ if (res.ok) {
+ if (d.action === 'create') return created(d, res.status, answer, { type, localId, threadTarget, mapping: m });
+ if (d.action === 'delete') db.prepare('UPDATE relay_message_map SET external_deleted_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP WHERE id = ?').run(mapped.id);
+ else db.prepare('UPDATE relay_message_map SET updated_at = CURRENT_TIMESTAMP WHERE id = ?').run(mapped.id);
+ return record(d, { ok: true, status: res.status });
+ }
+ // The message itself is gone on Discord (someone deleted it there): a delete is done, an edit has nothing to edit.
+ if (res.status === 404 && d.action !== 'create' && answer && Number(answer.code) === UNKNOWN_MESSAGE) {
+ db.prepare('UPDATE relay_message_map SET external_deleted_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP WHERE id = ?').run(mapped.id);
+ return d.action === 'delete' ? record(d, { ok: true, status: 404, note: 'the message was already gone on Discord' }) : settle(d, 'skipped', 'the message is gone on Discord');
+ }
+ const detail = answer && (answer.message || answer.error) ? String(answer.message || answer.error) : '';
let retryAfter = Number(res.headers.get('retry-after')) || 0;
let retryAfter = Number(res.headers.get('retry-after')) || 0;
- try { const j = await res.json(); detail = j && (j.message || j.error) ? String(j.message || j.error) : ''; if (j && j.retry_after) retryAfter = Math.max(retryAfter, Number(j.retry_after) || 0); } catch { /* not JSON */ }
+ if (answer && answer.retry_after) retryAfter = Math.max(retryAfter, Number(answer.retry_after) || 0);
const retry = res.status === 429 || res.status === 408 || res.status >= 500;
const retry = res.status === 429 || res.status === 408 || res.status >= 500;
- return record(d, { ok: false, retry, retryAfter, status: res.status, error: `Discord answered ${res.status}${detail ? `: ${detail}` : ''}` });
+ const outcome = record(d, { ok: false, retry, retryAfter, status: res.status, error: `Discord answered ${res.status}${detail ? `: ${detail}` : ''}` });
+ return res.status === 429 ? { outcome, rateLimited: true } : outcome;
+ }
+
+ /** A create was answered: map the message (both ways unique) and learn the mapping's channel. */
+ function created(d, status, msg, { type, localId, threadTarget, mapping }) {
+ const id = msg && SNOWFLAKE.test(String(msg.id || '')) ? String(msg.id) : null;
+ const channel = msg && SNOWFLAKE.test(String(msg.channel_id || '')) ? String(msg.channel_id) : null;
+ if (!id || !channel) return record(d, { ok: true, status, note: 'Discord did not answer with the message, so edits and deletes cannot follow it' });
+ return db.transaction(() => {
+ db.prepare(`INSERT INTO relay_message_map (platform, mapping_id, direction, local_type, local_id, thread_id, external_channel_id, external_thread_id, external_message_id, external_webhook_id)
+ VALUES ('discord', ?, 'out', ?, ?, ?, ?, ?, ?, ?) ON CONFLICT DO NOTHING`)
+ .run(mapping.id, type, localId, d.thread_id, channel, threadTarget ? channel : null, id, msg.webhook_id ? String(msg.webhook_id) : null);
+ if (!threadTarget) db.prepare('UPDATE relay_mappings SET discord_channel_id = ? WHERE id = ? AND discord_channel_id IS NULL').run(channel, mapping.id);
+ return record(d, { ok: true, status });
+ })();
}
}
- /** Send every due delivery once. One drain at a time. → { delivered, retry, failed } */
+ /**
+ * Send every due delivery once, oldest first. One drain at a time. A 429 leaves the rest of that
+ * webhook's queue for the next pass. → { delivered, retry, failed, skipped }
+ */
function drain() {
function drain() {
if (draining) return draining;
if (draining) return draining;
- if (stopped) return Promise.resolve({ delivered: 0, retry: 0, failed: 0 });
+ if (stopped) return Promise.resolve({ delivered: 0, retry: 0, failed: 0, skipped: 0 });
draining = (async () => {
draining = (async () => {
- const summary = { delivered: 0, retry: 0, failed: 0 };
+ const summary = { delivered: 0, retry: 0, failed: 0, skipped: 0 };
if (!enabled) return summary;
if (!enabled) return summary;
- const due = db.prepare(`SELECT d.*, m.webhook_url_ref FROM relay_deliveries d JOIN relay_mappings m ON m.id = d.mapping_id
- WHERE d.status = 'pending' AND m.enabled = 1 AND d.next_attempt_at <= ? ORDER BY d.id ASC LIMIT 50`).all(sqlTime(now()));
- for (const d of due) summary[await send(d)]++;
+ // Deletes go out even for a disabled mapping: what left the site leaves Discord too.
+ const due = db.prepare(`SELECT d.* FROM relay_deliveries d JOIN relay_mappings m ON m.id = d.mapping_id
+ WHERE d.status = 'pending' AND (m.enabled = 1 OR d.action = 'delete') AND d.next_attempt_at <= ?
+ ORDER BY d.id ASC LIMIT 50`).all(sqlTime(now()));
+ const paused = new Set();
+ for (const d of due) {
+ if (stopped) break;
+ if (paused.has(d.mapping_id)) continue;
+ const r = await send(d);
+ if (r && typeof r === 'object') { paused.add(d.mapping_id); summary[r.outcome]++; } else summary[r]++;
+ }
return summary;
return summary;
})().finally(() => { draining = null; });
})().finally(() => { draining = null; });
return draining;
return draining;
}
}
+ // ── lifecycle ────────────────────────────────────────────
+ /** The Events worker, the gateway and the inbound handler (app.js), started and stopped with the relay. */
+ function attach({ worker = null, gateway = null, inbound = null } = {}) { Object.assign(parts, { worker, gateway, inbound }); }
+ /** The Events worker says it queues creates (true) or no longer does. */
+ function useEventsForCreates(on) { createsFromEvents = !!on; }
+
function start() {
function start() {
if (!enabled || timer) return;
if (!enabled || timer) return;
stopped = false;
stopped = false;
timer = setInterval(() => { drain().catch((err) => console.warn('[Relay] drain failed:', err.message)); }, pollMs);
timer = setInterval(() => { drain().catch((err) => console.warn('[Relay] drain failed:', err.message)); }, pollMs);
if (timer.unref) timer.unref();
if (timer.unref) timer.unref();
kick();
kick();
+ if (parts.worker) parts.worker.start();
+ if (parts.gateway) parts.gateway.start();
+ }
+ /** Graceful stop: no further drains or reads; resolves when the drain and page in progress finished (the rest stay queued). */
+ function stop() {
+ stopped = true;
+ if (timer) clearInterval(timer);
+ timer = null;
+ const waits = [draining || Promise.resolve()];
+ if (parts.worker) waits.push(parts.worker.stop());
+ if (parts.gateway) waits.push(parts.gateway.stop());
+ return Promise.all(waits).then(() => undefined);
}
}
- /** Graceful stop: no further drains; resolves when the drain in progress has finished (the rest stay pending). */
- function stop() { stopped = true; if (timer) clearInterval(timer); timer = null; return draining || Promise.resolve(); }
// ── admin (staff) ────────────────────────────────────────
// ── admin (staff) ────────────────────────────────────────
function shapeMapping(m) {
function shapeMapping(m) {
- return { id: m.id, space: m.space_slug, direction: m.direction, webhook_url_ref: m.webhook_url_ref, webhook_configured: refAllowed(m.webhook_url_ref) && !!env[m.webhook_url_ref], enabled: !!m.enabled, created_at: isoTime(m.created_at) };
- }
- function listMappings() {
- return db.prepare('SELECT m.*, s.slug AS space_slug FROM relay_mappings m JOIN spaces s ON s.id = m.space_id ORDER BY m.id').all().map(shapeMapping);
+ return {
+ id: m.id, space: m.space_slug, direction: m.direction, webhook_url_ref: m.webhook_url_ref, webhook_configured: refAllowed(m.webhook_url_ref) && !!env[m.webhook_url_ref],
+ enabled: !!m.enabled, discord_channel_id: m.discord_channel_id || null, discord_thread_id: m.discord_thread_id || null, inbound: !!m.inbound, created_at: isoTime(m.created_at),
+ };
}
}
- function addMapping({ space_id, webhook_url_ref, enabled: on = true }) {
+ const mappingQuery = 'SELECT m.*, s.slug AS space_slug FROM relay_mappings m JOIN spaces s ON s.id = m.space_id';
+ function listMappings() { return db.prepare(`${mappingQuery} ORDER BY m.id`).all().map(shapeMapping); }
+ const snowflakeOrNull = (v, name) => {
+ if (v === undefined) return undefined;
+ if (v === null || v === '') return null;
+ if (!SNOWFLAKE.test(String(v))) throw new Error(`${name} is a Discord id (digits)`);
+ return String(v);
+ };
+ /** { space_id, webhook_url_ref, enabled?, discord_channel_id?, discord_thread_id?, inbound? } → mapping (made, or updated) */
+ function addMapping({ space_id, webhook_url_ref, enabled: on = true, discord_channel_id, discord_thread_id, inbound }) {
if (!refAllowed(webhook_url_ref)) throw new Error(`${webhook_url_ref} is not an allowed webhook variable`);
if (!refAllowed(webhook_url_ref)) throw new Error(`${webhook_url_ref} is not an allowed webhook variable`);
- db.prepare(`INSERT INTO relay_mappings (space_id, direction, webhook_url_ref, enabled) VALUES (?, 'out', ?, ?)
- ON CONFLICT(space_id, direction, webhook_url_ref) DO UPDATE SET enabled = excluded.enabled`).run(space_id, webhook_url_ref, on ? 1 : 0);
- return shapeMapping(db.prepare("SELECT m.*, s.slug AS space_slug FROM relay_mappings m JOIN spaces s ON s.id = m.space_id WHERE m.space_id = ? AND m.direction = 'out' AND m.webhook_url_ref = ?").get(space_id, webhook_url_ref));
+ const channel = snowflakeOrNull(discord_channel_id, 'discord_channel_id');
+ const thread = snowflakeOrNull(discord_thread_id, 'discord_thread_id');
+ db.prepare(`INSERT INTO relay_mappings (space_id, direction, webhook_url_ref, enabled, discord_channel_id, discord_thread_id, inbound) VALUES (?, 'out', ?, ?, ?, ?, ?)
+ ON CONFLICT(space_id, direction, webhook_url_ref) DO UPDATE SET enabled = excluded.enabled`).run(space_id, webhook_url_ref, on ? 1 : 0, channel || null, thread || null, inbound ? 1 : 0);
+ const row = db.prepare(`${mappingQuery} WHERE m.space_id = ? AND m.direction = 'out' AND m.webhook_url_ref = ?`).get(space_id, webhook_url_ref);
+ return updateMapping(row.id, { discord_channel_id: channel, discord_thread_id: thread, inbound });
}
}
- function setMappingEnabled(id, on) {
- if (!db.prepare('UPDATE relay_mappings SET enabled = ? WHERE id = ?').run(on ? 1 : 0, id).changes) return null;
- return shapeMapping(db.prepare('SELECT m.*, s.slug AS space_slug FROM relay_mappings m JOIN spaces s ON s.id = m.space_id WHERE m.id = ?').get(id));
+ /** { enabled?, inbound?, discord_channel_id?, discord_thread_id? } → mapping, or null when there is no such mapping */
+ function updateMapping(id, fields = {}) {
+ const sets = [];
+ const params = [];
+ if (fields.enabled !== undefined) { sets.push('enabled = ?'); params.push(fields.enabled ? 1 : 0); }
+ if (fields.inbound !== undefined) { sets.push('inbound = ?'); params.push(fields.inbound ? 1 : 0); }
+ for (const k of ['discord_channel_id', 'discord_thread_id']) {
+ const v = snowflakeOrNull(fields[k], k);
+ if (v !== undefined) { sets.push(`${k} = ?`); params.push(v); }
+ }
+ if (sets.length && !db.prepare(`UPDATE relay_mappings SET ${sets.join(', ')} WHERE id = ?`).run(...params, id).changes) return null;
+ const row = db.prepare(`${mappingQuery} WHERE m.id = ?`).get(id);
+ return row ? shapeMapping(row) : null;
}
}
- function listDeliveries({ status = null, limit = 50 } = {}) {
+ function setMappingEnabled(id, on) { return updateMapping(id, { enabled: on }); }
+
+ function listDeliveries({ status = null, action = null, limit = 50 } = {}) {
return db.prepare(`SELECT d.*, m.webhook_url_ref, t.title AS thread_title, t.slug AS thread_slug, s.slug AS space_slug
return db.prepare(`SELECT d.*, m.webhook_url_ref, t.title AS thread_title, t.slug AS thread_slug, s.slug AS space_slug
FROM relay_deliveries d JOIN relay_mappings m ON m.id = d.mapping_id JOIN threads t ON t.id = d.thread_id JOIN spaces s ON s.id = t.space_id
FROM relay_deliveries d JOIN relay_mappings m ON m.id = d.mapping_id JOIN threads t ON t.id = d.thread_id JOIN spaces s ON s.id = t.space_id
- WHERE (? IS NULL OR d.status = ?) ORDER BY d.id DESC LIMIT ?`).all(status, status, limit)
+ WHERE (? IS NULL OR d.status = ?) AND (? IS NULL OR d.action = ?) ORDER BY d.id DESC LIMIT ?`).all(status, status, action, action, limit)
.map((d) => ({
.map((d) => ({
- id: d.id, dedupe_key: d.dedupe_key, status: d.status, attempts: d.attempts, last_status: d.last_status, last_error: d.last_error,
- next_attempt_at: d.status === 'pending' ? isoTime(d.next_attempt_at) : null, delivered_at: isoTime(d.delivered_at), created_at: isoTime(d.created_at),
+ id: d.id, dedupe_key: d.dedupe_key, action: d.action, source: d.source, event_id: d.event_id || null,
+ status: d.status, attempts: d.attempts, last_status: d.last_status, last_error: d.last_error,
+ next_attempt_at: d.status === 'pending' ? isoTime(d.next_attempt_at) : null, delivered_at: isoTime(d.delivered_at), created_at: isoTime(d.created_at), updated_at: isoTime(d.updated_at),
mapping: { id: d.mapping_id, webhook_url_ref: d.webhook_url_ref },
mapping: { id: d.mapping_id, webhook_url_ref: d.webhook_url_ref },
thread: { id: d.thread_id, title: d.thread_title, url: `/s/${d.space_slug}/t/${d.thread_slug}` },
thread: { id: d.thread_id, title: d.thread_title, url: `/s/${d.space_slug}/t/${d.thread_slug}` },
+ post: d.post_id ? { id: d.post_id } : null,
}));
}));
}
}
- /** Put a failed (or pending) delivery back in the queue now, with a fresh attempt budget. */
+ /** Put a delivery that was not delivered back in the queue now, with a fresh attempt budget. */
function retry(id) {
function retry(id) {
const n = db.prepare("UPDATE relay_deliveries SET status = 'pending', attempts = 0, next_attempt_at = ?, updated_at = CURRENT_TIMESTAMP WHERE id = ? AND status <> 'delivered'").run(sqlTime(now()), id).changes;
const n = db.prepare("UPDATE relay_deliveries SET status = 'pending', attempts = 0, next_attempt_at = ?, updated_at = CURRENT_TIMESTAMP WHERE id = ? AND status <> 'delivered'").run(sqlTime(now()), id).changes;
if (n) kick();
if (n) kick();
return n;
return n;
}
}
+ /** Staff give up on a waiting or dead delivery: 'dropped', kept for the record. */
+ function drop(id) {
+ return db.prepare("UPDATE relay_deliveries SET status = 'dropped', updated_at = CURRENT_TIMESTAMP WHERE id = ? AND status IN ('pending', 'failed')").run(id).changes;
+ }
+
+ /** What staff (and /api/ready) see: the queue by status, and the Events worker's and the gateway's state. */
+ function status() {
+ const deliveries = Object.fromEntries(DELIVERY_STATUSES.map((s) => [s, 0]));
+ for (const r of db.prepare('SELECT status, COUNT(*) AS n FROM relay_deliveries GROUP BY status').all()) deliveries[r.status] = r.n;
+ const off = (reason) => ({ enabled: false, reason });
+ return {
+ enabled,
+ creates_from: createsFromEvents ? 'events' : 'forum',
+ deliveries,
+ dead_letters: deliveries.failed,
+ messages_mapped: db.prepare('SELECT COUNT(*) AS n FROM relay_message_map WHERE external_deleted_at IS NULL').get().n,
+ events_worker: parts.worker ? parts.worker.status() : off(enabled ? 'not configured' : 'the relay is off'),
+ inbound: parts.gateway ? { ...parts.gateway.status(), ...(parts.inbound ? { handled: parts.inbound.stats() } : {}) } : off(enabled ? 'DISCORD_RELAY_INBOUND is off' : 'the relay is off'),
+ inbound_failures: db.prepare('SELECT COUNT(*) AS n FROM relay_inbound_failures WHERE dismissed_at IS NULL').get().n,
+ };
+ }
- return { enabled, enqueueThread, drain, start, stop, listMappings, addMapping, setMappingEnabled, listDeliveries, retry, ENV_NAME, refAllowed, message };
+ return {
+ enabled, enqueueThread, enqueuePost, enqueueEdit, enqueueDelete, hideThreads, drain, start, stop, attach, useEventsForCreates, status,
+ listMappings, addMapping, updateMapping, setMappingEnabled, listDeliveries, retry, drop, deliveryKey, ENV_NAME, refAllowed, message, postMessage,
+ };
}
}
-module.exports = { createDiscordRelay, escapeDiscord, ENV_NAME, WEBHOOK_VAR };
+module.exports = { createDiscordRelay, escapeDiscord, ENV_NAME, WEBHOOK_VAR, SNOWFLAKE, DELIVERY_STATUSES };
new file mode 100644
new file mode 100644
@@ -0,0 +1,164 @@
@@ -0,0 +1,164 @@
+'use strict';
+
+/**
+ * The Discord relay's Events worker (roadmap WS-J task 6): reads Community's own
+ * community.thread.* and community.post.* from OpenVibe.Events' pull API (GET /api/v1/events,
+ * capability events.event.read, Community's service token on audience openvibe.events) and queues
+ * the relay's creates from them (relay/discord.js).
+ *
+ * - Cursor: the relay_cursors row 'discord-relay:events', moved in the same transaction as the
+ * deliveries its page queued. A crash replays at most the page in progress, and the dedupe keys
+ * make that replay queue nothing twice.
+ * - First start (no cursor yet): it begins at Events' head. Threads from before are never announced.
+ * - Events carry ids, never content: the worker reads the thread or post as it is now and the relay's
+ * rules decide (public spaces only, never members-only, never origin 'discord'). Redacted events
+ * and events from any other source are ignored.
+ * - A gap (retention pruned events the worker had not read yet) is logged and shown to staff; the
+ * creates in it are not relayed.
+ * - While it runs, the forum's own create calls queue nothing (the worker is the one path). Off
+ * without EVENTS_URL and OV_OAUTH_CLIENT_SECRET, or with DISCORD_RELAY_EVENTS=off: then the forum
+ * queues creates itself, as the first relay did. Edits and deletes are queued by the forum in both
+ * cases (Contracts has no community.thread/post update or delete events yet).
+ */
+const forumStore = require('../forum/store');
+
+const TOPICS = ['community.thread.*', 'community.post.*'];
+const CURSOR = 'discord-relay:events';
+const MAX_PAGES_PER_TICK = 20;
+
+/**
+ * events: an openvibe-sdk events client (tests); otherwise one is made from eventsUrl, clientSecret and
+ * tokenUrl (the Network's /oauth/token, client credentials of clientId).
+ */
+function createRelayEventsWorker({ db, relay, events = null, eventsUrl = null, clientSecret = null, tokenUrl = null, clientId = 'community', fetchImpl, enabled = true, pollMs = 5000, pageLimit = 100, now = () => Date.now() } = {}) {
+ let reason = null;
+ if (!relay || !relay.enabled) reason = 'the Discord relay is off (DISCORD_RELAY_ENABLED)';
+ else if (!enabled) reason = 'DISCORD_RELAY_EVENTS=off';
+ else if (!events && (!eventsUrl || !clientSecret || !tokenUrl)) reason = 'EVENTS_URL and OV_OAUTH_CLIENT_SECRET are not both set';
+ let client = events;
+ if (!reason && !client) {
+ const { createClient } = require('openvibe-sdk/core');
+ const { createServiceTokenClient } = require('openvibe-sdk/auth');
+ const { createEventsClient } = require('openvibe-sdk/events');
+ const tokens = createServiceTokenClient({ tokenUrl, clientId, clientSecret, fetch: fetchImpl });
+ const core = createClient({ baseUrls: { events: String(eventsUrl).replace(/\/+$/, '') }, tokenProvider: tokens, fetch: fetchImpl, retries: 0 });
+ client = createEventsClient(core, { source: 'community' });
+ }
+ const on = !reason;
+ const stats = { seen: 0, queued: 0, pages: 0, last_error: null, last_error_at: null, last_ok_at: null, last_gap: null, started_at_seq: null };
+ let timer = null;
+ let running = null;
+ let stopped = false;
+
+ const iso = () => new Date(now()).toISOString();
+ const readCursor = () => db.prepare('SELECT cursor, latest_seq FROM relay_cursors WHERE name = ?').get(CURSOR) || null;
+ function writeCursor(cursor, latest) {
+ db.prepare(`INSERT INTO relay_cursors (name, cursor, latest_seq, updated_at) VALUES (?, ?, ?, CURRENT_TIMESTAMP)
+ ON CONFLICT(name) DO UPDATE SET cursor = excluded.cursor, latest_seq = excluded.latest_seq, updated_at = CURRENT_TIMESTAMP`).run(CURSOR, cursor, latest == null ? null : latest);
+ }
+
+ /** One event → deliveries queued (0 when it is not for the relay). */
+ function handle(ev) {
+ stats.seen++;
+ if (!ev || ev.source !== 'community' || !ev.payload || ev.payload.redacted) return 0;
+ const p = ev.payload;
+ if (p.visibility && p.visibility !== 'public') return 0;
+ if (ev.event_type === 'community.thread.created') {
+ const thread = db.prepare('SELECT * FROM threads WHERE id = ?').get(Number(p.thread_id));
+ const space = thread ? forumStore.getSpaceById(db, thread.space_id) : null;
+ return relay.enqueueThread(thread, space, { source: 'events', eventId: ev.event_id || null });
+ }
+ if (ev.event_type === 'community.post.created') {
+ const post = forumStore.getPost(db, Number(p.post_id));
+ const thread = post ? db.prepare('SELECT * FROM threads WHERE id = ?').get(post.thread_id) : null;
+ const space = thread ? forumStore.getSpaceById(db, thread.space_id) : null;
+ return relay.enqueuePost(post, thread, space, { source: 'events', eventId: ev.event_id || null });
+ }
+ return 0;
+ }
+
+ /** Read everything new (at most MAX_PAGES_PER_TICK pages). One tick at a time. */
+ function tick() {
+ if (!on) return Promise.resolve({ queued: 0 });
+ if (running) return running;
+ running = (async () => {
+ let queuedNow = 0;
+ try {
+ let cur = readCursor();
+ if (!cur) {
+ const head = await client.pull({ topic: TOPICS, afterSeq: 0, limit: 1 });
+ const latest = Number(head.latest_seq) || 0;
+ writeCursor(latest, latest);
+ stats.started_at_seq = latest;
+ console.log(`[Relay] Events worker starts at seq ${latest} (earlier threads are not announced)`);
+ cur = { cursor: latest, latest_seq: latest };
+ }
+ let cursor = Number(cur.cursor) || 0;
+ for (let pages = 0; pages < MAX_PAGES_PER_TICK && !stopped; pages++) {
+ const page = await client.pull({ topic: TOPICS, afterSeq: cursor, limit: pageLimit });
+ if (stopped) break;
+ if (page.gap) {
+ stats.last_gap = { from_seq: page.gap.from_seq, to_seq: page.gap.to_seq, at: iso() };
+ console.warn(`[Relay] Events gap: seq ${page.gap.from_seq}..${page.gap.to_seq} were pruned before the relay read them; threads and replies in it are not relayed`);
+ }
+ const next = Number(page.next_after_seq);
+ const latest = Number(page.latest_seq);
+ const moved = Number.isFinite(next) && next > cursor ? next : cursor;
+ queuedNow += db.transaction(() => {
+ let n = 0;
+ for (const item of page.events || []) n += handle(item && item.event);
+ writeCursor(moved, Number.isFinite(latest) ? latest : null);
+ return n;
+ })();
+ stats.pages++;
+ if (moved === cursor || !Number.isFinite(latest) || moved >= latest) break;
+ cursor = moved;
+ }
+ stats.queued += queuedNow;
+ stats.last_ok_at = iso();
+ if (stats.last_error) console.log('[Relay] Events worker reading again');
+ stats.last_error = null;
+ stats.last_error_at = null;
+ } catch (err) {
+ const m = err && err.message ? err.message : String(err);
+ if (m !== stats.last_error) console.warn('[Relay] Events worker cannot read (will retry):', m);
+ stats.last_error = m;
+ stats.last_error_at = iso();
+ }
+ return { queued: queuedNow };
+ })().finally(() => { running = null; });
+ return running;
+ }
+
+ function start() {
+ if (!on || timer) return;
+ stopped = false;
+ relay.useEventsForCreates(true);
+ timer = setInterval(() => { tick(); }, pollMs);
+ if (timer.unref) timer.unref();
+ tick();
+ }
+ /** Graceful stop: no further reads; resolves when the page in progress has been handled. */
+ function stop() {
+ stopped = true;
+ if (timer) clearInterval(timer);
+ timer = null;
+ return running ? running.then(() => undefined) : Promise.resolve();
+ }
+
+ function status() {
+ if (!on) return { enabled: false, reason };
+ const cur = readCursor();
+ return {
+ enabled: true, running: !!timer, topics: TOPICS,
+ cursor: cur ? cur.cursor : null, latest_seq: cur ? cur.latest_seq : null,
+ lag: cur && cur.latest_seq != null ? Math.max(cur.latest_seq - cur.cursor, 0) : null,
+ events_seen: stats.seen, deliveries_queued: stats.queued, started_at_seq: stats.started_at_seq,
+ last_ok_at: stats.last_ok_at, last_error: stats.last_error, last_error_at: stats.last_error_at, last_gap: stats.last_gap,
+ };
+ }
+
+ return { enabled: on, reason, tick, start, stop, status, TOPICS, CURSOR };
+}
+
+module.exports = { createRelayEventsWorker, TOPICS, CURSOR };
new file mode 100644
new file mode 100644
@@ -0,0 +1,199 @@
@@ -0,0 +1,199 @@
+'use strict';
+
+/**
+ * Replies from Discord (roadmap WS-J task 6, inbound; DISCORD_RELAY_INBOUND=on with DISCORD_BOT_TOKEN).
+ * The gateway (relay/discord-gateway.js) hands message events here.
+ *
+ * What becomes a post: only messages in a mapped channel (a mapping with inbound on: its
+ * discord_channel_id or discord_thread_id, or a Discord thread started from one of its messages), and
+ * only replies to something in the external message map:
+ * - a Discord reply (message_reference) to a mapped message in the same channel, or
+ * - any message in a Discord thread started from a mapped message (that thread's id is the message's).
+ * Everything else in the channel is ignored. The post goes into the mapped message's local thread with
+ * origin 'discord', no author subject (never attributed to anyone on the site; the Discord display name
+ * is kept in posts.relay_author and shown with a "Discord" badge), and the map gets its direction 'in'
+ * row. Edits on Discord edit the post (a new revision, edited_by 'discord'); deletes on Discord delete
+ * it. A delete of a message the relay sent only marks it gone there (the thread stays).
+ *
+ * Loop prevention: webhook messages (the relay's own among them), bots, the bot itself and system
+ * messages are never taken in; a message already in the map is never taken twice (and the map's unique
+ * key refuses it anyway); posts with origin 'discord' are never relayed out (relay/discord.js).
+ * Limits: perMinute messages per Discord author per mapping and 10× that per mapping (a rolling
+ * minute), maxChars per message (longer text is cut). Mentions, roles, channels, custom emoji and
+ * timestamps become plain text and @everyone/@here are defused; the rest is Markdown, which the
+ * forum's safe renderer shows (render/markdown.js: no raw HTML, safe links only). Attachments are
+ * noted, never fetched. The thread must still be public, open (not locked) and not members-only.
+ * What could not be applied goes to relay_inbound_failures: ids and the reason, never the text.
+ */
+const forumStore = require('../forum/store');
+const { SNOWFLAKE } = require('./discord');
+
+const KEEP_FAILURES = 1000;
+const MESSAGE_TYPES = new Set([0, 19]); // DEFAULT, REPLY
+
+/** A Discord display name as plain text: no control or format characters, one line, at most 32. */
+function cleanName(s) {
+ const t = String(s == null ? '' : s).replace(/[\u0000-\u001F\u007F-\u009F\u200b-\u200F\u202A-\u202E\u2060-\u2064\uFEFF]/g, '').replace(/\s+/g, ' ').trim();
+ return t.slice(0, 32);
+}
+
+/** Discord message text → forum Markdown (see the header). '' when there is nothing to show. */
+function toMarkdown(d, maxChars = 4000) {
+ const names = new Map((Array.isArray(d.mentions) ? d.mentions : []).map((u) => [String(u && u.id), cleanName(u && (u.global_name || u.username))]));
+ let text = (typeof d.content === 'string' ? d.content : '')
+ .replace(/<@!?(\d{15,21})>/g, (_, id) => `@${names.get(id) || 'someone'}`)
+ .replace(/<@&\d{15,21}>/g, '@role')
+ .replace(/<#\d{15,21}>/g, '#channel')
+ .replace(/<\/([\w-]{1,32}(?: [\w-]{1,32}){0,2}):\d{15,21}>/g, '/$1')
+ .replace(/<a?:(\w{1,32}):\d{15,21}>/g, ':$1:')
+ .replace(/<t:(-?\d{1,13})(?::[tTdDfFR])?>/g, (_, sec) => {
+ const t = new Date(Number(sec) * 1000);
+ return Number.isNaN(t.getTime()) ? '' : `${t.toISOString().slice(0, 16).replace('T', ' ')} UTC`;
+ })
+ .replace(/@(everyone|here)/g, '@\u200b$1')
+ .replace(/\r\n?/g, '\n')
+ .replace(/[\u0000-\u0008\u000B\u000C\u000E-\u001F\u007F]/g, '')
+ .trim();
+ if (text.length > maxChars) text = `${text.slice(0, Math.max(maxChars - 1, 0)).trimEnd()}…`;
+ const files = Array.isArray(d.attachments) ? d.attachments.length : 0;
+ if (files) text += `${text ? '\n\n' : ''}_(${files} attachment${files === 1 ? '' : 's'} on Discord)_`;
+ return text;
+}
+
+function createDiscordInbound({ db, perMinute = 6, maxChars = 4000, now = () => Date.now() } = {}) {
+ const windows = new Map(); // key → timestamps in the last minute
+ const counts = { applied: 0, ignored: 0, failed: 0 };
+
+ function allow(key, limit) {
+ const t = now();
+ const list = (windows.get(key) || []).filter((x) => t - x < 60_000);
+ if (list.length >= limit) { windows.set(key, list); return false; }
+ list.push(t);
+ windows.set(key, list);
+ if (windows.size > 5000) for (const [k, l] of windows) if (!l.length || t - l[l.length - 1] >= 60_000) windows.delete(k);
+ return true;
+ }
+
+ const mapByMessage = (id) => db.prepare("SELECT * FROM relay_message_map WHERE platform = 'discord' AND external_message_id = ?").get(String(id)) || null;
+ const mappingById = (id) => db.prepare('SELECT * FROM relay_mappings WHERE id = ?').get(id) || null;
+
+ function ignore(reason) { counts.ignored++; return { status: 'ignored', reason }; }
+ function failure(event, d, mappingId, error) {
+ db.prepare(`INSERT INTO relay_inbound_failures (mapping_id, event, external_channel_id, external_message_id, external_author_id, error)
+ VALUES (?, ?, ?, ?, ?, ?)`).run(mappingId || null, event, d.channel_id ? String(d.channel_id) : null, d.id ? String(d.id) : null, d.author && d.author.id ? String(d.author.id) : null, String(error).slice(0, 300));
+ db.prepare('DELETE FROM relay_inbound_failures WHERE id <= (SELECT MAX(id) FROM relay_inbound_failures) - ?').run(KEEP_FAILURES);
+ counts.failed++;
+ return { status: 'failed', error };
+ }
+ /** Is the local thread still somewhere a reply from Discord may land? → null or why not */
+ function closed(thread) {
+ if (!thread || thread.deleted_at) return 'the thread was deleted';
+ const space = forumStore.getSpaceById(db, thread.space_id);
+ if (!space || space.visibility !== 'public' || space.members_only_owner || thread.members_only_owner) return 'the thread is not public';
+ if (thread.locked) return 'the thread is locked';
+ return null;
+ }
+
+ function create(d, botUserId) {
+ if (d.webhook_id) return ignore('a webhook message (the relay\'s own included)');
+ const author = d.author || {};
+ if (!author.id || author.bot || author.system || (botUserId && String(author.id) === String(botUserId))) return ignore('a bot or system message');
+ if (d.type !== undefined && !MESSAGE_TYPES.has(Number(d.type))) return ignore('not a plain message or a reply');
+ if (!SNOWFLAKE.test(String(d.id || '')) || !SNOWFLAKE.test(String(d.channel_id || ''))) return ignore('malformed ids');
+ if (mapByMessage(d.id)) return ignore('already relayed');
+ const channel = String(d.channel_id);
+ // A Discord thread started from a mapped message, else a reply to one in the same channel.
+ let anchor = mapByMessage(channel);
+ // (message_reference type 1 is a forward, not a reply: never taken in.)
+ if (!anchor && d.message_reference && d.message_reference.message_id && !Number(d.message_reference.type || 0)) {
+ const ref = mapByMessage(String(d.message_reference.message_id));
+ if (ref && ref.external_channel_id === channel) anchor = ref;
+ }
+ if (!anchor) return ignore('not a reply to a relayed message');
+ const mapping = mappingById(anchor.mapping_id);
+ if (!mapping || !mapping.enabled || !mapping.inbound) return ignore('inbound is off for this mapping');
+ const mapped = [mapping.discord_channel_id, mapping.discord_thread_id].filter(Boolean);
+ const inMapped = mapped.includes(channel) || (channel === anchor.external_message_id && mapped.includes(anchor.external_channel_id));
+ if (!inMapped) return ignore('not a mapped channel');
+ const thread = db.prepare('SELECT * FROM threads WHERE id = ?').get(anchor.thread_id);
+ const why = closed(thread);
+ if (why) return failure('MESSAGE_CREATE', d, mapping.id, why);
+ if (!allow(`a:${mapping.id}:${author.id}`, perMinute) || !allow(`m:${mapping.id}`, perMinute * 10)) return failure('MESSAGE_CREATE', d, mapping.id, 'rate limited');
+ const body = toMarkdown(d, maxChars);
+ if (!body) return failure('MESSAGE_CREATE', d, mapping.id, 'the message has no text: is the MESSAGE CONTENT intent on for the bot?');
+ const name = cleanName((d.member && d.member.nick) || author.global_name || author.username) || 'Discord';
+ try {
+ const post = db.transaction(() => {
+ const p = forumStore.addPost(db, { thread_id: thread.id, author_subject: null, origin: 'discord', body_markdown: body, relay_author: name });
+ db.prepare(`INSERT INTO relay_message_map (platform, mapping_id, direction, local_type, local_id, thread_id, external_channel_id, external_message_id)
+ VALUES ('discord', ?, 'in', 'post', ?, ?, ?, ?)`).run(mapping.id, p.id, thread.id, channel, String(d.id));
+ return p;
+ })();
+ counts.applied++;
+ return { status: 'applied', post_id: post.id };
+ } catch (err) {
+ if (/UNIQUE/.test(err.message)) return ignore('already relayed');
+ return failure('MESSAGE_CREATE', d, mapping.id, `could not save the post: ${err.message}`);
+ }
+ }
+
+ function update(d) {
+ if (!d.id) return ignore('malformed ids');
+ const row = mapByMessage(d.id);
+ if (!row) return ignore('not a relayed message');
+ if (row.direction !== 'in') return ignore('the relay\'s own message');
+ if (typeof d.content !== 'string') return ignore('no text change');
+ const mapping = mappingById(row.mapping_id);
+ if (!mapping || !mapping.enabled || !mapping.inbound) return ignore('inbound is off for this mapping');
+ const post = forumStore.getPost(db, row.local_id);
+ if (!post || post.deleted_at) return failure('MESSAGE_UPDATE', d, mapping.id, 'the post was deleted');
+ const why = closed(db.prepare('SELECT * FROM threads WHERE id = ?').get(post.thread_id));
+ if (why) return failure('MESSAGE_UPDATE', d, mapping.id, why);
+ const body = toMarkdown(d, maxChars);
+ if (!body) return failure('MESSAGE_UPDATE', d, mapping.id, 'the edit has no text');
+ forumStore.editPost(db, post.id, body, 'discord');
+ db.prepare('UPDATE relay_message_map SET updated_at = CURRENT_TIMESTAMP WHERE id = ?').run(row.id);
+ counts.applied++;
+ return { status: 'applied', post_id: post.id };
+ }
+
+ /** A delete on Discord. Applied whatever the mapping's switches: taking something down never waits. */
+ function remove(id) {
+ const row = mapByMessage(id);
+ if (!row) return ignore('not a relayed message');
+ db.transaction(() => {
+ db.prepare('UPDATE relay_message_map SET external_deleted_at = CURRENT_TIMESTAMP, updated_at = CURRENT_TIMESTAMP WHERE id = ? AND external_deleted_at IS NULL').run(row.id);
+ if (row.direction === 'in') forumStore.softDeletePost(db, row.local_id);
+ })();
+ counts.applied++;
+ return { status: 'applied', post_id: row.direction === 'in' ? row.local_id : null, note: row.direction === 'out' ? 'the relay\'s message was deleted on Discord; the thread stays' : undefined };
+ }
+
+ /** One gateway dispatch (type, data, { botUserId }) → result (an array for a bulk delete). */
+ function handle(type, d, { botUserId = null } = {}) {
+ if (!d || typeof d !== 'object') return ignore('no data');
+ if (type === 'MESSAGE_CREATE') return create(d, botUserId);
+ if (type === 'MESSAGE_UPDATE') return update(d);
+ if (type === 'MESSAGE_DELETE') return remove(String(d.id || ''));
+ if (type === 'MESSAGE_DELETE_BULK') return (Array.isArray(d.ids) ? d.ids : []).map((id) => remove(String(id)));
+ return ignore('not a message event');
+ }
+
+ // ── staff ────────────────────────────────────────────────
+ function listFailures({ all = false, limit = 50 } = {}) {
+ return db.prepare(`SELECT f.*, s.slug AS space_slug FROM relay_inbound_failures f LEFT JOIN relay_mappings m ON m.id = f.mapping_id LEFT JOIN spaces s ON s.id = m.space_id
+ WHERE (? = 1 OR f.dismissed_at IS NULL) ORDER BY f.id DESC LIMIT ?`).all(all ? 1 : 0, limit)
+ .map((f) => ({
+ id: f.id, event: f.event, error: f.error, mapping: f.mapping_id ? { id: f.mapping_id, space: f.space_slug || null } : null,
+ discord: { channel_id: f.external_channel_id, message_id: f.external_message_id, author_id: f.external_author_id },
+ dismissed: !!f.dismissed_at, created_at: f.created_at ? new Date(`${String(f.created_at).replace(' ', 'T')}Z`).toISOString() : null,
+ }));
+ }
+ function dismiss(id) {
+ return db.prepare('UPDATE relay_inbound_failures SET dismissed_at = CURRENT_TIMESTAMP WHERE id = ? AND dismissed_at IS NULL').run(id).changes;
+ }
+
+ return { handle, listFailures, dismiss, stats: () => ({ ...counts }) };
+}
+
+module.exports = { createDiscordInbound, toMarkdown, cleanName };
@@ -159,7 +159,7 @@ function authorPanel(p) {
@@ -159,7 +159,7 @@ function authorPanel(p) {
return `<aside class="postbit-author">
return `<aside class="postbit-author">
${pic ? `<img class="postbit-avatar" src="${esc(pic)}" alt="" width="64" height="64" loading="lazy">` : `<span class="postbit-avatar postbit-letter" aria-hidden="true">${initial}</span>`}
${pic ? `<img class="postbit-avatar" src="${esc(pic)}" alt="" width="64" height="64" loading="lazy">` : `<span class="postbit-avatar postbit-letter" aria-hidden="true">${initial}</span>`}
<span class="postbit-name">${a.username ? `<a href="https://openvibe.live/@${encodeURIComponent(a.username)}" rel="noopener">${esc(name)}</a>` : esc(name)}</span>
<span class="postbit-name">${a.username ? `<a href="https://openvibe.live/@${encodeURIComponent(a.username)}" rel="noopener">${esc(name)}</a>` : esc(name)}</span>
- ${a.is_ai ? '<span class="badge badge-ai" title="Written by AI, not by a person">AI</span>' : a.is_system ? '<span class="badge">OpenVibe</span>' : ''}
+ ${a.is_ai ? '<span class="badge badge-ai" title="Written by AI, not by a person">AI</span>' : a.is_system ? '<span class="badge">OpenVibe</span>' : a.is_relay ? '<span class="badge badge-relay" title="Written on Discord and relayed here">Discord</span>' : ''}
${st ? `<dl class="postbit-stats"><div><dt>Posts</dt><dd>${num(st.posts)}</dd></div>${monthYear(st.first_post_at) ? `<div><dt>Since</dt><dd>${esc(monthYear(st.first_post_at))}</dd></div>` : ''}</dl>
${st ? `<dl class="postbit-stats"><div><dt>Posts</dt><dd>${num(st.posts)}</dd></div>${monthYear(st.first_post_at) ? `<div><dt>Since</dt><dd>${esc(monthYear(st.first_post_at))}</dd></div>` : ''}</dl>
${(st.ratings || []).length ? `<p class="postbit-ratings" aria-label="Ratings received">${st.ratings.map((r) => `<span title="${esc(r.label)}">${r.emoji} ${num(r.count)}</span>`).join(' ')}</p>` : ''}` : ''}
${(st.ratings || []).length ? `<p class="postbit-ratings" aria-label="Ratings received">${st.ratings.map((r) => `<span title="${esc(r.label)}">${r.emoji} ${num(r.count)}</span>`).join(' ')}</p>` : ''}` : ''}
</aside>`;
</aside>`;
@@ -23,6 +23,7 @@ function who(a) {
@@ -23,6 +23,7 @@ function who(a) {
if (!a) return '<span class="author"><span class="avatar avatar-letter" aria-hidden="true">?</span><span>Anonymous</span></span>';
if (!a) return '<span class="author"><span class="avatar avatar-letter" aria-hidden="true">?</span><span>Anonymous</span></span>';
if (a.is_ai) return `<span class="author">${authorHtml({ display_name: a.display_name }, { link: false })}</span> <span class="badge badge-ai" title="Written by AI, not by a person">AI</span>`;
if (a.is_ai) return `<span class="author">${authorHtml({ display_name: a.display_name }, { link: false })}</span> <span class="badge badge-ai" title="Written by AI, not by a person">AI</span>`;
if (a.is_system) return `<span class="author">${authorHtml({ display_name: a.display_name }, { link: false })}</span>`;
if (a.is_system) return `<span class="author">${authorHtml({ display_name: a.display_name }, { link: false })}</span>`;
+ if (a.is_relay) return `<span class="author">${authorHtml({ display_name: a.display_name }, { link: false })}</span> <span class="badge badge-relay" title="Written on Discord and relayed here">Discord</span>`;
return authorHtml(a);
return authorHtml(a);
}
}
new file mode 100644
new file mode 100644
@@ -0,0 +1,97 @@
@@ -0,0 +1,97 @@
+'use strict';
+/**
+ * Stand-ins for Discord in the relay tests.
+ *
+ * startWebhooks(): an HTTP server speaking the webhook API the relay uses:
+ * POST /api/webhooks/:id/:token?wait=true[&thread_id=] → 200 the message { id, channel_id, webhook_id } (204 without wait)
+ * PATCH /api/webhooks/:id/:token/messages/:mid[?thread_id] → 200 the message
+ * DELETE /api/webhooks/:id/:token/messages/:mid[?thread_id] → 204
+ * an unknown or deleted message → 404 { code: 10008, message: 'Unknown Message' }
+ * Message ids count up from 1100000000000000000; a webhook's channel is 2000000000000000000 + its id,
+ * or the thread_id it was sent to. `plan` queues canned answers ({ status, body, headers } or
+ * { hang: true }) that take precedence; `hits` records every request.
+ *
+ * createGateway(): a fake WebSocket class (gw.WebSocket) the gateway client is given, with the
+ * server side scripted by the test: gw.sockets, gw.push(sock, payload), gw.drop(sock, code), and
+ * gw.onOpen / gw.onSend hooks. Sockets open on the next tick.
+ */
+const http = require('http');
+
+function startWebhooks() {
+ const hits = [];
+ const plan = [];
+ const messages = new Map();
+ let next = 1100000000000000000n;
+ const server = http.createServer((req, res) => {
+ let raw = '';
+ req.on('data', (c) => { raw += c; });
+ req.on('end', () => {
+ const u = new URL(req.url, 'http://discord.test');
+ const body = raw ? JSON.parse(raw) : null;
+ hits.push({ method: req.method, path: u.pathname + u.search, pathname: u.pathname, query: Object.fromEntries(u.searchParams), body });
+ const json = (status, obj) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(obj === undefined ? '' : JSON.stringify(obj)); };
+ const canned = plan.shift();
+ if (canned) {
+ if (canned.hang) return undefined;
+ res.writeHead(canned.status, { 'Content-Type': 'application/json', ...(canned.headers || {}) });
+ return res.end(canned.body ? JSON.stringify(canned.body) : '');
+ }
+ const m = u.pathname.match(/^\/api\/webhooks\/(\d+)\/([^/]+)(?:\/messages\/(\d+))?$/);
+ if (!m) return json(404, { message: 'Unknown Webhook', code: 10015 });
+ const [, hookId, , mid] = m;
+ if (req.method === 'POST' && !mid) {
+ const id = String(next++);
+ const channelId = u.searchParams.get('thread_id') || String(2000000000000000000n + BigInt(hookId));
+ messages.set(id, { id, channel_id: channelId, webhook_id: hookId, body, deleted: false });
+ if (u.searchParams.get('wait') !== 'true') { res.writeHead(204); return res.end(); }
+ return json(200, { id, channel_id: channelId, webhook_id: hookId, content: body && body.content });
+ }
+ const msg = mid ? messages.get(mid) : null;
+ if (!msg || msg.deleted) return json(404, { message: 'Unknown Message', code: 10008 });
+ if (req.method === 'PATCH') { msg.body = body; return json(200, { id: msg.id, channel_id: msg.channel_id, webhook_id: msg.webhook_id, content: body && body.content }); }
+ if (req.method === 'DELETE') { msg.deleted = true; res.writeHead(204); return res.end(); }
+ return json(405, { message: '405: Method Not Allowed', code: 0 });
+ });
+ });
+ return new Promise((resolve) => server.listen(0, '127.0.0.1', () => resolve({
+ url: `http://127.0.0.1:${server.address().port}`, hits, plan, messages,
+ close: () => new Promise((r) => server.close(r)),
+ })));
+}
+
+function createGateway() {
+ const gw = { sockets: [], onOpen: null, onSend: null };
+ gw.WebSocket = class FakeWebSocket extends EventTarget {
+ constructor(url) {
+ super();
+ this.url = url;
+ this.readyState = 0;
+ this.sent = [];
+ this.closedWith = null;
+ gw.sockets.push(this);
+ setImmediate(() => {
+ if (this.readyState !== 0) return;
+ this.readyState = 1;
+ this.dispatchEvent(new Event('open'));
+ if (gw.onOpen) gw.onOpen(this);
+ });
+ }
+ send(data) {
+ const p = JSON.parse(data);
+ this.sent.push(p);
+ if (gw.onSend) gw.onSend(this, p);
+ }
+ close(code = 1000, reason = '') {
+ if (this.readyState === 3) return;
+ this.readyState = 3;
+ this.closedWith = code;
+ setImmediate(() => this.dispatchEvent(Object.assign(new Event('close'), { code, reason })));
+ }
+ };
+ gw.push = (sock, payload) => sock.dispatchEvent(Object.assign(new Event('message'), { data: JSON.stringify(payload) }));
+ gw.drop = (sock, code, reason = '') => { sock.readyState = 3; sock.dispatchEvent(Object.assign(new Event('close'), { code, reason })); };
+ gw.last = () => gw.sockets[gw.sockets.length - 1];
+ return gw;
+}
+
+module.exports = { startWebhooks, createGateway };
new file mode 100644
new file mode 100644
@@ -0,0 +1,74 @@
@@ -0,0 +1,74 @@
+'use strict';
+/**
+ * A stand-in for OpenVibe.Events in the relay tests: the publish route Community's outbox uses and
+ * the pull route the relay's Events worker reads.
+ *
+ * POST /api/v1/events one envelope or { events: [...] } → { event_id, seq, duplicate } | { results }
+ * GET /api/v1/events?topic=a.*,b.*&after_seq=&limit= → { events: [{ seq, event }], next_after_seq, latest_seq, gap? }
+ *
+ * Topic patterns follow Events (`*` is one or more whole segments). Every call needs a Bearer token.
+ * `failNext` answers that many calls with 503; `prunedThrough` makes seq ≤ it look pruned (a gap).
+ */
+const http = require('http');
+
+function matcher(pattern) {
+ return new RegExp(`^${pattern.split('.').map((s) => (s === '*' ? '[a-z0-9_]+(?:\\.[a-z0-9_]+)*' : s)).join('\\.')}$`);
+}
+
+function startEvents() {
+ const log = []; // { seq, event }
+ const calls = [];
+ const state = { failNext: 0, prunedThrough: 0 };
+ const server = http.createServer((req, res) => {
+ let raw = '';
+ req.on('data', (c) => { raw += c; });
+ req.on('end', () => {
+ const u = new URL(req.url, 'http://events.test');
+ calls.push({ method: req.method, path: u.pathname + u.search, auth: req.headers.authorization || null });
+ const json = (status, obj) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(obj)); };
+ if (!/^Bearer \S+/.test(req.headers.authorization || '')) return json(401, { type: 'about:blank', code: 'auth.required', status: 401 });
+ if (state.failNext > 0) { state.failNext--; return json(503, { type: 'about:blank', code: 'events.unavailable', status: 503, detail: 'down for a test' }); }
+ if (u.pathname === '/api/v1/events' && req.method === 'POST') {
+ const body = JSON.parse(raw || '{}');
+ const batch = Array.isArray(body.events);
+ const results = (batch ? body.events : [body]).map((env) => {
+ const known = log.find((x) => x.event.event_id === env.event_id);
+ if (known) return { event_id: env.event_id, seq: known.seq, duplicate: true };
+ const seq = log.length + 1;
+ log.push({ seq, event: env });
+ return { event_id: env.event_id, seq, duplicate: false };
+ });
+ return json(201, batch ? { results } : results[0]);
+ }
+ if (u.pathname === '/api/v1/events' && req.method === 'GET') {
+ const patterns = String(u.searchParams.get('topic') || '*').split(',').map(matcher);
+ const after = Number(u.searchParams.get('after_seq') || 0);
+ const limit = Number(u.searchParams.get('limit') || 100);
+ const out = {};
+ let from = after;
+ if (state.prunedThrough && after < state.prunedThrough) { out.gap = { from_seq: after + 1, to_seq: state.prunedThrough }; from = state.prunedThrough; }
+ const rows = [];
+ let cursor = from;
+ for (const item of log) {
+ if (item.seq <= from) continue;
+ cursor = item.seq;
+ if (patterns.some((re) => re.test(item.event.event_type))) rows.push(item);
+ if (rows.length >= limit) break;
+ }
+ out.events = rows;
+ out.next_after_seq = cursor;
+ out.latest_seq = log.length;
+ return json(200, out);
+ }
+ return json(404, { type: 'about:blank', code: 'route.not_found', status: 404 });
+ });
+ });
+ /** Put an event straight into the log (as if some producer had published it). → seq */
+ function add(event) { log.push({ seq: log.length + 1, event }); return log.length; }
+ return new Promise((resolve) => server.listen(0, '127.0.0.1', () => resolve({
+ url: `http://127.0.0.1:${server.address().port}`, log, calls, state, add,
+ close: () => new Promise((r) => server.close(r)),
+ })));
+}
+
+module.exports = { startEvents };
new file mode 100644
new file mode 100644
@@ -0,0 +1,207 @@
@@ -0,0 +1,207 @@
+'use strict';
+/**
+ * The Discord relay as an Events worker (relay/events-worker.js): creates come from Community's own
+ * community.thread.created / community.post.created read back from OpenVibe.Events' pull API with a
+ * stored cursor. First start begins at the head (history is never announced); new events queue
+ * deliveries (source 'events', the event id) that are sent once; a replay queues nothing twice; the
+ * cursor survives a restart; members-only, Discord-origin, redacted and foreign events are ignored;
+ * Events being down or a retention gap shows in the status; while the worker runs the forum's own
+ * calls queue no creates. Last, the whole path in the app: an API write → Community's outbox → a fake
+ * Events → the worker → a fake Discord webhook.
+ */
+const assert = require('assert');
+const { ids } = require('openvibe-contracts');
+const { createClient } = require('openvibe-sdk/core');
+const { createEventsClient } = require('openvibe-sdk/events');
+const { boot, check, done } = require('./helpers/app');
+const { startWebhooks } = require('./helpers/fake-discord');
+const { startEvents } = require('./helpers/fake-events');
+const { openDb } = require('../server/db');
+const forumStore = require('../server/forum/store');
+const { createDiscordRelay } = require('../server/relay/discord');
+const { createRelayEventsWorker, CURSOR } = require('../server/relay/events-worker');
+
+let evn = 0;
+function envelope(type, payload, extra = {}) {
+ evn++;
+ return { event_id: `evt_TEST${String(evn).padStart(22, '0')}`, event_type: type, source: 'community', version: 1, timestamp: new Date().toISOString(), payload, ...extra };
+}
+const threadEvent = (t, visibility = 'public') => envelope('community.thread.created', { thread_id: t.id, space: 'general', visibility, author: null, url: null });
+const postEvent = (p, t, visibility = 'public') => envelope('community.post.created', { post_id: p.id, thread_id: t.id, space: 'general', visibility, author: null, url: null });
+
+(async () => {
+ const hook = await startWebhooks();
+ const events = await startEvents();
+ const db = openDb(':memory:');
+ const general = forumStore.getSpace(db, 'general');
+ const env = { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/1/general` };
+ const relay = createDiscordRelay({ db, config: { baseUrl: 'https://openvibe.community' }, env, enabled: true, baseMs: 1000, maxAttempts: 3 });
+ relay.addMapping({ space_id: general.id, webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' });
+ const client = createEventsClient(createClient({ baseUrls: { events: events.url }, getToken: async () => 'test-token', retries: 0 }), { source: 'community' });
+ const worker = createRelayEventsWorker({ db, relay, events: client, pollMs: 3_600_000, pageLimit: 2 });
+ relay.attach({ worker });
+ const alex = ids.newId('user');
+ const newThread = (title, extra = {}) => forumStore.createThread(db, { space_id: general.id, title, author_subject: alex, body_markdown: `${title} body`, ...extra }).thread;
+ const deliveries = () => db.prepare('SELECT * FROM relay_deliveries ORDER BY id').all();
+ const settle = () => new Promise((r) => setImmediate(r));
+
+ await check('off without Events or the client secret, or with DISCORD_RELAY_EVENTS=off; the forum queues creates then', async () => {
+ const a = createRelayEventsWorker({ db, relay, eventsUrl: null, clientSecret: 'x', tokenUrl: 'http://127.0.0.1:9/oauth/token' });
+ assert.deepStrictEqual([a.enabled, a.status().reason], [false, 'EVENTS_URL and OV_OAUTH_CLIENT_SECRET are not both set']);
+ const b = createRelayEventsWorker({ db, relay, eventsUrl: events.url, clientSecret: '', tokenUrl: 'http://127.0.0.1:9/oauth/token' });
+ assert.strictEqual(b.enabled, false);
+ const c = createRelayEventsWorker({ db, relay, events: client, enabled: false });
+ assert.deepStrictEqual([c.enabled, c.status().reason], [false, 'DISCORD_RELAY_EVENTS=off']);
+ const d = createRelayEventsWorker({ db, relay: createDiscordRelay({ db, env }), events: client });
+ assert.match(d.status().reason, /relay is off/);
+ c.start();
+ assert.strictEqual(relay.status().creates_from, 'forum');
+ assert.deepStrictEqual(await c.tick(), { queued: 0 });
+ });
+
+ let before;
+ await check('first start begins at the head: threads from before are never announced', async () => {
+ const old1 = newThread('Older one');
+ const old2 = newThread('Older two');
+ events.add(threadEvent(old1));
+ events.add(threadEvent(old2));
+ events.add(envelope('community.paste.created', { paste_id: 'x' }));
+ relay.start(); // starts the worker too (its interval is an hour here: ticks are driven by the test)
+ await worker.tick();
+ assert.strictEqual(relay.status().creates_from, 'events');
+ const st = worker.status();
+ assert.deepStrictEqual([st.enabled, st.cursor, st.latest_seq, st.started_at_seq, st.lag], [true, 3, 3, 3, 0]);
+ assert.strictEqual(deliveries().length, 0);
+ assert.deepStrictEqual(db.prepare('SELECT name, cursor FROM relay_cursors').all(), [{ name: CURSOR, cursor: 3 }]);
+ before = events.calls.length;
+ });
+
+ let t1;
+ let p1;
+ await check('new events queue creates (source events, the event id) that are sent once; the forum\'s own calls queue none', async () => {
+ t1 = newThread('From the stream');
+ assert.strictEqual(relay.enqueueThread(t1, general), 0, 'the forum queues no creates while the worker runs');
+ p1 = forumStore.addPost(db, { thread_id: t1.id, author_subject: alex, body_markdown: 'a reply via events' });
+ assert.strictEqual(relay.enqueuePost(p1, t1, general), 0);
+ const e1 = threadEvent(t1);
+ const e2 = postEvent(p1, t1);
+ events.add(e1);
+ events.add(envelope('community.comment.created', { comment_id: 1 })); // not on the relay's topics: never returned
+ events.add(e2);
+ assert.deepStrictEqual(await worker.tick(), { queued: 2 });
+ const rows = deliveries();
+ assert.deepStrictEqual(rows.map((d) => [d.action, d.source, d.event_id, d.post_id]), [['create', 'events', e1.event_id, null], ['create', 'events', e2.event_id, p1.id]]);
+ assert.ok(events.calls.slice(before).every((c) => /topic=community\.thread\.\*%2Ccommunity\.post\.\*/.test(c.path) && c.auth === 'Bearer test-token'), JSON.stringify(events.calls.slice(before)));
+ await settle(); await relay.drain(); await relay.drain();
+ assert.deepStrictEqual(hook.hits.map((h) => h.method), ['POST', 'POST']);
+ assert.match(hook.hits[1].body.content, /replied to \*\*From the stream\*\*/);
+ assert.deepStrictEqual([worker.status().cursor, worker.status().lag], [6, 0]);
+ });
+
+ await check('a replay (cursor moved back) queues nothing twice and posts nothing twice', async () => {
+ db.prepare('UPDATE relay_cursors SET cursor = 3 WHERE name = ?').run(CURSOR);
+ assert.deepStrictEqual(await worker.tick(), { queued: 0 });
+ await relay.drain();
+ assert.strictEqual(hook.hits.length, 2);
+ assert.strictEqual(worker.status().cursor, 6);
+ });
+
+ await check('the cursor survives a restart: a new worker carries on from it (no second "start at head")', async () => {
+ const t2 = newThread('While restarting');
+ events.add(threadEvent(t2));
+ const again = createRelayEventsWorker({ db, relay, events: client, pollMs: 3_600_000 });
+ assert.deepStrictEqual(await again.tick(), { queued: 1 });
+ assert.strictEqual(again.status().started_at_seq, null);
+ assert.strictEqual(deliveries().pop().thread_id, t2.id);
+ });
+
+ await check('members-only, Discord-origin, redacted and foreign events queue nothing', async () => {
+ const n = deliveries().length;
+ const gated = newThread('Gated', { members_only_owner: alex });
+ const fromDiscord = forumStore.addPost(db, { thread_id: t1.id, origin: 'discord', body_markdown: 'hi from discord', relay_author: 'Kim' });
+ const t3 = newThread('Looks public');
+ events.add(threadEvent(gated, 'members'));
+ events.add(threadEvent(gated)); // even a (wrongly) public-looking event: the row decides
+ events.add(postEvent(fromDiscord, t1));
+ events.add({ ...threadEvent(t3), source: 'live' });
+ events.add({ ...threadEvent(t3), payload: { redacted: true } });
+ assert.deepStrictEqual(await worker.tick(), { queued: 0 });
+ assert.strictEqual(deliveries().length, n);
+ });
+
+ await check('Events down: the error shows, the cursor stays, and it catches up once Events answers', async () => {
+ const t4 = newThread('After the outage');
+ events.add(threadEvent(t4));
+ const cursor = worker.status().cursor;
+ events.state.failNext = 1;
+ assert.deepStrictEqual(await worker.tick(), { queued: 0 });
+ const st = worker.status();
+ assert.ok(st.last_error, 'the error is shown');
+ assert.strictEqual(st.cursor, cursor);
+ assert.strictEqual(st.lag, st.latest_seq - cursor);
+ assert.deepStrictEqual(await worker.tick(), { queued: 1 });
+ assert.strictEqual(worker.status().last_error, null);
+ });
+
+ await check('a retention gap is logged and shown to staff; reading carries on after it', async () => {
+ db.prepare('UPDATE relay_cursors SET cursor = 1 WHERE name = ?').run(CURSOR);
+ events.state.prunedThrough = 5;
+ await worker.tick();
+ assert.deepStrictEqual([worker.status().last_gap.from_seq, worker.status().last_gap.to_seq], [2, 5]);
+ assert.strictEqual(worker.status().cursor, events.log.length);
+ events.state.prunedThrough = 0;
+ assert.strictEqual(relay.status().events_worker.last_gap.to_seq, 5);
+ });
+
+ await check('the whole path in the app: API write → outbox → Events → worker → Discord, and back out on edit and delete', async () => {
+ const hook2 = await startWebhooks();
+ const events2 = await startEvents();
+ process.env.EVENTS_URL = events2.url;
+ process.env.DISCORD_RELAY_EVENTS_POLL_MS = '40';
+ try {
+ const t = await boot({
+ authority: 'community',
+ appOpts: { relayOptions: { enabled: true, env: { DISCORD_WEBHOOK_GENERAL: `${hook2.url}/api/webhooks/7/app` } }, forumLimits: { threads: { cooldownSec: 0 }, posts: { cooldownSec: 0 } } },
+ });
+ const outbox = require('../server/events').init(t.db, { eventsUrl: events2.url, intervalMs: 40 });
+ assert.ok(outbox, 'the outbox publishes to the fake Events');
+ const net = t.network;
+ const samU = net.addUser({ network_user_id: 9, username: 'sam', display_name: 'Sam' });
+ const samJwt = net.sign({ id: 9, subject_id: samU.subject_id, username: 'sam', role: 'user' });
+ const adminJwt = net.sign({ id: 1, subject_id: ids.newId('user'), username: 'boss', role: 'admin' });
+ const call = (p, { method = 'GET', cookie, json } = {}) => t.get(p, { method, headers: json !== undefined ? { 'content-type': 'application/json' } : {}, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });
+ const waitFor = async (fn, what) => { for (let i = 0; i < 200; i++) { if (fn()) return; await new Promise((r) => setTimeout(r, 20)); } assert.fail(`timed out waiting for ${what}`); };
+ assert.strictEqual((await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' } })).status, 201);
+ await waitFor(() => t.app.locals.relay.status().events_worker.cursor != null, 'the worker\'s first read');
+ const st = (await call('/api/v1/relay/status', { cookie: adminJwt })).json();
+ assert.deepStrictEqual([st.creates_from, st.events_worker.enabled], ['events', true]);
+ const th = await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'Round trip', body: 'through Events' } });
+ assert.strictEqual(th.status, 201, th.text);
+ await waitFor(() => hook2.hits.length >= 1, 'the thread on Discord');
+ assert.strictEqual(hook2.hits[0].path, '/api/webhooks/7/app?wait=true');
+ assert.ok(events2.log.some((x) => x.event.event_type === 'community.thread.created'), 'it went through Events');
+ const d = t.db.prepare('SELECT * FROM relay_deliveries').get();
+ assert.deepStrictEqual([d.source, /^evt_/.test(d.event_id)], ['events', true]);
+ const reply = await call(`/api/v1/spaces/general/threads/${th.json().thread.slug}/posts`, { method: 'POST', cookie: samJwt, json: { body: 'and a reply' } });
+ await waitFor(() => hook2.hits.length >= 2, 'the reply on Discord');
+ assert.strictEqual((await call(`/api/v1/posts/${reply.json().post.id}`, { method: 'PUT', cookie: samJwt, json: { body: 'and a reply, edited' } })).status, 200);
+ await waitFor(() => hook2.hits.length >= 3, 'the edit on Discord');
+ assert.strictEqual((await call(`/api/v1/spaces/general/threads/${th.json().thread.slug}`, { method: 'DELETE', cookie: samJwt })).status, 200);
+ await waitFor(() => hook2.hits.length >= 5, 'the deletes on Discord');
+ assert.deepStrictEqual(hook2.hits.map((h) => h.method), ['POST', 'POST', 'PATCH', 'DELETE', 'DELETE']);
+ await new Promise((r) => setTimeout(r, 200));
+ assert.strictEqual(hook2.hits.length, 5, 'nothing was sent twice');
+ await t.app.locals.relay.stop();
+ await require('../server/events').stop();
+ await t.close();
+ } finally {
+ delete process.env.EVENTS_URL; delete process.env.DISCORD_RELAY_EVENTS_POLL_MS;
+ await hook2.close(); await events2.close();
+ }
+ });
+
+ await relay.stop();
+ await hook.close();
+ await events.close();
+ done();
+})();
new file mode 100644
new file mode 100644
@@ -0,0 +1,337 @@
@@ -0,0 +1,337 @@
+'use strict';
+/**
+ * Discord relay, inbound (relay/discord-gateway.js + relay/inbound.js), against a fake gateway:
+ * HELLO → IDENTIFY (token, intents GUILD_MESSAGES | MESSAGE_CONTENT), heartbeats with the last seq,
+ * RECONNECT and zombied connections resume, INVALID_SESSION identifies again, fatal close codes stop
+ * it with a reason, a missing HELLO reconnects, stop() closes. Then what becomes a post: replies to
+ * relayed messages (or messages in a Discord thread started from one) in mapped channels only, with
+ * origin 'discord', no author subject and the Discord name; mentions defused; size and rate limits;
+ * edits and deletes through the map; webhook/bot/own/system messages, unmapped channels and
+ * non-replies ignored; closed threads and empty messages recorded as failures; nothing from Discord
+ * ever relayed back out. Last, the app: status and failures at /api/v1/relay (staff, no token), and
+ * the post on the thread page with its Discord badge.
+ */
+const assert = require('assert');
+const { ids } = require('openvibe-contracts');
+const { boot, check, done } = require('./helpers/app');
+const { startWebhooks, createGateway } = require('./helpers/fake-discord');
+const { openDb } = require('../server/db');
+const forumStore = require('../server/forum/store');
+const { createDiscordRelay } = require('../server/relay/discord');
+const { createDiscordGateway, DEFAULT_URL, INTENTS } = require('../server/relay/discord-gateway');
+const { createDiscordInbound, toMarkdown } = require('../server/relay/inbound');
+const { createForumService } = require('../server/forum/service');
+
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+const BOT = '4000000000000000001';
+const KIM = '4000000000000000002';
+const LEE = '4000000000000000003';
+let snow = 5000000000000000000n;
+const nextId = () => String(snow++);
+
+/** A scripted gateway: HELLO on open, READY on IDENTIFY, RESUMED on RESUME, ACK on heartbeat (unless paused). */
+function scriptedGateway() {
+ const gw = createGateway();
+ gw.ack = true;
+ gw.seq = 0;
+ gw.onOpen = (sock) => gw.push(sock, { op: 10, d: { heartbeat_interval: 30 } });
+ gw.onSend = (sock, p) => {
+ if (p.op === 2) gw.push(sock, { op: 0, t: 'READY', s: ++gw.seq, d: { session_id: `sess-${gw.sockets.length}`, resume_gateway_url: 'wss://resume.discord.test', user: { id: BOT, bot: true } } });
+ if (p.op === 6) gw.push(sock, { op: 0, t: 'RESUMED', s: ++gw.seq, d: {} });
+ if (p.op === 1 && gw.ack) gw.push(sock, { op: 11 });
+ };
+ gw.dispatch = (t, d) => gw.push(gw.last(), { op: 0, t, s: ++gw.seq, d });
+ return gw;
+}
+const waitFor = async (fn, what, ms = 2000) => { const end = Date.now() + ms; while (Date.now() < end) { if (fn()) return; await sleep(5); } assert.fail(`timed out waiting for ${what}`); };
+const sent = (gw, op) => gw.sockets.flatMap((s) => s.sent).filter((p) => p.op === op);
+
+(async () => {
+ // ── the gateway client ───────────────────────────────────
+ const opts = { token: 'test-bot-token', minBackoffMs: 5, maxBackoffMs: 40, helloTimeoutMs: 150, invalidSessionMs: 5, jitter: () => 0 };
+
+ await check('off without a token or a WebSocket client, and says why', async () => {
+ const a = createDiscordGateway({ token: '' });
+ assert.deepStrictEqual([a.enabled, a.status()], [false, { enabled: false, reason: 'DISCORD_BOT_TOKEN is not set' }]);
+ const b = createDiscordGateway({ token: 'x', WebSocketImpl: null });
+ assert.match(b.status().reason, /no built-in WebSocket/);
+ b.start();
+ assert.strictEqual(typeof globalThis.WebSocket, 'function', 'Node 22 has the client the gateway uses');
+ });
+
+ await check('HELLO → IDENTIFY with the token and intents; heartbeats carry the last seq and are ACKed; READY is ready', async () => {
+ const gw = scriptedGateway();
+ const seen = [];
+ const g = createDiscordGateway({ ...opts, WebSocketImpl: gw.WebSocket, onDispatch: (t, d) => { seen.push([t, d.id]); } });
+ g.start();
+ await waitFor(() => g.status().state === 'ready', 'READY');
+ assert.strictEqual(gw.sockets[0].url, DEFAULT_URL);
+ const [identify] = sent(gw, 2);
+ assert.deepStrictEqual([identify.d.token, identify.d.intents], ['test-bot-token', INTENTS.GUILD_MESSAGES | INTENTS.MESSAGE_CONTENT]);
+ assert.strictEqual(INTENTS.GUILD_MESSAGES | INTENTS.MESSAGE_CONTENT, 33280);
+ await waitFor(() => sent(gw, 1).length >= 2, 'two heartbeats');
+ assert.strictEqual(sent(gw, 1).pop().d, 1, 'the last seq');
+ gw.dispatch('MESSAGE_CREATE', { id: '1' });
+ gw.dispatch('GUILD_CREATE', { id: '2' }); // not a message event: dropped here
+ gw.dispatch('MESSAGE_DELETE', { id: '3' });
+ assert.deepStrictEqual(seen, [['MESSAGE_CREATE', '1'], ['MESSAGE_DELETE', '3']]);
+ const st = g.status();
+ assert.deepStrictEqual([st.state, st.session, st.bot_user_id, st.events_handled], ['ready', true, BOT, 2]);
+ assert.ok(!JSON.stringify(st).includes('test-bot-token'), 'the token never shows');
+ await g.stop();
+ assert.strictEqual(gw.sockets[0].closedWith, 1000);
+ assert.strictEqual(g.status().state, 'stopped');
+ await sleep(60);
+ assert.strictEqual(gw.sockets.length, 1, 'no reconnect after stop()');
+ });
+
+ await check('RECONNECT, a zombied connection and an abnormal close all RESUME on the resume URL with the session and seq', async () => {
+ const gw = scriptedGateway();
+ const g = createDiscordGateway({ ...opts, WebSocketImpl: gw.WebSocket });
+ g.start();
+ await waitFor(() => g.status().state === 'ready', 'READY');
+ gw.dispatch('MESSAGE_CREATE', { id: '10' });
+ gw.push(gw.sockets[0], { op: 7, d: null });
+ await waitFor(() => gw.sockets.length === 2 && sent(gw, 6).length === 1, 'a resume');
+ assert.strictEqual(gw.sockets[0].closedWith, 4000, 'closed with a non-1000 code, so the session stays resumable');
+ assert.strictEqual(gw.sockets[1].url, 'wss://resume.discord.test/?v=10&encoding=json');
+ assert.deepStrictEqual(sent(gw, 6)[0].d, { token: 'test-bot-token', session_id: 'sess-1', seq: 2 });
+ await waitFor(() => g.status().state === 'ready', 'RESUMED');
+ // No ACK: the next heartbeat finds the last one unanswered and reconnects.
+ gw.ack = false;
+ await waitFor(() => gw.sockets.length === 3, 'a reconnect after a missing ACK', 1000);
+ gw.ack = true;
+ await waitFor(() => sent(gw, 6).length === 2, 'a second resume');
+ assert.match(g.status().last_reconnect.reason, /no heartbeat ACK \(zombied connection\)/);
+ // An abnormal close (1006) resumes too.
+ await waitFor(() => g.status().state === 'ready', 'ready again');
+ gw.drop(gw.last(), 1006);
+ await waitFor(() => sent(gw, 6).length === 3, 'a third resume');
+ assert.strictEqual(sent(gw, 2).length, 1, 'identified only once');
+ await g.stop();
+ });
+
+ await check('INVALID_SESSION (not resumable) and close codes 4007/4009 identify afresh; no HELLO reconnects', async () => {
+ const gw = scriptedGateway();
+ const g = createDiscordGateway({ ...opts, WebSocketImpl: gw.WebSocket });
+ g.start();
+ await waitFor(() => g.status().state === 'ready', 'READY');
+ gw.push(gw.last(), { op: 9, d: false });
+ await waitFor(() => sent(gw, 2).length === 2, 'a second IDENTIFY');
+ assert.strictEqual(gw.last().url, DEFAULT_URL, 'a fresh session starts on the main URL');
+ await waitFor(() => g.status().state === 'ready', 'READY again');
+ gw.drop(gw.last(), 4009);
+ await waitFor(() => sent(gw, 2).length === 3, 'IDENTIFY after 4009');
+ await waitFor(() => g.status().state === 'ready', 'READY after 4009');
+ // A gateway that never says HELLO.
+ gw.onOpen = null;
+ gw.drop(gw.last(), 1001);
+ await waitFor(() => gw.sockets.length >= 5, 'a reconnect after the HELLO timeout', 1500);
+ assert.match(g.status().last_error, /no HELLO/);
+ await g.stop();
+ });
+
+ await check('fatal close codes stop it with the reason (4004 a wrong token, 4014 the MESSAGE CONTENT intent)', async () => {
+ for (const [code, re] of [[4004, /DISCORD_BOT_TOKEN is wrong/], [4014, /MESSAGE CONTENT intent/]]) {
+ const gw = scriptedGateway();
+ const g = createDiscordGateway({ ...opts, WebSocketImpl: gw.WebSocket });
+ g.start();
+ await waitFor(() => g.status().state === 'ready', 'READY');
+ gw.drop(gw.last(), code);
+ await sleep(80);
+ assert.strictEqual(gw.sockets.length, 1, `no reconnect after ${code}`);
+ assert.strictEqual(g.status().state, 'failed');
+ assert.match(g.status().last_error, re);
+ await g.stop();
+ }
+ });
+
+ // ── inbound: what becomes a post ────────────────────────
+ const hook = await startWebhooks();
+ const db = openDb(':memory:');
+ const general = forumStore.getSpace(db, 'general');
+ const relay = createDiscordRelay({ db, config: { baseUrl: 'https://openvibe.community' }, env: { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/1/general` }, enabled: true });
+ relay.addMapping({ space_id: general.id, webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL', inbound: true });
+ const CHANNEL = '2000000000000000001'; // the fake webhook 1's channel (learned on the first send)
+ const inbound = createDiscordInbound({ db, perMinute: 3, maxChars: 200 });
+ const forum = createForumService({ db, relay, limits: { threads: { cooldownSec: 0, perMinute: 100 }, posts: { cooldownSec: 0, perMinute: 100 }, threadsPerDay: 0 } });
+ const sam = ids.newId('user');
+ const samV = { kind: 'user', subject: sam, staff: false };
+ const settle = async () => { await new Promise((r) => setImmediate(r)); await relay.drain(); await relay.drain(); };
+ const { thread } = await forum.createThread(samV, 'general', { title: 'Talk to Discord', body: 'Say hi over there' });
+ await settle();
+ const threadMsg = db.prepare("SELECT external_message_id FROM relay_message_map WHERE local_type = 'thread' AND local_id = ?").get(thread.id).external_message_id;
+ const failures = () => db.prepare('SELECT * FROM relay_inbound_failures ORDER BY id').all();
+ const msg = (extra = {}) => ({ id: nextId(), channel_id: CHANNEL, guild_id: '6000000000000000001', type: 19, content: 'hello from Discord', author: { id: KIM, username: 'kim', global_name: 'Kim' }, message_reference: { message_id: threadMsg, channel_id: CHANNEL }, mentions: [], attachments: [], ...extra });
+ const postsOf = (tid) => db.prepare('SELECT * FROM posts WHERE thread_id = ? ORDER BY id').all(tid);
+
+ await check('a Discord reply to a relayed message becomes a post: origin discord, nobody on the site, the Discord name, mapped', async () => {
+ const m = msg({ content: 'Hi <@4000000000000000003> and <@&4000000000000000009> in <#4000000000000000010> @everyone <:wave:4000000000000000011> <t:1790000000:R>', mentions: [{ id: LEE, username: 'lee', global_name: 'Lee' }], member: { nick: 'Kim (mod)' } });
+ const r = inbound.handle('MESSAGE_CREATE', m, { botUserId: BOT });
+ assert.strictEqual(r.status, 'applied', JSON.stringify(r));
+ const p = forumStore.getPost(db, r.post_id);
+ assert.deepStrictEqual([p.thread_id, p.origin, p.author_subject, p.relay_author], [thread.id, 'discord', null, 'Kim (mod)']);
+ assert.strictEqual(p.body_markdown, 'Hi @Lee and @role in #channel @\u200beveryone :wave: 2026-09-21 14:13 UTC');
+ const row = db.prepare('SELECT * FROM relay_message_map WHERE external_message_id = ?').get(m.id);
+ assert.deepStrictEqual([row.direction, row.local_type, row.local_id, row.thread_id, row.external_channel_id], ['in', 'post', p.id, thread.id, CHANNEL]);
+ const shown = (await forum.getThread(samV, 'general', thread.slug, {})).posts.find((x) => x.id === p.id);
+ assert.deepStrictEqual([shown.author.display_name, shown.author.is_relay, shown.author.subject, shown.origin], ['Kim (mod)', true, null, 'discord']);
+ // The same message again (a gateway replay) is not taken twice.
+ assert.deepStrictEqual(inbound.handle('MESSAGE_CREATE', m, { botUserId: BOT }), { status: 'ignored', reason: 'already relayed' });
+ });
+
+ await check('a message in a Discord thread started from a relayed message becomes a post in that thread', async () => {
+ const r = inbound.handle('MESSAGE_CREATE', msg({ channel_id: threadMsg, type: 0, message_reference: undefined, content: 'in the Discord thread' }));
+ assert.strictEqual(r.status, 'applied');
+ assert.strictEqual(forumStore.getPost(db, r.post_id).thread_id, thread.id);
+ // A reply to that Discord-origin post (mapped 'in') lands in the same thread too.
+ const inId = db.prepare("SELECT external_message_id FROM relay_message_map WHERE local_id = ? AND direction = 'in'").get(r.post_id).external_message_id;
+ const r2 = inbound.handle('MESSAGE_CREATE', msg({ channel_id: threadMsg, message_reference: { message_id: inId }, content: 'reply to a reply', author: { id: LEE, username: 'lee' } }));
+ assert.strictEqual(forumStore.getPost(db, r2.post_id).relay_author, 'lee');
+ });
+
+ await check('everything else is ignored: webhooks (the relay\'s own), bots, the bot itself, system messages, non-replies, other channels, inbound off', async () => {
+ const n = postsOf(thread.id).length;
+ const cases = [
+ [msg({ webhook_id: '1' }), /webhook message/],
+ [msg({ author: { id: LEE, username: 'lee', bot: true } }), /bot or system/],
+ [msg({ author: { id: BOT, username: 'relaybot' } }), /bot or system/],
+ [msg({ type: 7 }), /not a plain message/],
+ [msg({ message_reference: undefined, type: 0 }), /not a reply to a relayed message/],
+ [msg({ message_reference: { message_id: '9999999999999999999' } }), /not a reply to a relayed message/],
+ [msg({ channel_id: '7000000000000000001' }), /not a reply to a relayed message/],
+ [msg({ type: 0, content: '', message_reference: { type: 1, message_id: threadMsg, channel_id: CHANNEL } }), /not a reply to a relayed message/], // a forward
+ ];
+ for (const [m, re] of cases) {
+ const r = inbound.handle('MESSAGE_CREATE', m, { botUserId: BOT });
+ assert.strictEqual(r.status, 'ignored', JSON.stringify(m));
+ assert.match(r.reason, re);
+ }
+ relay.updateMapping(1, { inbound: false });
+ assert.match(inbound.handle('MESSAGE_CREATE', msg()).reason, /inbound is off/);
+ relay.updateMapping(1, { inbound: true, discord_channel_id: '7000000000000000002' });
+ assert.match(inbound.handle('MESSAGE_CREATE', msg()).reason, /not a mapped channel/);
+ relay.updateMapping(1, { discord_channel_id: CHANNEL });
+ assert.strictEqual(postsOf(thread.id).length, n);
+ assert.strictEqual(failures().length, 0, 'ignoring is not a failure');
+ });
+
+ await check('limits: a long message is cut, a fourth message a minute is refused, an empty one explains the intent; attachments are noted', async () => {
+ const long = inbound.handle('MESSAGE_CREATE', msg({ author: { id: '4000000000000000020', username: 'long' }, content: 'x'.repeat(500) }));
+ const body = forumStore.getPost(db, long.post_id).body_markdown;
+ assert.strictEqual(body.length, 200);
+ assert.ok(body.endsWith('…'));
+ const talker = { id: '4000000000000000021', username: 'talker' };
+ for (let i = 0; i < 3; i++) assert.strictEqual(inbound.handle('MESSAGE_CREATE', msg({ author: talker, content: `m${i}` })).status, 'applied');
+ const r = inbound.handle('MESSAGE_CREATE', msg({ author: talker, content: 'one too many' }));
+ assert.deepStrictEqual([r.status, r.error], ['failed', 'rate limited']);
+ const empty = inbound.handle('MESSAGE_CREATE', msg({ author: { id: '4000000000000000022', username: 'quiet' }, content: '' }));
+ assert.match(empty.error, /MESSAGE CONTENT intent/);
+ const pic = inbound.handle('MESSAGE_CREATE', msg({ author: { id: '4000000000000000023', username: 'pics' }, content: '', attachments: [{ id: '1' }, { id: '2' }] }));
+ assert.strictEqual(forumStore.getPost(db, pic.post_id).body_markdown, '_(2 attachments on Discord)_');
+ const f = failures();
+ assert.deepStrictEqual(f.map((x) => x.error.slice(0, 12)), ['rate limited', 'the message ']);
+ assert.deepStrictEqual([f[0].event, f[0].external_author_id, f[0].mapping_id], ['MESSAGE_CREATE', talker.id, 1]);
+ assert.ok(!JSON.stringify(f).includes('one too many'), 'failures never keep the text');
+ assert.strictEqual(toMarkdown({ content: ' <@123456789012345678> ' }), '@someone');
+ });
+
+ await check('closed threads refuse replies from Discord and say why: locked, members-only, deleted', async () => {
+ const t = (await forum.createThread(samV, 'general', { title: 'Soon closed', body: 'x' })).thread;
+ await settle();
+ const anchor = db.prepare("SELECT external_message_id FROM relay_message_map WHERE local_type = 'thread' AND local_id = ?").get(t.id).external_message_id;
+ const to = (extra = {}) => msg({ author: { id: '4000000000000000030', username: 'late' }, message_reference: { message_id: anchor }, ...extra });
+ forumStore.setThreadFlags(db, t.id, { locked: true });
+ assert.strictEqual(inbound.handle('MESSAGE_CREATE', to()).error, 'the thread is locked');
+ forumStore.setThreadFlags(db, t.id, { locked: false });
+ forumStore.setThreadMembersOnly(db, t.id, sam);
+ assert.strictEqual(inbound.handle('MESSAGE_CREATE', to()).error, 'the thread is not public');
+ forumStore.setThreadMembersOnly(db, t.id, null);
+ forumStore.softDeleteThread(db, t.id);
+ assert.strictEqual(inbound.handle('MESSAGE_CREATE', to()).error, 'the thread was deleted');
+ assert.strictEqual(postsOf(t.id).length, 1, 'only the opening post');
+ });
+
+ await check('edits and deletes on Discord follow through the map; a delete of the relay\'s own message leaves the thread', async () => {
+ const m = msg({ author: { id: '4000000000000000040', username: 'editor' }, content: 'first words' });
+ const { post_id: pid } = inbound.handle('MESSAGE_CREATE', m);
+ const r = inbound.handle('MESSAGE_UPDATE', { id: m.id, channel_id: CHANNEL, content: 'second words <@4000000000000000003>', mentions: [{ id: LEE, username: 'lee' }], edited_timestamp: new Date().toISOString() });
+ assert.strictEqual(r.status, 'applied');
+ const p = forumStore.getPost(db, pid);
+ assert.deepStrictEqual([p.body_markdown, p.revision], ['second words @lee', 2]);
+ assert.deepStrictEqual(forumStore.listPostVersions(db, pid).map((v) => v.edited_by), [null, 'discord']);
+ assert.match(inbound.handle('MESSAGE_UPDATE', { id: m.id, channel_id: CHANNEL, embeds: [] }).reason, /no text change/);
+ assert.match(inbound.handle('MESSAGE_UPDATE', { id: threadMsg, channel_id: CHANNEL, content: 'edited on Discord?' }).reason, /relay's own/);
+ const other = inbound.handle('MESSAGE_CREATE', msg({ author: { id: '4000000000000000041', username: 'bulk' }, content: 'bulk me' }));
+ const otherId = db.prepare("SELECT external_message_id FROM relay_message_map WHERE local_id = ? AND direction = 'in'").get(other.post_id).external_message_id;
+ assert.strictEqual(inbound.handle('MESSAGE_DELETE', { id: m.id, channel_id: CHANNEL }).status, 'applied');
+ assert.ok(forumStore.getPost(db, pid).deleted_at, 'the post is deleted');
+ const bulk = inbound.handle('MESSAGE_DELETE_BULK', { ids: [otherId, '1234567890123456789'], channel_id: CHANNEL });
+ assert.deepStrictEqual(bulk.map((x) => x.status), ['applied', 'ignored']);
+ assert.ok(forumStore.getPost(db, other.post_id).deleted_at);
+ // A moderator on Discord removed the relay's announcement: the map knows, the thread stays, no edit goes there again.
+ assert.match(inbound.handle('MESSAGE_DELETE', { id: threadMsg, channel_id: CHANNEL }).note, /thread stays/);
+ assert.ok(forumStore.getThread(db, thread.id), 'the thread is still here');
+ assert.ok(db.prepare('SELECT external_deleted_at FROM relay_message_map WHERE external_message_id = ?').get(threadMsg).external_deleted_at);
+ assert.strictEqual(relay.enqueueEdit(forumStore.getPost(db, db.prepare('SELECT id FROM posts WHERE thread_id = ? AND is_opening = 1').get(thread.id).id)), 0);
+ });
+
+ await check('loop prevention: nothing from Discord is ever relayed back out', async () => {
+ await settle();
+ const before = hook.hits.length;
+ const fromDiscord = postsOf(thread.id).filter((p) => p.origin === 'discord');
+ assert.ok(fromDiscord.length >= 5);
+ for (const p of fromDiscord) assert.strictEqual(relay.enqueuePost(p, forumStore.getThread(db, thread.id), general), 0);
+ assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM relay_deliveries WHERE post_id IN (SELECT id FROM posts WHERE origin = \'discord\')').get().n, 0);
+ await settle();
+ assert.strictEqual(hook.hits.length, before, 'no webhook call for anything that came from Discord');
+ });
+
+ await check('the app: the gateway starts with the relay; status and failures for staff only, without the token; the post shows with a Discord badge', async () => {
+ const gw = scriptedGateway();
+ process.env.DISCORD_RELAY_INBOUND = 'on';
+ process.env.DISCORD_BOT_TOKEN = 'app-bot-token-not-real';
+ let t;
+ try {
+ t = await boot({
+ authority: 'community',
+ appOpts: { relayOptions: { enabled: true, env: { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/5/app` } }, gatewayOptions: { WebSocketImpl: gw.WebSocket, minBackoffMs: 5, jitter: () => 0 }, forumLimits: { threads: { cooldownSec: 0 } } },
+ });
+ } finally { delete process.env.DISCORD_RELAY_INBOUND; delete process.env.DISCORD_BOT_TOKEN; }
+ const net = t.network;
+ const samU = net.addUser({ network_user_id: 9, username: 'sam', display_name: 'Sam' });
+ const samJwt = net.sign({ id: 9, subject_id: samU.subject_id, username: 'sam', role: 'user' });
+ const adminJwt = net.sign({ id: 1, subject_id: ids.newId('user'), username: 'boss', role: 'admin' });
+ const call = (p, { method = 'GET', cookie, json } = {}) => t.get(p, { method, headers: json !== undefined ? { 'content-type': 'application/json' } : {}, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });
+ await waitFor(() => t.app.locals.relay.status().inbound.state === 'ready', 'the app\'s gateway READY');
+ assert.strictEqual(sent(gw, 2)[0].d.token, 'app-bot-token-not-real');
+ const m = await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL', inbound: true } });
+ assert.strictEqual(m.json().mapping.inbound, true);
+ const th = await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'Ask Discord', body: 'what do you think?' } });
+ await waitFor(() => t.db.prepare("SELECT 1 FROM relay_message_map WHERE local_type = 'thread'").get(), 'the thread on Discord');
+ const anchor = t.db.prepare("SELECT external_message_id, external_channel_id FROM relay_message_map WHERE local_type = 'thread'").get();
+ gw.dispatch('MESSAGE_CREATE', { id: nextId(), channel_id: anchor.external_channel_id, type: 19, content: 'Looks **great**', author: { id: KIM, username: 'kim', global_name: 'Kim' }, message_reference: { message_id: anchor.external_message_id } });
+ const locked = nextId();
+ const page = await call(`/s/general/t/${th.json().thread.slug}`);
+ assert.strictEqual(page.status, 200);
+ assert.ok(page.text.includes('badge-relay') && page.text.includes('Kim') && page.text.includes('<strong>great</strong>'), 'the reply shows with its Discord badge');
+ t.db.prepare('UPDATE threads SET locked = 1').run();
+ gw.dispatch('MESSAGE_CREATE', { id: locked, channel_id: anchor.external_channel_id, type: 19, content: 'too late', author: { id: KIM, username: 'kim' }, message_reference: { message_id: anchor.external_message_id } });
+ assert.strictEqual((await call('/api/v1/relay/inbound', { cookie: samJwt })).status, 403);
+ const inb = await call('/api/v1/relay/inbound', { cookie: adminJwt });
+ assert.strictEqual(inb.status, 200);
+ assert.deepStrictEqual(inb.json().failures.map((f) => [f.error, f.discord.message_id, f.mapping.space]), [['the thread is locked', locked, 'general']]);
+ assert.strictEqual((await call(`/api/v1/relay/inbound/${inb.json().failures[0].id}/dismiss`, { method: 'POST', cookie: adminJwt })).status, 200);
+ assert.strictEqual((await call('/api/v1/relay/inbound', { cookie: adminJwt })).json().failures.length, 0);
+ assert.strictEqual((await call('/api/v1/relay/inbound?all=1', { cookie: adminJwt })).json().failures.length, 1);
+ const st = await call('/api/v1/relay/status', { cookie: adminJwt });
+ assert.deepStrictEqual([st.json().inbound.enabled, st.json().inbound.state, st.json().inbound.handled.applied, st.json().inbound_failures], [true, 'ready', 1, 0]);
+ assert.ok(!(st.text + inb.text + (await call('/api/ready')).text).includes('app-bot-token-not-real'), 'the token never shows');
+ await t.app.locals.relay.stop();
+ assert.strictEqual(gw.last().closedWith, 1000);
+ await t.close();
+ });
+
+ await hook.close();
+ done();
+})();
@@ -1,41 +1,31 @@
@@ -1,41 +1,31 @@
'use strict';
'use strict';
/**
/**
- * Discord relay (outbound): off unless DISCORD_RELAY_ENABLED; one delivery per (thread, mapping);
- * the webhook URL comes from the environment variable a mapping names; retries with backoff on
- * 5xx/429/network errors, immediate failure on other 4xx; loop prevention for Discord-origin
- * threads; members/staff spaces never relayed; the staff admin endpoint. A stub webhook server
- * stands in for Discord; the relay's clock is injected so backoff is tested without waiting.
+ * Discord relay, outbound (relay/discord.js): off unless DISCORD_RELAY_ENABLED; one delivery per
+ * dedupe key; the webhook URL comes from the environment variable a mapping names; creates are sent
+ * with ?wait=true and the answer's message id goes into the external message map (unique both ways);
+ * replies follow their thread (into the mapping's Discord thread when it names one); edits PATCH and
+ * deletes DELETE through the map; retries with backoff on 5xx/429/network errors, then the dead letter
+ * ('failed') that staff retry or drop; loop prevention for Discord-origin threads and posts; members,
+ * staff and VIP members-only content never relayed (and deleted when gated later); the staff API;
+ * the one-time rebuild of an older relay_deliveries table. A fake Discord stands in for the webhooks;
+ * the relay's clock is injected so backoff is tested without waiting. The forum queues creates
+ * directly here (no Events worker: see relay-events.test.js).
*/
*/
const assert = require('assert');
const assert = require('assert');
-const http = require('http');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
const { ids } = require('openvibe-contracts');
const { ids } = require('openvibe-contracts');
const { boot, check, done } = require('./helpers/app');
const { boot, check, done } = require('./helpers/app');
+const { startWebhooks } = require('./helpers/fake-discord');
const { openDb } = require('../server/db');
const { openDb } = require('../server/db');
const forumStore = require('../server/forum/store');
const forumStore = require('../server/forum/store');
const { createDiscordRelay } = require('../server/relay/discord');
const { createDiscordRelay } = require('../server/relay/discord');
+const { createForumService } = require('../server/forum/service');
const { sqlTime } = require('../server/http/v1');
const { sqlTime } = require('../server/http/v1');
-function stubWebhook() {
- const hits = [];
- const plan = []; // queued responses: { status, body, headers }
- const server = http.createServer((req, res) => {
- let raw = '';
- req.on('data', (c) => { raw += c; });
- req.on('end', () => {
- hits.push({ path: req.url, body: raw ? JSON.parse(raw) : null });
- const next = plan.shift() || { status: 204 };
- if (next.hang) return; // never answers
- res.writeHead(next.status, { 'Content-Type': 'application/json', ...(next.headers || {}) });
- res.end(next.body ? JSON.stringify(next.body) : '');
- });
- });
- return new Promise((resolve) => server.listen(0, '127.0.0.1', () => resolve({
- url: `http://127.0.0.1:${server.address().port}`, hits, plan, close: () => new Promise((r) => server.close(r)),
- })));
-}
-
(async () => {
(async () => {
- const hook = await stubWebhook();
+ const hook = await startWebhooks();
const db = openDb(':memory:');
const db = openDb(':memory:');
const general = forumStore.getSpace(db, 'general');
const general = forumStore.getSpace(db, 'general');
const feedback = forumStore.getSpace(db, 'feedback');
const feedback = forumStore.getSpace(db, 'feedback');
@@ -49,17 +39,23 @@ function stubWebhook() {
@@ -49,17 +39,23 @@ function stubWebhook() {
relay.addMapping({ space_id: feedback.id, webhook_url_ref: 'DISCORD_WEBHOOK_FEEDBACK' });
relay.addMapping({ space_id: feedback.id, webhook_url_ref: 'DISCORD_WEBHOOK_FEEDBACK' });
const newThread = (space, title, extra = {}) => forumStore.createThread(db, { space_id: space.id, title, author_subject: alex, body_markdown: '**Hello** there, [link](https://x.y)', ...extra }).thread;
const newThread = (space, title, extra = {}) => forumStore.createThread(db, { space_id: space.id, title, author_subject: alex, body_markdown: '**Hello** there, [link](https://x.y)', ...extra }).thread;
const deliveries = () => db.prepare('SELECT * FROM relay_deliveries ORDER BY id').all();
const deliveries = () => db.prepare('SELECT * FROM relay_deliveries ORDER BY id').all();
+ const mapOf = (type, id) => db.prepare('SELECT * FROM relay_message_map WHERE local_type = ? AND local_id = ? ORDER BY id').all(type, id);
const settle = () => new Promise((r) => setImmediate(r));
const settle = () => new Promise((r) => setImmediate(r));
+ const EMPTY = { delivered: 0, retry: 0, failed: 0, skipped: 0 };
await check('off by default: nothing is queued or sent', async () => {
await check('off by default: nothing is queued or sent', async () => {
const off = createDiscordRelay({ db, env, now });
const off = createDiscordRelay({ db, env, now });
assert.strictEqual(off.enabled, false);
assert.strictEqual(off.enabled, false);
- assert.strictEqual(off.enqueueThread(newThread(general, 'Quiet thread'), general), 0);
- assert.deepStrictEqual(await off.drain(), { delivered: 0, retry: 0, failed: 0 });
+ const th = newThread(general, 'Quiet thread');
+ assert.strictEqual(off.enqueueThread(th, general), 0);
+ assert.strictEqual(off.enqueuePost(forumStore.addPost(db, { thread_id: th.id, author_subject: alex, body_markdown: 'shh' }), th, general), 0);
+ assert.deepStrictEqual(await off.drain(), EMPTY);
assert.strictEqual(deliveries().length, 0);
assert.strictEqual(deliveries().length, 0);
+ const st = off.status();
+ assert.deepStrictEqual([st.enabled, st.events_worker.enabled, st.inbound.enabled], [false, false, false]);
});
});
- await check('a new thread in a mapped space is posted once, with attribution, a link, and no mentions', async () => {
+ await check('a new thread is posted once with ?wait=true, attribution, a link and no mentions; its message id is mapped', async () => {
const th = newThread(general, 'Relay me <please>');
const th = newThread(general, 'Relay me <please>');
assert.strictEqual(relay.enqueueThread(th, general), 1);
assert.strictEqual(relay.enqueueThread(th, general), 1);
assert.strictEqual(relay.enqueueThread(th, general), 0, 'dedupe key per (thread, mapping)');
assert.strictEqual(relay.enqueueThread(th, general), 0, 'dedupe key per (thread, mapping)');
@@ -67,7 +63,8 @@ function stubWebhook() {
@@ -67,7 +63,8 @@ function stubWebhook() {
await relay.drain();
await relay.drain();
assert.strictEqual(hook.hits.length, 1);
assert.strictEqual(hook.hits.length, 1);
const msg = hook.hits[0];
const msg = hook.hits[0];
- assert.strictEqual(msg.path, '/api/webhooks/1/general');
+ assert.strictEqual(msg.method, 'POST');
+ assert.strictEqual(msg.path, '/api/webhooks/1/general?wait=true');
assert.deepStrictEqual(msg.body.allowed_mentions, { parse: [] });
assert.deepStrictEqual(msg.body.allowed_mentions, { parse: [] });
assert.match(msg.body.content, /^New thread in \*\*s\/general\*\* by Alex @\u200beveryone: <https:\/\/openvibe\.community\/s\/general\/t\/relay-me-please>$/);
assert.match(msg.body.content, /^New thread in \*\*s\/general\*\* by Alex @\u200beveryone: <https:\/\/openvibe\.community\/s\/general\/t\/relay-me-please>$/);
assert.strictEqual(msg.body.embeds[0].title, 'Relay me <please>');
assert.strictEqual(msg.body.embeds[0].title, 'Relay me <please>');
@@ -75,9 +72,30 @@ function stubWebhook() {
@@ -75,9 +72,30 @@ function stubWebhook() {
assert.strictEqual(msg.body.embeds[0].description, 'Hello there, link');
assert.strictEqual(msg.body.embeds[0].description, 'Hello there, link');
assert.strictEqual(msg.body.embeds[0].footer.text, 'OpenVibe.Community · s/general');
assert.strictEqual(msg.body.embeds[0].footer.text, 'OpenVibe.Community · s/general');
const d = deliveries().find((x) => x.thread_id === th.id);
const d = deliveries().find((x) => x.thread_id === th.id);
- assert.deepStrictEqual([d.status, d.attempts, d.last_status, d.dedupe_key], ['delivered', 1, 204, `thread:${th.id}:mapping:1`]);
+ assert.deepStrictEqual([d.status, d.attempts, d.last_status, d.dedupe_key, d.action, d.source], ['delivered', 1, 200, `thread:${th.id}:mapping:1`, 'create', 'direct']);
+ const [row] = mapOf('thread', th.id);
+ assert.deepStrictEqual([row.platform, row.direction, row.mapping_id, row.thread_id, row.external_channel_id, row.external_thread_id, row.external_webhook_id],
+ ['discord', 'out', 1, th.id, '2000000000000000001', null, '1']);
+ assert.ok(hook.messages.has(row.external_message_id));
+ assert.strictEqual(db.prepare('SELECT discord_channel_id FROM relay_mappings WHERE id = 1').get().discord_channel_id, '2000000000000000001', 'the mapping learned its channel');
await relay.drain();
await relay.drain();
assert.strictEqual(hook.hits.length, 1, 'a delivered thread is never sent again');
assert.strictEqual(hook.hits.length, 1, 'a delivered thread is never sent again');
+ // Even a create queued again under another key is not sent twice: the map already has it.
+ db.prepare("INSERT INTO relay_deliveries (thread_id, mapping_id, dedupe_key, next_attempt_at) VALUES (?, 1, 'replay-of-the-same', ?)").run(th.id, sqlTime(clock));
+ await relay.drain();
+ assert.strictEqual(hook.hits.length, 1);
+ assert.deepStrictEqual([deliveries().pop().status, deliveries().pop().last_error], ['delivered', 'already on Discord']);
+ });
+
+ await check('the external message map is unique both ways', async () => {
+ const th = newThread(general, 'Unique map');
+ const put = (mapping, type, local, external) => db.prepare(`INSERT INTO relay_message_map (mapping_id, direction, local_type, local_id, thread_id, external_channel_id, external_message_id)
+ VALUES (?, 'out', ?, ?, ?, '2000000000000000001', ?)`).run(mapping, type, local, th.id, external);
+ put(1, 'thread', th.id, '1900000000000000001');
+ assert.throws(() => put(2, 'post', 999, '1900000000000000001'), /UNIQUE/, 'one Discord message is one local object');
+ assert.throws(() => put(1, 'thread', th.id, '1900000000000000002'), /UNIQUE/, 'one local object is one message per mapping');
+ put(2, 'thread', th.id, '1900000000000000003'); // the same thread through another mapping is fine
+ db.prepare('DELETE FROM relay_message_map WHERE thread_id = ?').run(th.id);
});
});
await check('retries with exponential backoff on 5xx, then delivers', async () => {
await check('retries with exponential backoff on 5xx, then delivers', async () => {
@@ -104,14 +122,20 @@ function stubWebhook() {
@@ -104,14 +122,20 @@ function stubWebhook() {
assert.strictEqual(hook.hits.length, 3);
assert.strictEqual(hook.hits.length, 3);
});
});
- await check('429 honours retry_after; exhausted attempts end as failed; other 4xx fail at once', async () => {
+ await check('429 honours retry_after and holds that webhook for the pass; used-up attempts end as failed; other 4xx fail at once', async () => {
const th = newThread(general, 'Rate limited');
const th = newThread(general, 'Rate limited');
+ const th2 = newThread(general, 'Behind it');
hook.plan.push({ status: 429, body: { message: 'You are being rate limited.', retry_after: 300 } });
hook.plan.push({ status: 429, body: { message: 'You are being rate limited.', retry_after: 300 } });
relay.enqueueThread(th, general);
relay.enqueueThread(th, general);
+ relay.enqueueThread(th2, general);
+ const before = hook.hits.length;
await settle(); await relay.drain();
await settle(); await relay.drain();
+ assert.strictEqual(hook.hits.length, before + 1, 'the next delivery to the same webhook waits for the next pass');
let d = deliveries().find((x) => x.thread_id === th.id);
let d = deliveries().find((x) => x.thread_id === th.id);
assert.strictEqual(d.status, 'pending');
assert.strictEqual(d.status, 'pending');
assert.strictEqual(Date.parse(`${d.next_attempt_at.replace(' ', 'T')}Z`) - clock, 300_000);
assert.strictEqual(Date.parse(`${d.next_attempt_at.replace(' ', 'T')}Z`) - clock, 300_000);
+ await relay.drain();
+ assert.strictEqual(deliveries().find((x) => x.thread_id === th2.id).status, 'delivered');
clock += 300_000;
clock += 300_000;
hook.plan.push({ status: 503 }, { status: 503 });
hook.plan.push({ status: 503 }, { status: 503 });
await relay.drain();
await relay.drain();
@@ -121,7 +145,7 @@ function stubWebhook() {
@@ -121,7 +145,7 @@ function stubWebhook() {
assert.deepStrictEqual([d.status, d.attempts, d.last_status], ['failed', 3, 503]);
assert.deepStrictEqual([d.status, d.attempts, d.last_status], ['failed', 3, 503]);
const gone = newThread(general, 'Webhook deleted');
const gone = newThread(general, 'Webhook deleted');
- hook.plan.push({ status: 404, body: { message: 'Unknown Webhook' } });
+ hook.plan.push({ status: 404, body: { message: 'Unknown Webhook', code: 10015 } });
relay.enqueueThread(gone, general);
relay.enqueueThread(gone, general);
await settle(); await relay.drain();
await settle(); await relay.drain();
d = deliveries().find((x) => x.thread_id === gone.id);
d = deliveries().find((x) => x.thread_id === gone.id);
@@ -148,11 +172,11 @@ function stubWebhook() {
@@ -148,11 +172,11 @@ function stubWebhook() {
await relay.drain();
await relay.drain();
d = deliveries().find((x) => x.thread_id === th.id);
d = deliveries().find((x) => x.thread_id === th.id);
assert.strictEqual(d.status, 'delivered');
assert.strictEqual(d.status, 'delivered');
- const everything = JSON.stringify(db.prepare('SELECT * FROM relay_mappings').all()) + JSON.stringify(deliveries());
+ const everything = JSON.stringify(db.prepare('SELECT * FROM relay_mappings').all()) + JSON.stringify(deliveries()) + JSON.stringify(db.prepare('SELECT * FROM relay_message_map').all());
assert.ok(!everything.includes('/api/webhooks/'), 'no webhook URL stored anywhere');
assert.ok(!everything.includes('/api/webhooks/'), 'no webhook URL stored anywhere');
});
});
- await check('loop prevention: Discord-origin threads are never relayed; members/staff spaces never leave the site', async () => {
+ await check('loop prevention: Discord-origin threads and posts are never relayed; members/staff spaces never leave the site', async () => {
const before = hook.hits.length;
const before = hook.hits.length;
const fromDiscord = newThread(general, 'Came from Discord', { origin: 'discord' });
const fromDiscord = newThread(general, 'Came from Discord', { origin: 'discord' });
assert.strictEqual(relay.enqueueThread(fromDiscord, general), 0);
assert.strictEqual(relay.enqueueThread(fromDiscord, general), 0);
@@ -161,14 +185,23 @@ function stubWebhook() {
@@ -161,14 +185,23 @@ function stubWebhook() {
await relay.drain();
await relay.drain();
const d = deliveries().find((x) => x.thread_id === fromDiscord.id);
const d = deliveries().find((x) => x.thread_id === fromDiscord.id);
assert.deepStrictEqual([d.status, d.last_error], ['failed', 'loop prevention: thread came from Discord']);
assert.deepStrictEqual([d.status, d.last_error], ['failed', 'loop prevention: thread came from Discord']);
+ // A reply that came from Discord, in a thread that is on Discord, stays here too.
+ const th = newThread(general, 'Mixed replies');
+ relay.enqueueThread(th, general);
+ await settle(); await relay.drain();
+ const inbound = forumStore.addPost(db, { thread_id: th.id, origin: 'discord', body_markdown: 'from discord', relay_author: 'Kim' });
+ assert.strictEqual(relay.enqueuePost(inbound, th, general), 0);
+ db.prepare('INSERT INTO relay_deliveries (thread_id, post_id, mapping_id, dedupe_key, next_attempt_at) VALUES (?, ?, 1, ?, ?)').run(th.id, inbound.id, `post:${inbound.id}:mapping:1`, sqlTime(clock));
+ await relay.drain();
+ assert.deepStrictEqual([deliveries().pop().status, deliveries().pop().last_error], ['failed', 'loop prevention: post came from Discord']);
db.prepare("INSERT INTO spaces (slug, name, visibility) VALUES ('insiders', 'Insiders', 'members')").run();
db.prepare("INSERT INTO spaces (slug, name, visibility) VALUES ('insiders', 'Insiders', 'members')").run();
const ins = forumStore.getSpace(db, 'insiders');
const ins = forumStore.getSpace(db, 'insiders');
relay.addMapping({ space_id: ins.id, webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' });
relay.addMapping({ space_id: ins.id, webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' });
assert.strictEqual(relay.enqueueThread(newThread(ins, 'Members only'), ins), 0);
assert.strictEqual(relay.enqueueThread(newThread(ins, 'Members only'), ins), 0);
- assert.strictEqual(hook.hits.length, before);
+ assert.strictEqual(hook.hits.length, before + 1, 'only the public thread went out');
});
});
- await check('disabled mappings queue nothing; retry() puts a failed delivery back with a fresh budget', async () => {
+ await check('disabled mappings queue nothing; retry() puts a failed delivery back with a fresh budget; drop() keeps it as dropped', async () => {
relay.setMappingEnabled(1, false);
relay.setMappingEnabled(1, false);
assert.strictEqual(relay.enqueueThread(newThread(general, 'Mapping off'), general), 0);
assert.strictEqual(relay.enqueueThread(newThread(general, 'Mapping off'), general), 0);
relay.setMappingEnabled(1, true);
relay.setMappingEnabled(1, true);
@@ -177,64 +210,285 @@ function stubWebhook() {
@@ -177,64 +210,285 @@ function stubWebhook() {
await settle(); await relay.drain();
await settle(); await relay.drain();
assert.strictEqual(deliveries().find((x) => x.id === failed.id).status, 'delivered');
assert.strictEqual(deliveries().find((x) => x.id === failed.id).status, 'delivered');
assert.strictEqual(relay.retry(failed.id), 0, 'delivered ones stay delivered');
assert.strictEqual(relay.retry(failed.id), 0, 'delivered ones stay delivered');
+ const dead = deliveries().find((x) => x.status === 'failed' && x.last_status === 503);
+ assert.strictEqual(relay.drop(dead.id), 1);
+ assert.strictEqual(deliveries().find((x) => x.id === dead.id).status, 'dropped');
+ assert.strictEqual(relay.drop(dead.id), 0);
+ assert.strictEqual(relay.status().deliveries.dropped, 1);
+ });
+
+ // ── replies, edits and deletes through the forum service ─────────────────────────
+ const forum = createForumService({ db, relay, limits: { threads: { cooldownSec: 0, perMinute: 100 }, posts: { cooldownSec: 0, perMinute: 100 }, threadsPerDay: 0 } });
+ const sam = ids.newId('user');
+ require('../server/pastes/store').upsertProjection(db, { subject_id: sam, username: 'sam', display_name: 'Sam' });
+ const samV = { kind: 'user', subject: sam, staff: false };
+ const alexV = { kind: 'user', subject: alex, staff: false };
+ const modV = { kind: 'user', subject: ids.newId('user'), staff: true };
+ const drainAll = async () => { await settle(); for (let i = 0; i < 5; i++) await relay.drain(); };
+
+ await check('replies follow their thread: one message per reply, linked to the post, mapped; a reply waits for its thread\'s message', async () => {
+ hook.hits.length = 0;
+ const { thread } = await forum.createThread(samV, 'general', { title: 'Replies go too', body: 'Opening words' });
+ const { post } = await forum.reply(alexV, 'general', thread.slug, { body: 'First **reply**' });
+ await drainAll();
+ assert.deepStrictEqual(hook.hits.map((h) => h.method), ['POST', 'POST']);
+ const r = hook.hits[1];
+ assert.strictEqual(r.path, '/api/webhooks/1/general?wait=true');
+ assert.match(r.body.content, /^Alex @\u200beveryone replied to \*\*Replies go too\*\* in \*\*s\/general\*\*: <https:\/\/openvibe\.community\/s\/general\/t\/replies-go-too#post-\d+>$/);
+ assert.strictEqual(r.body.embeds[0].description, 'First reply');
+ assert.deepStrictEqual(r.body.allowed_mentions, { parse: [] });
+ const [row] = mapOf('post', post.id);
+ assert.deepStrictEqual([row.direction, row.thread_id], ['out', thread.id]);
+ // The thread's own message failing for now: the reply waits, without using an attempt.
+ hook.hits.length = 0;
+ hook.plan.push({ status: 500 });
+ const t2 = await forum.createThread(samV, 'general', { title: 'Thread first', body: 'x' });
+ const p2 = (await forum.reply(alexV, 'general', t2.thread.slug, { body: 'waits' })).post;
+ await drainAll();
+ const reply = deliveries().find((x) => x.post_id === p2.id);
+ assert.deepStrictEqual([reply.status, reply.attempts, reply.last_error], ['pending', 0, "waiting for the thread's own message"]);
+ assert.strictEqual(hook.hits.length, 1, 'only the thread was tried');
+ clock += 60_000;
+ await drainAll();
+ assert.deepStrictEqual(hook.hits.map((h) => h.body.content.startsWith('New thread') ? 'thread' : 'reply'), ['thread', 'thread', 'reply'], 'the thread, then its reply');
+ assert.strictEqual(deliveries().find((x) => x.post_id === p2.id).status, 'delivered');
+ // A thread that was never relayed (made before the mapping): its replies are not either.
+ const old = newThread(general, 'Before the mapping');
+ assert.strictEqual(relay.enqueuePost(forumStore.addPost(db, { thread_id: old.id, author_subject: sam, body_markdown: 'late' }), old, general), 0);
});
});
- await check('app wiring: a thread created through the API is relayed; the admin endpoint is staff-only and URL-free', async () => {
+ await check('a mapping that names a Discord thread posts into it (thread_id) and edits and deletes there', async () => {
+ hook.hits.length = 0;
+ relay.updateMapping(2, { discord_thread_id: '3000000000000000002' });
+ const { thread } = await forum.createThread(samV, 'feedback', { title: 'Into a Discord thread', body: 'hi' });
+ const { post } = await forum.reply(samV, 'feedback', thread.slug, { body: 'reply in thread' });
+ await drainAll();
+ assert.deepStrictEqual(hook.hits.map((h) => h.path), ['/api/webhooks/2/feedback?wait=true&thread_id=3000000000000000002', '/api/webhooks/2/feedback?wait=true&thread_id=3000000000000000002'], JSON.stringify(hook.hits.map((h) => h.path)));
+ const [row] = mapOf('post', post.id);
+ assert.deepStrictEqual([row.external_channel_id, row.external_thread_id], ['3000000000000000002', '3000000000000000002']);
+ await forum.editPost(samV, post.id, { body: 'reply in thread, edited' });
+ await drainAll();
+ await forum.deletePost(samV, post.id);
+ await drainAll();
+ assert.deepStrictEqual(hook.hits.slice(2).map((h) => `${h.method} ${h.path}`), [
+ `PATCH /api/webhooks/2/feedback/messages/${row.external_message_id}?thread_id=3000000000000000002`,
+ `DELETE /api/webhooks/2/feedback/messages/${row.external_message_id}?thread_id=3000000000000000002`,
+ ]);
+ assert.strictEqual(db.prepare('SELECT discord_channel_id FROM relay_mappings WHERE id = 2').get().discord_channel_id, '2000000000000000002', 'the channel learned before stays');
+ relay.updateMapping(2, { discord_thread_id: null });
+ });
+
+ await check('edits follow through the map: a reply\'s message and, for the opening post, the thread\'s message are PATCHed', async () => {
+ hook.hits.length = 0;
+ const { thread, post: opening } = await forum.createThread(samV, 'general', { title: 'Edit me', body: 'Version one' });
+ const { post } = await forum.reply(samV, 'general', thread.slug, { body: 'Reply one' });
+ await drainAll();
+ const threadMsg = mapOf('thread', thread.id)[0].external_message_id;
+ const postMsg = mapOf('post', post.id)[0].external_message_id;
+ await forum.editPost(samV, post.id, { body: 'Reply **two**' });
+ await forum.editPost(samV, opening.id, { body: 'Version two' });
+ await forum.editPost(samV, opening.id, { body: 'Version two' }); // no change: no revision, nothing queued
+ await drainAll();
+ const edits = hook.hits.filter((h) => h.method === 'PATCH');
+ assert.deepStrictEqual(edits.map((h) => h.pathname), [`/api/webhooks/1/general/messages/${postMsg}`, `/api/webhooks/1/general/messages/${threadMsg}`]);
+ assert.strictEqual(edits[0].body.embeds[0].description, 'Reply two');
+ assert.strictEqual(edits[1].body.embeds[0].description, 'Version two');
+ assert.deepStrictEqual(edits[1].body.allowed_mentions, { parse: [] });
+ assert.strictEqual(hook.messages.get(threadMsg).body.embeds[0].description, 'Version two');
+ const keys = deliveries().filter((d) => d.action === 'edit').map((d) => d.dedupe_key);
+ assert.ok(keys.includes(`edit:post:${post.id}:mapping:1:r2`) && keys.includes(`edit:thread:${thread.id}:mapping:1:r2`), keys.join());
+ // Someone deleted the reply's message on Discord: the next edit finds nothing and says so.
+ hook.messages.get(postMsg).deleted = true;
+ await forum.editPost(samV, post.id, { body: 'Reply three' });
+ await drainAll();
+ const last = deliveries().pop();
+ assert.deepStrictEqual([last.action, last.status, last.last_error], ['edit', 'skipped', 'the message is gone on Discord']);
+ assert.ok(mapOf('post', post.id)[0].external_deleted_at, 'the map knows it is gone');
+ const queued = deliveries().length;
+ await forum.editPost(samV, post.id, { body: 'Reply four' });
+ assert.strictEqual(deliveries().length, queued, 'nothing more is queued for a message that is gone');
+ // An edit right before a delete is skipped: only the delete goes out.
+ const { post: p2 } = await forum.reply(samV, 'general', thread.slug, { body: 'short-lived' });
+ await drainAll();
+ hook.hits.length = 0;
+ await forum.editPost(samV, p2.id, { body: 'short-lived, edited' });
+ await forum.deletePost(samV, p2.id);
+ await drainAll();
+ assert.deepStrictEqual(hook.hits.map((h) => h.method), ['DELETE']);
+ assert.strictEqual(deliveries().find((d) => d.post_id === p2.id && d.action === 'edit').status, 'skipped');
+ });
+
+ await check('deletes follow through the map: a reply, and a thread with every reply of it; what was not sent yet is skipped', async () => {
+ hook.hits.length = 0;
+ const { thread } = await forum.createThread(samV, 'general', { title: 'Delete me', body: 'soon gone' });
+ const a = (await forum.reply(samV, 'general', thread.slug, { body: 'a' })).post;
+ const b = (await forum.reply(alexV, 'general', thread.slug, { body: 'b' })).post;
+ await drainAll();
+ const msgs = { thread: mapOf('thread', thread.id)[0].external_message_id, a: mapOf('post', a.id)[0].external_message_id, b: mapOf('post', b.id)[0].external_message_id };
+ await forum.deletePost(modV, a.id); // moderated away
+ await drainAll();
+ assert.deepStrictEqual(hook.hits.filter((h) => h.method === 'DELETE').map((h) => h.pathname), [`/api/webhooks/1/general/messages/${msgs.a}`]);
+ const c = (await forum.reply(samV, 'general', thread.slug, { body: 'c, never sent' })).post; // queued, not sent yet
+ await forum.deleteThread(samV, 'general', thread.slug);
+ await drainAll();
+ const deleted = hook.hits.filter((h) => h.method === 'DELETE').map((h) => h.pathname).sort();
+ assert.deepStrictEqual(deleted, [msgs.thread, msgs.a, msgs.b].map((m) => `/api/webhooks/1/general/messages/${m}`).sort());
+ assert.ok(!hook.hits.some((h) => h.method === 'POST' && /c, never sent/.test(JSON.stringify(h.body))), 'the reply queued before the delete never went out');
+ assert.deepStrictEqual([deliveries().find((d) => d.post_id === c.id).status], ['skipped']);
+ assert.ok(mapOf('thread', thread.id).every((r) => r.external_deleted_at));
+ // A delete whose message is already gone on Discord counts as done.
+ const t3 = (await forum.createThread(samV, 'general', { title: 'Gone already', body: 'x' })).thread;
+ await drainAll();
+ hook.messages.get(mapOf('thread', t3.id)[0].external_message_id).deleted = true;
+ await forum.deleteThread(samV, 'general', t3.slug);
+ await drainAll();
+ const del = deliveries().pop();
+ assert.deepStrictEqual([del.action, del.status, del.last_status, del.last_error], ['delete', 'delivered', 404, 'the message was already gone on Discord']);
+ });
+
+ await check('members-only (VIP): gated threads and spaces never go out, and gating later deletes what went out', async () => {
+ hook.hits.length = 0;
+ const { thread } = await forum.createThread(samV, 'general', { title: 'Public for now', body: 'x' });
+ await forum.reply(alexV, 'general', thread.slug, { body: 'a public reply' });
+ await drainAll();
+ const posted = hook.hits.length;
+ assert.strictEqual(posted, 2);
+ await forum.setThreadMembersOnly(samV, 'general', thread.slug, { members_only: sam });
+ await drainAll();
+ assert.strictEqual(hook.hits.filter((h) => h.method === 'DELETE').length, 2, 'the thread\'s and the reply\'s messages were deleted');
+ // Replies in the gated thread (the creator may still post) never go out.
+ const gatedThread = db.prepare('SELECT * FROM threads WHERE id = ?').get(thread.id);
+ const reply = forumStore.addPost(db, { thread_id: thread.id, author_subject: sam, body_markdown: 'members only' });
+ assert.strictEqual(relay.enqueuePost(reply, gatedThread, general), 0);
+ // A members-only thread from the start is never queued, nor is anything in a VIP space.
+ const gated = newThread(general, 'VIP from the start', { members_only_owner: sam });
+ assert.strictEqual(relay.enqueueThread(gated, general), 0);
+ db.prepare('UPDATE spaces SET members_only_owner = ? WHERE id = ?').run(sam, feedback.id);
+ const vipSpace = forumStore.getSpace(db, 'feedback');
+ assert.strictEqual(relay.enqueueThread(newThread(vipSpace, 'In a VIP space'), vipSpace), 0);
+ db.prepare('UPDATE spaces SET members_only_owner = NULL WHERE id = ?').run(feedback.id);
+ await drainAll();
+ assert.ok(!hook.hits.slice(posted).some((h) => h.method !== 'DELETE'), 'nothing but the two deletes');
+ });
+
+ await check('an older database: relay_deliveries is rebuilt once, rows kept, the new columns in place', async () => {
+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-relay-migrate-'));
+ const file = path.join(dir, 'community.db');
+ const old = openDb(file);
+ const th = forumStore.createThread(old, { space_id: 1, title: 'Old row', body_markdown: 'x' }).thread;
+ old.exec(`DROP TABLE relay_deliveries;
+ CREATE TABLE relay_deliveries (id INTEGER PRIMARY KEY AUTOINCREMENT, thread_id INTEGER NOT NULL REFERENCES threads(id) ON DELETE CASCADE,
+ mapping_id INTEGER NOT NULL REFERENCES relay_mappings(id) ON DELETE CASCADE, dedupe_key TEXT UNIQUE NOT NULL,
+ status TEXT NOT NULL DEFAULT 'pending' CHECK(status IN ('pending', 'delivered', 'failed')), attempts INTEGER NOT NULL DEFAULT 0,
+ next_attempt_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, last_status INTEGER, last_error TEXT, delivered_at DATETIME,
+ created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, UNIQUE (thread_id, mapping_id));
+ CREATE INDEX idx_relay_deliveries_due ON relay_deliveries(status, next_attempt_at);`);
+ old.prepare("INSERT INTO relay_mappings (space_id, webhook_url_ref) VALUES (1, 'DISCORD_WEBHOOK_GENERAL')").run();
+ old.prepare("INSERT INTO relay_deliveries (thread_id, mapping_id, dedupe_key, status, attempts, last_status) VALUES (?, 1, ?, 'delivered', 1, 204)").run(th.id, `thread:${th.id}:mapping:1`);
+ old.close();
+ const reopened = openDb(file);
+ const cols = reopened.prepare('PRAGMA table_info(relay_deliveries)').all().map((c) => c.name);
+ for (const c of ['post_id', 'action', 'source', 'event_id']) assert.ok(cols.includes(c), c);
+ const row = reopened.prepare('SELECT * FROM relay_deliveries').get();
+ assert.deepStrictEqual([row.thread_id, row.dedupe_key, row.status, row.action, row.source, row.last_status], [th.id, `thread:${th.id}:mapping:1`, 'delivered', 'create', 'direct', 204]);
+ reopened.prepare("INSERT INTO relay_deliveries (thread_id, mapping_id, dedupe_key, status) VALUES (?, 1, 'x', 'dropped')").run(th.id);
+ assert.ok(reopened.prepare("SELECT 1 FROM sqlite_master WHERE name = 'idx_relay_deliveries_due'").get());
+ const mcols = reopened.prepare('PRAGMA table_info(relay_mappings)').all().map((c) => c.name);
+ assert.ok(['discord_channel_id', 'discord_thread_id', 'inbound'].every((c) => mcols.includes(c)));
+ reopened.close();
+ const again = openDb(file); // idempotent
+ assert.strictEqual(again.prepare('SELECT COUNT(*) AS n FROM relay_deliveries').get().n, 2);
+ again.close();
+ fs.rmSync(dir, { recursive: true, force: true });
+ });
+
+ await check('app wiring: threads, replies, edits and deletes through the API are relayed; the admin endpoints are staff-only and URL-free', async () => {
const t = await boot({
const t = await boot({
authority: 'community',
authority: 'community',
- appOpts: { relayOptions: { enabled: true, env: { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/9/app` } }, forumLimits: { threads: { cooldownSec: 0 } } },
+ appOpts: { relayOptions: { enabled: true, env: { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/9/app` } }, forumLimits: { threads: { cooldownSec: 0 }, posts: { cooldownSec: 0 } } },
});
});
const net = t.network;
const net = t.network;
- const sam = net.addUser({ network_user_id: 9, username: 'sam', display_name: 'Sam' });
- const samJwt = net.sign({ id: 9, subject_id: sam.subject_id, username: 'sam', role: 'user' });
+ const samU = net.addUser({ network_user_id: 9, username: 'sam', display_name: 'Sam' });
+ const samJwt = net.sign({ id: 9, subject_id: samU.subject_id, username: 'sam', role: 'user' });
const adminJwt = net.sign({ id: 1, subject_id: ids.newId('user'), username: 'boss', role: 'admin' });
const adminJwt = net.sign({ id: 1, subject_id: ids.newId('user'), username: 'boss', role: 'admin' });
- const call = (path, { method = 'GET', cookie, token, json, headers = {} } = {}) => {
+ const call = (p, { method = 'GET', cookie, token, json, headers = {} } = {}) => {
const h = { ...headers };
const h = { ...headers };
if (token) h.authorization = `Bearer ${token}`;
if (token) h.authorization = `Bearer ${token}`;
if (json !== undefined) h['content-type'] = 'application/json';
if (json !== undefined) h['content-type'] = 'application/json';
- return t.get(path, { method, headers: h, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });
+ return t.get(p, { method, headers: h, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });
};
};
+ const r = t.app.locals.relay;
+ assert.strictEqual(r.status().creates_from, 'forum', 'no EVENTS_URL: the forum queues creates');
+ assert.deepStrictEqual([r.status().events_worker.enabled, r.status().inbound.enabled], [false, false]);
assert.strictEqual((await call('/api/v1/relay/mappings', { method: 'POST', cookie: samJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' } })).status, 403);
assert.strictEqual((await call('/api/v1/relay/mappings', { method: 'POST', cookie: samJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' } })).status, 403);
const bad = await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'https://discord.com/api/webhooks/secret' } });
const bad = await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'https://discord.com/api/webhooks/secret' } });
assert.strictEqual(bad.status, 400);
assert.strictEqual(bad.status, 400);
assert.strictEqual(bad.json().code, 'relay.invalid_ref');
assert.strictEqual(bad.json().code, 'relay.invalid_ref');
+ assert.strictEqual((await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL', discord_channel_id: 'nope' } })).json().code, 'relay.invalid_discord_id');
const m = await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' } });
const m = await call('/api/v1/relay/mappings', { method: 'POST', cookie: adminJwt, json: { space: 'general', webhook_url_ref: 'DISCORD_WEBHOOK_GENERAL' } });
assert.strictEqual(m.status, 201, m.text);
assert.strictEqual(m.status, 201, m.text);
- assert.deepStrictEqual([m.json().mapping.space, m.json().mapping.webhook_configured, m.json().mapping.enabled], ['general', true, true]);
+ assert.deepStrictEqual([m.json().mapping.space, m.json().mapping.webhook_configured, m.json().mapping.enabled, m.json().mapping.inbound, m.json().mapping.discord_channel_id], ['general', true, true, false, null]);
+ const put = await call(`/api/v1/relay/mappings/${m.json().mapping.id}`, { method: 'PUT', cookie: adminJwt, json: { inbound: true, discord_channel_id: '2000000000000000009' } });
+ assert.deepStrictEqual([put.json().mapping.inbound, put.json().mapping.discord_channel_id, put.json().mapping.enabled], [true, '2000000000000000009', true]);
const before = hook.hits.length;
const before = hook.hits.length;
+ const waitHits = async (n) => { for (let i = 0; i < 100 && hook.hits.length < n; i++) await new Promise((res) => setTimeout(res, 10)); };
const th = await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'Through the app', body: 'hi' } });
const th = await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'Through the app', body: 'hi' } });
assert.strictEqual(th.status, 201);
assert.strictEqual(th.status, 201);
- for (let i = 0; i < 50 && hook.hits.length === before; i++) await new Promise((r) => setTimeout(r, 10));
- assert.strictEqual(hook.hits.length, before + 1);
- assert.strictEqual(hook.hits[hook.hits.length - 1].path, '/api/webhooks/9/app');
+ await waitHits(before + 1);
+ assert.strictEqual(hook.hits[hook.hits.length - 1].path, '/api/webhooks/9/app?wait=true');
+ const reply = await call(`/api/v1/spaces/general/threads/${th.json().thread.slug}/posts`, { method: 'POST', cookie: samJwt, json: { body: 'a reply' } });
+ assert.strictEqual(reply.status, 201, reply.text);
+ await waitHits(before + 2);
+ assert.match(hook.hits[hook.hits.length - 1].body.content, /replied to \*\*Through the app\*\*/);
+ assert.strictEqual((await call(`/api/v1/posts/${reply.json().post.id}`, { method: 'PUT', cookie: samJwt, json: { body: 'a reply, edited' } })).status, 200);
+ await waitHits(before + 3);
+ assert.strictEqual(hook.hits[hook.hits.length - 1].method, 'PATCH');
+ assert.strictEqual((await call(`/api/v1/posts/${reply.json().post.id}`, { method: 'DELETE', cookie: samJwt })).status, 200);
+ await waitHits(before + 4);
+ assert.strictEqual(hook.hits[hook.hits.length - 1].method, 'DELETE');
hook.plan.push({ status: 400, body: { message: 'Invalid Form Body' } });
hook.plan.push({ status: 400, body: { message: 'Invalid Form Body' } });
await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'This one fails', body: 'hi' } });
await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'This one fails', body: 'hi' } });
- for (let i = 0; i < 50 && hook.hits.length === before + 1; i++) await new Promise((r) => setTimeout(r, 10));
- await t.app.locals.relay.drain();
+ await waitHits(before + 5);
+ await r.drain();
assert.strictEqual((await call('/api/v1/relay/deliveries')).status, 401);
assert.strictEqual((await call('/api/v1/relay/deliveries')).status, 401);
assert.strictEqual((await call('/api/v1/relay/deliveries', { cookie: samJwt })).status, 403);
assert.strictEqual((await call('/api/v1/relay/deliveries', { cookie: samJwt })).status, 403);
+ assert.strictEqual((await call('/api/v1/relay/status', { cookie: samJwt })).status, 403);
const failed = await call('/api/v1/relay/deliveries?status=failed', { cookie: adminJwt });
const failed = await call('/api/v1/relay/deliveries?status=failed', { cookie: adminJwt });
assert.strictEqual(failed.status, 200, failed.text);
assert.strictEqual(failed.status, 200, failed.text);
assert.strictEqual(failed.json().enabled, true);
assert.strictEqual(failed.json().enabled, true);
assert.strictEqual(failed.json().deliveries.length, 1);
assert.strictEqual(failed.json().deliveries.length, 1);
- assert.match(failed.json().deliveries[0].last_error, /400: Invalid Form Body/);
- assert.strictEqual(failed.json().deliveries[0].thread.url, '/s/general/t/this-one-fails');
+ const dl = failed.json().deliveries[0];
+ assert.match(dl.last_error, /400: Invalid Form Body/);
+ assert.deepStrictEqual([dl.thread.url, dl.action, dl.source], ['/s/general/t/this-one-fails', 'create', 'direct']);
+ const edits = await call('/api/v1/relay/deliveries?action=edit', { cookie: adminJwt });
+ assert.deepStrictEqual(edits.json().deliveries.map((x) => [x.action, x.status, x.post.id]), [['edit', 'delivered', reply.json().post.id]]);
+ const st = await call('/api/v1/relay/status', { cookie: adminJwt });
+ assert.strictEqual(st.status, 200);
+ assert.deepStrictEqual([st.json().dead_letters, st.json().deliveries.failed, st.json().creates_from], [1, 1, 'forum']);
+ assert.strictEqual((await call(`/api/v1/relay/deliveries/${dl.id}/drop`, { method: 'POST', cookie: adminJwt })).status, 200);
+ assert.strictEqual((await call(`/api/v1/relay/deliveries/${dl.id}/drop`, { method: 'POST', cookie: adminJwt })).status, 404, 'dropped already');
+ assert.strictEqual((await call(`/api/v1/relay/deliveries/${dl.id}/retry`, { method: 'POST', cookie: adminJwt })).status, 200, 'a dropped one can be retried');
+ await waitHits(before + 6);
+ await r.drain();
+ assert.strictEqual((await call('/api/v1/relay/deliveries?status=failed', { cookie: adminJwt })).json().deliveries.length, 0);
+ const ready = await call('/api/ready');
+ assert.deepStrictEqual([ready.json().discord_relay.enabled, ready.json().discord_relay.events_worker.enabled, ready.json().discord_relay.deliveries.delivered > 0], [true, false, true]);
const svcAdmin = await call('/api/v1/relay/deliveries', { token: net.signService({ cap: ['community.comment.moderate'] }) });
const svcAdmin = await call('/api/v1/relay/deliveries', { token: net.signService({ cap: ['community.comment.moderate'] }) });
assert.strictEqual(svcAdmin.status, 200);
assert.strictEqual(svcAdmin.status, 200);
- assert.ok(!(failed.text + svcAdmin.text + (await call('/api/v1/relay/mappings', { cookie: adminJwt })).text).includes('/api/webhooks/'), 'responses never carry a webhook URL');
+ assert.ok(!(failed.text + svcAdmin.text + st.text + (await call('/api/v1/relay/mappings', { cookie: adminJwt })).text).includes('/api/webhooks/'), 'responses never carry a webhook URL');
await t.close();
await t.close();
});
});
await check('staff can map only allow-listed webhook variables, never the URL in some other env var', async () => {
await check('staff can map only allow-listed webhook variables, never the URL in some other env var', async () => {
const secretHits = [];
const secretHits = [];
const internal = { url: 'http://127.0.0.1:9/internal', hits: secretHits };
const internal = { url: 'http://127.0.0.1:9/internal', hits: secretHits };
- const env = { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/1/ok`, OV_MEDIA_INTERNAL_URL: internal.url, NETWORK_INTERNAL_URL: internal.url, DISCORD_WEBHOOKS_X: internal.url };
+ const env2 = { DISCORD_WEBHOOK_GENERAL: `${hook.url}/api/webhooks/1/ok`, OV_MEDIA_INTERNAL_URL: internal.url, NETWORK_INTERNAL_URL: internal.url, DISCORD_WEBHOOKS_X: internal.url };
const fetchSpy = (url, o) => { if (String(url).startsWith(internal.url)) secretHits.push(url); return fetch(url, o); };
const fetchSpy = (url, o) => { if (String(url).startsWith(internal.url)) secretHits.push(url); return fetch(url, o); };
- const t = await boot({ authority: 'community', appOpts: { relayOptions: { enabled: true, env, fetchImpl: fetchSpy }, forumLimits: { threads: { cooldownSec: 0 } } } });
+ const t = await boot({ authority: 'community', appOpts: { relayOptions: { enabled: true, env: env2, fetchImpl: fetchSpy }, forumLimits: { threads: { cooldownSec: 0 } } } });
const net = t.network;
const net = t.network;
const adminJwt = net.sign({ id: 1, subject_id: ids.newId('user'), username: 'boss', role: 'admin' });
const adminJwt = net.sign({ id: 1, subject_id: ids.newId('user'), username: 'boss', role: 'admin' });
const samJwt = net.sign({ id: 9, subject_id: net.addUser({ network_user_id: 9, username: 'sam' }).subject_id, username: 'sam', role: 'user' });
const samJwt = net.sign({ id: 9, subject_id: net.addUser({ network_user_id: 9, username: 'sam' }).subject_id, username: 'sam', role: 'user' });
- const call = (path, { method = 'GET', cookie, token, json } = {}) => t.get(path, { method, headers: { ...(token ? { authorization: `Bearer ${token}` } : {}), ...(json !== undefined ? { 'content-type': 'application/json' } : {}) }, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });
+ const call = (p, { method = 'GET', cookie, token, json } = {}) => t.get(p, { method, headers: { ...(token ? { authorization: `Bearer ${token}` } : {}), ...(json !== undefined ? { 'content-type': 'application/json' } : {}) }, body: json !== undefined ? JSON.stringify(json) : undefined, cookies: cookie ? [`ov_token=${cookie}`] : [] });
const map = (ref, who = { cookie: adminJwt }) => call('/api/v1/relay/mappings', { method: 'POST', json: { space: 'general', webhook_url_ref: ref }, ...who });
const map = (ref, who = { cookie: adminJwt }) => call('/api/v1/relay/mappings', { method: 'POST', json: { space: 'general', webhook_url_ref: ref }, ...who });
for (const ref of ['OV_MEDIA_INTERNAL_URL', 'NETWORK_INTERNAL_URL', 'DISCORD_WEBHOOKS_X', 'PATH', 'DISCORD_WEBHOOK_']) {
for (const ref of ['OV_MEDIA_INTERNAL_URL', 'NETWORK_INTERNAL_URL', 'DISCORD_WEBHOOKS_X', 'PATH', 'DISCORD_WEBHOOK_']) {
const r = await map(ref);
const r = await map(ref);
@@ -247,9 +501,9 @@ function stubWebhook() {
@@ -247,9 +501,9 @@ function stubWebhook() {
assert.throws(() => t.app.locals.relay.addMapping({ space_id: 1, webhook_url_ref: 'OV_MEDIA_INTERNAL_URL' }), /not an allowed webhook variable/);
assert.throws(() => t.app.locals.relay.addMapping({ space_id: 1, webhook_url_ref: 'OV_MEDIA_INTERNAL_URL' }), /not an allowed webhook variable/);
// A mapping that predates the allow-list (straight into the table) is never sent to, and
// A mapping that predates the allow-list (straight into the table) is never sent to, and
// does not reveal whether that variable is set.
// does not reveal whether that variable is set.
- const general = t.db.prepare("SELECT id FROM spaces WHERE slug = 'general'").get();
- t.db.prepare("INSERT INTO relay_mappings (space_id, direction, webhook_url_ref, enabled) VALUES (?, 'out', 'OV_MEDIA_INTERNAL_URL', 1)").run(general.id);
- const listed = (await call('/api/v1/relay/mappings', { cookie: adminJwt })).json().mappings.find((m) => m.webhook_url_ref === 'OV_MEDIA_INTERNAL_URL');
+ const gen = t.db.prepare("SELECT id FROM spaces WHERE slug = 'general'").get();
+ t.db.prepare("INSERT INTO relay_mappings (space_id, direction, webhook_url_ref, enabled) VALUES (?, 'out', 'OV_MEDIA_INTERNAL_URL', 1)").run(gen.id);
+ const listed = (await call('/api/v1/relay/mappings', { cookie: adminJwt })).json().mappings.find((x) => x.webhook_url_ref === 'OV_MEDIA_INTERNAL_URL');
assert.strictEqual(listed.webhook_configured, false);
assert.strictEqual(listed.webhook_configured, false);
assert.strictEqual((await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'Would go inside', body: 'hi' } })).status, 201);
assert.strictEqual((await call('/api/v1/spaces/general/threads', { method: 'POST', cookie: samJwt, json: { title: 'Would go inside', body: 'hi' } })).status, 201);
await t.app.locals.relay.drain();
await t.app.locals.relay.drain();
@@ -260,11 +514,25 @@ function stubWebhook() {
@@ -260,11 +514,25 @@ function stubWebhook() {
assert.strictEqual((await map('DISCORD_WEBHOOK_GENERAL')).status, 201, 'the conventional names still work');
assert.strictEqual((await map('DISCORD_WEBHOOK_GENERAL')).status, 201, 'the conventional names still work');
await t.close();
await t.close();
// DISCORD_RELAY_WEBHOOK_VARS narrows it to exact names.
// DISCORD_RELAY_WEBHOOK_VARS narrows it to exact names.
- const { createDiscordRelay } = require('../server/relay/discord');
- const strict = createDiscordRelay({ db: t.db, env, webhookVars: ['DISCORD_WEBHOOK_GENERAL'] });
+ const strict = createDiscordRelay({ db: t.db, env: env2, webhookVars: ['DISCORD_WEBHOOK_GENERAL'] });
assert.strictEqual(strict.refAllowed('DISCORD_WEBHOOK_GENERAL'), true);
assert.strictEqual(strict.refAllowed('DISCORD_WEBHOOK_GENERAL'), true);
assert.strictEqual(strict.refAllowed('DISCORD_WEBHOOK_OTHER'), false);
assert.strictEqual(strict.refAllowed('DISCORD_WEBHOOK_OTHER'), false);
- assert.strictEqual(createDiscordRelay({ db: t.db, env }).refAllowed('DISCORD_WEBHOOK_OTHER'), true);
+ assert.strictEqual(createDiscordRelay({ db: t.db, env: env2 }).refAllowed('DISCORD_WEBHOOK_OTHER'), true);
+ });
+
+ await check('disabled by default in the app: no relay, no Events worker, no gateway, whatever else is set', async () => {
+ process.env.DISCORD_BOT_TOKEN = 'not-a-real-token';
+ process.env.DISCORD_RELAY_INBOUND = 'on';
+ process.env.EVENTS_URL = 'http://127.0.0.1:9';
+ try {
+ const t = await boot({ authority: 'community' });
+ const st = t.app.locals.relay.status();
+ assert.deepStrictEqual([st.enabled, st.events_worker.enabled, st.inbound.enabled, t.app.locals.relayInbound], [false, false, false, null]);
+ assert.match(st.events_worker.reason, /relay is off/);
+ await t.close();
+ } finally {
+ delete process.env.DISCORD_BOT_TOKEN; delete process.env.DISCORD_RELAY_INBOUND; delete process.env.EVENTS_URL;
+ }
});
});
await hook.close();
await hook.close();
@@ -1,7 +1,7 @@
@@ -1,7 +1,7 @@
'use strict';
'use strict';
/**
/**
* Graceful stop (roadmap WS-P lifecycle; server/graceful.js), on the real process with the events
* Graceful stop (roadmap WS-P lifecycle; server/graceful.js), on the real process with the events
- * outbox, the Discord relay, the Pulse subscriptions and the profile and search scans all on (pointed
+ * outbox, the Discord relay (with its Events worker and gateway), the Pulse subscriptions and the profile and search scans all on (pointed
* at nothing): SIGTERM while a request is in flight and a keep-alive connection sits idle. New
* at nothing): SIGTERM while a request is in flight and a keep-alive connection sits idle. New
* connections are refused, the request is answered (Connection: close), every background worker is
* connections are refused, the request is answered (Connection: close), every background worker is
* stopped, community.db is closed, and the process exits 0 within the manifest's 5 s; the idle
* stopped, community.db is closed, and the process exits 0 within the manifest's 5 s; the idle
@@ -27,6 +27,7 @@ const freePort = () => new Promise((resolve) => { const s = net.createServer();
@@ -27,6 +27,7 @@ const freePort = () => new Promise((resolve) => { const s = net.createServer();
...process.env, PORT: String(port), HOST: '127.0.0.1', NODE_ENV: 'test', COMMUNITY_DB_PATH: path.join(dir, 'community.db'),
...process.env, PORT: String(port), HOST: '127.0.0.1', NODE_ENV: 'test', COMMUNITY_DB_PATH: path.join(dir, 'community.db'),
EVENTS_URL: 'http://127.0.0.1:9', OV_NETWORK_INTERNAL_URL: 'http://127.0.0.1:9', OV_NETWORK_URL: 'http://127.0.0.1:9',
EVENTS_URL: 'http://127.0.0.1:9', OV_NETWORK_INTERNAL_URL: 'http://127.0.0.1:9', OV_NETWORK_URL: 'http://127.0.0.1:9',
OV_OAUTH_CLIENT_SECRET: 'test-secret-not-real', COMMUNITY_EVENTS_SECRET: 'e'.repeat(40), DISCORD_RELAY_ENABLED: '1',
OV_OAUTH_CLIENT_SECRET: 'test-secret-not-real', COMMUNITY_EVENTS_SECRET: 'e'.repeat(40), DISCORD_RELAY_ENABLED: '1',
+ DISCORD_RELAY_INBOUND: 'on', DISCORD_BOT_TOKEN: 'not-a-real-bot-token', DISCORD_GATEWAY_URL: 'ws://127.0.0.1:9/?v=10&encoding=json',
},
},
stdio: ['ignore', 'pipe', 'pipe'],
stdio: ['ignore', 'pipe', 'pipe'],
});
});
@@ -43,6 +44,7 @@ const freePort = () => new Promise((resolve) => { const s = net.createServer();
@@ -43,6 +44,7 @@ const freePort = () => new Promise((resolve) => { const s = net.createServer();
}
}
assert.ok(up, `the server did not start:\n${out}`);
assert.ok(up, `the server did not start:\n${out}`);
assert.match(out, /\[Events\] community →/, 'the outbox is on');
assert.match(out, /\[Events\] community →/, 'the outbox is on');
+ assert.match(out, /\[Relay\] Discord relay on: creates queued by the Events worker; events worker on; inbound on/, 'the relay, its Events worker and the gateway are on');
// An idle keep-alive connection (the server keeps them 65 s).
// An idle keep-alive connection (the server keeps them 65 s).
const agent = new http.Agent({ keepAlive: true });
const agent = new http.Agent({ keepAlive: true });
@@ -73,7 +75,7 @@ const freePort = () => new Promise((resolve) => { const s = net.createServer();
@@ -73,7 +75,7 @@ const freePort = () => new Promise((resolve) => { const s = net.createServer();
const ms = Date.now() - t0;
const ms = Date.now() - t0;
assert.strictEqual(code, 0, `exit 0 (got ${code} ${signal})\n${out.slice(-2000)}`);
assert.strictEqual(code, 0, `exit 0 (got ${code} ${signal})\n${out.slice(-2000)}`);
assert.ok(ms < 5000, `within the manifest's 5 s (${ms} ms)`);
assert.ok(ms < 5000, `within the manifest's 5 s (${ms} ms)`);
- assert.match(out, /\[Community\] stopped: subscriptions stopped, profile scan stopped, search scans stopped, relay stopped, outbox stopped \(\d+ pending\)/, out.slice(-2000));
+ assert.match(out, /\[Community\] stopped: subscriptions stopped, profile scan stopped, search scans stopped, relay stopped \(events worker stopped, gateway stopped\), outbox stopped \(\d+ pending\)/, out.slice(-2000));
assert.match(out, /\[Community\] stopped in \d+ ms/);
assert.match(out, /\[Community\] stopped in \d+ ms/);
agent.destroy();
agent.destroy();
console.log(`shutdown: SIGTERM → exit 0 in ${ms} ms, the request in flight answered, every worker stopped`);
console.log(`shutdown: SIGTERM → exit 0 in ${ms} ms, the request in flight answered, every worker stopped`);