openvibers/openvibe.mail AGENTS.md

AGENTS.md

main 13.9 KB · 183 lines · markdown

Raw HistoryBlame Repository
1# AGENTS.md — OpenVibe.Mail
2
3## Project Overview
4
5OpenVibe.Mail is `openvibe.email`: masks that protect you, burners that self-destruct, an inbox for every agent, and
6person mail (read your mailbox, compose, custom domains, forwarding, read-only IMAP of another provider, opt-in Inbox
7AI). M1 is **receive-only**: Stalwart Community listens on port 25; the service hands out addresses that resolve into
8mailboxes, and nothing sends or relays outside OpenVibe unless `MAIL_OUTBOUND=on` and an outbound provider are both
9configured (external sending is M2 and off by default).
10
11- **Domain:** `openvibe.email` · **Port:** 5250 (API; SMTP 25, JMAP/IMAP loopback) · **Service id:** `mail` ·
12 **Env prefix:** `MAIL`
13- **Runtime:** Node 22, Express 4, PostgreSQL 18 through `openvibe-sdk/db` (PGlite in development/tests), Stalwart
14 Community 1.0.3 pinned in [deploy/stalwart/VERSION](deploy/stalwart/VERSION), `imapflow`/`mailparser`/`nodemailer`
15 for the optional person-mail paths.
16- **Read first:** [README.md](README.md), [docs/INBOX-AI.md](docs/INBOX-AI.md), [docs/INSTALL.md](docs/INSTALL.md),
17 [docs/DNS.md](docs/DNS.md), [docs/INBOX.md](docs/INBOX.md).
18
19**Network context:** identity, tokens, staff roles, verified emails and the trust check are OpenVibe.Network
20(`identity.trust.read`, `identity.emails.read`). Account export/deletion is ADR-033 through OpenVibe.Events. Mail AI
21(`server/ai/inbox.js`) is optional. The mail server is Stalwart's own process; this repository ships its declarative
22config and a Node API, not a modified Stalwart. Message bodies are read from Stalwart over loopback JMAP and are never
23written to logs, events or the database (the one exception is Inbox AI's opt-in summary/editable draft, and the draft
24is the only generated body persisted).
25
26## Commands
27
28```bash
29npm ci # install (contracts/sdk/shared pinned by tag)
30npm start # node server/index.js (API; Stalwart runs separately)
31npm test # unit: PGlite, in-process Network and in-memory mail server
32npm run test:pg # the same through PostgreSQL
33npm run test:stalwart # acceptance: real Stalwart + PostgreSQL (MinIO with STALWART_BLOB_STORE=s3)
34npm run init-db # node deploy/init-db.js (apply migrations before systemd starts the service)
35npm run stalwart:fetch # verify/download the pinned Stalwart (sha256)
36npm run stalwart:render # config.toml → rendered/config.json + rendered/plan.json
37npm run stalwart:apply # reconcile the plan over loopback JMAP
38node --check <file.js> # syntax check (no linter configured)
39```
40
41No build step: the page and assets are served from [public/](public/) (`css/inbox.css`, `css/mail.css`, `js/inbox.js`,
42`js/compose.js`, `js/mail.js`, `js/mail-providers.js`).
43
44## Architecture at a Glance
45
46- **Entry:** [server/index.js](server/index.js) — database/migrations, mail server client, events relay, purge,
47 agent-mail, Inbox AI, IMAP pull and forwarding jobs, then the listener; each job is due-based and stops on SIGTERM.
48- **App factory:** [server/app.js](server/app.js) — helmet/CSP, `/auth`, CORS, `/api/v1`, `/internal/`, machine
49 endpoints, the page/static mount [server/http/pages.js](server/http/pages.js), error handler.
50- **Config:** [server/config.js](server/config.js) — `load(env)` is pure; production `/etc/openvibe/mail.env`.
51- **Domains:** [server/domain/](server/domain/) — `masks.js`, `burners.js`, `agent-inboxes.js`, `agent-answering.js`,
52 `agent-reply.js`, `addresses.js`, `messages.js`, `mailboxes.js`, `reading.js` (HTML cleaning), `sending.js`,
53 `compose.js`, `custom-domains.js`, `forwarding.js`, `person-mail.js`, `mail-search.js`, `inbox-ai*.js`,
54 `imap-connections.js`, `imap-secrets.js`, `outbound-providers.js`, `provider-*.js`, `limits.js`,
55 `network-policy.js`.
56- **Mail server:** [server/stalwart/](server/stalwart/) — `client.js` (loopback JMAP), `smtp.js`, `code.js`;
57 [server/imap/client.js](server/imap/client.js) for the read-only IMAP pull.
58- **HTTP:** [server/http/](server/http/) — `api.js`, `principal.js`, `person-routes.js`, `reading-routes.js`,
59 `agent-mail.js`, `imap-routes.js`, `workspace-routes.js`, `cors.js`, `discovery.js`, `pages.js`, `mail-copy.js`.
60- **Jobs:** [server/jobs/](server/jobs/) — `purge.js` (expiry), `agent-mail.js`, `inbox-ai.js`, `imap-pull.js`,
61 `forwarding.js`, `workspace.js`.
62- **Database:** [server/db.js](server/db.js) and [migrations/](migrations/): `mail_addresses`, `mail_directory_accounts`
63 (the SQL directory view Stalwart reads), settings/outbox, rate limits, observed messages, mask expiry, agent
64 answering, Inbox AI consent/triage, IMAP mailboxes, outbound providers/domains, person mail, workspace, spam
65 scoring, `event_outbox` (with its rejected partial index).
66- **Stalwart config:** [deploy/stalwart/config.toml](deploy/stalwart/config.toml) is the source of truth;
67 `render.js` produces the data store and plan, `apply.js` reconciles over JMAP. It creates the SQL directory, the
68 base/mask/burner domains (`allowRelaying = false`), the blob store (fs or S3), spam filter and exactly three
69 listeners (SMTP 25, JMAP/management + IMAP on loopback); every default submission/HTTPS/POP3/Sieve listener is
70 removed. The directory-created account gets the `User` role or Stalwart refuses its own mail.
71- **Deploy:** [deploy/systemd/openvibe-mail.service](deploy/systemd/openvibe-mail.service) (API),
72 [deploy/systemd/openvibe-stalwart.service](deploy/systemd/openvibe-stalwart.service) and
73 [deploy/systemd/openvibe-stalwart.slice](deploy/systemd/openvibe-stalwart.slice) (memory cap),
74 [deploy/nginx/openvibe.email.conf](deploy/nginx/openvibe.email.conf) (API/health/ready/auth; `/internal/` closed).
75 M1 is **not deployed from this repository** — [docs/INSTALL.md](docs/INSTALL.md) is the operator's handoff.
76
77## Ownership and Boundaries
78
79Mail owns masks, burners, agent inboxes and their answering state, the directory view, person-mail addresses,
80settings, observed message metadata, rate limits, Inbox AI consent/summaries/drafts, IMAP mailbox settings and the
81outbox. It does **not** own identity/verified emails/trust (Network), mail delivery or blobs (Stalwart), event delivery
82(Events), AI (the AI service), or the host/DNS (Host/operator). It never stores a message body as a row; message
83content lives in Stalwart and is read on demand for its owner. Another person's mask, burner or inbox answers **404,
84never 403**.
85
86## Addresses, Reading and Sending
87
88- **Masks** (`<site>.<code>@mask.openvibe.email`) deliver into the owner's mailbox; disabling one makes it bounce. A
89 mask may carry `expires_days`; the purge job disables it and the directory stops resolving it.
90- **Burners** are receive-only and expire after 10m/1h/1d/7d or the first message; the 10-minute burner needs no
91 account and is read with its returned `x-mail-burner-token`. Every unexpired burner is its **own** directory
92 account, so one burner's mail is never readable in another's. Expired: the mail is deleted, the account removed and
93 the address bounces.
94- **Agent inboxes** (`<name>.<owner>@openvibe.email`) are aliases on the owner's mailbox. Answering is a switch, **off
95 by default**, turned on for one agent; the due poll then publishes `mail.agent_inbox.received` once per unseen
96 non-spam, non-blocked letter. The agent reads with `mail.agent.read` and replies with `mail.agent.reply`, always
97 from the agent's address to the sender and nobody else, plain text, with `In-Reply-To`/`References`.
98- **Delivered-To:** a mask/agent inbox is matched only against the first `Delivered-To` header Stalwart stamps —
99 never a sender-written To/Cc/Delivered-To. A burner's account is read with no recipient filtering at all.
100- **HTML reading:** [server/domain/reading.js](server/domain/reading.js) sanitizes with an allow-list; `style` is
101 limited to layout/colour with no `url()`, `expression()` or `@import`; links only http(s)/mailto with
102 `rel="noopener noreferrer"`. Remote images become alt text (`blocked_images`) until `?images=1` (https only); the
103 page renders it in an iframe with no `allow-scripts` and its own `default-src 'none'` CSP.
104- **Person mail:** a person can create a named address/alias, compose plain text, attach a custom domain and forward a
105 copy to a verified Network email. Sending needs a verified account email, a one-day-old account, a verified sender
106 domain and a live trust check. 20 recipients/day rising to 50/100 with age and history; 10 recipients/message; no
107 bulk mode. Hard bounces/complaints pause the sender; every message carries a report link and
108 `X-OpenVibe-Report-Abuse`.
109- **Forwarding** rechecks Network ownership at creation and before every copy; only non-junk plain-text messages
110 without attachments are copied; an ambiguous provider answer is not retried automatically.
111- **Outbound is closed by default.** `MAIL_OUTBOUND=on` plus an enabled provider is required; with it off, a reply to
112 a non-OpenVibe address answers 409 `mail.outbound_closed` and stays pending, never lost.
113- **IMAP pull** is TLS only, read-only, app password sealed with `MAIL_SECRETS_KEY`; it never writes back.
114- Limits: 20 masks, 5 live burners/person; account-less 10-minute burners 10/IP/hour and 2,000 live globally
115 (503 `mail.burner_capacity`); counters are in PostgreSQL, so a restart is not a fresh allowance.
116
117## API and Scopes
118
119All API routes are `/api/v1`, take a Network bearer or the same-site session cookie; a cross-origin browser may use a
120Bearer from a first-party origin (CORS); a cookie write must come from openvibe.email (`Sec-Fetch-Site`/`Origin`).
121An app token names the person (`on_behalf_of`) and holds `mail.addresses.write` or `mail.messages.read`; the
122Companion-style token holds `mail.mask.manage`; a typed token is refused. A service acting for a person uses
123`X-OV-Subject`.
124
125Core routes: `GET /me/mail`, masks (`/masks`, `/:id`, `/:id/disable`), burners (`/burners`,
126`/:id/messages`, `/:id/code`), agent inboxes (`/agent-inboxes`, `/:id/messages`, `/:id/answering`),
127blocks (`/me/mail/blocks`), `GET|POST /me/mail/messages[...]` (reading, read/unread, delete, attachments),
128person-mail/addresses/compose/domains/forwarding/IMAP/Inbox AI per [docs/INBOX-AI.md](docs/INBOX-AI.md), and the
129internal agent routes. The message list returns `{ id, from, subject, received_at }` and nothing else; `/code`
130returns the code, never the body.
131
132## Conventions
133
134- CommonJS, 4-space indent, single quotes, semicolons; no linter/formatter.
135- **Never log a body, a subject, an address's mail or a code.** Metadata-only events: `mail.mask.created`,
136 `mail.received` (address, sender, message id, time — no subject/body/code), `mail.agent_inbox.received` (excerpt
137 ≤4000 chars, never HTML/attachment, nothing for spam/blocked).
138- New schema = a new `migrations/NNNN_name.sql` (expand-only); never edit one that has run. The directory view is
139 read by Stalwart, so a column change is a coordinated deploy.
140- Stalwart config is **declarative**: edit [deploy/stalwart/config.toml](deploy/stalwart/config.toml), then render and
141 apply. Never edit the rendered data store by hand or leave a default listener enabled.
142- CSP: add origins to `helmet` in [server/app.js](server/app.js); the message iframe has its own stricter policy.
143- Phone first: the one page and the inbox fit 390 px; no Font Awesome; "Calm reduces, only Off removes" (a block or
144 disabled mask suppresses, only purge/delete removes).
145- Empty states use the shared kit; keep the page's copy to what M1 does today (receive, show a code, never a body
146 unless its owner asks).
147
148## Key Pitfalls
149
150- **One burner = one account.** Never pool or merge burners into one mailbox; a forgeable To/Cc/Delivered-To must not
151 move mail between burners or agent inboxes. [test/stalwart/integration.test.js](test/stalwart/integration.test.js)
152 proves it with a real Stalwart.
153- **Directory vs expiry:** an expired/disabled address must resolve to no directory row and bounce; do not "keep it
154 working" for a grace period.
155- **Bodies:** only the owner reads a message; the code extractor must discard the body as soon as it has the code.
156 An attachment is allow-listed/`application/octet-stream` with `nosniff`.
157- **Outbound off** is the default safety: do not make a route that quietly sends externally because a provider key is
158 present. `GET /internal/mail/outbound` tells callers the truth.
159- **Inbox AI** is opt-in and consent-scoped; the draft is editable and is the only generated body persisted; a reply
160 stays a draft until the person sends it (see [docs/INBOX-AI.md](docs/INBOX-AI.md)).
161- **Custom domains:** verify DNS/ownership before accepting; the inbound path and states are in
162 [docs/DNS.md](docs/DNS.md). Never show a domain as verified before it is.
163- **Ports:** only 25 inbound is open; 587/465 stay closed. JMAP/IMAP listen on loopback. Do not expose them.
164- **`event_outbox`** needs its rejected partial index (migration 0007).
165- **M1 is not deployed from this repo.** Do not add a deploy step here; the operator follows INSTALL.md.
166
167## Testing
168
169`npm test` runs unit tests on PGlite with an in-process Network and an in-memory mail server: masks, burners, agent
170inboxes/answering, directory, reading/sanitizing/images, person mail, provider adapters/router/webhooks, IMAP,
171Inbox AI, purge, API/page, CORS/nginx, Stalwart client, companion masks, workspace and your-address. `npm run test:pg`
172for PostgreSQL. `npm run test:stalwart` is the real acceptance run (real Stalwart + PostgreSQL) and also runs in CI.
173Always `node --check` changed files.
174
175## Documentation
176
177- [README.md](README.md) — person mail, M1.5 reading, M1 pieces, API, events, agent answering, contracts, Stalwart,
178 tests, layout, limits, install.
179- [docs/INBOX-AI.md](docs/INBOX-AI.md) — consent, rules, journal, undo and receipts.
180- [docs/INBOX.md](docs/INBOX.md), [docs/INSTALL.md](docs/INSTALL.md), [docs/DNS.md](docs/DNS.md).
181- [deploy/stalwart/config.toml](deploy/stalwart/config.toml), [deploy/stalwart/VERSION](deploy/stalwart/VERSION),
182 [SECURITY.md](SECURITY.md), [.env.example](.env.example).
183