openvibers/openvibe.publishing AGENTS.md
AGENTS.md
| 1 | # AGENTS.md — OpenVibe.Publishing |
| 2 | |
| 3 | ## Project Overview |
| 4 | |
| 5 | OpenVibe.Publishing is the shared **library** of publishing behaviour for the OpenVibe publication products (Wiki, Blog, |
| 6 | News, Reviews, Deals, Coupons, Trade) and for the record products that use its Search feed/relay: revisions, |
| 7 | scheduling, taxonomy, citations, media/discussion references, the indexability gate, feeds/sitemaps, authorship and AI |
| 8 | disclosure, Search documents and envelopes, server-rendered page layout, the block model, the Sources ingest chassis |
| 9 | and the publication glue. |
| 10 | |
| 11 | - **Package:** `openvibe-publishing` · **Version:** 1.7.1 · **Runtime:** CommonJS, Node ≥20 (production Node 22) · |
| 12 | **License:** MIT |
| 13 | - **It is code, never state.** It has no runtime, port, domain, database, API, migration directory or deploy of its own. |
| 14 | It must never get one. Products pin the release tarball; the state lives in each product's database with that |
| 15 | product's table prefix. |
| 16 | - **Read first:** [README.md](README.md), [docs/MIGRATION.md](docs/MIGRATION.md), [CHANGELOG.md](CHANGELOG.md), |
| 17 | [examples/two-products/](examples/two-products/). |
| 18 | |
| 19 | **Network context:** the package implements **no capability and holds no grant**. Modules call services with the |
| 20 | **product's** token client: `discussion` → OpenVibe.Community `community.comment.write`; `ai` → OpenVibe.AI |
| 21 | `ai.run.create`/`ai.run.read`; `media.verify()` → OpenVibe.Media `media.object.read`; `ingest` → OpenVibe.Sources |
| 22 | `sources.item.read`/`sources.source.read`. It stores references, never comment content, media bytes or source items. |
| 23 | |
| 24 | ## Commands |
| 25 | |
| 26 | ```bash |
| 27 | npm ci # install (devDependencies include contracts; peers come from the consumer) |
| 28 | npm test # every test/*.test.js on a fresh in-process PGlite schema, no network |
| 29 | eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" && npm test # PostgreSQL 18 + PgBouncer, as CI |
| 30 | node --check <file.js> # syntax check (no linter configured) |
| 31 | ``` |
| 32 | |
| 33 | There is no start/dev/build/deploy script. A release is a git tag `vX.Y.Z` recorded in |
| 34 | [CHANGELOG.md](CHANGELOG.md); products pin the tag's tarball and ship it with their own |
| 35 | `sudo ovhost deploy <product>`. A bad release is undone by the products pinning the previous tag. |
| 36 | |
| 37 | ## Module Map |
| 38 | |
| 39 | Each entry point loads alone (`require('openvibe-publishing/<module>')`; the root export also exposes them lazily). |
| 40 | |
| 41 | | Module | What it does | Tables (in the product's DB, `<prefix>_…`) | |
| 42 | |---|---|---| |
| 43 | | `revisions` | drafts, immutable revisions with parent pointers, optimistic concurrency (412 `revision.conflict`), diff, revert-as-new-revision, audited purge | `_revisions`, `_drafts`, `_revision_purges` | |
| 44 | | `schedule` | idempotent scheduled publish/unpublish, lease-based claim, retries/backoff | `_schedule_jobs` | |
| 45 | | `taxonomy` | tags/terms, hierarchical categories, cycle checks | `_terms`, `_term_links` | |
| 46 | | `citations` | append-only Sources item/URL references, retrieved_at, quote span, licence note, carry-forward | `_citations`, `_citation_purges` | |
| 47 | | `media` | attachments by `med_…`/`legacy:…`, explicit `broken` state, `verify()` never flips on an outage, `<figure>` with an honest placeholder | `_attachments` | |
| 48 | | `discussion` | Community thread references (thread id only) | `_discussion_refs` | |
| 49 | | `seo` | deterministic indexability gate, canonical URLs, slug redirects (301/410), meta/robots, sitemaps, RSS/Atom/JSON Feed, JSON-LD from given fields | `_redirects` | |
| 50 | | `authorship` | human/ai/hybrid/imported records, workflow + run id, disclosure labels, AI held as draft + noindex until review | `_reviews` | |
| 51 | | `index-hooks` | `search.index-document@1` docs/tombstones, `<owner>.index_document.*` events, monotonic sequencer, product publish events | `_index_revisions` | |
| 52 | | `ai` | run a registered OpenVibe.AI workflow, poll a slow run, read citations, coded `AiRunError`s | — | |
| 53 | | `ssr` | auto-escaping `html` templates + `raw`, safe Markdown subset, text extraction, pagination, breadcrumbs, diff, `<time>` | — | |
| 54 | | `blocks` | Markdown + `:::name{attrs}` blocks, parse/validate/render/toText, plus `blocks.css` | — | |
| 55 | | `ingest` | Sources client, named change cursor, `pullChanges` with per-item savepoints, signed exactly-once event consumer, normalisers, PSL host helpers | `_ingest_cursor` | |
| 56 | | `publication` | gate → document → `sequencer.stamp` → events → `outbox.enqueue` on the caller's transaction; tombstones; IndexNow | — | |
| 57 | | `search-feed` | record pages in Search from the write's transaction, `sweep()` re-sends only what changed | — | |
| 58 | | `search-relay` | deliver a product's `<product>_outbox` rows to Search (oldest first, supersede, conflicts, 429/5xx backoff, ≤100 a tick) | — | |
| 59 | | `layout` | `renderDocument(o)` through `openvibe-shared/shell page()` with gate robots/canonical, article times, prev/next, feeds, icon, stylesheets, footer init | — | |
| 60 | |
| 61 | `schema({ revisions, citations, seo, … })` returns the DDL for several stores at once; each store also has |
| 62 | `schema()` and `ensureSchema()` (tests/PGlite only). |
| 63 | |
| 64 | ## The Rule: Packages, Not an Authority |
| 65 | |
| 66 | Share **how** something is done (revisions, the gate, feeds, indexing); never share **what is true** (which revision |
| 67 | is live, who may read it, editorial rules). Two products share the code, not the data: Wiki's database holds `wiki_*` |
| 68 | tables, Blog's holds `blog_*`; [test/two-products.test.js](test/two-products.test.js) proves it. A generic |
| 69 | `OpenVibe.Content` authority is explicitly not what this is. It does not decide editorial policy — each product |
| 70 | passes its own gate policy and permission checks. |
| 71 | |
| 72 | ## Using the Stores on PostgreSQL |
| 73 | |
| 74 | - Every store takes the **product's** `openvibe-sdk/db` handle and a prefix; creating it runs no query; data methods |
| 75 | are `async` and take an optional transaction handle as their **first** argument. With one, the product's write and |
| 76 | the store's writes commit together or not at all. |
| 77 | - The DDL goes into the product's migrations (the runtime PgBouncer role cannot create tables); the stores never |
| 78 | create their own tables. `schema()` text is idempotent, PostgreSQL-typed (`bigint` identity, `jsonb`, epoch ms, |
| 79 | `timestamptz` for citations, `COLLATE "C"` identifiers) and has an index under every query a store sends |
| 80 | ([test/schema.test.js](test/schema.test.js) checks with EXPLAIN). |
| 81 | - Pin the release tarball (never a `file:` link or a vendored copy). `openvibe-sdk` and `openvibe-shared` are peers |
| 82 | resolved from the product's install, so there is one database handle/pool and one Shared copy per service. |
| 83 | |
| 84 | ## Conventions |
| 85 | |
| 86 | - CommonJS, 4-space indent, single quotes, semicolons. Every value reaches PostgreSQL as a bind parameter; the only |
| 87 | identifiers in SQL are the validated prefix's table names. |
| 88 | - **Escaping:** `ssr.html` auto-escapes every value; never interpolate raw HTML without `raw()`. The Markdown subset |
| 89 | stays linear-time (the ReDoS fix is a guarantee). JSON-LD is escaped by `openvibe-shared`. |
| 90 | - **Indexability:** the gate is deterministic with stable reasons (`seo.REASONS`). A product supplies the fields; |
| 91 | the package never invents one. AI content is a draft + noindex until a person's review. Sitemaps/feeds list only |
| 92 | what the gate says indexable. |
| 93 | - **Events/Search:** the document and envelope shapes are `search.index-document@1` / `events.event-envelope@1`; |
| 94 | `stamp` and `enqueue` must be on the same transaction as the change, and a monotonic revision means an older |
| 95 | document never overwrites a newer one. |
| 96 | - **Slug/redirects:** a request path with several leading slashes stays on the site's canonical URL |
| 97 | (`//feed/` used to 500 on News, Deals and Blog — v1.4.1). Old slug → 301, gone → 410. |
| 98 | - **Schema changes:** a later release that changes DDL gives the migration in its CHANGELOG; an applied product |
| 99 | migration is never edited. |
| 100 | - No Font Awesome and no site classes in `blocks.css`; use the shared theme tokens. The library renders no page shell |
| 101 | of its own beyond `layout`. |
| 102 | |
| 103 | ## Key Pitfalls |
| 104 | |
| 105 | - **Never add state, a server, a port, a database or an API here.** A change that needs one belongs in the product. |
| 106 | - **Never share data between products.** Prefixes isolate the tables; do not add a cross-product query, a shared |
| 107 | service database, or a default prefix. |
| 108 | - **Transactions:** pass the caller's handle through; a store opening its own transaction inside the caller's would |
| 109 | break atomicity. A purge racing writes, racing workers and `get-or-create` are tested on real PostgreSQL |
| 110 | ([test/postgres.test.js](test/postgres.test.js)). |
| 111 | - **AI:** no partial draft on a failed run; an `AiRunError` is coded; AI output stays noindex until review; the |
| 112 | authorship record keeps the workflow/run/model. |
| 113 | - **Media:** `verify()` never flips a state on an outage; a missing object renders the honest placeholder, not a |
| 114 | broken image. |
| 115 | - **Ingest:** per-item savepoint isolation means one bad item does not roll back the page; a removed item is handled |
| 116 | as removal, never silently ignored; the cursor advances only when the work commits. |
| 117 | - **Search relay:** a stale revision is done, a conflict is sent again one revision up, a 429 pauses until |
| 118 | `Retry-After`, a 5xx/network error backs off; other refusals are logged and set aside, never dropped silently. |
| 119 | - **Peers:** do not add a runtime dependency on a driver; `lib/` never loads `pg`/PGlite. Products bring their handle. |
| 120 | |
| 121 | ## Testing |
| 122 | |
| 123 | `npm test` runs every [test/](test/) file on a fresh in-process PGlite schema with no network: immutable revisions/ |
| 124 | diff/revert, scheduling across worker restarts, citations, honest media states, discussion refs, the gate/redirects/ |
| 125 | structured data/feeds, AI held until review, contracts-valid Search documents/envelopes and transactional stamping, |
| 126 | `ssr` escaping/ReDoS, the ingest normalisers (119 captured cases), the publication glue/tombstones/IndexNow, every |
| 127 | subpath loading alone without a driver, DDL idempotency/63-byte names/checks/guards/indexes, cross-store transactions/ |
| 128 | rollbacks/savepoints, and the two-product exit proof. `test/postgres.test.js` runs the race tests on PostgreSQL 18 + |
| 129 | PgBouncer when `OV_TEST_PG_URL` is set (CI always does); otherwise it reports skipped with the reason. |
| 130 | |
| 131 | ## Documentation |
| 132 | |
| 133 | - [README.md](README.md) — purpose, owns, the rule, module map, install, using the stores on PostgreSQL, moving |
| 134 | between releases, the gate, honesty guarantees, events/Search, ingest, publication glue, exit proof, capabilities, |
| 135 | acceptance, security, deploy, development, does-not-own, depends-on. |
| 136 | - [docs/MIGRATION.md](docs/MIGRATION.md) — moving a product between releases. |
| 137 | - [CHANGELOG.md](CHANGELOG.md) — every release, its additive/breaking changes and any DDL a product must add. |
| 138 | - [examples/two-products/](examples/two-products/) — the exit proof (two products, one library, no shared data). |
| 139 | - [SECURITY.md](SECURITY.md). |
| 140 |