openvibers/openvibe.publishing Commits
openvibe-publishing 0.2.0: index-hooks emits the released search.index-document@1 (contracts v0.12.0) and the <owner>.index_document.upserted|deleted events Search consumes, with a monotonic index-revision sequencer; proposal schema removed
Changes
@@@ -4,6 +4,52 @@ All notable changes to `openvibe-publishing`. Versions follow [semver](https://sbreaking change to any exported function, table layout, reason code or document shape is a newmajor (a minor while 0.x). A release is the git tag `vX.Y.Z`; consumers pin the tag's tarball.++## 0.2.0 — 2026-09-22++++**Breaking: `index-hooks` now emits the released `search.index-document@1`** (openvibe-contracts++v0.12.0, owned by OpenVibe.Search) instead of the 0.1.0 proposal, and builds the events Search++actually consumes. Products on 0.1.0 must update their calls; nothing else changed.++++### Changed (breaking)++- `buildIndexDocument()` takes `owner` (was `service`) and returns exactly the contract shape:++ `owner`, `type`, `id`, `revision`, `deleted`, `visibility`, `acl`, `canonical_url`, `title`,++ `summary`, `body`, `facets`, `language` (was `locale`), `authorship`, `provenance`,++ `publication_state`, `published_at`, `updated_at`, `indexability { decision, reasons }`.++ Gone: `schema`, `owner_service`, `resource_type`, `resource_id`, `acl.public`, `body_truncated`,++ the `indexability.indexable/listable/gate` fields and the provenance object.++- Mappings onto the contract enums: visibility `gated` → `members`; authorship `hybrid` →++ `ai_assisted`, `ai` → `ai_generated`; gate reason codes → Search's known reasons where one exists++ (`thin` → `thin_content`, `ai_generated_unreviewed` → `ai_unreviewed`, `unreviewed_sensitive` →++ `sensitive_unreviewed`, `gated` → `members_only`, `unpublished` → `not_published`,++ `missing_canonical` → `missing_canonical_url`, `noindex_requested` → `owner_decision`; the rest++ keep the gate's code). `provenance` is now an array of typed references built from citations++ (Sources items, or the owner's own citation records), the AI run (with `stub: true` for stub++ output) and extra `provenance` refs.++- `tombstone()` returns exactly `{ owner, type, id, revision, deleted: true }`. Anything not++ published (draft, scheduled, unpublished, retracted, archived, deleted) is still a tombstone, and++ so is unlisted content unless `includeUnlisted: true` (then `visibility: 'unlisted'`).++- ACL follows the contract: `subjects` are `usr_`/`gst_` only, plus `groups` and `entitlements`;++ private needs subjects, members needs at least one of the three. Text is clipped to the contract's++ limits (title 500, summary 4000, body 48000); facets are checked against its rules.++- `publicationEvent()` (the product's own `<product>.<type>.<action>` event) no longer embeds the++ document; its payload is `{ canonical_url, publication_state, indexability }`.++++### Added++- `indexEvent({ document })`: `<owner>.index_document.upserted` (payload = document) or++ `<owner>.index_document.deleted` (payload = `{ type, id, revision }`), subject++ `{ type, id, revision }`, visibility internal — the envelopes OpenVibe.Search's webhook consumes.++- `createIndexSequencer(db, { prefix })`: a monotonic index revision per resource in++ `<prefix>_index_revisions`. Search refuses a different document at an equal revision and lets a++ tombstone win ties, so every indexed change (content, visibility, state, canonical URL,++ decision) needs a higher revision; an unchanged document keeps its revision (a safe replay).++- `searchIndexability(decision)`, `SEARCH_REASONS`, `AUTHORSHIP`, `LIMITS`.++- Tests validate every emitted document with `validate('search.index-document@1')` and every++ envelope with `validate('events.event-envelope@1')` from openvibe-contracts v0.12.0, and mirror++ Search's webhook checks (owner = source, subject matches the payload).++++### Removed++- `docs/contracts-proposal/search.index-document.v1.json` (superseded by the released contract).++## 0.1.0 — 2026-09-22First version (roadmap Wave 15). Packages only: no runtime, no domain, no database.@@@ -3,8 +3,9 @@> Shared publishing packages for the OpenVibe publication products: Wiki, Blog, News, Reviews,> Deals, Coupons and Trade.--**Status:** alpha, 0.1.0 (roadmap Wave 15). The modules are tested and the exit proof--(`examples/two-products`) runs in `npm test`; not tagged or released yet. No product uses them in production yet: Wiki and Blog (Wave 16) are the++**Status:** alpha, 0.2.0 (roadmap Wave 15). The modules are tested and the exit proof++(`examples/two-products`) runs in `npm test`. v0.1.0 is tagged; 0.2.0 (the released Search++document shape, a breaking change — see [CHANGELOG.md](CHANGELOG.md)) is not tagged yet. No product uses them in production yet: Wiki and Blog (Wave 16) are thefirst planned consumers.**Package:** `openvibe-publishing` (CommonJS, Node ≥ 20, production runs Node 22).**License:** MIT, like OpenVibe.Shared.@@@ -36,7 +37,7 @@ Every module is its own entry point and can be used alone.| `openvibe-publishing/discussion` | Community comment-thread references: resolves via `POST /api/v1/comments/threads/resolve`, stores the thread id only (no comment content) | `<prefix>_discussion_refs` || `openvibe-publishing/seo` | The deterministic **indexability gate**; canonical URLs; history-aware redirects (old slug → 301, gone → 410); meta/robots tags; sitemaps; RSS, Atom and JSON Feed; JSON-LD built only from provided fields | `<prefix>_redirects` || `openvibe-publishing/authorship` | human / ai / hybrid / imported records with the OpenVibe.AI workflow + run id, disclosure labels, AI content held as draft + noindex until a person's review | `<prefix>_reviews` |--| `openvibe-publishing/index-hooks` | Search index documents (and tombstones) from a published revision; `<product>.<type>.published\|updated\|unpublished\|deleted` event envelopes for the product's outbox | — |++| `openvibe-publishing/index-hooks` | `search.index-document@1` documents and tombstones, the `<owner>.index_document.upserted\|deleted` events OpenVibe.Search consumes, a monotonic index-revision sequencer, and the product's own `<product>.<type>.published\|updated\|unpublished\|deleted` events | `<prefix>_index_revisions` || `openvibe-publishing/ssr` | auto-escaping `html` tagged templates with `raw()`, a safe Markdown subset, plain-text extraction, word count, server pagination, breadcrumbs, diff markup, honest `<time>` | — |`require('openvibe-publishing')` exposes all of them lazily (`.revisions`, `.seo`, `.indexHooks`, …).@@@ -46,7 +47,7 @@ Every module is its own entry point and can be used alone.Pin the release tarball, like every OpenVibe package (never a `file:` link or a vendored copy):```json--"openvibe-publishing": "https://codeload.github.com/OpenVibers/OpenVibe.Publishing/tar.gz/refs/tags/v0.1.0"++"openvibe-publishing": "https://codeload.github.com/OpenVibers/OpenVibe.Publishing/tar.gz/refs/tags/v0.2.0"````better-sqlite3` (≥ 11) is a peer dependency: the product brings its own handle. The only runtime@@@ -96,13 +97,32 @@ published document) all require the decision, so an object without one cannot re## Events and Search--`index-hooks.buildIndexDocument()` produces the Search document ([REF] §12.3 fields plus the gate--decision) or a tombstone; the proposed schema is--[docs/contracts-proposal/search.index-document.v1.json](docs/contracts-proposal/search.index-document.v1.json).--`publicationEvent()` wraps it in an `events.event-envelope@1` envelope with `event_type`--`<product>.<type>.<action>` and no `event_id` (the OpenVibe.Events outbox assigns one); events about--private, gated or unlisted content are `internal`. Enqueue it with the Events outbox in the same--transaction as the product's state change.++`index-hooks.buildIndexDocument()` produces exactly the released `search.index-document@1`++(openvibe-contracts v0.12.0, owned by OpenVibe.Search): visibility `public | unlisted | members |++private`, ACL `subjects | groups | entitlements`, authorship `human | ai_assisted | ai_generated |++imported`, provenance as typed references (Sources items, the product's citation records, the AI++run with `stub: true` for stub output), and `indexability { decision: index|noindex, reasons }` with++the gate's codes mapped onto Search's known reasons (`hooks.SEARCH_REASONS`). Anything not++published, and unlisted content unless `includeUnlisted`, becomes the tombstone++`{ owner, type, id, revision, deleted: true }`.++++Search orders documents by `revision`, refuses a different document at an equal revision and lets a++tombstone win ties, so the revision must rise with every indexed change, not only content edits.++`createIndexSequencer(db, { prefix })` keeps that counter in the product's database.++++```js++const seq = hooks.createIndexSequencer(db, { prefix: 'wiki' });++const doc = seq.stamp(hooks.buildIndexDocument({ owner: 'wiki', type: 'page', id, revision: 0, state, visibility, … , decision }));++outbox.enqueue(hooks.indexEvent({ document: doc })); // wiki.index_document.upserted | .deleted++```++++`indexEvent()` builds the envelope Search's webhook consumes (payload = the document, or++`{ type, id, revision }` for a deletion; subject `{ type, id, revision }`; visibility internal).++`publicationEvent()` builds the product's own `<product>.<type>.<action>` event (payload: canonical++URL, publication state, indexability; public only for public, listable content). Neither sets++`event_id`: the OpenVibe.Events outbox assigns it. Enqueue both in the same transaction as the++product's state change. Tests validate every document and envelope with openvibe-contracts++v0.12.0 and mirror Search's webhook checks.## Exit proof: two products@@@ -135,4 +155,5 @@ fnm exec --using=22.22.1 npm test # every test/*.test.js, temp database- `openvibe-shared` v1.0.0 (`seo`), at runtime.- `better-sqlite3` ≥ 11, supplied by the consumer.- Contracts it produces for: `common.entity-ref@1`, `media.media-ref@1`, `events.event-envelope@1`-- (validated in tests against `openvibe-contracts` v0.7.0), and the proposed `search.index-document@1`.++ and `search.index-document@1` (validated in tests against `openvibe-contracts` v0.12.0, a++ devDependency: nothing in `lib/` needs it at runtime).@@@ -5,17 +5,36 @@"wave": "W15","kind": "packages","package": "openvibe-publishing",-- "version": "0.1.0",++ "version": "0.2.0","plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)","runtime": null,"port": null,"domain": null,"database": null,-- "modules": ["revisions", "schedule", "taxonomy", "citations", "media", "discussion", "seo", "authorship", "index-hooks", "ssr"],++ "modules": [++ "revisions",++ "schedule",++ "taxonomy",++ "citations",++ "media",++ "discussion",++ "seo",++ "authorship",++ "index-hooks",++ "ssr"++ ],"consumers": [],-- "plannedConsumers": ["OpenVibe.Wiki", "OpenVibe.Blog", "OpenVibe.News", "OpenVibe.Reviews", "OpenVibe.Deals", "OpenVibe.Coupons", "OpenVibe.Trade"],++ "plannedConsumers": [++ "OpenVibe.Wiki",++ "OpenVibe.Blog",++ "OpenVibe.News",++ "OpenVibe.Reviews",++ "OpenVibe.Deals",++ "OpenVibe.Coupons",++ "OpenVibe.Trade"++ ],"exitProof": "examples/two-products + test/two-products.test.js",-- "contracts": "openvibe-contracts v0.7.0 (tests); proposes search.index-document@1",++ "contracts": "openvibe-contracts v0.12.0: emits search.index-document@1 and events.event-envelope@1 (validated in tests)","released": false,"updated": "2026-09-22"}ddeleted file mode 100644@@@ -1,113 +0,0 @@--{-- "$schema": "https://json-schema.org/draft/2020-12/schema",-- "$id": "https://openvibe.network/contracts/search/index-document.v1.json",-- "title": "SearchIndexDocument",-- "description": "PROPOSAL (openvibe-publishing 0.1.0, for OpenVibe.Contracts and OpenVibe.Search). The document a publication product sends to Search for one revision of one resource, or the tombstone that removes it. Owner, resource, revision, visibility + ACL, canonical URL, title/summary/body, facets, provenance, publication state, deletion marker ([REF] §12.3), plus the publishing SEO gate's indexability decision. Search must never return a document to a query that fails its ACL, and must not list a document whose indexability.listable is false on public surfaces.",-- "oneOf": [-- {-- "title": "IndexDocument",-- "type": "object",-- "required": ["schema", "owner_service", "resource_type", "resource_id", "revision", "visibility", "acl", "canonical_url", "title", "summary", "body", "facets", "provenance", "publication_state", "deleted", "indexability", "published_at", "updated_at"],-- "additionalProperties": false,-- "properties": {-- "schema": { "const": "search.index-document@1" },-- "owner_service": { "$ref": "#/$defs/service" },-- "resource_type": { "$ref": "#/$defs/segment" },-- "resource_id": { "type": "string", "minLength": 1, "maxLength": 200 },-- "revision": { "type": "integer", "minimum": 0 },-- "visibility": { "enum": ["public", "private", "gated"] },-- "acl": {-- "oneOf": [-- { "type": "object", "required": ["public"], "additionalProperties": false, "properties": { "public": { "const": true } } },-- {-- "type": "object", "required": ["public"], "additionalProperties": false,-- "properties": {-- "public": { "const": false },-- "subjects": { "type": "array", "minItems": 1, "items": { "type": "string", "pattern": "^(usr|gst|app|mod)_[0-9A-HJKMNP-TV-Z]{26}$" } },-- "entitlements": { "type": "array", "minItems": 1, "items": { "type": "string", "minLength": 1 } }-- },-- "anyOf": [{ "required": ["subjects"] }, { "required": ["entitlements"] }]-- }-- ]-- },-- "canonical_url": { "type": "string", "pattern": "^https?://" },-- "title": { "type": "string", "minLength": 1 },-- "summary": { "type": ["string", "null"] },-- "body": { "type": "string", "maxLength": 100000 },-- "body_truncated": { "const": true },-- "locale": { "type": "string" },-- "facets": {-- "type": "object",-- "propertyNames": { "pattern": "^[a-z][a-z0-9_]{0,39}$" },-- "additionalProperties": {-- "anyOf": [-- { "type": ["string", "number", "boolean"] },-- { "type": "array", "items": { "type": ["string", "number", "boolean"] } }-- ]-- }-- },-- "provenance": {-- "type": "object",-- "additionalProperties": false,-- "properties": {-- "authorship": {-- "type": "object", "required": ["mode"], "additionalProperties": false,-- "properties": {-- "mode": { "enum": ["human", "ai", "hybrid", "imported"] },-- "reviewed": { "type": "boolean" },-- "workflow": { "type": "object", "required": ["id", "run_id"], "additionalProperties": false, "properties": { "id": { "type": "string" }, "run_id": { "type": "string" } } }-- }-- },-- "citations": {-- "type": "array",-- "items": {-- "type": "object", "additionalProperties": false,-- "properties": {-- "source_item_id": { "type": "string" },-- "url": { "type": "string" },-- "retrieved_at": { "type": "string", "format": "date-time" },-- "license_note": { "type": "string" }-- },-- "anyOf": [{ "required": ["source_item_id"] }, { "required": ["url"] }]-- }-- },-- "derived_from": { "type": "object" }-- }-- },-- "publication_state": { "const": "published" },-- "deleted": { "const": false },-- "indexability": {-- "type": "object", "required": ["indexable", "listable", "reasons", "gate"], "additionalProperties": false,-- "properties": {-- "indexable": { "type": "boolean" },-- "listable": { "type": "boolean" },-- "reasons": { "type": "array", "items": { "type": "string", "pattern": "^[a-z][a-z_]*$" } },-- "gate": { "type": ["string", "null"] }-- }-- },-- "published_at": { "type": ["string", "null"], "format": "date-time" },-- "updated_at": { "type": ["string", "null"], "format": "date-time" }-- }-- },-- {-- "title": "IndexTombstone",-- "type": "object",-- "required": ["schema", "owner_service", "resource_type", "resource_id", "revision", "publication_state", "deleted", "updated_at"],-- "additionalProperties": false,-- "properties": {-- "schema": { "const": "search.index-document@1" },-- "owner_service": { "$ref": "#/$defs/service" },-- "resource_type": { "$ref": "#/$defs/segment" },-- "resource_id": { "type": "string", "minLength": 1, "maxLength": 200 },-- "revision": { "type": "integer", "minimum": 0 },-- "publication_state": { "enum": ["draft", "scheduled", "published", "unpublished", "deleted"] },-- "deleted": { "const": true },-- "updated_at": { "type": ["string", "null"], "format": "date-time" }-- }-- }-- ],-- "$defs": {-- "service": { "type": "string", "pattern": "^[a-z][a-z0-9-]{1,39}$" },-- "segment": { "type": "string", "pattern": "^[a-z][a-z0-9_]{0,39}$" }-- }--}@@@ -6,7 +6,7 @@* node examples/two-products/wiki/app.js # http://127.0.0.1:4801 with a temp database** Uses: revisions (wiki_page_*), citations (wiki_citations), seo gate + redirects + sitemap +-- * Atom/JSON feeds, authorship (AI pages start as drafts), index-hooks (events into wiki_outbox), ssr.++ * Atom/JSON feeds, authorship (AI pages start as drafts), index-hooks (Search index events + product events into wiki_outbox), ssr.*/const http = require('http');const path = require('path');@@@ -47,6 +47,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Dateconst citations = createCitationStore(db, { prefix: 'wiki', now, revisions });const redirects = seo.createRedirectStore(db, { prefix: 'wiki_page', now });const reviews = authorship.createReviewLog(db, { prefix: 'wiki_page', now });++ const sequencer = hooks.createIndexSequencer(db, { prefix: 'wiki', now });const pathOf = (slug) => `/p/${slug}`;const urlOf = (slug) => seo.canonicalUrl(origin, pathOf(slug));@@@ -64,23 +65,32 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Datefunction document(page, rev, decision) {return hooks.buildIndexDocument({-- service: 'wiki', type: 'page', id: page.id, revision: rev ? rev.number : 0, state: page.state, visibility: page.visibility,++ owner: 'wiki', type: 'page', id: page.id, revision: 0, state: page.state, visibility: page.visibility,canonicalUrl: urlOf(page.slug), title: rev ? rev.fields.title : page.slug, summary: rev ? ssr.markdownToText(rev.content, 160) : null,body: rev ? ssr.markdownToText(rev.content) : '', decision, acl: { subjects: [page.owner] },-- provenance: { authorship: rev && rev.meta.authorship, citations: rev ? citations.forRevision(page.id, rev.number) : [] },++ authorship: rev && rev.meta.authorship, citations: rev ? citations.forRevision(page.id, rev.number) : [],++ facets: { content_revision: rev ? rev.number : 0 },publishedAt: page.published_at, updatedAt: page.updated_at,});}-- function emit(before, page, actor) {++ /**++ * After any change: send Search the current document (or tombstone) when it differs from the++ * last one sent, and the product event when the publication state moved. Both go to the++ * outbox in the caller's transaction.++ */++ function sync(before, page, actor) {const rev = page.published_revision ? revisions.get(page.id, page.published_revision) : null;++ const decision = rev ? decide(page, rev) : null;++ const sentBefore = sequencer.current('wiki', 'page', page.id);++ const doc = sequencer.stamp(document(page, rev, decision));++ const out = [];++ if (doc.revision !== sentBefore) out.push(hooks.indexEvent({ document: doc, now: now() }));const action = hooks.actionFor(before && { state: before.state, visibility: before.visibility, revision: before.published_revision },{ state: page.state, visibility: page.visibility, revision: page.published_revision });-- if (!action) return null;-- const decision = rev ? decide(page, rev) : null;-- const env = hooks.publicationEvent({ product: 'wiki', type: 'page', action, id: page.id, revision: page.published_revision || 0, actor, document: document(page, rev, decision), decision, now: now() });-- db.prepare('INSERT INTO wiki_outbox (envelope) VALUES (?)').run(JSON.stringify(env));-- return env;++ if (action) out.push(hooks.publicationEvent({ product: 'wiki', type: 'page', action, id: page.id, revision: page.published_revision || 0, actor, document: doc, decision, now: now() }));++ for (const env of out) db.prepare('INSERT INTO wiki_outbox (envelope) VALUES (?)').run(JSON.stringify(env));++ return out;}const api = {@@@ -126,7 +136,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Dateconst ok = authorship.canPublish(rec, reviews.latest(id, revision));if (!ok.ok) throw Object.assign(new Error(`cannot publish: ${ok.reason}`), { status: 409, code: ok.reason });db.prepare("UPDATE wiki_pages SET state = 'published', published_revision = ?, published_at = COALESCE(published_at, ?), updated_at = ? WHERE id = ?").run(revision, now(), now(), id);-- return emit(before, getPage(id), actor);++ return sync(before, getPage(id), actor);})();},@@@ -134,7 +144,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Datereturn db.transaction(() => {const before = getPage(id);db.prepare('UPDATE wiki_pages SET visibility = ?, updated_at = ? WHERE id = ?').run(visibility, now(), id);-- return emit(before, getPage(id), actor);++ return sync(before, getPage(id), actor);})();},@@@ -144,6 +154,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Dateconst slug = slugify(newTitle);redirects.recordMove(id, pathOf(page.slug), pathOf(slug));db.prepare('UPDATE wiki_pages SET slug = ?, updated_at = ? WHERE id = ?').run(slug, now(), id);++ sync(page, getPage(id), 'svc:wiki'); // the canonical URL changed: Search gets a new revisionreturn slug;})();},@@@ -152,7 +163,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Datereturn db.transaction(() => {const before = getPage(id);db.prepare("UPDATE wiki_pages SET state = 'deleted', updated_at = ? WHERE id = ?").run(now(), id);-- return emit(before, getPage(id), actor);++ return sync(before, getPage(id), actor);})();},@@@ -1,40 +1,91 @@'use strict';/**-- * openvibe-publishing/index-hooks — Search index documents and publication event envelopes.++ * openvibe-publishing/index-hooks — OpenVibe.Search index documents and the events that carry them.** const hooks = require('openvibe-publishing/index-hooks');-- * const doc = hooks.buildIndexDocument({ service: 'wiki', type: 'page', id: 'pg_1', revision: 4, state: 'published',-- * visibility: 'public', canonicalUrl, title, summary, body, facets: { space: 'food' }, provenance, decision });-- * const action = hooks.actionFor(before, after); // 'published' | 'updated' | 'unpublished' | 'deleted' | null-- * const env = hooks.publicationEvent({ product: 'wiki', type: 'page', action, id: 'pg_1', revision: 4,-- * actor: 'usr_…', document: doc, decision, now: Date.now() });-- * outbox.enqueue(env); // OpenVibe.Events createOutbox, in the same transaction as the product's change++ * const doc = hooks.buildIndexDocument({ owner: 'wiki', type: 'page', id: 'pg_1', revision: 4, state: 'published',++ * visibility: 'public', canonicalUrl, title, summary, body, facets: { space: 'food' },++ * authorship: rec, citations, decision });++ * outbox.enqueue(hooks.indexEvent({ document: doc, actor: 'svc:wiki' })); // wiki.index_document.upserted|deleted++ * const action = hooks.actionFor(before, after); // product domain event, optional++ * if (action) outbox.enqueue(hooks.publicationEvent({ product: 'wiki', type: 'page', action, id, revision, actor, document: doc, decision }));*-- * The index document is the Search contract of [REF] §12.3 (owner service, resource type/id,-- * revision, visibility + ACL, canonical URL, title, summary, body, facets, provenance, publication-- * state, deletion marker) plus the gate's indexability decision. Proposed schema:-- * docs/contracts-proposal/search.index-document.v1.json.++ * Documents follow the released contract `search.index-document@1` (openvibe-contracts ≥ 0.12.0,++ * contracts/search/index-document.v1.json) exactly: { owner, type, id, revision, deleted,++ * visibility, acl, canonical_url, title, summary, body, facets, language, authorship, provenance,++ * publication_state, published_at, updated_at, indexability { decision, reasons } }, and a++ * tombstone is exactly { owner, type, id, revision, deleted: true }.*-- * Safety rules built in:-- * - Anything not published, deleted, or unlisted becomes a tombstone (deletion marker, no-- * title/summary/body), so a visibility change or deletion removes content from the index.-- * - private content must carry an ACL naming subjects; gated content must name entitlements.-- * There is no way to build a private document that reads as world-visible.-- * - Events about non-public or non-listable content are 'internal', never 'public'.-- * - The envelope carries no event_id; the outbox assigns evt_<ULID> (or pass eventId).++ * Mapping from this package's vocabulary:++ * visibility public → public · unlisted → unlisted (only with includeUnlisted; else a tombstone)++ * gated → members (acl entitlements/groups/subjects) · private → private (acl.subjects)++ * authorship human → human · hybrid → ai_assisted · ai → ai_generated · imported → imported++ * reasons the gate's codes, renamed to Search's known reasons where one exists (SEARCH_REASONS)++ * provenance Sources items → { service: 'sources', type: 'item', id }; URL-only citations →++ * the owner's own citation record { service: owner, type: 'citation', id };++ * the AI run → { service: 'ai', type: 'run', id, stub? }; plus any refs passed in++ *++ * Stricter than the schema requires, on purpose: anything not published (draft, scheduled,++ * unpublished, retracted, archived, deleted) and unlisted content (unless includeUnlisted) is sent++ * as a tombstone, so a state or visibility change always removes the old copy from Search. A++ * private document must name its subjects and a members document its audience — there is no way to++ * build a non-public document that reads as world-visible.++ *++ * Revisions: Search orders by the document's `revision`; at an equal revision a different++ * document is a conflict (first write wins) and a tombstone wins, so the number must grow with++ * EVERY change to what is indexed (content, visibility, state, canonical URL, gate decision), not++ * only with content revisions. createIndexSequencer(db, { prefix }) keeps that counter in the++ * product's database: same document → same revision (a safe replay), anything different → +1.++ *++ * Events: Search consumes `<owner>.index_document.upserted` (payload = the document) and++ * `<owner>.index_document.deleted` (payload = { type, id, revision }), subject { type, id, revision },++ * visibility internal (OpenVibe.Search server/api/webhook.js). Envelopes carry no event_id; the++ * OpenVibe.Events outbox assigns evt_<ULID>. Owners must be slugs without dashes because event++ * types are dot-separated [a-z0-9_] segments.*/--const SERVICE_RE = /^[a-z][a-z0-9-]{1,39}$/;--const SEGMENT_RE = /^[a-z][a-z0-9_]{0,39}$/;--const ULID = '[0-9A-HJKMNP-TV-Z]{26}';--const SUBJECT_ID_RE = new RegExp(`^(usr|gst|app|mod)_${ULID}$`);--const ACTIONS = ['published', 'updated', 'unpublished', 'deleted'];++const OWNER_RE = /^[a-z][a-z0-9_]{1,39}$/; // service slug that is also a valid event_type segment++const TYPE_RE = /^[a-z][a-z0-9_]{1,39}$/;++const ID_RE = /^[A-Za-z0-9][A-Za-z0-9._:~-]{0,127}$/;++const PERSON_RE = /^(usr|gst)_[0-9A-HJKMNP-TV-Z]{26}$/;++const KEY_RE = /^[a-z][a-z0-9_.:-]{0,127}$/;++const FACET_RE = /^[a-z][a-z0-9_]{0,39}$/;++const LANG_RE = /^[a-zA-Z]{2,3}(-[A-Za-z0-9]{2,8})*$/;++const REF_SERVICE_RE = /^[a-z][a-z0-9-]{1,39}$/;++const SCHEMA = 'search.index-document@1';--const MAX_BODY = 100000;++const ACTIONS = ['published', 'updated', 'unpublished', 'deleted'];++const LIMITS = Object.freeze({ title: 500, summary: 4000, body: 48000, facets: 20, facetArray: 50, facetString: 200, provenance: 50, reasons: 20, acl: { subjects: 200, groups: 100, entitlements: 100 } });--function assertDecision(d) {-- if (!d || typeof d.indexable !== 'boolean' || typeof d.listable !== 'boolean') throw new TypeError('decision (seo.evaluate) is required');-- return { indexable: d.indexable, listable: d.listable, reasons: Array.isArray(d.codes) ? [...d.codes] : [], gate: d.gate || null };--}++const AUTHORSHIP = Object.freeze({ human: 'human', hybrid: 'ai_assisted', ai: 'ai_generated', imported: 'imported' });++++/** Gate reason code → Search's known reason (roadmap §32.3 names as listed in the contract). */++const SEARCH_REASONS = Object.freeze({++ deleted: 'deleted',++ private: 'private',++ gated: 'members_only',++ unlisted: 'unlisted',++ draft: 'draft',++ unpublished: 'not_published',++ ai_generated_unreviewed: 'ai_unreviewed',++ stub_provider: 'stub_provider',++ unreviewed_sensitive: 'sensitive_unreviewed',++ thin: 'thin_content',++ unsourced: 'unsourced',++ unsupported_claims: 'unsupported_claims',++ missing_canonical: 'missing_canonical_url',++ noindex_requested: 'owner_decision',++ // no Search equivalent; sent under the gate's own stable code++ takedown: 'takedown',++ retracted: 'retracted',++ expired: 'expired',++ stale_price: 'stale_price',++ duplicate_of: 'duplicate_of',++});++++const clip = (s, max) => {++ const t = String(s == null ? '' : s);++ return t.length > max ? t.slice(0, max) : t;++};function iso(v, name) {if (v == null || v === '') return null;@@@ -43,109 +94,190 @@ function iso(v, name) {return d.toISOString();}--function checkFacets(facets) {-- const out = {};-- for (const [k, v] of Object.entries(facets || {})) {-- if (!SEGMENT_RE.test(k)) throw new TypeError(`facet name "${k}" must match ${SEGMENT_RE}`);-- if (v == null) continue;-- const ok = (x) => typeof x === 'string' || typeof x === 'boolean' || (typeof x === 'number' && Number.isFinite(x));-- if (Array.isArray(v) ? !v.every(ok) : !ok(v)) throw new TypeError(`facet "${k}" must be a string, number, boolean or an array of them`);-- out[k] = Array.isArray(v) ? [...v] : v;-- }++function checkIdentity({ owner, type, id, revision }) {++ if (!OWNER_RE.test(String(owner || ''))) throw new TypeError('owner must be a service slug of [a-z0-9_] (it prefixes event types)');++ if (!TYPE_RE.test(String(type || ''))) throw new TypeError('type must match ^[a-z][a-z0-9_]{1,39}$');++ if (typeof id !== 'string' || !ID_RE.test(id)) throw new TypeError('id must match ^[A-Za-z0-9][A-Za-z0-9._:~-]{0,127}$');++ if (!Number.isInteger(revision) || revision < 0) throw new TypeError('revision must be a non-negative integer');++}++++/** The gate decision as Search's indexability { decision, reasons }. */++function searchIndexability(decision) {++ if (!decision || typeof decision.indexable !== 'boolean' || typeof decision.listable !== 'boolean') throw new TypeError('decision (seo.evaluate) is required');++ const codes = Array.isArray(decision.codes) ? decision.codes : [];++ const reasons = [...new Set(codes.map((c) => SEARCH_REASONS[c] || c))].slice(0, LIMITS.reasons);++ return { decision: decision.indexable ? 'index' : 'noindex', reasons };++}++++function uniqueKeys(list, re, max, what) {++ const out = [...new Set((list || []).map(String))];++ if (out.length > max) throw new TypeError(`acl.${what} holds at most ${max} entries`);++ for (const k of out) if (!re.test(k)) throw new TypeError(`acl.${what} entry "${k}" is not valid`);return out;}--function checkProvenance(p = {}) {++function checkFacets(facets) {++ const entries = Object.entries(facets || {}).filter(([, v]) => v != null);++ if (entries.length > LIMITS.facets) throw new TypeError(`at most ${LIMITS.facets} facets`);const out = {};-- if (p.authorship) {-- out.authorship = { mode: p.authorship.mode };-- if (p.authorship.workflow) out.authorship.workflow = { id: p.authorship.workflow.id, run_id: p.authorship.workflow.runId };-- if (p.authorship.reviewed != null) out.authorship.reviewed = Boolean(p.authorship.reviewed);-- }-- if (Array.isArray(p.citations)) {-- out.citations = p.citations.map((c) => {-- const o = {};-- if (c.sourceItemId) o.source_item_id = String(c.sourceItemId);-- if (c.url) o.url = String(c.url);-- if (c.retrievedAt) o.retrieved_at = iso(c.retrievedAt, 'citation.retrievedAt');-- if (c.licenseNote) o.license_note = String(c.licenseNote);-- return o;-- }).filter((o) => o.source_item_id || o.url);++ for (const [k, v] of entries) {++ if (!FACET_RE.test(k)) throw new TypeError(`facet name "${k}" must match ${FACET_RE}`);++ if (Array.isArray(v)) {++ if (v.length > LIMITS.facetArray) throw new TypeError(`facet "${k}" holds at most ${LIMITS.facetArray} values`);++ if (!v.every((x) => typeof x === 'string')) throw new TypeError(`facet "${k}": arrays must hold strings`);++ out[k] = v.map((x) => clip(x, LIMITS.facetString));++ } else if (typeof v === 'string') out[k] = clip(v, LIMITS.facetString);++ else if (typeof v === 'boolean' || (typeof v === 'number' && Number.isFinite(v))) out[k] = v;++ else throw new TypeError(`facet "${k}" must be a string, number, boolean or an array of strings`);}-- if (p.derivedFrom) out.derived_from = p.derivedFrom;return out;}--function checkIdentity({ service, type, id, revision }) {-- if (!SERVICE_RE.test(String(service || ''))) throw new TypeError('service must be a service slug');-- if (!SEGMENT_RE.test(String(type || ''))) throw new TypeError('type must match ^[a-z][a-z0-9_]{0,39}$');-- if (typeof id !== 'string' || !id || id.length > 200) throw new TypeError('id must be a non-empty string');-- if (!Number.isInteger(revision) || revision < 0) throw new TypeError('revision must be a non-negative integer');++function httpUrl(u) {++ if (!u) return null;++ try { const x = new URL(String(u)); return x.protocol === 'http:' || x.protocol === 'https:' ? x.toString() : null; } catch { return null; }}--/** A deletion marker: Search removes (resource, revision ≤ this) and caches purge it. */--function tombstone({ service, type, id, revision, state = 'deleted', updatedAt = null }) {-- checkIdentity({ service, type, id, revision });-- return {-- schema: SCHEMA, owner_service: service, resource_type: type, resource_id: id, revision,-- publication_state: state, deleted: true, updated_at: iso(updatedAt, 'updatedAt'),++/**++ * Provenance references from citations (citations store rows), the authorship record and extra refs++ * ([{ service, type, id, revision?, label?, url?, retrievedAt?, stub? }]).++ */++function buildProvenance(owner, { citations = [], authorship = null, refs = [] } = {}) {++ const out = [];++ const push = (r) => {++ const o = { service: r.service, type: r.type, id: clip(r.id, 128) };++ if (!REF_SERVICE_RE.test(o.service) || !TYPE_RE.test(o.type) || !o.id) throw new TypeError(`invalid provenance reference ${JSON.stringify(r)}`);++ if (Number.isInteger(r.revision) && r.revision >= 0) o.revision = r.revision;++ if (r.label) o.label = clip(r.label, 200);++ const url = httpUrl(r.url);++ if (url) o.url = url;++ const at = iso(r.retrievedAt, 'retrievedAt');++ if (at) o.retrieved_at = at;++ if (r.stub === true) o.stub = true;++ out.push(o);};++ if (authorship && authorship.workflow) push({ service: 'ai', type: 'run', id: authorship.workflow.runId, label: authorship.workflow.id, stub: Boolean(authorship.stubProvider) });++ for (const c of citations) {++ if (!c) continue;++ if (c.sourceItemId) push({ service: 'sources', type: 'item', id: c.sourceItemId, label: c.title, url: c.url, retrievedAt: c.retrievedAt });++ else if (c.url && c.id != null) push({ service: owner.replace(/_/g, '-'), type: 'citation', id: String(c.id), label: c.title, url: c.url, retrievedAt: c.retrievedAt });++ }++ for (const r of refs) push(r);++ if (out.length > LIMITS.provenance) throw new TypeError(`at most ${LIMITS.provenance} provenance references`);++ return out;++}++++/** The deletion marker: removes (owner, type, id) at this revision or older from every result. */++function tombstone({ owner, type, id, revision }) {++ checkIdentity({ owner, type, id, revision });++ return { owner, type, id, revision, deleted: true };}/**-- * Build the Search document for a revision. Non-published, deleted and unlisted resources come-- * back as tombstones.++ * The Search document for one revision, or a tombstone when the resource must not be in Search.++ * state: draft | scheduled | published | unpublished | retracted | archived | deleted++ * visibility: public | unlisted | gated | private*/function buildIndexDocument({-- service, type, id, revision, state, visibility, acl = {}, canonicalUrl, title, summary = null, body = '',-- facets = {}, provenance = {}, decision, publishedAt = null, updatedAt = null, locale = null, deleted = false,++ owner, type, id, revision, state, visibility, acl = {}, canonicalUrl, title, summary = null, body = '',++ facets = {}, authorship = null, citations = [], provenance = [], decision, publishedAt = null, updatedAt = null,++ language = null, deleted = false, includeUnlisted = false,} = {}) {-- checkIdentity({ service, type, id, revision });-- if (deleted || state !== 'published' || visibility === 'unlisted') {-- return tombstone({ service, type, id, revision, state: deleted ? 'deleted' : state || 'deleted', updatedAt });-- }-- const indexability = assertDecision(decision);-- let docAcl;-- if (visibility === 'public') docAcl = { public: true };-- else if (visibility === 'private') {-- const subjects = (acl.subjects || []).map(String);-- if (!subjects.length || !subjects.every((s) => SUBJECT_ID_RE.test(s))) throw new TypeError('private documents need acl.subjects (usr_/gst_/app_/mod_ ids)');-- docAcl = { public: false, subjects };-- } else if (visibility === 'gated') {-- const entitlements = (acl.entitlements || []).map(String).filter(Boolean);-- if (!entitlements.length) throw new TypeError('gated documents need acl.entitlements');-- docAcl = { public: false, entitlements, ...(acl.subjects && acl.subjects.length ? { subjects: acl.subjects.map(String) } : {}) };++ checkIdentity({ owner, type, id, revision });++ if (deleted || state !== 'published' || (visibility === 'unlisted' && !includeUnlisted)) return tombstone({ owner, type, id, revision });++++ const doc = { owner, type, id, revision, deleted: false };++ if (visibility === 'public') {++ doc.visibility = 'public';++ } else if (visibility === 'unlisted') {++ doc.visibility = 'unlisted';++ const subjects = uniqueKeys(acl.subjects, PERSON_RE, LIMITS.acl.subjects, 'subjects');++ if (subjects.length) doc.acl = { subjects };++ } else if (visibility === 'private') {++ const subjects = uniqueKeys(acl.subjects, PERSON_RE, LIMITS.acl.subjects, 'subjects');++ if (!subjects.length) throw new TypeError('private documents need acl.subjects (usr_/gst_ ids)');++ doc.visibility = 'private';++ doc.acl = { subjects };++ } else if (visibility === 'gated' || visibility === 'members') {++ const a = {++ subjects: uniqueKeys(acl.subjects, PERSON_RE, LIMITS.acl.subjects, 'subjects'),++ groups: uniqueKeys(acl.groups, KEY_RE, LIMITS.acl.groups, 'groups'),++ entitlements: uniqueKeys(acl.entitlements, KEY_RE, LIMITS.acl.entitlements, 'entitlements'),++ };++ for (const k of Object.keys(a)) if (!a[k].length) delete a[k];++ if (!Object.keys(a).length) throw new TypeError('gated (members) documents need acl.entitlements, acl.groups or acl.subjects');++ doc.visibility = 'members';++ doc.acl = a;} else {-- throw new TypeError('visibility must be public, private, gated or unlisted');++ throw new TypeError('visibility must be public, unlisted, gated or private');}-- if (!canonicalUrl || !/^https?:\/\//.test(canonicalUrl)) throw new TypeError('canonicalUrl must be absolute');++++ const url = httpUrl(canonicalUrl);++ if (!url) throw new TypeError('canonicalUrl must be an absolute http(s) URL');if (!title) throw new TypeError('title is required');-- const text = String(body == null ? '' : body);-- const doc = {-- schema: SCHEMA, owner_service: service, resource_type: type, resource_id: id, revision,-- visibility, acl: docAcl, canonical_url: canonicalUrl, title: String(title),-- summary: summary == null ? null : String(summary), body: text.slice(0, MAX_BODY),-- facets: checkFacets(facets), provenance: checkProvenance(provenance),-- publication_state: 'published', deleted: false, indexability,-- published_at: iso(publishedAt, 'publishedAt'), updated_at: iso(updatedAt, 'updatedAt'),-- };-- if (text.length > MAX_BODY) doc.body_truncated = true;-- if (locale) doc.locale = String(locale);++ doc.canonical_url = url;++ doc.title = clip(title, LIMITS.title);++ if (summary != null && summary !== '') doc.summary = clip(summary, LIMITS.summary);++ doc.body = clip(body, LIMITS.body);++ doc.facets = checkFacets(facets);++ if (language) {++ if (!LANG_RE.test(language)) throw new TypeError('language must be a BCP 47 tag');++ doc.language = language;++ }++ if (authorship) {++ if (!AUTHORSHIP[authorship.mode]) throw new TypeError('authorship.mode must be human, ai, hybrid or imported');++ doc.authorship = AUTHORSHIP[authorship.mode];++ }++ doc.provenance = buildProvenance(owner, { citations, authorship, refs: provenance });++ doc.publication_state = 'published';++ doc.published_at = iso(publishedAt, 'publishedAt');++ doc.updated_at = iso(updatedAt, 'updatedAt');++ doc.indexability = searchIndexability(decision);return doc;}/**-- * Which publication event a transition produces. before/after: { state, visibility, revision }-- * (before may be null for a new resource). → 'published' | 'updated' | 'unpublished' | 'deleted' | null++ * Monotonic index revisions per resource, in <prefix>_index_revisions of the product's database.++ * const seq = createIndexSequencer(db, { prefix: 'wiki' });++ * const doc = seq.stamp(hooks.buildIndexDocument({ ..., revision: 0 })); // revision replaced*/--function actionFor(before, after) {-- const was = Boolean(before && before.state === 'published');-- if (after.state === 'deleted') return before && before.state === 'deleted' ? null : (before ? 'deleted' : null);-- const is = after.state === 'published';-- if (!was && is) return 'published';-- if (was && !is) return 'unpublished';-- if (was && is && (before.revision !== after.revision || before.visibility !== after.visibility)) return 'updated';-- return null;++function createIndexSequencer(db, { prefix, now } = {}) {++ const { assertDb, assertPrefix, clockOf, sha256, stableStringify } = require('./internal');++ assertDb(db);++ assertPrefix(prefix);++ const clock = clockOf(now);++ const T = `${prefix}_index_revisions`;++ db.exec(`++ CREATE TABLE IF NOT EXISTS ${T} (++ owner TEXT NOT NULL,++ type TEXT NOT NULL,++ id TEXT NOT NULL,++ revision INTEGER NOT NULL,++ hash TEXT NOT NULL,++ updated_at INTEGER NOT NULL,++ PRIMARY KEY (owner, type, id)++ );++ `);++ const get = db.prepare(`SELECT revision, hash FROM ${T} WHERE owner = ? AND type = ? AND id = ?`);++ const put = db.prepare(`INSERT INTO ${T} (owner, type, id, revision, hash, updated_at) VALUES (@owner, @type, @id, @revision, @hash, @now)++ ON CONFLICT (owner, type, id) DO UPDATE SET revision = excluded.revision, hash = excluded.hash, updated_at = excluded.updated_at`);++ return {++ table: T,++ /** The document with its revision set: unchanged → the last revision, changed → last + 1. */++ stamp(document) {++ checkIdentity(document);++ const { revision: _ignored, ...rest } = document;++ const hash = sha256(stableStringify(rest));++ return db.transaction(() => {++ const cur = get.get(document.owner, document.type, document.id);++ if (cur && cur.hash === hash) return { ...document, revision: cur.revision };++ const revision = Math.max(cur ? cur.revision + 1 : 1, document.revision);++ put.run({ owner: document.owner, type: document.type, id: document.id, revision, hash, now: clock() });++ return { ...document, revision };++ })();++ },++ current(owner, type, id) { const r = get.get(owner, type, id); return r ? r.revision : null; },++ };}/** 'usr_…' | 'svc:wiki' | 'service:wiki' | { type, id } → SubjectRef. */@@@ -160,38 +292,88 @@ function subjectRef(actor) {throw new TypeError('actor must be a subject (usr_…, gst_…, svc:<service>) or a SubjectRef');}++function envelope({ source, eventType, actor, now, subject, payload, visibility, traceId, eventId }) {++ const env = {++ event_type: eventType,++ version: 1,++ source,++ actor: subjectRef(actor),++ timestamp: new Date(now).toISOString(),++ visibility,++ subject,++ payload,++ };++ if (eventId) env.event_id = eventId;++ if (traceId && /^[0-9a-f]{32}$/.test(traceId)) env.trace_id = traceId;++ return env;++}++++/**++ * The event OpenVibe.Search consumes for a document or tombstone:++ * `<owner>.index_document.upserted` (payload = document) or `.deleted` (payload = { type, id, revision }).++ * actor defaults to the owner service itself.++ */++function indexEvent({ document, actor = null, now = Date.now(), traceId = null, eventId = null } = {}) {++ if (!document || typeof document !== 'object') throw new TypeError('document (buildIndexDocument/tombstone) is required');++ checkIdentity(document);++ const { owner, type, id, revision } = document;++ const deleted = document.deleted === true;++ return envelope({++ source: owner,++ eventType: `${owner}.index_document.${deleted ? 'deleted' : 'upserted'}`,++ actor: actor || { type: 'service', id: owner.replace(/_/g, '-') },++ now, traceId, eventId,++ subject: { type, id, revision },++ visibility: 'internal',++ payload: deleted ? { type, id, revision } : document,++ });++}++/**-- * `<product>.<type>.<action>` envelope (contracts events/event-envelope@1) for the product's outbox.-- * document: the index document (or tombstone) for this revision — it becomes the payload.++ * Which product event a transition produces. before/after: { state, visibility, revision }++ * (before may be null for a new resource). → 'published' | 'updated' | 'unpublished' | 'deleted' | null++ */++function actionFor(before, after) {++ const was = Boolean(before && before.state === 'published');++ if (after.state === 'deleted') return before && before.state === 'deleted' ? null : (before ? 'deleted' : null);++ const is = after.state === 'published';++ if (!was && is) return 'published';++ if (was && !is) return 'unpublished';++ if (was && is && (before.revision !== after.revision || before.visibility !== after.visibility)) return 'updated';++ return null;++}++++/**++ * The product's own domain event `<product>.<type>.<action>` (e.g. wiki.page.published), for++ * subscribers other than Search. Payload: canonical URL, publication state, Search-shaped++ * indexability — not the body (Search gets that through indexEvent). Public only when the++ * resource is public and the gate lets it be listed; otherwise internal.*/function publicationEvent({ product, type, action, id, revision, actor, document, decision = null, now = Date.now(), traceId = null, eventId = null, extra = {} } = {}) {-- if (!SERVICE_RE.test(String(product || '')) || !SEGMENT_RE.test(String(product))) throw new TypeError('product must be a lowercase service slug without dashes, e.g. "wiki"');-- if (!SEGMENT_RE.test(String(type || ''))) throw new TypeError('type must match ^[a-z][a-z0-9_]{0,39}$');++ if (!OWNER_RE.test(String(product || ''))) throw new TypeError('product must be a service slug of [a-z0-9_], e.g. "wiki"');++ if (!TYPE_RE.test(String(type || ''))) throw new TypeError('type must match ^[a-z][a-z0-9_]{1,39}$');if (!ACTIONS.includes(action)) throw new TypeError(`action must be one of ${ACTIONS.join(', ')}`);-- if (!document || document.resource_id !== id) throw new TypeError('document (buildIndexDocument/tombstone) for this resource is required');-- if ((action === 'deleted' || action === 'unpublished') && !document.deleted) throw new TypeError(`${action} events carry a tombstone`);-- // A published-but-unlisted resource is a tombstone with publication_state 'published': it may be updated, not listed.-- if ((action === 'published' || action === 'updated') && document.publication_state !== 'published') throw new TypeError(`${action} events carry a published document`);-- const isPublic = !document.deleted && document.visibility === 'public' && document.indexability && document.indexability.listable;-- const env = {-- event_type: `${product}.${type}.${action}`,-- version: 1,++ if (!document || document.id !== id) throw new TypeError('document (buildIndexDocument/tombstone) for this resource is required');++ const removal = action === 'deleted' || action === 'unpublished';++ if (action === 'published' && document.deleted) throw new TypeError('published events carry a published document');++ if (removal && !document.deleted) throw new TypeError(`${action} events carry a tombstone`);++ const isPublic = !document.deleted && document.visibility === 'public' && Boolean(decision && decision.listable);++ return envelope({source: product,-- actor: subjectRef(actor),-- timestamp: new Date(now).toISOString(),-- visibility: isPublic ? 'public' : 'internal',++ eventType: `${product}.${type}.${action}`,++ actor, now, traceId, eventId,subject: { type, id, revision },++ visibility: isPublic ? 'public' : 'internal',payload: {canonical_url: document.canonical_url || null,-- publication_state: document.publication_state,-- indexability: decision ? assertDecision(decision) : document.indexability || null,-- document,++ publication_state: document.deleted ? (action === 'deleted' ? 'deleted' : 'unpublished') : document.publication_state,++ indexability: document.indexability || (decision ? searchIndexability(decision) : null),...extra,},-- };-- if (eventId) env.event_id = eventId;-- if (traceId && /^[0-9a-f]{32}$/.test(traceId)) env.trace_id = traceId;-- return env;++ });}--module.exports = { SCHEMA, ACTIONS, buildIndexDocument, tombstone, actionFor, publicationEvent, subjectRef };++module.exports = {++ SCHEMA, ACTIONS, AUTHORSHIP, SEARCH_REASONS, LIMITS,++ buildIndexDocument, tombstone, searchIndexability, createIndexSequencer, buildProvenance, indexEvent, actionFor, publicationEvent, subjectRef,++};@@@ -1,12 +1,12 @@{"name": "openvibe-publishing",-- "version": "0.1.0",++ "version": "0.2.0","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "openvibe-publishing",-- "version": "0.1.0",++ "version": "0.2.0","license": "MIT","dependencies": {"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.0.0"@@@ -15,7 +15,7 @@"ajv": "^8.17.1","ajv-formats": "^3.0.1","better-sqlite3": "^11.10.0",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.7.0"++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.12.0"},"engines": {"node": ">=20"@@@ -461,9 +461,9 @@}},"node_modules/openvibe-contracts": {-- "version": "0.7.0",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.7.0",-- "integrity": "sha512-gUrjxbTy/HX0QMQGoDPphmFdwm7iitmNMMchs9tDSDGR47FRFHL/RgBgIKYR1S7M6ICu261YycFV8+YVglafXA==",++ "version": "0.12.0",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.12.0",++ "integrity": "sha512-Cf7rxAk0YXCg7G+HcFldH65+URPVeHY2hoOsqpJ/+KmAFyaK8lX6D1OrdyozH52t+rfsQcRlRqkO5ldEVdZRHA==","dev": true,"license": "AGPL-3.0-only","dependencies": {@@@ -1,6 +1,6 @@{"name": "openvibe-publishing",-- "version": "0.1.0",++ "version": "0.2.0","description": "Shared publishing packages for the OpenVibe publication products: revisions, scheduling, taxonomy, citations, media attachments, discussion references, the SEO indexability gate, authorship, index hooks and SSR helpers. Packages, not an authority.","main": "index.js","exports": {@@@ -20,7 +20,6 @@"files": ["index.js","lib/",-- "docs/","CHANGELOG.md"],"scripts": {@@@ -50,6 +49,6 @@"ajv": "^8.17.1","ajv-formats": "^3.0.1","better-sqlite3": "^11.10.0",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.7.0"++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.12.0"}}@@@ -1,67 +1,164 @@'use strict';const assert = require('assert');--const fs = require('fs');--const path = require('path');--const Ajv = require('ajv/dist/2020');--const addFormats = require('ajv-formats');const contracts = require('openvibe-contracts');const hooks = require('../lib/index-hooks');const seo = require('../lib/seo');--const { suite } = require('./helpers/db');++const authorship = require('../lib/authorship');++const { openDb, fakeClock, suite } = require('./helpers/db');const { test, run } = suite();const USER = 'usr_01J8Z6Q3KX0000000000000000';--const ajv = new Ajv({ allErrors: true, strict: false });--addFormats(ajv);--const validateDoc = ajv.compile(JSON.parse(fs.readFileSync(path.join(__dirname, '..', 'docs', 'contracts-proposal', 'search.index-document.v1.json'), 'utf8')));++const RUN = 'run_01J8Z6Q3KX0000000000000000';++++const valid = (doc) => {++ const v = contracts.validate('search.index-document@1', doc);++ assert.ok(v.valid, `search.index-document@1: ${JSON.stringify(v.errors)} in ${JSON.stringify(doc)}`);++ return doc;++};++const validEnvelope = (env) => {++ const v = contracts.validate('events.event-envelope@1', { ...env, event_id: contracts.ids.newId('event') });++ assert.ok(v.valid, `events.event-envelope@1: ${JSON.stringify(v.errors)}`);++ return env;++};++++/** What OpenVibe.Search's webhook (server/api/webhook.js documentFromEvent) does with an envelope. */++function searchReads(env) {++ const m = /^([a-z][a-z0-9-]{1,39})\.index_document\.(upserted|deleted)$/.exec(env.event_type);++ assert.ok(m, `not an index_document event: ${env.event_type}`);++ assert.strictEqual(m[1], env.source, 'event_type owner must be the source');++ const p = env.payload;++ if (p.owner !== undefined) assert.strictEqual(p.owner, env.source);++ const doc = m[2] === 'deleted' ? { owner: env.source, type: p.type, id: p.id, revision: p.revision, deleted: true } : { ...p, owner: env.source };++ assert.strictEqual(env.subject.type, doc.type);++ assert.strictEqual(env.subject.id, doc.id);++ assert.strictEqual(env.subject.revision, doc.revision);++ assert.strictEqual(env.visibility, 'internal');++ return valid(doc);++}const facts = { state: 'published', visibility: 'public', canonicalUrl: 'https://openvibe.wiki/p/rye', wordCount: 400, citationCount: 2 };--const decision = seo.evaluate(facts, { policy: { requireSources: true } });++const decisionFor = (extra = {}) => seo.evaluate({ ...facts, ...extra }, { policy: { requireSources: true } });const input = (extra = {}) => ({-- service: 'wiki', type: 'page', id: 'pg_1', revision: 4, state: 'published', visibility: 'public',++ owner: 'wiki', type: 'page', id: 'pg_1', revision: 4, state: 'published', visibility: 'public',canonicalUrl: 'https://openvibe.wiki/p/rye', title: 'Rye', summary: 'A cereal grass.', body: 'Rye is a grass grown as a grain.',-- facets: { space: 'food', tags: ['grain', 'bread'] },-- provenance: { authorship: { mode: 'human' }, citations: [{ sourceItemId: 'src_1', url: 'https://example.org/rye', retrievedAt: '2026-09-20T10:00:00Z' }] },-- decision, publishedAt: '2026-09-20T12:00:00Z', updatedAt: '2026-09-21T12:00:00Z', ...extra,++ facets: { space: 'food', tags: ['grain', 'bread'], words: 400 },++ authorship: authorship.record({ mode: 'human', authors: [USER] }),++ citations: [++ { id: 7, sourceItemId: 'src_1', url: 'https://example.org/rye', title: 'Rye', retrievedAt: '2026-09-20T10:00:00Z' },++ { id: 8, url: 'https://example.org/other' },++ ],++ decision: decisionFor(), publishedAt: '2026-09-20T12:00:00Z', updatedAt: '2026-09-21T12:00:00Z', language: 'en', ...extra,});--function withEventId(env) { return { ...env, event_id: contracts.ids.newId('event') }; }----test('a published revision becomes a full index document with every §12.3 field', () => {-- const doc = hooks.buildIndexDocument(input());-- for (const k of ['owner_service', 'resource_type', 'resource_id', 'revision', 'visibility', 'acl', 'canonical_url', 'title', 'summary', 'body', 'facets', 'provenance', 'publication_state', 'deleted', 'indexability']) {-- assert.ok(k in doc, `missing ${k}`);-- }-- assert.deepStrictEqual(doc.acl, { public: true });-- assert.deepStrictEqual(doc.indexability, { indexable: true, listable: true, reasons: [], gate: 'openvibe-publishing/seo@1' });-- assert.deepStrictEqual(doc.provenance.citations, [{ source_item_id: 'src_1', url: 'https://example.org/rye', retrieved_at: '2026-09-20T10:00:00.000Z' }]);-- assert.ok(validateDoc(doc), JSON.stringify(validateDoc.errors));++test('a published revision becomes exactly the released search.index-document@1 shape', () => {++ const doc = valid(hooks.buildIndexDocument(input()));++ assert.deepStrictEqual(Object.keys(doc), ['owner', 'type', 'id', 'revision', 'deleted', 'visibility', 'canonical_url', 'title', 'summary', 'body',++ 'facets', 'language', 'authorship', 'provenance', 'publication_state', 'published_at', 'updated_at', 'indexability']);++ assert.strictEqual(doc.owner, 'wiki');++ assert.strictEqual(doc.visibility, 'public');++ assert.ok(!('acl' in doc), 'a public document carries no ACL');++ assert.strictEqual(doc.authorship, 'human');++ assert.deepStrictEqual(doc.indexability, { decision: 'index', reasons: [] });++ assert.deepStrictEqual(doc.provenance, [++ { service: 'sources', type: 'item', id: 'src_1', label: 'Rye', url: 'https://example.org/rye', retrieved_at: '2026-09-20T10:00:00.000Z' },++ { service: 'wiki', type: 'citation', id: '8', url: 'https://example.org/other' },++ ], 'no retrieved_at invented for the second citation');});--test('deleted, unpublished, draft and unlisted resources become tombstones with no content', () => {-- for (const extra of [{ deleted: true }, { state: 'unpublished' }, { state: 'draft' }, { visibility: 'unlisted' }]) {-- const doc = hooks.buildIndexDocument(input(extra));-- assert.strictEqual(doc.deleted, true, JSON.stringify(extra));-- for (const k of ['title', 'summary', 'body', 'canonical_url', 'acl']) assert.ok(!(k in doc), `${k} leaked in a tombstone`);-- assert.ok(validateDoc(doc), JSON.stringify(validateDoc.errors));-- }++test('authorship, visibility and reason codes map onto the contract enums', () => {++ const ai = authorship.record({ mode: 'ai', workflow: { id: 'wiki.generate_page', runId: RUN }, stubProvider: true });++ const hybrid = authorship.record({ mode: 'hybrid', authors: [USER], workflow: { id: 'blog.draft_post', runId: RUN } });++ const imported = authorship.record({ mode: 'imported', importedFrom: { label: 'legacy' } });++ assert.strictEqual(valid(hooks.buildIndexDocument(input({ authorship: hybrid }))).authorship, 'ai_assisted');++ assert.strictEqual(valid(hooks.buildIndexDocument(input({ authorship: imported }))).authorship, 'imported');++ const aiDoc = valid(hooks.buildIndexDocument(input({ authorship: ai, decision: decisionFor(authorship.gateFacts(ai)) })));++ assert.strictEqual(aiDoc.authorship, 'ai_generated');++ assert.deepStrictEqual(aiDoc.provenance[0], { service: 'ai', type: 'run', id: RUN, label: 'wiki.generate_page', stub: true });++ assert.deepStrictEqual(aiDoc.indexability, { decision: 'noindex', reasons: ['ai_unreviewed', 'stub_provider'] });++++ const thin = valid(hooks.buildIndexDocument(input({ decision: decisionFor({ wordCount: 3, citationCount: 0, noindex: true, canonicalUrl: undefined }) })));++ assert.deepStrictEqual(thin.indexability.reasons, ['thin_content', 'unsourced', 'missing_canonical_url', 'owner_decision']);++ const dup = valid(hooks.buildIndexDocument(input({ decision: decisionFor({ duplicateOf: 'https://openvibe.wiki/p/x' }) })));++ assert.deepStrictEqual(dup.indexability.reasons, ['duplicate_of']);++ for (const code of Object.keys(hooks.SEARCH_REASONS)) assert.match(hooks.SEARCH_REASONS[code], /^[a-z][a-z0-9_]{1,63}$/);});--test('private content needs an ACL of subjects; gated content needs entitlements', () => {-- const priv = seo.evaluate({ ...facts, visibility: 'private' }, { policy: { requireSources: true } });++test('private needs subjects; gated becomes members with entitlements/groups; nothing non-public reads as public', () => {++ const priv = decisionFor({ visibility: 'private' });assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'private', decision: priv })), /acl.subjects/);-- assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'private', decision: priv, acl: { subjects: ['everyone'] } })), /acl.subjects/);-- const doc = hooks.buildIndexDocument(input({ visibility: 'private', decision: priv, acl: { subjects: [USER] } }));-- assert.deepStrictEqual(doc.acl, { public: false, subjects: [USER] });-- assert.strictEqual(doc.indexability.listable, false);-- assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'gated', decision: priv })), /entitlements/);-- const vip = hooks.buildIndexDocument(input({ visibility: 'gated', decision: priv, acl: { entitlements: ['vip.tier.gold'] } }));-- assert.deepStrictEqual(vip.acl, { public: false, entitlements: ['vip.tier.gold'] });-- assert.ok(validateDoc(vip));++ assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'private', decision: priv, acl: { subjects: ['everyone'] } })), /not valid/);++ const doc = valid(hooks.buildIndexDocument(input({ visibility: 'private', decision: priv, acl: { subjects: [USER, USER] } })));++ assert.strictEqual(doc.visibility, 'private');++ assert.deepStrictEqual(doc.acl, { subjects: [USER] });++ assert.deepStrictEqual(doc.indexability.reasons, ['private']);++ const gated = decisionFor({ visibility: 'gated' });++ assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'gated', decision: gated })), /members/);++ const vip = valid(hooks.buildIndexDocument(input({ visibility: 'gated', decision: gated, acl: { entitlements: ['vip.plan:pln_gold'], groups: ['wiki.space:food:member'] } })));++ assert.strictEqual(vip.visibility, 'members');++ assert.deepStrictEqual(vip.acl, { groups: ['wiki.space:food:member'], entitlements: ['vip.plan:pln_gold'] });++ assert.deepStrictEqual(vip.indexability.reasons, ['members_only']);++ assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'friends' })), /visibility/);assert.throws(() => hooks.buildIndexDocument(input({ decision: undefined })), /decision/);});--test('actionFor maps state transitions to event actions', () => {++test('tombstones are exactly { owner, type, id, revision, deleted: true } for anything that must leave Search', () => {++ for (const extra of [{ deleted: true }, { state: 'unpublished' }, { state: 'draft' }, { state: 'scheduled' }, { state: 'retracted' }, { state: 'archived' }, { visibility: 'unlisted' }]) {++ const doc = valid(hooks.buildIndexDocument(input(extra)));++ assert.deepStrictEqual(doc, { owner: 'wiki', type: 'page', id: 'pg_1', revision: 4, deleted: true }, JSON.stringify(extra));++ }++ const unlisted = valid(hooks.buildIndexDocument(input({ visibility: 'unlisted', includeUnlisted: true, decision: decisionFor({ visibility: 'unlisted' }) })));++ assert.strictEqual(unlisted.visibility, 'unlisted');++ assert.deepStrictEqual(unlisted.indexability.reasons, ['unlisted']);++});++++test('limits of the contract are respected: long text is clipped, bad facets and ids refused', () => {++ const doc = valid(hooks.buildIndexDocument(input({ title: 't'.repeat(900), summary: 's'.repeat(5000), body: 'b'.repeat(60000) })));++ assert.strictEqual(doc.title.length, 500);++ assert.strictEqual(doc.summary.length, 4000);++ assert.strictEqual(doc.body.length, 48000);++ assert.throws(() => hooks.buildIndexDocument(input({ facets: { tags: [1, 2] } })), /strings/);++ assert.throws(() => hooks.buildIndexDocument(input({ facets: Object.fromEntries(Array.from({ length: 21 }, (_, i) => [`f${i}`, 'x'])) })), /20 facets/);++ assert.throws(() => hooks.buildIndexDocument(input({ id: 'has space' })), /id must match/);++ assert.throws(() => hooks.buildIndexDocument(input({ owner: 'open-re' })), /owner/);++ assert.throws(() => hooks.buildIndexDocument(input({ language: 'english!' })), /BCP 47/);++ assert.ok(!('summary' in valid(hooks.buildIndexDocument(input({ summary: null })))));++});++++test('index events are what Search consumes: <owner>.index_document.upserted|deleted', () => {++ const doc = hooks.buildIndexDocument(input());++ const up = validEnvelope(hooks.indexEvent({ document: doc, now: Date.parse('2026-09-22T12:00:00Z') }));++ assert.strictEqual(up.event_type, 'wiki.index_document.upserted');++ assert.strictEqual(up.source, 'wiki');++ assert.deepStrictEqual(up.actor, { type: 'service', id: 'wiki' });++ assert.deepStrictEqual(up.subject, { type: 'page', id: 'pg_1', revision: 4 });++ assert.ok(!('event_id' in up), 'the outbox assigns event_id');++ assert.deepStrictEqual(searchReads(up), doc);++++ const tomb = hooks.tombstone({ owner: 'wiki', type: 'page', id: 'pg_1', revision: 5 });++ const del = validEnvelope(hooks.indexEvent({ document: tomb, actor: USER }));++ assert.strictEqual(del.event_type, 'wiki.index_document.deleted');++ assert.deepStrictEqual(del.payload, { type: 'page', id: 'pg_1', revision: 5 });++ assert.deepStrictEqual(searchReads(del), tomb);++ assert.throws(() => hooks.indexEvent({}), /document/);++});++++test('the sequencer bumps the index revision on every indexed change and replays unchanged documents', () => {++ const seq = hooks.createIndexSequencer(openDb('idx'), { prefix: 'wiki', now: fakeClock() });++ const a = seq.stamp(hooks.buildIndexDocument(input({ revision: 0 })));++ assert.strictEqual(a.revision, 1);++ assert.strictEqual(seq.stamp(hooks.buildIndexDocument(input({ revision: 0 }))).revision, 1, 'same document, same revision');++ const priv = seq.stamp(hooks.buildIndexDocument(input({ revision: 0, visibility: 'private', acl: { subjects: [USER] }, decision: decisionFor({ visibility: 'private' }) })));++ assert.strictEqual(priv.revision, 2, 'a visibility change with the same content still gets a new revision');++ const gone = seq.stamp(hooks.tombstone({ owner: 'wiki', type: 'page', id: 'pg_1', revision: 0 }));++ assert.strictEqual(gone.revision, 3);++ const back = seq.stamp(hooks.buildIndexDocument(input({ revision: 0 })));++ assert.strictEqual(back.revision, 4, 'a restore outranks the tombstone');++ assert.strictEqual(seq.current('wiki', 'page', 'pg_1'), 4);++ valid(back);++ assert.strictEqual(seq.table, 'wiki_index_revisions');++});++++test('actionFor maps state transitions to product events', () => {const pub = { state: 'published', visibility: 'public', revision: 1 };assert.strictEqual(hooks.actionFor(null, pub), 'published');assert.strictEqual(hooks.actionFor({ state: 'draft', revision: 1 }, pub), 'published');@@@ -71,41 +168,26 @@ test('actionFor maps state transitions to event actions', () => {assert.strictEqual(hooks.actionFor(pub, { ...pub, state: 'unpublished' }), 'unpublished');assert.strictEqual(hooks.actionFor(pub, { ...pub, state: 'deleted' }), 'deleted');assert.strictEqual(hooks.actionFor({ state: 'deleted' }, { state: 'deleted' }), null);-- assert.strictEqual(hooks.actionFor({ state: 'draft' }, { state: 'draft' }), null);});--test('publication events are valid contracts envelopes: <product>.<type>.<action>', () => {++test('product events <product>.<type>.<action> are valid envelopes, internal unless public and listable', () => {++ const decision = decisionFor();const doc = hooks.buildIndexDocument(input());-- const env = hooks.publicationEvent({ product: 'wiki', type: 'page', action: 'published', id: 'pg_1', revision: 4, actor: USER, document: doc, decision, now: Date.parse('2026-09-22T12:00:00Z') });++ const env = validEnvelope(hooks.publicationEvent({ product: 'wiki', type: 'page', action: 'published', id: 'pg_1', revision: 4, actor: USER, document: doc, decision }));assert.strictEqual(env.event_type, 'wiki.page.published');assert.strictEqual(env.visibility, 'public');-- assert.deepStrictEqual(env.actor, { type: 'user', id: USER });-- assert.deepStrictEqual(env.subject, { type: 'page', id: 'pg_1', revision: 4 });-- assert.strictEqual(env.payload.document, doc);-- assert.ok(!('event_id' in env), 'the outbox assigns event_id');-- const check = contracts.validate('events.event-envelope@1', withEventId(env));-- assert.ok(check.valid, JSON.stringify(check.errors));---- const tomb = hooks.buildIndexDocument(input({ deleted: true }));-- const del = hooks.publicationEvent({ product: 'blog', type: 'post', action: 'deleted', id: 'pg_1', revision: 4, actor: 'svc:blog', document: tomb });-- assert.strictEqual(del.event_type, 'blog.post.deleted');++ assert.deepStrictEqual(env.payload, { canonical_url: 'https://openvibe.wiki/p/rye', publication_state: 'published', indexability: { decision: 'index', reasons: [] } });++ assert.ok(!('body' in env.payload), 'no body in the product event');++ const priv = decisionFor({ visibility: 'private' });++ const pdoc = hooks.buildIndexDocument(input({ visibility: 'private', acl: { subjects: [USER] }, decision: priv }));++ assert.strictEqual(validEnvelope(hooks.publicationEvent({ product: 'wiki', type: 'page', action: 'updated', id: 'pg_1', revision: 4, actor: USER, document: pdoc, decision: priv })).visibility, 'internal');++ const tomb = hooks.tombstone({ owner: 'blog', type: 'post', id: 'pg_1', revision: 9 });++ const del = validEnvelope(hooks.publicationEvent({ product: 'blog', type: 'post', action: 'deleted', id: 'pg_1', revision: 4, actor: 'svc:blog', document: tomb }));assert.strictEqual(del.visibility, 'internal');-- assert.deepStrictEqual(del.actor, { type: 'service', id: 'blog' });-- assert.ok(contracts.validate('events.event-envelope@1', withEventId(del)).valid);++ assert.strictEqual(del.payload.publication_state, 'deleted');assert.throws(() => hooks.publicationEvent({ product: 'blog', type: 'post', action: 'deleted', id: 'pg_1', revision: 4, actor: USER, document: doc }), /tombstone/);assert.throws(() => hooks.publicationEvent({ product: 'blog', type: 'post', action: 'published', id: 'pg_1', revision: 4, actor: USER, document: tomb }), /published document/);assert.throws(() => hooks.publicationEvent({ product: 'blog', type: 'post', action: 'moved', id: 'pg_1', revision: 4, actor: USER, document: doc }), /action/);});--test('events about private or non-listable content are internal', () => {-- const priv = seo.evaluate({ ...facts, visibility: 'private' }, { policy: { requireSources: true } });-- const doc = hooks.buildIndexDocument(input({ visibility: 'private', decision: priv, acl: { subjects: [USER] } }));-- const env = hooks.publicationEvent({ product: 'wiki', type: 'page', action: 'updated', id: 'pg_1', revision: 4, actor: USER, document: doc });-- assert.strictEqual(env.visibility, 'internal');-- const unlisted = hooks.buildIndexDocument(input({ visibility: 'unlisted' }));-- const env2 = hooks.publicationEvent({ product: 'wiki', type: 'page', action: 'updated', id: 'pg_1', revision: 4, actor: USER, document: unlisted });-- assert.strictEqual(env2.visibility, 'internal');-- assert.ok(contracts.validate('events.event-envelope@1', withEventId(env2)).valid);--});--run();@@@ -7,6 +7,7 @@ const assert = require('assert');const fs = require('fs');const http = require('http');const path = require('path');++const contracts = require('openvibe-contracts');const { tempDir, fakeClock, suite } = require('./helpers/db');const { createWiki } = require('../examples/two-products/wiki/app');const { createBlog } = require('../examples/two-products/blog/app');@@@ -191,10 +192,22 @@ test('HTTP: pages useful without JavaScript, 301 for old slugs, private/deletedassert.doesNotMatch(await (await w.get('/sitemap.xml')).text(), /rye/);assert.doesNotMatch(await (await w.get('/feed.atom')).text(), /<entry>/);assert.strictEqual(JSON.parse(await (await w.get('/feed.json')).text()).items.length, 0);-- const events = wiki.outbox();-- assert.deepStrictEqual(events.map((e) => e.event_type), ['wiki.page.published', 'wiki.page.updated']);-- assert.strictEqual(events[1].visibility, 'internal');-- assert.deepStrictEqual(events[1].payload.document.acl, { public: false, subjects: [ALEX] });++ let events = wiki.outbox();++ assert.deepStrictEqual(events.map((e) => e.event_type), [++ 'wiki.index_document.upserted', 'wiki.page.published', // publish++ 'wiki.index_document.upserted', // rename: new canonical URL, new index revision++ 'wiki.index_document.upserted', 'wiki.page.updated', // made private++ ]);++ const docs = events.filter((e) => e.event_type.startsWith('wiki.index_document.')).map((e) => e.payload);++ assert.deepStrictEqual(docs.map((d) => d.revision), [1, 2, 3], 'every indexed change is a higher revision');++ assert.strictEqual(docs[1].canonical_url, 'https://openvibe.wiki/p/rye-grain');++ assert.strictEqual(docs[2].visibility, 'private');++ assert.deepStrictEqual(docs[2].acl, { subjects: [ALEX] });++ assert.strictEqual(events[4].visibility, 'internal');++ for (const e of events) {++ assert.ok(contracts.validate('events.event-envelope@1', { ...e, event_id: contracts.ids.newId('event') }).valid, e.event_type);++ }++ for (const d of docs) assert.ok(contracts.validate('search.index-document@1', d).valid);blog.setVisibility(post, 'gated');assert.strictEqual((await b.get('/posts/crumb-shots')).status, 404);@@@ -206,10 +219,12 @@ test('HTTP: pages useful without JavaScript, 301 for old slugs, private/deletedwiki.remove(id, { actor: ALEX });assert.strictEqual((await w.get('/p/rye-grain')).status, 410);assert.strictEqual((await w.get('/p/rye')).status, 410);-- const last = wiki.outbox().pop();-- assert.strictEqual(last.event_type, 'wiki.page.deleted');-- assert.strictEqual(last.payload.document.deleted, true);-- assert.ok(!('body' in last.payload.document));++ events = wiki.outbox().slice(-2);++ assert.deepStrictEqual(events.map((e) => e.event_type), ['wiki.index_document.deleted', 'wiki.page.deleted']);++ assert.deepStrictEqual(events[0].payload, { type: 'page', id, revision: 4 });++ assert.deepStrictEqual(events[0].subject, { type: 'page', id, revision: 4 });++ assert.strictEqual(events[1].payload.publication_state, 'deleted');++ for (const e of events) assert.ok(contracts.validate('events.event-envelope@1', { ...e, event_id: contracts.ids.newId('event') }).valid);} finally { w.close(); b.close(); wiki.close(); blog.close(); }});
@@ -4,6 +4,52 @@ All notable changes to `openvibe-publishing`. Versions follow [semver](https://s
@@ -4,6 +4,52 @@ All notable changes to `openvibe-publishing`. Versions follow [semver](https://s
breaking change to any exported function, table layout, reason code or document shape is a new
breaking change to any exported function, table layout, reason code or document shape is a new
major (a minor while 0.x). A release is the git tag `vX.Y.Z`; consumers pin the tag's tarball.
major (a minor while 0.x). A release is the git tag `vX.Y.Z`; consumers pin the tag's tarball.
+## 0.2.0 — 2026-09-22
+
+**Breaking: `index-hooks` now emits the released `search.index-document@1`** (openvibe-contracts
+v0.12.0, owned by OpenVibe.Search) instead of the 0.1.0 proposal, and builds the events Search
+actually consumes. Products on 0.1.0 must update their calls; nothing else changed.
+
+### Changed (breaking)
+- `buildIndexDocument()` takes `owner` (was `service`) and returns exactly the contract shape:
+ `owner`, `type`, `id`, `revision`, `deleted`, `visibility`, `acl`, `canonical_url`, `title`,
+ `summary`, `body`, `facets`, `language` (was `locale`), `authorship`, `provenance`,
+ `publication_state`, `published_at`, `updated_at`, `indexability { decision, reasons }`.
+ Gone: `schema`, `owner_service`, `resource_type`, `resource_id`, `acl.public`, `body_truncated`,
+ the `indexability.indexable/listable/gate` fields and the provenance object.
+- Mappings onto the contract enums: visibility `gated` → `members`; authorship `hybrid` →
+ `ai_assisted`, `ai` → `ai_generated`; gate reason codes → Search's known reasons where one exists
+ (`thin` → `thin_content`, `ai_generated_unreviewed` → `ai_unreviewed`, `unreviewed_sensitive` →
+ `sensitive_unreviewed`, `gated` → `members_only`, `unpublished` → `not_published`,
+ `missing_canonical` → `missing_canonical_url`, `noindex_requested` → `owner_decision`; the rest
+ keep the gate's code). `provenance` is now an array of typed references built from citations
+ (Sources items, or the owner's own citation records), the AI run (with `stub: true` for stub
+ output) and extra `provenance` refs.
+- `tombstone()` returns exactly `{ owner, type, id, revision, deleted: true }`. Anything not
+ published (draft, scheduled, unpublished, retracted, archived, deleted) is still a tombstone, and
+ so is unlisted content unless `includeUnlisted: true` (then `visibility: 'unlisted'`).
+- ACL follows the contract: `subjects` are `usr_`/`gst_` only, plus `groups` and `entitlements`;
+ private needs subjects, members needs at least one of the three. Text is clipped to the contract's
+ limits (title 500, summary 4000, body 48000); facets are checked against its rules.
+- `publicationEvent()` (the product's own `<product>.<type>.<action>` event) no longer embeds the
+ document; its payload is `{ canonical_url, publication_state, indexability }`.
+
+### Added
+- `indexEvent({ document })`: `<owner>.index_document.upserted` (payload = document) or
+ `<owner>.index_document.deleted` (payload = `{ type, id, revision }`), subject
+ `{ type, id, revision }`, visibility internal — the envelopes OpenVibe.Search's webhook consumes.
+- `createIndexSequencer(db, { prefix })`: a monotonic index revision per resource in
+ `<prefix>_index_revisions`. Search refuses a different document at an equal revision and lets a
+ tombstone win ties, so every indexed change (content, visibility, state, canonical URL,
+ decision) needs a higher revision; an unchanged document keeps its revision (a safe replay).
+- `searchIndexability(decision)`, `SEARCH_REASONS`, `AUTHORSHIP`, `LIMITS`.
+- Tests validate every emitted document with `validate('search.index-document@1')` and every
+ envelope with `validate('events.event-envelope@1')` from openvibe-contracts v0.12.0, and mirror
+ Search's webhook checks (owner = source, subject matches the payload).
+
+### Removed
+- `docs/contracts-proposal/search.index-document.v1.json` (superseded by the released contract).
+
## 0.1.0 — 2026-09-22
## 0.1.0 — 2026-09-22
First version (roadmap Wave 15). Packages only: no runtime, no domain, no database.
First version (roadmap Wave 15). Packages only: no runtime, no domain, no database.
@@ -3,8 +3,9 @@
@@ -3,8 +3,9 @@
> Shared publishing packages for the OpenVibe publication products: Wiki, Blog, News, Reviews,
> Shared publishing packages for the OpenVibe publication products: Wiki, Blog, News, Reviews,
> Deals, Coupons and Trade.
> Deals, Coupons and Trade.
-**Status:** alpha, 0.1.0 (roadmap Wave 15). The modules are tested and the exit proof
-(`examples/two-products`) runs in `npm test`; not tagged or released yet. No product uses them in production yet: Wiki and Blog (Wave 16) are the
+**Status:** alpha, 0.2.0 (roadmap Wave 15). The modules are tested and the exit proof
+(`examples/two-products`) runs in `npm test`. v0.1.0 is tagged; 0.2.0 (the released Search
+document shape, a breaking change — see [CHANGELOG.md](CHANGELOG.md)) is not tagged yet. No product uses them in production yet: Wiki and Blog (Wave 16) are the
first planned consumers.
first planned consumers.
**Package:** `openvibe-publishing` (CommonJS, Node ≥ 20, production runs Node 22).
**Package:** `openvibe-publishing` (CommonJS, Node ≥ 20, production runs Node 22).
**License:** MIT, like OpenVibe.Shared.
**License:** MIT, like OpenVibe.Shared.
@@ -36,7 +37,7 @@ Every module is its own entry point and can be used alone.
@@ -36,7 +37,7 @@ Every module is its own entry point and can be used alone.
| `openvibe-publishing/discussion` | Community comment-thread references: resolves via `POST /api/v1/comments/threads/resolve`, stores the thread id only (no comment content) | `<prefix>_discussion_refs` |
| `openvibe-publishing/discussion` | Community comment-thread references: resolves via `POST /api/v1/comments/threads/resolve`, stores the thread id only (no comment content) | `<prefix>_discussion_refs` |
| `openvibe-publishing/seo` | The deterministic **indexability gate**; canonical URLs; history-aware redirects (old slug → 301, gone → 410); meta/robots tags; sitemaps; RSS, Atom and JSON Feed; JSON-LD built only from provided fields | `<prefix>_redirects` |
| `openvibe-publishing/seo` | The deterministic **indexability gate**; canonical URLs; history-aware redirects (old slug → 301, gone → 410); meta/robots tags; sitemaps; RSS, Atom and JSON Feed; JSON-LD built only from provided fields | `<prefix>_redirects` |
| `openvibe-publishing/authorship` | human / ai / hybrid / imported records with the OpenVibe.AI workflow + run id, disclosure labels, AI content held as draft + noindex until a person's review | `<prefix>_reviews` |
| `openvibe-publishing/authorship` | human / ai / hybrid / imported records with the OpenVibe.AI workflow + run id, disclosure labels, AI content held as draft + noindex until a person's review | `<prefix>_reviews` |
-| `openvibe-publishing/index-hooks` | Search index documents (and tombstones) from a published revision; `<product>.<type>.published\|updated\|unpublished\|deleted` event envelopes for the product's outbox | — |
+| `openvibe-publishing/index-hooks` | `search.index-document@1` documents and tombstones, the `<owner>.index_document.upserted\|deleted` events OpenVibe.Search consumes, a monotonic index-revision sequencer, and the product's own `<product>.<type>.published\|updated\|unpublished\|deleted` events | `<prefix>_index_revisions` |
| `openvibe-publishing/ssr` | auto-escaping `html` tagged templates with `raw()`, a safe Markdown subset, plain-text extraction, word count, server pagination, breadcrumbs, diff markup, honest `<time>` | — |
| `openvibe-publishing/ssr` | auto-escaping `html` tagged templates with `raw()`, a safe Markdown subset, plain-text extraction, word count, server pagination, breadcrumbs, diff markup, honest `<time>` | — |
`require('openvibe-publishing')` exposes all of them lazily (`.revisions`, `.seo`, `.indexHooks`, …).
`require('openvibe-publishing')` exposes all of them lazily (`.revisions`, `.seo`, `.indexHooks`, …).
@@ -46,7 +47,7 @@ Every module is its own entry point and can be used alone.
@@ -46,7 +47,7 @@ Every module is its own entry point and can be used alone.
Pin the release tarball, like every OpenVibe package (never a `file:` link or a vendored copy):
Pin the release tarball, like every OpenVibe package (never a `file:` link or a vendored copy):
```json
```json
-"openvibe-publishing": "https://codeload.github.com/OpenVibers/OpenVibe.Publishing/tar.gz/refs/tags/v0.1.0"
+"openvibe-publishing": "https://codeload.github.com/OpenVibers/OpenVibe.Publishing/tar.gz/refs/tags/v0.2.0"
```
```
`better-sqlite3` (≥ 11) is a peer dependency: the product brings its own handle. The only runtime
`better-sqlite3` (≥ 11) is a peer dependency: the product brings its own handle. The only runtime
@@ -96,13 +97,32 @@ published document) all require the decision, so an object without one cannot re
@@ -96,13 +97,32 @@ published document) all require the decision, so an object without one cannot re
## Events and Search
## Events and Search
-`index-hooks.buildIndexDocument()` produces the Search document ([REF] §12.3 fields plus the gate
-decision) or a tombstone; the proposed schema is
-[docs/contracts-proposal/search.index-document.v1.json](docs/contracts-proposal/search.index-document.v1.json).
-`publicationEvent()` wraps it in an `events.event-envelope@1` envelope with `event_type`
-`<product>.<type>.<action>` and no `event_id` (the OpenVibe.Events outbox assigns one); events about
-private, gated or unlisted content are `internal`. Enqueue it with the Events outbox in the same
-transaction as the product's state change.
+`index-hooks.buildIndexDocument()` produces exactly the released `search.index-document@1`
+(openvibe-contracts v0.12.0, owned by OpenVibe.Search): visibility `public | unlisted | members |
+private`, ACL `subjects | groups | entitlements`, authorship `human | ai_assisted | ai_generated |
+imported`, provenance as typed references (Sources items, the product's citation records, the AI
+run with `stub: true` for stub output), and `indexability { decision: index|noindex, reasons }` with
+the gate's codes mapped onto Search's known reasons (`hooks.SEARCH_REASONS`). Anything not
+published, and unlisted content unless `includeUnlisted`, becomes the tombstone
+`{ owner, type, id, revision, deleted: true }`.
+
+Search orders documents by `revision`, refuses a different document at an equal revision and lets a
+tombstone win ties, so the revision must rise with every indexed change, not only content edits.
+`createIndexSequencer(db, { prefix })` keeps that counter in the product's database.
+
+```js
+const seq = hooks.createIndexSequencer(db, { prefix: 'wiki' });
+const doc = seq.stamp(hooks.buildIndexDocument({ owner: 'wiki', type: 'page', id, revision: 0, state, visibility, … , decision }));
+outbox.enqueue(hooks.indexEvent({ document: doc })); // wiki.index_document.upserted | .deleted
+```
+
+`indexEvent()` builds the envelope Search's webhook consumes (payload = the document, or
+`{ type, id, revision }` for a deletion; subject `{ type, id, revision }`; visibility internal).
+`publicationEvent()` builds the product's own `<product>.<type>.<action>` event (payload: canonical
+URL, publication state, indexability; public only for public, listable content). Neither sets
+`event_id`: the OpenVibe.Events outbox assigns it. Enqueue both in the same transaction as the
+product's state change. Tests validate every document and envelope with openvibe-contracts
+v0.12.0 and mirror Search's webhook checks.
## Exit proof: two products
## Exit proof: two products
@@ -135,4 +155,5 @@ fnm exec --using=22.22.1 npm test # every test/*.test.js, temp database
@@ -135,4 +155,5 @@ fnm exec --using=22.22.1 npm test # every test/*.test.js, temp database
- `openvibe-shared` v1.0.0 (`seo`), at runtime.
- `openvibe-shared` v1.0.0 (`seo`), at runtime.
- `better-sqlite3` ≥ 11, supplied by the consumer.
- `better-sqlite3` ≥ 11, supplied by the consumer.
- Contracts it produces for: `common.entity-ref@1`, `media.media-ref@1`, `events.event-envelope@1`
- Contracts it produces for: `common.entity-ref@1`, `media.media-ref@1`, `events.event-envelope@1`
- (validated in tests against `openvibe-contracts` v0.7.0), and the proposed `search.index-document@1`.
+ and `search.index-document@1` (validated in tests against `openvibe-contracts` v0.12.0, a
+ devDependency: nothing in `lib/` needs it at runtime).
@@ -5,17 +5,36 @@
@@ -5,17 +5,36 @@
"wave": "W15",
"wave": "W15",
"kind": "packages",
"kind": "packages",
"package": "openvibe-publishing",
"package": "openvibe-publishing",
- "version": "0.1.0",
+ "version": "0.2.0",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
"runtime": null,
"runtime": null,
"port": null,
"port": null,
"domain": null,
"domain": null,
"database": null,
"database": null,
- "modules": ["revisions", "schedule", "taxonomy", "citations", "media", "discussion", "seo", "authorship", "index-hooks", "ssr"],
+ "modules": [
+ "revisions",
+ "schedule",
+ "taxonomy",
+ "citations",
+ "media",
+ "discussion",
+ "seo",
+ "authorship",
+ "index-hooks",
+ "ssr"
+ ],
"consumers": [],
"consumers": [],
- "plannedConsumers": ["OpenVibe.Wiki", "OpenVibe.Blog", "OpenVibe.News", "OpenVibe.Reviews", "OpenVibe.Deals", "OpenVibe.Coupons", "OpenVibe.Trade"],
+ "plannedConsumers": [
+ "OpenVibe.Wiki",
+ "OpenVibe.Blog",
+ "OpenVibe.News",
+ "OpenVibe.Reviews",
+ "OpenVibe.Deals",
+ "OpenVibe.Coupons",
+ "OpenVibe.Trade"
+ ],
"exitProof": "examples/two-products + test/two-products.test.js",
"exitProof": "examples/two-products + test/two-products.test.js",
- "contracts": "openvibe-contracts v0.7.0 (tests); proposes search.index-document@1",
+ "contracts": "openvibe-contracts v0.12.0: emits search.index-document@1 and events.event-envelope@1 (validated in tests)",
"released": false,
"released": false,
"updated": "2026-09-22"
"updated": "2026-09-22"
}
}
deleted file mode 100644
deleted file mode 100644
@@ -1,113 +0,0 @@
@@ -1,113 +0,0 @@
-{
- "$schema": "https://json-schema.org/draft/2020-12/schema",
- "$id": "https://openvibe.network/contracts/search/index-document.v1.json",
- "title": "SearchIndexDocument",
- "description": "PROPOSAL (openvibe-publishing 0.1.0, for OpenVibe.Contracts and OpenVibe.Search). The document a publication product sends to Search for one revision of one resource, or the tombstone that removes it. Owner, resource, revision, visibility + ACL, canonical URL, title/summary/body, facets, provenance, publication state, deletion marker ([REF] §12.3), plus the publishing SEO gate's indexability decision. Search must never return a document to a query that fails its ACL, and must not list a document whose indexability.listable is false on public surfaces.",
- "oneOf": [
- {
- "title": "IndexDocument",
- "type": "object",
- "required": ["schema", "owner_service", "resource_type", "resource_id", "revision", "visibility", "acl", "canonical_url", "title", "summary", "body", "facets", "provenance", "publication_state", "deleted", "indexability", "published_at", "updated_at"],
- "additionalProperties": false,
- "properties": {
- "schema": { "const": "search.index-document@1" },
- "owner_service": { "$ref": "#/$defs/service" },
- "resource_type": { "$ref": "#/$defs/segment" },
- "resource_id": { "type": "string", "minLength": 1, "maxLength": 200 },
- "revision": { "type": "integer", "minimum": 0 },
- "visibility": { "enum": ["public", "private", "gated"] },
- "acl": {
- "oneOf": [
- { "type": "object", "required": ["public"], "additionalProperties": false, "properties": { "public": { "const": true } } },
- {
- "type": "object", "required": ["public"], "additionalProperties": false,
- "properties": {
- "public": { "const": false },
- "subjects": { "type": "array", "minItems": 1, "items": { "type": "string", "pattern": "^(usr|gst|app|mod)_[0-9A-HJKMNP-TV-Z]{26}$" } },
- "entitlements": { "type": "array", "minItems": 1, "items": { "type": "string", "minLength": 1 } }
- },
- "anyOf": [{ "required": ["subjects"] }, { "required": ["entitlements"] }]
- }
- ]
- },
- "canonical_url": { "type": "string", "pattern": "^https?://" },
- "title": { "type": "string", "minLength": 1 },
- "summary": { "type": ["string", "null"] },
- "body": { "type": "string", "maxLength": 100000 },
- "body_truncated": { "const": true },
- "locale": { "type": "string" },
- "facets": {
- "type": "object",
- "propertyNames": { "pattern": "^[a-z][a-z0-9_]{0,39}$" },
- "additionalProperties": {
- "anyOf": [
- { "type": ["string", "number", "boolean"] },
- { "type": "array", "items": { "type": ["string", "number", "boolean"] } }
- ]
- }
- },
- "provenance": {
- "type": "object",
- "additionalProperties": false,
- "properties": {
- "authorship": {
- "type": "object", "required": ["mode"], "additionalProperties": false,
- "properties": {
- "mode": { "enum": ["human", "ai", "hybrid", "imported"] },
- "reviewed": { "type": "boolean" },
- "workflow": { "type": "object", "required": ["id", "run_id"], "additionalProperties": false, "properties": { "id": { "type": "string" }, "run_id": { "type": "string" } } }
- }
- },
- "citations": {
- "type": "array",
- "items": {
- "type": "object", "additionalProperties": false,
- "properties": {
- "source_item_id": { "type": "string" },
- "url": { "type": "string" },
- "retrieved_at": { "type": "string", "format": "date-time" },
- "license_note": { "type": "string" }
- },
- "anyOf": [{ "required": ["source_item_id"] }, { "required": ["url"] }]
- }
- },
- "derived_from": { "type": "object" }
- }
- },
- "publication_state": { "const": "published" },
- "deleted": { "const": false },
- "indexability": {
- "type": "object", "required": ["indexable", "listable", "reasons", "gate"], "additionalProperties": false,
- "properties": {
- "indexable": { "type": "boolean" },
- "listable": { "type": "boolean" },
- "reasons": { "type": "array", "items": { "type": "string", "pattern": "^[a-z][a-z_]*$" } },
- "gate": { "type": ["string", "null"] }
- }
- },
- "published_at": { "type": ["string", "null"], "format": "date-time" },
- "updated_at": { "type": ["string", "null"], "format": "date-time" }
- }
- },
- {
- "title": "IndexTombstone",
- "type": "object",
- "required": ["schema", "owner_service", "resource_type", "resource_id", "revision", "publication_state", "deleted", "updated_at"],
- "additionalProperties": false,
- "properties": {
- "schema": { "const": "search.index-document@1" },
- "owner_service": { "$ref": "#/$defs/service" },
- "resource_type": { "$ref": "#/$defs/segment" },
- "resource_id": { "type": "string", "minLength": 1, "maxLength": 200 },
- "revision": { "type": "integer", "minimum": 0 },
- "publication_state": { "enum": ["draft", "scheduled", "published", "unpublished", "deleted"] },
- "deleted": { "const": true },
- "updated_at": { "type": ["string", "null"], "format": "date-time" }
- }
- }
- ],
- "$defs": {
- "service": { "type": "string", "pattern": "^[a-z][a-z0-9-]{1,39}$" },
- "segment": { "type": "string", "pattern": "^[a-z][a-z0-9_]{0,39}$" }
- }
-}
@@ -6,7 +6,7 @@
@@ -6,7 +6,7 @@
* node examples/two-products/wiki/app.js # http://127.0.0.1:4801 with a temp database
* node examples/two-products/wiki/app.js # http://127.0.0.1:4801 with a temp database
*
*
* Uses: revisions (wiki_page_*), citations (wiki_citations), seo gate + redirects + sitemap +
* Uses: revisions (wiki_page_*), citations (wiki_citations), seo gate + redirects + sitemap +
- * Atom/JSON feeds, authorship (AI pages start as drafts), index-hooks (events into wiki_outbox), ssr.
+ * Atom/JSON feeds, authorship (AI pages start as drafts), index-hooks (Search index events + product events into wiki_outbox), ssr.
*/
*/
const http = require('http');
const http = require('http');
const path = require('path');
const path = require('path');
@@ -47,6 +47,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
@@ -47,6 +47,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
const citations = createCitationStore(db, { prefix: 'wiki', now, revisions });
const citations = createCitationStore(db, { prefix: 'wiki', now, revisions });
const redirects = seo.createRedirectStore(db, { prefix: 'wiki_page', now });
const redirects = seo.createRedirectStore(db, { prefix: 'wiki_page', now });
const reviews = authorship.createReviewLog(db, { prefix: 'wiki_page', now });
const reviews = authorship.createReviewLog(db, { prefix: 'wiki_page', now });
+ const sequencer = hooks.createIndexSequencer(db, { prefix: 'wiki', now });
const pathOf = (slug) => `/p/${slug}`;
const pathOf = (slug) => `/p/${slug}`;
const urlOf = (slug) => seo.canonicalUrl(origin, pathOf(slug));
const urlOf = (slug) => seo.canonicalUrl(origin, pathOf(slug));
@@ -64,23 +65,32 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
@@ -64,23 +65,32 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
function document(page, rev, decision) {
function document(page, rev, decision) {
return hooks.buildIndexDocument({
return hooks.buildIndexDocument({
- service: 'wiki', type: 'page', id: page.id, revision: rev ? rev.number : 0, state: page.state, visibility: page.visibility,
+ owner: 'wiki', type: 'page', id: page.id, revision: 0, state: page.state, visibility: page.visibility,
canonicalUrl: urlOf(page.slug), title: rev ? rev.fields.title : page.slug, summary: rev ? ssr.markdownToText(rev.content, 160) : null,
canonicalUrl: urlOf(page.slug), title: rev ? rev.fields.title : page.slug, summary: rev ? ssr.markdownToText(rev.content, 160) : null,
body: rev ? ssr.markdownToText(rev.content) : '', decision, acl: { subjects: [page.owner] },
body: rev ? ssr.markdownToText(rev.content) : '', decision, acl: { subjects: [page.owner] },
- provenance: { authorship: rev && rev.meta.authorship, citations: rev ? citations.forRevision(page.id, rev.number) : [] },
+ authorship: rev && rev.meta.authorship, citations: rev ? citations.forRevision(page.id, rev.number) : [],
+ facets: { content_revision: rev ? rev.number : 0 },
publishedAt: page.published_at, updatedAt: page.updated_at,
publishedAt: page.published_at, updatedAt: page.updated_at,
});
});
}
}
- function emit(before, page, actor) {
+ /**
+ * After any change: send Search the current document (or tombstone) when it differs from the
+ * last one sent, and the product event when the publication state moved. Both go to the
+ * outbox in the caller's transaction.
+ */
+ function sync(before, page, actor) {
const rev = page.published_revision ? revisions.get(page.id, page.published_revision) : null;
const rev = page.published_revision ? revisions.get(page.id, page.published_revision) : null;
+ const decision = rev ? decide(page, rev) : null;
+ const sentBefore = sequencer.current('wiki', 'page', page.id);
+ const doc = sequencer.stamp(document(page, rev, decision));
+ const out = [];
+ if (doc.revision !== sentBefore) out.push(hooks.indexEvent({ document: doc, now: now() }));
const action = hooks.actionFor(before && { state: before.state, visibility: before.visibility, revision: before.published_revision },
const action = hooks.actionFor(before && { state: before.state, visibility: before.visibility, revision: before.published_revision },
{ state: page.state, visibility: page.visibility, revision: page.published_revision });
{ state: page.state, visibility: page.visibility, revision: page.published_revision });
- if (!action) return null;
- const decision = rev ? decide(page, rev) : null;
- const env = hooks.publicationEvent({ product: 'wiki', type: 'page', action, id: page.id, revision: page.published_revision || 0, actor, document: document(page, rev, decision), decision, now: now() });
- db.prepare('INSERT INTO wiki_outbox (envelope) VALUES (?)').run(JSON.stringify(env));
- return env;
+ if (action) out.push(hooks.publicationEvent({ product: 'wiki', type: 'page', action, id: page.id, revision: page.published_revision || 0, actor, document: doc, decision, now: now() }));
+ for (const env of out) db.prepare('INSERT INTO wiki_outbox (envelope) VALUES (?)').run(JSON.stringify(env));
+ return out;
}
}
const api = {
const api = {
@@ -126,7 +136,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
@@ -126,7 +136,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
const ok = authorship.canPublish(rec, reviews.latest(id, revision));
const ok = authorship.canPublish(rec, reviews.latest(id, revision));
if (!ok.ok) throw Object.assign(new Error(`cannot publish: ${ok.reason}`), { status: 409, code: ok.reason });
if (!ok.ok) throw Object.assign(new Error(`cannot publish: ${ok.reason}`), { status: 409, code: ok.reason });
db.prepare("UPDATE wiki_pages SET state = 'published', published_revision = ?, published_at = COALESCE(published_at, ?), updated_at = ? WHERE id = ?").run(revision, now(), now(), id);
db.prepare("UPDATE wiki_pages SET state = 'published', published_revision = ?, published_at = COALESCE(published_at, ?), updated_at = ? WHERE id = ?").run(revision, now(), now(), id);
- return emit(before, getPage(id), actor);
+ return sync(before, getPage(id), actor);
})();
})();
},
},
@@ -134,7 +144,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
@@ -134,7 +144,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
return db.transaction(() => {
return db.transaction(() => {
const before = getPage(id);
const before = getPage(id);
db.prepare('UPDATE wiki_pages SET visibility = ?, updated_at = ? WHERE id = ?').run(visibility, now(), id);
db.prepare('UPDATE wiki_pages SET visibility = ?, updated_at = ? WHERE id = ?').run(visibility, now(), id);
- return emit(before, getPage(id), actor);
+ return sync(before, getPage(id), actor);
})();
})();
},
},
@@ -144,6 +154,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
@@ -144,6 +154,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
const slug = slugify(newTitle);
const slug = slugify(newTitle);
redirects.recordMove(id, pathOf(page.slug), pathOf(slug));
redirects.recordMove(id, pathOf(page.slug), pathOf(slug));
db.prepare('UPDATE wiki_pages SET slug = ?, updated_at = ? WHERE id = ?').run(slug, now(), id);
db.prepare('UPDATE wiki_pages SET slug = ?, updated_at = ? WHERE id = ?').run(slug, now(), id);
+ sync(page, getPage(id), 'svc:wiki'); // the canonical URL changed: Search gets a new revision
return slug;
return slug;
})();
})();
},
},
@@ -152,7 +163,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
@@ -152,7 +163,7 @@ function createWiki({ dbPath, origin = 'https://openvibe.wiki', now = () => Date
return db.transaction(() => {
return db.transaction(() => {
const before = getPage(id);
const before = getPage(id);
db.prepare("UPDATE wiki_pages SET state = 'deleted', updated_at = ? WHERE id = ?").run(now(), id);
db.prepare("UPDATE wiki_pages SET state = 'deleted', updated_at = ? WHERE id = ?").run(now(), id);
- return emit(before, getPage(id), actor);
+ return sync(before, getPage(id), actor);
})();
})();
},
},
@@ -1,40 +1,91 @@
@@ -1,40 +1,91 @@
'use strict';
'use strict';
/**
/**
- * openvibe-publishing/index-hooks — Search index documents and publication event envelopes.
+ * openvibe-publishing/index-hooks — OpenVibe.Search index documents and the events that carry them.
*
*
* const hooks = require('openvibe-publishing/index-hooks');
* const hooks = require('openvibe-publishing/index-hooks');
- * const doc = hooks.buildIndexDocument({ service: 'wiki', type: 'page', id: 'pg_1', revision: 4, state: 'published',
- * visibility: 'public', canonicalUrl, title, summary, body, facets: { space: 'food' }, provenance, decision });
- * const action = hooks.actionFor(before, after); // 'published' | 'updated' | 'unpublished' | 'deleted' | null
- * const env = hooks.publicationEvent({ product: 'wiki', type: 'page', action, id: 'pg_1', revision: 4,
- * actor: 'usr_…', document: doc, decision, now: Date.now() });
- * outbox.enqueue(env); // OpenVibe.Events createOutbox, in the same transaction as the product's change
+ * const doc = hooks.buildIndexDocument({ owner: 'wiki', type: 'page', id: 'pg_1', revision: 4, state: 'published',
+ * visibility: 'public', canonicalUrl, title, summary, body, facets: { space: 'food' },
+ * authorship: rec, citations, decision });
+ * outbox.enqueue(hooks.indexEvent({ document: doc, actor: 'svc:wiki' })); // wiki.index_document.upserted|deleted
+ * const action = hooks.actionFor(before, after); // product domain event, optional
+ * if (action) outbox.enqueue(hooks.publicationEvent({ product: 'wiki', type: 'page', action, id, revision, actor, document: doc, decision }));
*
*
- * The index document is the Search contract of [REF] §12.3 (owner service, resource type/id,
- * revision, visibility + ACL, canonical URL, title, summary, body, facets, provenance, publication
- * state, deletion marker) plus the gate's indexability decision. Proposed schema:
- * docs/contracts-proposal/search.index-document.v1.json.
+ * Documents follow the released contract `search.index-document@1` (openvibe-contracts ≥ 0.12.0,
+ * contracts/search/index-document.v1.json) exactly: { owner, type, id, revision, deleted,
+ * visibility, acl, canonical_url, title, summary, body, facets, language, authorship, provenance,
+ * publication_state, published_at, updated_at, indexability { decision, reasons } }, and a
+ * tombstone is exactly { owner, type, id, revision, deleted: true }.
*
*
- * Safety rules built in:
- * - Anything not published, deleted, or unlisted becomes a tombstone (deletion marker, no
- * title/summary/body), so a visibility change or deletion removes content from the index.
- * - private content must carry an ACL naming subjects; gated content must name entitlements.
- * There is no way to build a private document that reads as world-visible.
- * - Events about non-public or non-listable content are 'internal', never 'public'.
- * - The envelope carries no event_id; the outbox assigns evt_<ULID> (or pass eventId).
+ * Mapping from this package's vocabulary:
+ * visibility public → public · unlisted → unlisted (only with includeUnlisted; else a tombstone)
+ * gated → members (acl entitlements/groups/subjects) · private → private (acl.subjects)
+ * authorship human → human · hybrid → ai_assisted · ai → ai_generated · imported → imported
+ * reasons the gate's codes, renamed to Search's known reasons where one exists (SEARCH_REASONS)
+ * provenance Sources items → { service: 'sources', type: 'item', id }; URL-only citations →
+ * the owner's own citation record { service: owner, type: 'citation', id };
+ * the AI run → { service: 'ai', type: 'run', id, stub? }; plus any refs passed in
+ *
+ * Stricter than the schema requires, on purpose: anything not published (draft, scheduled,
+ * unpublished, retracted, archived, deleted) and unlisted content (unless includeUnlisted) is sent
+ * as a tombstone, so a state or visibility change always removes the old copy from Search. A
+ * private document must name its subjects and a members document its audience — there is no way to
+ * build a non-public document that reads as world-visible.
+ *
+ * Revisions: Search orders by the document's `revision`; at an equal revision a different
+ * document is a conflict (first write wins) and a tombstone wins, so the number must grow with
+ * EVERY change to what is indexed (content, visibility, state, canonical URL, gate decision), not
+ * only with content revisions. createIndexSequencer(db, { prefix }) keeps that counter in the
+ * product's database: same document → same revision (a safe replay), anything different → +1.
+ *
+ * Events: Search consumes `<owner>.index_document.upserted` (payload = the document) and
+ * `<owner>.index_document.deleted` (payload = { type, id, revision }), subject { type, id, revision },
+ * visibility internal (OpenVibe.Search server/api/webhook.js). Envelopes carry no event_id; the
+ * OpenVibe.Events outbox assigns evt_<ULID>. Owners must be slugs without dashes because event
+ * types are dot-separated [a-z0-9_] segments.
*/
*/
-const SERVICE_RE = /^[a-z][a-z0-9-]{1,39}$/;
-const SEGMENT_RE = /^[a-z][a-z0-9_]{0,39}$/;
-const ULID = '[0-9A-HJKMNP-TV-Z]{26}';
-const SUBJECT_ID_RE = new RegExp(`^(usr|gst|app|mod)_${ULID}$`);
-const ACTIONS = ['published', 'updated', 'unpublished', 'deleted'];
+const OWNER_RE = /^[a-z][a-z0-9_]{1,39}$/; // service slug that is also a valid event_type segment
+const TYPE_RE = /^[a-z][a-z0-9_]{1,39}$/;
+const ID_RE = /^[A-Za-z0-9][A-Za-z0-9._:~-]{0,127}$/;
+const PERSON_RE = /^(usr|gst)_[0-9A-HJKMNP-TV-Z]{26}$/;
+const KEY_RE = /^[a-z][a-z0-9_.:-]{0,127}$/;
+const FACET_RE = /^[a-z][a-z0-9_]{0,39}$/;
+const LANG_RE = /^[a-zA-Z]{2,3}(-[A-Za-z0-9]{2,8})*$/;
+const REF_SERVICE_RE = /^[a-z][a-z0-9-]{1,39}$/;
+
const SCHEMA = 'search.index-document@1';
const SCHEMA = 'search.index-document@1';
-const MAX_BODY = 100000;
+const ACTIONS = ['published', 'updated', 'unpublished', 'deleted'];
+const LIMITS = Object.freeze({ title: 500, summary: 4000, body: 48000, facets: 20, facetArray: 50, facetString: 200, provenance: 50, reasons: 20, acl: { subjects: 200, groups: 100, entitlements: 100 } });
-function assertDecision(d) {
- if (!d || typeof d.indexable !== 'boolean' || typeof d.listable !== 'boolean') throw new TypeError('decision (seo.evaluate) is required');
- return { indexable: d.indexable, listable: d.listable, reasons: Array.isArray(d.codes) ? [...d.codes] : [], gate: d.gate || null };
-}
+const AUTHORSHIP = Object.freeze({ human: 'human', hybrid: 'ai_assisted', ai: 'ai_generated', imported: 'imported' });
+
+/** Gate reason code → Search's known reason (roadmap §32.3 names as listed in the contract). */
+const SEARCH_REASONS = Object.freeze({
+ deleted: 'deleted',
+ private: 'private',
+ gated: 'members_only',
+ unlisted: 'unlisted',
+ draft: 'draft',
+ unpublished: 'not_published',
+ ai_generated_unreviewed: 'ai_unreviewed',
+ stub_provider: 'stub_provider',
+ unreviewed_sensitive: 'sensitive_unreviewed',
+ thin: 'thin_content',
+ unsourced: 'unsourced',
+ unsupported_claims: 'unsupported_claims',
+ missing_canonical: 'missing_canonical_url',
+ noindex_requested: 'owner_decision',
+ // no Search equivalent; sent under the gate's own stable code
+ takedown: 'takedown',
+ retracted: 'retracted',
+ expired: 'expired',
+ stale_price: 'stale_price',
+ duplicate_of: 'duplicate_of',
+});
+
+const clip = (s, max) => {
+ const t = String(s == null ? '' : s);
+ return t.length > max ? t.slice(0, max) : t;
+};
function iso(v, name) {
function iso(v, name) {
if (v == null || v === '') return null;
if (v == null || v === '') return null;
@@ -43,109 +94,190 @@ function iso(v, name) {
@@ -43,109 +94,190 @@ function iso(v, name) {
return d.toISOString();
return d.toISOString();
}
}
-function checkFacets(facets) {
- const out = {};
- for (const [k, v] of Object.entries(facets || {})) {
- if (!SEGMENT_RE.test(k)) throw new TypeError(`facet name "${k}" must match ${SEGMENT_RE}`);
- if (v == null) continue;
- const ok = (x) => typeof x === 'string' || typeof x === 'boolean' || (typeof x === 'number' && Number.isFinite(x));
- if (Array.isArray(v) ? !v.every(ok) : !ok(v)) throw new TypeError(`facet "${k}" must be a string, number, boolean or an array of them`);
- out[k] = Array.isArray(v) ? [...v] : v;
- }
+function checkIdentity({ owner, type, id, revision }) {
+ if (!OWNER_RE.test(String(owner || ''))) throw new TypeError('owner must be a service slug of [a-z0-9_] (it prefixes event types)');
+ if (!TYPE_RE.test(String(type || ''))) throw new TypeError('type must match ^[a-z][a-z0-9_]{1,39}$');
+ if (typeof id !== 'string' || !ID_RE.test(id)) throw new TypeError('id must match ^[A-Za-z0-9][A-Za-z0-9._:~-]{0,127}$');
+ if (!Number.isInteger(revision) || revision < 0) throw new TypeError('revision must be a non-negative integer');
+}
+
+/** The gate decision as Search's indexability { decision, reasons }. */
+function searchIndexability(decision) {
+ if (!decision || typeof decision.indexable !== 'boolean' || typeof decision.listable !== 'boolean') throw new TypeError('decision (seo.evaluate) is required');
+ const codes = Array.isArray(decision.codes) ? decision.codes : [];
+ const reasons = [...new Set(codes.map((c) => SEARCH_REASONS[c] || c))].slice(0, LIMITS.reasons);
+ return { decision: decision.indexable ? 'index' : 'noindex', reasons };
+}
+
+function uniqueKeys(list, re, max, what) {
+ const out = [...new Set((list || []).map(String))];
+ if (out.length > max) throw new TypeError(`acl.${what} holds at most ${max} entries`);
+ for (const k of out) if (!re.test(k)) throw new TypeError(`acl.${what} entry "${k}" is not valid`);
return out;
return out;
}
}
-function checkProvenance(p = {}) {
+function checkFacets(facets) {
+ const entries = Object.entries(facets || {}).filter(([, v]) => v != null);
+ if (entries.length > LIMITS.facets) throw new TypeError(`at most ${LIMITS.facets} facets`);
const out = {};
const out = {};
- if (p.authorship) {
- out.authorship = { mode: p.authorship.mode };
- if (p.authorship.workflow) out.authorship.workflow = { id: p.authorship.workflow.id, run_id: p.authorship.workflow.runId };
- if (p.authorship.reviewed != null) out.authorship.reviewed = Boolean(p.authorship.reviewed);
- }
- if (Array.isArray(p.citations)) {
- out.citations = p.citations.map((c) => {
- const o = {};
- if (c.sourceItemId) o.source_item_id = String(c.sourceItemId);
- if (c.url) o.url = String(c.url);
- if (c.retrievedAt) o.retrieved_at = iso(c.retrievedAt, 'citation.retrievedAt');
- if (c.licenseNote) o.license_note = String(c.licenseNote);
- return o;
- }).filter((o) => o.source_item_id || o.url);
+ for (const [k, v] of entries) {
+ if (!FACET_RE.test(k)) throw new TypeError(`facet name "${k}" must match ${FACET_RE}`);
+ if (Array.isArray(v)) {
+ if (v.length > LIMITS.facetArray) throw new TypeError(`facet "${k}" holds at most ${LIMITS.facetArray} values`);
+ if (!v.every((x) => typeof x === 'string')) throw new TypeError(`facet "${k}": arrays must hold strings`);
+ out[k] = v.map((x) => clip(x, LIMITS.facetString));
+ } else if (typeof v === 'string') out[k] = clip(v, LIMITS.facetString);
+ else if (typeof v === 'boolean' || (typeof v === 'number' && Number.isFinite(v))) out[k] = v;
+ else throw new TypeError(`facet "${k}" must be a string, number, boolean or an array of strings`);
}
}
- if (p.derivedFrom) out.derived_from = p.derivedFrom;
return out;
return out;
}
}
-function checkIdentity({ service, type, id, revision }) {
- if (!SERVICE_RE.test(String(service || ''))) throw new TypeError('service must be a service slug');
- if (!SEGMENT_RE.test(String(type || ''))) throw new TypeError('type must match ^[a-z][a-z0-9_]{0,39}$');
- if (typeof id !== 'string' || !id || id.length > 200) throw new TypeError('id must be a non-empty string');
- if (!Number.isInteger(revision) || revision < 0) throw new TypeError('revision must be a non-negative integer');
+function httpUrl(u) {
+ if (!u) return null;
+ try { const x = new URL(String(u)); return x.protocol === 'http:' || x.protocol === 'https:' ? x.toString() : null; } catch { return null; }
}
}
-/** A deletion marker: Search removes (resource, revision ≤ this) and caches purge it. */
-function tombstone({ service, type, id, revision, state = 'deleted', updatedAt = null }) {
- checkIdentity({ service, type, id, revision });
- return {
- schema: SCHEMA, owner_service: service, resource_type: type, resource_id: id, revision,
- publication_state: state, deleted: true, updated_at: iso(updatedAt, 'updatedAt'),
+/**
+ * Provenance references from citations (citations store rows), the authorship record and extra refs
+ * ([{ service, type, id, revision?, label?, url?, retrievedAt?, stub? }]).
+ */
+function buildProvenance(owner, { citations = [], authorship = null, refs = [] } = {}) {
+ const out = [];
+ const push = (r) => {
+ const o = { service: r.service, type: r.type, id: clip(r.id, 128) };
+ if (!REF_SERVICE_RE.test(o.service) || !TYPE_RE.test(o.type) || !o.id) throw new TypeError(`invalid provenance reference ${JSON.stringify(r)}`);
+ if (Number.isInteger(r.revision) && r.revision >= 0) o.revision = r.revision;
+ if (r.label) o.label = clip(r.label, 200);
+ const url = httpUrl(r.url);
+ if (url) o.url = url;
+ const at = iso(r.retrievedAt, 'retrievedAt');
+ if (at) o.retrieved_at = at;
+ if (r.stub === true) o.stub = true;
+ out.push(o);
};
};
+ if (authorship && authorship.workflow) push({ service: 'ai', type: 'run', id: authorship.workflow.runId, label: authorship.workflow.id, stub: Boolean(authorship.stubProvider) });
+ for (const c of citations) {
+ if (!c) continue;
+ if (c.sourceItemId) push({ service: 'sources', type: 'item', id: c.sourceItemId, label: c.title, url: c.url, retrievedAt: c.retrievedAt });
+ else if (c.url && c.id != null) push({ service: owner.replace(/_/g, '-'), type: 'citation', id: String(c.id), label: c.title, url: c.url, retrievedAt: c.retrievedAt });
+ }
+ for (const r of refs) push(r);
+ if (out.length > LIMITS.provenance) throw new TypeError(`at most ${LIMITS.provenance} provenance references`);
+ return out;
+}
+
+/** The deletion marker: removes (owner, type, id) at this revision or older from every result. */
+function tombstone({ owner, type, id, revision }) {
+ checkIdentity({ owner, type, id, revision });
+ return { owner, type, id, revision, deleted: true };
}
}
/**
/**
- * Build the Search document for a revision. Non-published, deleted and unlisted resources come
- * back as tombstones.
+ * The Search document for one revision, or a tombstone when the resource must not be in Search.
+ * state: draft | scheduled | published | unpublished | retracted | archived | deleted
+ * visibility: public | unlisted | gated | private
*/
*/
function buildIndexDocument({
function buildIndexDocument({
- service, type, id, revision, state, visibility, acl = {}, canonicalUrl, title, summary = null, body = '',
- facets = {}, provenance = {}, decision, publishedAt = null, updatedAt = null, locale = null, deleted = false,
+ owner, type, id, revision, state, visibility, acl = {}, canonicalUrl, title, summary = null, body = '',
+ facets = {}, authorship = null, citations = [], provenance = [], decision, publishedAt = null, updatedAt = null,
+ language = null, deleted = false, includeUnlisted = false,
} = {}) {
} = {}) {
- checkIdentity({ service, type, id, revision });
- if (deleted || state !== 'published' || visibility === 'unlisted') {
- return tombstone({ service, type, id, revision, state: deleted ? 'deleted' : state || 'deleted', updatedAt });
- }
- const indexability = assertDecision(decision);
- let docAcl;
- if (visibility === 'public') docAcl = { public: true };
- else if (visibility === 'private') {
- const subjects = (acl.subjects || []).map(String);
- if (!subjects.length || !subjects.every((s) => SUBJECT_ID_RE.test(s))) throw new TypeError('private documents need acl.subjects (usr_/gst_/app_/mod_ ids)');
- docAcl = { public: false, subjects };
- } else if (visibility === 'gated') {
- const entitlements = (acl.entitlements || []).map(String).filter(Boolean);
- if (!entitlements.length) throw new TypeError('gated documents need acl.entitlements');
- docAcl = { public: false, entitlements, ...(acl.subjects && acl.subjects.length ? { subjects: acl.subjects.map(String) } : {}) };
+ checkIdentity({ owner, type, id, revision });
+ if (deleted || state !== 'published' || (visibility === 'unlisted' && !includeUnlisted)) return tombstone({ owner, type, id, revision });
+
+ const doc = { owner, type, id, revision, deleted: false };
+ if (visibility === 'public') {
+ doc.visibility = 'public';
+ } else if (visibility === 'unlisted') {
+ doc.visibility = 'unlisted';
+ const subjects = uniqueKeys(acl.subjects, PERSON_RE, LIMITS.acl.subjects, 'subjects');
+ if (subjects.length) doc.acl = { subjects };
+ } else if (visibility === 'private') {
+ const subjects = uniqueKeys(acl.subjects, PERSON_RE, LIMITS.acl.subjects, 'subjects');
+ if (!subjects.length) throw new TypeError('private documents need acl.subjects (usr_/gst_ ids)');
+ doc.visibility = 'private';
+ doc.acl = { subjects };
+ } else if (visibility === 'gated' || visibility === 'members') {
+ const a = {
+ subjects: uniqueKeys(acl.subjects, PERSON_RE, LIMITS.acl.subjects, 'subjects'),
+ groups: uniqueKeys(acl.groups, KEY_RE, LIMITS.acl.groups, 'groups'),
+ entitlements: uniqueKeys(acl.entitlements, KEY_RE, LIMITS.acl.entitlements, 'entitlements'),
+ };
+ for (const k of Object.keys(a)) if (!a[k].length) delete a[k];
+ if (!Object.keys(a).length) throw new TypeError('gated (members) documents need acl.entitlements, acl.groups or acl.subjects');
+ doc.visibility = 'members';
+ doc.acl = a;
} else {
} else {
- throw new TypeError('visibility must be public, private, gated or unlisted');
+ throw new TypeError('visibility must be public, unlisted, gated or private');
}
}
- if (!canonicalUrl || !/^https?:\/\//.test(canonicalUrl)) throw new TypeError('canonicalUrl must be absolute');
+
+ const url = httpUrl(canonicalUrl);
+ if (!url) throw new TypeError('canonicalUrl must be an absolute http(s) URL');
if (!title) throw new TypeError('title is required');
if (!title) throw new TypeError('title is required');
- const text = String(body == null ? '' : body);
- const doc = {
- schema: SCHEMA, owner_service: service, resource_type: type, resource_id: id, revision,
- visibility, acl: docAcl, canonical_url: canonicalUrl, title: String(title),
- summary: summary == null ? null : String(summary), body: text.slice(0, MAX_BODY),
- facets: checkFacets(facets), provenance: checkProvenance(provenance),
- publication_state: 'published', deleted: false, indexability,
- published_at: iso(publishedAt, 'publishedAt'), updated_at: iso(updatedAt, 'updatedAt'),
- };
- if (text.length > MAX_BODY) doc.body_truncated = true;
- if (locale) doc.locale = String(locale);
+ doc.canonical_url = url;
+ doc.title = clip(title, LIMITS.title);
+ if (summary != null && summary !== '') doc.summary = clip(summary, LIMITS.summary);
+ doc.body = clip(body, LIMITS.body);
+ doc.facets = checkFacets(facets);
+ if (language) {
+ if (!LANG_RE.test(language)) throw new TypeError('language must be a BCP 47 tag');
+ doc.language = language;
+ }
+ if (authorship) {
+ if (!AUTHORSHIP[authorship.mode]) throw new TypeError('authorship.mode must be human, ai, hybrid or imported');
+ doc.authorship = AUTHORSHIP[authorship.mode];
+ }
+ doc.provenance = buildProvenance(owner, { citations, authorship, refs: provenance });
+ doc.publication_state = 'published';
+ doc.published_at = iso(publishedAt, 'publishedAt');
+ doc.updated_at = iso(updatedAt, 'updatedAt');
+ doc.indexability = searchIndexability(decision);
return doc;
return doc;
}
}
/**
/**
- * Which publication event a transition produces. before/after: { state, visibility, revision }
- * (before may be null for a new resource). → 'published' | 'updated' | 'unpublished' | 'deleted' | null
+ * Monotonic index revisions per resource, in <prefix>_index_revisions of the product's database.
+ * const seq = createIndexSequencer(db, { prefix: 'wiki' });
+ * const doc = seq.stamp(hooks.buildIndexDocument({ ..., revision: 0 })); // revision replaced
*/
*/
-function actionFor(before, after) {
- const was = Boolean(before && before.state === 'published');
- if (after.state === 'deleted') return before && before.state === 'deleted' ? null : (before ? 'deleted' : null);
- const is = after.state === 'published';
- if (!was && is) return 'published';
- if (was && !is) return 'unpublished';
- if (was && is && (before.revision !== after.revision || before.visibility !== after.visibility)) return 'updated';
- return null;
+function createIndexSequencer(db, { prefix, now } = {}) {
+ const { assertDb, assertPrefix, clockOf, sha256, stableStringify } = require('./internal');
+ assertDb(db);
+ assertPrefix(prefix);
+ const clock = clockOf(now);
+ const T = `${prefix}_index_revisions`;
+ db.exec(`
+ CREATE TABLE IF NOT EXISTS ${T} (
+ owner TEXT NOT NULL,
+ type TEXT NOT NULL,
+ id TEXT NOT NULL,
+ revision INTEGER NOT NULL,
+ hash TEXT NOT NULL,
+ updated_at INTEGER NOT NULL,
+ PRIMARY KEY (owner, type, id)
+ );
+ `);
+ const get = db.prepare(`SELECT revision, hash FROM ${T} WHERE owner = ? AND type = ? AND id = ?`);
+ const put = db.prepare(`INSERT INTO ${T} (owner, type, id, revision, hash, updated_at) VALUES (@owner, @type, @id, @revision, @hash, @now)
+ ON CONFLICT (owner, type, id) DO UPDATE SET revision = excluded.revision, hash = excluded.hash, updated_at = excluded.updated_at`);
+ return {
+ table: T,
+ /** The document with its revision set: unchanged → the last revision, changed → last + 1. */
+ stamp(document) {
+ checkIdentity(document);
+ const { revision: _ignored, ...rest } = document;
+ const hash = sha256(stableStringify(rest));
+ return db.transaction(() => {
+ const cur = get.get(document.owner, document.type, document.id);
+ if (cur && cur.hash === hash) return { ...document, revision: cur.revision };
+ const revision = Math.max(cur ? cur.revision + 1 : 1, document.revision);
+ put.run({ owner: document.owner, type: document.type, id: document.id, revision, hash, now: clock() });
+ return { ...document, revision };
+ })();
+ },
+ current(owner, type, id) { const r = get.get(owner, type, id); return r ? r.revision : null; },
+ };
}
}
/** 'usr_…' | 'svc:wiki' | 'service:wiki' | { type, id } → SubjectRef. */
/** 'usr_…' | 'svc:wiki' | 'service:wiki' | { type, id } → SubjectRef. */
@@ -160,38 +292,88 @@ function subjectRef(actor) {
@@ -160,38 +292,88 @@ function subjectRef(actor) {
throw new TypeError('actor must be a subject (usr_…, gst_…, svc:<service>) or a SubjectRef');
throw new TypeError('actor must be a subject (usr_…, gst_…, svc:<service>) or a SubjectRef');
}
}
+function envelope({ source, eventType, actor, now, subject, payload, visibility, traceId, eventId }) {
+ const env = {
+ event_type: eventType,
+ version: 1,
+ source,
+ actor: subjectRef(actor),
+ timestamp: new Date(now).toISOString(),
+ visibility,
+ subject,
+ payload,
+ };
+ if (eventId) env.event_id = eventId;
+ if (traceId && /^[0-9a-f]{32}$/.test(traceId)) env.trace_id = traceId;
+ return env;
+}
+
+/**
+ * The event OpenVibe.Search consumes for a document or tombstone:
+ * `<owner>.index_document.upserted` (payload = document) or `.deleted` (payload = { type, id, revision }).
+ * actor defaults to the owner service itself.
+ */
+function indexEvent({ document, actor = null, now = Date.now(), traceId = null, eventId = null } = {}) {
+ if (!document || typeof document !== 'object') throw new TypeError('document (buildIndexDocument/tombstone) is required');
+ checkIdentity(document);
+ const { owner, type, id, revision } = document;
+ const deleted = document.deleted === true;
+ return envelope({
+ source: owner,
+ eventType: `${owner}.index_document.${deleted ? 'deleted' : 'upserted'}`,
+ actor: actor || { type: 'service', id: owner.replace(/_/g, '-') },
+ now, traceId, eventId,
+ subject: { type, id, revision },
+ visibility: 'internal',
+ payload: deleted ? { type, id, revision } : document,
+ });
+}
+
/**
/**
- * `<product>.<type>.<action>` envelope (contracts events/event-envelope@1) for the product's outbox.
- * document: the index document (or tombstone) for this revision — it becomes the payload.
+ * Which product event a transition produces. before/after: { state, visibility, revision }
+ * (before may be null for a new resource). → 'published' | 'updated' | 'unpublished' | 'deleted' | null
+ */
+function actionFor(before, after) {
+ const was = Boolean(before && before.state === 'published');
+ if (after.state === 'deleted') return before && before.state === 'deleted' ? null : (before ? 'deleted' : null);
+ const is = after.state === 'published';
+ if (!was && is) return 'published';
+ if (was && !is) return 'unpublished';
+ if (was && is && (before.revision !== after.revision || before.visibility !== after.visibility)) return 'updated';
+ return null;
+}
+
+/**
+ * The product's own domain event `<product>.<type>.<action>` (e.g. wiki.page.published), for
+ * subscribers other than Search. Payload: canonical URL, publication state, Search-shaped
+ * indexability — not the body (Search gets that through indexEvent). Public only when the
+ * resource is public and the gate lets it be listed; otherwise internal.
*/
*/
function publicationEvent({ product, type, action, id, revision, actor, document, decision = null, now = Date.now(), traceId = null, eventId = null, extra = {} } = {}) {
function publicationEvent({ product, type, action, id, revision, actor, document, decision = null, now = Date.now(), traceId = null, eventId = null, extra = {} } = {}) {
- if (!SERVICE_RE.test(String(product || '')) || !SEGMENT_RE.test(String(product))) throw new TypeError('product must be a lowercase service slug without dashes, e.g. "wiki"');
- if (!SEGMENT_RE.test(String(type || ''))) throw new TypeError('type must match ^[a-z][a-z0-9_]{0,39}$');
+ if (!OWNER_RE.test(String(product || ''))) throw new TypeError('product must be a service slug of [a-z0-9_], e.g. "wiki"');
+ if (!TYPE_RE.test(String(type || ''))) throw new TypeError('type must match ^[a-z][a-z0-9_]{1,39}$');
if (!ACTIONS.includes(action)) throw new TypeError(`action must be one of ${ACTIONS.join(', ')}`);
if (!ACTIONS.includes(action)) throw new TypeError(`action must be one of ${ACTIONS.join(', ')}`);
- if (!document || document.resource_id !== id) throw new TypeError('document (buildIndexDocument/tombstone) for this resource is required');
- if ((action === 'deleted' || action === 'unpublished') && !document.deleted) throw new TypeError(`${action} events carry a tombstone`);
- // A published-but-unlisted resource is a tombstone with publication_state 'published': it may be updated, not listed.
- if ((action === 'published' || action === 'updated') && document.publication_state !== 'published') throw new TypeError(`${action} events carry a published document`);
- const isPublic = !document.deleted && document.visibility === 'public' && document.indexability && document.indexability.listable;
- const env = {
- event_type: `${product}.${type}.${action}`,
- version: 1,
+ if (!document || document.id !== id) throw new TypeError('document (buildIndexDocument/tombstone) for this resource is required');
+ const removal = action === 'deleted' || action === 'unpublished';
+ if (action === 'published' && document.deleted) throw new TypeError('published events carry a published document');
+ if (removal && !document.deleted) throw new TypeError(`${action} events carry a tombstone`);
+ const isPublic = !document.deleted && document.visibility === 'public' && Boolean(decision && decision.listable);
+ return envelope({
source: product,
source: product,
- actor: subjectRef(actor),
- timestamp: new Date(now).toISOString(),
- visibility: isPublic ? 'public' : 'internal',
+ eventType: `${product}.${type}.${action}`,
+ actor, now, traceId, eventId,
subject: { type, id, revision },
subject: { type, id, revision },
+ visibility: isPublic ? 'public' : 'internal',
payload: {
payload: {
canonical_url: document.canonical_url || null,
canonical_url: document.canonical_url || null,
- publication_state: document.publication_state,
- indexability: decision ? assertDecision(decision) : document.indexability || null,
- document,
+ publication_state: document.deleted ? (action === 'deleted' ? 'deleted' : 'unpublished') : document.publication_state,
+ indexability: document.indexability || (decision ? searchIndexability(decision) : null),
...extra,
...extra,
},
},
- };
- if (eventId) env.event_id = eventId;
- if (traceId && /^[0-9a-f]{32}$/.test(traceId)) env.trace_id = traceId;
- return env;
+ });
}
}
-module.exports = { SCHEMA, ACTIONS, buildIndexDocument, tombstone, actionFor, publicationEvent, subjectRef };
+module.exports = {
+ SCHEMA, ACTIONS, AUTHORSHIP, SEARCH_REASONS, LIMITS,
+ buildIndexDocument, tombstone, searchIndexability, createIndexSequencer, buildProvenance, indexEvent, actionFor, publicationEvent, subjectRef,
+};
@@ -1,12 +1,12 @@
@@ -1,12 +1,12 @@
{
{
"name": "openvibe-publishing",
"name": "openvibe-publishing",
- "version": "0.1.0",
+ "version": "0.2.0",
"lockfileVersion": 3,
"lockfileVersion": 3,
"requires": true,
"requires": true,
"packages": {
"packages": {
"": {
"": {
"name": "openvibe-publishing",
"name": "openvibe-publishing",
- "version": "0.1.0",
+ "version": "0.2.0",
"license": "MIT",
"license": "MIT",
"dependencies": {
"dependencies": {
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.0.0"
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.0.0"
@@ -15,7 +15,7 @@
@@ -15,7 +15,7 @@
"ajv": "^8.17.1",
"ajv": "^8.17.1",
"ajv-formats": "^3.0.1",
"ajv-formats": "^3.0.1",
"better-sqlite3": "^11.10.0",
"better-sqlite3": "^11.10.0",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.7.0"
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.12.0"
},
},
"engines": {
"engines": {
"node": ">=20"
"node": ">=20"
@@ -461,9 +461,9 @@
@@ -461,9 +461,9 @@
}
}
},
},
"node_modules/openvibe-contracts": {
"node_modules/openvibe-contracts": {
- "version": "0.7.0",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.7.0",
- "integrity": "sha512-gUrjxbTy/HX0QMQGoDPphmFdwm7iitmNMMchs9tDSDGR47FRFHL/RgBgIKYR1S7M6ICu261YycFV8+YVglafXA==",
+ "version": "0.12.0",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.12.0",
+ "integrity": "sha512-Cf7rxAk0YXCg7G+HcFldH65+URPVeHY2hoOsqpJ/+KmAFyaK8lX6D1OrdyozH52t+rfsQcRlRqkO5ldEVdZRHA==",
"dev": true,
"dev": true,
"license": "AGPL-3.0-only",
"license": "AGPL-3.0-only",
"dependencies": {
"dependencies": {
@@ -1,6 +1,6 @@
@@ -1,6 +1,6 @@
{
{
"name": "openvibe-publishing",
"name": "openvibe-publishing",
- "version": "0.1.0",
+ "version": "0.2.0",
"description": "Shared publishing packages for the OpenVibe publication products: revisions, scheduling, taxonomy, citations, media attachments, discussion references, the SEO indexability gate, authorship, index hooks and SSR helpers. Packages, not an authority.",
"description": "Shared publishing packages for the OpenVibe publication products: revisions, scheduling, taxonomy, citations, media attachments, discussion references, the SEO indexability gate, authorship, index hooks and SSR helpers. Packages, not an authority.",
"main": "index.js",
"main": "index.js",
"exports": {
"exports": {
@@ -20,7 +20,6 @@
@@ -20,7 +20,6 @@
"files": [
"files": [
"index.js",
"index.js",
"lib/",
"lib/",
- "docs/",
"CHANGELOG.md"
"CHANGELOG.md"
],
],
"scripts": {
"scripts": {
@@ -50,6 +49,6 @@
@@ -50,6 +49,6 @@
"ajv": "^8.17.1",
"ajv": "^8.17.1",
"ajv-formats": "^3.0.1",
"ajv-formats": "^3.0.1",
"better-sqlite3": "^11.10.0",
"better-sqlite3": "^11.10.0",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.7.0"
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.12.0"
}
}
}
}
@@ -1,67 +1,164 @@
@@ -1,67 +1,164 @@
'use strict';
'use strict';
const assert = require('assert');
const assert = require('assert');
-const fs = require('fs');
-const path = require('path');
-const Ajv = require('ajv/dist/2020');
-const addFormats = require('ajv-formats');
const contracts = require('openvibe-contracts');
const contracts = require('openvibe-contracts');
const hooks = require('../lib/index-hooks');
const hooks = require('../lib/index-hooks');
const seo = require('../lib/seo');
const seo = require('../lib/seo');
-const { suite } = require('./helpers/db');
+const authorship = require('../lib/authorship');
+const { openDb, fakeClock, suite } = require('./helpers/db');
const { test, run } = suite();
const { test, run } = suite();
const USER = 'usr_01J8Z6Q3KX0000000000000000';
const USER = 'usr_01J8Z6Q3KX0000000000000000';
-const ajv = new Ajv({ allErrors: true, strict: false });
-addFormats(ajv);
-const validateDoc = ajv.compile(JSON.parse(fs.readFileSync(path.join(__dirname, '..', 'docs', 'contracts-proposal', 'search.index-document.v1.json'), 'utf8')));
+const RUN = 'run_01J8Z6Q3KX0000000000000000';
+
+const valid = (doc) => {
+ const v = contracts.validate('search.index-document@1', doc);
+ assert.ok(v.valid, `search.index-document@1: ${JSON.stringify(v.errors)} in ${JSON.stringify(doc)}`);
+ return doc;
+};
+const validEnvelope = (env) => {
+ const v = contracts.validate('events.event-envelope@1', { ...env, event_id: contracts.ids.newId('event') });
+ assert.ok(v.valid, `events.event-envelope@1: ${JSON.stringify(v.errors)}`);
+ return env;
+};
+
+/** What OpenVibe.Search's webhook (server/api/webhook.js documentFromEvent) does with an envelope. */
+function searchReads(env) {
+ const m = /^([a-z][a-z0-9-]{1,39})\.index_document\.(upserted|deleted)$/.exec(env.event_type);
+ assert.ok(m, `not an index_document event: ${env.event_type}`);
+ assert.strictEqual(m[1], env.source, 'event_type owner must be the source');
+ const p = env.payload;
+ if (p.owner !== undefined) assert.strictEqual(p.owner, env.source);
+ const doc = m[2] === 'deleted' ? { owner: env.source, type: p.type, id: p.id, revision: p.revision, deleted: true } : { ...p, owner: env.source };
+ assert.strictEqual(env.subject.type, doc.type);
+ assert.strictEqual(env.subject.id, doc.id);
+ assert.strictEqual(env.subject.revision, doc.revision);
+ assert.strictEqual(env.visibility, 'internal');
+ return valid(doc);
+}
const facts = { state: 'published', visibility: 'public', canonicalUrl: 'https://openvibe.wiki/p/rye', wordCount: 400, citationCount: 2 };
const facts = { state: 'published', visibility: 'public', canonicalUrl: 'https://openvibe.wiki/p/rye', wordCount: 400, citationCount: 2 };
-const decision = seo.evaluate(facts, { policy: { requireSources: true } });
+const decisionFor = (extra = {}) => seo.evaluate({ ...facts, ...extra }, { policy: { requireSources: true } });
const input = (extra = {}) => ({
const input = (extra = {}) => ({
- service: 'wiki', type: 'page', id: 'pg_1', revision: 4, state: 'published', visibility: 'public',
+ owner: 'wiki', type: 'page', id: 'pg_1', revision: 4, state: 'published', visibility: 'public',
canonicalUrl: 'https://openvibe.wiki/p/rye', title: 'Rye', summary: 'A cereal grass.', body: 'Rye is a grass grown as a grain.',
canonicalUrl: 'https://openvibe.wiki/p/rye', title: 'Rye', summary: 'A cereal grass.', body: 'Rye is a grass grown as a grain.',
- facets: { space: 'food', tags: ['grain', 'bread'] },
- provenance: { authorship: { mode: 'human' }, citations: [{ sourceItemId: 'src_1', url: 'https://example.org/rye', retrievedAt: '2026-09-20T10:00:00Z' }] },
- decision, publishedAt: '2026-09-20T12:00:00Z', updatedAt: '2026-09-21T12:00:00Z', ...extra,
+ facets: { space: 'food', tags: ['grain', 'bread'], words: 400 },
+ authorship: authorship.record({ mode: 'human', authors: [USER] }),
+ citations: [
+ { id: 7, sourceItemId: 'src_1', url: 'https://example.org/rye', title: 'Rye', retrievedAt: '2026-09-20T10:00:00Z' },
+ { id: 8, url: 'https://example.org/other' },
+ ],
+ decision: decisionFor(), publishedAt: '2026-09-20T12:00:00Z', updatedAt: '2026-09-21T12:00:00Z', language: 'en', ...extra,
});
});
-function withEventId(env) { return { ...env, event_id: contracts.ids.newId('event') }; }
-
-test('a published revision becomes a full index document with every §12.3 field', () => {
- const doc = hooks.buildIndexDocument(input());
- for (const k of ['owner_service', 'resource_type', 'resource_id', 'revision', 'visibility', 'acl', 'canonical_url', 'title', 'summary', 'body', 'facets', 'provenance', 'publication_state', 'deleted', 'indexability']) {
- assert.ok(k in doc, `missing ${k}`);
- }
- assert.deepStrictEqual(doc.acl, { public: true });
- assert.deepStrictEqual(doc.indexability, { indexable: true, listable: true, reasons: [], gate: 'openvibe-publishing/seo@1' });
- assert.deepStrictEqual(doc.provenance.citations, [{ source_item_id: 'src_1', url: 'https://example.org/rye', retrieved_at: '2026-09-20T10:00:00.000Z' }]);
- assert.ok(validateDoc(doc), JSON.stringify(validateDoc.errors));
+test('a published revision becomes exactly the released search.index-document@1 shape', () => {
+ const doc = valid(hooks.buildIndexDocument(input()));
+ assert.deepStrictEqual(Object.keys(doc), ['owner', 'type', 'id', 'revision', 'deleted', 'visibility', 'canonical_url', 'title', 'summary', 'body',
+ 'facets', 'language', 'authorship', 'provenance', 'publication_state', 'published_at', 'updated_at', 'indexability']);
+ assert.strictEqual(doc.owner, 'wiki');
+ assert.strictEqual(doc.visibility, 'public');
+ assert.ok(!('acl' in doc), 'a public document carries no ACL');
+ assert.strictEqual(doc.authorship, 'human');
+ assert.deepStrictEqual(doc.indexability, { decision: 'index', reasons: [] });
+ assert.deepStrictEqual(doc.provenance, [
+ { service: 'sources', type: 'item', id: 'src_1', label: 'Rye', url: 'https://example.org/rye', retrieved_at: '2026-09-20T10:00:00.000Z' },
+ { service: 'wiki', type: 'citation', id: '8', url: 'https://example.org/other' },
+ ], 'no retrieved_at invented for the second citation');
});
});
-test('deleted, unpublished, draft and unlisted resources become tombstones with no content', () => {
- for (const extra of [{ deleted: true }, { state: 'unpublished' }, { state: 'draft' }, { visibility: 'unlisted' }]) {
- const doc = hooks.buildIndexDocument(input(extra));
- assert.strictEqual(doc.deleted, true, JSON.stringify(extra));
- for (const k of ['title', 'summary', 'body', 'canonical_url', 'acl']) assert.ok(!(k in doc), `${k} leaked in a tombstone`);
- assert.ok(validateDoc(doc), JSON.stringify(validateDoc.errors));
- }
+test('authorship, visibility and reason codes map onto the contract enums', () => {
+ const ai = authorship.record({ mode: 'ai', workflow: { id: 'wiki.generate_page', runId: RUN }, stubProvider: true });
+ const hybrid = authorship.record({ mode: 'hybrid', authors: [USER], workflow: { id: 'blog.draft_post', runId: RUN } });
+ const imported = authorship.record({ mode: 'imported', importedFrom: { label: 'legacy' } });
+ assert.strictEqual(valid(hooks.buildIndexDocument(input({ authorship: hybrid }))).authorship, 'ai_assisted');
+ assert.strictEqual(valid(hooks.buildIndexDocument(input({ authorship: imported }))).authorship, 'imported');
+ const aiDoc = valid(hooks.buildIndexDocument(input({ authorship: ai, decision: decisionFor(authorship.gateFacts(ai)) })));
+ assert.strictEqual(aiDoc.authorship, 'ai_generated');
+ assert.deepStrictEqual(aiDoc.provenance[0], { service: 'ai', type: 'run', id: RUN, label: 'wiki.generate_page', stub: true });
+ assert.deepStrictEqual(aiDoc.indexability, { decision: 'noindex', reasons: ['ai_unreviewed', 'stub_provider'] });
+
+ const thin = valid(hooks.buildIndexDocument(input({ decision: decisionFor({ wordCount: 3, citationCount: 0, noindex: true, canonicalUrl: undefined }) })));
+ assert.deepStrictEqual(thin.indexability.reasons, ['thin_content', 'unsourced', 'missing_canonical_url', 'owner_decision']);
+ const dup = valid(hooks.buildIndexDocument(input({ decision: decisionFor({ duplicateOf: 'https://openvibe.wiki/p/x' }) })));
+ assert.deepStrictEqual(dup.indexability.reasons, ['duplicate_of']);
+ for (const code of Object.keys(hooks.SEARCH_REASONS)) assert.match(hooks.SEARCH_REASONS[code], /^[a-z][a-z0-9_]{1,63}$/);
});
});
-test('private content needs an ACL of subjects; gated content needs entitlements', () => {
- const priv = seo.evaluate({ ...facts, visibility: 'private' }, { policy: { requireSources: true } });
+test('private needs subjects; gated becomes members with entitlements/groups; nothing non-public reads as public', () => {
+ const priv = decisionFor({ visibility: 'private' });
assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'private', decision: priv })), /acl.subjects/);
assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'private', decision: priv })), /acl.subjects/);
- assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'private', decision: priv, acl: { subjects: ['everyone'] } })), /acl.subjects/);
- const doc = hooks.buildIndexDocument(input({ visibility: 'private', decision: priv, acl: { subjects: [USER] } }));
- assert.deepStrictEqual(doc.acl, { public: false, subjects: [USER] });
- assert.strictEqual(doc.indexability.listable, false);
- assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'gated', decision: priv })), /entitlements/);
- const vip = hooks.buildIndexDocument(input({ visibility: 'gated', decision: priv, acl: { entitlements: ['vip.tier.gold'] } }));
- assert.deepStrictEqual(vip.acl, { public: false, entitlements: ['vip.tier.gold'] });
- assert.ok(validateDoc(vip));
+ assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'private', decision: priv, acl: { subjects: ['everyone'] } })), /not valid/);
+ const doc = valid(hooks.buildIndexDocument(input({ visibility: 'private', decision: priv, acl: { subjects: [USER, USER] } })));
+ assert.strictEqual(doc.visibility, 'private');
+ assert.deepStrictEqual(doc.acl, { subjects: [USER] });
+ assert.deepStrictEqual(doc.indexability.reasons, ['private']);
+ const gated = decisionFor({ visibility: 'gated' });
+ assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'gated', decision: gated })), /members/);
+ const vip = valid(hooks.buildIndexDocument(input({ visibility: 'gated', decision: gated, acl: { entitlements: ['vip.plan:pln_gold'], groups: ['wiki.space:food:member'] } })));
+ assert.strictEqual(vip.visibility, 'members');
+ assert.deepStrictEqual(vip.acl, { groups: ['wiki.space:food:member'], entitlements: ['vip.plan:pln_gold'] });
+ assert.deepStrictEqual(vip.indexability.reasons, ['members_only']);
+ assert.throws(() => hooks.buildIndexDocument(input({ visibility: 'friends' })), /visibility/);
assert.throws(() => hooks.buildIndexDocument(input({ decision: undefined })), /decision/);
assert.throws(() => hooks.buildIndexDocument(input({ decision: undefined })), /decision/);
});
});
-test('actionFor maps state transitions to event actions', () => {
+test('tombstones are exactly { owner, type, id, revision, deleted: true } for anything that must leave Search', () => {
+ for (const extra of [{ deleted: true }, { state: 'unpublished' }, { state: 'draft' }, { state: 'scheduled' }, { state: 'retracted' }, { state: 'archived' }, { visibility: 'unlisted' }]) {
+ const doc = valid(hooks.buildIndexDocument(input(extra)));
+ assert.deepStrictEqual(doc, { owner: 'wiki', type: 'page', id: 'pg_1', revision: 4, deleted: true }, JSON.stringify(extra));
+ }
+ const unlisted = valid(hooks.buildIndexDocument(input({ visibility: 'unlisted', includeUnlisted: true, decision: decisionFor({ visibility: 'unlisted' }) })));
+ assert.strictEqual(unlisted.visibility, 'unlisted');
+ assert.deepStrictEqual(unlisted.indexability.reasons, ['unlisted']);
+});
+
+test('limits of the contract are respected: long text is clipped, bad facets and ids refused', () => {
+ const doc = valid(hooks.buildIndexDocument(input({ title: 't'.repeat(900), summary: 's'.repeat(5000), body: 'b'.repeat(60000) })));
+ assert.strictEqual(doc.title.length, 500);
+ assert.strictEqual(doc.summary.length, 4000);
+ assert.strictEqual(doc.body.length, 48000);
+ assert.throws(() => hooks.buildIndexDocument(input({ facets: { tags: [1, 2] } })), /strings/);
+ assert.throws(() => hooks.buildIndexDocument(input({ facets: Object.fromEntries(Array.from({ length: 21 }, (_, i) => [`f${i}`, 'x'])) })), /20 facets/);
+ assert.throws(() => hooks.buildIndexDocument(input({ id: 'has space' })), /id must match/);
+ assert.throws(() => hooks.buildIndexDocument(input({ owner: 'open-re' })), /owner/);
+ assert.throws(() => hooks.buildIndexDocument(input({ language: 'english!' })), /BCP 47/);
+ assert.ok(!('summary' in valid(hooks.buildIndexDocument(input({ summary: null })))));
+});
+
+test('index events are what Search consumes: <owner>.index_document.upserted|deleted', () => {
+ const doc = hooks.buildIndexDocument(input());
+ const up = validEnvelope(hooks.indexEvent({ document: doc, now: Date.parse('2026-09-22T12:00:00Z') }));
+ assert.strictEqual(up.event_type, 'wiki.index_document.upserted');
+ assert.strictEqual(up.source, 'wiki');
+ assert.deepStrictEqual(up.actor, { type: 'service', id: 'wiki' });
+ assert.deepStrictEqual(up.subject, { type: 'page', id: 'pg_1', revision: 4 });
+ assert.ok(!('event_id' in up), 'the outbox assigns event_id');
+ assert.deepStrictEqual(searchReads(up), doc);
+
+ const tomb = hooks.tombstone({ owner: 'wiki', type: 'page', id: 'pg_1', revision: 5 });
+ const del = validEnvelope(hooks.indexEvent({ document: tomb, actor: USER }));
+ assert.strictEqual(del.event_type, 'wiki.index_document.deleted');
+ assert.deepStrictEqual(del.payload, { type: 'page', id: 'pg_1', revision: 5 });
+ assert.deepStrictEqual(searchReads(del), tomb);
+ assert.throws(() => hooks.indexEvent({}), /document/);
+});
+
+test('the sequencer bumps the index revision on every indexed change and replays unchanged documents', () => {
+ const seq = hooks.createIndexSequencer(openDb('idx'), { prefix: 'wiki', now: fakeClock() });
+ const a = seq.stamp(hooks.buildIndexDocument(input({ revision: 0 })));
+ assert.strictEqual(a.revision, 1);
+ assert.strictEqual(seq.stamp(hooks.buildIndexDocument(input({ revision: 0 }))).revision, 1, 'same document, same revision');
+ const priv = seq.stamp(hooks.buildIndexDocument(input({ revision: 0, visibility: 'private', acl: { subjects: [USER] }, decision: decisionFor({ visibility: 'private' }) })));
+ assert.strictEqual(priv.revision, 2, 'a visibility change with the same content still gets a new revision');
+ const gone = seq.stamp(hooks.tombstone({ owner: 'wiki', type: 'page', id: 'pg_1', revision: 0 }));
+ assert.strictEqual(gone.revision, 3);
+ const back = seq.stamp(hooks.buildIndexDocument(input({ revision: 0 })));
+ assert.strictEqual(back.revision, 4, 'a restore outranks the tombstone');
+ assert.strictEqual(seq.current('wiki', 'page', 'pg_1'), 4);
+ valid(back);
+ assert.strictEqual(seq.table, 'wiki_index_revisions');
+});
+
+test('actionFor maps state transitions to product events', () => {
const pub = { state: 'published', visibility: 'public', revision: 1 };
const pub = { state: 'published', visibility: 'public', revision: 1 };
assert.strictEqual(hooks.actionFor(null, pub), 'published');
assert.strictEqual(hooks.actionFor(null, pub), 'published');
assert.strictEqual(hooks.actionFor({ state: 'draft', revision: 1 }, pub), 'published');
assert.strictEqual(hooks.actionFor({ state: 'draft', revision: 1 }, pub), 'published');
@@ -71,41 +168,26 @@ test('actionFor maps state transitions to event actions', () => {
@@ -71,41 +168,26 @@ test('actionFor maps state transitions to event actions', () => {
assert.strictEqual(hooks.actionFor(pub, { ...pub, state: 'unpublished' }), 'unpublished');
assert.strictEqual(hooks.actionFor(pub, { ...pub, state: 'unpublished' }), 'unpublished');
assert.strictEqual(hooks.actionFor(pub, { ...pub, state: 'deleted' }), 'deleted');
assert.strictEqual(hooks.actionFor(pub, { ...pub, state: 'deleted' }), 'deleted');
assert.strictEqual(hooks.actionFor({ state: 'deleted' }, { state: 'deleted' }), null);
assert.strictEqual(hooks.actionFor({ state: 'deleted' }, { state: 'deleted' }), null);
- assert.strictEqual(hooks.actionFor({ state: 'draft' }, { state: 'draft' }), null);
});
});
-test('publication events are valid contracts envelopes: <product>.<type>.<action>', () => {
+test('product events <product>.<type>.<action> are valid envelopes, internal unless public and listable', () => {
+ const decision = decisionFor();
const doc = hooks.buildIndexDocument(input());
const doc = hooks.buildIndexDocument(input());
- const env = hooks.publicationEvent({ product: 'wiki', type: 'page', action: 'published', id: 'pg_1', revision: 4, actor: USER, document: doc, decision, now: Date.parse('2026-09-22T12:00:00Z') });
+ const env = validEnvelope(hooks.publicationEvent({ product: 'wiki', type: 'page', action: 'published', id: 'pg_1', revision: 4, actor: USER, document: doc, decision }));
assert.strictEqual(env.event_type, 'wiki.page.published');
assert.strictEqual(env.event_type, 'wiki.page.published');
assert.strictEqual(env.visibility, 'public');
assert.strictEqual(env.visibility, 'public');
- assert.deepStrictEqual(env.actor, { type: 'user', id: USER });
- assert.deepStrictEqual(env.subject, { type: 'page', id: 'pg_1', revision: 4 });
- assert.strictEqual(env.payload.document, doc);
- assert.ok(!('event_id' in env), 'the outbox assigns event_id');
- const check = contracts.validate('events.event-envelope@1', withEventId(env));
- assert.ok(check.valid, JSON.stringify(check.errors));
-
- const tomb = hooks.buildIndexDocument(input({ deleted: true }));
- const del = hooks.publicationEvent({ product: 'blog', type: 'post', action: 'deleted', id: 'pg_1', revision: 4, actor: 'svc:blog', document: tomb });
- assert.strictEqual(del.event_type, 'blog.post.deleted');
+ assert.deepStrictEqual(env.payload, { canonical_url: 'https://openvibe.wiki/p/rye', publication_state: 'published', indexability: { decision: 'index', reasons: [] } });
+ assert.ok(!('body' in env.payload), 'no body in the product event');
+ const priv = decisionFor({ visibility: 'private' });
+ const pdoc = hooks.buildIndexDocument(input({ visibility: 'private', acl: { subjects: [USER] }, decision: priv }));
+ assert.strictEqual(validEnvelope(hooks.publicationEvent({ product: 'wiki', type: 'page', action: 'updated', id: 'pg_1', revision: 4, actor: USER, document: pdoc, decision: priv })).visibility, 'internal');
+ const tomb = hooks.tombstone({ owner: 'blog', type: 'post', id: 'pg_1', revision: 9 });
+ const del = validEnvelope(hooks.publicationEvent({ product: 'blog', type: 'post', action: 'deleted', id: 'pg_1', revision: 4, actor: 'svc:blog', document: tomb }));
assert.strictEqual(del.visibility, 'internal');
assert.strictEqual(del.visibility, 'internal');
- assert.deepStrictEqual(del.actor, { type: 'service', id: 'blog' });
- assert.ok(contracts.validate('events.event-envelope@1', withEventId(del)).valid);
+ assert.strictEqual(del.payload.publication_state, 'deleted');
assert.throws(() => hooks.publicationEvent({ product: 'blog', type: 'post', action: 'deleted', id: 'pg_1', revision: 4, actor: USER, document: doc }), /tombstone/);
assert.throws(() => hooks.publicationEvent({ product: 'blog', type: 'post', action: 'deleted', id: 'pg_1', revision: 4, actor: USER, document: doc }), /tombstone/);
assert.throws(() => hooks.publicationEvent({ product: 'blog', type: 'post', action: 'published', id: 'pg_1', revision: 4, actor: USER, document: tomb }), /published document/);
assert.throws(() => hooks.publicationEvent({ product: 'blog', type: 'post', action: 'published', id: 'pg_1', revision: 4, actor: USER, document: tomb }), /published document/);
assert.throws(() => hooks.publicationEvent({ product: 'blog', type: 'post', action: 'moved', id: 'pg_1', revision: 4, actor: USER, document: doc }), /action/);
assert.throws(() => hooks.publicationEvent({ product: 'blog', type: 'post', action: 'moved', id: 'pg_1', revision: 4, actor: USER, document: doc }), /action/);
});
});
-test('events about private or non-listable content are internal', () => {
- const priv = seo.evaluate({ ...facts, visibility: 'private' }, { policy: { requireSources: true } });
- const doc = hooks.buildIndexDocument(input({ visibility: 'private', decision: priv, acl: { subjects: [USER] } }));
- const env = hooks.publicationEvent({ product: 'wiki', type: 'page', action: 'updated', id: 'pg_1', revision: 4, actor: USER, document: doc });
- assert.strictEqual(env.visibility, 'internal');
- const unlisted = hooks.buildIndexDocument(input({ visibility: 'unlisted' }));
- const env2 = hooks.publicationEvent({ product: 'wiki', type: 'page', action: 'updated', id: 'pg_1', revision: 4, actor: USER, document: unlisted });
- assert.strictEqual(env2.visibility, 'internal');
- assert.ok(contracts.validate('events.event-envelope@1', withEventId(env2)).valid);
-});
-
run();
run();
@@ -7,6 +7,7 @@ const assert = require('assert');
@@ -7,6 +7,7 @@ const assert = require('assert');
const fs = require('fs');
const fs = require('fs');
const http = require('http');
const http = require('http');
const path = require('path');
const path = require('path');
+const contracts = require('openvibe-contracts');
const { tempDir, fakeClock, suite } = require('./helpers/db');
const { tempDir, fakeClock, suite } = require('./helpers/db');
const { createWiki } = require('../examples/two-products/wiki/app');
const { createWiki } = require('../examples/two-products/wiki/app');
const { createBlog } = require('../examples/two-products/blog/app');
const { createBlog } = require('../examples/two-products/blog/app');
@@ -191,10 +192,22 @@ test('HTTP: pages useful without JavaScript, 301 for old slugs, private/deleted
@@ -191,10 +192,22 @@ test('HTTP: pages useful without JavaScript, 301 for old slugs, private/deleted
assert.doesNotMatch(await (await w.get('/sitemap.xml')).text(), /rye/);
assert.doesNotMatch(await (await w.get('/sitemap.xml')).text(), /rye/);
assert.doesNotMatch(await (await w.get('/feed.atom')).text(), /<entry>/);
assert.doesNotMatch(await (await w.get('/feed.atom')).text(), /<entry>/);
assert.strictEqual(JSON.parse(await (await w.get('/feed.json')).text()).items.length, 0);
assert.strictEqual(JSON.parse(await (await w.get('/feed.json')).text()).items.length, 0);
- const events = wiki.outbox();
- assert.deepStrictEqual(events.map((e) => e.event_type), ['wiki.page.published', 'wiki.page.updated']);
- assert.strictEqual(events[1].visibility, 'internal');
- assert.deepStrictEqual(events[1].payload.document.acl, { public: false, subjects: [ALEX] });
+ let events = wiki.outbox();
+ assert.deepStrictEqual(events.map((e) => e.event_type), [
+ 'wiki.index_document.upserted', 'wiki.page.published', // publish
+ 'wiki.index_document.upserted', // rename: new canonical URL, new index revision
+ 'wiki.index_document.upserted', 'wiki.page.updated', // made private
+ ]);
+ const docs = events.filter((e) => e.event_type.startsWith('wiki.index_document.')).map((e) => e.payload);
+ assert.deepStrictEqual(docs.map((d) => d.revision), [1, 2, 3], 'every indexed change is a higher revision');
+ assert.strictEqual(docs[1].canonical_url, 'https://openvibe.wiki/p/rye-grain');
+ assert.strictEqual(docs[2].visibility, 'private');
+ assert.deepStrictEqual(docs[2].acl, { subjects: [ALEX] });
+ assert.strictEqual(events[4].visibility, 'internal');
+ for (const e of events) {
+ assert.ok(contracts.validate('events.event-envelope@1', { ...e, event_id: contracts.ids.newId('event') }).valid, e.event_type);
+ }
+ for (const d of docs) assert.ok(contracts.validate('search.index-document@1', d).valid);
blog.setVisibility(post, 'gated');
blog.setVisibility(post, 'gated');
assert.strictEqual((await b.get('/posts/crumb-shots')).status, 404);
assert.strictEqual((await b.get('/posts/crumb-shots')).status, 404);
@@ -206,10 +219,12 @@ test('HTTP: pages useful without JavaScript, 301 for old slugs, private/deleted
@@ -206,10 +219,12 @@ test('HTTP: pages useful without JavaScript, 301 for old slugs, private/deleted
wiki.remove(id, { actor: ALEX });
wiki.remove(id, { actor: ALEX });
assert.strictEqual((await w.get('/p/rye-grain')).status, 410);
assert.strictEqual((await w.get('/p/rye-grain')).status, 410);
assert.strictEqual((await w.get('/p/rye')).status, 410);
assert.strictEqual((await w.get('/p/rye')).status, 410);
- const last = wiki.outbox().pop();
- assert.strictEqual(last.event_type, 'wiki.page.deleted');
- assert.strictEqual(last.payload.document.deleted, true);
- assert.ok(!('body' in last.payload.document));
+ events = wiki.outbox().slice(-2);
+ assert.deepStrictEqual(events.map((e) => e.event_type), ['wiki.index_document.deleted', 'wiki.page.deleted']);
+ assert.deepStrictEqual(events[0].payload, { type: 'page', id, revision: 4 });
+ assert.deepStrictEqual(events[0].subject, { type: 'page', id, revision: 4 });
+ assert.strictEqual(events[1].payload.publication_state, 'deleted');
+ for (const e of events) assert.ok(contracts.validate('events.event-envelope@1', { ...e, event_id: contracts.ids.newId('event') }).valid);
} finally { w.close(); b.close(); wiki.close(); blog.close(); }
} finally { w.close(); b.close(); wiki.close(); blog.close(); }
});
});