openvibers/openvibe.search Commits

The SQLite era is gone (on PostgreSQL since 2026-09-28; no compatibility kept): the one-time migrate-to-postgres script, the *_DB_PATH settings (config, .env.example, systemd unit, test helpers), the unused better-sqlite3 dependency and its lockfile entries, and docs, docstrings and STATUS lines about SQLite or its rollback window. Reviewed independently before shipping. npm test 12/12 test files passed.

OP OpenVibers committed 29 Sept 2026 · 7be6c4231d · 19 files, +170 −230


Changes

ddiff --git a/.env.example b/.env.example
iindex a3920cd..be8a1ec 100644
---- a/.env.example
++++ b/.env.example
@@@ -2,7 +2,7 @@
# OpenVibe.Search — Environment Configuration
# Copy to .env and fill in values.
# In production this lives at /etc/openvibe/search.env (0600); the systemd unit sets
--# NODE_ENV, PORT and SEARCH_DB_PATH itself.
++# NODE_ENV and PORT itself.
# ═══════════════════════════════════════════════════════════════
# Server
@@@ -27,16 +27,13 @@ OV_NETWORK_INTERNAL_URL=http://127.0.0.1:4000
OV_OAUTH_CLIENT_ID=search
OV_OAUTH_CLIENT_SECRET=
--# Store (SQLite, FTS5). Production: /var/lib/openvibe-search/search.db (systemd StateDirectory).
--# PostgreSQL (ADR-035), written by OpenVibe.Host roles/data/add-service.sh search: the runtime role through PgBouncer,
--# the owner on a direct connection (migrations and the one-time import), and Valkey (shared limit counters).
++# Store (PostgreSQL, ADR-035), written by OpenVibe.Host roles/data/add-service.sh search: the runtime role through
++# PgBouncer, the owner on a direct connection (migrations), and Valkey (shared limit counters).
# Without DATABASE_URL, development uses an embedded PGlite database in data/pglite.
DATABASE_URL=
DATABASE_DIRECT_URL=
VALKEY_URL=
VALKEY_PREFIX=
--# The SQLite file of releases before PostgreSQL: read once by scripts/migrate-to-postgres.js.
--SEARCH_DB_PATH=./data/search.db
# OpenVibe.Events. The outbox relay runs only when EVENTS_URL is set.
#EVENTS_URL=http://127.0.0.1:4300
@@@ -52,7 +49,7 @@ SEARCH_DB_PATH=./data/search.db
#SEARCH_MAX_BODY_BYTES=262144
# Ranking: freshness weighting of full-text relevance (README "Ranking").
--# score = bm25 × (1 + WEIGHT × 2^(−age_days / HALF_LIFE_DAYS)); age from published_at, else
++# score = relevance × (1 + WEIGHT × 2^(−age_days / HALF_LIFE_DAYS)); age from published_at, else
# updated_at. WEIGHT 0 turns it off.
#SEARCH_FRESHNESS_WEIGHT=1
#SEARCH_FRESHNESS_HALF_LIFE_DAYS=30
ddiff --git a/README.md b/README.md
iindex 37a9510..4ad0d5e 100644
---- a/README.md
++++ b/README.md
@@@ -25,17 +25,17 @@ revision-ordered copy and can be reconciled against it.
npm install
cp .env.example .env
npm run dev # http://127.0.0.1:4710
--npm test # every test/*.test.js, temp databases, no network needed
++npm test # every test/*.test.js, a PGlite database per file, no network needed
```
Node 22 in production (`fnm exec --using=22.22.1 npm test`). Production: `/opt/openvibe.search`,
env `/etc/openvibe/search.env`, unit [deploy/systemd/openvibe-search.service](deploy/systemd/openvibe-search.service),
--database `ov_search` on the host's data role (ADR-035; `sudo /opt/openvibe.host/roles/data/add-service.sh search`; the one-time move from `/var/lib/openvibe-search/search.db` is `scripts/migrate-to-postgres.js`, and the full-text index rebuilds itself from the documents at boot), nginx [deploy/nginx/search.openvibe.network.conf](deploy/nginx/search.openvibe.network.conf)
++database `ov_search` on the host's data role (ADR-035; `sudo /opt/openvibe.host/roles/data/add-service.sh search`; the full-text index rebuilds itself from the documents at boot), nginx [deploy/nginx/search.openvibe.network.conf](deploy/nginx/search.openvibe.network.conf)
(the public vhost exposes the search page, the GET query API, saved searches and health; the owner
API and the Events webhook are loopback-only, and owners and Events call `127.0.0.1:4710`).
`GET /api/health` is liveness. `GET /api/ready` (openvibe-shared/ready) is 503 only when the
--database (the documents and both full-text tables) fails; a Network signing key that has not loaded
++database (the documents and the full-text table) fails; a Network signing key that has not loaded
(anonymous queries still answer) and a full-text index out of step with the documents table degrade
it. It also reports document counts by exposure, the outbox backlog and whether the Events webhook
is on. `GET /metrics` (openvibe-shared/metrics) answers direct loopback callers only: golden signals
@@@ -51,7 +51,7 @@ GET /api/v1/documents/:owner/:type/:id
```
- `q` is plain text: word tokens only, every word required, a trailing `*` makes the last word a
-- prefix. FTS operators and column filters in `q` are neutralised. No `q` = newest first.
++ prefix. Query operators and column filters in `q` are neutralised. No `q` = newest first.
- Results carry `owner, type, id, revision, visibility, title, summary, canonical_url, facets,
language, authorship, provenance, published_at, updated_at, indexable` and a `snippet_html`
(escaped text with `<mark>`). Never the ACL. `next_cursor` is bound to the query that made it.
@@@ -69,7 +69,7 @@ Who sees what:
Drafts, unpublished, retracted and deleted documents are returned to nobody. A hidden document and
a missing one give the same 404. A Bearer that does not verify is a 401 (never downgraded to
anonymous); delegation headers from a browser are ignored. Visibility is enforced in the engine
--SQL (two FTS tables split by audience plus an ACL predicate) and re-checked per hit — see
++SQL (the full-text table split by audience plus an ACL predicate) and re-checked per hit — see
[docs/adr-engine.md](docs/adr-engine.md).
The search page at `/` runs the same query as HTML for browsers (no JavaScript; no-store; result
@@@ -78,11 +78,11 @@ route index at `/`.
### Ranking
--Full-text results are ordered by bm25 relevance (title ×10, summary ×4, body ×1) multiplied by a
++Full-text results are ordered by full-text relevance (title ×10, summary ×4, body ×1) multiplied by a
freshness boost:
```
--score = bm25 × (1 + W × 2^(−age_days / H)) W = SEARCH_FRESHNESS_WEIGHT (default 1)
++score = relevance × (1 + W × 2^(−age_days / H)) W = SEARCH_FRESHNESS_WEIGHT (default 1)
H = SEARCH_FRESHNESS_HALF_LIFE_DAYS (default 30)
```
@@@ -236,7 +236,7 @@ Never limited: the signed Events deliveries (`POST /internal/events`), `/api/hea
## Acceptance (tests)
- a private or draft document is never returned to an unauthorized query — hits, snippets, facet
-- counts, suggestions, direct gets, cursors and FTS-syntax injection (`test/acl-leak.test.js`)
++ counts, suggestions, direct gets, cursors and full-text-syntax injection (`test/acl-leak.test.js`)
- a visibility change or deletion removes the document from results immediately and emits
`search.document.removed` (`test/acl-leak.test.js`)
- revision ordering, tombstones, restore, reconciliation (`test/documents.test.js`)
ddiff --git a/STATUS.json b/STATUS.json
iindex 452898d..c0f9b98 100644
---- a/STATUS.json
++++ b/STATUS.json
@@@ -4,7 +4,7 @@
"code": true,
"wave": "W14",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
-- "runtime": "node 22, express, better-sqlite3 (FTS5)",
++ "runtime": "node 22, express, PostgreSQL full-text search (openvibe-sdk/db)",
"port": 4710,
"domain": "search.openvibe.network (vhost in deploy/nginx; the service manifest records the search page and query API as public since 2026-09-23)",
"contracts": "openvibe-contracts v0.76.0",
@@@ -17,14 +17,14 @@
"contractsProposed": [
"search.index-document@1 (released in openvibe-contracts v0.12.0)"
],
-- "engine": "sqlite-fts5 (docs/adr-engine.md)",
++ "engine": "postgresql-fts (docs/adr-engine.md)",
"deployed": true,
"deployment": "Deployed with `sudo ovhost deploy search`: /opt/openvibe.search, unit openvibe-search, 127.0.0.1:4710, principal search, Events subscription *.index_document.*, accepted event owners sources, wiki, blog, news, reviews, trade, deals, coupons. On 2026-09-23 the production index held 10 documents, all tombstones (wiki), 0 public. Restore drill passed 2026-09-23.",
"indexedProducers": [
"wiki (tombstones only so far)"
],
"features": [
-- "ACL-filtered FTS5 query API",
++ "ACL-filtered full-text query API",
"facets",
"suggestions",
"freshness-weighted ranking",
ddiff --git a/deploy/systemd/openvibe-search.service b/deploy/systemd/openvibe-search.service
iindex 43d2f19..d001028 100644
---- a/deploy/systemd/openvibe-search.service
++++ b/deploy/systemd/openvibe-search.service
@@@ -1,5 +1,5 @@
[Unit]
--Description=OpenVibe.Search — permission-aware index and query API (SQLite FTS5)
++Description=OpenVibe.Search — permission-aware index and query API (PostgreSQL full-text search)
# Network (openvibe-network) signs the tokens this service verifies. Search starts without it
# and keeps retrying the JWKS fetch; /api/ready stays 503 until the key has loaded.
After=network.target postgresql.service pgbouncer.service valkey-server.service
@@@ -23,9 +23,6 @@ NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
--# The code tree stays read-only; the index lives in /var/lib/openvibe-search.
--StateDirectory=openvibe-search
--Environment=SEARCH_DB_PATH=/var/lib/openvibe-search/search.db
[Install]
WantedBy=multi-user.target
ddiff --git a/docs/adr-engine.md b/docs/adr-engine.md
iindex 189f01f..2399cf7 100644
---- a/docs/adr-engine.md
++++ b/docs/adr-engine.md
@@@ -1,66 +1,54 @@
--# ADR: search engine for Wave 14 — SQLite FTS5 now, PostgreSQL FTS when Postgres exists
++# ADR: search engine for Wave 14 — PostgreSQL full-text search
--**Status:** accepted for alpha (2026-09-22) · **Owner:** OpenVibe.Search · **Roadmap:** §4.2 A, §15.12, Wave 14
++**Status:** accepted (2026-09-22; engine updated to PostgreSQL on 2026-09-28, ADR-035) · **Owner:** OpenVibe.Search · **Roadmap:** §4.2 A, §15.12, Wave 14
## Context
The roadmap asks for an event-fed, permission-aware index with the engine chosen only after
measuring need: *"PostgreSQL FTS first; Meilisearch or OpenSearch only if justified"*, and warns
--against engine shopping. The network runs every service on one host with one SQLite database per
--service; PostgreSQL is not provisioned yet (it arrives with the legacy-retirement track). There
--is no corpus to measure: no product indexes anything today.
++against engine shopping. No external search engine is wanted: the corpus is SQL rows, the ACL
++model is SQL, and a visibility change must leave results in the same transaction that makes it.
++There is no corpus to measure: no product indexes anything today.
## Decision
--1. **SQLite FTS5 inside Search's own database** (`/var/lib/openvibe-search/search.db`), no
-- external engine, no new daemon. FTS5 ships in better-sqlite3's bundled SQLite (3.49), gives
-- BM25 ranking, snippets and prefix queries, and commits in the same transaction as the
-- document row, ACL rows, facet rows, the inbox receipt and the outbox event. That transaction
-- is what makes "a visibility change or deletion removes the document from results immediately"
-- true without a second system to converge.
--2. **Behind a small engine interface** — [server/engine/fts5.js](../server/engine/fts5.js):
-- `put(rid, doc, exposure)`, `remove(rid)`, `search`, `browse`, `facetCounts`, `suggest`. The
-- store, the revision rules, the ACL model and the APIs do not know FTS5 exists.
--3. **PostgreSQL FTS is the next engine**, as the roadmap says, once a PostgreSQL instance exists
-- for services: a `tsvector` column with `websearch_to_tsquery`/`ts_rank_cd`, GIN index, the same
-- ACL predicate as SQL. Meilisearch/OpenSearch stay out unless a measurement justifies them.
++1. **PostgreSQL full-text search in the service's own database** (`search_fts`, ADR-035), no
++ external engine, no new daemon. A `tsvector` weighted title A, summary B, body C with a GIN
++ index gives ranked matches, `ts_headline` snippets and prefix queries, and commits in the same
++ transaction as the document row, ACL rows, facet rows, the inbox receipt and the outbox event.
++ That transaction is what makes "a visibility change or deletion removes the document from
++ results immediately" true without a second system to converge.
++2. **Behind a small engine interface** — [server/engine/pg.js](../server/engine/pg.js):
++ `put(rid, doc, exposure)`, `remove(rid)`, `search`, `browse`, `facetCounts`, `suggest`, `counts`,
++ `reconcile`. The store, the revision rules, the ACL model and the APIs do not know the engine.
++3. **Meilisearch/OpenSearch stay out** unless a measurement justifies them, as the roadmap says.
## How visibility is enforced inside the engine
--- Documents are split into two FTS tables by audience: `fts_public` holds only public,
-- published, indexable documents; `fts_restricted` only published unlisted/members/private
-- documents. Drafts, unpublished, deleted and public-noindex documents are in neither, so no text
-- query can reach them.
--- An anonymous query (or a viewer without a subject) only ever reads `fts_public`. BM25 statistics
-- are per table, so private documents cannot even shift the ranking an anonymous caller sees.
--- A signed-in query adds a second branch over `fts_restricted`, filtered in the same SQL by the
-- viewer's ACL keys (`doc_acl`: subject, group, entitlement). Facet counts and suggestions reuse
-- the exact same visible-set SQL, so they cannot count or complete what the viewer cannot see.
--- Known limitation: within `fts_restricted`, BM25 term statistics include restricted documents the
++- One table, `search_fts`, split by the `audience` column: exposure 3 holds only public, published,
++ indexable documents; exposure 1 holds published unlisted/members/private documents. Drafts,
++ unpublished, deleted and public-noindex documents have no row, so no text query can reach them.
++- An anonymous query (or a viewer without a subject) only ever reads audience 3. Ranking statistics
++ are computed over the same visible rows, so private documents cannot shift what an anonymous
++ caller sees.
++- A signed-in query adds a second branch over audience 1, filtered in the same SQL by the viewer's
++ ACL keys (`doc_acl`: subject, group, entitlement). Facet counts and suggestions reuse the exact
++ same visible-set SQL, so they cannot count or complete what the viewer cannot see.
++- Known limitation: within audience 1, `ts_rank` term statistics include restricted documents the
viewer cannot see. The effect on ordering is statistical and returns no content, but it is a
-- theoretical side channel between signed-in users; a per-audience or per-tenant index (or
-- PostgreSQL with ranking that ignores global statistics) removes it if it ever matters.
++ theoretical side channel between signed-in users; a per-audience or per-tenant index removes it
++ if it ever matters.
## Consequences
--- One process and one file: backups, restores and moves are the same as every other service.
--- Write throughput is SQLite's (single writer, WAL). That is ample for the expected corpus (tens to
-- hundreds of thousands of documents); the trigger to revisit is measured: p95 query latency over
-- 200 ms at the public edge, index size over a few GB, or write contention in `/api/ready`.
--- Tokenizer is `unicode61 remove_diacritics 2` without stemming: predictable across languages
-- (Japanese/Chinese get no word segmentation; a later engine or an ICU tokenizer fixes that).
--- Moving to PostgreSQL is an engine swap plus a re-index from owners (the reconciliation API
-- exists for exactly that), not a contract change.
--
--## Update 2026-09-28: PostgreSQL full-text search (ADR-035)
--
--Search moved to PostgreSQL with every other service. The engine interface held: `server/engine/pg.js` replaced
--`server/engine/fts5.js` and nothing else learned the engine changed. One table, `search_fts`, split by audience
--(exposure 3 public listed, 1 restricted), with a `tsvector` weighted title A, summary B, body C (FTS5's 10 / 4 / 1 as
--`ts_rank` weights) and a GIN index. Text is folded (diacritics removed) and split into runs of letters and digits before
--the `simple` configuration sees it, so tokens are exactly FTS5 unicode61's: on a copy of production, the match counts
--for sampled terms were identical. The rank is minus `ts_rank` times the same freshness boost, so ascending order and
--the keyset cursors work unchanged. Snippets are `ts_headline` over the summary and body (the title when only it
--matches), computed for the returned page only. The index is derived: at boot, when its counts disagree with the
--documents table (after the one-time import, whose FTS5 tables are not copied), it is rebuilt from `documents.doc`.
--
++- One database: backups, restores and moves are the same as every other service (PostgreSQL,
++ ADR-035).
++- Ranking is `ts_rank` with title/summary/body weights 10/4/1 times a freshness boost
++ ([README "Ranking"](../README.md#ranking)), returned as minus the score so ascending keyset
++ cursors work.
++- Text is folded (diacritics removed) and split into runs of letters and digits before the
++ `simple` configuration sees it, so tokens are predictable across languages (Japanese/Chinese get
++ no word segmentation; a later engine or ICU tokenizer fixes that) and a query and the index
++ always agree on what a word is.
++- The index is derived from `documents.doc`: at boot, when its counts disagree with the documents
++ table, it is rebuilt (the reconciliation API lets owners re-push everything if needed).
ddiff --git a/docs/service-manifest-proposal.json b/docs/service-manifest-proposal.json
iindex d9ed0b0..a0aa501 100644
---- a/docs/service-manifest-proposal.json
++++ b/docs/service-manifest-proposal.json
@@@ -26,5 +26,5 @@
"*.index_document.deleted"
],
"namespacesOwned": [],
-- "notes": "Canonical index document (search.index-document@1), event-fed and direct indexing with revision ordering and tombstones, permission-aware query over SQLite FTS5 (docs/adr-engine.md). Anonymous queries need no token."
++ "notes": "Canonical index document (search.index-document@1), event-fed and direct indexing with revision ordering and tombstones, permission-aware query over PostgreSQL full-text search (docs/adr-engine.md). Anonymous queries need no token."
}
ddiff --git a/package-lock.json b/package-lock.json
iindex 770eed6..247121a 100644
---- a/package-lock.json
++++ b/package-lock.json
@@@ -11,7 +11,6 @@
"dependencies": {
"ajv": "^8.17.1",
"ajv-formats": "^3.0.1",
-- "better-sqlite3": "^11.10.0",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",
@@@ -103,7 +102,9 @@
"url": "https://feross.org/support"
}
],
-- "license": "MIT"
++ "license": "MIT",
++ "optional": true,
++ "peer": true
},
"node_modules/better-sqlite3": {
"version": "11.10.0",
@@@ -111,6 +112,8 @@
"integrity": "sha512-EwhOpyXiOEL/lKzHz9AW1msWFNzGc/z+LzeB3/jnFJpxu+th2yqvzsSWas1v9jgs9+xiXJcD5A8CJxAG2TaghQ==",
"hasInstallScript": true,
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"bindings": "^1.5.0",
"prebuild-install": "^7.1.1"
@@@ -121,6 +124,8 @@
"resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
"integrity": "sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"file-uri-to-path": "1.0.0"
}
@@@ -130,6 +135,8 @@
"resolved": "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz",
"integrity": "sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"buffer": "^5.5.0",
"inherits": "^2.0.4",
@@@ -179,6 +186,8 @@
}
],
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"base64-js": "^1.3.1",
"ieee754": "^1.1.13"
@@@ -232,7 +241,9 @@
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz",
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
-- "license": "ISC"
++ "license": "ISC",
++ "optional": true,
++ "peer": true
},
"node_modules/content-disposition": {
"version": "0.5.4",
@@@ -284,6 +295,8 @@
"resolved": "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz",
"integrity": "sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"mimic-response": "^3.1.0"
},
@@@ -299,6 +312,8 @@
"resolved": "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz",
"integrity": "sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"engines": {
"node": ">=4.0.0"
}
@@@ -327,6 +342,8 @@
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
"integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
"license": "Apache-2.0",
++ "optional": true,
++ "peer": true,
"engines": {
"node": ">=8"
}
@@@ -386,6 +403,8 @@
"resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",
"integrity": "sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"once": "^1.4.0"
}
@@@ -440,6 +459,8 @@
"resolved": "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz",
"integrity": "sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg==",
"license": "(MIT OR WTFPL)",
++ "optional": true,
++ "peer": true,
"engines": {
"node": ">=6"
}
@@@ -516,7 +537,9 @@
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz",
"integrity": "sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw==",
-- "license": "MIT"
++ "license": "MIT",
++ "optional": true,
++ "peer": true
},
"node_modules/finalhandler": {
"version": "1.3.2",
@@@ -558,7 +581,9 @@
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz",
"integrity": "sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow==",
-- "license": "MIT"
++ "license": "MIT",
++ "optional": true,
++ "peer": true
},
"node_modules/function-bind": {
"version": "1.1.2",
@@@ -610,7 +635,9 @@
"version": "0.0.0",
"resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",
"integrity": "sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw==",
-- "license": "MIT"
++ "license": "MIT",
++ "optional": true,
++ "peer": true
},
"node_modules/gopd": {
"version": "1.2.0",
@@@ -698,7 +725,9 @@
"url": "https://feross.org/support"
}
],
-- "license": "BSD-3-Clause"
++ "license": "BSD-3-Clause",
++ "optional": true,
++ "peer": true
},
"node_modules/inherits": {
"version": "2.0.4",
@@@ -710,7 +739,9 @@
"version": "1.3.8",
"resolved": "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz",
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
-- "license": "ISC"
++ "license": "ISC",
++ "optional": true,
++ "peer": true
},
"node_modules/ipaddr.js": {
"version": "1.9.1",
@@@ -892,6 +923,8 @@
"resolved": "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz",
"integrity": "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"engines": {
"node": ">=10"
},
@@@ -904,6 +937,8 @@
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
@@@ -912,7 +947,9 @@
"version": "0.5.3",
"resolved": "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz",
"integrity": "sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A==",
-- "license": "MIT"
++ "license": "MIT",
++ "optional": true,
++ "peer": true
},
"node_modules/ms": {
"version": "2.0.0",
@@@ -924,7 +961,9 @@
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz",
"integrity": "sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA==",
-- "license": "MIT"
++ "license": "MIT",
++ "optional": true,
++ "peer": true
},
"node_modules/negotiator": {
"version": "0.6.3",
@@@ -940,6 +979,8 @@
"resolved": "https://registry.npmjs.org/node-abi/-/node-abi-3.96.0.tgz",
"integrity": "sha512-rebQ/lz7i0EkoLzUVSrKRzA69zMkwLp95kKMWoMDkkM00Suxz0D7zEQPwRml5fQum24mj7bPvmlgLAmu2JCiYg==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"semver": "^7.3.5"
},
@@@ -976,6 +1017,8 @@
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
"integrity": "sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==",
"license": "ISC",
++ "optional": true,
++ "peer": true,
"dependencies": {
"wrappy": "1"
}
@@@ -1183,6 +1226,8 @@
"integrity": "sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug==",
"deprecated": "No longer maintained. Please contact the author of the relevant native addon; alternatives are available.",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"detect-libc": "^2.0.0",
"expand-template": "^2.0.3",
@@@ -1226,6 +1271,8 @@
"resolved": "https://registry.npmjs.org/pump/-/pump-3.0.4.tgz",
"integrity": "sha512-VS7sjc6KR7e1ukRFhQSY5LM2uBWAUPiOPa/A3mkKmiMwSmRFUITt0xuj+/lesgnCv+dPIEYlkzrcyXgquIHMcA==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"end-of-stream": "^1.1.0",
"once": "^1.3.1"
@@@ -1276,6 +1323,8 @@
"resolved": "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz",
"integrity": "sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw==",
"license": "(BSD-2-Clause OR MIT OR Apache-2.0)",
++ "optional": true,
++ "peer": true,
"dependencies": {
"deep-extend": "^0.6.0",
"ini": "~1.3.0",
@@@ -1291,6 +1340,8 @@
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
"integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"inherits": "^2.0.3",
"string_decoder": "^1.1.1",
@@@ -1488,7 +1539,9 @@
"url": "https://feross.org/support"
}
],
-- "license": "MIT"
++ "license": "MIT",
++ "optional": true,
++ "peer": true
},
"node_modules/simple-get": {
"version": "4.0.1",
@@@ -1509,6 +1562,8 @@
}
],
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"decompress-response": "^6.0.0",
"once": "^1.3.1",
@@@ -1538,6 +1593,8 @@
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
"integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"safe-buffer": "~5.2.0"
}
@@@ -1547,6 +1604,8 @@
"resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz",
"integrity": "sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"engines": {
"node": ">=0.10.0"
}
@@@ -1556,6 +1615,8 @@
"resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.5.tgz",
"integrity": "sha512-OboTd8mmMhZDNPV+UjQcK9yKAatXu2aJ+r1w4im1Otd4M4fl2hwvdoXUxIYHFTHWK/3y3FarBP70v3vwmGlOxw==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"chownr": "^1.1.1",
"mkdirp-classic": "^0.5.2",
@@@ -1568,6 +1629,8 @@
"resolved": "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz",
"integrity": "sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==",
"license": "MIT",
++ "optional": true,
++ "peer": true,
"dependencies": {
"bl": "^4.0.3",
"end-of-stream": "^1.4.1",
@@@ -1593,6 +1656,8 @@
"resolved": "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz",
"integrity": "sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==",
"license": "Apache-2.0",
++ "optional": true,
++ "peer": true,
"dependencies": {
"safe-buffer": "^5.0.1"
},
@@@ -1626,7 +1691,9 @@
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
-- "license": "MIT"
++ "license": "MIT",
++ "optional": true,
++ "peer": true
},
"node_modules/utils-merge": {
"version": "1.0.1",
@@@ -1650,7 +1717,9 @@
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
-- "license": "ISC"
++ "license": "ISC",
++ "optional": true,
++ "peer": true
},
"node_modules/xtend": {
"version": "4.0.2",
ddiff --git a/package.json b/package.json
iindex 67b284f..9bcdd44 100644
---- a/package.json
++++ b/package.json
@@@ -2,7 +2,7 @@
"name": "openvibe-search",
"version": "0.1.0",
"private": true,
-- "description": "OpenVibe.Search: the canonical index document, event-fed and direct indexing with revision ordering and tombstones, and a permission-aware query API over SQLite FTS5.",
++ "description": "OpenVibe.Search: the canonical index document, event-fed and direct indexing with revision ordering and tombstones, and a permission-aware query API over PostgreSQL full-text search.",
"main": "server/index.js",
"scripts": {
"start": "node server/index.js",
@@@ -14,7 +14,7 @@
"keywords": [
"openvibe",
"search",
-- "fts5",
++ "full-text",
"acl"
],
"author": "OpenVibers",
@@@ -23,7 +23,6 @@
"dependencies": {
"ajv": "^8.17.1",
"ajv-formats": "^3.0.1",
-- "better-sqlite3": "^11.10.0",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",
ddiff --git a/scripts/migrate-to-postgres.js b/scripts/migrate-to-postgres.js
ddeleted file mode 100644
iindex dc9803b..0000000
---- a/scripts/migrate-to-postgres.js
++++ /dev/null
@@@ -1,26 +0,0 @@
--#!/usr/bin/env node
--'use strict';
--/**
-- * The one-time move of Search's SQLite database (SEARCH_DB_PATH) into its PostgreSQL schema (ADR-035; the procedure is
-- * openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by openvibe-sdk/db runSqliteMigration).
-- *
-- * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]
-- *
-- * Applies migrations/ as the owner (DATABASE_DIRECT_URL), copies every table into emptied tables, verifies row counts
-- * and checksums, and exits 1 unless everything verified. The SQLite file is opened read-only. Every table keeps its
-- * name and columns.
-- */
--require('dotenv').config();
--const { runSqliteMigration } = require('openvibe-sdk/db');
--const configLib = require('../server/config');
--const { MIGRATIONS } = require('../server/db');
--
--const TABLES = {};
--
--if (require.main === module) {
-- const config = configLib.load();
-- runSqliteMigration({ service: 'search', sqlite: config.dbPath, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES })
-- .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });
--}
--
--module.exports = { TABLES };
ddiff --git a/server/api/webhook.js b/server/api/webhook.js
iindex 4f858fd..6f7ce89 100644
---- a/server/api/webhook.js
++++ b/server/api/webhook.js
@@@ -9,7 +9,7 @@
* with envelope subject { type, id, revision } and visibility "internal".
*
* Exactly once: the inbox receipt (consumer, event_id) and the document change commit in one
-- * SQLite transaction. A redelivery of a processed event is answered 204 and changes nothing.
++ * database transaction. A redelivery of a processed event is answered 204 and changes nothing.
* Revision order is the store's rule, so out-of-order delivery is harmless.
*
* Refusals (unknown owner, a source writing another owner's document, a bad document, a subject
ddiff --git a/server/config.js b/server/config.js
iindex e64f299..9c2ea28 100644
---- a/server/config.js
++++ b/server/config.js
@@@ -72,8 +72,6 @@ function load(env = process.env) {
// PostgreSQL (ADR-035): DATABASE_URL serves (PgBouncer), DATABASE_DIRECT_URL migrates (owner role).
db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '' },
-- // The SQLite file of releases before the switch: read once by scripts/migrate-to-postgres.js.
-- dbPath: env.SEARCH_DB_PATH || './data/search.db',
// Events: the outbox relays only when EVENTS_URL is set (rows wait otherwise).
events: {
ddiff --git a/server/engine/pg.js b/server/engine/pg.js
iindex 7c45c0d..1c310a7 100644
---- a/server/engine/pg.js
++++ b/server/engine/pg.js
@@@ -1,7 +1,7 @@
'use strict';
/**
-- * Search engine: PostgreSQL full-text search in the service's own database (ADR-035; it replaced SQLite FTS5, the
-- * engine docs/adr-engine.md chose, with the same interface and the same guarantees).
++ * Search engine: PostgreSQL full-text search in the service's own database (ADR-035; the engine docs/adr-engine.md
++ * chose, with the same interface and the same guarantees).
*
* One table, search_fts, split by audience so an anonymous query never touches, ranks against or counts anything but
* public listed documents:
@@@ -13,15 +13,16 @@
* also filtered by `documents.exposure` and, for restricted rows, by the viewer's ACL keys (doc_acl) in the same SQL:
* visibility is decided before anything is ranked, counted, snippeted or suggested.
*
-- * Text is folded before it is indexed or queried (NFKD, combining marks removed: FTS5's remove_diacritics), split into
-- * runs of letters and digits (FTS5's unicode61) and tokenized by the `simple` configuration (lower case, no stemming, no stop words: FTS5's unicode61). Title, summary and
-- * body weigh 10, 4 and 1 (tsvector weights A, B, C). Every function here is async; put and remove run inside the store's
++ * Text is folded before it is indexed or queried (NFKD, combining marks removed), split into runs of letters and
++ * digits and tokenized by the `simple` configuration (lower case, no stemming, no stop words): the index and every
++ * query see the same text. Title, summary and body weigh 10, 4 and 1 (tsvector weights A, B, C), with the freshness
++ * boost described below. Every function here is async; put and remove run inside the store's
* transaction (ambient). Nothing outside this file writes SQL against search_fts.
*/
const { EXPOSURE } = require('../document');
const CONFIG = 'simple';
--const WEIGHTS = '{0.1, 0.1, 0.4, 1.0}'; // D, C body, B summary, A title: FTS5's bm25 weights 10 / 4 / 1
++const WEIGHTS = '{0.1, 0.1, 0.4, 1.0}'; // D, C body, B summary, A title: ts_rank weights 1 / 4 / 10
const MARK_OPEN = '\u0001';
const MARK_CLOSE = '\u0002';
const HEADLINE = `StartSel=${MARK_OPEN}, StopSel=${MARK_CLOSE}, MaxWords=16, MinWords=8, ShortWord=0, MaxFragments=1, FragmentDelimiter=" … "`;
@@@ -47,15 +48,15 @@ function freshnessBoost(iso, nowMs, halfLifeDays, weight) {
return 1 + weight * Math.pow(2, -ageDays / halfLifeDays);
}
--/** Diacritics folded, as FTS5's remove_diacritics did: the index and every query see the same text. */
++/** Diacritics folded, so the index and every query see the same text. */
function fold(s) {
return String(s || '').normalize('NFKD').replace(/\p{M}+/gu, '').normalize('NFC');
}
/**
-- * The words FTS5's unicode61 tokenizer saw: runs of letters and digits, everything else a separator. PostgreSQL's
-- * parser would keep "OpenVibe.Live", "a_b" or "3.5" as single tokens (host, word, number); separating them first makes
-- * "openvibe" find "OpenVibe.Live" as it did.
++ * Word tokens: runs of letters and digits, everything else a separator. PostgreSQL's parser would
++ * keep "OpenVibe.Live", "a_b" or "3.5" as single tokens (host, word, number); separating them first
++ * makes "openvibe" find "OpenVibe.Live".
*/
function words(s) {
return fold(s).replace(/[^\p{L}\p{N}]+/gu, ' ');
@@@ -124,7 +125,7 @@ function createEngine(db, { freshness = { weight: 1, halfLifeDays: 30 } } = {})
/**
* The visible, matching set as (rid, rank) rows. match = toMatch()'s query. rank is minus ts_rank times the
-- * freshness boost at reference time `now` (ms), so lower ranks first (as FTS5's bm25 did) and pages of one query
++ * freshness boost at reference time `now` (ms), so lower ranks first and pages of one query
* (which carry their first page's `now` in the cursor) rank against the same clock.
*/
function matchedSet({ match, filters, viewer, now = Date.now() }, params) {
@@@ -234,8 +235,8 @@ function createEngine(db, { freshness = { weight: 1, halfLifeDays: 30 } } = {})
}
/**
-- * Rebuild the index from documents.doc when its counts disagree with the documents table (after the one-time import
-- * from SQLite, whose FTS5 tables are not copied). Returns what it did.
++ * Rebuild the index from documents.doc when its counts disagree with the documents table (for instance after a
++ * restore). Returns what it did.
*/
async function reconcile() {
const docs = await db.prepare(`SELECT
ddiff --git a/server/events/outbox.js b/server/events/outbox.js
iindex 7b3943c..26729a8 100644
---- a/server/events/outbox.js
++++ b/server/events/outbox.js
@@@ -16,22 +16,6 @@ const { ids, validate, serviceAuth } = require('openvibe-contracts');
const BACKOFF_MS = [1000, 5000, 30000, 120000, 600000];
--function ensureSchema(db) {
-- db.exec(`CREATE TABLE IF NOT EXISTS event_outbox (
-- seq INTEGER PRIMARY KEY AUTOINCREMENT,
-- event_id TEXT NOT NULL UNIQUE,
-- event_type TEXT NOT NULL,
-- envelope TEXT NOT NULL,
-- created_at INTEGER NOT NULL,
-- attempts INTEGER NOT NULL DEFAULT 0,
-- next_attempt_at INTEGER NOT NULL DEFAULT 0,
-- sent_at INTEGER,
-- rejected_at INTEGER,
-- last_error TEXT
-- );
-- CREATE INDEX IF NOT EXISTS idx_event_outbox_due ON event_outbox(sent_at, rejected_at, next_attempt_at);`);
--}
--
function createOutbox(db, { source, now = () => Date.now() }) {
const insert = db.prepare('INSERT INTO event_outbox (event_id, event_type, envelope, created_at) VALUES (?, ?, ?, ?)');
@@@ -176,4 +160,4 @@ function createRelay({ outbox, eventsUrl, intervalMs = 2000, tokenClient, tokenO
return { start, stop, flush, running: () => Boolean(timer) };
}
--module.exports = { ensureSchema, createOutbox, createRelay };
++module.exports = { createOutbox, createRelay };
ddiff --git a/server/index.js b/server/index.js
iindex 9c14b2f..3f6b018 100644
---- a/server/index.js
++++ b/server/index.js
@@@ -22,7 +22,7 @@ async function start({ config, db: givenDb = null, now = () => Date.now(), fetch
// PostgreSQL (ADR-035): opened and migrated here unless the caller (a test) hands in a migrated handle.
const db = givenDb || await openDb(config, { log });
const engine = createEngine(db, { freshness: config.freshness });
-- // The index is derived from documents: rebuilt when it disagrees (after the one-time import from SQLite).
++ // The index is derived from documents: rebuilt at boot when it disagrees with the documents table.
const rebuilt = await engine.reconcile();
if (rebuilt.rebuilt) log.log(`[search] full-text index rebuilt: ${rebuilt.rebuilt} document(s)`);
const outbox = createOutbox(db, { source: config.serviceId, now });
ddiff --git a/server/observability.js b/server/observability.js
iindex 0d9ce3e..2832efa 100644
---- a/server/observability.js
++++ b/server/observability.js
@@@ -3,16 +3,16 @@
* Track O: truthful readiness for GET /api/ready and the Search gauges on GET /metrics
* (openvibe-shared/ready and openvibe-shared/metrics).
*
-- * db required a real read of the documents table and of both full-text tables (they
-- * live in the same SQLite file): without them nothing can be indexed or found
++ * db required a real read of the documents table and of the full-text table (they
++ * live in the same database): without them nothing can be indexed or found
* network_jwks optional the Network signing key has loaded. Without it anonymous queries still
* answer (public documents only), but no token can be verified: owner
* writes and signed-in queries fail, so it degrades rather than fails
-- * index optional the full-text index agrees with the documents table: fts_public holds
-- * exactly the public_listed documents and fts_restricted the restricted
-- * ones. A mismatch means queries miss (or would surface) documents
++ * index optional the full-text index agrees with the documents table: search_fts holds
++ * exactly the public_listed and restricted documents. A mismatch means
++ * queries miss (or would surface) documents
*
-- * Gauges: documents by exposure, full-text entries by table, and the events outbox backlog.
++ * Gauges: documents by exposure, full-text entries by audience, and the events outbox backlog.
*/
const { createReadiness } = require('openvibe-shared/ready');
const { EXPOSURE } = require('./document');
@@@ -90,7 +90,7 @@ function registerSearchGauges(registry, { db, engine, outbox, purges = null }) {
collect: async () => Object.entries(await read.documents()).map(([exposure, value]) => ({ labels: { exposure }, value })),
});
registry.gauge({
-- name: 'search_documents_indexed', help: 'Documents in the full-text index, by table', labelNames: ['index'],
++ name: 'search_documents_indexed', help: 'Documents in the full-text index, by audience', labelNames: ['index'],
collect: async () => Object.entries(await read.indexed()).map(([index, value]) => ({ labels: { index }, value })),
});
registry.gauge({ name: 'search_outbox_pending', help: 'Events waiting in the outbox', collect: async () => await outbox.pending() });
ddiff --git a/server/purge.js b/server/purge.js
iindex 8ac6155..a2ea55f 100644
---- a/server/purge.js
++++ b/server/purge.js
@@@ -25,42 +25,6 @@ const MAX_ATTEMPTS = 8;
const FILES_PER_CALL = 30;
const DETAIL_MAX = 300;
--function ensureSchema(db) {
-- db.exec(`
-- CREATE TABLE IF NOT EXISTS removals (
-- seq INTEGER PRIMARY KEY AUTOINCREMENT,
-- event_id TEXT NOT NULL, -- the search.document.removed event
-- owner TEXT NOT NULL,
-- type TEXT NOT NULL,
-- id TEXT NOT NULL,
-- revision INTEGER NOT NULL,
-- reason TEXT NOT NULL,
-- previous_exposure TEXT NOT NULL,
-- exposure TEXT NOT NULL,
-- canonical_url TEXT, -- only a URL that had been public
-- at INTEGER NOT NULL
-- );
-- CREATE INDEX IF NOT EXISTS idx_removals_owner ON removals (owner, seq);
--
-- CREATE TABLE IF NOT EXISTS cdn_purges (
-- seq INTEGER PRIMARY KEY AUTOINCREMENT,
-- removal_seq INTEGER NOT NULL,
-- provider TEXT NOT NULL DEFAULT 'cloudflare',
-- zone_id TEXT,
-- url TEXT NOT NULL,
-- state TEXT NOT NULL CHECK (state IN ('pending', 'purged', 'failed', 'skipped')),
-- detail TEXT,
-- attempts INTEGER NOT NULL DEFAULT 0,
-- next_attempt_at INTEGER NOT NULL DEFAULT 0,
-- created_at INTEGER NOT NULL,
-- done_at INTEGER
-- );
-- -- One pending purge per URL: a sitemap shared by many removals is purged once per batch.
-- CREATE UNIQUE INDEX IF NOT EXISTS idx_cdn_purges_pending ON cdn_purges (zone_id, url) WHERE state = 'pending';
-- CREATE INDEX IF NOT EXISTS idx_cdn_purges_due ON cdn_purges (state, next_attempt_at, seq);
-- `);
--}
--
/** The zone a URL belongs to: exact host or a parent domain in the map, longest first. */
function zoneFor(url, zones) {
let u;
@@@ -254,4 +218,4 @@ function createPurger({ db, config, fetchImpl = globalThis.fetch, log = console,
return { start, stop, flush, running: () => Boolean(timer) };
}
--module.exports = { ensureSchema, createPurgeQueue, createPurger, zoneFor, purgeUrls, MAX_ATTEMPTS, FILES_PER_CALL };
++module.exports = { createPurgeQueue, createPurger, zoneFor, purgeUrls, MAX_ATTEMPTS, FILES_PER_CALL };
ddiff --git a/server/saved.js b/server/saved.js
iindex fbb346e..0a249a5 100644
---- a/server/saved.js
++++ b/server/saved.js
@@@ -13,24 +13,6 @@ const { ids } = require('openvibe-contracts');
const ID_RE = /^svs_[0-9a-z]{26}$/;
--function ensureSchema(db) {
-- db.exec(`
-- CREATE TABLE IF NOT EXISTS saved_searches (
-- id TEXT PRIMARY KEY,
-- subject TEXT NOT NULL, -- usr_… (the person who saved it)
-- name TEXT NOT NULL,
-- q TEXT NOT NULL DEFAULT '',
-- filters TEXT NOT NULL, -- canonical JSON {owner,type,language,facets}
-- query_hash TEXT NOT NULL, -- same query saved twice = one row
-- created_at INTEGER NOT NULL,
-- updated_at INTEGER NOT NULL,
-- last_run_at INTEGER,
-- UNIQUE (subject, query_hash)
-- );
-- CREATE INDEX IF NOT EXISTS idx_saved_searches_subject ON saved_searches (subject, created_at);
-- `);
--}
--
/** Filters in one canonical shape, so equal queries hash equally whatever order they came in. */
function canonicalFilters(f) {
const out = {};
@@@ -111,4 +93,4 @@ function createSavedSearches(db, { maxPerSubject = 50, now = () => Date.now() }
};
}
--module.exports = { ensureSchema, createSavedSearches, canonicalFilters, ID_RE };
++module.exports = { createSavedSearches, canonicalFilters, ID_RE };
ddiff --git a/test/helpers.js b/test/helpers.js
iindex 5721a78..4a04818 100644
---- a/test/helpers.js
++++ b/test/helpers.js
@@@ -5,9 +5,6 @@
* Events delivery helper. Nothing here needs the network or a running OpenVibe.Network.
*/
const crypto = require('crypto');
--const fs = require('fs');
--const os = require('os');
--const path = require('path');
const { serviceAuth, ids } = require('openvibe-contracts');
const { load } = require('../server/config');
const { start } = require('../server/index');
@@@ -37,21 +34,10 @@ function userToken({ subjectId = ids.newId('user'), role = 'user', aud = ['openv
}, key);
}
--const made = [];
--process.on('exit', () => { for (const d of made) fs.rmSync(d, { recursive: true, force: true }); });
--
--function tmpDir() {
-- const d = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-search-test-'));
-- made.push(d);
-- return d;
--}
--
async function boot({ env = {}, fetchImpl, tokenClient, now } = {}) {
-- const dir = tmpDir();
const config = load({
NODE_ENV: 'test',
PORT: '0',
-- SEARCH_DB_PATH: path.join(dir, 'search.db'),
OV_NETWORK_PUBLIC_KEY: publicKey,
SEARCH_EVENTS_SECRET: WEBHOOK_SECRET,
...env,
@@@ -60,7 +46,7 @@ async function boot({ env = {}, fetchImpl, tokenClient, now } = {}) {
const testdb = await require('./db').testDb();
const h = await start({ config, db: testdb.db, log: silent, fetchImpl, tokenClient, now });
const base = `http://127.0.0.1:${h.server.address().port}`;
-- return { ...h, base, dir, async stop() { await h.close(); await testdb.close(); } };
++ return { ...h, base, async stop() { await h.close(); await testdb.close(); } };
}
async function request(base, method, p, { token, body, headers = {}, raw } = {}) {
@@@ -145,6 +131,6 @@ function suite(name) {
}
module.exports = {
-- ISSUER, WEBHOOK_SECRET, privateKey, publicKey, silent, serviceToken, userToken, tmpDir, boot, request,
++ ISSUER, WEBHOOK_SECRET, privateKey, publicKey, silent, serviceToken, userToken, boot, request,
doc, indexEvent, deliver, suite,
};
ddiff --git a/test/run.js b/test/run.js
iindex cb17beb..7d40967 100644
---- a/test/run.js
++++ b/test/run.js
@@@ -1,8 +1,9 @@
#!/usr/bin/env node
/**
* Runs every test in test/ — the files named *.test.js — each in its own process, and fails
-- * if any of them fails. They use temp SQLite databases, generated RSA keys and stub subscribers
-- * on random ports; none of them needs the network or a running OpenVibe.Network.
++ * if any of them fails. They boot a PGlite database per test file (PostgreSQL with
++ * SEARCH_TEST_STORE=pg), with generated RSA keys and stub subscribers on random ports; none of
++ * them needs the network or a running OpenVibe.Network.
*
* npm test # everything
* npm test -- publish sse # only files whose name contains one of the words