openvibers/openvibe.contracts Commits
ADR-035 (accepted, owner decision 2026-09-28): PostgreSQL and Valkey as the data tier. PostgreSQL 18 becomes every service's system of record, one cluster in the data role (on this host until a data server exists): a database, an owner role and a pooled application role per service, PgBouncer transaction pooling, loopback and private-network only, scram-sha-256. pgBackRest keeps 14 days of point-in-time recovery in an encrypted B2 repository, with monthly restore drills. Valkey 9 (BSD, chosen over Redis for its licence) is the shared, never-authoritative store for caches, per-actor limits, queues, pub/sub, presence and sessions: an ACL user per service on its own key prefix, volatile-lru eviction so only TTL keys go, AOF. The SDK gains an async data layer with PostgreSQL and SQLite adapters, and cache, queue and pubsub helpers. New services start on it; existing services migrate one at a time, smallest first (Wiki first, Live last): data access onto the layer while still on SQLite, then a verified copy, then the switch, with SQLite kept read-only for the 7-day N-1 window. Supersedes ADR-007's 2026-09-26 trigger rule; the ownership rule and the 2026-09-24 runtime posture (pooling, replicas, SERIALIZABLE money paths, non-authoritative cache) still bind. ADR-007's status line says so. npm test passes.
Changes
@@@ -1,6 +1,6 @@# ADR-007: Logical data ownership and database technology--**Status:** Accepted 2026-09-22. Amended 2026-09-24: PostgreSQL runtime posture (pooling, replicas, money-path isolation, Redis non-authoritative; roadmap §28.3 m10).++**Status:** Accepted 2026-09-22. Superseded in part 2026-09-28 by ADR-035 (PostgreSQL and Valkey now, not on a trigger; the ownership rule and the 2026-09-24 runtime posture still hold). Amended 2026-09-24: PostgreSQL runtime posture (pooling, replicas, money-path isolation, Redis non-authoritative; roadmap §28.3 m10).## Context and current evidencennew file mode 100644@@@ -0,0 +1,86 @@++# ADR-035: PostgreSQL and Valkey as the data tier++++**Status:** Accepted 2026-09-28, the owner's decision ("we should just get it over with now … so we don't waste time later"). Supersedes the "a service moves only on a measured trigger" rule of ADR-007's 2026-09-26 amendment. ADR-007's ownership rule and its 2026-09-24 runtime posture stay in force and bind every step below. Roadmap workstream WS-X2, requirement D48.++++## Context and current evidence++++- Every service runs one SQLite database (better-sqlite3, synchronous) on one host. Load is far below SQLite's limits (ADR-007 amendment 2026-09-26). The owner has decided that the platform should be built for many hosts and much larger scale now (ADR-034; roadmap section 4B, decisions 1 and 2), not migrated later under pressure.++- Several things need a shared, networked store before a second host exists, because today they only work in one process:++ - per-actor limits;++ - realtime fan-out;++ - session caches;++ - job queues.++- The host runs Ubuntu 26.04 with 4 vCPUs and 15 GB RAM. Its distribution provides PostgreSQL 18, PgBouncer 1.25, pgBackRest 2.58 and Valkey 9.0.++- Two projects joining the network already use PostgreSQL and Redis-compatible stores: OpenVibe: Source (PostgreSQL and Redis) and GameServerLists (TimescaleDB and Redis).++++## Decision++++1. **PostgreSQL is the system of record for every service.** One cluster serves the "data" role (ADR-034 section 12; roadmap WS-X1). It runs on the current host until a data server is bought, when the role moves by inventory.++ - **Per service:** a database, an owner role (migrations, direct connections), and an application role that connects through PgBouncer in transaction-pooling mode (ADR-007, 2026-09-24, rule 1). No service connects to another's database.++ - **Access:** listening only on loopback and the private network; `scram-sha-256` authentication; TLS on the private network. Passwords go into each service's env file, never into logs.++ - **Money paths** stay `SERIALIZABLE` with row locks and retries (rule 3). Reads that may lag go to replicas when replicas exist (rule 2).++2. **Valkey is the shared, non-authoritative store** (rule 4 applies; Valkey replaces the "Redis" of rule 4):++ - uses: caches, per-actor limit counters, job queues (streams), pub/sub fan-out, presence and session caches;++ - one ACL user per service, restricted to its key prefix `ov:<service>:*`;++ - `maxmemory-policy volatile-lru`, so only keys with a TTL (caches) are evicted, never queues or counters without a TTL; append-only persistence every second;++ - losing Valkey loses no write and changes no answer about money, accounts, grants or bans.++ - Valkey (BSD-licensed, Linux Foundation) is chosen over Redis because of its licence.++3. **Backups.**++ - pgBackRest archives WAL continuously and takes a full backup weekly and a differential daily to an encrypted repository in B2, keeping 14 days of point-in-time recovery. `ovhost backup` and `ovhost drill` cover it, and a monthly drill restores a service's database to a point in time.++ - Valkey needs no disaster recovery, because nothing in it is authoritative.++4. **One async data layer in the SDK** (`openvibe-sdk/db`), with PostgreSQL and SQLite adapters behind the same interface:++ - tagged-template queries with parameters;++ - transactions, with bounded retry on serialization failure;++ - statement timeouts and tracing (`traceparent`);++ - versioned migrations, each labelled expand, migrate or contract (ADR-028);++ - a readiness check that reports the store in use.++++ **`openvibe-sdk/cache`, `/queue` and `/pubsub`** wrap Valkey the same way.++5. **New services start on PostgreSQL and Valkey.** Every product in roadmap section 4B does.++6. **Existing services migrate one at a time, smallest first**, following expand/migrate/contract:++ 1. Move the service's data access onto `openvibe-sdk/db` while it still runs on SQLite. The suite must pass unchanged.++ 2. Create the PostgreSQL schema, then copy with verification: row counts and per-table checksums, under a short write freeze or dual writes.++ 3. Switch reads and writes to PostgreSQL, keeping the SQLite file read-only for the N-1 window (ADR-016, 7 days) as the rollback.++ 4. Then delete the SQLite file after a final backup.++++ Order: Wiki, Blog, News, Reviews, Deals, Coupons, Trade, Tips, VIP, Search, Sources, Codes, Host, AI, Events, Chat, Community, Billing, Media, Tools, Games, OpenRe, Network, Live. Each migration is its own change, with its own rehearsal on a copy of the production data.++7. **Sizing on the shared host**, until the data role has its own machine:++ - PostgreSQL: `shared_buffers` 1 GB, `effective_cache_size` 4 GB, `work_mem` 16 MB, `max_connections` 120, pooled behind PgBouncer;++ - Valkey: `maxmemory` 768 MB.++++ These are revisited when the role moves.++8. **Observability.** Exporters for PostgreSQL, PgBouncer and Valkey feed Prometheus. Alerts cover:++ - connections near the limit;++ - WAL archive failure;++ - backup age over 26 hours;++ - replication lag, once replicas exist;++ - Valkey memory over 90%.++++## Alternatives considered++++- **Stay on SQLite until a trigger** (ADR-007, 2026-09-26): rejected by the owner, to avoid a larger migration later.++- **Redis:** its licence moved from BSD to RSAL/SSPL in 7.4, and to AGPL in 8. Valkey stays BSD and is compatible.++- **Managed databases:** cost and lock-in against an "open, affordable" platform. They are possible later for the control plane.++- **Distributed SQL (CockroachDB, YugabyteDB):** more operational weight than the workload needs. The cell design (ADR-034) scales out with ordinary PostgreSQL clusters.++- **One shared schema:** rejected by ADR-007 (one writer per dataset).++++## Migration consequences++++- Synchronous `better-sqlite3` call sites become async. That is the bulk of each service's migration, and step 1 of item 6 isolates it from the storage change.++- Session-level PostgreSQL features cannot be used through the pooler (ADR-007 2026-09-24 rule 1).++- Per-process state (limits, fan-out, caches) moves to Valkey. That is also what makes services able to run more than one process.++++## Rollback++++- **Per service, within its N-1 window:** point it back at its read-only SQLite file after replaying writes made since the switch. The migration keeps a change log for that window.++- **The data tier itself:** each service keeps working on SQLite until its own migration switches, so the tier can be removed before any switch without effect.++++## Acceptance tests++++- **ADR-007's first-move tests:**++ - the suite passes on PostgreSQL behind PgBouncer in transaction mode;++ - a concurrent-debit race lets exactly one debit through;++ - a lagging read is never used for a decision;++ - flushing Valkey mid-run changes no money or account answer.++- **A point-in-time restore drill** from pgBackRest reproduces a service's database at a chosen time.++- **Truthful readiness:** each migrated service's `/ready` reports `postgresql` and its pool state, and a service not yet migrated reports `sqlite`.@@@ -38,3 +38,4 @@ Platform-wide decisions from the OpenVibe development roadmap (section 10). Each| [ADR-032](ADR-032-containers.md) | Containers: platform services stay systemd units (one adapter in ovhost, a reserved `runtime` field); tenant code isolation belongs to Stage C | Accepted 2026-09-26 || [ADR-033](ADR-033-account-export-and-deletion.md) | Account export and deletion: services push their export parts to Network; deletion after a 30-day grace, each service erases its rows and confirms; money rows kept pseudonymised | Accepted 2026-09-27; implementation in progress || [ADR-034](ADR-034-platform-north-star.md) | Platform north star, an open affordable cloud: boundaries now (projects, `ovrn:` resource names, scoped grants, generated contracts, metering), infrastructure when measured; control/data plane and cells; regions, edge nodes, load balancing and a geo API; pulled, gated deploys; four extension levels under one manifest | Proposed 2026-09-28, for the owner's review |++| [ADR-035](ADR-035-postgresql-and-valkey.md) | PostgreSQL 18 as every service's system of record (database + roles per service, PgBouncer transaction pooling, pgBackRest PITR to B2) and Valkey as the shared non-authoritative store; the SDK data layer; services migrate one at a time, smallest first; supersedes ADR-007's trigger rule | Accepted 2026-09-28 (owner decision) |
@@ -1,6 +1,6 @@
@@ -1,6 +1,6 @@
# ADR-007: Logical data ownership and database technology
# ADR-007: Logical data ownership and database technology
-**Status:** Accepted 2026-09-22. Amended 2026-09-24: PostgreSQL runtime posture (pooling, replicas, money-path isolation, Redis non-authoritative; roadmap §28.3 m10).
+**Status:** Accepted 2026-09-22. Superseded in part 2026-09-28 by ADR-035 (PostgreSQL and Valkey now, not on a trigger; the ownership rule and the 2026-09-24 runtime posture still hold). Amended 2026-09-24: PostgreSQL runtime posture (pooling, replicas, money-path isolation, Redis non-authoritative; roadmap §28.3 m10).
## Context and current evidence
## Context and current evidence
new file mode 100644
new file mode 100644
@@ -0,0 +1,86 @@
@@ -0,0 +1,86 @@
+# ADR-035: PostgreSQL and Valkey as the data tier
+
+**Status:** Accepted 2026-09-28, the owner's decision ("we should just get it over with now … so we don't waste time later"). Supersedes the "a service moves only on a measured trigger" rule of ADR-007's 2026-09-26 amendment. ADR-007's ownership rule and its 2026-09-24 runtime posture stay in force and bind every step below. Roadmap workstream WS-X2, requirement D48.
+
+## Context and current evidence
+
+- Every service runs one SQLite database (better-sqlite3, synchronous) on one host. Load is far below SQLite's limits (ADR-007 amendment 2026-09-26). The owner has decided that the platform should be built for many hosts and much larger scale now (ADR-034; roadmap section 4B, decisions 1 and 2), not migrated later under pressure.
+- Several things need a shared, networked store before a second host exists, because today they only work in one process:
+ - per-actor limits;
+ - realtime fan-out;
+ - session caches;
+ - job queues.
+- The host runs Ubuntu 26.04 with 4 vCPUs and 15 GB RAM. Its distribution provides PostgreSQL 18, PgBouncer 1.25, pgBackRest 2.58 and Valkey 9.0.
+- Two projects joining the network already use PostgreSQL and Redis-compatible stores: OpenVibe: Source (PostgreSQL and Redis) and GameServerLists (TimescaleDB and Redis).
+
+## Decision
+
+1. **PostgreSQL is the system of record for every service.** One cluster serves the "data" role (ADR-034 section 12; roadmap WS-X1). It runs on the current host until a data server is bought, when the role moves by inventory.
+ - **Per service:** a database, an owner role (migrations, direct connections), and an application role that connects through PgBouncer in transaction-pooling mode (ADR-007, 2026-09-24, rule 1). No service connects to another's database.
+ - **Access:** listening only on loopback and the private network; `scram-sha-256` authentication; TLS on the private network. Passwords go into each service's env file, never into logs.
+ - **Money paths** stay `SERIALIZABLE` with row locks and retries (rule 3). Reads that may lag go to replicas when replicas exist (rule 2).
+2. **Valkey is the shared, non-authoritative store** (rule 4 applies; Valkey replaces the "Redis" of rule 4):
+ - uses: caches, per-actor limit counters, job queues (streams), pub/sub fan-out, presence and session caches;
+ - one ACL user per service, restricted to its key prefix `ov:<service>:*`;
+ - `maxmemory-policy volatile-lru`, so only keys with a TTL (caches) are evicted, never queues or counters without a TTL; append-only persistence every second;
+ - losing Valkey loses no write and changes no answer about money, accounts, grants or bans.
+ - Valkey (BSD-licensed, Linux Foundation) is chosen over Redis because of its licence.
+3. **Backups.**
+ - pgBackRest archives WAL continuously and takes a full backup weekly and a differential daily to an encrypted repository in B2, keeping 14 days of point-in-time recovery. `ovhost backup` and `ovhost drill` cover it, and a monthly drill restores a service's database to a point in time.
+ - Valkey needs no disaster recovery, because nothing in it is authoritative.
+4. **One async data layer in the SDK** (`openvibe-sdk/db`), with PostgreSQL and SQLite adapters behind the same interface:
+ - tagged-template queries with parameters;
+ - transactions, with bounded retry on serialization failure;
+ - statement timeouts and tracing (`traceparent`);
+ - versioned migrations, each labelled expand, migrate or contract (ADR-028);
+ - a readiness check that reports the store in use.
+
+ **`openvibe-sdk/cache`, `/queue` and `/pubsub`** wrap Valkey the same way.
+5. **New services start on PostgreSQL and Valkey.** Every product in roadmap section 4B does.
+6. **Existing services migrate one at a time, smallest first**, following expand/migrate/contract:
+ 1. Move the service's data access onto `openvibe-sdk/db` while it still runs on SQLite. The suite must pass unchanged.
+ 2. Create the PostgreSQL schema, then copy with verification: row counts and per-table checksums, under a short write freeze or dual writes.
+ 3. Switch reads and writes to PostgreSQL, keeping the SQLite file read-only for the N-1 window (ADR-016, 7 days) as the rollback.
+ 4. Then delete the SQLite file after a final backup.
+
+ Order: Wiki, Blog, News, Reviews, Deals, Coupons, Trade, Tips, VIP, Search, Sources, Codes, Host, AI, Events, Chat, Community, Billing, Media, Tools, Games, OpenRe, Network, Live. Each migration is its own change, with its own rehearsal on a copy of the production data.
+7. **Sizing on the shared host**, until the data role has its own machine:
+ - PostgreSQL: `shared_buffers` 1 GB, `effective_cache_size` 4 GB, `work_mem` 16 MB, `max_connections` 120, pooled behind PgBouncer;
+ - Valkey: `maxmemory` 768 MB.
+
+ These are revisited when the role moves.
+8. **Observability.** Exporters for PostgreSQL, PgBouncer and Valkey feed Prometheus. Alerts cover:
+ - connections near the limit;
+ - WAL archive failure;
+ - backup age over 26 hours;
+ - replication lag, once replicas exist;
+ - Valkey memory over 90%.
+
+## Alternatives considered
+
+- **Stay on SQLite until a trigger** (ADR-007, 2026-09-26): rejected by the owner, to avoid a larger migration later.
+- **Redis:** its licence moved from BSD to RSAL/SSPL in 7.4, and to AGPL in 8. Valkey stays BSD and is compatible.
+- **Managed databases:** cost and lock-in against an "open, affordable" platform. They are possible later for the control plane.
+- **Distributed SQL (CockroachDB, YugabyteDB):** more operational weight than the workload needs. The cell design (ADR-034) scales out with ordinary PostgreSQL clusters.
+- **One shared schema:** rejected by ADR-007 (one writer per dataset).
+
+## Migration consequences
+
+- Synchronous `better-sqlite3` call sites become async. That is the bulk of each service's migration, and step 1 of item 6 isolates it from the storage change.
+- Session-level PostgreSQL features cannot be used through the pooler (ADR-007 2026-09-24 rule 1).
+- Per-process state (limits, fan-out, caches) moves to Valkey. That is also what makes services able to run more than one process.
+
+## Rollback
+
+- **Per service, within its N-1 window:** point it back at its read-only SQLite file after replaying writes made since the switch. The migration keeps a change log for that window.
+- **The data tier itself:** each service keeps working on SQLite until its own migration switches, so the tier can be removed before any switch without effect.
+
+## Acceptance tests
+
+- **ADR-007's first-move tests:**
+ - the suite passes on PostgreSQL behind PgBouncer in transaction mode;
+ - a concurrent-debit race lets exactly one debit through;
+ - a lagging read is never used for a decision;
+ - flushing Valkey mid-run changes no money or account answer.
+- **A point-in-time restore drill** from pgBackRest reproduces a service's database at a chosen time.
+- **Truthful readiness:** each migrated service's `/ready` reports `postgresql` and its pool state, and a service not yet migrated reports `sqlite`.
@@ -38,3 +38,4 @@ Platform-wide decisions from the OpenVibe development roadmap (section 10). Each
@@ -38,3 +38,4 @@ Platform-wide decisions from the OpenVibe development roadmap (section 10). Each
| [ADR-032](ADR-032-containers.md) | Containers: platform services stay systemd units (one adapter in ovhost, a reserved `runtime` field); tenant code isolation belongs to Stage C | Accepted 2026-09-26 |
| [ADR-032](ADR-032-containers.md) | Containers: platform services stay systemd units (one adapter in ovhost, a reserved `runtime` field); tenant code isolation belongs to Stage C | Accepted 2026-09-26 |
| [ADR-033](ADR-033-account-export-and-deletion.md) | Account export and deletion: services push their export parts to Network; deletion after a 30-day grace, each service erases its rows and confirms; money rows kept pseudonymised | Accepted 2026-09-27; implementation in progress |
| [ADR-033](ADR-033-account-export-and-deletion.md) | Account export and deletion: services push their export parts to Network; deletion after a 30-day grace, each service erases its rows and confirms; money rows kept pseudonymised | Accepted 2026-09-27; implementation in progress |
| [ADR-034](ADR-034-platform-north-star.md) | Platform north star, an open affordable cloud: boundaries now (projects, `ovrn:` resource names, scoped grants, generated contracts, metering), infrastructure when measured; control/data plane and cells; regions, edge nodes, load balancing and a geo API; pulled, gated deploys; four extension levels under one manifest | Proposed 2026-09-28, for the owner's review |
| [ADR-034](ADR-034-platform-north-star.md) | Platform north star, an open affordable cloud: boundaries now (projects, `ovrn:` resource names, scoped grants, generated contracts, metering), infrastructure when measured; control/data plane and cells; regions, edge nodes, load balancing and a geo API; pulled, gated deploys; four extension levels under one manifest | Proposed 2026-09-28, for the owner's review |
+| [ADR-035](ADR-035-postgresql-and-valkey.md) | PostgreSQL 18 as every service's system of record (database + roles per service, PgBouncer transaction pooling, pgBackRest PITR to B2) and Valkey as the shared non-authoritative store; the SDK data layer; services migrate one at a time, smallest first; supersedes ADR-007's trigger rule | Accepted 2026-09-28 (owner decision) |