openvibers/openvibe.host Commits
Host Stage B moves to PostgreSQL and Valkey (ADR-035, roadmap WS-X2, repository-pass group G2), async from end to end.
Changes
@@@ -20,6 +20,14 @@ HOST_SITES_DOMAIN=openvibe.host# SQLite database and content-addressed objects (the production unit sets both under# /var/lib/openvibe-host-api)++# PostgreSQL (ADR-035), written by OpenVibe.Host roles/data/add-service.sh host: the runtime role through PgBouncer,++# the owner on a direct connection (migrations and the one-time import), and Valkey (shared limit counters).++# Without DATABASE_URL, development uses an embedded PGlite database in data/pglite.++DATABASE_URL=++DATABASE_DIRECT_URL=++VALKEY_URL=++VALKEY_PREFIX=++# The SQLite file of releases before PostgreSQL: read once by scripts/migrate-to-postgres.js.HOST_DB_PATH=./data/host.dbHOST_STORAGE_DIR=./data/objects@@@ -10,6 +10,9 @@ jobs:with:syntax-dirs: lib servercontracts-service: host++ # ADR-035: PostgreSQL 18 + PgBouncer (transaction mode) + Valkey 9 containers (openvibe-sdk++ # scripts/test-services.sh). npm test runs every file on PGlite; test:pg runs them all through PgBouncer.++ test: 'eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" && npm test && npm run test:pg'contracts-src: lib serverextra: node --check bin/ovhost && for f in scripts/*.sh; do bash -n "$f"; done# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).@@@ -49,10 +49,10 @@ Stage A adds a few safety rules of its own:## Depends on--- OpenVibe.Contracts (`openvibe-contracts` v0.66.0; `host.*` capabilities and the `host` manifest were released in v0.24.0, and v0.32.0 adds the takedown routes to `host.site.manage`): service manifests (vhost rendering, snapshots, the first-party domain list), ids, problem+json, service-token verification, capability checks.++- OpenVibe.Contracts (`openvibe-contracts` v0.76.0; `host.*` capabilities and the `host` manifest were released in v0.24.0, and v0.32.0 adds the takedown routes to `host.site.manage`): service manifests (vhost rendering, snapshots, the first-party domain list), ids, problem+json, service-token verification, capability checks.- OpenVibe.Network (Stage B): SSO for the dashboard, the JWKS that verifies user and service tokens, client-credentials tokens for the outbox relay.--- OpenVibe.Events (Stage B): `host.*` events through the `openvibe-sdk` v0.12.0 transactional outbox (openvibe-sdk/limits for the per-actor limits).--- `openvibe-shared` v1.25.0 (Stage B): shared chrome, legal pages, `/release.json`, `/metrics`, `/api/ready`.++- OpenVibe.Events (Stage B): `host.*` events through the `openvibe-sdk` v0.20.3 transactional outbox (openvibe-sdk/limits for the per-actor limits).++- `openvibe-shared` v1.27.0 (Stage B): shared chrome, legal pages, `/release.json`, `/metrics`, `/api/ready`.- OpenVibe.Media: not yet. The roadmap stores artifacts "through Media where practical"; Stage B keeps them on local disk for now (see [Not done yet](#not-done-yet-stage-b)).## Capabilities@@@ -212,8 +212,7 @@ Two things deploy from this repository:The unit is `openvibe-host.service` on `127.0.0.1:4910`, the env file `/etc/openvibe/host.env`. Its state is`/var/lib/openvibe-host-api` (not ovhost's own `/var/lib/openvibe-host`).Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the-- restart; afterwards `sudo ovhost rollback host --to <sha>`. Nothing blocks a rollback: the schema-- code only adds tables and columns.++ restart; afterwards `sudo ovhost rollback host --to <sha>`. Migrations only add tables and columns.## Installing on the host (for the operator)@@@ -303,11 +302,11 @@ The three capability ids and the service manifest are released in `openvibe-cont### Activation and rollback--Activation is one SQLite transaction: a compare-and-set on the site's `active_deploy_id`, an activation record and the `host.deploy.activated` event. If any part fails, none of it happens. `expected_active` turns a racing operator's switch into a 409. Rollback goes to a named ready deploy of the same site, or to the most recent previously active one that still exists. The active deploy cannot be deleted. Deleting another deploy removes its objects when no other deploy of the project uses them.++Activation is one PostgreSQL transaction: a compare-and-set on the site's `active_deploy_id`, an activation record and the `host.deploy.activated` event. If any part fails, none of it happens. `expected_active` turns a racing operator's switch into a 409. Rollback goes to a named ready deploy of the same site, or to the most recent previously active one that still exists. The active deploy cannot be deleted. Deleting another deploy removes its objects when no other deploy of the project uses them.### Events--Through the `openvibe-sdk` v0.5.0 transactional outbox (`event_outbox`), inside the transaction that makes the change:++Through the `openvibe-sdk` v0.20.3 transactional outbox (`event_outbox`), inside the transaction that makes the change:| Event | When | Payload ||---|---|---|@@@ -322,7 +321,7 @@ These tenant events have subject `deploy` and visibility `internal`; `host.deplo### Dashboard--Server-rendered pages with the shared chrome (`openvibe-shared` v1.5.1: navbar and theme loader from the Network, `<noscript>` navigation, SSR footer, app icon, legal pages). Every action is a plain form, so it works without JavaScript: projects, quotas and usage, members, sites, folder or archive upload, deploys with activate/rollback/delete, activation history, upload logs and file lists, domains with their DNS records and a "check DNS now" button. Pages are `private, no-store`, `noindex`, `frame-ancestors 'none'`.++Server-rendered pages with the shared chrome (`openvibe-shared` v1.27.0: navbar and theme loader from the Network, `<noscript>` navigation, SSR footer, app icon, legal pages). Every action is a plain form, so it works without JavaScript: projects, quotas and usage, members, sites, folder or archive upload, deploys with activate/rollback/delete, activation history, upload logs and file lists, domains with their DNS records and a "check DNS now" button. Pages are `private, no-store`, `noindex`, `frame-ancestors 'none'`.### Observability@@@ -333,7 +332,7 @@ Server-rendered pages with the shared chrome (`openvibe-shared` v1.5.1: navbar a- OpenVibe.Network OAuth client **`host`**, redirect `https://openvibe.host/auth/callback`, scope `profile theme`. The same client is the service principal `svc:host`.- Grant `[host, events.event.publish, openvibe.events]`.- Callers of Host get `[<client>, host.site.manage | host.deploy.create | host.domain.manage, openvibe.host]` as needed. None exist yet (Codes, the expected first caller, is not built).--- Released in `openvibe-contracts` v0.24.0 (CI contract check blocking); v0.32.0 adds the takedown routes to `host.site.manage.implementedBy`.++- Released in `openvibe-contracts` v0.76.0 (CI contract check blocking); v0.32.0 adds the takedown routes to `host.site.manage.implementedBy`.### Deploying Stage B (for the operator)@@@ -352,7 +351,7 @@ The service runs on the host (loopback, since 2026-09-23) and the pending tenant### Not done yet (Stage B)- Uploads are held in memory while they are validated (bounded by `HOST_MAX_UPLOAD_BYTES` and `HOST_MAX_UNPACKED_BYTES`), so the service needs that much headroom per concurrent upload.--- Objects live on the service's local disk, not in OpenVibe.Media; there is no replication and no backup beyond `ovhost backup host` (the SQLite database only).++- Objects live on the service's local disk, not in OpenVibe.Media; there is no replication and no backup beyond `ovhost backup host`. The database (`ov_host` on the host's data role, ADR-035; schema in [migrations/](migrations/)) is backed up with the others by pgBackRest.- Certificates for custom domains are issued by hand (step 8); there is no automatic ACME flow.- Projects are created in Host. When Network has projects (ADR-014), Host should accept only Network project ids and read membership from Network.- `openvibe.host` is not on the Public Suffix List, so a tenant page can still set `Domain=openvibe.host` cookies (a cookie bomb breaks the dashboard and other tenant sites for that visitor). The protections above do not depend on it. [docs/threat-review.md §5](docs/threat-review.md#5-the-public-suffix-list-question) records the decision (launch without it) and the recommended follow-up (dashboard off the tenant zone, then list the zone).@@@ -4,16 +4,30 @@"code": true,"wave": "W21 Stage A + Stage B","plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",-- "runtime": "node 22; Stage A: CLI (ovhost); Stage B: express 4 + better-sqlite3 11 service",++ "runtime": "node 22; Stage A: CLI (ovhost); Stage B: express 4 + PostgreSQL 18 (openvibe-sdk/db, async) + Valkey 9 service","port": 4910,"domain": "openvibe.host","tenantDomain": "*.openvibe.host",-- "contracts": "openvibe-contracts v0.66.0",-- "packages": ["openvibe-contracts v0.66.0 (host.* capabilities and manifest released in v0.24.0; v0.32.0 adds the takedown routes to host.site.manage)", "openvibe-sdk v0.12.0", "openvibe-shared v1.25.0"],-- "capabilities": ["host.site.manage", "host.deploy.create", "host.domain.manage"],++ "contracts": "openvibe-contracts v0.76.0",++ "packages": [++ "openvibe-contracts v0.76.0",++ "openvibe-sdk v0.20.3",++ "openvibe-shared v1.27.0"++ ],++ "capabilities": [++ "host.site.manage",++ "host.deploy.create",++ "host.domain.manage"++ ],"capabilitiesRegistered": true,"serviceManifestRegistered": "host manifest v0.2.0 released (openvibe-contracts v0.24.0+), with the four host.* events",-- "events": ["host.deploy.created", "host.deploy.activated", "host.deploy.failed", "host.domain.verified", "host.release.published"],++ "events": [++ "host.deploy.created",++ "host.deploy.activated",++ "host.deploy.failed",++ "host.domain.verified",++ "host.release.published"++ ],"stages": {"A": "alpha: ovhost CLI installed on the production host 2026-09-23 (/usr/local/bin/ovhost, inventory /etc/openvibe/host.json); every service deploys with `sudo ovhost deploy <svc>` since the 2026-09-27 cutover (roadmap WS-N task 11: Live, Tools, Sites, Games and OpenRe through the release-layout, multi-app, static-build and pnpm-build strategies); nightly backups, restore drills (19 passed 2026-09-23, docs/restore-drills.md), release notifications, incidents and maintenance, the alerts relay; the D41 deploy-and-rollback proof (scripts/d41-proof.sh) is written and tested","B": "alpha: running on the host on loopback :4910 (2026-09-23 audit), not launched; launch-rule items met in code and tests, launch steps in docs/launch.md, threat review in docs/threat-review.md",@@@ -3,8 +3,8 @@ Description=OpenVibe.Host Stage B: tenant static hosting (dashboard, API and *.o# Network (sign-in, JWKS, service tokens) and Events (the outbox relay) should be up first, but Host# degrades gracefully without them: tenant sites keep serving; sign-in, API calls and events wait# (see /api/ready).--After=network.target--Wants=openvibe-network.service++After=network.target postgresql.service pgbouncer.service valkey-server.service++Wants=openvibe-network.service postgresql.service pgbouncer.service valkey-server.service[Service]Type=simplennew file mode 100644@@@ -0,0 +1,185 @@++-- phase: expand++-- OpenVibe.Host on PostgreSQL (ADR-035, roadmap WS-X2): the tables as they were on SQLite (converted by openvibe-sdk++-- tools/asyncify/sqlite-schema-to-pg: text COLLATE "C" compares like SQLite, integers are bigint, identities keep their ids),++-- then the openvibe-publishing stores and the openvibe-sdk inbox and outbox. Generated once on 2026-09-28; never edited after it runs.++++CREATE TABLE host_projects (++ id text COLLATE "C" PRIMARY KEY, -- prj_<ULID>++ seq bigint GENERATED ALWAYS AS IDENTITY UNIQUE, -- insertion order (the SQLite rowid tiebreak)++ network_project_id text COLLATE "C", -- OpenVibe.Network project id (ADR-014), when it exists++ owner_subject text COLLATE "C" NOT NULL, -- usr_…++ name text COLLATE "C" NOT NULL,++ environment text COLLATE "C" NOT NULL DEFAULT 'production' CHECK (environment IN ('production','sandbox')),++ status text COLLATE "C" NOT NULL DEFAULT 'active' CHECK (status IN ('active','deleted')),++ created_at bigint NOT NULL,++ updated_at bigint NOT NULL,++ deleted_at bigint++);++CREATE UNIQUE INDEX host_projects_network ON host_projects (network_project_id) WHERE network_project_id IS NOT NULL AND status = 'active';++CREATE INDEX host_projects_owner ON host_projects (owner_subject, status);++++CREATE TABLE host_project_members (++ project_id text COLLATE "C" NOT NULL REFERENCES host_projects(id),++ principal text COLLATE "C" NOT NULL, -- usr_… | app:app_… | svc:<id>++ role text COLLATE "C" NOT NULL CHECK (role IN ('owner','maintainer','deployer')),++ added_by text COLLATE "C",++ created_at bigint NOT NULL,++ PRIMARY KEY (project_id, principal)++);++CREATE INDEX host_project_members_principal ON host_project_members (principal);++++CREATE TABLE host_quotas (++ project_id text COLLATE "C" PRIMARY KEY REFERENCES host_projects(id),++ storage_bytes bigint,++ deploys_per_day bigint,++ max_files bigint,++ max_file_bytes bigint,++ sites bigint,++ custom_domains bigint,++ updated_by text COLLATE "C",++ updated_at bigint NOT NULL++);++++CREATE TABLE host_sites (++ id text COLLATE "C" PRIMARY KEY, -- site_<ULID>++ seq bigint GENERATED ALWAYS AS IDENTITY UNIQUE, -- insertion order (the SQLite rowid tiebreak)++ project_id text COLLATE "C" NOT NULL REFERENCES host_projects(id),++ name text COLLATE "C" NOT NULL, -- the <name>.openvibe.host label++ active_deploy_id text COLLATE "C",++ status text COLLATE "C" NOT NULL DEFAULT 'active' CHECK (status IN ('active','deleted')),++ created_by text COLLATE "C" NOT NULL,++ created_at bigint NOT NULL,++ updated_at bigint NOT NULL,++ deleted_at bigint++);++CREATE UNIQUE INDEX host_sites_name ON host_sites (name) WHERE status = 'active';++CREATE INDEX host_sites_project ON host_sites (project_id, status);++++CREATE TABLE host_deploys (++ id text COLLATE "C" PRIMARY KEY, -- dpl_<ULID>++ seq bigint GENERATED ALWAYS AS IDENTITY UNIQUE, -- insertion order (the SQLite rowid tiebreak)++ project_id text COLLATE "C" NOT NULL REFERENCES host_projects(id),++ site_id text COLLATE "C" NOT NULL REFERENCES host_sites(id),++ state text COLLATE "C" NOT NULL CHECK (state IN ('ready','failed','deleted')),++ source text COLLATE "C" NOT NULL CHECK (source IN ('archive','files')),++ manifest text COLLATE "C", -- JSON host.deploy-manifest@1 (NULL when failed)++ manifest_sha256 text COLLATE "C",++ file_count bigint NOT NULL DEFAULT 0,++ total_bytes bigint NOT NULL DEFAULT 0,++ new_bytes bigint NOT NULL DEFAULT 0, -- bytes this deploy added to the project's storage++ failure_code text COLLATE "C",++ created_by text COLLATE "C" NOT NULL,++ created_at bigint NOT NULL,++ deleted_at bigint++);++CREATE INDEX host_deploys_site ON host_deploys (site_id, created_at);++CREATE INDEX host_deploys_project_day ON host_deploys (project_id, created_at);++++CREATE TABLE host_deploy_files (++ deploy_id text COLLATE "C" NOT NULL REFERENCES host_deploys(id),++ path text COLLATE "C" NOT NULL,++ sha256 text COLLATE "C" NOT NULL,++ size bigint NOT NULL,++ content_type text COLLATE "C" NOT NULL,++ PRIMARY KEY (deploy_id, path)++);++CREATE INDEX host_deploy_files_sha ON host_deploy_files (sha256);++++CREATE TABLE host_blobs (++ project_id text COLLATE "C" NOT NULL,++ sha256 text COLLATE "C" NOT NULL,++ size bigint NOT NULL,++ created_at bigint NOT NULL,++ PRIMARY KEY (project_id, sha256)++);++++CREATE TABLE host_activations (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ site_id text COLLATE "C" NOT NULL REFERENCES host_sites(id),++ deploy_id text COLLATE "C" NOT NULL,++ previous_deploy_id text COLLATE "C",++ kind text COLLATE "C" NOT NULL CHECK (kind IN ('activate','rollback')),++ actor text COLLATE "C" NOT NULL,++ created_at bigint NOT NULL++);++CREATE INDEX host_activations_site ON host_activations (site_id, id);++++CREATE TABLE host_deploy_logs (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ deploy_id text COLLATE "C" NOT NULL,++ level text COLLATE "C" NOT NULL CHECK (level IN ('info','warn','error')),++ message text COLLATE "C" NOT NULL,++ created_at bigint NOT NULL++);++CREATE INDEX host_deploy_logs_deploy ON host_deploy_logs (deploy_id, id);++++CREATE TABLE host_domains (++ id text COLLATE "C" PRIMARY KEY, -- dom_<ULID>++ seq bigint GENERATED ALWAYS AS IDENTITY UNIQUE, -- insertion order (the SQLite rowid tiebreak)++ project_id text COLLATE "C" NOT NULL REFERENCES host_projects(id),++ site_id text COLLATE "C" NOT NULL REFERENCES host_sites(id),++ hostname text COLLATE "C" NOT NULL,++ kind text COLLATE "C" NOT NULL CHECK (kind IN ('default','custom')),++ status text COLLATE "C" NOT NULL CHECK (status IN ('pending','verified','failed','lapsed')),++ token text COLLATE "C", -- the TXT verification value (custom only; public in DNS, not a secret)++ created_by text COLLATE "C" NOT NULL,++ created_at bigint NOT NULL,++ verified_at bigint,++ last_checked_at bigint,++ last_error text COLLATE "C",++ check_count bigint NOT NULL DEFAULT 0,++ record_missing_since bigint -- verified domain whose TXT record disappeared++);++CREATE UNIQUE INDEX host_domains_verified ON host_domains (hostname) WHERE status = 'verified';++CREATE UNIQUE INDEX host_domains_site_host ON host_domains (site_id, hostname);++CREATE INDEX host_domains_status ON host_domains (kind, status, last_checked_at);++++-- Abuse takedowns (staff only): a site, or every site of a project, stops being served (451) while++-- its deploys, files and objects are KEPT for review. Rows are never deleted: a lift is recorded.++CREATE TABLE host_takedowns (++ id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,++ target_kind text COLLATE "C" NOT NULL CHECK (target_kind IN ('project','site')),++ target_id text COLLATE "C" NOT NULL, -- prj_… or site_…++ reason text COLLATE "C" NOT NULL,++ created_by text COLLATE "C" NOT NULL,++ created_at bigint NOT NULL,++ lifted_by text COLLATE "C",++ lifted_at bigint,++ lift_note text COLLATE "C"++);++CREATE UNIQUE INDEX host_takedowns_active ON host_takedowns (target_kind, target_id) WHERE lifted_at IS NULL;++++-- Deploy artifacts are immutable: only state / deleted_at may change, and never back from 'deleted'.++CREATE FUNCTION host_deploys_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF TRUE THEN RAISE EXCEPTION 'host: deploy artifacts are immutable'; END IF; RETURN NEW; END $$;++CREATE TRIGGER host_deploys_immutable BEFORE UPDATE OF id, project_id, site_id, source, manifest, manifest_sha256, file_count, total_bytes, new_bytes, created_by, created_at ON host_deploys FOR EACH ROW EXECUTE FUNCTION host_deploys_immutable();++CREATE FUNCTION host_deploys_no_undelete() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF OLD.state = 'deleted' OR (OLD.state = 'failed' AND NEW.state = 'ready') THEN RAISE EXCEPTION 'host: a deleted or failed deploy cannot become ready'; END IF; RETURN NEW; END $$;++CREATE TRIGGER host_deploys_no_undelete BEFORE UPDATE OF state ON host_deploys FOR EACH ROW EXECUTE FUNCTION host_deploys_no_undelete();++CREATE FUNCTION host_deploy_files_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF TRUE THEN RAISE EXCEPTION 'host: deploy files are immutable'; END IF; RETURN NEW; END $$;++CREATE TRIGGER host_deploy_files_immutable BEFORE UPDATE ON host_deploy_files FOR EACH ROW EXECUTE FUNCTION host_deploy_files_immutable();++CREATE FUNCTION host_deploy_files_delete_only_deleted() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF (SELECT state FROM host_deploys WHERE id = OLD.deploy_id) <> 'deleted' THEN RAISE EXCEPTION 'host: files of a live deploy cannot be removed'; END IF; RETURN OLD; END $$;++CREATE TRIGGER host_deploy_files_delete_only_deleted BEFORE DELETE ON host_deploy_files FOR EACH ROW EXECUTE FUNCTION host_deploy_files_delete_only_deleted();++++-- openvibe-sdk/events inbox: one receipt per (consumer, event) handled++CREATE TABLE IF NOT EXISTS idempotency_receipts (++ consumer text NOT NULL,++ event_id text NOT NULL,++ processed_at bigint NOT NULL,++ PRIMARY KEY (consumer, event_id)++);++++-- openvibe-sdk/events PostgreSQL outbox++CREATE TABLE IF NOT EXISTS event_outbox (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ event_id text NOT NULL UNIQUE,++ envelope jsonb NOT NULL,++ traceparent text,++ created_at bigint NOT NULL,++ attempts integer NOT NULL DEFAULT 0,++ next_attempt_at bigint NOT NULL DEFAULT 0,++ sent_at bigint,++ seq bigint,++ rejected_at bigint,++ last_error text++);++CREATE INDEX IF NOT EXISTS event_outbox_due ON event_outbox (next_attempt_at, id) WHERE sent_at IS NULL AND rejected_at IS NULL;++CREATE INDEX IF NOT EXISTS event_outbox_sent ON event_outbox (sent_at) WHERE sent_at IS NOT NULL;@@@ -16,14 +16,17 @@"dotenv": "^16.4.7","express": "^4.21.2","express-rate-limit": "^7.5.0",++ "iovalkey": "^0.4.0","openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",-- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"++ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",++ "pg": "^8.23.0"},"bin": {"ovhost": "bin/ovhost"},"devDependencies": {++ "@electric-sql/pglite": "^0.5.8","jsonwebtoken": "^9.0.2"},"engines": {@@@ -338,6 +341,19 @@"node": ">=18.0.0"}},++ "node_modules/@electric-sql/pglite": {++ "version": "0.5.8",++ "resolved": "https://registry.npmjs.org/@electric-sql/pglite/-/pglite-0.5.8.tgz",++ "integrity": "sha512-n9tsbUOhwx2epK1V0ZG9Ar4SHWUju04dhmzZXiSBXwBoleOvIfals33NAaWgagQVAL4Rbvx/Ptsu3P+pA09f6Q==",++ "devOptional": true,++ "license": "Apache-2.0"++ },++ "node_modules/@iovalkey/commands": {++ "version": "0.1.0",++ "resolved": "https://registry.npmjs.org/@iovalkey/commands/-/commands-0.1.0.tgz",++ "integrity": "sha512-/B9W4qKSSITDii5nkBCHyPkIkAi+ealUtr1oqBJsLxjSRLka4pxun2VvMNSmcwgAMxgXtQfl0qRv7TE+udPJzg==",++ "license": "MIT"++ },"node_modules/@smithy/core": {"version": "3.35.0","resolved": "https://registry.npmjs.org/@smithy/core/-/core-3.35.0.tgz",@@@ -637,6 +653,15 @@"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==","license": "ISC"},++ "node_modules/cluster-key-slot": {++ "version": "1.1.2",++ "resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",++ "integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==",++ "license": "Apache-2.0",++ "engines": {++ "node": ">=0.10.0"++ }++ },"node_modules/content-disposition": {"version": "0.5.4","resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",@@@ -719,6 +744,15 @@"node": ">=4.0.0"}},++ "node_modules/denque": {++ "version": "2.1.0",++ "resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",++ "integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",++ "license": "Apache-2.0",++ "engines": {++ "node": ">=0.10"++ }++ },"node_modules/depd": {"version": "2.0.0","resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",@@@ -1143,6 +1177,49 @@"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==","license": "ISC"},++ "node_modules/iovalkey": {++ "version": "0.4.0",++ "resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.4.0.tgz",++ "integrity": "sha512-OSUKxJ+s44CLdUTaicX4+pVrBN/zHKIYwr2oKhRDuczr19FwcSCXEZHzcMUYWZNh1mNSCV9bNJHW/T6cHVm9Aw==",++ "license": "MIT",++ "dependencies": {++ "@iovalkey/commands": "^0.1.0",++ "cluster-key-slot": "^1.1.0",++ "debug": "^4.3.4",++ "denque": "^2.1.0",++ "lodash.defaults": "^4.2.0",++ "lodash.isarguments": "^3.1.0",++ "redis-errors": "^1.2.0",++ "redis-parser": "^3.0.0",++ "standard-as-callback": "^2.1.0"++ },++ "engines": {++ "node": ">=18.12.0"++ }++ },++ "node_modules/iovalkey/node_modules/debug": {++ "version": "4.4.3",++ "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",++ "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",++ "license": "MIT",++ "dependencies": {++ "ms": "^2.1.3"++ },++ "engines": {++ "node": ">=6.0"++ },++ "peerDependenciesMeta": {++ "supports-color": {++ "optional": true++ }++ }++ },++ "node_modules/iovalkey/node_modules/ms": {++ "version": "2.1.3",++ "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",++ "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",++ "license": "MIT"++ },"node_modules/ipaddr.js": {"version": "1.9.1","resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",@@@ -1207,12 +1284,24 @@"safe-buffer": "^5.0.1"}},++ "node_modules/lodash.defaults": {++ "version": "4.2.0",++ "resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz",++ "integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==",++ "license": "MIT"++ },"node_modules/lodash.includes": {"version": "4.3.0","resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz","integrity": "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w==","license": "MIT"},++ "node_modules/lodash.isarguments": {++ "version": "3.1.0",++ "resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz",++ "integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==",++ "license": "MIT"++ },"node_modules/lodash.isboolean": {"version": "3.0.3","resolved": "https://registry.npmjs.org/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz",@@@ -1428,18 +1517,38 @@}},"node_modules/openvibe-sdk": {-- "version": "0.12.0",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",-- "integrity": "sha512-iG1h905O1f7y/PnP8xLA6GCVzXEBDfSaBj2tc1ucNreWg2sT/AJWosb8NW0WRMK5WS4sIQ6tMsprmDLj/fjjeQ==",++ "version": "0.20.3",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",++ "integrity": "sha512-+g+ySveItzOBnUsw9aZZIJVKRrGuj0aO328W4N25T0Z8mY4fE6++ImvLnZ55LH2BRh0WYozC1KAVIShACm2KGQ==","license": "MIT","engines": {"node": ">=20"++ },++ "peerDependencies": {++ "@electric-sql/pglite": "^0.5.8",++ "better-sqlite3": ">=11",++ "iovalkey": "^0.4.0",++ "pg": "^8.23.0"++ },++ "peerDependenciesMeta": {++ "@electric-sql/pglite": {++ "optional": true++ },++ "better-sqlite3": {++ "optional": true++ },++ "iovalkey": {++ "optional": true++ },++ "pg": {++ "optional": true++ }}},"node_modules/openvibe-shared": {-- "version": "1.25.0",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0",-- "integrity": "sha512-oqZ+PvljrWi8IEbtg6bDYai17ZV2sALFx/ezMptniELSr10TgFUiGUQt+b1IWwiHv0T2Y1U2iSrkirJxMNgjSw==",++ "version": "1.27.0",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",++ "integrity": "sha512-/H7dyFw+cURcdjyfbpyY7mNc6F+Pcb9TwkVwZHR+9d3M62q70KDhZLo/gZi71eaYxUjD8lO8tmbXu70EAa9paQ==","license": "MIT","dependencies": {"jsonwebtoken": "^9.0.0"@@@ -1460,6 +1569,134 @@"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==","license": "MIT"},++ "node_modules/pg": {++ "version": "8.23.0",++ "resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",++ "integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",++ "license": "MIT",++ "dependencies": {++ "pg-connection-string": "^2.14.0",++ "pg-pool": "^3.14.0",++ "pg-protocol": "^1.16.0",++ "pg-types": "2.2.0",++ "pgpass": "1.0.5"++ },++ "engines": {++ "node": ">= 16.0.0"++ },++ "optionalDependencies": {++ "pg-cloudflare": "^1.4.0"++ },++ "peerDependencies": {++ "pg-native": ">=3.0.1"++ },++ "peerDependenciesMeta": {++ "pg-native": {++ "optional": true++ }++ }++ },++ "node_modules/pg-cloudflare": {++ "version": "1.4.0",++ "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",++ "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",++ "license": "MIT",++ "optional": true++ },++ "node_modules/pg-connection-string": {++ "version": "2.14.0",++ "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",++ "integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",++ "license": "MIT"++ },++ "node_modules/pg-int8": {++ "version": "1.0.1",++ "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",++ "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",++ "license": "ISC",++ "engines": {++ "node": ">=4.0.0"++ }++ },++ "node_modules/pg-pool": {++ "version": "3.14.0",++ "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",++ "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",++ "license": "MIT",++ "peerDependencies": {++ "pg": ">=8.0"++ }++ },++ "node_modules/pg-protocol": {++ "version": "1.16.0",++ "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz",++ "integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==",++ "license": "MIT"++ },++ "node_modules/pg-types": {++ "version": "2.2.0",++ "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",++ "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",++ "license": "MIT",++ "dependencies": {++ "pg-int8": "1.0.1",++ "postgres-array": "~2.0.0",++ "postgres-bytea": "~1.0.0",++ "postgres-date": "~1.0.4",++ "postgres-interval": "^1.1.0"++ },++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/pgpass": {++ "version": "1.0.5",++ "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",++ "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",++ "license": "MIT",++ "dependencies": {++ "split2": "^4.1.0"++ }++ },++ "node_modules/postgres-array": {++ "version": "2.0.0",++ "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",++ "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",++ "license": "MIT",++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/postgres-bytea": {++ "version": "1.0.1",++ "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",++ "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",++ "license": "MIT",++ "engines": {++ "node": ">=0.10.0"++ }++ },++ "node_modules/postgres-date": {++ "version": "1.0.7",++ "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",++ "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",++ "license": "MIT",++ "engines": {++ "node": ">=0.10.0"++ }++ },++ "node_modules/postgres-interval": {++ "version": "1.2.0",++ "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",++ "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",++ "license": "MIT",++ "dependencies": {++ "xtend": "^4.0.0"++ },++ "engines": {++ "node": ">=0.10.0"++ }++ },"node_modules/prebuild-install": {"version": "7.1.3","resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",@@@ -1583,6 +1820,27 @@"node": ">= 6"}},++ "node_modules/redis-errors": {++ "version": "1.2.0",++ "resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",++ "integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==",++ "license": "MIT",++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/redis-parser": {++ "version": "3.0.0",++ "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz",++ "integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==",++ "license": "MIT",++ "dependencies": {++ "redis-errors": "^1.0.0"++ },++ "engines": {++ "node": ">=4"++ }++ },"node_modules/require-from-string": {"version": "2.0.2","resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",@@@ -1798,6 +2056,21 @@"simple-concat": "^1.0.0"}},++ "node_modules/split2": {++ "version": "4.2.0",++ "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",++ "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",++ "license": "ISC",++ "engines": {++ "node": ">= 10.x"++ }++ },++ "node_modules/standard-as-callback": {++ "version": "2.1.0",++ "resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz",++ "integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==",++ "license": "MIT"++ },"node_modules/statuses": {"version": "2.0.2","resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",@@@ -1939,6 +2212,15 @@"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz","integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==","license": "ISC"++ },++ "node_modules/xtend": {++ "version": "4.0.2",++ "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",++ "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",++ "license": "MIT",++ "engines": {++ "node": ">=0.4"++ }}}}@@@ -10,7 +10,8 @@"scripts": {"start": "node server/index.js","dev": "NODE_ENV=development node server/index.js",-- "test": "node test/run.js"++ "test": "node test/run.js",++ "test:pg": "HOST_TEST_STORE=pg node test/run.js"},"keywords": ["openvibe",@@@ -31,14 +32,17 @@"dotenv": "^16.4.7","express": "^4.21.2","express-rate-limit": "^7.5.0",++ "iovalkey": "^0.4.0","openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",-- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"++ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",++ "pg": "^8.23.0"},"engines": {"node": ">=22.0.0"},"devDependencies": {-- "jsonwebtoken": "^9.0.2"++ "jsonwebtoken": "^9.0.2",++ "@electric-sql/pglite": "^0.5.8"}}@@@ -1,7 +1,7 @@#!/usr/bin/env bash# roles/data/switch-service.sh — the production switch of one service from its SQLite file to PostgreSQL (ADR-035).#--# sudo roles/data/switch-service.sh <service>++# sudo roles/data/switch-service.sh <service> [<sqlite file>]## Before: the service's postgres branch is merged to main with green CI, its auto-deploy is frozen# (ovhost freeze <svc> --reason …), and add-service.sh <svc> has written its database settings.@@@ -13,6 +13,8 @@# register C-89: sudo ovhost rollback <svc> to the release before, which reads the file).set -euo pipefailSVC=$1; UNIT=openvibe-$SVC.service; DIR=/opt/openvibe.$SVC; DATA=/var/lib/openvibe-$SVC; DB=$DATA/$SVC.db++# A second argument names the SQLite file when it is not /var/lib/openvibe-<svc>/<svc>.db (Host: /var/lib/openvibe-host-api/host.db).++if [ -n "${2:-}" ]; then DB=$2; DATA=$(dirname "$DB"); fiWORK=/var/tmp/pg-$SVC; UB=$(id -u ubuntu); GB=$(id -g ubuntu)[ -f "$DB" ] || { echo "no $DB"; exit 1; }rm -rf "$WORK"; install -d -o ubuntu -g ubuntu "$WORK"nnew file mode 100644@@@ -0,0 +1,26 @@++#!/usr/bin/env node++'use strict';++/**++ * The one-time move of Host's SQLite database (HOST_DB_PATH) into its PostgreSQL schema (ADR-035; the procedure is++ * openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by openvibe-sdk/db runSqliteMigration).++ *++ * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]++ *++ * Applies migrations/ as the owner (DATABASE_DIRECT_URL), copies every table into emptied tables, verifies row counts++ * and checksums, and exits 1 unless everything verified. The SQLite file is opened read-only. Every table keeps its++ * name and columns.++ */++require('dotenv').config();++const { runSqliteMigration } = require('openvibe-sdk/db');++const configLib = require('../server/config');++const { MIGRATIONS } = require('../server/db');++++const TABLES = {};++++if (require.main === module) {++ const config = (configLib.load || configLib.loadConfig)();++ runSqliteMigration({ service: 'host', sqlite: config.dbPath, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES })++ .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });++}++++module.exports = { TABLES };@@@ -65,10 +65,11 @@ const DASHBOARD_CSP_HEADER = Object.entries(DASHBOARD_CSP).map(([k, v]) => `${k}* opts: config, store | dbPath, now (clock), fetchImpl, auth (a createAuthClient-like object),* resolver ({ resolveTxt }), log*/--function createApp(opts = {}) {++async function createApp(opts = {}) {const config = opts.config || configLib.load();const log = opts.log || console;-- const store = opts.store || openStore(opts.dbPath || config.dbPath, { now: opts.now });++ // PostgreSQL (ADR-035): opened and migrated here unless the caller (a test, a script) hands in a store.++ const store = opts.store || await openStore(config, { now: opts.now, log });const blobs = createBlobStore(opts.storageDir || config.storageDir);require('./http/errors').setLogger(log);@@@ -97,11 +98,14 @@ function createApp(opts = {}) {const httpMetrics = sharedMetrics.httpMetrics(registry, { normalize: (req) => (req.hostTarget === 'site' ? 'tenant_site' : req.hostTarget === 'unknown' ? 'unknown_host' : null) });const proc = sharedMetrics.processMetrics(registry);// Per-actor limits on writes (server/http/actor-limits.js; roadmap WS-R task 4), one budget for the API and the dashboard.-- ctx.actorLimits = require('./http/actor-limits').createHostActorLimits({ registry });++ // Valkey (ADR-035): shared, never-authoritative state (per-actor limit counters). Optional.++ const valkey = opts.valkey !== undefined ? opts.valkey : (config.valkey.url ? require('openvibe-sdk/valkey').createValkey({ url: config.valkey.url, prefix: config.valkey.prefix, log }) : null);++ ctx.valkey = valkey;++ ctx.actorLimits = require('./http/actor-limits').createHostActorLimits({ registry, valkey });sharedMetrics.releaseInfo(registry, { service: 'host', release: release.release });registry.gauge({name: 'host_sites', help: 'Active tenant sites',-- collect: () => [{ labels: {}, value: store.db.prepare("SELECT COUNT(*) AS n FROM host_sites WHERE status = 'active'").get().n }],++ collect: async () => [{ labels: {}, value: (await store.db.prepare("SELECT COUNT(*) AS n FROM host_sites WHERE status = 'active'").get()).n }],});ctx.stopMetrics = proc.stop;app.locals.metrics = registry;@@@ -110,11 +114,11 @@ function createApp(opts = {}) {app.use(httpMetrics.middleware);// ── Host dispatch: tenant sites never reach anything below ──-- app.use((req, res, next) => {-- const target = tenant.resolve(req.headers.host);++ app.use(async (req, res, next) => {++ const target = await tenant.resolve(req.headers.host);req.hostTarget = target.kind;if (target.kind === 'dashboard') return next();-- if (target.kind === 'site') return tenant.handle(req, res, target.site);++ if (target.kind === 'site') return await tenant.handle(req, res, target.site);return tenant.unknownHost(req, res);});@@@ -135,7 +139,7 @@ function createApp(opts = {}) {release.mount(app, { registry: registry });// GET /limits.json: the developer limits enforced here, from config (WS-N task 7; Codes renders them).require('./limits').mountLimits(app, config);-- const readiness = createHostReadiness({ store, blobs, auth, outbox, release: release.release, minFreeBytes: () => config.uploads.minFreeBytes });++ const readiness = createHostReadiness({ store, blobs, auth, outbox, release: release.release, minFreeBytes: () => config.uploads.minFreeBytes, valkey: ctx.valkey });app.get('/api/ready', readiness.handler);app.get('/metrics', sharedMetrics.metricsHandler(registry));// The dashboard host: only the public front page and the legal pages are crawlable. Tenant sites@@@ -71,7 +71,7 @@ function createViewerResolver({ auth, config }) {const token = bearerToken(req);if (!token) return ANONYMOUS;const payload = decodeJwtPayload(token);-- if (payload && typeof payload.sub === 'string' && PRINCIPAL_SUB.test(payload.sub)) return fromServiceToken(req, token);++ if (payload && typeof payload.sub === 'string' && PRINCIPAL_SUB.test(payload.sub)) return await fromServiceToken(req, token);const v = await fromUserToken(token);if (!v) throw new ViewerError(401, 'auth.invalid_token', 'the bearer token is not a valid OpenVibe token');return v;@@@ -44,6 +44,10 @@ function load(env = process.env) {sitesDomain,trustProxy: env.TRUST_PROXY != null ? Number(env.TRUST_PROXY) : 1, // nginx (realip) → Node++ // PostgreSQL (ADR-035): DATABASE_URL serves (PgBouncer), DATABASE_DIRECT_URL migrates (owner role).++ db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '' },++ valkey: { url: env.VALKEY_URL || '', prefix: env.VALKEY_PREFIX || 'ov:host:' },++ // The SQLite file of releases before the switch: read once by scripts/migrate-to-postgres.js.dbPath: env.HOST_DB_PATH || './data/host.db',// Content-addressed deploy objects, one directory per project (tenancy keyed by project id).storageDir: env.HOST_STORAGE_DIR || './data/objects',@@@ -1,7 +1,7 @@'use strict';/**-- * Host's own SQLite database (WAL), created on boot, idempotently. Nothing here is shared with++ * Host's own PostgreSQL database (ADR-035, roadmap WS-X2): the schema is migrations/NNNN_*.sql, applied at boot. Nothing here is shared with* another service.** host_projects tenant projects: owner subject (usr_…), environment (production|sandbox),@@@ -23,181 +23,49 @@*/const fs = require('fs');const path = require('path');--const Database = require('better-sqlite3');++const { createDb } = require('openvibe-sdk/db');--const SCHEMA = `--CREATE TABLE IF NOT EXISTS host_projects (-- id TEXT PRIMARY KEY, -- prj_<ULID>-- network_project_id TEXT, -- OpenVibe.Network project id (ADR-014), when it exists-- owner_subject TEXT NOT NULL, -- usr_…-- name TEXT NOT NULL,-- environment TEXT NOT NULL DEFAULT 'production' CHECK (environment IN ('production','sandbox')),-- status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active','deleted')),-- created_at INTEGER NOT NULL,-- updated_at INTEGER NOT NULL,-- deleted_at INTEGER--);--CREATE UNIQUE INDEX IF NOT EXISTS host_projects_network ON host_projects (network_project_id) WHERE network_project_id IS NOT NULL AND status = 'active';--CREATE INDEX IF NOT EXISTS host_projects_owner ON host_projects (owner_subject, status);----CREATE TABLE IF NOT EXISTS host_project_members (-- project_id TEXT NOT NULL REFERENCES host_projects(id),-- principal TEXT NOT NULL, -- usr_… | app:app_… | svc:<id>-- role TEXT NOT NULL CHECK (role IN ('owner','maintainer','deployer')),-- added_by TEXT,-- created_at INTEGER NOT NULL,-- PRIMARY KEY (project_id, principal)--);--CREATE INDEX IF NOT EXISTS host_project_members_principal ON host_project_members (principal);----CREATE TABLE IF NOT EXISTS host_quotas (-- project_id TEXT PRIMARY KEY REFERENCES host_projects(id),-- storage_bytes INTEGER,-- deploys_per_day INTEGER,-- max_files INTEGER,-- max_file_bytes INTEGER,-- sites INTEGER,-- custom_domains INTEGER,-- updated_by TEXT,-- updated_at INTEGER NOT NULL--);----CREATE TABLE IF NOT EXISTS host_sites (-- id TEXT PRIMARY KEY, -- site_<ULID>-- project_id TEXT NOT NULL REFERENCES host_projects(id),-- name TEXT NOT NULL, -- the <name>.openvibe.host label-- active_deploy_id TEXT,-- status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active','deleted')),-- created_by TEXT NOT NULL,-- created_at INTEGER NOT NULL,-- updated_at INTEGER NOT NULL,-- deleted_at INTEGER--);--CREATE UNIQUE INDEX IF NOT EXISTS host_sites_name ON host_sites (name) WHERE status = 'active';--CREATE INDEX IF NOT EXISTS host_sites_project ON host_sites (project_id, status);----CREATE TABLE IF NOT EXISTS host_deploys (-- id TEXT PRIMARY KEY, -- dpl_<ULID>-- project_id TEXT NOT NULL REFERENCES host_projects(id),-- site_id TEXT NOT NULL REFERENCES host_sites(id),-- state TEXT NOT NULL CHECK (state IN ('ready','failed','deleted')),-- source TEXT NOT NULL CHECK (source IN ('archive','files')),-- manifest TEXT, -- JSON host.deploy-manifest@1 (NULL when failed)-- manifest_sha256 TEXT,-- file_count INTEGER NOT NULL DEFAULT 0,-- total_bytes INTEGER NOT NULL DEFAULT 0,-- new_bytes INTEGER NOT NULL DEFAULT 0, -- bytes this deploy added to the project's storage-- failure_code TEXT,-- created_by TEXT NOT NULL,-- created_at INTEGER NOT NULL,-- deleted_at INTEGER--);--CREATE INDEX IF NOT EXISTS host_deploys_site ON host_deploys (site_id, created_at);--CREATE INDEX IF NOT EXISTS host_deploys_project_day ON host_deploys (project_id, created_at);----CREATE TABLE IF NOT EXISTS host_deploy_files (-- deploy_id TEXT NOT NULL REFERENCES host_deploys(id),-- path TEXT NOT NULL,-- sha256 TEXT NOT NULL,-- size INTEGER NOT NULL,-- content_type TEXT NOT NULL,-- PRIMARY KEY (deploy_id, path)--) WITHOUT ROWID;--CREATE INDEX IF NOT EXISTS host_deploy_files_sha ON host_deploy_files (sha256);----CREATE TABLE IF NOT EXISTS host_blobs (-- project_id TEXT NOT NULL,-- sha256 TEXT NOT NULL,-- size INTEGER NOT NULL,-- created_at INTEGER NOT NULL,-- PRIMARY KEY (project_id, sha256)--) WITHOUT ROWID;----CREATE TABLE IF NOT EXISTS host_activations (-- id INTEGER PRIMARY KEY AUTOINCREMENT,-- site_id TEXT NOT NULL REFERENCES host_sites(id),-- deploy_id TEXT NOT NULL,-- previous_deploy_id TEXT,-- kind TEXT NOT NULL CHECK (kind IN ('activate','rollback')),-- actor TEXT NOT NULL,-- created_at INTEGER NOT NULL--);--CREATE INDEX IF NOT EXISTS host_activations_site ON host_activations (site_id, id);----CREATE TABLE IF NOT EXISTS host_deploy_logs (-- id INTEGER PRIMARY KEY AUTOINCREMENT,-- deploy_id TEXT NOT NULL,-- level TEXT NOT NULL CHECK (level IN ('info','warn','error')),-- message TEXT NOT NULL,-- created_at INTEGER NOT NULL--);--CREATE INDEX IF NOT EXISTS host_deploy_logs_deploy ON host_deploy_logs (deploy_id, id);----CREATE TABLE IF NOT EXISTS host_domains (-- id TEXT PRIMARY KEY, -- dom_<ULID>-- project_id TEXT NOT NULL REFERENCES host_projects(id),-- site_id TEXT NOT NULL REFERENCES host_sites(id),-- hostname TEXT NOT NULL,-- kind TEXT NOT NULL CHECK (kind IN ('default','custom')),-- status TEXT NOT NULL CHECK (status IN ('pending','verified','failed','lapsed')),-- token TEXT, -- the TXT verification value (custom only; public in DNS, not a secret)-- created_by TEXT NOT NULL,-- created_at INTEGER NOT NULL,-- verified_at INTEGER,-- last_checked_at INTEGER,-- last_error TEXT,-- check_count INTEGER NOT NULL DEFAULT 0,-- record_missing_since INTEGER -- verified domain whose TXT record disappeared--);--CREATE UNIQUE INDEX IF NOT EXISTS host_domains_verified ON host_domains (hostname) WHERE status = 'verified';--CREATE UNIQUE INDEX IF NOT EXISTS host_domains_site_host ON host_domains (site_id, hostname);--CREATE INDEX IF NOT EXISTS host_domains_status ON host_domains (kind, status, last_checked_at);----CREATE TABLE IF NOT EXISTS host_takedowns (-- id INTEGER PRIMARY KEY,-- target_kind TEXT NOT NULL CHECK (target_kind IN ('project','site')),-- target_id TEXT NOT NULL, -- prj_… or site_…-- reason TEXT NOT NULL,-- created_by TEXT NOT NULL,-- created_at INTEGER NOT NULL,-- lifted_by TEXT,-- lifted_at INTEGER,-- lift_note TEXT--);--CREATE UNIQUE INDEX IF NOT EXISTS host_takedowns_active ON host_takedowns (target_kind, target_id) WHERE lifted_at IS NULL;----CREATE TRIGGER IF NOT EXISTS host_deploys_immutable--BEFORE UPDATE OF id, project_id, site_id, source, manifest, manifest_sha256, file_count, total_bytes, new_bytes, created_by, created_at ON host_deploys--BEGIN SELECT RAISE(ABORT, 'host: deploy artifacts are immutable'); END;--CREATE TRIGGER IF NOT EXISTS host_deploys_no_undelete--BEFORE UPDATE OF state ON host_deploys WHEN OLD.state = 'deleted' OR (OLD.state = 'failed' AND NEW.state = 'ready')--BEGIN SELECT RAISE(ABORT, 'host: a deleted or failed deploy cannot become ready'); END;--CREATE TRIGGER IF NOT EXISTS host_deploy_files_immutable--BEFORE UPDATE ON host_deploy_files--BEGIN SELECT RAISE(ABORT, 'host: deploy files are immutable'); END;--CREATE TRIGGER IF NOT EXISTS host_deploy_files_delete_only_deleted--BEFORE DELETE ON host_deploy_files WHEN (SELECT state FROM host_deploys WHERE id = OLD.deploy_id) <> 'deleted'--BEGIN SELECT RAISE(ABORT, 'host: files of a live deploy cannot be removed'); END;--`;++const MIGRATIONS = path.join(__dirname, '..', 'migrations');++const DEV_PGLITE = path.join(__dirname, '..', 'data', 'pglite');const CHARTER_TABLES = ['host_projects', 'host_project_members', 'host_quotas', 'host_sites', 'host_deploys', 'host_deploy_files', 'host_blobs', 'host_activations', 'host_deploy_logs', 'host_domains', 'host_takedowns'];--function openStore(dbPath, { now = () => Date.now() } = {}) {-- if (dbPath !== ':memory:') fs.mkdirSync(path.dirname(path.resolve(dbPath)), { recursive: true });-- const db = new Database(dbPath);-- db.pragma('journal_mode = WAL');-- db.pragma('foreign_keys = ON');-- db.pragma('busy_timeout = 5000');-- db.exec(SCHEMA);++/**++ * The serving handle (ADR-035): DATABASE_URL through PgBouncer; in development without it, an embedded PGlite database++ * in data/pglite. Migrations run first, as the owner (DATABASE_DIRECT_URL), or on the embedded handle.++ */++async function openDb(config, { log = console, registry } = {}) {++ if (!config.db.url) {++ if (config.isProduction) throw new Error('DATABASE_URL is not set: production serves from PostgreSQL (OpenVibe.Host roles/data add-service.sh host)');++ log.warn(`[Host] DATABASE_URL unset: embedded PGlite database in ${DEV_PGLITE} (development only, one process)`);++ fs.mkdirSync(DEV_PGLITE, { recursive: true });++ const db = createDb({ pglite: DEV_PGLITE, service: 'host', registry, log });++ await db.migrate({ dir: MIGRATIONS, log });++ return db;++ }++ if (!config.db.directUrl) throw new Error('DATABASE_DIRECT_URL is not set: migrations run with the owner role on a direct connection');++ const owner = createDb({ url: config.db.directUrl, service: 'host-migrate', max: 1, log });++ try { await owner.migrate({ dir: MIGRATIONS, log }); } finally { await owner.close(); }++ return createDb({ url: config.db.url, service: 'host', registry, log });++}++++/**++ * Every store on a migrated database handle. opts.now — injectable clock (epoch ms), so tests and replays are++ * deterministic. store.tx(fn) is a transaction; inside it, plain db calls join it (ambient).++ */++function createStore(db, { now = () => Date.now() } = {}) {return {db,now,-- tx: (fn) => db.transaction(fn)(),++++ tx: async (fn) => await db.tx(() => fn()),close: () => db.close(),};}--module.exports = { openStore, CHARTER_TABLES };++/** openDb + createStore. */++async function openStore(config, { now, log } = {}) {++ return createStore(await openDb(config, { log }), { now });++}++++module.exports = { openDb, openStore, createStore, MIGRATIONS, CHARTER_TABLES };@@@ -47,10 +47,10 @@ function createAccess({ store }) {const { db } = store;const memberStmt = db.prepare('SELECT role FROM host_project_members WHERE project_id = ? AND principal = ?');-- function roleOf(project, viewer) {++ async function roleOf(project, viewer) {const p = principalOf(viewer);if (!p) return null;-- const row = memberStmt.get(project.id, p);++ const row = await memberStmt.get(project.id, p);return row ? row.role : null;}@@@ -58,7 +58,7 @@ function createAccess({ store }) {* Throws unless the viewer may act on the project at `need` (read|deploy|maintain|own|staff).* `notFound` is the problem thrown when the viewer has no business knowing the resource exists.*/-- function authorize(project, viewer, need, notFound = new ApiError(404, 'project.not_found', 'no such project')) {++ async function authorize(project, viewer, need, notFound = new ApiError(404, 'project.not_found', 'no such project')) {if (!project || project.status !== 'active') throw notFound;if (!viewer || viewer.kind === 'anonymous') throw new ApiError(401, 'auth.required', 'sign in with OpenVibe, or present a service token');if (viewer.kind === 'service' && viewer.env === 'sandbox' && project.environment !== 'sandbox') {@@@ -68,7 +68,7 @@ function createAccess({ store }) {if (viewer.staff) return 'staff';throw new ApiError(403, 'auth.staff_only', 'only OpenVibe staff can change quotas');}-- const role = roleOf(project, viewer);++ const role = await roleOf(project, viewer);if (!role) {if (viewer.staff && need !== 'deploy') return 'staff';throw notFound;@@@ -27,18 +27,18 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, oconst { db } = store;const q = {byId: db.prepare('SELECT * FROM host_deploys WHERE id = ?'),-- forSite: db.prepare("SELECT * FROM host_deploys WHERE site_id = ? AND state <> 'deleted' ORDER BY created_at DESC, rowid DESC LIMIT ?"),++ forSite: db.prepare("SELECT * FROM host_deploys WHERE site_id = ? AND state <> 'deleted' ORDER BY created_at DESC, seq DESC LIMIT ?"),insert: db.prepare(`INSERT INTO host_deploys (id, project_id, site_id, state, source, manifest, manifest_sha256, file_count, total_bytes, new_bytes, failure_code, created_by, created_at)VALUES (@id, @project_id, @site_id, @state, @source, @manifest, @manifest_sha256, @file_count, @total_bytes, @new_bytes, @failure_code, @created_by, @created_at)`),insertFile: db.prepare('INSERT INTO host_deploy_files (deploy_id, path, sha256, size, content_type) VALUES (?, ?, ?, ?, ?)'),-- insertBlob: db.prepare('INSERT OR IGNORE INTO host_blobs (project_id, sha256, size, created_at) VALUES (?, ?, ?, ?)'),++ insertBlob: db.prepare('INSERT INTO host_blobs (project_id, sha256, size, created_at) VALUES (?, ?, ?, ?) ON CONFLICT DO NOTHING'),hasBlob: db.prepare('SELECT 1 FROM host_blobs WHERE project_id = ? AND sha256 = ?'),-- storageUsed: db.prepare('SELECT COALESCE(SUM(size), 0) AS n FROM host_blobs WHERE project_id = ?'),++ storageUsed: db.prepare('SELECT COALESCE(SUM(size), 0)::bigint AS n FROM host_blobs WHERE project_id = ?'),deploysSince: db.prepare('SELECT COUNT(*) AS n FROM host_deploys WHERE project_id = ? AND created_at > ?'),logLine: db.prepare('INSERT INTO host_deploy_logs (deploy_id, level, message, created_at) VALUES (?, ?, ?, ?)'),logFor: db.prepare('SELECT level, message, created_at FROM host_deploy_logs WHERE deploy_id = ? ORDER BY id'),files: db.prepare('SELECT path, sha256, size, content_type FROM host_deploy_files WHERE deploy_id = ? ORDER BY path'),-- setPointer: db.prepare('UPDATE host_sites SET active_deploy_id = ?, updated_at = ? WHERE id = ? AND status = \'active\' AND active_deploy_id IS ?'),++ setPointer: db.prepare('UPDATE host_sites SET active_deploy_id = ?, updated_at = ? WHERE id = ? AND status = \'active\' AND active_deploy_id IS NOT DISTINCT FROM ?'),activation: db.prepare('INSERT INTO host_activations (site_id, deploy_id, previous_deploy_id, kind, actor, created_at) VALUES (?, ?, ?, ?, ?, ?)'),activations: db.prepare('SELECT deploy_id, previous_deploy_id, kind, actor, created_at FROM host_activations WHERE site_id = ? ORDER BY id DESC LIMIT ?'),markDeleted: db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE id = ?"),@@@ -49,32 +49,32 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, odropBlob: db.prepare('DELETE FROM host_blobs WHERE project_id = ? AND sha256 = ?'),};-- function get(id) {-- return isId('deploy', id) ? q.byId.get(id) || null : null;++ async function get(id) {++ return isId('deploy', id) ? await q.byId.get(id) || null : null;}/** The deploy, its site and project, authorized at `need`; 404 for anyone without access. */-- function load(viewer, id, need) {-- const deploy = get(id);++ async function load(viewer, id, need) {++ const deploy = await get(id);const notFound = new ApiError(404, 'deploy.not_found', 'no such deploy');if (!deploy || deploy.state === 'deleted') throw notFound;-- const site = sites.get(deploy.site_id);++ const site = await sites.get(deploy.site_id);if (!site || site.status !== 'active') throw notFound;-- const project = projects.get(deploy.project_id);-- access.authorize(project, viewer, need, notFound);++ const project = await projects.get(deploy.project_id);++ await access.authorize(project, viewer, need, notFound);return { deploy, site, project };}/** Limits for an upload into this project (the archive reader and the validator share them). */-- function limitsFor(project) {-- const quota = projects.quotaOf(project);++ async function limitsFor(project) {++ const quota = await projects.quotaOf(project);return { maxFiles: quota.maxFiles, maxFileBytes: quota.maxFileBytes, maxTotalBytes: quota.storageBytes, quota };}/** Before any byte is read: may this caller deploy to this site right now? */-- function precheck(viewer, siteId) {-- const { site, project } = sites.load(viewer, siteId, 'deploy');-- takedowns.assertOpen(site);++ async function precheck(viewer, siteId) {++ const { site, project } = await sites.load(viewer, siteId, 'deploy');++ await takedowns.assertOpen(site);const minFree = config ? config.uploads.minFreeBytes : 0;if (minFree > 0) {let free = Infinity;@@@ -84,20 +84,20 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, othrow new ApiError(507, 'storage.host_full', 'Host is short of disk space and is not accepting deploys right now; try again later');}}-- const limits = limitsFor(project);-- const recent = q.deploysSince.get(project.id, store.now() - DAY).n;++ const limits = await limitsFor(project);++ const recent = (await q.deploysSince.get(project.id, store.now() - DAY)).n;if (recent >= limits.quota.deploysPerDay) {throw new ApiError(429, 'quota.deploys_per_day', `this project has made ${recent} deploys in the last 24 hours; the limit is ${limits.quota.deploysPerDay}`);}return { site, project, limits };}-- function writeLog(deployId, lines, now) {-- for (const l of lines) q.logLine.run(deployId, l.level, String(l.message).slice(0, 2000), now);++ async function writeLog(deployId, lines, now) {++ for (const l of lines) await q.logLine.run(deployId, l.level, String(l.message).slice(0, 2000), now);}/** A refused upload becomes a failed deploy (with its log) and host.deploy.failed. */-- function recordFailure(viewer, { site, project }, { source, code, problems = [], notes = [], traceparent }) {++ async function recordFailure(viewer, { site, project }, { source, code, problems = [], notes = [], traceparent }) {const id = newId('deploy', store.now());const now = store.now();const lines = [@@@ -105,40 +105,40 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o...problems.map((p) => ({ level: 'error', message: `${p.code}: ${p.message}` })),{ level: 'error', message: `deploy refused (${code}); nothing was stored and the active deploy is unchanged` },];-- store.tx(() => {-- q.insert.run({ id, project_id: project.id, site_id: site.id, state: 'failed', source, manifest: null, manifest_sha256: null, file_count: 0, total_bytes: 0, new_bytes: 0, failure_code: code, created_by: principalOf(viewer), created_at: now });-- writeLog(id, lines, now);-- outbox.emit({++ await store.tx(async () => {++ await q.insert.run({ id, project_id: project.id, site_id: site.id, state: 'failed', source, manifest: null, manifest_sha256: null, file_count: 0, total_bytes: 0, new_bytes: 0, failure_code: code, created_by: principalOf(viewer), created_at: now });++ await writeLog(id, lines, now);++ await outbox.emit({event_type: 'host.deploy.failed', actor: actorRef(viewer), visibility: 'internal', priority: 'low',subject: { type: 'deploy', id },payload: { project_id: project.id, site_id: site.id, site: site.name, code, problems: problems.slice(0, 10).map((p) => p.code) },}, { traceparent });});outbox.kick();-- return { deploy: get(id), log: q.logFor.all(id) };++ return { deploy: await get(id), log: await q.logFor.all(id) };}/*** entries: [{ path, data }] already normalised by the reader. -> { deploy, log, activated }* Throws ApiError (with extra.deploy_id and extra.log) when the upload is refused.*/-- function create(viewer, ctx, entries, { source, activate = false, notes = [], traceparent } = {}) {++ async function create(viewer, ctx, entries, { source, activate = false, notes = [], traceparent } = {}) {const { site, project, limits } = ctx;-- const fail = (status, code, problems) => {-- const r = recordFailure(viewer, ctx, { source, code, problems, notes, traceparent });++ const fail = async (status, code, problems) => {++ const r = await recordFailure(viewer, ctx, { source, code, problems, notes, traceparent });throw new ApiError(status, code, problems[0] ? problems[0].message : code, { deploy_id: r.deploy.id, log: r.log.map((l) => `${l.level}: ${l.message}`) });};const built = buildManifest(entries, limits);-- if (!built.ok) fail(/^quota\.|deploy\.too_large/.test(built.code) ? 413 : 422, built.code, built.problems);++ if (!built.ok) await fail(/^quota\.|deploy\.too_large/.test(built.code) ? 413 : 422, built.code, built.problems);// Storage quota: only bytes this project does not store yet count.const fresh = new Map();-- for (const f of built.files) if (!q.hasBlob.get(project.id, f.sha256) && !fresh.has(f.sha256)) fresh.set(f.sha256, f);++ for (const f of built.files) if (!await q.hasBlob.get(project.id, f.sha256) && !fresh.has(f.sha256)) fresh.set(f.sha256, f);const newBytes = [...fresh.values()].reduce((n, f) => n + f.size, 0);-- const used = q.storageUsed.get(project.id).n;++ const used = (await q.storageUsed.get(project.id)).n;if (used + newBytes > limits.quota.storageBytes) {-- fail(413, 'quota.storage', [{ code: 'quota.storage', message: `this deploy adds ${fmtBytes(newBytes)}; the project stores ${fmtBytes(used)} of ${fmtBytes(limits.quota.storageBytes)}` }]);++ await fail(413, 'quota.storage', [{ code: 'quota.storage', message: `this deploy adds ${fmtBytes(newBytes)}; the project stores ${fmtBytes(used)} of ${fmtBytes(limits.quota.storageBytes)}` }]);}// Objects first (idempotent, content-addressed); rows only after they are durable.@@@ -156,30 +156,30 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o{ level: 'info', message: 'no build step: Stage B serves the uploaded files as they are; nothing was executed' },];let activated = null;-- store.tx(() => {-- q.insert.run({ id, project_id: project.id, site_id: site.id, state: 'ready', source, manifest: built.manifestJson, manifest_sha256: built.manifestSha256, file_count: built.manifest.file_count, total_bytes: built.manifest.total_bytes, new_bytes: newBytes, failure_code: null, created_by: who, created_at: now });-- for (const f of built.files) q.insertFile.run(id, f.path, f.sha256, f.size, f.content_type);-- for (const f of fresh.values()) q.insertBlob.run(project.id, f.sha256, f.size, now);-- outbox.emit({++ await store.tx(async () => {++ await q.insert.run({ id, project_id: project.id, site_id: site.id, state: 'ready', source, manifest: built.manifestJson, manifest_sha256: built.manifestSha256, file_count: built.manifest.file_count, total_bytes: built.manifest.total_bytes, new_bytes: newBytes, failure_code: null, created_by: who, created_at: now });++ for (const f of built.files) await q.insertFile.run(id, f.path, f.sha256, f.size, f.content_type);++ for (const f of fresh.values()) await q.insertBlob.run(project.id, f.sha256, f.size, now);++ await outbox.emit({event_type: 'host.deploy.created', actor: actorRef(viewer), visibility: 'internal', priority: 'low',subject: { type: 'deploy', id },payload: { project_id: project.id, site_id: site.id, site: site.name, source, file_count: built.manifest.file_count, total_bytes: built.manifest.total_bytes, manifest_sha256: built.manifestSha256 },}, { traceparent });if (activate) {-- activated = switchPointer(viewer, site, get(id), 'activate', { traceparent });++ activated = await switchPointer(viewer, site, await get(id), 'activate', { traceparent });lines.push({ level: 'info', message: 'activated: the site now serves this deploy' });} else {lines.push({ level: 'info', message: 'ready; not active until you activate it' });}-- writeLog(id, lines, now);++ await writeLog(id, lines, now);});outbox.kick();-- return { deploy: get(id), log: q.logFor.all(id), activated };++ return { deploy: await get(id), log: await q.logFor.all(id), activated };}/** Inside a transaction. Compare-and-set on the pointer read in the same transaction. */-- function switchPointer(viewer, site, deploy, kind, { expectedActive, traceparent } = {}) {-- const current = db.prepare('SELECT active_deploy_id, status FROM host_sites WHERE id = ?').get(site.id);++ async function switchPointer(viewer, site, deploy, kind, { expectedActive, traceparent } = {}) {++ const current = await db.prepare('SELECT active_deploy_id, status FROM host_sites WHERE id = ?').get(site.id);if (!current || current.status !== 'active') throw new ApiError(404, 'site.not_found', 'no such site');if (deploy.state !== 'ready' || deploy.site_id !== site.id) throw new ApiError(409, 'deploy.not_ready', 'only a ready deploy of this site can be activated');if (expectedActive !== undefined && (expectedActive || null) !== current.active_deploy_id) {@@@ -187,10 +187,10 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o}if (current.active_deploy_id === deploy.id) return { changed: false, deploy_id: deploy.id, previous_deploy_id: deploy.id };const now = store.now();-- const r = q.setPointer.run(deploy.id, now, site.id, current.active_deploy_id);++ const r = await q.setPointer.run(deploy.id, now, site.id, current.active_deploy_id);if (r.changes !== 1) throw new ApiError(409, 'site.active_changed', 'the active deploy changed during the switch');-- q.activation.run(site.id, deploy.id, current.active_deploy_id, kind, principalOf(viewer) || 'host', now);-- outbox.emit({++ await q.activation.run(site.id, deploy.id, current.active_deploy_id, kind, principalOf(viewer) || 'host', now);++ await outbox.emit({event_type: 'host.deploy.activated', actor: actorRef(viewer), visibility: 'internal', priority: 'low',subject: { type: 'deploy', id: deploy.id },payload: { project_id: site.project_id, site_id: site.id, site: site.name, deploy_id: deploy.id, previous_deploy_id: current.active_deploy_id, rollback: kind === 'rollback' },@@@ -198,45 +198,45 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, oreturn { changed: true, deploy_id: deploy.id, previous_deploy_id: current.active_deploy_id };}-- function activate(viewer, id, { expectedActive, traceparent } = {}) {-- const { deploy, site } = load(viewer, id, 'deploy');-- takedowns.assertOpen(site);-- const out = store.tx(() => switchPointer(viewer, site, deploy, 'activate', { expectedActive, traceparent }));++ async function activate(viewer, id, { expectedActive, traceparent } = {}) {++ const { deploy, site } = await load(viewer, id, 'deploy');++ await takedowns.assertOpen(site);++ const out = await store.tx(async () => await switchPointer(viewer, site, deploy, 'activate', { expectedActive, traceparent }));outbox.kick();return out;}/** To `deployId`, or else to the most recent previously active deploy that still exists. */-- function rollback(viewer, siteId, { deployId, expectedActive, traceparent } = {}) {-- const { site } = sites.load(viewer, siteId, 'deploy');-- takedowns.assertOpen(site);-- const out = store.tx(() => {-- const current = db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id).active_deploy_id;++ async function rollback(viewer, siteId, { deployId, expectedActive, traceparent } = {}) {++ const { site } = await sites.load(viewer, siteId, 'deploy');++ await takedowns.assertOpen(site);++ const out = await store.tx(async () => {++ const current = (await db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id)).active_deploy_id;let target = null;if (deployId) {-- target = get(deployId);++ target = await get(deployId);if (!target || target.site_id !== site.id || target.state === 'deleted') throw new ApiError(404, 'deploy.not_found', 'no such deploy on this site');} else {-- for (const a of q.activations.all(site.id, 200)) {++ for (const a of await q.activations.all(site.id, 200)) {for (const cand of [a.previous_deploy_id, a.deploy_id]) {if (!cand || cand === current) continue;-- const d = get(cand);++ const d = await get(cand);if (d && d.state === 'ready' && d.site_id === site.id) { target = d; break; }}if (target) break;}if (!target) throw new ApiError(409, 'deploy.no_previous', 'there is no previous deploy to roll back to');}-- return switchPointer(viewer, site, target, 'rollback', { expectedActive, traceparent });++ return await switchPointer(viewer, site, target, 'rollback', { expectedActive, traceparent });});outbox.kick();return out;}/** Inside a transaction: drop blob rows no deploy of the project references. -> freed shas */-- function collectGarbage(projectId) {-- const freed = q.orphans.all(projectId).map((r) => r.sha256);-- for (const sha of freed) q.dropBlob.run(projectId, sha);++ async function collectGarbage(projectId) {++ const freed = (await q.orphans.all(projectId)).map((r) => r.sha256);++ for (const sha of freed) await q.dropBlob.run(projectId, sha);return freed;}@@@ -246,16 +246,16 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o}}-- function remove(viewer, id) {-- const { deploy, site, project } = load(viewer, id, 'maintain');-- takedowns.assertDeletable(viewer, { site });++ async function remove(viewer, id) {++ const { deploy, site, project } = await load(viewer, id, 'maintain');++ await takedowns.assertDeletable(viewer, { site });let freed = [];-- store.tx(() => {-- const active = db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id).active_deploy_id;++ await store.tx(async () => {++ const active = (await db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id)).active_deploy_id;if (active === deploy.id) throw new ApiError(409, 'deploy.active', 'this deploy is being served: activate another one first');-- q.markDeleted.run(store.now(), deploy.id);-- q.dropFiles.run(deploy.id);-- freed = collectGarbage(project.id);++ await q.markDeleted.run(store.now(), deploy.id);++ await q.dropFiles.run(deploy.id);++ freed = await collectGarbage(project.id);});unlinkBlobs(project.id, freed);return { deleted: true, objects_removed: freed.length };@@@ -263,10 +263,10 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, oreturn {get, load, precheck, create, recordFailure, activate, rollback, remove, collectGarbage, unlinkBlobs, limitsFor,-- list: (siteId, limit = 100) => q.forSite.all(siteId, Math.min(Math.max(limit, 1), 200)),-- logOf: (id) => q.logFor.all(id),-- filesOf: (id) => q.files.all(id),-- activationsOf: (siteId, limit = 50) => q.activations.all(siteId, limit),++ list: async (siteId, limit = 100) => await q.forSite.all(siteId, Math.min(Math.max(limit, 1), 200)),++ logOf: async (id) => await q.logFor.all(id),++ filesOf: async (id) => await q.files.all(id),++ activationsOf: async (siteId, limit = 50) => await q.activations.all(siteId, limit),};}@@@ -57,7 +57,7 @@ function createDomains({ store, config, access, projects, sites, outbox, resolveconst reserved = firstPartyDomains(config);const q = {byId: db.prepare('SELECT * FROM host_domains WHERE id = ?'),-- forSite: db.prepare("SELECT * FROM host_domains WHERE site_id = ? ORDER BY kind DESC, created_at, rowid"),++ forSite: db.prepare("SELECT * FROM host_domains WHERE site_id = ? ORDER BY kind DESC, created_at, seq"),verifiedByHost: db.prepare("SELECT * FROM host_domains WHERE hostname = ? AND status = 'verified'"),onSite: db.prepare('SELECT * FROM host_domains WHERE site_id = ? AND hostname = ?'),insert: db.prepare(`INSERT INTO host_domains (id, project_id, site_id, hostname, kind, status, token, created_by, created_at)@@@ -74,13 +74,13 @@ function createDomains({ store, config, access, projects, sites, outbox, resolveconst challengeName = (hostname) => `${CHALLENGE_PREFIX}.${hostname}`;-- function load(viewer, id, need) {-- const domain = isId('domain', id) ? q.byId.get(id) : null;++ async function load(viewer, id, need) {++ const domain = isId('domain', id) ? await q.byId.get(id) : null;const notFound = new ApiError(404, 'domain.not_found', 'no such domain');if (!domain) throw notFound;-- const site = sites.get(domain.site_id);++ const site = await sites.get(domain.site_id);if (!site || site.status !== 'active') throw notFound;-- access.authorize(projects.get(domain.project_id), viewer, need, notFound);++ await access.authorize(await projects.get(domain.project_id), viewer, need, notFound);return { domain, site };}@@@ -109,18 +109,18 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve};}-- function add(viewer, siteId, input = {}) {-- const { site, project } = sites.load(viewer, siteId, 'maintain');++ async function add(viewer, siteId, input = {}) {++ const { site, project } = await sites.load(viewer, siteId, 'maintain');const hostname = checkHostname(input.hostname);if (project.environment === 'sandbox') throw new ApiError(403, 'domain.sandbox', 'sandbox projects are served on their default domain only');-- const quota = projects.quotaOf(project);-- if (projects.usageOf(project).customDomains >= quota.customDomains) throw new ApiError(429, 'quota.custom_domains', `this project may have ${quota.customDomains} custom domains`);-- if (q.onSite.get(site.id, hostname)) throw new ApiError(409, 'domain.exists', `${hostname} is already on this site`);-- if (q.verifiedByHost.get(hostname)) throw new ApiError(409, 'domain.taken', `${hostname} is already verified for another site`);++ const quota = await projects.quotaOf(project);++ if ((await projects.usageOf(project)).customDomains >= quota.customDomains) throw new ApiError(429, 'quota.custom_domains', `this project may have ${quota.customDomains} custom domains`);++ if (await q.onSite.get(site.id, hostname)) throw new ApiError(409, 'domain.exists', `${hostname} is already on this site`);++ if (await q.verifiedByHost.get(hostname)) throw new ApiError(409, 'domain.taken', `${hostname} is already verified for another site`);const id = newId('domain', store.now());const token = crypto.randomBytes(20).toString('hex');-- q.insert.run(id, project.id, site.id, hostname, token, principalOf(viewer), store.now());-- const domain = q.byId.get(id);++ await q.insert.run(id, project.id, site.id, hostname, token, principalOf(viewer), store.now());++ const domain = await q.byId.get(id);return { domain, site };}@@@ -143,40 +143,40 @@ function createDomains({ store, config, access, projects, sites, outbox, resolveif (domain.last_checked_at && now - domain.last_checked_at < MIN_RECHECK_MS) return domain;const r = await lookup(domain.hostname);const expected = `${TXT_PREFIX}${domain.token}`;-- if (!r.ok) { q.checked.run(store.now(), r.error, domain.id); return q.byId.get(domain.id); }++ if (!r.ok) { await q.checked.run(store.now(), r.error, domain.id); return await q.byId.get(domain.id); }if (!r.values.includes(expected)) {-- q.checked.run(store.now(), r.values.length ? `${challengeName(domain.hostname)} has TXT records, but not the expected value` : `no TXT record at ${challengeName(domain.hostname)} yet`, domain.id);-- return q.byId.get(domain.id);++ await q.checked.run(store.now(), r.values.length ? `${challengeName(domain.hostname)} has TXT records, but not the expected value` : `no TXT record at ${challengeName(domain.hostname)} yet`, domain.id);++ return await q.byId.get(domain.id);}try {-- store.tx(() => {-- if (q.verifiedByHost.get(domain.hostname)) throw new ApiError(409, 'domain.taken', `${domain.hostname} was verified for another site first`);-- const site = sites.get(domain.site_id);++ await store.tx(async () => {++ if (await q.verifiedByHost.get(domain.hostname)) throw new ApiError(409, 'domain.taken', `${domain.hostname} was verified for another site first`);++ const site = await sites.get(domain.site_id);if (!site || site.status !== 'active') throw new ApiError(404, 'domain.not_found', 'no such domain');-- if (q.verified.run(store.now(), store.now(), domain.id).changes !== 1) return;-- outbox.emit({++ if ((await q.verified.run(store.now(), store.now(), domain.id)).changes !== 1) return;++ await outbox.emit({event_type: 'host.domain.verified', actor: viewer ? actorRef(viewer) : { type: 'service', id: 'host' }, visibility: 'internal', priority: 'low',subject: { type: 'domain', id: domain.id },payload: { project_id: domain.project_id, site_id: domain.site_id, site: site.name, hostname: domain.hostname },}, { traceparent });});} catch (err) {-- if (err instanceof ApiError && err.code === 'domain.taken') { q.setStatus.run('failed', err.message, domain.id); return q.byId.get(domain.id); }++ if (err instanceof ApiError && err.code === 'domain.taken') { await q.setStatus.run('failed', err.message, domain.id); return await q.byId.get(domain.id); }throw err;}outbox.kick();-- return q.byId.get(domain.id);++ return await q.byId.get(domain.id);}async function verify(viewer, id, opts = {}) {-- const { domain } = load(viewer, id, 'maintain');-- return verifyDomain(domain, viewer, opts);++ const { domain } = await load(viewer, id, 'maintain');++ return await verifyDomain(domain, viewer, opts);}-- function remove(viewer, id) {-- const { domain } = load(viewer, id, 'maintain');++ async function remove(viewer, id) {++ const { domain } = await load(viewer, id, 'maintain');if (domain.kind === 'default') throw new ApiError(409, 'domain.default', 'the default domain goes away only with the site');-- q.remove.run(domain.id);++ await q.remove.run(domain.id);return { deleted: true };}@@@ -184,9 +184,9 @@ function createDomains({ store, config, access, projects, sites, outbox, resolveasync function recheck({ limit = 50 } = {}) {const now = store.now();const summary = { verified: 0, failed: 0, lapsed: 0, checked: 0 };-- for (const d of q.duePending.all(now - config.domains.recheckIntervalMs + 1000, limit)) {++ for (const d of await q.duePending.all(now - config.domains.recheckIntervalMs + 1000, limit)) {if (now - d.created_at > config.domains.pendingDays * DAY) {-- q.setStatus.run('failed', `not verified within ${config.domains.pendingDays} days`, d.id);++ await q.setStatus.run('failed', `not verified within ${config.domains.pendingDays} days`, d.id);summary.failed++;continue;}@@@ -194,15 +194,15 @@ function createDomains({ store, config, access, projects, sites, outbox, resolvesummary.checked++;if (after.status === 'verified') summary.verified++;}-- for (const d of q.dueVerified.all(now - DAY, limit)) {++ for (const d of await q.dueVerified.all(now - DAY, limit)) {const r = await lookup(d.hostname);summary.checked++;-- if (!r.ok) { q.checked.run(store.now(), r.error, d.id); continue; }-- if (r.values.includes(`${TXT_PREFIX}${d.token}`)) { q.present.run(store.now(), d.id); continue; }-- q.missing.run(store.now(), store.now(), `the TXT record at ${challengeName(d.hostname)} is gone`, d.id);-- const row = q.byId.get(d.id);++ if (!r.ok) { await q.checked.run(store.now(), r.error, d.id); continue; }++ if (r.values.includes(`${TXT_PREFIX}${d.token}`)) { await q.present.run(store.now(), d.id); continue; }++ await q.missing.run(store.now(), store.now(), `the TXT record at ${challengeName(d.hostname)} is gone`, d.id);++ const row = await q.byId.get(d.id);if (row.record_missing_since && store.now() - row.record_missing_since > config.domains.lapseDays * DAY) {-- q.setStatus.run('lapsed', `the TXT record has been gone for more than ${config.domains.lapseDays} days; the domain is no longer served`, d.id);++ await q.setStatus.run('lapsed', `the TXT record has been gone for more than ${config.domains.lapseDays} days; the domain is no longer served`, d.id);summary.lapsed++;log.warn(`[Host] custom domain ${d.hostname} lapsed (TXT record gone)`);}@@@ -212,9 +212,9 @@ function createDomains({ store, config, access, projects, sites, outbox, resolvereturn {load, add, verify, verifyDomain, remove, recheck, instructions, checkHostname,-- listForSite: (siteId) => q.forSite.all(siteId),-- get: (id) => (isId('domain', id) ? q.byId.get(id) || null : null),-- verifiedCustom: () => db.prepare("SELECT hostname, site_id FROM host_domains WHERE kind = 'custom' AND status = 'verified' ORDER BY hostname").all(),++ listForSite: async (siteId) => await q.forSite.all(siteId),++ get: async (id) => (isId('domain', id) ? await q.byId.get(id) || null : null),++ verifiedCustom: async () => await db.prepare("SELECT hostname, site_id FROM host_domains WHERE kind = 'custom' AND status = 'verified' ORDER BY hostname").all(),challengeName, TXT_PREFIX,};}@@@ -23,8 +23,8 @@ function createProjects({ store, config, access, blobs, takedowns, log = consoleconst q = {byId: db.prepare('SELECT * FROM host_projects WHERE id = ?'),forPrincipal: db.prepare(`SELECT p.*, m.role FROM host_projects p JOIN host_project_members m ON m.project_id = p.id-- WHERE m.principal = ? AND p.status = 'active' ORDER BY p.created_at DESC, p.rowid DESC LIMIT 200`),-- all: db.prepare("SELECT p.*, NULL AS role FROM host_projects p WHERE p.status = 'active' ORDER BY p.created_at DESC, p.rowid DESC LIMIT 200"),++ WHERE m.principal = ? AND p.status = 'active' ORDER BY p.created_at DESC, p.seq DESC LIMIT 200`),++ all: db.prepare("SELECT p.*, NULL AS role FROM host_projects p WHERE p.status = 'active' ORDER BY p.created_at DESC, p.seq DESC LIMIT 200"),ownedCount: db.prepare("SELECT COUNT(*) AS n FROM host_projects WHERE owner_subject = ? AND status = 'active'"),insert: db.prepare(`INSERT INTO host_projects (id, network_project_id, owner_subject, name, environment, status, created_at, updated_at)VALUES (?, ?, ?, ?, ?, 'active', ?, ?)`),@@@ -33,38 +33,38 @@ function createProjects({ store, config, access, blobs, takedowns, log = consoleremoveMember: db.prepare('DELETE FROM host_project_members WHERE project_id = ? AND principal = ?'),members: db.prepare('SELECT principal, role, added_by, created_at FROM host_project_members WHERE project_id = ? ORDER BY created_at'),quotaRow: db.prepare('SELECT * FROM host_quotas WHERE project_id = ?'),-- storageUsed: db.prepare('SELECT COALESCE(SUM(size), 0) AS n, COUNT(*) AS c FROM host_blobs WHERE project_id = ?'),++ storageUsed: db.prepare('SELECT COALESCE(SUM(size), 0)::bigint AS n, COUNT(*) AS c FROM host_blobs WHERE project_id = ?'),deploysSince: db.prepare('SELECT COUNT(*) AS n FROM host_deploys WHERE project_id = ? AND created_at > ?'),siteCount: db.prepare("SELECT COUNT(*) AS n FROM host_sites WHERE project_id = ? AND status = 'active'"),customDomainCount: db.prepare("SELECT COUNT(*) AS n FROM host_domains WHERE project_id = ? AND kind = 'custom' AND status IN ('pending','verified')"),markDeleted: db.prepare("UPDATE host_projects SET status = 'deleted', deleted_at = ?, updated_at = ? WHERE id = ?"),};-- function get(id) {-- return isId('project', id) ? q.byId.get(id) || null : null;++ async function get(id) {++ return isId('project', id) ? await q.byId.get(id) || null : null;}/** The effective quota: the project's override where set, else the environment's default. */-- function quotaOf(project) {++ async function quotaOf(project) {const defaults = config.quotas[project.environment] || config.quotas.production;-- const row = q.quotaRow.get(project.id) || {};++ const row = await q.quotaRow.get(project.id) || {};const out = {};for (const [col, key] of Object.entries(QUOTA_FIELDS)) out[key] = row[col] != null ? row[col] : defaults[key];return out;}-- function usageOf(project) {-- const s = q.storageUsed.get(project.id);++ async function usageOf(project) {++ const s = await q.storageUsed.get(project.id);return {storageBytes: s.n,objects: s.c,-- deploysLast24h: q.deploysSince.get(project.id, store.now() - DAY).n,-- sites: q.siteCount.get(project.id).n,-- customDomains: q.customDomainCount.get(project.id).n,++ deploysLast24h: (await q.deploysSince.get(project.id, store.now() - DAY)).n,++ sites: (await q.siteCount.get(project.id)).n,++ customDomains: (await q.customDomainCount.get(project.id)).n,};}-- function create(viewer, input = {}) {++ async function create(viewer, input = {}) {const owner = principalOf(viewer);if (!owner || !ids.isSubjectId('user', owner)) {throw new ApiError(403, 'project.owner_must_be_user', 'a project is owned by a person (usr_…): an app or service must name the owner with X-OV-Subject');@@@ -82,51 +82,51 @@ function createProjects({ store, config, access, blobs, takedowns, log = consoleif (!NETWORK_PROJECT_RE.test(networkProjectId)) throw new ApiError(422, 'project.invalid_network_project_id', 'network_project_id is not a project id');}const maxOwned = (config.projects && config.projects.maxPerOwner) || 10;-- if (q.ownedCount.get(owner).n >= maxOwned) throw new ApiError(429, 'quota.projects', `you already own ${maxOwned} projects`);++ if ((await q.ownedCount.get(owner)).n >= maxOwned) throw new ApiError(429, 'quota.projects', `you already own ${maxOwned} projects`);const id = newId('project', store.now());const now = store.now();try {-- store.tx(() => {-- q.insert.run(id, networkProjectId, owner, name, environment, now, now);-- q.addMember.run(id, owner, 'owner', owner, now);++ await store.tx(async () => {++ await q.insert.run(id, networkProjectId, owner, name, environment, now, now);++ await q.addMember.run(id, owner, 'owner', owner, now);});} catch (err) {-- if (/UNIQUE/.test(err.message)) throw new ApiError(409, 'project.network_project_taken', 'that Network project already has a Host project');++ if (err.code === '23505') throw new ApiError(409, 'project.network_project_taken', 'that Network project already has a Host project');throw err;}-- return get(id);++ return await get(id);}-- function listFor(viewer) {-- if (viewer.kind === 'user' && viewer.staff && viewer.allProjects) return q.all.all();++ async function listFor(viewer) {++ if (viewer.kind === 'user' && viewer.staff && viewer.allProjects) return await q.all.all();const p = principalOf(viewer);-- return p ? q.forPrincipal.all(p) : [];++ return p ? await q.forPrincipal.all(p) : [];}-- function members(project) {-- return q.members.all(project.id);++ async function members(project) {++ return await q.members.all(project.id);}-- function setMember(viewer, project, principal, role) {-- access.authorize(project, viewer, 'own');++ async function setMember(viewer, project, principal, role) {++ await access.authorize(project, viewer, 'own');if (!PRINCIPAL_RE.test(String(principal))) throw new ApiError(422, 'member.invalid_principal', 'a member is a usr_ subject, an app:app_ principal or a svc: principal');if (!['maintainer', 'deployer', 'owner'].includes(role)) throw new ApiError(422, 'member.invalid_role', 'role is owner, maintainer or deployer');if (role === 'owner' && !principal.startsWith('usr_')) throw new ApiError(422, 'member.invalid_role', 'only a person can be an owner');if (principal === project.owner_subject && role !== 'owner') throw new ApiError(409, 'member.owner_fixed', 'the project owner stays owner');-- q.addMember.run(project.id, principal, role, principalOf(viewer), store.now());-- return members(project);++ await q.addMember.run(project.id, principal, role, principalOf(viewer), store.now());++ return await members(project);}-- function removeMember(viewer, project, principal) {-- access.authorize(project, viewer, 'own');++ async function removeMember(viewer, project, principal) {++ await access.authorize(project, viewer, 'own');if (principal === project.owner_subject) throw new ApiError(409, 'member.owner_fixed', 'the project owner cannot be removed');-- q.removeMember.run(project.id, principal);-- return members(project);++ await q.removeMember.run(project.id, principal);++ return await members(project);}-- function setQuota(viewer, project, input = {}) {-- access.authorize(project, viewer, 'staff');-- const row = q.quotaRow.get(project.id) || {};++ async function setQuota(viewer, project, input = {}) {++ await access.authorize(project, viewer, 'staff');++ const row = await q.quotaRow.get(project.id) || {};const next = {};for (const col of Object.keys(QUOTA_FIELDS)) {if (!(col in input)) { next[col] = row[col] != null ? row[col] : null; continue; }@@@ -135,27 +135,27 @@ function createProjects({ store, config, access, blobs, takedowns, log = consoleif (!Number.isSafeInteger(v) || v < 0) throw new ApiError(422, 'quota.invalid', `${col} must be a non-negative integer or null (the default)`);next[col] = v;}-- db.prepare(`INSERT INTO host_quotas (project_id, storage_bytes, deploys_per_day, max_files, max_file_bytes, sites, custom_domains, updated_by, updated_at)++ await db.prepare(`INSERT INTO host_quotas (project_id, storage_bytes, deploys_per_day, max_files, max_file_bytes, sites, custom_domains, updated_by, updated_at)VALUES (@project_id, @storage_bytes, @deploys_per_day, @max_files, @max_file_bytes, @sites, @custom_domains, @updated_by, @updated_at)ON CONFLICT (project_id) DO UPDATE SET storage_bytes = excluded.storage_bytes, deploys_per_day = excluded.deploys_per_day,max_files = excluded.max_files, max_file_bytes = excluded.max_file_bytes, sites = excluded.sites,custom_domains = excluded.custom_domains, updated_by = excluded.updated_by, updated_at = excluded.updated_at`).run({ project_id: project.id, ...next, updated_by: principalOf(viewer), updated_at: store.now() });-- return quotaOf(project);++ return await quotaOf(project);}/** Owner (or staff) deletes the project: every site stops serving at once; objects are removed. */-- function remove(viewer, project, { sites }) {-- access.authorize(project, viewer, 'own');-- takedowns.assertDeletable(viewer, { projectId: project.id });++ async function remove(viewer, project, { sites }) {++ await access.authorize(project, viewer, 'own');++ await takedowns.assertDeletable(viewer, { projectId: project.id });const now = store.now();-- store.tx(() => {-- for (const s of sites.listForProject(project.id)) sites.markDeleted(s, now);-- db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE project_id = ? AND state <> 'deleted'").run(now, project.id);-- db.prepare("DELETE FROM host_deploy_files WHERE deploy_id IN (SELECT id FROM host_deploys WHERE project_id = ?)").run(project.id);-- db.prepare('DELETE FROM host_blobs WHERE project_id = ?').run(project.id);-- db.prepare('DELETE FROM host_domains WHERE project_id = ?').run(project.id);-- q.markDeleted.run(now, now, project.id);++ await store.tx(async () => {++ for (const s of await sites.listForProject(project.id)) await sites.markDeleted(s, now);++ await db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE project_id = ? AND state <> 'deleted'").run(now, project.id);++ await db.prepare("DELETE FROM host_deploy_files WHERE deploy_id IN (SELECT id FROM host_deploys WHERE project_id = ?)").run(project.id);++ await db.prepare('DELETE FROM host_blobs WHERE project_id = ?').run(project.id);++ await db.prepare('DELETE FROM host_domains WHERE project_id = ?').run(project.id);++ await q.markDeleted.run(now, now, project.id);});try { blobs.removeProject(project.id); } catch (err) { log.warn('[Host] could not remove project objects:', err.message); }return { deleted: true };@@@ -28,7 +28,7 @@ function createSites({ store, config, access, projects, takedowns }) {byId: db.prepare('SELECT * FROM host_sites WHERE id = ?'),byName: db.prepare("SELECT * FROM host_sites WHERE name = ? AND status = 'active'"),held: db.prepare("SELECT project_id FROM host_sites WHERE name = ? AND status = 'deleted' AND deleted_at > ? ORDER BY deleted_at DESC LIMIT 1"),-- forProject: db.prepare("SELECT * FROM host_sites WHERE project_id = ? AND status = 'active' ORDER BY created_at, rowid"),++ forProject: db.prepare("SELECT * FROM host_sites WHERE project_id = ? AND status = 'active' ORDER BY created_at, seq"),insert: db.prepare(`INSERT INTO host_sites (id, project_id, name, active_deploy_id, status, created_by, created_at, updated_at)VALUES (?, ?, ?, NULL, 'active', ?, ?, ?)`),insertDefaultDomain: db.prepare(`INSERT INTO host_domains (id, project_id, site_id, hostname, kind, status, token, created_by, created_at, verified_at)@@@ -38,17 +38,17 @@ function createSites({ store, config, access, projects, takedowns }) {const defaultHostname = (name) => `${name}.${config.sitesDomain}`;-- function get(id) {-- return isId('site', id) ? q.byId.get(id) || null : null;++ async function get(id) {++ return isId('site', id) ? await q.byId.get(id) || null : null;}/** The site and its project, authorized at `need`; 404 for anyone without access. */-- function load(viewer, id, need) {-- const site = get(id);++ async function load(viewer, id, need) {++ const site = await get(id);const notFound = new ApiError(404, 'site.not_found', 'no such site');if (!site || site.status !== 'active') throw notFound;-- const project = projects.get(site.project_id);-- const role = access.authorize(project, viewer, need, notFound);++ const project = await projects.get(site.project_id);++ const role = await access.authorize(project, viewer, need, notFound);return { site, project, role };}@@@ -59,56 +59,56 @@ function createSites({ store, config, access, projects, takedowns }) {if (reserved.has(name) || /^xn--/.test(name) || /^openvibe/.test(name)) throw new ApiError(422, 'site.name_reserved', `"${name}" is reserved`);}-- function create(viewer, project, input = {}) {-- access.authorize(project, viewer, 'maintain');++ async function create(viewer, project, input = {}) {++ await access.authorize(project, viewer, 'maintain');const name = String(input.name || '').trim().toLowerCase();checkName(name);-- const quota = projects.quotaOf(project);-- if (projects.usageOf(project).sites >= quota.sites) throw new ApiError(429, 'quota.sites', `this project may have ${quota.sites} sites`);-- if (q.byName.get(name)) throw new ApiError(409, 'site.name_taken', `${defaultHostname(name)} is taken`);-- const held = q.held.get(name, store.now() - HOLD_MS);++ const quota = await projects.quotaOf(project);++ if ((await projects.usageOf(project)).sites >= quota.sites) throw new ApiError(429, 'quota.sites', `this project may have ${quota.sites} sites`);++ if (await q.byName.get(name)) throw new ApiError(409, 'site.name_taken', `${defaultHostname(name)} is taken`);++ const held = await q.held.get(name, store.now() - HOLD_MS);if (held && held.project_id !== project.id) throw new ApiError(409, 'site.name_held', `${defaultHostname(name)} was deleted recently and is held for 30 days`);const id = newId('site', store.now());const now = store.now();const who = principalOf(viewer);try {-- store.tx(() => {-- q.insert.run(id, project.id, name, who, now, now);-- q.insertDefaultDomain.run(newId('domain', now), project.id, id, defaultHostname(name), who, now, now);++ await store.tx(async () => {++ await q.insert.run(id, project.id, name, who, now, now);++ await q.insertDefaultDomain.run(newId('domain', now), project.id, id, defaultHostname(name), who, now, now);});} catch (err) {-- if (/UNIQUE/.test(err.message)) throw new ApiError(409, 'site.name_taken', `${defaultHostname(name)} is taken`);++ if (err.code === '23505') throw new ApiError(409, 'site.name_taken', `${defaultHostname(name)} is taken`);throw err;}-- return get(id);++ return await get(id);}-- function listForProject(projectId) {-- return q.forProject.all(projectId);++ async function listForProject(projectId) {++ return await q.forProject.all(projectId);}/** Inside a transaction: the site stops serving immediately; deploys and domains go with it. */-- function markDeleted(site, now) {-- q.markDeleted.run(now, now, site.id);-- db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE site_id = ? AND state <> 'deleted'").run(now, site.id);-- db.prepare('DELETE FROM host_deploy_files WHERE deploy_id IN (SELECT id FROM host_deploys WHERE site_id = ?)').run(site.id);-- db.prepare('DELETE FROM host_domains WHERE site_id = ?').run(site.id);++ async function markDeleted(site, now) {++ await q.markDeleted.run(now, now, site.id);++ await db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE site_id = ? AND state <> 'deleted'").run(now, site.id);++ await db.prepare('DELETE FROM host_deploy_files WHERE deploy_id IN (SELECT id FROM host_deploys WHERE site_id = ?)').run(site.id);++ await db.prepare('DELETE FROM host_domains WHERE site_id = ?').run(site.id);}-- function remove(viewer, id, { deploys }) {-- const { site, project } = load(viewer, id, 'maintain');-- takedowns.assertDeletable(viewer, { site });++ async function remove(viewer, id, { deploys }) {++ const { site, project } = await load(viewer, id, 'maintain');++ await takedowns.assertDeletable(viewer, { site });const now = store.now();let freed = [];-- store.tx(() => {-- markDeleted(site, now);-- freed = deploys.collectGarbage(project.id);++ await store.tx(async () => {++ await markDeleted(site, now);++ freed = await deploys.collectGarbage(project.id);});deploys.unlinkBlobs(project.id, freed);return { deleted: true };}-- return { get, load, create, listForProject, markDeleted, remove, defaultHostname, checkName, byName: (n) => q.byName.get(n) || null };++ return { get, load, create, listForProject, markDeleted, remove, defaultHostname, checkName, byName: async (n) => await q.byName.get(n) || null };}module.exports = { createSites, RESERVED, NAME_RE };@@@ -34,20 +34,20 @@ function createTakedowns({ store }) {const shape = (row) => (row ? { scope: row.target_kind, reason: row.reason, since: row.created_at } : null);/** The takedown that covers a site (its own, or its project's), or null. */-- const ofSite = (site) => shape(q.forSite.get(site.id, site.project_id));++ const ofSite = async (site) => shape(await q.forSite.get(site.id, site.project_id));/** The project-wide takedown, or null. */-- const ofProject = (projectId) => shape(q.forProject.get(projectId));++ const ofProject = async (projectId) => shape(await q.forProject.get(projectId));/** Members may not publish into, or change, content that is taken down. */-- function assertOpen(site) {-- const t = ofSite(site);++ async function assertOpen(site) {++ const t = await ofSite(site);if (t) throw new ApiError(403, 'site.taken_down', `this ${t.scope} was taken down by OpenVibe staff (${t.reason}); nothing can be published or changed until staff lift it`);}/** Deleting taken-down content would destroy what staff are reviewing: staff only. */-- function assertDeletable(viewer, { site = null, projectId = null }) {++ async function assertDeletable(viewer, { site = null, projectId = null }) {if (viewer && viewer.staff) return;-- const row = site ? q.forSite.get(site.id, site.project_id) : q.anyInProject.get({ p: projectId });++ const row = site ? await q.forSite.get(site.id, site.project_id) : await q.anyInProject.get({ p: projectId });if (row) throw new ApiError(409, 'site.taken_down', `this ${row.target_kind} was taken down by OpenVibe staff and is kept for review; it cannot be deleted until staff lift the takedown`);}@@@ -56,22 +56,22 @@ function createTakedowns({ store }) {}/** kind: 'site' | 'project'. -> the takedown */-- function takeDown(viewer, kind, targetId, reasonRaw) {++ async function takeDown(viewer, kind, targetId, reasonRaw) {requireStaff(viewer);const reason = String(reasonRaw || '').trim();if (!reason || reason.length > REASON_MAX) throw new ApiError(422, 'takedown.reason_required', `a takedown needs a reason of 1–${REASON_MAX} characters (members see it)`);try {-- q.insert.run(kind, targetId, reason, principalOf(viewer), store.now());++ await q.insert.run(kind, targetId, reason, principalOf(viewer), store.now());} catch (err) {-- if (/UNIQUE/.test(err.message)) throw new ApiError(409, 'takedown.exists', `this ${kind} is already taken down`);++ if (err.code === '23505') throw new ApiError(409, 'takedown.exists', `this ${kind} is already taken down`);throw err;}return { scope: kind, reason, since: store.now() };}-- function lift(viewer, kind, targetId, note) {++ async function lift(viewer, kind, targetId, note) {requireStaff(viewer);-- const r = q.lift.run(principalOf(viewer), store.now(), note == null ? null : String(note).slice(0, REASON_MAX), kind, targetId);++ const r = await q.lift.run(principalOf(viewer), store.now(), note == null ? null : String(note).slice(0, REASON_MAX), kind, targetId);if (r.changes !== 1) throw new ApiError(404, 'takedown.not_found', `this ${kind} is not taken down`);return { lifted: true };}@@@ -15,7 +15,7 @@*/const { createClient } = require('openvibe-sdk/core');const { createServiceTokenClient } = require('openvibe-sdk/auth');--const { createEventsClient, createOutbox } = require('openvibe-sdk/events');++const { createEventsClient, createPgOutbox } = require('openvibe-sdk/events');const EVENT_TYPES = Object.freeze(['host.deploy.created', 'host.deploy.activated', 'host.deploy.failed', 'host.domain.verified']);@@@ -33,7 +33,9 @@ function createHostOutbox({ db, config, fetchImpl, now, log = console }) {}const events = createEventsClient(createClient(clientOpts), { source: 'host' });let lastError = null;-- const outbox = createOutbox(db, {++ // The PostgreSQL outbox: rows are written in the change's own transaction (enqueue(db, …) joins the ambient++ // transaction); several processes relay one table safely (leases).++ const outbox = createPgOutbox(db, {events,intervalMs: config.events.intervalMs,now,@@@ -43,12 +45,11 @@ function createHostOutbox({ db, config, fetchImpl, now, log = console }) {lastError = msg;},});-- outbox.ensureSchema();/** Inside the caller's transaction. Returns the complete envelope (with its event_id). */-- function emit(envelope, { traceparent } = {}) {++ async function emit(envelope, { traceparent } = {}) {if (!EVENT_TYPES.includes(envelope.event_type)) throw new Error(`undeclared event type ${envelope.event_type}`);-- return outbox.enqueue(envelope, { traceparent });++ return await outbox.enqueue(db, envelope, { traceparent });}return {@@@ -58,7 +59,7 @@ function createHostOutbox({ db, config, fetchImpl, now, log = console }) {start() { if (enabled) outbox.start(); },stop: () => outbox.stop(),kick() { if (enabled) outbox.kick(); },-- status: () => ({ enabled, pending: outbox.pending(), rejected: outbox.rejected(), last_error: lastError }),++ status: async () => ({ enabled, pending: await outbox.pending(), rejected: await outbox.rejected(), last_error: lastError }),};}@@@ -10,7 +10,7 @@*/'use strict';--const { createActorLimiter } = require('openvibe-sdk/limits');++const { createActorLimiter, createValkeyLimitStore } = require('openvibe-sdk/limits');const num = (v, d) => { const n = parseInt(v, 10); return Number.isFinite(n) && n > 0 ? n : d; };@@@ -21,7 +21,7 @@ const ROUTES = [['host.project.create', /^POST$/, /^\/projects\/?$/, { minute: 5, hour: 30 }],];--function createHostActorLimits({ env = process.env, registry = null, now } = {}) {++function createHostActorLimits({ env = process.env, registry = null, now, valkey = null } = {}) {// HOST_ACTOR_LIMITS=off: a rollback lever (and what the Stage B suites use: they deploy dozens of times a minute).if (/^(off|0|false)$/i.test(String(env.HOST_ACTOR_LIMITS || ''))) return function noLimits(req, res, next) { next(); };const refused = registry && typeof registry.counter === 'function'@@@ -38,6 +38,8 @@ function createHostActorLimits({ env = process.env, registry = null, now } = {})limits: { minute: num(env.HOST_LIMITS_MINUTE, 120), hour: num(env.HOST_LIMITS_HOUR, 3000) },actor,...(now ? { now } : {}),++ // Shared across processes on Valkey (ADR-035) when VALKEY_URL is set; in-process otherwise.++ ...(valkey ? { store: createValkeyLimitStore(valkey) } : {}),onLimited(e) {console.warn(`[Limits] ${e.name}: ${e.actor} refused, over ${e.limit} per ${e.window}`);if (refused) refused.inc({ limit: e.name, window: e.window });@@@ -49,103 +49,103 @@ function createApi(ctx) {const u = new URL(config.baseUrl);return `${u.protocol}//${hostname}${u.port ? `:${u.port}` : ''}`;};-- const siteOut = (s) => { const hostname = sites.defaultHostname(s.name); return out.site(s, { hostname, url: siteUrl(hostname), takedown: takedowns.ofSite(s) }); };-- const projectOut = (p, role) => out.project(p, { role, quota: projects.quotaOf(p), usage: projects.usageOf(p), takedown: takedowns.ofProject(p.id) });++ const siteOut = async (s) => { const hostname = sites.defaultHostname(s.name); return out.site(s, { hostname, url: siteUrl(hostname), takedown: await takedowns.ofSite(s) }); };++ const projectOut = async (p, role) => out.project(p, { role, quota: await projects.quotaOf(p), usage: await projects.usageOf(p), takedown: await takedowns.ofProject(p.id) });const tp = (req) => (req.ov ? req.ov.traceparent : undefined);-- const loadProject = (req, need) => {-- const project = projects.get(req.params.id);-- const role = ctx.access.authorize(project, req.viewer, need);++ const loadProject = async (req, need) => {++ const project = await projects.get(req.params.id);++ const role = await ctx.access.authorize(project, req.viewer, need);return { project, role };};// ── Projects ────────────────────────────────────────────-- router.get('/projects', guard('host.site.manage'), run((req) => {++ router.get('/projects', guard('host.site.manage'), run(async (req) => {if (req.viewer.kind === 'anonymous') throw new ApiError(401, 'auth.required', 'sign in with OpenVibe, or present a service token');-- return { projects: projects.listFor(req.viewer).map((p) => out.project(p, { takedown: takedowns.ofProject(p.id) })) };++ return { projects: (await Promise.all((await projects.listFor(req.viewer)).map(async (p) => out.project(p, { takedown: await takedowns.ofProject(p.id) })))) };}));-- router.post('/projects', guard('host.site.manage'), jsonBody, run((req) => {++ router.post('/projects', guard('host.site.manage'), jsonBody, run(async (req) => {if (req.viewer.kind === 'anonymous') throw new ApiError(401, 'auth.required', 'sign in with OpenVibe, or present a service token');-- const p = projects.create(req.viewer, req.body || {});-- return { project: projectOut(p, 'owner') };++ const p = await projects.create(req.viewer, req.body || {});++ return { project: await projectOut(p, 'owner') };}, 201));-- router.get('/projects/:id', guard('host.site.manage'), run((req) => {-- const { project, role } = loadProject(req, 'read');-- return { project: projectOut(project, role), sites: sites.listForProject(project.id).map(siteOut), members: projects.members(project).map((m) => ({ principal: m.principal, role: m.role, added_at: out.iso(m.created_at) })) };-- }));-- router.delete('/projects/:id', guard('host.site.manage'), run((req) => {-- const { project } = loadProject(req, 'read');-- return projects.remove(req.viewer, project, { sites });-- }));-- router.get('/projects/:id/quota', guard('host.site.manage'), run((req) => {-- const { project } = loadProject(req, 'read');-- return { quota: out.quotaOut(projects.quotaOf(project)), usage: out.usageOut(projects.usageOf(project)) };-- }));-- router.put('/projects/:id/quota', guard('host.site.manage'), jsonBody, run((req) => {-- const project = projects.get(req.params.id);-- const q = projects.setQuota(req.viewer, project, req.body || {});-- return { quota: out.quotaOut(q), usage: out.usageOut(projects.usageOf(project)) };-- }));-- router.put('/projects/:id/members/:principal', guard('host.site.manage'), jsonBody, run((req) => {-- const { project } = loadProject(req, 'read');-- const list = projects.setMember(req.viewer, project, req.params.principal, String((req.body || {}).role || ''));++ router.get('/projects/:id', guard('host.site.manage'), run(async (req) => {++ const { project, role } = await loadProject(req, 'read');++ return { project: await projectOut(project, role), sites: (await Promise.all((await sites.listForProject(project.id)).map(siteOut))), members: (await projects.members(project)).map((m) => ({ principal: m.principal, role: m.role, added_at: out.iso(m.created_at) })) };++ }));++ router.delete('/projects/:id', guard('host.site.manage'), run(async (req) => {++ const { project } = await loadProject(req, 'read');++ return await projects.remove(req.viewer, project, { sites });++ }));++ router.get('/projects/:id/quota', guard('host.site.manage'), run(async (req) => {++ const { project } = await loadProject(req, 'read');++ return { quota: out.quotaOut(await projects.quotaOf(project)), usage: out.usageOut(await projects.usageOf(project)) };++ }));++ router.put('/projects/:id/quota', guard('host.site.manage'), jsonBody, run(async (req) => {++ const project = await projects.get(req.params.id);++ const q = await projects.setQuota(req.viewer, project, req.body || {});++ return { quota: out.quotaOut(q), usage: out.usageOut(await projects.usageOf(project)) };++ }));++ router.put('/projects/:id/members/:principal', guard('host.site.manage'), jsonBody, run(async (req) => {++ const { project } = await loadProject(req, 'read');++ const list = await projects.setMember(req.viewer, project, req.params.principal, String((req.body || {}).role || ''));return { members: list.map((m) => ({ principal: m.principal, role: m.role, added_at: out.iso(m.created_at) })) };}));-- router.delete('/projects/:id/members/:principal', guard('host.site.manage'), run((req) => {-- const { project } = loadProject(req, 'read');-- const list = projects.removeMember(req.viewer, project, req.params.principal);++ router.delete('/projects/:id/members/:principal', guard('host.site.manage'), run(async (req) => {++ const { project } = await loadProject(req, 'read');++ const list = await projects.removeMember(req.viewer, project, req.params.principal);return { members: list.map((m) => ({ principal: m.principal, role: m.role, added_at: out.iso(m.created_at) })) };}));// ── Takedowns (staff only; members see them on the project and site) ──// Serving stops at once and the content is kept for review (domain/takedowns.js).-- router.post('/projects/:id/takedown', guard('host.site.manage'), jsonBody, run((req) => {-- const { project } = loadProject(req, 'read');-- return { takedown: out.takedownOut(takedowns.takeDown(req.viewer, 'project', project.id, (req.body || {}).reason)) };++ router.post('/projects/:id/takedown', guard('host.site.manage'), jsonBody, run(async (req) => {++ const { project } = await loadProject(req, 'read');++ return { takedown: out.takedownOut(await takedowns.takeDown(req.viewer, 'project', project.id, (req.body || {}).reason)) };}, 201));-- router.delete('/projects/:id/takedown', guard('host.site.manage'), jsonBody, run((req) => {-- const { project } = loadProject(req, 'read');-- return takedowns.lift(req.viewer, 'project', project.id, (req.body || {}).note);++ router.delete('/projects/:id/takedown', guard('host.site.manage'), jsonBody, run(async (req) => {++ const { project } = await loadProject(req, 'read');++ return await takedowns.lift(req.viewer, 'project', project.id, (req.body || {}).note);}));-- router.post('/sites/:id/takedown', guard('host.site.manage'), jsonBody, run((req) => {-- const { site } = sites.load(req.viewer, req.params.id, 'read');-- return { takedown: out.takedownOut(takedowns.takeDown(req.viewer, 'site', site.id, (req.body || {}).reason)) };++ router.post('/sites/:id/takedown', guard('host.site.manage'), jsonBody, run(async (req) => {++ const { site } = await sites.load(req.viewer, req.params.id, 'read');++ return { takedown: out.takedownOut(await takedowns.takeDown(req.viewer, 'site', site.id, (req.body || {}).reason)) };}, 201));-- router.delete('/sites/:id/takedown', guard('host.site.manage'), jsonBody, run((req) => {-- const { site } = sites.load(req.viewer, req.params.id, 'read');-- return takedowns.lift(req.viewer, 'site', site.id, (req.body || {}).note);++ router.delete('/sites/:id/takedown', guard('host.site.manage'), jsonBody, run(async (req) => {++ const { site } = await sites.load(req.viewer, req.params.id, 'read');++ return await takedowns.lift(req.viewer, 'site', site.id, (req.body || {}).note);}));// ── Sites ───────────────────────────────────────────────-- router.get('/projects/:id/sites', guard('host.site.manage'), run((req) => {-- const { project } = loadProject(req, 'read');-- return { sites: sites.listForProject(project.id).map(siteOut) };-- }));-- router.post('/projects/:id/sites', guard('host.site.manage'), jsonBody, run((req) => {-- const project = projects.get(req.params.id);-- ctx.access.authorize(project, req.viewer, 'read');-- return { site: siteOut(sites.create(req.viewer, project, req.body || {})) };++ router.get('/projects/:id/sites', guard('host.site.manage'), run(async (req) => {++ const { project } = await loadProject(req, 'read');++ return { sites: (await Promise.all((await sites.listForProject(project.id)).map(siteOut))) };++ }));++ router.post('/projects/:id/sites', guard('host.site.manage'), jsonBody, run(async (req) => {++ const project = await projects.get(req.params.id);++ await ctx.access.authorize(project, req.viewer, 'read');++ return { site: await siteOut(await sites.create(req.viewer, project, req.body || {})) };}, 201));-- router.get('/sites/:id', guard('host.site.manage'), run((req) => {-- const { site } = sites.load(req.viewer, req.params.id, 'read');-- return { site: siteOut(site), domains: domains.listForSite(site.id).map((d) => out.domain(d, domains.instructions(d, site))) };++ router.get('/sites/:id', guard('host.site.manage'), run(async (req) => {++ const { site } = await sites.load(req.viewer, req.params.id, 'read');++ return { site: await siteOut(site), domains: (await domains.listForSite(site.id)).map((d) => out.domain(d, domains.instructions(d, site))) };}));-- router.delete('/sites/:id', guard('host.site.manage'), run((req) => sites.remove(req.viewer, req.params.id, { deploys })));++ router.delete('/sites/:id', guard('host.site.manage'), run(async (req) => await sites.remove(req.viewer, req.params.id, { deploys })));// ── Deploys ─────────────────────────────────────────────-- router.get('/sites/:id/deploys', guard('host.deploy.create'), run((req) => {++ router.get('/sites/:id/deploys', guard('host.deploy.create'), run(async (req) => {// Reads need the least member role (deployer); staff may read them too, e.g. to review a takedown.-- const { site } = sites.load(req.viewer, req.params.id, 'read');++ const { site } = await sites.load(req.viewer, req.params.id, 'read');return {active_deploy_id: site.active_deploy_id || null,-- deploys: deploys.list(site.id, Number(req.query.limit) || 50).map((d) => out.deploy(d, { active: d.id === site.active_deploy_id })),-- activations: deploys.activationsOf(site.id, 20).map(out.activation),++ deploys: (await deploys.list(site.id, Number(req.query.limit) || 50)).map((d) => out.deploy(d, { active: d.id === site.active_deploy_id })),++ activations: (await deploys.activationsOf(site.id, 20)).map(out.activation),};}));router.post('/sites/:id/deploys', guard('host.deploy.create'), async (req, res) => {let release = null;try {-- const pre = deploys.precheck(req.viewer, req.params.id);++ const pre = await deploys.precheck(req.viewer, req.params.id);release = ctx.uploadGate.enter();if (!release) { res.set('Retry-After', '30'); throw new ApiError(503, 'upload.busy', 'Host is validating other uploads right now; try again in 30 seconds'); }let up;@@@ -155,12 +155,12 @@ function createApi(ctx) {if (!(err instanceof UploadError)) throw err;if (err.closeConnection) res.set('Connection', 'close');const multipart = String(req.headers['content-type'] || '').startsWith('multipart/');-- const r = deploys.recordFailure(req.viewer, pre, { source: err.source || (multipart ? 'files' : 'archive'), code: err.code, problems: [{ code: err.code, message: err.message }], traceparent: tp(req) });++ const r = await deploys.recordFailure(req.viewer, pre, { source: err.source || (multipart ? 'files' : 'archive'), code: err.code, problems: [{ code: err.code, message: err.message }], traceparent: tp(req) });return contracts.http.sendProblem(res, err.status, err.code, { detail: err.message, ctx: req.ov, extra: { deploy_id: r.deploy.id, log: r.log.map((l) => `${l.level}: ${l.message}`) } });}const activate = truthy(up.fields.activate != null ? up.fields.activate : req.query.activate);-- const r = deploys.create(req.viewer, pre, up.entries, { source: up.source, activate, notes: up.notes, traceparent: tp(req) });-- const site = sites.get(pre.site.id);++ const r = await deploys.create(req.viewer, pre, up.entries, { source: up.source, activate, notes: up.notes, traceparent: tp(req) });++ const site = await sites.get(pre.site.id);res.status(201).json({ deploy: out.deploy(r.deploy, { active: site.active_deploy_id === r.deploy.id, log: r.log }), activated: Boolean(r.activated && r.activated.changed), url: siteUrl(sites.defaultHostname(site.name)) });} catch (err) {// Refused before the body was read (404, 429, 503): close instead of draining a large upload.@@@ -171,41 +171,41 @@ function createApi(ctx) {}});-- router.get('/deploys/:id', guard('host.deploy.create'), run((req) => {-- const { deploy, site } = deploys.load(req.viewer, req.params.id, 'read');-- return { deploy: out.deploy(deploy, { active: site.active_deploy_id === deploy.id, files: deploys.filesOf(deploy.id) }) };++ router.get('/deploys/:id', guard('host.deploy.create'), run(async (req) => {++ const { deploy, site } = await deploys.load(req.viewer, req.params.id, 'read');++ return { deploy: out.deploy(deploy, { active: site.active_deploy_id === deploy.id, files: await deploys.filesOf(deploy.id) }) };}));-- router.get('/deploys/:id/log', guard('host.deploy.create'), run((req) => {-- const { deploy } = deploys.load(req.viewer, req.params.id, 'read');-- return { deploy_id: deploy.id, state: deploy.state, log: deploys.logOf(deploy.id).map(out.logLine) };++ router.get('/deploys/:id/log', guard('host.deploy.create'), run(async (req) => {++ const { deploy } = await deploys.load(req.viewer, req.params.id, 'read');++ return { deploy_id: deploy.id, state: deploy.state, log: (await deploys.logOf(deploy.id)).map(out.logLine) };}));-- router.post('/deploys/:id/activate', guard('host.deploy.create'), jsonBody, run((req) => {++ router.post('/deploys/:id/activate', guard('host.deploy.create'), jsonBody, run(async (req) => {const body = req.body || {};-- const r = deploys.activate(req.viewer, req.params.id, { expectedActive: 'expected_active' in body ? body.expected_active : undefined, traceparent: tp(req) });++ const r = await deploys.activate(req.viewer, req.params.id, { expectedActive: 'expected_active' in body ? body.expected_active : undefined, traceparent: tp(req) });return { active_deploy_id: r.deploy_id, previous_deploy_id: r.previous_deploy_id, changed: r.changed };}));-- router.post('/sites/:id/rollback', guard('host.deploy.create'), jsonBody, run((req) => {++ router.post('/sites/:id/rollback', guard('host.deploy.create'), jsonBody, run(async (req) => {const body = req.body || {};-- const r = deploys.rollback(req.viewer, req.params.id, { deployId: body.deploy_id || null, expectedActive: 'expected_active' in body ? body.expected_active : undefined, traceparent: tp(req) });++ const r = await deploys.rollback(req.viewer, req.params.id, { deployId: body.deploy_id || null, expectedActive: 'expected_active' in body ? body.expected_active : undefined, traceparent: tp(req) });return { active_deploy_id: r.deploy_id, previous_deploy_id: r.previous_deploy_id, changed: r.changed };}));-- router.delete('/deploys/:id', guard('host.site.manage'), run((req) => deploys.remove(req.viewer, req.params.id)));++ router.delete('/deploys/:id', guard('host.site.manage'), run(async (req) => await deploys.remove(req.viewer, req.params.id)));// ── Domains ─────────────────────────────────────────────-- router.get('/sites/:id/domains', guard('host.domain.manage'), run((req) => {-- const { site } = sites.load(req.viewer, req.params.id, 'read');-- return { domains: domains.listForSite(site.id).map((d) => out.domain(d, domains.instructions(d, site))) };++ router.get('/sites/:id/domains', guard('host.domain.manage'), run(async (req) => {++ const { site } = await sites.load(req.viewer, req.params.id, 'read');++ return { domains: (await domains.listForSite(site.id)).map((d) => out.domain(d, domains.instructions(d, site))) };}));-- router.post('/sites/:id/domains', guard('host.domain.manage'), jsonBody, run((req) => {-- const { domain, site } = domains.add(req.viewer, req.params.id, req.body || {});++ router.post('/sites/:id/domains', guard('host.domain.manage'), jsonBody, run(async (req) => {++ const { domain, site } = await domains.add(req.viewer, req.params.id, req.body || {});return { domain: out.domain(domain, domains.instructions(domain, site)) };}, 201));router.post('/domains/:id/verify', guard('host.domain.manage'), run(async (req) => {const d = await domains.verify(req.viewer, req.params.id, { traceparent: tp(req) });-- const site = sites.get(d.site_id);++ const site = await sites.get(d.site_id);return { domain: out.domain(d, domains.instructions(d, site)) };}));-- router.delete('/domains/:id', guard('host.domain.manage'), run((req) => domains.remove(req.viewer, req.params.id)));++ router.delete('/domains/:id', guard('host.domain.manage'), run(async (req) => await domains.remove(req.viewer, req.params.id)));router.use((req, res) => contracts.http.sendProblem(res, 404, 'route.not_found', { detail: `no route ${req.method} ${req.baseUrl}${req.path}`, ctx: req.ov }));return router;@@@ -66,102 +66,102 @@ function createDashboard(ctx) {// ── Pages ───────────────────────────────────────────────// What shipped on OpenVibe.Host: the shared update log every OpenVibe site has.router.get('/updates', view((req, res) => page(req, res, 200, 'What shipped on OpenVibe.Host', frame.updatesBody({ service: 'host', siteName: 'OpenVibe.Host' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>`, null, { indexable: true })));-- router.get('/', view((req, res) => {++ router.get('/', view(async (req, res) => {// The signed-out front page is the public page of openvibe.host (sitemap.xml): indexable.// Everything behind sign-in stays noindex and private.if (req.viewer.kind !== 'user') return page(req, res, 200, null, pages.signedOut(), null, { indexable: !req.query || !Object.keys(req.query).length });-- page(req, res, 200, 'Projects', pages.home({ projects: projects.listFor(req.viewer), csrf: csrfToken(config, req.viewer) }), notice(req));++ page(req, res, 200, 'Projects', pages.home({ projects: await projects.listFor(req.viewer), csrf: csrfToken(config, req.viewer) }), notice(req));}));-- router.get('/projects/:id', view((req, res) => {++ router.get('/projects/:id', view(async (req, res) => {if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');-- const project = projects.get(req.params.id);-- const role = access.authorize(project, req.viewer, 'read');++ const project = await projects.get(req.params.id);++ const role = await access.authorize(project, req.viewer, 'read');page(req, res, 200, project.name, pages.project({-- project, role, sites: sites.listForProject(project.id), quota: projects.quotaOf(project), usage: projects.usageOf(project),-- members: projects.members(project), csrf: csrfToken(config, req.viewer), siteUrl,-- }), takedownNotice(ctx.takedowns.ofProject(project.id)) || notice(req));++ project, role, sites: await sites.listForProject(project.id), quota: await projects.quotaOf(project), usage: await projects.usageOf(project),++ members: await projects.members(project), csrf: csrfToken(config, req.viewer), siteUrl,++ }), takedownNotice(await ctx.takedowns.ofProject(project.id)) || notice(req));}));-- router.get('/sites/:id', view((req, res) => {++ router.get('/sites/:id', view(async (req, res) => {if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');-- const { site, project, role } = sites.load(req.viewer, req.params.id, 'read');++ const { site, project, role } = await sites.load(req.viewer, req.params.id, 'read');page(req, res, 200, site.name, pages.site({site, project, role, url: siteUrl(site), csrf: csrfToken(config, req.viewer),-- deploys: deploys.list(site.id, 50), activations: deploys.activationsOf(site.id, 10),-- domains: domains.listForSite(site.id).map((d) => ({ domain: d, instructions: domains.instructions(d, site) })),-- }), takedownNotice(ctx.takedowns.ofSite(site)) || notice(req));++ deploys: await deploys.list(site.id, 50), activations: await deploys.activationsOf(site.id, 10),++ domains: (await domains.listForSite(site.id)).map((d) => ({ domain: d, instructions: domains.instructions(d, site) })),++ }), takedownNotice(await ctx.takedowns.ofSite(site)) || notice(req));}));-- router.get('/deploys/:id', view((req, res) => {++ router.get('/deploys/:id', view(async (req, res) => {if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');-- const { deploy, site, project } = deploys.load(req.viewer, req.params.id, 'read');++ const { deploy, site, project } = await deploys.load(req.viewer, req.params.id, 'read');page(req, res, 200, `Deploy ${deploy.id.slice(0, 12)}`, pages.deploy({-- deploy, site, project, active: site.active_deploy_id === deploy.id, files: deploys.filesOf(deploy.id), log: deploys.logOf(deploy.id),++ deploy, site, project, active: site.active_deploy_id === deploy.id, files: await deploys.filesOf(deploy.id), log: await deploys.logOf(deploy.id),}), notice(req));}));// ── Actions ─────────────────────────────────────────────-- router.post('/projects', action((req) => {-- const p = projects.create(req.viewer, { name: req.body.name, environment: req.body.environment });++ router.post('/projects', action(async (req) => {++ const p = await projects.create(req.viewer, { name: req.body.name, environment: req.body.environment });return back(`/projects/${p.id}`, 'Project created.');}));-- router.post('/projects/:id/sites', action((req) => {-- const project = projects.get(req.params.id);-- access.authorize(project, req.viewer, 'read');-- const s = sites.create(req.viewer, project, { name: req.body.name });++ router.post('/projects/:id/sites', action(async (req) => {++ const project = await projects.get(req.params.id);++ await access.authorize(project, req.viewer, 'read');++ const s = await sites.create(req.viewer, project, { name: req.body.name });return back(`/sites/${s.id}`, `Site created: ${siteUrl(s)}`);}));-- router.post('/projects/:id/members', action((req) => {-- const project = projects.get(req.params.id);-- access.authorize(project, req.viewer, 'read');-- projects.setMember(req.viewer, project, String(req.body.principal || '').trim(), String(req.body.role || ''));++ router.post('/projects/:id/members', action(async (req) => {++ const project = await projects.get(req.params.id);++ await access.authorize(project, req.viewer, 'read');++ await projects.setMember(req.viewer, project, String(req.body.principal || '').trim(), String(req.body.role || ''));return back(`/projects/${project.id}`, 'Member saved.');}));-- router.post('/projects/:id/members/remove', action((req) => {-- const project = projects.get(req.params.id);-- access.authorize(project, req.viewer, 'read');-- projects.removeMember(req.viewer, project, String(req.body.principal || ''));++ router.post('/projects/:id/members/remove', action(async (req) => {++ const project = await projects.get(req.params.id);++ await access.authorize(project, req.viewer, 'read');++ await projects.removeMember(req.viewer, project, String(req.body.principal || ''));return back(`/projects/${project.id}`, 'Member removed.');}));-- router.post('/projects/:id/delete', action((req) => {-- const project = projects.get(req.params.id);-- access.authorize(project, req.viewer, 'read');++ router.post('/projects/:id/delete', action(async (req) => {++ const project = await projects.get(req.params.id);++ await access.authorize(project, req.viewer, 'read');if (req.body.confirm !== 'yes') throw new ApiError(422, 'form.confirm', 'tick the confirmation box to delete');-- projects.remove(req.viewer, project, { sites });++ await projects.remove(req.viewer, project, { sites });return back('/', 'Project deleted.');}));-- router.post('/sites/:id/domains', action((req) => {-- const { domain } = domains.add(req.viewer, req.params.id, { hostname: req.body.hostname });++ router.post('/sites/:id/domains', action(async (req) => {++ const { domain } = await domains.add(req.viewer, req.params.id, { hostname: req.body.hostname });return back(`/sites/${domain.site_id}`, `Added ${domain.hostname}: publish the DNS records below, then check.`);}));-- router.post('/sites/:id/rollback', action((req) => {-- const r = deploys.rollback(req.viewer, req.params.id, { expectedActive: req.body.expected_active || null, traceparent: req.ov && req.ov.traceparent });++ router.post('/sites/:id/rollback', action(async (req) => {++ const r = await deploys.rollback(req.viewer, req.params.id, { expectedActive: req.body.expected_active || null, traceparent: req.ov && req.ov.traceparent });return back(`/sites/${req.params.id}`, r.changed ? `Rolled back: ${r.deploy_id} is active.` : 'Nothing changed.');}));-- router.post('/sites/:id/delete', action((req) => {-- const { site } = sites.load(req.viewer, req.params.id, 'maintain');++ router.post('/sites/:id/delete', action(async (req) => {++ const { site } = await sites.load(req.viewer, req.params.id, 'maintain');if (req.body.confirm !== 'yes') throw new ApiError(422, 'form.confirm', 'tick the confirmation box to delete');-- sites.remove(req.viewer, site.id, { deploys });++ await sites.remove(req.viewer, site.id, { deploys });return back(`/projects/${site.project_id}`, `${site.name} deleted.`);}));-- router.post('/deploys/:id/activate', action((req) => {-- const { site } = deploys.load(req.viewer, req.params.id, 'deploy');-- const r = deploys.activate(req.viewer, req.params.id, { expectedActive: req.body.expected_active || null, traceparent: req.ov && req.ov.traceparent });++ router.post('/deploys/:id/activate', action(async (req) => {++ const { site } = await deploys.load(req.viewer, req.params.id, 'deploy');++ const r = await deploys.activate(req.viewer, req.params.id, { expectedActive: req.body.expected_active || null, traceparent: req.ov && req.ov.traceparent });return back(`/sites/${site.id}`, r.changed ? `${r.deploy_id} is active.` : 'That deploy was already active.');}));-- router.post('/deploys/:id/delete', action((req) => {-- const { site } = deploys.load(req.viewer, req.params.id, 'maintain');-- deploys.remove(req.viewer, req.params.id);++ router.post('/deploys/:id/delete', action(async (req) => {++ const { site } = await deploys.load(req.viewer, req.params.id, 'maintain');++ await deploys.remove(req.viewer, req.params.id);return back(`/sites/${site.id}`, 'Deploy deleted.');}));router.post('/domains/:id/verify', action(async (req) => {const d = await domains.verify(req.viewer, req.params.id, { traceparent: req.ov && req.ov.traceparent });return back(`/sites/${d.site_id}`, d.status === 'verified' ? `${d.hostname} is verified and served.` : `${d.hostname}: ${d.last_error || d.status}`, d.status === 'verified' ? null : 'error');}));-- router.post('/domains/:id/delete', action((req) => {-- const { domain } = domains.load(req.viewer, req.params.id, 'maintain');-- domains.remove(req.viewer, domain.id);++ router.post('/domains/:id/delete', action(async (req) => {++ const { domain } = await domains.load(req.viewer, req.params.id, 'maintain');++ await domains.remove(req.viewer, domain.id);return back(`/sites/${domain.site_id}`, `${domain.hostname} removed.`);}));@@@ -171,7 +171,7 @@ function createDashboard(ctx) {try {if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');if (!sameOrigin(config, req)) throw new ApiError(403, 'form.invalid', 'this form did not come from this dashboard');-- const pre = deploys.precheck(req.viewer, req.params.id);++ const pre = await deploys.precheck(req.viewer, req.params.id);release = ctx.uploadGate.enter();if (!release) { res.set('Retry-After', '30'); throw new ApiError(503, 'upload.busy', 'Host is validating other uploads right now; try again in 30 seconds'); }let up;@@@ -181,12 +181,12 @@ function createDashboard(ctx) {if (!(err instanceof UploadError)) throw err;if (err.closeConnection) res.set('Connection', 'close');if (!checkCsrf(config, req.viewer, err.fields && err.fields.csrf) && !err.closeConnection) throw new ApiError(403, 'form.invalid', 'this form expired; reload the page and try again');-- const r = deploys.recordFailure(req.viewer, pre, { source: err.source || 'files', code: err.code, problems: [{ code: err.code, message: err.message }], traceparent: req.ov && req.ov.traceparent });++ const r = await deploys.recordFailure(req.viewer, pre, { source: err.source || 'files', code: err.code, problems: [{ code: err.code, message: err.message }], traceparent: req.ov && req.ov.traceparent });return res.redirect(303, back(`/deploys/${r.deploy.id}`, `Upload refused: ${err.message}`, 'error'));}if (!checkCsrf(config, req.viewer, up.fields.csrf)) throw new ApiError(403, 'form.invalid', 'this form expired; reload the page and try again');try {-- const r = deploys.create(req.viewer, pre, up.entries, { source: up.source, activate: up.fields.activate === '1', notes: up.notes, traceparent: req.ov && req.ov.traceparent });++ const r = await deploys.create(req.viewer, pre, up.entries, { source: up.source, activate: up.fields.activate === '1', notes: up.notes, traceparent: req.ov && req.ov.traceparent });return res.redirect(303, back(`/deploys/${r.deploy.id}`, r.activated && r.activated.changed ? 'Deployed and activated.' : 'Deployed. Activate it from the site page.'));} catch (err) {if (err instanceof ApiError && err.extra && err.extra.deploy_id) return res.redirect(303, back(`/deploys/${err.extra.deploy_id}`, `Upload refused: ${err.message}`, 'error'));@@@ -89,18 +89,18 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons/*** -> { kind: 'dashboard' } | { kind: 'site', site } | { kind: 'unknown', host }*/-- function resolve(rawHost) {++ async function resolve(rawHost) {const host = normaliseHost(rawHost);if (!host) return { kind: 'unknown', host: null };if (host === config.dashboardHost || LOOPBACK_HOSTS.has(host)) return { kind: 'dashboard' };if (host.endsWith(suffix)) {const label = host.slice(0, -suffix.length);if (!label || label.includes('.')) return { kind: 'unknown', host };-- const site = q.siteByName.get(label);++ const site = await q.siteByName.get(label);return site ? { kind: 'site', site, host } : { kind: 'unknown', host };}if (!isHostname(host)) return { kind: 'unknown', host };-- const site = q.siteByCustom.get(host);++ const site = await q.siteByCustom.get(host);return site ? { kind: 'site', site, host } : { kind: 'unknown', host };}@@@ -160,8 +160,8 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = consstream.pipe(res);}-- function notFound(req, res, site, reason) {-- const page = site.active_deploy_id ? q.file.get(site.active_deploy_id, '404.html') : null;++ async function notFound(req, res, site, reason) {++ const page = site.active_deploy_id ? await q.file.get(site.active_deploy_id, '404.html') : null;if (page && page.project_id === site.project_id) return send(req, res, site, page, { status: 404, cache: 'no-cache' });res.statusCode = 404;res.setHeader('Content-Type', 'text/html; charset=utf-8');@@@ -174,13 +174,13 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons}/** Serves one request for a resolved site. */-- function handle(req, res, site) {++ async function handle(req, res, site) {baseHeaders(res, { sandbox: site.environment === 'sandbox' });if (req.method !== 'GET' && req.method !== 'HEAD') {return plain(res, 405, 'Static site: only GET and HEAD are supported.', { Allow: 'GET, HEAD' });}// Taken down by staff: nothing of the tenant's is served, on any of its hosts, until lifted.-- if (takedowns && takedowns.ofSite(site)) {++ if (takedowns && await takedowns.ofSite(site)) {// Clear-Site-Data asks the browser to drop what the site left behind for this visitor// (caches, storage and service workers a phishing page may have installed).return plain(res, 451, 'This site is unavailable: OpenVibe.Host staff took it down after a report.', { 'Clear-Site-Data': '"cache", "storage"' });@@@ -196,12 +196,12 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = consconst q0 = target.indexOf('?');const rawPath = q0 >= 0 ? target.slice(0, q0) : target;const search = q0 >= 0 ? target.slice(q0) : '';-- if (!site.active_deploy_id) return notFound(req, res, site, 'no-deploy');++ if (!site.active_deploy_id) return await notFound(req, res, site, 'no-deploy');const c = candidates(rawPath);-- if (!c) return notFound(req, res, site, 'invalid');++ if (!c) return await notFound(req, res, site, 'invalid');const deployId = site.active_deploy_id;for (const p of c.list) {-- const row = q.file.get(deployId, p);++ const row = await q.file.get(deployId, p);if (row && row.project_id === site.project_id) {const hashed = isHashedAsset(p);// Browsers keep fingerprinted assets for a year; a shared CDN in front of the tenant@@@ -211,14 +211,14 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = consreturn send(req, res, site, row, { cache: hashed ? 'public, max-age=31536000, immutable' : 'public, max-age=0, must-revalidate' });}}-- if (!c.dir && c.dirIndex && q.file.get(deployId, c.dirIndex)) {++ if (!c.dir && c.dirIndex && await q.file.get(deployId, c.dirIndex)) {// /docs → /docs/ so relative links inside docs/index.html resolve; built from the raw path.res.statusCode = 301;res.setHeader('Location', `${rawPath}/${search}`);res.setHeader('Cache-Control', 'no-cache');return res.end();}-- return notFound(req, res, site, 'missing');++ return await notFound(req, res, site, 'missing');}function unknownHost(req, res) {@@@ -9,7 +9,8 @@*/const { createApp } = require('./app');--const { app, ctx } = createApp();++(async () => {++const { app, ctx } = await createApp();const { config } = ctx;const server = app.listen(config.port, config.host, () => {@@@ -27,10 +28,11 @@ function shutdown(signal) {server.close(async () => {try { await ctx.outbox.stop(); } catch { /* best effort */ }try { ctx.stopMetrics(); } catch { /* best effort */ }-- try { ctx.store.close(); } catch { /* already closed */ }++ try { await ctx.store.close(); } catch { /* already closed */ }process.exit(0);});setTimeout(() => process.exit(0), 5000).unref();}process.on('SIGTERM', () => shutdown('SIGTERM'));process.on('SIGINT', () => shutdown('SIGINT'));++})().catch((err) => { console.error('[Host] failed to start:', err); process.exit(1); });@@@ -12,7 +12,7 @@const { createReadiness } = require('openvibe-shared/ready');const { CHARTER_TABLES } = require('./db');--function createHostReadiness({ store, blobs, auth, outbox, release = null, minFreeBytes = () => 0 }) {++function createHostReadiness({ store, blobs, auth, outbox, release = null, minFreeBytes = () => 0, valkey = null }) {const { db } = store;return createReadiness({service: 'host',@@@ -20,8 +20,8 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFrchecks: [{name: 'db', required: true,-- check: () => {-- const names = new Set(db.prepare("SELECT name FROM sqlite_master WHERE type = 'table'").all().map((r) => r.name));++ check: async () => {++ const names = new Set((await db.prepare("SELECT table_name AS name FROM information_schema.tables WHERE table_schema = current_schema()").all()).map((r) => r.name));const missing = CHARTER_TABLES.filter((t) => !names.has(t));return missing.length ? `missing ${missing.join(', ')}` : true;},@@@ -36,6 +36,7 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFrreturn free >= floor ? { ok: true, detail: { free_bytes: free } } : `${free} bytes free, below HOST_MIN_FREE_BYTES (${floor}): new deploys are refused`;},},++ { name: 'valkey', required: false, check: async () => (valkey ? valkey.ready() : { skipped: 'VALKEY_URL not set: per-actor limits count in this process only' }) },{name: 'network_jwks', required: false,check: () => {@@@ -46,8 +47,8 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFr},{name: 'events_relay', required: false,-- check: () => {-- const s = outbox.status();++ check: async () => {++ const s = await outbox.status();if (!s.enabled) return `relay off (Events URL or service credentials not configured); ${s.pending} events waiting`;if (s.rejected) return `${s.rejected} events rejected by OpenVibe.Events`;return { ok: true, detail: { pending: s.pending } };@@@ -55,8 +56,8 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFr},{name: 'domain_checks', required: false,-- check: () => {-- const stale = db.prepare("SELECT COUNT(*) AS n FROM host_domains WHERE kind = 'custom' AND status = 'pending' AND COALESCE(last_checked_at, created_at) < ?").get(store.now() - 3600 * 1000).n;++ check: async () => {++ const stale = (await db.prepare("SELECT COUNT(*) AS n FROM host_domains WHERE kind = 'custom' AND status = 'pending' AND COALESCE(last_checked_at, created_at) < ?").get(store.now() - 3600 * 1000)).n;return stale ? `${stale} pending custom domains not checked in the last hour (is the worker on?)` : true;},},@@@ -29,11 +29,11 @@ function createWorker({ config, store, domains, blobs, outbox, log = console })}/** Remove objects no row references, once they are older than `graceMs`. */-- function sweepObjects({ graceMs = 3600 * 1000 } = {}) {++ async function sweepObjects({ graceMs = 3600 * 1000 } = {}) {const known = store.db.prepare('SELECT 1 FROM host_blobs WHERE project_id = ? AND sha256 = ?');let removed = 0;for (const o of blobs.list()) {-- if (known.get(o.projectId, o.sha256)) continue;++ if (await known.get(o.projectId, o.sha256)) continue;try {if (Date.now() - fs.statSync(o.file).mtimeMs < graceMs) continue;fs.unlinkSync(o.file);@@@ -51,8 +51,8 @@ function createWorker({ config, store, domains, blobs, outbox, log = console })if (!config.worker.enabled) return;const every = (ms, fn) => { const t = setInterval(fn, ms); t.unref(); timers.push(t); };every(config.domains.recheckIntervalMs, domainTick);-- every(3600 * 1000, () => { try { sweepObjects(); } catch (err) { log.warn('[Host] object sweep failed:', err.message); } });-- every(24 * 3600 * 1000, () => { try { outbox.outbox.prune(); } catch (err) { log.warn('[Host] outbox prune failed:', err.message); } });++ every(3600 * 1000, async () => { try { await sweepObjects(); } catch (err) { log.warn('[Host] object sweep failed:', err.message); } });++ every(24 * 3600 * 1000, async () => { try { await outbox.outbox.prune(); } catch (err) { log.warn('[Host] outbox prune failed:', err.message); } });setTimeout(domainTick, 5000).unref();},stop() { for (const t of timers) clearInterval(t); timers.length = 0; },@@@ -149,7 +149,7 @@ async function runProof(host, args) {for (const t of ['ovhost', 'systemctl', 'runuser']) { fs.writeFileSync(path.join(bin, t), STUB); fs.chmodSync(path.join(bin, t), 0o755); }fs.symlinkSync(process.execPath, path.join(bin, 'node'));const state = path.join(dir, 'd41');-- return withRpc(host, (rpc) => new Promise((resolve) => {++ return await withRpc(host, (rpc) => new Promise((resolve) => {const child = spawn('bash', [SCRIPT, ...args], {env: {PATH: `${bin}:${process.env.PATH}`, HOME: dir, D41_RPC: rpc, D41_REQUIRE_ROOT: '0', D41_STATE_DIR: state,nnew file mode 100644@@@ -0,0 +1,11 @@++'use strict';++/**++ * A migrated database for one test run (ADR-035), from openvibe-sdk/testing: PGlite by default; with++ * HOST_TEST_STORE=pg (npm run test:pg) the PostgreSQL + PgBouncer containers, with roles and a schema of this run's own.++ */++const { createTestDb, pgAvailable } = require('openvibe-sdk/testing');++const { MIGRATIONS } = require('../../server/db');++++const testDb = ({ store = process.env.HOST_TEST_STORE || 'pglite', max = 4 } = {}) => createTestDb({ migrations: MIGRATIONS, store, service: 'host', max });++++module.exports = { testDb, pgAvailable };@@@ -97,7 +97,7 @@ const { site: siteTar } = require('./stageb/tar');assert.strictEqual((await t.get('alpha.openvibe.host', '/')).text, 'ALPHA TWO');assert.strictEqual((await t.get('www.alpha-example.org', '/')).text, 'ALPHA TWO');assert.strictEqual((await t.api('DELETE', `/api/v1/sites/${alpha.id}/takedown`, { as: staff })).status, 404);-- const rows = t.ctx.store.db.prepare('SELECT target_kind, reason, lifted_at IS NOT NULL AS lifted, lift_note FROM host_takedowns').all();++ const rows = await t.ctx.store.db.prepare('SELECT target_kind, reason, (lifted_at IS NOT NULL)::int AS lifted, lift_note FROM host_takedowns').all();assert.deepStrictEqual(rows, [{ target_kind: 'site', reason: 'phishing report #42', lifted: 1, lift_note: 'reviewed: false positive' }], 'the history is kept');});@@@ -134,7 +134,7 @@ const { site: siteTar } = require('./stageb/tar');const pc = await t.project(alice, 'C');const gamma = await t.site(alice, pc.id, 'gamma');const body = siteTar({ 'index.html': 'GAMMA' });-- const failedBefore = t.ctx.store.db.prepare("SELECT COUNT(*) AS n FROM host_deploys WHERE state = 'failed'").get().n;++ const failedBefore = (await t.ctx.store.db.prepare("SELECT COUNT(*) AS n FROM host_deploys WHERE state = 'failed'").get()).n;// The first upload sends its headers and half its body, then waits.let firstResponse;let finish;@@@ -157,7 +157,7 @@ const { site: siteTar } = require('./stageb/tar');await first;assert.strictEqual(firstResponse.status, 201, firstResponse.text);assert.strictEqual(t.ctx.uploadGate.inFlight, 0, 'the slot is released');-- assert.strictEqual(t.ctx.store.db.prepare("SELECT COUNT(*) AS n FROM host_deploys WHERE state = 'failed'").get().n, failedBefore, 'a busy refusal is not the tenant\'s failed deploy');++ assert.strictEqual((await t.ctx.store.db.prepare("SELECT COUNT(*) AS n FROM host_deploys WHERE state = 'failed'").get()).n, failedBefore, 'a busy refusal is not the tenant\'s failed deploy');const third = await t.upload(alice, gamma.id, { 'index.html': 'THIRD' });assert.strictEqual(third.status, 201, third.text);// A refused upload releases its slot too.@@@ -79,7 +79,7 @@ const DIR = path.join(__dirname, '..', 'docs', 'capabilities-proposal');await t.ctx.outbox.outbox.flush();assert.deepStrictEqual(received.map((e) => e.event_type).sort(), ['host.deploy.activated', 'host.deploy.created', 'host.deploy.failed']);assert.ok(received.every((e) => e.source === 'host'));-- assert.strictEqual(t.ctx.outbox.status().pending, 0);++ assert.strictEqual((await t.ctx.outbox.status()).pending, 0);await t.close();await events.close();});@@@ -63,7 +63,7 @@ const DAY = 24 * 3600 * 1000;const r = await t.api('POST', `/api/v1/domains/${dom.id}/verify`, { as: alice });assert.strictEqual(r.json().domain.status, 'verified');assert.strictEqual(r.json().domain.served, true);-- const ev = t.events('host.domain.verified');++ const ev = await t.events('host.domain.verified');assert.strictEqual(ev.length, 1);assert.deepStrictEqual(ev[0].payload, { project_id: pa.id, site_id: sa.id, site: 'alice-site', hostname: HOST });const g = await t.get(HOST, '/');@@@ -115,7 +115,7 @@ const DAY = 24 * 3600 * 1000;const never = (await t.api('POST', `/api/v1/sites/${sb.id}/domains`, { as: bob, json: { hostname: 'never.example.org' } })).json().domain;t.clock.advance(8 * DAY);await t.ctx.worker.domainTick();-- const after = t.ctx.domains.get(never.id);++ const after = await t.ctx.domains.get(never.id);assert.strictEqual(after.status, 'failed');});@@@ -123,10 +123,10 @@ const DAY = 24 * 3600 * 1000;t.dns.delete(`_openvibe-host.${HOST}`);t.clock.advance(DAY + 1000);await t.ctx.worker.domainTick();-- assert.strictEqual(t.ctx.domains.get(dom.id).status, 'verified', 'still served during the grace period');-- assert.ok(t.ctx.domains.get(dom.id).record_missing_since);++ assert.strictEqual((await t.ctx.domains.get(dom.id)).status, 'verified', 'still served during the grace period');++ assert.ok((await t.ctx.domains.get(dom.id)).record_missing_since);for (let i = 0; i < 8; i++) { t.clock.advance(DAY + 1000); await t.ctx.worker.domainTick(); }-- assert.strictEqual(t.ctx.domains.get(dom.id).status, 'lapsed');++ assert.strictEqual((await t.ctx.domains.get(dom.id)).status, 'lapsed');assert.strictEqual((await t.get(HOST, '/')).status, 404);});@@@ -142,8 +142,8 @@ const { boot, check, done } = require('./stageb/boot');await check('events: created/activated in the outbox with valid envelopes and no file contents', async () => {const contracts = require('openvibe-contracts');-- const created = t.events('host.deploy.created');-- const activated = t.events('host.deploy.activated');++ const created = await t.events('host.deploy.created');++ const activated = await t.events('host.deploy.activated');assert.ok(created.length >= 4);assert.ok(activated.some((e) => e.payload.rollback === true));for (const e of [...created, ...activated]) {@@@ -163,7 +163,7 @@ const DAY = 24 * 3600 * 1000;const up = await t.api('POST', `/api/v1/sites/${site.id}/deploys?activate=1`, { as: ci, body: siteTar({ 'index.html': 'from CI' }), headers: { 'content-type': 'application/gzip' } });assert.strictEqual(up.status, 201, up.text);assert.strictEqual(up.json().deploy.created_by, `app:${appId}`);-- assert.deepStrictEqual(t.events('host.deploy.activated').pop().actor, { type: 'app', id: appId });++ assert.deepStrictEqual((await t.events('host.deploy.activated')).pop().actor, { type: 'app', id: appId });assert.strictEqual((await t.get('quota-site.openvibe.host', '/')).text, 'from CI');const mk = await t.api('POST', `/api/v1/projects/${project.id}/sites`, { as: ci, json: { name: 'ci-site' } });assert.strictEqual(mk.status, 403);@@@ -49,15 +49,15 @@ const { boot, check, done } = require('./stageb/boot');await check('a failure inside the switch leaves the pointer, history and outbox untouched', async () => {await t.api('POST', `/api/v1/deploys/${v1.id}/activate`, { as: alice, json: {} });const db = t.ctx.store.db;-- const before = { active: db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id).active_deploy_id, acts: db.prepare('SELECT COUNT(*) AS n FROM host_activations').get().n, events: db.prepare('SELECT COUNT(*) AS n FROM event_outbox').get().n };++ const before = { active: (await db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id)).active_deploy_id, acts: (await db.prepare('SELECT COUNT(*) AS n FROM host_activations').get()).n, events: (await db.prepare('SELECT COUNT(*) AS n FROM event_outbox').get()).n };const emit = t.ctx.outbox.emit;t.ctx.outbox.emit = () => { throw new Error('simulated crash after the pointer moved'); };const r = await t.api('POST', `/api/v1/deploys/${v2.id}/activate`, { as: alice, json: {} });t.ctx.outbox.emit = emit;assert.strictEqual(r.status, 500);-- assert.strictEqual(db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id).active_deploy_id, before.active);-- assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM host_activations').get().n, before.acts);-- assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM event_outbox').get().n, before.events);++ assert.strictEqual((await db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id)).active_deploy_id, before.active);++ assert.strictEqual((await db.prepare('SELECT COUNT(*) AS n FROM host_activations').get()).n, before.acts);++ assert.strictEqual((await db.prepare('SELECT COUNT(*) AS n FROM event_outbox').get()).n, before.events);assert.strictEqual((await t.get('flip.openvibe.host', '/')).text, 'version-1');});@@@ -86,12 +86,12 @@ const { boot, check, done } = require('./stageb/boot');await check('artifacts are immutable in the database itself', async () => {const db = t.ctx.store.db;-- assert.throws(() => db.prepare("UPDATE host_deploys SET manifest = '{}' WHERE id = ?").run(v1.id), /immutable/);-- assert.throws(() => db.prepare('UPDATE host_deploys SET total_bytes = 1 WHERE id = ?').run(v1.id), /immutable/);-- assert.throws(() => db.prepare("UPDATE host_deploy_files SET sha256 = 'x' WHERE deploy_id = ?").run(v1.id), /immutable/);-- assert.throws(() => db.prepare('DELETE FROM host_deploy_files WHERE deploy_id = ?').run(v1.id), /live deploy/);-- db.prepare("UPDATE host_deploys SET state = 'deleted' WHERE id = (SELECT id FROM host_deploys WHERE state = 'failed' LIMIT 1)").run();-- assert.throws(() => db.prepare("UPDATE host_deploys SET state = 'ready' WHERE state = 'deleted'").run(), /cannot become ready/);++ await assert.rejects(db.prepare("UPDATE host_deploys SET manifest = '{}' WHERE id = ?").run(v1.id), /immutable/);++ await assert.rejects(db.prepare('UPDATE host_deploys SET total_bytes = 1 WHERE id = ?').run(v1.id), /immutable/);++ await assert.rejects(db.prepare("UPDATE host_deploy_files SET sha256 = 'x' WHERE deploy_id = ?").run(v1.id), /immutable/);++ await assert.rejects(db.prepare('DELETE FROM host_deploy_files WHERE deploy_id = ?').run(v1.id), /live deploy/);++ await db.prepare("UPDATE host_deploys SET state = 'deleted' WHERE id = (SELECT id FROM host_deploys WHERE state = 'failed' LIMIT 1)").run();++ await assert.rejects(db.prepare("UPDATE host_deploys SET state = 'ready' WHERE state = 'deleted'").run(), /cannot become ready/);});await check('the active pointer survives a restart', async () => {@@@ -42,7 +42,7 @@ const ORIGIN = 'https://openvibe.host';}const deployKeys = Object.keys(d).sort();assert.deepStrictEqual(deployKeys, ['active', 'created_at', 'created_by', 'failure_code', 'file_count', 'id', 'log', 'manifest_sha256', 'new_bytes', 'project_id', 'site_id', 'source', 'state', 'total_bytes'].sort());-- const events = JSON.stringify(t.events());++ const events = JSON.stringify(await t.events());assert.ok(!events.includes('sk_live') && !events.includes(SECRET));});@@@ -22,7 +22,7 @@ const { tarball } = require('./stageb/tar');let baseline = countObjects();async function refused(entries, code, { gzip = true, raw = null, contentType } = {}) {-- const before = t.events('host.deploy.failed').length;++ const before = (await t.events('host.deploy.failed')).length;const r = await t.api('POST', `/api/v1/sites/${site.id}/deploys?activate=1`, { as: alice, body: raw || tarball(entries, { gzip }), headers: { 'content-type': contentType || 'application/gzip' } });assert.ok([413, 422].includes(r.status), `${code}: status ${r.status} ${r.text}`);const body = r.json();@@@ -33,7 +33,7 @@ const { tarball } = require('./stageb/tar');assert.strictEqual(failed.state, 'failed');assert.strictEqual(failed.failure_code, code);assert.strictEqual(failed.file_count, 0);-- assert.strictEqual(t.events('host.deploy.failed').length, before + 1);++ assert.strictEqual((await t.events('host.deploy.failed')).length, before + 1);assert.strictEqual((await t.get('uploads.openvibe.host', '/')).text, 'GOOD', 'the active deploy is unchanged');assert.strictEqual(countObjects(), baseline, 'nothing was stored');return body;@@@ -121,7 +121,7 @@ const { tarball } = require('./stageb/tar');const one = list.deploys.find((d) => d.failure_code === 'deploy.link_refused');const log = (await t.api('GET', `/api/v1/deploys/${one.id}/log`, { as: alice })).json().log;assert.ok(log.some((l) => l.level === 'error' && /link/.test(l.message)));-- for (const e of t.events('host.deploy.failed')) assert.ok(!JSON.stringify(e).includes('sk_live'), 'no file contents in events');++ for (const e of await t.events('host.deploy.failed')) assert.ok(!JSON.stringify(e).includes('sk_live'), 'no file contents in events');});await t.close();@@@ -53,6 +53,9 @@ async function boot(opts = {}) {};const configLib = require('../../server/config');const { createApp } = require('../../server/app');++ const { createStore } = require('../../server/db');++ // One database per boot (PGlite, or HOST_TEST_STORE=pg: the containers); a restart keeps it, like a file did.++ const testdb = await require('../helpers/db').testDb();const quiet = { log() {}, warn() {}, error: (...a) => { if (process.env.VERBOSE) console.error(...a); } };let server = null;@@@ -61,7 +64,7 @@ async function boot(opts = {}) {async function start() {const config = configLib.load(env);-- built = createApp({ config, now: clock.now, log: quiet, resolver, fetchImpl: opts.fetchImpl });++ built = await createApp({ config, store: createStore(testdb.db, { now: clock.now }), now: clock.now, log: quiet, resolver, fetchImpl: opts.fetchImpl });await built.ctx.auth.ensureKey();server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });t.port = server.address().port;@@@ -71,7 +74,7 @@ async function boot(opts = {}) {}async function stop() {if (server) { server.closeAllConnections(); await new Promise((r) => server.close(r)); }-- if (built) { built.ctx.worker.stop(); await built.ctx.outbox.stop(); built.ctx.stopMetrics(); built.ctx.store.close(); }++ if (built) { built.ctx.worker.stop(); await built.ctx.outbox.stop(); built.ctx.stopMetrics(); }server = null; built = null;}@@@ -107,7 +110,7 @@ async function boot(opts = {}) {t.get = (host, p, o = {}) => request({ ...o, method: o.method || 'GET', host, path: p });/** Outbox rows as parsed envelopes. */-- t.events = (type = null) => t.ctx.store.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all().map((r) => JSON.parse(r.envelope)).filter((e) => !type || e.event_type === type);++ t.events = async (type = null) => (await t.ctx.store.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all()).map((r) => (typeof r.envelope === 'string' ? (typeof r.envelope === 'string' ? (typeof r.envelope === 'string' ? JSON.parse(r.envelope) : r.envelope) : r.envelope) : r.envelope)).filter((e) => !type || e.event_type === type);t.user = (name, extra) => network.addUser(name, extra);t.project = async (user, name = 'Project', extra = {}) => {@@@ -131,7 +134,7 @@ async function boot(opts = {}) {};t.restart = async () => { await stop(); await start(); };-- t.close = async () => { await stop(); await network.close(); fs.rmSync(dir, { recursive: true, force: true }); };++ t.close = async () => { await stop(); await testdb.close(); await network.close(); fs.rmSync(dir, { recursive: true, force: true }); };await start();return t;}
@@ -20,6 +20,14 @@ HOST_SITES_DOMAIN=openvibe.host
@@ -20,6 +20,14 @@ HOST_SITES_DOMAIN=openvibe.host
# SQLite database and content-addressed objects (the production unit sets both under
# SQLite database and content-addressed objects (the production unit sets both under
# /var/lib/openvibe-host-api)
# /var/lib/openvibe-host-api)
+# PostgreSQL (ADR-035), written by OpenVibe.Host roles/data/add-service.sh host: the runtime role through PgBouncer,
+# the owner on a direct connection (migrations and the one-time import), and Valkey (shared limit counters).
+# Without DATABASE_URL, development uses an embedded PGlite database in data/pglite.
+DATABASE_URL=
+DATABASE_DIRECT_URL=
+VALKEY_URL=
+VALKEY_PREFIX=
+# The SQLite file of releases before PostgreSQL: read once by scripts/migrate-to-postgres.js.
HOST_DB_PATH=./data/host.db
HOST_DB_PATH=./data/host.db
HOST_STORAGE_DIR=./data/objects
HOST_STORAGE_DIR=./data/objects
@@ -10,6 +10,9 @@ jobs:
@@ -10,6 +10,9 @@ jobs:
with:
with:
syntax-dirs: lib server
syntax-dirs: lib server
contracts-service: host
contracts-service: host
+ # ADR-035: PostgreSQL 18 + PgBouncer (transaction mode) + Valkey 9 containers (openvibe-sdk
+ # scripts/test-services.sh). npm test runs every file on PGlite; test:pg runs them all through PgBouncer.
+ test: 'eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" && npm test && npm run test:pg'
contracts-src: lib server
contracts-src: lib server
extra: node --check bin/ovhost && for f in scripts/*.sh; do bash -n "$f"; done
extra: node --check bin/ovhost && for f in scripts/*.sh; do bash -n "$f"; done
# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).
# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).
@@ -49,10 +49,10 @@ Stage A adds a few safety rules of its own:
@@ -49,10 +49,10 @@ Stage A adds a few safety rules of its own:
## Depends on
## Depends on
-- OpenVibe.Contracts (`openvibe-contracts` v0.66.0; `host.*` capabilities and the `host` manifest were released in v0.24.0, and v0.32.0 adds the takedown routes to `host.site.manage`): service manifests (vhost rendering, snapshots, the first-party domain list), ids, problem+json, service-token verification, capability checks.
+- OpenVibe.Contracts (`openvibe-contracts` v0.76.0; `host.*` capabilities and the `host` manifest were released in v0.24.0, and v0.32.0 adds the takedown routes to `host.site.manage`): service manifests (vhost rendering, snapshots, the first-party domain list), ids, problem+json, service-token verification, capability checks.
- OpenVibe.Network (Stage B): SSO for the dashboard, the JWKS that verifies user and service tokens, client-credentials tokens for the outbox relay.
- OpenVibe.Network (Stage B): SSO for the dashboard, the JWKS that verifies user and service tokens, client-credentials tokens for the outbox relay.
-- OpenVibe.Events (Stage B): `host.*` events through the `openvibe-sdk` v0.12.0 transactional outbox (openvibe-sdk/limits for the per-actor limits).
-- `openvibe-shared` v1.25.0 (Stage B): shared chrome, legal pages, `/release.json`, `/metrics`, `/api/ready`.
+- OpenVibe.Events (Stage B): `host.*` events through the `openvibe-sdk` v0.20.3 transactional outbox (openvibe-sdk/limits for the per-actor limits).
+- `openvibe-shared` v1.27.0 (Stage B): shared chrome, legal pages, `/release.json`, `/metrics`, `/api/ready`.
- OpenVibe.Media: not yet. The roadmap stores artifacts "through Media where practical"; Stage B keeps them on local disk for now (see [Not done yet](#not-done-yet-stage-b)).
- OpenVibe.Media: not yet. The roadmap stores artifacts "through Media where practical"; Stage B keeps them on local disk for now (see [Not done yet](#not-done-yet-stage-b)).
## Capabilities
## Capabilities
@@ -212,8 +212,7 @@ Two things deploy from this repository:
@@ -212,8 +212,7 @@ Two things deploy from this repository:
The unit is `openvibe-host.service` on `127.0.0.1:4910`, the env file `/etc/openvibe/host.env`. Its state is
The unit is `openvibe-host.service` on `127.0.0.1:4910`, the env file `/etc/openvibe/host.env`. Its state is
`/var/lib/openvibe-host-api` (not ovhost's own `/var/lib/openvibe-host`).
`/var/lib/openvibe-host-api` (not ovhost's own `/var/lib/openvibe-host`).
Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the
Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the
- restart; afterwards `sudo ovhost rollback host --to <sha>`. Nothing blocks a rollback: the schema
- code only adds tables and columns.
+ restart; afterwards `sudo ovhost rollback host --to <sha>`. Migrations only add tables and columns.
## Installing on the host (for the operator)
## Installing on the host (for the operator)
@@ -303,11 +302,11 @@ The three capability ids and the service manifest are released in `openvibe-cont
@@ -303,11 +302,11 @@ The three capability ids and the service manifest are released in `openvibe-cont
### Activation and rollback
### Activation and rollback
-Activation is one SQLite transaction: a compare-and-set on the site's `active_deploy_id`, an activation record and the `host.deploy.activated` event. If any part fails, none of it happens. `expected_active` turns a racing operator's switch into a 409. Rollback goes to a named ready deploy of the same site, or to the most recent previously active one that still exists. The active deploy cannot be deleted. Deleting another deploy removes its objects when no other deploy of the project uses them.
+Activation is one PostgreSQL transaction: a compare-and-set on the site's `active_deploy_id`, an activation record and the `host.deploy.activated` event. If any part fails, none of it happens. `expected_active` turns a racing operator's switch into a 409. Rollback goes to a named ready deploy of the same site, or to the most recent previously active one that still exists. The active deploy cannot be deleted. Deleting another deploy removes its objects when no other deploy of the project uses them.
### Events
### Events
-Through the `openvibe-sdk` v0.5.0 transactional outbox (`event_outbox`), inside the transaction that makes the change:
+Through the `openvibe-sdk` v0.20.3 transactional outbox (`event_outbox`), inside the transaction that makes the change:
| Event | When | Payload |
| Event | When | Payload |
|---|---|---|
|---|---|---|
@@ -322,7 +321,7 @@ These tenant events have subject `deploy` and visibility `internal`; `host.deplo
@@ -322,7 +321,7 @@ These tenant events have subject `deploy` and visibility `internal`; `host.deplo
### Dashboard
### Dashboard
-Server-rendered pages with the shared chrome (`openvibe-shared` v1.5.1: navbar and theme loader from the Network, `<noscript>` navigation, SSR footer, app icon, legal pages). Every action is a plain form, so it works without JavaScript: projects, quotas and usage, members, sites, folder or archive upload, deploys with activate/rollback/delete, activation history, upload logs and file lists, domains with their DNS records and a "check DNS now" button. Pages are `private, no-store`, `noindex`, `frame-ancestors 'none'`.
+Server-rendered pages with the shared chrome (`openvibe-shared` v1.27.0: navbar and theme loader from the Network, `<noscript>` navigation, SSR footer, app icon, legal pages). Every action is a plain form, so it works without JavaScript: projects, quotas and usage, members, sites, folder or archive upload, deploys with activate/rollback/delete, activation history, upload logs and file lists, domains with their DNS records and a "check DNS now" button. Pages are `private, no-store`, `noindex`, `frame-ancestors 'none'`.
### Observability
### Observability
@@ -333,7 +332,7 @@ Server-rendered pages with the shared chrome (`openvibe-shared` v1.5.1: navbar a
@@ -333,7 +332,7 @@ Server-rendered pages with the shared chrome (`openvibe-shared` v1.5.1: navbar a
- OpenVibe.Network OAuth client **`host`**, redirect `https://openvibe.host/auth/callback`, scope `profile theme`. The same client is the service principal `svc:host`.
- OpenVibe.Network OAuth client **`host`**, redirect `https://openvibe.host/auth/callback`, scope `profile theme`. The same client is the service principal `svc:host`.
- Grant `[host, events.event.publish, openvibe.events]`.
- Grant `[host, events.event.publish, openvibe.events]`.
- Callers of Host get `[<client>, host.site.manage | host.deploy.create | host.domain.manage, openvibe.host]` as needed. None exist yet (Codes, the expected first caller, is not built).
- Callers of Host get `[<client>, host.site.manage | host.deploy.create | host.domain.manage, openvibe.host]` as needed. None exist yet (Codes, the expected first caller, is not built).
-- Released in `openvibe-contracts` v0.24.0 (CI contract check blocking); v0.32.0 adds the takedown routes to `host.site.manage.implementedBy`.
+- Released in `openvibe-contracts` v0.76.0 (CI contract check blocking); v0.32.0 adds the takedown routes to `host.site.manage.implementedBy`.
### Deploying Stage B (for the operator)
### Deploying Stage B (for the operator)
@@ -352,7 +351,7 @@ The service runs on the host (loopback, since 2026-09-23) and the pending tenant
@@ -352,7 +351,7 @@ The service runs on the host (loopback, since 2026-09-23) and the pending tenant
### Not done yet (Stage B)
### Not done yet (Stage B)
- Uploads are held in memory while they are validated (bounded by `HOST_MAX_UPLOAD_BYTES` and `HOST_MAX_UNPACKED_BYTES`), so the service needs that much headroom per concurrent upload.
- Uploads are held in memory while they are validated (bounded by `HOST_MAX_UPLOAD_BYTES` and `HOST_MAX_UNPACKED_BYTES`), so the service needs that much headroom per concurrent upload.
-- Objects live on the service's local disk, not in OpenVibe.Media; there is no replication and no backup beyond `ovhost backup host` (the SQLite database only).
+- Objects live on the service's local disk, not in OpenVibe.Media; there is no replication and no backup beyond `ovhost backup host`. The database (`ov_host` on the host's data role, ADR-035; schema in [migrations/](migrations/)) is backed up with the others by pgBackRest.
- Certificates for custom domains are issued by hand (step 8); there is no automatic ACME flow.
- Certificates for custom domains are issued by hand (step 8); there is no automatic ACME flow.
- Projects are created in Host. When Network has projects (ADR-014), Host should accept only Network project ids and read membership from Network.
- Projects are created in Host. When Network has projects (ADR-014), Host should accept only Network project ids and read membership from Network.
- `openvibe.host` is not on the Public Suffix List, so a tenant page can still set `Domain=openvibe.host` cookies (a cookie bomb breaks the dashboard and other tenant sites for that visitor). The protections above do not depend on it. [docs/threat-review.md §5](docs/threat-review.md#5-the-public-suffix-list-question) records the decision (launch without it) and the recommended follow-up (dashboard off the tenant zone, then list the zone).
- `openvibe.host` is not on the Public Suffix List, so a tenant page can still set `Domain=openvibe.host` cookies (a cookie bomb breaks the dashboard and other tenant sites for that visitor). The protections above do not depend on it. [docs/threat-review.md §5](docs/threat-review.md#5-the-public-suffix-list-question) records the decision (launch without it) and the recommended follow-up (dashboard off the tenant zone, then list the zone).
@@ -4,16 +4,30 @@
@@ -4,16 +4,30 @@
"code": true,
"code": true,
"wave": "W21 Stage A + Stage B",
"wave": "W21 Stage A + Stage B",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
"plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026)",
- "runtime": "node 22; Stage A: CLI (ovhost); Stage B: express 4 + better-sqlite3 11 service",
+ "runtime": "node 22; Stage A: CLI (ovhost); Stage B: express 4 + PostgreSQL 18 (openvibe-sdk/db, async) + Valkey 9 service",
"port": 4910,
"port": 4910,
"domain": "openvibe.host",
"domain": "openvibe.host",
"tenantDomain": "*.openvibe.host",
"tenantDomain": "*.openvibe.host",
- "contracts": "openvibe-contracts v0.66.0",
- "packages": ["openvibe-contracts v0.66.0 (host.* capabilities and manifest released in v0.24.0; v0.32.0 adds the takedown routes to host.site.manage)", "openvibe-sdk v0.12.0", "openvibe-shared v1.25.0"],
- "capabilities": ["host.site.manage", "host.deploy.create", "host.domain.manage"],
+ "contracts": "openvibe-contracts v0.76.0",
+ "packages": [
+ "openvibe-contracts v0.76.0",
+ "openvibe-sdk v0.20.3",
+ "openvibe-shared v1.27.0"
+ ],
+ "capabilities": [
+ "host.site.manage",
+ "host.deploy.create",
+ "host.domain.manage"
+ ],
"capabilitiesRegistered": true,
"capabilitiesRegistered": true,
"serviceManifestRegistered": "host manifest v0.2.0 released (openvibe-contracts v0.24.0+), with the four host.* events",
"serviceManifestRegistered": "host manifest v0.2.0 released (openvibe-contracts v0.24.0+), with the four host.* events",
- "events": ["host.deploy.created", "host.deploy.activated", "host.deploy.failed", "host.domain.verified", "host.release.published"],
+ "events": [
+ "host.deploy.created",
+ "host.deploy.activated",
+ "host.deploy.failed",
+ "host.domain.verified",
+ "host.release.published"
+ ],
"stages": {
"stages": {
"A": "alpha: ovhost CLI installed on the production host 2026-09-23 (/usr/local/bin/ovhost, inventory /etc/openvibe/host.json); every service deploys with `sudo ovhost deploy <svc>` since the 2026-09-27 cutover (roadmap WS-N task 11: Live, Tools, Sites, Games and OpenRe through the release-layout, multi-app, static-build and pnpm-build strategies); nightly backups, restore drills (19 passed 2026-09-23, docs/restore-drills.md), release notifications, incidents and maintenance, the alerts relay; the D41 deploy-and-rollback proof (scripts/d41-proof.sh) is written and tested",
"A": "alpha: ovhost CLI installed on the production host 2026-09-23 (/usr/local/bin/ovhost, inventory /etc/openvibe/host.json); every service deploys with `sudo ovhost deploy <svc>` since the 2026-09-27 cutover (roadmap WS-N task 11: Live, Tools, Sites, Games and OpenRe through the release-layout, multi-app, static-build and pnpm-build strategies); nightly backups, restore drills (19 passed 2026-09-23, docs/restore-drills.md), release notifications, incidents and maintenance, the alerts relay; the D41 deploy-and-rollback proof (scripts/d41-proof.sh) is written and tested",
"B": "alpha: running on the host on loopback :4910 (2026-09-23 audit), not launched; launch-rule items met in code and tests, launch steps in docs/launch.md, threat review in docs/threat-review.md",
"B": "alpha: running on the host on loopback :4910 (2026-09-23 audit), not launched; launch-rule items met in code and tests, launch steps in docs/launch.md, threat review in docs/threat-review.md",
@@ -3,8 +3,8 @@ Description=OpenVibe.Host Stage B: tenant static hosting (dashboard, API and *.o
@@ -3,8 +3,8 @@ Description=OpenVibe.Host Stage B: tenant static hosting (dashboard, API and *.o
# Network (sign-in, JWKS, service tokens) and Events (the outbox relay) should be up first, but Host
# Network (sign-in, JWKS, service tokens) and Events (the outbox relay) should be up first, but Host
# degrades gracefully without them: tenant sites keep serving; sign-in, API calls and events wait
# degrades gracefully without them: tenant sites keep serving; sign-in, API calls and events wait
# (see /api/ready).
# (see /api/ready).
-After=network.target
-Wants=openvibe-network.service
+After=network.target postgresql.service pgbouncer.service valkey-server.service
+Wants=openvibe-network.service postgresql.service pgbouncer.service valkey-server.service
[Service]
[Service]
Type=simple
Type=simple
new file mode 100644
new file mode 100644
@@ -0,0 +1,185 @@
@@ -0,0 +1,185 @@
+-- phase: expand
+-- OpenVibe.Host on PostgreSQL (ADR-035, roadmap WS-X2): the tables as they were on SQLite (converted by openvibe-sdk
+-- tools/asyncify/sqlite-schema-to-pg: text COLLATE "C" compares like SQLite, integers are bigint, identities keep their ids),
+-- then the openvibe-publishing stores and the openvibe-sdk inbox and outbox. Generated once on 2026-09-28; never edited after it runs.
+
+CREATE TABLE host_projects (
+ id text COLLATE "C" PRIMARY KEY, -- prj_<ULID>
+ seq bigint GENERATED ALWAYS AS IDENTITY UNIQUE, -- insertion order (the SQLite rowid tiebreak)
+ network_project_id text COLLATE "C", -- OpenVibe.Network project id (ADR-014), when it exists
+ owner_subject text COLLATE "C" NOT NULL, -- usr_…
+ name text COLLATE "C" NOT NULL,
+ environment text COLLATE "C" NOT NULL DEFAULT 'production' CHECK (environment IN ('production','sandbox')),
+ status text COLLATE "C" NOT NULL DEFAULT 'active' CHECK (status IN ('active','deleted')),
+ created_at bigint NOT NULL,
+ updated_at bigint NOT NULL,
+ deleted_at bigint
+);
+CREATE UNIQUE INDEX host_projects_network ON host_projects (network_project_id) WHERE network_project_id IS NOT NULL AND status = 'active';
+CREATE INDEX host_projects_owner ON host_projects (owner_subject, status);
+
+CREATE TABLE host_project_members (
+ project_id text COLLATE "C" NOT NULL REFERENCES host_projects(id),
+ principal text COLLATE "C" NOT NULL, -- usr_… | app:app_… | svc:<id>
+ role text COLLATE "C" NOT NULL CHECK (role IN ('owner','maintainer','deployer')),
+ added_by text COLLATE "C",
+ created_at bigint NOT NULL,
+ PRIMARY KEY (project_id, principal)
+);
+CREATE INDEX host_project_members_principal ON host_project_members (principal);
+
+CREATE TABLE host_quotas (
+ project_id text COLLATE "C" PRIMARY KEY REFERENCES host_projects(id),
+ storage_bytes bigint,
+ deploys_per_day bigint,
+ max_files bigint,
+ max_file_bytes bigint,
+ sites bigint,
+ custom_domains bigint,
+ updated_by text COLLATE "C",
+ updated_at bigint NOT NULL
+);
+
+CREATE TABLE host_sites (
+ id text COLLATE "C" PRIMARY KEY, -- site_<ULID>
+ seq bigint GENERATED ALWAYS AS IDENTITY UNIQUE, -- insertion order (the SQLite rowid tiebreak)
+ project_id text COLLATE "C" NOT NULL REFERENCES host_projects(id),
+ name text COLLATE "C" NOT NULL, -- the <name>.openvibe.host label
+ active_deploy_id text COLLATE "C",
+ status text COLLATE "C" NOT NULL DEFAULT 'active' CHECK (status IN ('active','deleted')),
+ created_by text COLLATE "C" NOT NULL,
+ created_at bigint NOT NULL,
+ updated_at bigint NOT NULL,
+ deleted_at bigint
+);
+CREATE UNIQUE INDEX host_sites_name ON host_sites (name) WHERE status = 'active';
+CREATE INDEX host_sites_project ON host_sites (project_id, status);
+
+CREATE TABLE host_deploys (
+ id text COLLATE "C" PRIMARY KEY, -- dpl_<ULID>
+ seq bigint GENERATED ALWAYS AS IDENTITY UNIQUE, -- insertion order (the SQLite rowid tiebreak)
+ project_id text COLLATE "C" NOT NULL REFERENCES host_projects(id),
+ site_id text COLLATE "C" NOT NULL REFERENCES host_sites(id),
+ state text COLLATE "C" NOT NULL CHECK (state IN ('ready','failed','deleted')),
+ source text COLLATE "C" NOT NULL CHECK (source IN ('archive','files')),
+ manifest text COLLATE "C", -- JSON host.deploy-manifest@1 (NULL when failed)
+ manifest_sha256 text COLLATE "C",
+ file_count bigint NOT NULL DEFAULT 0,
+ total_bytes bigint NOT NULL DEFAULT 0,
+ new_bytes bigint NOT NULL DEFAULT 0, -- bytes this deploy added to the project's storage
+ failure_code text COLLATE "C",
+ created_by text COLLATE "C" NOT NULL,
+ created_at bigint NOT NULL,
+ deleted_at bigint
+);
+CREATE INDEX host_deploys_site ON host_deploys (site_id, created_at);
+CREATE INDEX host_deploys_project_day ON host_deploys (project_id, created_at);
+
+CREATE TABLE host_deploy_files (
+ deploy_id text COLLATE "C" NOT NULL REFERENCES host_deploys(id),
+ path text COLLATE "C" NOT NULL,
+ sha256 text COLLATE "C" NOT NULL,
+ size bigint NOT NULL,
+ content_type text COLLATE "C" NOT NULL,
+ PRIMARY KEY (deploy_id, path)
+);
+CREATE INDEX host_deploy_files_sha ON host_deploy_files (sha256);
+
+CREATE TABLE host_blobs (
+ project_id text COLLATE "C" NOT NULL,
+ sha256 text COLLATE "C" NOT NULL,
+ size bigint NOT NULL,
+ created_at bigint NOT NULL,
+ PRIMARY KEY (project_id, sha256)
+);
+
+CREATE TABLE host_activations (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ site_id text COLLATE "C" NOT NULL REFERENCES host_sites(id),
+ deploy_id text COLLATE "C" NOT NULL,
+ previous_deploy_id text COLLATE "C",
+ kind text COLLATE "C" NOT NULL CHECK (kind IN ('activate','rollback')),
+ actor text COLLATE "C" NOT NULL,
+ created_at bigint NOT NULL
+);
+CREATE INDEX host_activations_site ON host_activations (site_id, id);
+
+CREATE TABLE host_deploy_logs (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ deploy_id text COLLATE "C" NOT NULL,
+ level text COLLATE "C" NOT NULL CHECK (level IN ('info','warn','error')),
+ message text COLLATE "C" NOT NULL,
+ created_at bigint NOT NULL
+);
+CREATE INDEX host_deploy_logs_deploy ON host_deploy_logs (deploy_id, id);
+
+CREATE TABLE host_domains (
+ id text COLLATE "C" PRIMARY KEY, -- dom_<ULID>
+ seq bigint GENERATED ALWAYS AS IDENTITY UNIQUE, -- insertion order (the SQLite rowid tiebreak)
+ project_id text COLLATE "C" NOT NULL REFERENCES host_projects(id),
+ site_id text COLLATE "C" NOT NULL REFERENCES host_sites(id),
+ hostname text COLLATE "C" NOT NULL,
+ kind text COLLATE "C" NOT NULL CHECK (kind IN ('default','custom')),
+ status text COLLATE "C" NOT NULL CHECK (status IN ('pending','verified','failed','lapsed')),
+ token text COLLATE "C", -- the TXT verification value (custom only; public in DNS, not a secret)
+ created_by text COLLATE "C" NOT NULL,
+ created_at bigint NOT NULL,
+ verified_at bigint,
+ last_checked_at bigint,
+ last_error text COLLATE "C",
+ check_count bigint NOT NULL DEFAULT 0,
+ record_missing_since bigint -- verified domain whose TXT record disappeared
+);
+CREATE UNIQUE INDEX host_domains_verified ON host_domains (hostname) WHERE status = 'verified';
+CREATE UNIQUE INDEX host_domains_site_host ON host_domains (site_id, hostname);
+CREATE INDEX host_domains_status ON host_domains (kind, status, last_checked_at);
+
+-- Abuse takedowns (staff only): a site, or every site of a project, stops being served (451) while
+-- its deploys, files and objects are KEPT for review. Rows are never deleted: a lift is recorded.
+CREATE TABLE host_takedowns (
+ id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
+ target_kind text COLLATE "C" NOT NULL CHECK (target_kind IN ('project','site')),
+ target_id text COLLATE "C" NOT NULL, -- prj_… or site_…
+ reason text COLLATE "C" NOT NULL,
+ created_by text COLLATE "C" NOT NULL,
+ created_at bigint NOT NULL,
+ lifted_by text COLLATE "C",
+ lifted_at bigint,
+ lift_note text COLLATE "C"
+);
+CREATE UNIQUE INDEX host_takedowns_active ON host_takedowns (target_kind, target_id) WHERE lifted_at IS NULL;
+
+-- Deploy artifacts are immutable: only state / deleted_at may change, and never back from 'deleted'.
+CREATE FUNCTION host_deploys_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF TRUE THEN RAISE EXCEPTION 'host: deploy artifacts are immutable'; END IF; RETURN NEW; END $$;
+CREATE TRIGGER host_deploys_immutable BEFORE UPDATE OF id, project_id, site_id, source, manifest, manifest_sha256, file_count, total_bytes, new_bytes, created_by, created_at ON host_deploys FOR EACH ROW EXECUTE FUNCTION host_deploys_immutable();
+CREATE FUNCTION host_deploys_no_undelete() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF OLD.state = 'deleted' OR (OLD.state = 'failed' AND NEW.state = 'ready') THEN RAISE EXCEPTION 'host: a deleted or failed deploy cannot become ready'; END IF; RETURN NEW; END $$;
+CREATE TRIGGER host_deploys_no_undelete BEFORE UPDATE OF state ON host_deploys FOR EACH ROW EXECUTE FUNCTION host_deploys_no_undelete();
+CREATE FUNCTION host_deploy_files_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF TRUE THEN RAISE EXCEPTION 'host: deploy files are immutable'; END IF; RETURN NEW; END $$;
+CREATE TRIGGER host_deploy_files_immutable BEFORE UPDATE ON host_deploy_files FOR EACH ROW EXECUTE FUNCTION host_deploy_files_immutable();
+CREATE FUNCTION host_deploy_files_delete_only_deleted() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN IF (SELECT state FROM host_deploys WHERE id = OLD.deploy_id) <> 'deleted' THEN RAISE EXCEPTION 'host: files of a live deploy cannot be removed'; END IF; RETURN OLD; END $$;
+CREATE TRIGGER host_deploy_files_delete_only_deleted BEFORE DELETE ON host_deploy_files FOR EACH ROW EXECUTE FUNCTION host_deploy_files_delete_only_deleted();
+
+-- openvibe-sdk/events inbox: one receipt per (consumer, event) handled
+CREATE TABLE IF NOT EXISTS idempotency_receipts (
+ consumer text NOT NULL,
+ event_id text NOT NULL,
+ processed_at bigint NOT NULL,
+ PRIMARY KEY (consumer, event_id)
+);
+
+-- openvibe-sdk/events PostgreSQL outbox
+CREATE TABLE IF NOT EXISTS event_outbox (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ event_id text NOT NULL UNIQUE,
+ envelope jsonb NOT NULL,
+ traceparent text,
+ created_at bigint NOT NULL,
+ attempts integer NOT NULL DEFAULT 0,
+ next_attempt_at bigint NOT NULL DEFAULT 0,
+ sent_at bigint,
+ seq bigint,
+ rejected_at bigint,
+ last_error text
+);
+CREATE INDEX IF NOT EXISTS event_outbox_due ON event_outbox (next_attempt_at, id) WHERE sent_at IS NULL AND rejected_at IS NULL;
+CREATE INDEX IF NOT EXISTS event_outbox_sent ON event_outbox (sent_at) WHERE sent_at IS NOT NULL;
@@ -16,14 +16,17 @@
@@ -16,14 +16,17 @@
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"express": "^4.21.2",
"express-rate-limit": "^7.5.0",
"express-rate-limit": "^7.5.0",
+ "iovalkey": "^0.4.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",
- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",
- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"
+ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",
+ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",
+ "pg": "^8.23.0"
},
},
"bin": {
"bin": {
"ovhost": "bin/ovhost"
"ovhost": "bin/ovhost"
},
},
"devDependencies": {
"devDependencies": {
+ "@electric-sql/pglite": "^0.5.8",
"jsonwebtoken": "^9.0.2"
"jsonwebtoken": "^9.0.2"
},
},
"engines": {
"engines": {
@@ -338,6 +341,19 @@
@@ -338,6 +341,19 @@
"node": ">=18.0.0"
"node": ">=18.0.0"
}
}
},
},
+ "node_modules/@electric-sql/pglite": {
+ "version": "0.5.8",
+ "resolved": "https://registry.npmjs.org/@electric-sql/pglite/-/pglite-0.5.8.tgz",
+ "integrity": "sha512-n9tsbUOhwx2epK1V0ZG9Ar4SHWUju04dhmzZXiSBXwBoleOvIfals33NAaWgagQVAL4Rbvx/Ptsu3P+pA09f6Q==",
+ "devOptional": true,
+ "license": "Apache-2.0"
+ },
+ "node_modules/@iovalkey/commands": {
+ "version": "0.1.0",
+ "resolved": "https://registry.npmjs.org/@iovalkey/commands/-/commands-0.1.0.tgz",
+ "integrity": "sha512-/B9W4qKSSITDii5nkBCHyPkIkAi+ealUtr1oqBJsLxjSRLka4pxun2VvMNSmcwgAMxgXtQfl0qRv7TE+udPJzg==",
+ "license": "MIT"
+ },
"node_modules/@smithy/core": {
"node_modules/@smithy/core": {
"version": "3.35.0",
"version": "3.35.0",
"resolved": "https://registry.npmjs.org/@smithy/core/-/core-3.35.0.tgz",
"resolved": "https://registry.npmjs.org/@smithy/core/-/core-3.35.0.tgz",
@@ -637,6 +653,15 @@
@@ -637,6 +653,15 @@
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"integrity": "sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg==",
"license": "ISC"
"license": "ISC"
},
},
+ "node_modules/cluster-key-slot": {
+ "version": "1.1.2",
+ "resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",
+ "integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==",
+ "license": "Apache-2.0",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
"node_modules/content-disposition": {
"node_modules/content-disposition": {
"version": "0.5.4",
"version": "0.5.4",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
"resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
@@ -719,6 +744,15 @@
@@ -719,6 +744,15 @@
"node": ">=4.0.0"
"node": ">=4.0.0"
}
}
},
},
+ "node_modules/denque": {
+ "version": "2.1.0",
+ "resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
+ "integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",
+ "license": "Apache-2.0",
+ "engines": {
+ "node": ">=0.10"
+ }
+ },
"node_modules/depd": {
"node_modules/depd": {
"version": "2.0.0",
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
@@ -1143,6 +1177,49 @@
@@ -1143,6 +1177,49 @@
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"integrity": "sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew==",
"license": "ISC"
"license": "ISC"
},
},
+ "node_modules/iovalkey": {
+ "version": "0.4.0",
+ "resolved": "https://registry.npmjs.org/iovalkey/-/iovalkey-0.4.0.tgz",
+ "integrity": "sha512-OSUKxJ+s44CLdUTaicX4+pVrBN/zHKIYwr2oKhRDuczr19FwcSCXEZHzcMUYWZNh1mNSCV9bNJHW/T6cHVm9Aw==",
+ "license": "MIT",
+ "dependencies": {
+ "@iovalkey/commands": "^0.1.0",
+ "cluster-key-slot": "^1.1.0",
+ "debug": "^4.3.4",
+ "denque": "^2.1.0",
+ "lodash.defaults": "^4.2.0",
+ "lodash.isarguments": "^3.1.0",
+ "redis-errors": "^1.2.0",
+ "redis-parser": "^3.0.0",
+ "standard-as-callback": "^2.1.0"
+ },
+ "engines": {
+ "node": ">=18.12.0"
+ }
+ },
+ "node_modules/iovalkey/node_modules/debug": {
+ "version": "4.4.3",
+ "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
+ "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
+ "license": "MIT",
+ "dependencies": {
+ "ms": "^2.1.3"
+ },
+ "engines": {
+ "node": ">=6.0"
+ },
+ "peerDependenciesMeta": {
+ "supports-color": {
+ "optional": true
+ }
+ }
+ },
+ "node_modules/iovalkey/node_modules/ms": {
+ "version": "2.1.3",
+ "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
+ "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
+ "license": "MIT"
+ },
"node_modules/ipaddr.js": {
"node_modules/ipaddr.js": {
"version": "1.9.1",
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
@@ -1207,12 +1284,24 @@
@@ -1207,12 +1284,24 @@
"safe-buffer": "^5.0.1"
"safe-buffer": "^5.0.1"
}
}
},
},
+ "node_modules/lodash.defaults": {
+ "version": "4.2.0",
+ "resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz",
+ "integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==",
+ "license": "MIT"
+ },
"node_modules/lodash.includes": {
"node_modules/lodash.includes": {
"version": "4.3.0",
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz",
"resolved": "https://registry.npmjs.org/lodash.includes/-/lodash.includes-4.3.0.tgz",
"integrity": "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w==",
"integrity": "sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w==",
"license": "MIT"
"license": "MIT"
},
},
+ "node_modules/lodash.isarguments": {
+ "version": "3.1.0",
+ "resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz",
+ "integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==",
+ "license": "MIT"
+ },
"node_modules/lodash.isboolean": {
"node_modules/lodash.isboolean": {
"version": "3.0.3",
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz",
"resolved": "https://registry.npmjs.org/lodash.isboolean/-/lodash.isboolean-3.0.3.tgz",
@@ -1428,18 +1517,38 @@
@@ -1428,18 +1517,38 @@
}
}
},
},
"node_modules/openvibe-sdk": {
"node_modules/openvibe-sdk": {
- "version": "0.12.0",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",
- "integrity": "sha512-iG1h905O1f7y/PnP8xLA6GCVzXEBDfSaBj2tc1ucNreWg2sT/AJWosb8NW0WRMK5WS4sIQ6tMsprmDLj/fjjeQ==",
+ "version": "0.20.3",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",
+ "integrity": "sha512-+g+ySveItzOBnUsw9aZZIJVKRrGuj0aO328W4N25T0Z8mY4fE6++ImvLnZ55LH2BRh0WYozC1KAVIShACm2KGQ==",
"license": "MIT",
"license": "MIT",
"engines": {
"engines": {
"node": ">=20"
"node": ">=20"
+ },
+ "peerDependencies": {
+ "@electric-sql/pglite": "^0.5.8",
+ "better-sqlite3": ">=11",
+ "iovalkey": "^0.4.0",
+ "pg": "^8.23.0"
+ },
+ "peerDependenciesMeta": {
+ "@electric-sql/pglite": {
+ "optional": true
+ },
+ "better-sqlite3": {
+ "optional": true
+ },
+ "iovalkey": {
+ "optional": true
+ },
+ "pg": {
+ "optional": true
+ }
}
}
},
},
"node_modules/openvibe-shared": {
"node_modules/openvibe-shared": {
- "version": "1.25.0",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0",
- "integrity": "sha512-oqZ+PvljrWi8IEbtg6bDYai17ZV2sALFx/ezMptniELSr10TgFUiGUQt+b1IWwiHv0T2Y1U2iSrkirJxMNgjSw==",
+ "version": "1.27.0",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",
+ "integrity": "sha512-/H7dyFw+cURcdjyfbpyY7mNc6F+Pcb9TwkVwZHR+9d3M62q70KDhZLo/gZi71eaYxUjD8lO8tmbXu70EAa9paQ==",
"license": "MIT",
"license": "MIT",
"dependencies": {
"dependencies": {
"jsonwebtoken": "^9.0.0"
"jsonwebtoken": "^9.0.0"
@@ -1460,6 +1569,134 @@
@@ -1460,6 +1569,134 @@
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"license": "MIT"
"license": "MIT"
},
},
+ "node_modules/pg": {
+ "version": "8.23.0",
+ "resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",
+ "integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",
+ "license": "MIT",
+ "dependencies": {
+ "pg-connection-string": "^2.14.0",
+ "pg-pool": "^3.14.0",
+ "pg-protocol": "^1.16.0",
+ "pg-types": "2.2.0",
+ "pgpass": "1.0.5"
+ },
+ "engines": {
+ "node": ">= 16.0.0"
+ },
+ "optionalDependencies": {
+ "pg-cloudflare": "^1.4.0"
+ },
+ "peerDependencies": {
+ "pg-native": ">=3.0.1"
+ },
+ "peerDependenciesMeta": {
+ "pg-native": {
+ "optional": true
+ }
+ }
+ },
+ "node_modules/pg-cloudflare": {
+ "version": "1.4.0",
+ "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",
+ "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",
+ "license": "MIT",
+ "optional": true
+ },
+ "node_modules/pg-connection-string": {
+ "version": "2.14.0",
+ "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",
+ "integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",
+ "license": "MIT"
+ },
+ "node_modules/pg-int8": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
+ "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
+ "license": "ISC",
+ "engines": {
+ "node": ">=4.0.0"
+ }
+ },
+ "node_modules/pg-pool": {
+ "version": "3.14.0",
+ "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",
+ "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",
+ "license": "MIT",
+ "peerDependencies": {
+ "pg": ">=8.0"
+ }
+ },
+ "node_modules/pg-protocol": {
+ "version": "1.16.0",
+ "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz",
+ "integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==",
+ "license": "MIT"
+ },
+ "node_modules/pg-types": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
+ "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
+ "license": "MIT",
+ "dependencies": {
+ "pg-int8": "1.0.1",
+ "postgres-array": "~2.0.0",
+ "postgres-bytea": "~1.0.0",
+ "postgres-date": "~1.0.4",
+ "postgres-interval": "^1.1.0"
+ },
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/pgpass": {
+ "version": "1.0.5",
+ "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
+ "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
+ "license": "MIT",
+ "dependencies": {
+ "split2": "^4.1.0"
+ }
+ },
+ "node_modules/postgres-array": {
+ "version": "2.0.0",
+ "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
+ "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/postgres-bytea": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
+ "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/postgres-date": {
+ "version": "1.0.7",
+ "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
+ "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/postgres-interval": {
+ "version": "1.2.0",
+ "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
+ "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
+ "license": "MIT",
+ "dependencies": {
+ "xtend": "^4.0.0"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
"node_modules/prebuild-install": {
"node_modules/prebuild-install": {
"version": "7.1.3",
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
@@ -1583,6 +1820,27 @@
@@ -1583,6 +1820,27 @@
"node": ">= 6"
"node": ">= 6"
}
}
},
},
+ "node_modules/redis-errors": {
+ "version": "1.2.0",
+ "resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",
+ "integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/redis-parser": {
+ "version": "3.0.0",
+ "resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz",
+ "integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==",
+ "license": "MIT",
+ "dependencies": {
+ "redis-errors": "^1.0.0"
+ },
+ "engines": {
+ "node": ">=4"
+ }
+ },
"node_modules/require-from-string": {
"node_modules/require-from-string": {
"version": "2.0.2",
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
"resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz",
@@ -1798,6 +2056,21 @@
@@ -1798,6 +2056,21 @@
"simple-concat": "^1.0.0"
"simple-concat": "^1.0.0"
}
}
},
},
+ "node_modules/split2": {
+ "version": "4.2.0",
+ "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
+ "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
+ "license": "ISC",
+ "engines": {
+ "node": ">= 10.x"
+ }
+ },
+ "node_modules/standard-as-callback": {
+ "version": "2.1.0",
+ "resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz",
+ "integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==",
+ "license": "MIT"
+ },
"node_modules/statuses": {
"node_modules/statuses": {
"version": "2.0.2",
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
@@ -1939,6 +2212,15 @@
@@ -1939,6 +2212,15 @@
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC"
"license": "ISC"
+ },
+ "node_modules/xtend": {
+ "version": "4.0.2",
+ "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
+ "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.4"
+ }
}
}
}
}
}
}
@@ -10,7 +10,8 @@
@@ -10,7 +10,8 @@
"scripts": {
"scripts": {
"start": "node server/index.js",
"start": "node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
- "test": "node test/run.js"
+ "test": "node test/run.js",
+ "test:pg": "HOST_TEST_STORE=pg node test/run.js"
},
},
"keywords": [
"keywords": [
"openvibe",
"openvibe",
@@ -31,14 +32,17 @@
@@ -31,14 +32,17 @@
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"express": "^4.21.2",
"express-rate-limit": "^7.5.0",
"express-rate-limit": "^7.5.0",
+ "iovalkey": "^0.4.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",
"openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",
- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.12.0",
- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"
+ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",
+ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",
+ "pg": "^8.23.0"
},
},
"engines": {
"engines": {
"node": ">=22.0.0"
"node": ">=22.0.0"
},
},
"devDependencies": {
"devDependencies": {
- "jsonwebtoken": "^9.0.2"
+ "jsonwebtoken": "^9.0.2",
+ "@electric-sql/pglite": "^0.5.8"
}
}
}
}
@@ -1,7 +1,7 @@
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
#!/usr/bin/env bash
# roles/data/switch-service.sh — the production switch of one service from its SQLite file to PostgreSQL (ADR-035).
# roles/data/switch-service.sh — the production switch of one service from its SQLite file to PostgreSQL (ADR-035).
#
#
-# sudo roles/data/switch-service.sh <service>
+# sudo roles/data/switch-service.sh <service> [<sqlite file>]
#
#
# Before: the service's postgres branch is merged to main with green CI, its auto-deploy is frozen
# Before: the service's postgres branch is merged to main with green CI, its auto-deploy is frozen
# (ovhost freeze <svc> --reason …), and add-service.sh <svc> has written its database settings.
# (ovhost freeze <svc> --reason …), and add-service.sh <svc> has written its database settings.
@@ -13,6 +13,8 @@
@@ -13,6 +13,8 @@
# register C-89: sudo ovhost rollback <svc> to the release before, which reads the file).
# register C-89: sudo ovhost rollback <svc> to the release before, which reads the file).
set -euo pipefail
set -euo pipefail
SVC=$1; UNIT=openvibe-$SVC.service; DIR=/opt/openvibe.$SVC; DATA=/var/lib/openvibe-$SVC; DB=$DATA/$SVC.db
SVC=$1; UNIT=openvibe-$SVC.service; DIR=/opt/openvibe.$SVC; DATA=/var/lib/openvibe-$SVC; DB=$DATA/$SVC.db
+# A second argument names the SQLite file when it is not /var/lib/openvibe-<svc>/<svc>.db (Host: /var/lib/openvibe-host-api/host.db).
+if [ -n "${2:-}" ]; then DB=$2; DATA=$(dirname "$DB"); fi
WORK=/var/tmp/pg-$SVC; UB=$(id -u ubuntu); GB=$(id -g ubuntu)
WORK=/var/tmp/pg-$SVC; UB=$(id -u ubuntu); GB=$(id -g ubuntu)
[ -f "$DB" ] || { echo "no $DB"; exit 1; }
[ -f "$DB" ] || { echo "no $DB"; exit 1; }
rm -rf "$WORK"; install -d -o ubuntu -g ubuntu "$WORK"
rm -rf "$WORK"; install -d -o ubuntu -g ubuntu "$WORK"
new file mode 100644
new file mode 100644
@@ -0,0 +1,26 @@
@@ -0,0 +1,26 @@
+#!/usr/bin/env node
+'use strict';
+/**
+ * The one-time move of Host's SQLite database (HOST_DB_PATH) into its PostgreSQL schema (ADR-035; the procedure is
+ * openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by openvibe-sdk/db runSqliteMigration).
+ *
+ * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]
+ *
+ * Applies migrations/ as the owner (DATABASE_DIRECT_URL), copies every table into emptied tables, verifies row counts
+ * and checksums, and exits 1 unless everything verified. The SQLite file is opened read-only. Every table keeps its
+ * name and columns.
+ */
+require('dotenv').config();
+const { runSqliteMigration } = require('openvibe-sdk/db');
+const configLib = require('../server/config');
+const { MIGRATIONS } = require('../server/db');
+
+const TABLES = {};
+
+if (require.main === module) {
+ const config = (configLib.load || configLib.loadConfig)();
+ runSqliteMigration({ service: 'host', sqlite: config.dbPath, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES })
+ .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });
+}
+
+module.exports = { TABLES };
@@ -65,10 +65,11 @@ const DASHBOARD_CSP_HEADER = Object.entries(DASHBOARD_CSP).map(([k, v]) => `${k}
@@ -65,10 +65,11 @@ const DASHBOARD_CSP_HEADER = Object.entries(DASHBOARD_CSP).map(([k, v]) => `${k}
* opts: config, store | dbPath, now (clock), fetchImpl, auth (a createAuthClient-like object),
* opts: config, store | dbPath, now (clock), fetchImpl, auth (a createAuthClient-like object),
* resolver ({ resolveTxt }), log
* resolver ({ resolveTxt }), log
*/
*/
-function createApp(opts = {}) {
+async function createApp(opts = {}) {
const config = opts.config || configLib.load();
const config = opts.config || configLib.load();
const log = opts.log || console;
const log = opts.log || console;
- const store = opts.store || openStore(opts.dbPath || config.dbPath, { now: opts.now });
+ // PostgreSQL (ADR-035): opened and migrated here unless the caller (a test, a script) hands in a store.
+ const store = opts.store || await openStore(config, { now: opts.now, log });
const blobs = createBlobStore(opts.storageDir || config.storageDir);
const blobs = createBlobStore(opts.storageDir || config.storageDir);
require('./http/errors').setLogger(log);
require('./http/errors').setLogger(log);
@@ -97,11 +98,14 @@ function createApp(opts = {}) {
@@ -97,11 +98,14 @@ function createApp(opts = {}) {
const httpMetrics = sharedMetrics.httpMetrics(registry, { normalize: (req) => (req.hostTarget === 'site' ? 'tenant_site' : req.hostTarget === 'unknown' ? 'unknown_host' : null) });
const httpMetrics = sharedMetrics.httpMetrics(registry, { normalize: (req) => (req.hostTarget === 'site' ? 'tenant_site' : req.hostTarget === 'unknown' ? 'unknown_host' : null) });
const proc = sharedMetrics.processMetrics(registry);
const proc = sharedMetrics.processMetrics(registry);
// Per-actor limits on writes (server/http/actor-limits.js; roadmap WS-R task 4), one budget for the API and the dashboard.
// Per-actor limits on writes (server/http/actor-limits.js; roadmap WS-R task 4), one budget for the API and the dashboard.
- ctx.actorLimits = require('./http/actor-limits').createHostActorLimits({ registry });
+ // Valkey (ADR-035): shared, never-authoritative state (per-actor limit counters). Optional.
+ const valkey = opts.valkey !== undefined ? opts.valkey : (config.valkey.url ? require('openvibe-sdk/valkey').createValkey({ url: config.valkey.url, prefix: config.valkey.prefix, log }) : null);
+ ctx.valkey = valkey;
+ ctx.actorLimits = require('./http/actor-limits').createHostActorLimits({ registry, valkey });
sharedMetrics.releaseInfo(registry, { service: 'host', release: release.release });
sharedMetrics.releaseInfo(registry, { service: 'host', release: release.release });
registry.gauge({
registry.gauge({
name: 'host_sites', help: 'Active tenant sites',
name: 'host_sites', help: 'Active tenant sites',
- collect: () => [{ labels: {}, value: store.db.prepare("SELECT COUNT(*) AS n FROM host_sites WHERE status = 'active'").get().n }],
+ collect: async () => [{ labels: {}, value: (await store.db.prepare("SELECT COUNT(*) AS n FROM host_sites WHERE status = 'active'").get()).n }],
});
});
ctx.stopMetrics = proc.stop;
ctx.stopMetrics = proc.stop;
app.locals.metrics = registry;
app.locals.metrics = registry;
@@ -110,11 +114,11 @@ function createApp(opts = {}) {
@@ -110,11 +114,11 @@ function createApp(opts = {}) {
app.use(httpMetrics.middleware);
app.use(httpMetrics.middleware);
// ── Host dispatch: tenant sites never reach anything below ──
// ── Host dispatch: tenant sites never reach anything below ──
- app.use((req, res, next) => {
- const target = tenant.resolve(req.headers.host);
+ app.use(async (req, res, next) => {
+ const target = await tenant.resolve(req.headers.host);
req.hostTarget = target.kind;
req.hostTarget = target.kind;
if (target.kind === 'dashboard') return next();
if (target.kind === 'dashboard') return next();
- if (target.kind === 'site') return tenant.handle(req, res, target.site);
+ if (target.kind === 'site') return await tenant.handle(req, res, target.site);
return tenant.unknownHost(req, res);
return tenant.unknownHost(req, res);
});
});
@@ -135,7 +139,7 @@ function createApp(opts = {}) {
@@ -135,7 +139,7 @@ function createApp(opts = {}) {
release.mount(app, { registry: registry });
release.mount(app, { registry: registry });
// GET /limits.json: the developer limits enforced here, from config (WS-N task 7; Codes renders them).
// GET /limits.json: the developer limits enforced here, from config (WS-N task 7; Codes renders them).
require('./limits').mountLimits(app, config);
require('./limits').mountLimits(app, config);
- const readiness = createHostReadiness({ store, blobs, auth, outbox, release: release.release, minFreeBytes: () => config.uploads.minFreeBytes });
+ const readiness = createHostReadiness({ store, blobs, auth, outbox, release: release.release, minFreeBytes: () => config.uploads.minFreeBytes, valkey: ctx.valkey });
app.get('/api/ready', readiness.handler);
app.get('/api/ready', readiness.handler);
app.get('/metrics', sharedMetrics.metricsHandler(registry));
app.get('/metrics', sharedMetrics.metricsHandler(registry));
// The dashboard host: only the public front page and the legal pages are crawlable. Tenant sites
// The dashboard host: only the public front page and the legal pages are crawlable. Tenant sites
@@ -71,7 +71,7 @@ function createViewerResolver({ auth, config }) {
@@ -71,7 +71,7 @@ function createViewerResolver({ auth, config }) {
const token = bearerToken(req);
const token = bearerToken(req);
if (!token) return ANONYMOUS;
if (!token) return ANONYMOUS;
const payload = decodeJwtPayload(token);
const payload = decodeJwtPayload(token);
- if (payload && typeof payload.sub === 'string' && PRINCIPAL_SUB.test(payload.sub)) return fromServiceToken(req, token);
+ if (payload && typeof payload.sub === 'string' && PRINCIPAL_SUB.test(payload.sub)) return await fromServiceToken(req, token);
const v = await fromUserToken(token);
const v = await fromUserToken(token);
if (!v) throw new ViewerError(401, 'auth.invalid_token', 'the bearer token is not a valid OpenVibe token');
if (!v) throw new ViewerError(401, 'auth.invalid_token', 'the bearer token is not a valid OpenVibe token');
return v;
return v;
@@ -44,6 +44,10 @@ function load(env = process.env) {
@@ -44,6 +44,10 @@ function load(env = process.env) {
sitesDomain,
sitesDomain,
trustProxy: env.TRUST_PROXY != null ? Number(env.TRUST_PROXY) : 1, // nginx (realip) → Node
trustProxy: env.TRUST_PROXY != null ? Number(env.TRUST_PROXY) : 1, // nginx (realip) → Node
+ // PostgreSQL (ADR-035): DATABASE_URL serves (PgBouncer), DATABASE_DIRECT_URL migrates (owner role).
+ db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '' },
+ valkey: { url: env.VALKEY_URL || '', prefix: env.VALKEY_PREFIX || 'ov:host:' },
+ // The SQLite file of releases before the switch: read once by scripts/migrate-to-postgres.js.
dbPath: env.HOST_DB_PATH || './data/host.db',
dbPath: env.HOST_DB_PATH || './data/host.db',
// Content-addressed deploy objects, one directory per project (tenancy keyed by project id).
// Content-addressed deploy objects, one directory per project (tenancy keyed by project id).
storageDir: env.HOST_STORAGE_DIR || './data/objects',
storageDir: env.HOST_STORAGE_DIR || './data/objects',
@@ -1,7 +1,7 @@
@@ -1,7 +1,7 @@
'use strict';
'use strict';
/**
/**
- * Host's own SQLite database (WAL), created on boot, idempotently. Nothing here is shared with
+ * Host's own PostgreSQL database (ADR-035, roadmap WS-X2): the schema is migrations/NNNN_*.sql, applied at boot. Nothing here is shared with
* another service.
* another service.
*
*
* host_projects tenant projects: owner subject (usr_…), environment (production|sandbox),
* host_projects tenant projects: owner subject (usr_…), environment (production|sandbox),
@@ -23,181 +23,49 @@
@@ -23,181 +23,49 @@
*/
*/
const fs = require('fs');
const fs = require('fs');
const path = require('path');
const path = require('path');
-const Database = require('better-sqlite3');
+const { createDb } = require('openvibe-sdk/db');
-const SCHEMA = `
-CREATE TABLE IF NOT EXISTS host_projects (
- id TEXT PRIMARY KEY, -- prj_<ULID>
- network_project_id TEXT, -- OpenVibe.Network project id (ADR-014), when it exists
- owner_subject TEXT NOT NULL, -- usr_…
- name TEXT NOT NULL,
- environment TEXT NOT NULL DEFAULT 'production' CHECK (environment IN ('production','sandbox')),
- status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active','deleted')),
- created_at INTEGER NOT NULL,
- updated_at INTEGER NOT NULL,
- deleted_at INTEGER
-);
-CREATE UNIQUE INDEX IF NOT EXISTS host_projects_network ON host_projects (network_project_id) WHERE network_project_id IS NOT NULL AND status = 'active';
-CREATE INDEX IF NOT EXISTS host_projects_owner ON host_projects (owner_subject, status);
-
-CREATE TABLE IF NOT EXISTS host_project_members (
- project_id TEXT NOT NULL REFERENCES host_projects(id),
- principal TEXT NOT NULL, -- usr_… | app:app_… | svc:<id>
- role TEXT NOT NULL CHECK (role IN ('owner','maintainer','deployer')),
- added_by TEXT,
- created_at INTEGER NOT NULL,
- PRIMARY KEY (project_id, principal)
-);
-CREATE INDEX IF NOT EXISTS host_project_members_principal ON host_project_members (principal);
-
-CREATE TABLE IF NOT EXISTS host_quotas (
- project_id TEXT PRIMARY KEY REFERENCES host_projects(id),
- storage_bytes INTEGER,
- deploys_per_day INTEGER,
- max_files INTEGER,
- max_file_bytes INTEGER,
- sites INTEGER,
- custom_domains INTEGER,
- updated_by TEXT,
- updated_at INTEGER NOT NULL
-);
-
-CREATE TABLE IF NOT EXISTS host_sites (
- id TEXT PRIMARY KEY, -- site_<ULID>
- project_id TEXT NOT NULL REFERENCES host_projects(id),
- name TEXT NOT NULL, -- the <name>.openvibe.host label
- active_deploy_id TEXT,
- status TEXT NOT NULL DEFAULT 'active' CHECK (status IN ('active','deleted')),
- created_by TEXT NOT NULL,
- created_at INTEGER NOT NULL,
- updated_at INTEGER NOT NULL,
- deleted_at INTEGER
-);
-CREATE UNIQUE INDEX IF NOT EXISTS host_sites_name ON host_sites (name) WHERE status = 'active';
-CREATE INDEX IF NOT EXISTS host_sites_project ON host_sites (project_id, status);
-
-CREATE TABLE IF NOT EXISTS host_deploys (
- id TEXT PRIMARY KEY, -- dpl_<ULID>
- project_id TEXT NOT NULL REFERENCES host_projects(id),
- site_id TEXT NOT NULL REFERENCES host_sites(id),
- state TEXT NOT NULL CHECK (state IN ('ready','failed','deleted')),
- source TEXT NOT NULL CHECK (source IN ('archive','files')),
- manifest TEXT, -- JSON host.deploy-manifest@1 (NULL when failed)
- manifest_sha256 TEXT,
- file_count INTEGER NOT NULL DEFAULT 0,
- total_bytes INTEGER NOT NULL DEFAULT 0,
- new_bytes INTEGER NOT NULL DEFAULT 0, -- bytes this deploy added to the project's storage
- failure_code TEXT,
- created_by TEXT NOT NULL,
- created_at INTEGER NOT NULL,
- deleted_at INTEGER
-);
-CREATE INDEX IF NOT EXISTS host_deploys_site ON host_deploys (site_id, created_at);
-CREATE INDEX IF NOT EXISTS host_deploys_project_day ON host_deploys (project_id, created_at);
-
-CREATE TABLE IF NOT EXISTS host_deploy_files (
- deploy_id TEXT NOT NULL REFERENCES host_deploys(id),
- path TEXT NOT NULL,
- sha256 TEXT NOT NULL,
- size INTEGER NOT NULL,
- content_type TEXT NOT NULL,
- PRIMARY KEY (deploy_id, path)
-) WITHOUT ROWID;
-CREATE INDEX IF NOT EXISTS host_deploy_files_sha ON host_deploy_files (sha256);
-
-CREATE TABLE IF NOT EXISTS host_blobs (
- project_id TEXT NOT NULL,
- sha256 TEXT NOT NULL,
- size INTEGER NOT NULL,
- created_at INTEGER NOT NULL,
- PRIMARY KEY (project_id, sha256)
-) WITHOUT ROWID;
-
-CREATE TABLE IF NOT EXISTS host_activations (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- site_id TEXT NOT NULL REFERENCES host_sites(id),
- deploy_id TEXT NOT NULL,
- previous_deploy_id TEXT,
- kind TEXT NOT NULL CHECK (kind IN ('activate','rollback')),
- actor TEXT NOT NULL,
- created_at INTEGER NOT NULL
-);
-CREATE INDEX IF NOT EXISTS host_activations_site ON host_activations (site_id, id);
-
-CREATE TABLE IF NOT EXISTS host_deploy_logs (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- deploy_id TEXT NOT NULL,
- level TEXT NOT NULL CHECK (level IN ('info','warn','error')),
- message TEXT NOT NULL,
- created_at INTEGER NOT NULL
-);
-CREATE INDEX IF NOT EXISTS host_deploy_logs_deploy ON host_deploy_logs (deploy_id, id);
-
-CREATE TABLE IF NOT EXISTS host_domains (
- id TEXT PRIMARY KEY, -- dom_<ULID>
- project_id TEXT NOT NULL REFERENCES host_projects(id),
- site_id TEXT NOT NULL REFERENCES host_sites(id),
- hostname TEXT NOT NULL,
- kind TEXT NOT NULL CHECK (kind IN ('default','custom')),
- status TEXT NOT NULL CHECK (status IN ('pending','verified','failed','lapsed')),
- token TEXT, -- the TXT verification value (custom only; public in DNS, not a secret)
- created_by TEXT NOT NULL,
- created_at INTEGER NOT NULL,
- verified_at INTEGER,
- last_checked_at INTEGER,
- last_error TEXT,
- check_count INTEGER NOT NULL DEFAULT 0,
- record_missing_since INTEGER -- verified domain whose TXT record disappeared
-);
-CREATE UNIQUE INDEX IF NOT EXISTS host_domains_verified ON host_domains (hostname) WHERE status = 'verified';
-CREATE UNIQUE INDEX IF NOT EXISTS host_domains_site_host ON host_domains (site_id, hostname);
-CREATE INDEX IF NOT EXISTS host_domains_status ON host_domains (kind, status, last_checked_at);
-
-CREATE TABLE IF NOT EXISTS host_takedowns (
- id INTEGER PRIMARY KEY,
- target_kind TEXT NOT NULL CHECK (target_kind IN ('project','site')),
- target_id TEXT NOT NULL, -- prj_… or site_…
- reason TEXT NOT NULL,
- created_by TEXT NOT NULL,
- created_at INTEGER NOT NULL,
- lifted_by TEXT,
- lifted_at INTEGER,
- lift_note TEXT
-);
-CREATE UNIQUE INDEX IF NOT EXISTS host_takedowns_active ON host_takedowns (target_kind, target_id) WHERE lifted_at IS NULL;
-
-CREATE TRIGGER IF NOT EXISTS host_deploys_immutable
-BEFORE UPDATE OF id, project_id, site_id, source, manifest, manifest_sha256, file_count, total_bytes, new_bytes, created_by, created_at ON host_deploys
-BEGIN SELECT RAISE(ABORT, 'host: deploy artifacts are immutable'); END;
-CREATE TRIGGER IF NOT EXISTS host_deploys_no_undelete
-BEFORE UPDATE OF state ON host_deploys WHEN OLD.state = 'deleted' OR (OLD.state = 'failed' AND NEW.state = 'ready')
-BEGIN SELECT RAISE(ABORT, 'host: a deleted or failed deploy cannot become ready'); END;
-CREATE TRIGGER IF NOT EXISTS host_deploy_files_immutable
-BEFORE UPDATE ON host_deploy_files
-BEGIN SELECT RAISE(ABORT, 'host: deploy files are immutable'); END;
-CREATE TRIGGER IF NOT EXISTS host_deploy_files_delete_only_deleted
-BEFORE DELETE ON host_deploy_files WHEN (SELECT state FROM host_deploys WHERE id = OLD.deploy_id) <> 'deleted'
-BEGIN SELECT RAISE(ABORT, 'host: files of a live deploy cannot be removed'); END;
-`;
+const MIGRATIONS = path.join(__dirname, '..', 'migrations');
+const DEV_PGLITE = path.join(__dirname, '..', 'data', 'pglite');
const CHARTER_TABLES = ['host_projects', 'host_project_members', 'host_quotas', 'host_sites', 'host_deploys', 'host_deploy_files', 'host_blobs', 'host_activations', 'host_deploy_logs', 'host_domains', 'host_takedowns'];
const CHARTER_TABLES = ['host_projects', 'host_project_members', 'host_quotas', 'host_sites', 'host_deploys', 'host_deploy_files', 'host_blobs', 'host_activations', 'host_deploy_logs', 'host_domains', 'host_takedowns'];
-function openStore(dbPath, { now = () => Date.now() } = {}) {
- if (dbPath !== ':memory:') fs.mkdirSync(path.dirname(path.resolve(dbPath)), { recursive: true });
- const db = new Database(dbPath);
- db.pragma('journal_mode = WAL');
- db.pragma('foreign_keys = ON');
- db.pragma('busy_timeout = 5000');
- db.exec(SCHEMA);
+/**
+ * The serving handle (ADR-035): DATABASE_URL through PgBouncer; in development without it, an embedded PGlite database
+ * in data/pglite. Migrations run first, as the owner (DATABASE_DIRECT_URL), or on the embedded handle.
+ */
+async function openDb(config, { log = console, registry } = {}) {
+ if (!config.db.url) {
+ if (config.isProduction) throw new Error('DATABASE_URL is not set: production serves from PostgreSQL (OpenVibe.Host roles/data add-service.sh host)');
+ log.warn(`[Host] DATABASE_URL unset: embedded PGlite database in ${DEV_PGLITE} (development only, one process)`);
+ fs.mkdirSync(DEV_PGLITE, { recursive: true });
+ const db = createDb({ pglite: DEV_PGLITE, service: 'host', registry, log });
+ await db.migrate({ dir: MIGRATIONS, log });
+ return db;
+ }
+ if (!config.db.directUrl) throw new Error('DATABASE_DIRECT_URL is not set: migrations run with the owner role on a direct connection');
+ const owner = createDb({ url: config.db.directUrl, service: 'host-migrate', max: 1, log });
+ try { await owner.migrate({ dir: MIGRATIONS, log }); } finally { await owner.close(); }
+ return createDb({ url: config.db.url, service: 'host', registry, log });
+}
+
+/**
+ * Every store on a migrated database handle. opts.now — injectable clock (epoch ms), so tests and replays are
+ * deterministic. store.tx(fn) is a transaction; inside it, plain db calls join it (ambient).
+ */
+function createStore(db, { now = () => Date.now() } = {}) {
return {
return {
db,
db,
now,
now,
- tx: (fn) => db.transaction(fn)(),
+
+ tx: async (fn) => await db.tx(() => fn()),
close: () => db.close(),
close: () => db.close(),
};
};
}
}
-module.exports = { openStore, CHARTER_TABLES };
+/** openDb + createStore. */
+async function openStore(config, { now, log } = {}) {
+ return createStore(await openDb(config, { log }), { now });
+}
+
+module.exports = { openDb, openStore, createStore, MIGRATIONS, CHARTER_TABLES };
@@ -47,10 +47,10 @@ function createAccess({ store }) {
@@ -47,10 +47,10 @@ function createAccess({ store }) {
const { db } = store;
const { db } = store;
const memberStmt = db.prepare('SELECT role FROM host_project_members WHERE project_id = ? AND principal = ?');
const memberStmt = db.prepare('SELECT role FROM host_project_members WHERE project_id = ? AND principal = ?');
- function roleOf(project, viewer) {
+ async function roleOf(project, viewer) {
const p = principalOf(viewer);
const p = principalOf(viewer);
if (!p) return null;
if (!p) return null;
- const row = memberStmt.get(project.id, p);
+ const row = await memberStmt.get(project.id, p);
return row ? row.role : null;
return row ? row.role : null;
}
}
@@ -58,7 +58,7 @@ function createAccess({ store }) {
@@ -58,7 +58,7 @@ function createAccess({ store }) {
* Throws unless the viewer may act on the project at `need` (read|deploy|maintain|own|staff).
* Throws unless the viewer may act on the project at `need` (read|deploy|maintain|own|staff).
* `notFound` is the problem thrown when the viewer has no business knowing the resource exists.
* `notFound` is the problem thrown when the viewer has no business knowing the resource exists.
*/
*/
- function authorize(project, viewer, need, notFound = new ApiError(404, 'project.not_found', 'no such project')) {
+ async function authorize(project, viewer, need, notFound = new ApiError(404, 'project.not_found', 'no such project')) {
if (!project || project.status !== 'active') throw notFound;
if (!project || project.status !== 'active') throw notFound;
if (!viewer || viewer.kind === 'anonymous') throw new ApiError(401, 'auth.required', 'sign in with OpenVibe, or present a service token');
if (!viewer || viewer.kind === 'anonymous') throw new ApiError(401, 'auth.required', 'sign in with OpenVibe, or present a service token');
if (viewer.kind === 'service' && viewer.env === 'sandbox' && project.environment !== 'sandbox') {
if (viewer.kind === 'service' && viewer.env === 'sandbox' && project.environment !== 'sandbox') {
@@ -68,7 +68,7 @@ function createAccess({ store }) {
@@ -68,7 +68,7 @@ function createAccess({ store }) {
if (viewer.staff) return 'staff';
if (viewer.staff) return 'staff';
throw new ApiError(403, 'auth.staff_only', 'only OpenVibe staff can change quotas');
throw new ApiError(403, 'auth.staff_only', 'only OpenVibe staff can change quotas');
}
}
- const role = roleOf(project, viewer);
+ const role = await roleOf(project, viewer);
if (!role) {
if (!role) {
if (viewer.staff && need !== 'deploy') return 'staff';
if (viewer.staff && need !== 'deploy') return 'staff';
throw notFound;
throw notFound;
@@ -27,18 +27,18 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
@@ -27,18 +27,18 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
const { db } = store;
const { db } = store;
const q = {
const q = {
byId: db.prepare('SELECT * FROM host_deploys WHERE id = ?'),
byId: db.prepare('SELECT * FROM host_deploys WHERE id = ?'),
- forSite: db.prepare("SELECT * FROM host_deploys WHERE site_id = ? AND state <> 'deleted' ORDER BY created_at DESC, rowid DESC LIMIT ?"),
+ forSite: db.prepare("SELECT * FROM host_deploys WHERE site_id = ? AND state <> 'deleted' ORDER BY created_at DESC, seq DESC LIMIT ?"),
insert: db.prepare(`INSERT INTO host_deploys (id, project_id, site_id, state, source, manifest, manifest_sha256, file_count, total_bytes, new_bytes, failure_code, created_by, created_at)
insert: db.prepare(`INSERT INTO host_deploys (id, project_id, site_id, state, source, manifest, manifest_sha256, file_count, total_bytes, new_bytes, failure_code, created_by, created_at)
VALUES (@id, @project_id, @site_id, @state, @source, @manifest, @manifest_sha256, @file_count, @total_bytes, @new_bytes, @failure_code, @created_by, @created_at)`),
VALUES (@id, @project_id, @site_id, @state, @source, @manifest, @manifest_sha256, @file_count, @total_bytes, @new_bytes, @failure_code, @created_by, @created_at)`),
insertFile: db.prepare('INSERT INTO host_deploy_files (deploy_id, path, sha256, size, content_type) VALUES (?, ?, ?, ?, ?)'),
insertFile: db.prepare('INSERT INTO host_deploy_files (deploy_id, path, sha256, size, content_type) VALUES (?, ?, ?, ?, ?)'),
- insertBlob: db.prepare('INSERT OR IGNORE INTO host_blobs (project_id, sha256, size, created_at) VALUES (?, ?, ?, ?)'),
+ insertBlob: db.prepare('INSERT INTO host_blobs (project_id, sha256, size, created_at) VALUES (?, ?, ?, ?) ON CONFLICT DO NOTHING'),
hasBlob: db.prepare('SELECT 1 FROM host_blobs WHERE project_id = ? AND sha256 = ?'),
hasBlob: db.prepare('SELECT 1 FROM host_blobs WHERE project_id = ? AND sha256 = ?'),
- storageUsed: db.prepare('SELECT COALESCE(SUM(size), 0) AS n FROM host_blobs WHERE project_id = ?'),
+ storageUsed: db.prepare('SELECT COALESCE(SUM(size), 0)::bigint AS n FROM host_blobs WHERE project_id = ?'),
deploysSince: db.prepare('SELECT COUNT(*) AS n FROM host_deploys WHERE project_id = ? AND created_at > ?'),
deploysSince: db.prepare('SELECT COUNT(*) AS n FROM host_deploys WHERE project_id = ? AND created_at > ?'),
logLine: db.prepare('INSERT INTO host_deploy_logs (deploy_id, level, message, created_at) VALUES (?, ?, ?, ?)'),
logLine: db.prepare('INSERT INTO host_deploy_logs (deploy_id, level, message, created_at) VALUES (?, ?, ?, ?)'),
logFor: db.prepare('SELECT level, message, created_at FROM host_deploy_logs WHERE deploy_id = ? ORDER BY id'),
logFor: db.prepare('SELECT level, message, created_at FROM host_deploy_logs WHERE deploy_id = ? ORDER BY id'),
files: db.prepare('SELECT path, sha256, size, content_type FROM host_deploy_files WHERE deploy_id = ? ORDER BY path'),
files: db.prepare('SELECT path, sha256, size, content_type FROM host_deploy_files WHERE deploy_id = ? ORDER BY path'),
- setPointer: db.prepare('UPDATE host_sites SET active_deploy_id = ?, updated_at = ? WHERE id = ? AND status = \'active\' AND active_deploy_id IS ?'),
+ setPointer: db.prepare('UPDATE host_sites SET active_deploy_id = ?, updated_at = ? WHERE id = ? AND status = \'active\' AND active_deploy_id IS NOT DISTINCT FROM ?'),
activation: db.prepare('INSERT INTO host_activations (site_id, deploy_id, previous_deploy_id, kind, actor, created_at) VALUES (?, ?, ?, ?, ?, ?)'),
activation: db.prepare('INSERT INTO host_activations (site_id, deploy_id, previous_deploy_id, kind, actor, created_at) VALUES (?, ?, ?, ?, ?, ?)'),
activations: db.prepare('SELECT deploy_id, previous_deploy_id, kind, actor, created_at FROM host_activations WHERE site_id = ? ORDER BY id DESC LIMIT ?'),
activations: db.prepare('SELECT deploy_id, previous_deploy_id, kind, actor, created_at FROM host_activations WHERE site_id = ? ORDER BY id DESC LIMIT ?'),
markDeleted: db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE id = ?"),
markDeleted: db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE id = ?"),
@@ -49,32 +49,32 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
@@ -49,32 +49,32 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
dropBlob: db.prepare('DELETE FROM host_blobs WHERE project_id = ? AND sha256 = ?'),
dropBlob: db.prepare('DELETE FROM host_blobs WHERE project_id = ? AND sha256 = ?'),
};
};
- function get(id) {
- return isId('deploy', id) ? q.byId.get(id) || null : null;
+ async function get(id) {
+ return isId('deploy', id) ? await q.byId.get(id) || null : null;
}
}
/** The deploy, its site and project, authorized at `need`; 404 for anyone without access. */
/** The deploy, its site and project, authorized at `need`; 404 for anyone without access. */
- function load(viewer, id, need) {
- const deploy = get(id);
+ async function load(viewer, id, need) {
+ const deploy = await get(id);
const notFound = new ApiError(404, 'deploy.not_found', 'no such deploy');
const notFound = new ApiError(404, 'deploy.not_found', 'no such deploy');
if (!deploy || deploy.state === 'deleted') throw notFound;
if (!deploy || deploy.state === 'deleted') throw notFound;
- const site = sites.get(deploy.site_id);
+ const site = await sites.get(deploy.site_id);
if (!site || site.status !== 'active') throw notFound;
if (!site || site.status !== 'active') throw notFound;
- const project = projects.get(deploy.project_id);
- access.authorize(project, viewer, need, notFound);
+ const project = await projects.get(deploy.project_id);
+ await access.authorize(project, viewer, need, notFound);
return { deploy, site, project };
return { deploy, site, project };
}
}
/** Limits for an upload into this project (the archive reader and the validator share them). */
/** Limits for an upload into this project (the archive reader and the validator share them). */
- function limitsFor(project) {
- const quota = projects.quotaOf(project);
+ async function limitsFor(project) {
+ const quota = await projects.quotaOf(project);
return { maxFiles: quota.maxFiles, maxFileBytes: quota.maxFileBytes, maxTotalBytes: quota.storageBytes, quota };
return { maxFiles: quota.maxFiles, maxFileBytes: quota.maxFileBytes, maxTotalBytes: quota.storageBytes, quota };
}
}
/** Before any byte is read: may this caller deploy to this site right now? */
/** Before any byte is read: may this caller deploy to this site right now? */
- function precheck(viewer, siteId) {
- const { site, project } = sites.load(viewer, siteId, 'deploy');
- takedowns.assertOpen(site);
+ async function precheck(viewer, siteId) {
+ const { site, project } = await sites.load(viewer, siteId, 'deploy');
+ await takedowns.assertOpen(site);
const minFree = config ? config.uploads.minFreeBytes : 0;
const minFree = config ? config.uploads.minFreeBytes : 0;
if (minFree > 0) {
if (minFree > 0) {
let free = Infinity;
let free = Infinity;
@@ -84,20 +84,20 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
@@ -84,20 +84,20 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
throw new ApiError(507, 'storage.host_full', 'Host is short of disk space and is not accepting deploys right now; try again later');
throw new ApiError(507, 'storage.host_full', 'Host is short of disk space and is not accepting deploys right now; try again later');
}
}
}
}
- const limits = limitsFor(project);
- const recent = q.deploysSince.get(project.id, store.now() - DAY).n;
+ const limits = await limitsFor(project);
+ const recent = (await q.deploysSince.get(project.id, store.now() - DAY)).n;
if (recent >= limits.quota.deploysPerDay) {
if (recent >= limits.quota.deploysPerDay) {
throw new ApiError(429, 'quota.deploys_per_day', `this project has made ${recent} deploys in the last 24 hours; the limit is ${limits.quota.deploysPerDay}`);
throw new ApiError(429, 'quota.deploys_per_day', `this project has made ${recent} deploys in the last 24 hours; the limit is ${limits.quota.deploysPerDay}`);
}
}
return { site, project, limits };
return { site, project, limits };
}
}
- function writeLog(deployId, lines, now) {
- for (const l of lines) q.logLine.run(deployId, l.level, String(l.message).slice(0, 2000), now);
+ async function writeLog(deployId, lines, now) {
+ for (const l of lines) await q.logLine.run(deployId, l.level, String(l.message).slice(0, 2000), now);
}
}
/** A refused upload becomes a failed deploy (with its log) and host.deploy.failed. */
/** A refused upload becomes a failed deploy (with its log) and host.deploy.failed. */
- function recordFailure(viewer, { site, project }, { source, code, problems = [], notes = [], traceparent }) {
+ async function recordFailure(viewer, { site, project }, { source, code, problems = [], notes = [], traceparent }) {
const id = newId('deploy', store.now());
const id = newId('deploy', store.now());
const now = store.now();
const now = store.now();
const lines = [
const lines = [
@@ -105,40 +105,40 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
@@ -105,40 +105,40 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
...problems.map((p) => ({ level: 'error', message: `${p.code}: ${p.message}` })),
...problems.map((p) => ({ level: 'error', message: `${p.code}: ${p.message}` })),
{ level: 'error', message: `deploy refused (${code}); nothing was stored and the active deploy is unchanged` },
{ level: 'error', message: `deploy refused (${code}); nothing was stored and the active deploy is unchanged` },
];
];
- store.tx(() => {
- q.insert.run({ id, project_id: project.id, site_id: site.id, state: 'failed', source, manifest: null, manifest_sha256: null, file_count: 0, total_bytes: 0, new_bytes: 0, failure_code: code, created_by: principalOf(viewer), created_at: now });
- writeLog(id, lines, now);
- outbox.emit({
+ await store.tx(async () => {
+ await q.insert.run({ id, project_id: project.id, site_id: site.id, state: 'failed', source, manifest: null, manifest_sha256: null, file_count: 0, total_bytes: 0, new_bytes: 0, failure_code: code, created_by: principalOf(viewer), created_at: now });
+ await writeLog(id, lines, now);
+ await outbox.emit({
event_type: 'host.deploy.failed', actor: actorRef(viewer), visibility: 'internal', priority: 'low',
event_type: 'host.deploy.failed', actor: actorRef(viewer), visibility: 'internal', priority: 'low',
subject: { type: 'deploy', id },
subject: { type: 'deploy', id },
payload: { project_id: project.id, site_id: site.id, site: site.name, code, problems: problems.slice(0, 10).map((p) => p.code) },
payload: { project_id: project.id, site_id: site.id, site: site.name, code, problems: problems.slice(0, 10).map((p) => p.code) },
}, { traceparent });
}, { traceparent });
});
});
outbox.kick();
outbox.kick();
- return { deploy: get(id), log: q.logFor.all(id) };
+ return { deploy: await get(id), log: await q.logFor.all(id) };
}
}
/**
/**
* entries: [{ path, data }] already normalised by the reader. -> { deploy, log, activated }
* entries: [{ path, data }] already normalised by the reader. -> { deploy, log, activated }
* Throws ApiError (with extra.deploy_id and extra.log) when the upload is refused.
* Throws ApiError (with extra.deploy_id and extra.log) when the upload is refused.
*/
*/
- function create(viewer, ctx, entries, { source, activate = false, notes = [], traceparent } = {}) {
+ async function create(viewer, ctx, entries, { source, activate = false, notes = [], traceparent } = {}) {
const { site, project, limits } = ctx;
const { site, project, limits } = ctx;
- const fail = (status, code, problems) => {
- const r = recordFailure(viewer, ctx, { source, code, problems, notes, traceparent });
+ const fail = async (status, code, problems) => {
+ const r = await recordFailure(viewer, ctx, { source, code, problems, notes, traceparent });
throw new ApiError(status, code, problems[0] ? problems[0].message : code, { deploy_id: r.deploy.id, log: r.log.map((l) => `${l.level}: ${l.message}`) });
throw new ApiError(status, code, problems[0] ? problems[0].message : code, { deploy_id: r.deploy.id, log: r.log.map((l) => `${l.level}: ${l.message}`) });
};
};
const built = buildManifest(entries, limits);
const built = buildManifest(entries, limits);
- if (!built.ok) fail(/^quota\.|deploy\.too_large/.test(built.code) ? 413 : 422, built.code, built.problems);
+ if (!built.ok) await fail(/^quota\.|deploy\.too_large/.test(built.code) ? 413 : 422, built.code, built.problems);
// Storage quota: only bytes this project does not store yet count.
// Storage quota: only bytes this project does not store yet count.
const fresh = new Map();
const fresh = new Map();
- for (const f of built.files) if (!q.hasBlob.get(project.id, f.sha256) && !fresh.has(f.sha256)) fresh.set(f.sha256, f);
+ for (const f of built.files) if (!await q.hasBlob.get(project.id, f.sha256) && !fresh.has(f.sha256)) fresh.set(f.sha256, f);
const newBytes = [...fresh.values()].reduce((n, f) => n + f.size, 0);
const newBytes = [...fresh.values()].reduce((n, f) => n + f.size, 0);
- const used = q.storageUsed.get(project.id).n;
+ const used = (await q.storageUsed.get(project.id)).n;
if (used + newBytes > limits.quota.storageBytes) {
if (used + newBytes > limits.quota.storageBytes) {
- fail(413, 'quota.storage', [{ code: 'quota.storage', message: `this deploy adds ${fmtBytes(newBytes)}; the project stores ${fmtBytes(used)} of ${fmtBytes(limits.quota.storageBytes)}` }]);
+ await fail(413, 'quota.storage', [{ code: 'quota.storage', message: `this deploy adds ${fmtBytes(newBytes)}; the project stores ${fmtBytes(used)} of ${fmtBytes(limits.quota.storageBytes)}` }]);
}
}
// Objects first (idempotent, content-addressed); rows only after they are durable.
// Objects first (idempotent, content-addressed); rows only after they are durable.
@@ -156,30 +156,30 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
@@ -156,30 +156,30 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
{ level: 'info', message: 'no build step: Stage B serves the uploaded files as they are; nothing was executed' },
{ level: 'info', message: 'no build step: Stage B serves the uploaded files as they are; nothing was executed' },
];
];
let activated = null;
let activated = null;
- store.tx(() => {
- q.insert.run({ id, project_id: project.id, site_id: site.id, state: 'ready', source, manifest: built.manifestJson, manifest_sha256: built.manifestSha256, file_count: built.manifest.file_count, total_bytes: built.manifest.total_bytes, new_bytes: newBytes, failure_code: null, created_by: who, created_at: now });
- for (const f of built.files) q.insertFile.run(id, f.path, f.sha256, f.size, f.content_type);
- for (const f of fresh.values()) q.insertBlob.run(project.id, f.sha256, f.size, now);
- outbox.emit({
+ await store.tx(async () => {
+ await q.insert.run({ id, project_id: project.id, site_id: site.id, state: 'ready', source, manifest: built.manifestJson, manifest_sha256: built.manifestSha256, file_count: built.manifest.file_count, total_bytes: built.manifest.total_bytes, new_bytes: newBytes, failure_code: null, created_by: who, created_at: now });
+ for (const f of built.files) await q.insertFile.run(id, f.path, f.sha256, f.size, f.content_type);
+ for (const f of fresh.values()) await q.insertBlob.run(project.id, f.sha256, f.size, now);
+ await outbox.emit({
event_type: 'host.deploy.created', actor: actorRef(viewer), visibility: 'internal', priority: 'low',
event_type: 'host.deploy.created', actor: actorRef(viewer), visibility: 'internal', priority: 'low',
subject: { type: 'deploy', id },
subject: { type: 'deploy', id },
payload: { project_id: project.id, site_id: site.id, site: site.name, source, file_count: built.manifest.file_count, total_bytes: built.manifest.total_bytes, manifest_sha256: built.manifestSha256 },
payload: { project_id: project.id, site_id: site.id, site: site.name, source, file_count: built.manifest.file_count, total_bytes: built.manifest.total_bytes, manifest_sha256: built.manifestSha256 },
}, { traceparent });
}, { traceparent });
if (activate) {
if (activate) {
- activated = switchPointer(viewer, site, get(id), 'activate', { traceparent });
+ activated = await switchPointer(viewer, site, await get(id), 'activate', { traceparent });
lines.push({ level: 'info', message: 'activated: the site now serves this deploy' });
lines.push({ level: 'info', message: 'activated: the site now serves this deploy' });
} else {
} else {
lines.push({ level: 'info', message: 'ready; not active until you activate it' });
lines.push({ level: 'info', message: 'ready; not active until you activate it' });
}
}
- writeLog(id, lines, now);
+ await writeLog(id, lines, now);
});
});
outbox.kick();
outbox.kick();
- return { deploy: get(id), log: q.logFor.all(id), activated };
+ return { deploy: await get(id), log: await q.logFor.all(id), activated };
}
}
/** Inside a transaction. Compare-and-set on the pointer read in the same transaction. */
/** Inside a transaction. Compare-and-set on the pointer read in the same transaction. */
- function switchPointer(viewer, site, deploy, kind, { expectedActive, traceparent } = {}) {
- const current = db.prepare('SELECT active_deploy_id, status FROM host_sites WHERE id = ?').get(site.id);
+ async function switchPointer(viewer, site, deploy, kind, { expectedActive, traceparent } = {}) {
+ const current = await db.prepare('SELECT active_deploy_id, status FROM host_sites WHERE id = ?').get(site.id);
if (!current || current.status !== 'active') throw new ApiError(404, 'site.not_found', 'no such site');
if (!current || current.status !== 'active') throw new ApiError(404, 'site.not_found', 'no such site');
if (deploy.state !== 'ready' || deploy.site_id !== site.id) throw new ApiError(409, 'deploy.not_ready', 'only a ready deploy of this site can be activated');
if (deploy.state !== 'ready' || deploy.site_id !== site.id) throw new ApiError(409, 'deploy.not_ready', 'only a ready deploy of this site can be activated');
if (expectedActive !== undefined && (expectedActive || null) !== current.active_deploy_id) {
if (expectedActive !== undefined && (expectedActive || null) !== current.active_deploy_id) {
@@ -187,10 +187,10 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
@@ -187,10 +187,10 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
}
}
if (current.active_deploy_id === deploy.id) return { changed: false, deploy_id: deploy.id, previous_deploy_id: deploy.id };
if (current.active_deploy_id === deploy.id) return { changed: false, deploy_id: deploy.id, previous_deploy_id: deploy.id };
const now = store.now();
const now = store.now();
- const r = q.setPointer.run(deploy.id, now, site.id, current.active_deploy_id);
+ const r = await q.setPointer.run(deploy.id, now, site.id, current.active_deploy_id);
if (r.changes !== 1) throw new ApiError(409, 'site.active_changed', 'the active deploy changed during the switch');
if (r.changes !== 1) throw new ApiError(409, 'site.active_changed', 'the active deploy changed during the switch');
- q.activation.run(site.id, deploy.id, current.active_deploy_id, kind, principalOf(viewer) || 'host', now);
- outbox.emit({
+ await q.activation.run(site.id, deploy.id, current.active_deploy_id, kind, principalOf(viewer) || 'host', now);
+ await outbox.emit({
event_type: 'host.deploy.activated', actor: actorRef(viewer), visibility: 'internal', priority: 'low',
event_type: 'host.deploy.activated', actor: actorRef(viewer), visibility: 'internal', priority: 'low',
subject: { type: 'deploy', id: deploy.id },
subject: { type: 'deploy', id: deploy.id },
payload: { project_id: site.project_id, site_id: site.id, site: site.name, deploy_id: deploy.id, previous_deploy_id: current.active_deploy_id, rollback: kind === 'rollback' },
payload: { project_id: site.project_id, site_id: site.id, site: site.name, deploy_id: deploy.id, previous_deploy_id: current.active_deploy_id, rollback: kind === 'rollback' },
@@ -198,45 +198,45 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
@@ -198,45 +198,45 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
return { changed: true, deploy_id: deploy.id, previous_deploy_id: current.active_deploy_id };
return { changed: true, deploy_id: deploy.id, previous_deploy_id: current.active_deploy_id };
}
}
- function activate(viewer, id, { expectedActive, traceparent } = {}) {
- const { deploy, site } = load(viewer, id, 'deploy');
- takedowns.assertOpen(site);
- const out = store.tx(() => switchPointer(viewer, site, deploy, 'activate', { expectedActive, traceparent }));
+ async function activate(viewer, id, { expectedActive, traceparent } = {}) {
+ const { deploy, site } = await load(viewer, id, 'deploy');
+ await takedowns.assertOpen(site);
+ const out = await store.tx(async () => await switchPointer(viewer, site, deploy, 'activate', { expectedActive, traceparent }));
outbox.kick();
outbox.kick();
return out;
return out;
}
}
/** To `deployId`, or else to the most recent previously active deploy that still exists. */
/** To `deployId`, or else to the most recent previously active deploy that still exists. */
- function rollback(viewer, siteId, { deployId, expectedActive, traceparent } = {}) {
- const { site } = sites.load(viewer, siteId, 'deploy');
- takedowns.assertOpen(site);
- const out = store.tx(() => {
- const current = db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id).active_deploy_id;
+ async function rollback(viewer, siteId, { deployId, expectedActive, traceparent } = {}) {
+ const { site } = await sites.load(viewer, siteId, 'deploy');
+ await takedowns.assertOpen(site);
+ const out = await store.tx(async () => {
+ const current = (await db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id)).active_deploy_id;
let target = null;
let target = null;
if (deployId) {
if (deployId) {
- target = get(deployId);
+ target = await get(deployId);
if (!target || target.site_id !== site.id || target.state === 'deleted') throw new ApiError(404, 'deploy.not_found', 'no such deploy on this site');
if (!target || target.site_id !== site.id || target.state === 'deleted') throw new ApiError(404, 'deploy.not_found', 'no such deploy on this site');
} else {
} else {
- for (const a of q.activations.all(site.id, 200)) {
+ for (const a of await q.activations.all(site.id, 200)) {
for (const cand of [a.previous_deploy_id, a.deploy_id]) {
for (const cand of [a.previous_deploy_id, a.deploy_id]) {
if (!cand || cand === current) continue;
if (!cand || cand === current) continue;
- const d = get(cand);
+ const d = await get(cand);
if (d && d.state === 'ready' && d.site_id === site.id) { target = d; break; }
if (d && d.state === 'ready' && d.site_id === site.id) { target = d; break; }
}
}
if (target) break;
if (target) break;
}
}
if (!target) throw new ApiError(409, 'deploy.no_previous', 'there is no previous deploy to roll back to');
if (!target) throw new ApiError(409, 'deploy.no_previous', 'there is no previous deploy to roll back to');
}
}
- return switchPointer(viewer, site, target, 'rollback', { expectedActive, traceparent });
+ return await switchPointer(viewer, site, target, 'rollback', { expectedActive, traceparent });
});
});
outbox.kick();
outbox.kick();
return out;
return out;
}
}
/** Inside a transaction: drop blob rows no deploy of the project references. -> freed shas */
/** Inside a transaction: drop blob rows no deploy of the project references. -> freed shas */
- function collectGarbage(projectId) {
- const freed = q.orphans.all(projectId).map((r) => r.sha256);
- for (const sha of freed) q.dropBlob.run(projectId, sha);
+ async function collectGarbage(projectId) {
+ const freed = (await q.orphans.all(projectId)).map((r) => r.sha256);
+ for (const sha of freed) await q.dropBlob.run(projectId, sha);
return freed;
return freed;
}
}
@@ -246,16 +246,16 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
@@ -246,16 +246,16 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
}
}
}
}
- function remove(viewer, id) {
- const { deploy, site, project } = load(viewer, id, 'maintain');
- takedowns.assertDeletable(viewer, { site });
+ async function remove(viewer, id) {
+ const { deploy, site, project } = await load(viewer, id, 'maintain');
+ await takedowns.assertDeletable(viewer, { site });
let freed = [];
let freed = [];
- store.tx(() => {
- const active = db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id).active_deploy_id;
+ await store.tx(async () => {
+ const active = (await db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id)).active_deploy_id;
if (active === deploy.id) throw new ApiError(409, 'deploy.active', 'this deploy is being served: activate another one first');
if (active === deploy.id) throw new ApiError(409, 'deploy.active', 'this deploy is being served: activate another one first');
- q.markDeleted.run(store.now(), deploy.id);
- q.dropFiles.run(deploy.id);
- freed = collectGarbage(project.id);
+ await q.markDeleted.run(store.now(), deploy.id);
+ await q.dropFiles.run(deploy.id);
+ freed = await collectGarbage(project.id);
});
});
unlinkBlobs(project.id, freed);
unlinkBlobs(project.id, freed);
return { deleted: true, objects_removed: freed.length };
return { deleted: true, objects_removed: freed.length };
@@ -263,10 +263,10 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
@@ -263,10 +263,10 @@ function createDeploys({ store, config = null, access, projects, sites, blobs, o
return {
return {
get, load, precheck, create, recordFailure, activate, rollback, remove, collectGarbage, unlinkBlobs, limitsFor,
get, load, precheck, create, recordFailure, activate, rollback, remove, collectGarbage, unlinkBlobs, limitsFor,
- list: (siteId, limit = 100) => q.forSite.all(siteId, Math.min(Math.max(limit, 1), 200)),
- logOf: (id) => q.logFor.all(id),
- filesOf: (id) => q.files.all(id),
- activationsOf: (siteId, limit = 50) => q.activations.all(siteId, limit),
+ list: async (siteId, limit = 100) => await q.forSite.all(siteId, Math.min(Math.max(limit, 1), 200)),
+ logOf: async (id) => await q.logFor.all(id),
+ filesOf: async (id) => await q.files.all(id),
+ activationsOf: async (siteId, limit = 50) => await q.activations.all(siteId, limit),
};
};
}
}
@@ -57,7 +57,7 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
@@ -57,7 +57,7 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
const reserved = firstPartyDomains(config);
const reserved = firstPartyDomains(config);
const q = {
const q = {
byId: db.prepare('SELECT * FROM host_domains WHERE id = ?'),
byId: db.prepare('SELECT * FROM host_domains WHERE id = ?'),
- forSite: db.prepare("SELECT * FROM host_domains WHERE site_id = ? ORDER BY kind DESC, created_at, rowid"),
+ forSite: db.prepare("SELECT * FROM host_domains WHERE site_id = ? ORDER BY kind DESC, created_at, seq"),
verifiedByHost: db.prepare("SELECT * FROM host_domains WHERE hostname = ? AND status = 'verified'"),
verifiedByHost: db.prepare("SELECT * FROM host_domains WHERE hostname = ? AND status = 'verified'"),
onSite: db.prepare('SELECT * FROM host_domains WHERE site_id = ? AND hostname = ?'),
onSite: db.prepare('SELECT * FROM host_domains WHERE site_id = ? AND hostname = ?'),
insert: db.prepare(`INSERT INTO host_domains (id, project_id, site_id, hostname, kind, status, token, created_by, created_at)
insert: db.prepare(`INSERT INTO host_domains (id, project_id, site_id, hostname, kind, status, token, created_by, created_at)
@@ -74,13 +74,13 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
@@ -74,13 +74,13 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
const challengeName = (hostname) => `${CHALLENGE_PREFIX}.${hostname}`;
const challengeName = (hostname) => `${CHALLENGE_PREFIX}.${hostname}`;
- function load(viewer, id, need) {
- const domain = isId('domain', id) ? q.byId.get(id) : null;
+ async function load(viewer, id, need) {
+ const domain = isId('domain', id) ? await q.byId.get(id) : null;
const notFound = new ApiError(404, 'domain.not_found', 'no such domain');
const notFound = new ApiError(404, 'domain.not_found', 'no such domain');
if (!domain) throw notFound;
if (!domain) throw notFound;
- const site = sites.get(domain.site_id);
+ const site = await sites.get(domain.site_id);
if (!site || site.status !== 'active') throw notFound;
if (!site || site.status !== 'active') throw notFound;
- access.authorize(projects.get(domain.project_id), viewer, need, notFound);
+ await access.authorize(await projects.get(domain.project_id), viewer, need, notFound);
return { domain, site };
return { domain, site };
}
}
@@ -109,18 +109,18 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
@@ -109,18 +109,18 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
};
};
}
}
- function add(viewer, siteId, input = {}) {
- const { site, project } = sites.load(viewer, siteId, 'maintain');
+ async function add(viewer, siteId, input = {}) {
+ const { site, project } = await sites.load(viewer, siteId, 'maintain');
const hostname = checkHostname(input.hostname);
const hostname = checkHostname(input.hostname);
if (project.environment === 'sandbox') throw new ApiError(403, 'domain.sandbox', 'sandbox projects are served on their default domain only');
if (project.environment === 'sandbox') throw new ApiError(403, 'domain.sandbox', 'sandbox projects are served on their default domain only');
- const quota = projects.quotaOf(project);
- if (projects.usageOf(project).customDomains >= quota.customDomains) throw new ApiError(429, 'quota.custom_domains', `this project may have ${quota.customDomains} custom domains`);
- if (q.onSite.get(site.id, hostname)) throw new ApiError(409, 'domain.exists', `${hostname} is already on this site`);
- if (q.verifiedByHost.get(hostname)) throw new ApiError(409, 'domain.taken', `${hostname} is already verified for another site`);
+ const quota = await projects.quotaOf(project);
+ if ((await projects.usageOf(project)).customDomains >= quota.customDomains) throw new ApiError(429, 'quota.custom_domains', `this project may have ${quota.customDomains} custom domains`);
+ if (await q.onSite.get(site.id, hostname)) throw new ApiError(409, 'domain.exists', `${hostname} is already on this site`);
+ if (await q.verifiedByHost.get(hostname)) throw new ApiError(409, 'domain.taken', `${hostname} is already verified for another site`);
const id = newId('domain', store.now());
const id = newId('domain', store.now());
const token = crypto.randomBytes(20).toString('hex');
const token = crypto.randomBytes(20).toString('hex');
- q.insert.run(id, project.id, site.id, hostname, token, principalOf(viewer), store.now());
- const domain = q.byId.get(id);
+ await q.insert.run(id, project.id, site.id, hostname, token, principalOf(viewer), store.now());
+ const domain = await q.byId.get(id);
return { domain, site };
return { domain, site };
}
}
@@ -143,40 +143,40 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
@@ -143,40 +143,40 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
if (domain.last_checked_at && now - domain.last_checked_at < MIN_RECHECK_MS) return domain;
if (domain.last_checked_at && now - domain.last_checked_at < MIN_RECHECK_MS) return domain;
const r = await lookup(domain.hostname);
const r = await lookup(domain.hostname);
const expected = `${TXT_PREFIX}${domain.token}`;
const expected = `${TXT_PREFIX}${domain.token}`;
- if (!r.ok) { q.checked.run(store.now(), r.error, domain.id); return q.byId.get(domain.id); }
+ if (!r.ok) { await q.checked.run(store.now(), r.error, domain.id); return await q.byId.get(domain.id); }
if (!r.values.includes(expected)) {
if (!r.values.includes(expected)) {
- q.checked.run(store.now(), r.values.length ? `${challengeName(domain.hostname)} has TXT records, but not the expected value` : `no TXT record at ${challengeName(domain.hostname)} yet`, domain.id);
- return q.byId.get(domain.id);
+ await q.checked.run(store.now(), r.values.length ? `${challengeName(domain.hostname)} has TXT records, but not the expected value` : `no TXT record at ${challengeName(domain.hostname)} yet`, domain.id);
+ return await q.byId.get(domain.id);
}
}
try {
try {
- store.tx(() => {
- if (q.verifiedByHost.get(domain.hostname)) throw new ApiError(409, 'domain.taken', `${domain.hostname} was verified for another site first`);
- const site = sites.get(domain.site_id);
+ await store.tx(async () => {
+ if (await q.verifiedByHost.get(domain.hostname)) throw new ApiError(409, 'domain.taken', `${domain.hostname} was verified for another site first`);
+ const site = await sites.get(domain.site_id);
if (!site || site.status !== 'active') throw new ApiError(404, 'domain.not_found', 'no such domain');
if (!site || site.status !== 'active') throw new ApiError(404, 'domain.not_found', 'no such domain');
- if (q.verified.run(store.now(), store.now(), domain.id).changes !== 1) return;
- outbox.emit({
+ if ((await q.verified.run(store.now(), store.now(), domain.id)).changes !== 1) return;
+ await outbox.emit({
event_type: 'host.domain.verified', actor: viewer ? actorRef(viewer) : { type: 'service', id: 'host' }, visibility: 'internal', priority: 'low',
event_type: 'host.domain.verified', actor: viewer ? actorRef(viewer) : { type: 'service', id: 'host' }, visibility: 'internal', priority: 'low',
subject: { type: 'domain', id: domain.id },
subject: { type: 'domain', id: domain.id },
payload: { project_id: domain.project_id, site_id: domain.site_id, site: site.name, hostname: domain.hostname },
payload: { project_id: domain.project_id, site_id: domain.site_id, site: site.name, hostname: domain.hostname },
}, { traceparent });
}, { traceparent });
});
});
} catch (err) {
} catch (err) {
- if (err instanceof ApiError && err.code === 'domain.taken') { q.setStatus.run('failed', err.message, domain.id); return q.byId.get(domain.id); }
+ if (err instanceof ApiError && err.code === 'domain.taken') { await q.setStatus.run('failed', err.message, domain.id); return await q.byId.get(domain.id); }
throw err;
throw err;
}
}
outbox.kick();
outbox.kick();
- return q.byId.get(domain.id);
+ return await q.byId.get(domain.id);
}
}
async function verify(viewer, id, opts = {}) {
async function verify(viewer, id, opts = {}) {
- const { domain } = load(viewer, id, 'maintain');
- return verifyDomain(domain, viewer, opts);
+ const { domain } = await load(viewer, id, 'maintain');
+ return await verifyDomain(domain, viewer, opts);
}
}
- function remove(viewer, id) {
- const { domain } = load(viewer, id, 'maintain');
+ async function remove(viewer, id) {
+ const { domain } = await load(viewer, id, 'maintain');
if (domain.kind === 'default') throw new ApiError(409, 'domain.default', 'the default domain goes away only with the site');
if (domain.kind === 'default') throw new ApiError(409, 'domain.default', 'the default domain goes away only with the site');
- q.remove.run(domain.id);
+ await q.remove.run(domain.id);
return { deleted: true };
return { deleted: true };
}
}
@@ -184,9 +184,9 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
@@ -184,9 +184,9 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
async function recheck({ limit = 50 } = {}) {
async function recheck({ limit = 50 } = {}) {
const now = store.now();
const now = store.now();
const summary = { verified: 0, failed: 0, lapsed: 0, checked: 0 };
const summary = { verified: 0, failed: 0, lapsed: 0, checked: 0 };
- for (const d of q.duePending.all(now - config.domains.recheckIntervalMs + 1000, limit)) {
+ for (const d of await q.duePending.all(now - config.domains.recheckIntervalMs + 1000, limit)) {
if (now - d.created_at > config.domains.pendingDays * DAY) {
if (now - d.created_at > config.domains.pendingDays * DAY) {
- q.setStatus.run('failed', `not verified within ${config.domains.pendingDays} days`, d.id);
+ await q.setStatus.run('failed', `not verified within ${config.domains.pendingDays} days`, d.id);
summary.failed++;
summary.failed++;
continue;
continue;
}
}
@@ -194,15 +194,15 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
@@ -194,15 +194,15 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
summary.checked++;
summary.checked++;
if (after.status === 'verified') summary.verified++;
if (after.status === 'verified') summary.verified++;
}
}
- for (const d of q.dueVerified.all(now - DAY, limit)) {
+ for (const d of await q.dueVerified.all(now - DAY, limit)) {
const r = await lookup(d.hostname);
const r = await lookup(d.hostname);
summary.checked++;
summary.checked++;
- if (!r.ok) { q.checked.run(store.now(), r.error, d.id); continue; }
- if (r.values.includes(`${TXT_PREFIX}${d.token}`)) { q.present.run(store.now(), d.id); continue; }
- q.missing.run(store.now(), store.now(), `the TXT record at ${challengeName(d.hostname)} is gone`, d.id);
- const row = q.byId.get(d.id);
+ if (!r.ok) { await q.checked.run(store.now(), r.error, d.id); continue; }
+ if (r.values.includes(`${TXT_PREFIX}${d.token}`)) { await q.present.run(store.now(), d.id); continue; }
+ await q.missing.run(store.now(), store.now(), `the TXT record at ${challengeName(d.hostname)} is gone`, d.id);
+ const row = await q.byId.get(d.id);
if (row.record_missing_since && store.now() - row.record_missing_since > config.domains.lapseDays * DAY) {
if (row.record_missing_since && store.now() - row.record_missing_since > config.domains.lapseDays * DAY) {
- q.setStatus.run('lapsed', `the TXT record has been gone for more than ${config.domains.lapseDays} days; the domain is no longer served`, d.id);
+ await q.setStatus.run('lapsed', `the TXT record has been gone for more than ${config.domains.lapseDays} days; the domain is no longer served`, d.id);
summary.lapsed++;
summary.lapsed++;
log.warn(`[Host] custom domain ${d.hostname} lapsed (TXT record gone)`);
log.warn(`[Host] custom domain ${d.hostname} lapsed (TXT record gone)`);
}
}
@@ -212,9 +212,9 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
@@ -212,9 +212,9 @@ function createDomains({ store, config, access, projects, sites, outbox, resolve
return {
return {
load, add, verify, verifyDomain, remove, recheck, instructions, checkHostname,
load, add, verify, verifyDomain, remove, recheck, instructions, checkHostname,
- listForSite: (siteId) => q.forSite.all(siteId),
- get: (id) => (isId('domain', id) ? q.byId.get(id) || null : null),
- verifiedCustom: () => db.prepare("SELECT hostname, site_id FROM host_domains WHERE kind = 'custom' AND status = 'verified' ORDER BY hostname").all(),
+ listForSite: async (siteId) => await q.forSite.all(siteId),
+ get: async (id) => (isId('domain', id) ? await q.byId.get(id) || null : null),
+ verifiedCustom: async () => await db.prepare("SELECT hostname, site_id FROM host_domains WHERE kind = 'custom' AND status = 'verified' ORDER BY hostname").all(),
challengeName, TXT_PREFIX,
challengeName, TXT_PREFIX,
};
};
}
}
@@ -23,8 +23,8 @@ function createProjects({ store, config, access, blobs, takedowns, log = console
@@ -23,8 +23,8 @@ function createProjects({ store, config, access, blobs, takedowns, log = console
const q = {
const q = {
byId: db.prepare('SELECT * FROM host_projects WHERE id = ?'),
byId: db.prepare('SELECT * FROM host_projects WHERE id = ?'),
forPrincipal: db.prepare(`SELECT p.*, m.role FROM host_projects p JOIN host_project_members m ON m.project_id = p.id
forPrincipal: db.prepare(`SELECT p.*, m.role FROM host_projects p JOIN host_project_members m ON m.project_id = p.id
- WHERE m.principal = ? AND p.status = 'active' ORDER BY p.created_at DESC, p.rowid DESC LIMIT 200`),
- all: db.prepare("SELECT p.*, NULL AS role FROM host_projects p WHERE p.status = 'active' ORDER BY p.created_at DESC, p.rowid DESC LIMIT 200"),
+ WHERE m.principal = ? AND p.status = 'active' ORDER BY p.created_at DESC, p.seq DESC LIMIT 200`),
+ all: db.prepare("SELECT p.*, NULL AS role FROM host_projects p WHERE p.status = 'active' ORDER BY p.created_at DESC, p.seq DESC LIMIT 200"),
ownedCount: db.prepare("SELECT COUNT(*) AS n FROM host_projects WHERE owner_subject = ? AND status = 'active'"),
ownedCount: db.prepare("SELECT COUNT(*) AS n FROM host_projects WHERE owner_subject = ? AND status = 'active'"),
insert: db.prepare(`INSERT INTO host_projects (id, network_project_id, owner_subject, name, environment, status, created_at, updated_at)
insert: db.prepare(`INSERT INTO host_projects (id, network_project_id, owner_subject, name, environment, status, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, 'active', ?, ?)`),
VALUES (?, ?, ?, ?, ?, 'active', ?, ?)`),
@@ -33,38 +33,38 @@ function createProjects({ store, config, access, blobs, takedowns, log = console
@@ -33,38 +33,38 @@ function createProjects({ store, config, access, blobs, takedowns, log = console
removeMember: db.prepare('DELETE FROM host_project_members WHERE project_id = ? AND principal = ?'),
removeMember: db.prepare('DELETE FROM host_project_members WHERE project_id = ? AND principal = ?'),
members: db.prepare('SELECT principal, role, added_by, created_at FROM host_project_members WHERE project_id = ? ORDER BY created_at'),
members: db.prepare('SELECT principal, role, added_by, created_at FROM host_project_members WHERE project_id = ? ORDER BY created_at'),
quotaRow: db.prepare('SELECT * FROM host_quotas WHERE project_id = ?'),
quotaRow: db.prepare('SELECT * FROM host_quotas WHERE project_id = ?'),
- storageUsed: db.prepare('SELECT COALESCE(SUM(size), 0) AS n, COUNT(*) AS c FROM host_blobs WHERE project_id = ?'),
+ storageUsed: db.prepare('SELECT COALESCE(SUM(size), 0)::bigint AS n, COUNT(*) AS c FROM host_blobs WHERE project_id = ?'),
deploysSince: db.prepare('SELECT COUNT(*) AS n FROM host_deploys WHERE project_id = ? AND created_at > ?'),
deploysSince: db.prepare('SELECT COUNT(*) AS n FROM host_deploys WHERE project_id = ? AND created_at > ?'),
siteCount: db.prepare("SELECT COUNT(*) AS n FROM host_sites WHERE project_id = ? AND status = 'active'"),
siteCount: db.prepare("SELECT COUNT(*) AS n FROM host_sites WHERE project_id = ? AND status = 'active'"),
customDomainCount: db.prepare("SELECT COUNT(*) AS n FROM host_domains WHERE project_id = ? AND kind = 'custom' AND status IN ('pending','verified')"),
customDomainCount: db.prepare("SELECT COUNT(*) AS n FROM host_domains WHERE project_id = ? AND kind = 'custom' AND status IN ('pending','verified')"),
markDeleted: db.prepare("UPDATE host_projects SET status = 'deleted', deleted_at = ?, updated_at = ? WHERE id = ?"),
markDeleted: db.prepare("UPDATE host_projects SET status = 'deleted', deleted_at = ?, updated_at = ? WHERE id = ?"),
};
};
- function get(id) {
- return isId('project', id) ? q.byId.get(id) || null : null;
+ async function get(id) {
+ return isId('project', id) ? await q.byId.get(id) || null : null;
}
}
/** The effective quota: the project's override where set, else the environment's default. */
/** The effective quota: the project's override where set, else the environment's default. */
- function quotaOf(project) {
+ async function quotaOf(project) {
const defaults = config.quotas[project.environment] || config.quotas.production;
const defaults = config.quotas[project.environment] || config.quotas.production;
- const row = q.quotaRow.get(project.id) || {};
+ const row = await q.quotaRow.get(project.id) || {};
const out = {};
const out = {};
for (const [col, key] of Object.entries(QUOTA_FIELDS)) out[key] = row[col] != null ? row[col] : defaults[key];
for (const [col, key] of Object.entries(QUOTA_FIELDS)) out[key] = row[col] != null ? row[col] : defaults[key];
return out;
return out;
}
}
- function usageOf(project) {
- const s = q.storageUsed.get(project.id);
+ async function usageOf(project) {
+ const s = await q.storageUsed.get(project.id);
return {
return {
storageBytes: s.n,
storageBytes: s.n,
objects: s.c,
objects: s.c,
- deploysLast24h: q.deploysSince.get(project.id, store.now() - DAY).n,
- sites: q.siteCount.get(project.id).n,
- customDomains: q.customDomainCount.get(project.id).n,
+ deploysLast24h: (await q.deploysSince.get(project.id, store.now() - DAY)).n,
+ sites: (await q.siteCount.get(project.id)).n,
+ customDomains: (await q.customDomainCount.get(project.id)).n,
};
};
}
}
- function create(viewer, input = {}) {
+ async function create(viewer, input = {}) {
const owner = principalOf(viewer);
const owner = principalOf(viewer);
if (!owner || !ids.isSubjectId('user', owner)) {
if (!owner || !ids.isSubjectId('user', owner)) {
throw new ApiError(403, 'project.owner_must_be_user', 'a project is owned by a person (usr_…): an app or service must name the owner with X-OV-Subject');
throw new ApiError(403, 'project.owner_must_be_user', 'a project is owned by a person (usr_…): an app or service must name the owner with X-OV-Subject');
@@ -82,51 +82,51 @@ function createProjects({ store, config, access, blobs, takedowns, log = console
@@ -82,51 +82,51 @@ function createProjects({ store, config, access, blobs, takedowns, log = console
if (!NETWORK_PROJECT_RE.test(networkProjectId)) throw new ApiError(422, 'project.invalid_network_project_id', 'network_project_id is not a project id');
if (!NETWORK_PROJECT_RE.test(networkProjectId)) throw new ApiError(422, 'project.invalid_network_project_id', 'network_project_id is not a project id');
}
}
const maxOwned = (config.projects && config.projects.maxPerOwner) || 10;
const maxOwned = (config.projects && config.projects.maxPerOwner) || 10;
- if (q.ownedCount.get(owner).n >= maxOwned) throw new ApiError(429, 'quota.projects', `you already own ${maxOwned} projects`);
+ if ((await q.ownedCount.get(owner)).n >= maxOwned) throw new ApiError(429, 'quota.projects', `you already own ${maxOwned} projects`);
const id = newId('project', store.now());
const id = newId('project', store.now());
const now = store.now();
const now = store.now();
try {
try {
- store.tx(() => {
- q.insert.run(id, networkProjectId, owner, name, environment, now, now);
- q.addMember.run(id, owner, 'owner', owner, now);
+ await store.tx(async () => {
+ await q.insert.run(id, networkProjectId, owner, name, environment, now, now);
+ await q.addMember.run(id, owner, 'owner', owner, now);
});
});
} catch (err) {
} catch (err) {
- if (/UNIQUE/.test(err.message)) throw new ApiError(409, 'project.network_project_taken', 'that Network project already has a Host project');
+ if (err.code === '23505') throw new ApiError(409, 'project.network_project_taken', 'that Network project already has a Host project');
throw err;
throw err;
}
}
- return get(id);
+ return await get(id);
}
}
- function listFor(viewer) {
- if (viewer.kind === 'user' && viewer.staff && viewer.allProjects) return q.all.all();
+ async function listFor(viewer) {
+ if (viewer.kind === 'user' && viewer.staff && viewer.allProjects) return await q.all.all();
const p = principalOf(viewer);
const p = principalOf(viewer);
- return p ? q.forPrincipal.all(p) : [];
+ return p ? await q.forPrincipal.all(p) : [];
}
}
- function members(project) {
- return q.members.all(project.id);
+ async function members(project) {
+ return await q.members.all(project.id);
}
}
- function setMember(viewer, project, principal, role) {
- access.authorize(project, viewer, 'own');
+ async function setMember(viewer, project, principal, role) {
+ await access.authorize(project, viewer, 'own');
if (!PRINCIPAL_RE.test(String(principal))) throw new ApiError(422, 'member.invalid_principal', 'a member is a usr_ subject, an app:app_ principal or a svc: principal');
if (!PRINCIPAL_RE.test(String(principal))) throw new ApiError(422, 'member.invalid_principal', 'a member is a usr_ subject, an app:app_ principal or a svc: principal');
if (!['maintainer', 'deployer', 'owner'].includes(role)) throw new ApiError(422, 'member.invalid_role', 'role is owner, maintainer or deployer');
if (!['maintainer', 'deployer', 'owner'].includes(role)) throw new ApiError(422, 'member.invalid_role', 'role is owner, maintainer or deployer');
if (role === 'owner' && !principal.startsWith('usr_')) throw new ApiError(422, 'member.invalid_role', 'only a person can be an owner');
if (role === 'owner' && !principal.startsWith('usr_')) throw new ApiError(422, 'member.invalid_role', 'only a person can be an owner');
if (principal === project.owner_subject && role !== 'owner') throw new ApiError(409, 'member.owner_fixed', 'the project owner stays owner');
if (principal === project.owner_subject && role !== 'owner') throw new ApiError(409, 'member.owner_fixed', 'the project owner stays owner');
- q.addMember.run(project.id, principal, role, principalOf(viewer), store.now());
- return members(project);
+ await q.addMember.run(project.id, principal, role, principalOf(viewer), store.now());
+ return await members(project);
}
}
- function removeMember(viewer, project, principal) {
- access.authorize(project, viewer, 'own');
+ async function removeMember(viewer, project, principal) {
+ await access.authorize(project, viewer, 'own');
if (principal === project.owner_subject) throw new ApiError(409, 'member.owner_fixed', 'the project owner cannot be removed');
if (principal === project.owner_subject) throw new ApiError(409, 'member.owner_fixed', 'the project owner cannot be removed');
- q.removeMember.run(project.id, principal);
- return members(project);
+ await q.removeMember.run(project.id, principal);
+ return await members(project);
}
}
- function setQuota(viewer, project, input = {}) {
- access.authorize(project, viewer, 'staff');
- const row = q.quotaRow.get(project.id) || {};
+ async function setQuota(viewer, project, input = {}) {
+ await access.authorize(project, viewer, 'staff');
+ const row = await q.quotaRow.get(project.id) || {};
const next = {};
const next = {};
for (const col of Object.keys(QUOTA_FIELDS)) {
for (const col of Object.keys(QUOTA_FIELDS)) {
if (!(col in input)) { next[col] = row[col] != null ? row[col] : null; continue; }
if (!(col in input)) { next[col] = row[col] != null ? row[col] : null; continue; }
@@ -135,27 +135,27 @@ function createProjects({ store, config, access, blobs, takedowns, log = console
@@ -135,27 +135,27 @@ function createProjects({ store, config, access, blobs, takedowns, log = console
if (!Number.isSafeInteger(v) || v < 0) throw new ApiError(422, 'quota.invalid', `${col} must be a non-negative integer or null (the default)`);
if (!Number.isSafeInteger(v) || v < 0) throw new ApiError(422, 'quota.invalid', `${col} must be a non-negative integer or null (the default)`);
next[col] = v;
next[col] = v;
}
}
- db.prepare(`INSERT INTO host_quotas (project_id, storage_bytes, deploys_per_day, max_files, max_file_bytes, sites, custom_domains, updated_by, updated_at)
+ await db.prepare(`INSERT INTO host_quotas (project_id, storage_bytes, deploys_per_day, max_files, max_file_bytes, sites, custom_domains, updated_by, updated_at)
VALUES (@project_id, @storage_bytes, @deploys_per_day, @max_files, @max_file_bytes, @sites, @custom_domains, @updated_by, @updated_at)
VALUES (@project_id, @storage_bytes, @deploys_per_day, @max_files, @max_file_bytes, @sites, @custom_domains, @updated_by, @updated_at)
ON CONFLICT (project_id) DO UPDATE SET storage_bytes = excluded.storage_bytes, deploys_per_day = excluded.deploys_per_day,
ON CONFLICT (project_id) DO UPDATE SET storage_bytes = excluded.storage_bytes, deploys_per_day = excluded.deploys_per_day,
max_files = excluded.max_files, max_file_bytes = excluded.max_file_bytes, sites = excluded.sites,
max_files = excluded.max_files, max_file_bytes = excluded.max_file_bytes, sites = excluded.sites,
custom_domains = excluded.custom_domains, updated_by = excluded.updated_by, updated_at = excluded.updated_at`)
custom_domains = excluded.custom_domains, updated_by = excluded.updated_by, updated_at = excluded.updated_at`)
.run({ project_id: project.id, ...next, updated_by: principalOf(viewer), updated_at: store.now() });
.run({ project_id: project.id, ...next, updated_by: principalOf(viewer), updated_at: store.now() });
- return quotaOf(project);
+ return await quotaOf(project);
}
}
/** Owner (or staff) deletes the project: every site stops serving at once; objects are removed. */
/** Owner (or staff) deletes the project: every site stops serving at once; objects are removed. */
- function remove(viewer, project, { sites }) {
- access.authorize(project, viewer, 'own');
- takedowns.assertDeletable(viewer, { projectId: project.id });
+ async function remove(viewer, project, { sites }) {
+ await access.authorize(project, viewer, 'own');
+ await takedowns.assertDeletable(viewer, { projectId: project.id });
const now = store.now();
const now = store.now();
- store.tx(() => {
- for (const s of sites.listForProject(project.id)) sites.markDeleted(s, now);
- db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE project_id = ? AND state <> 'deleted'").run(now, project.id);
- db.prepare("DELETE FROM host_deploy_files WHERE deploy_id IN (SELECT id FROM host_deploys WHERE project_id = ?)").run(project.id);
- db.prepare('DELETE FROM host_blobs WHERE project_id = ?').run(project.id);
- db.prepare('DELETE FROM host_domains WHERE project_id = ?').run(project.id);
- q.markDeleted.run(now, now, project.id);
+ await store.tx(async () => {
+ for (const s of await sites.listForProject(project.id)) await sites.markDeleted(s, now);
+ await db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE project_id = ? AND state <> 'deleted'").run(now, project.id);
+ await db.prepare("DELETE FROM host_deploy_files WHERE deploy_id IN (SELECT id FROM host_deploys WHERE project_id = ?)").run(project.id);
+ await db.prepare('DELETE FROM host_blobs WHERE project_id = ?').run(project.id);
+ await db.prepare('DELETE FROM host_domains WHERE project_id = ?').run(project.id);
+ await q.markDeleted.run(now, now, project.id);
});
});
try { blobs.removeProject(project.id); } catch (err) { log.warn('[Host] could not remove project objects:', err.message); }
try { blobs.removeProject(project.id); } catch (err) { log.warn('[Host] could not remove project objects:', err.message); }
return { deleted: true };
return { deleted: true };
@@ -28,7 +28,7 @@ function createSites({ store, config, access, projects, takedowns }) {
@@ -28,7 +28,7 @@ function createSites({ store, config, access, projects, takedowns }) {
byId: db.prepare('SELECT * FROM host_sites WHERE id = ?'),
byId: db.prepare('SELECT * FROM host_sites WHERE id = ?'),
byName: db.prepare("SELECT * FROM host_sites WHERE name = ? AND status = 'active'"),
byName: db.prepare("SELECT * FROM host_sites WHERE name = ? AND status = 'active'"),
held: db.prepare("SELECT project_id FROM host_sites WHERE name = ? AND status = 'deleted' AND deleted_at > ? ORDER BY deleted_at DESC LIMIT 1"),
held: db.prepare("SELECT project_id FROM host_sites WHERE name = ? AND status = 'deleted' AND deleted_at > ? ORDER BY deleted_at DESC LIMIT 1"),
- forProject: db.prepare("SELECT * FROM host_sites WHERE project_id = ? AND status = 'active' ORDER BY created_at, rowid"),
+ forProject: db.prepare("SELECT * FROM host_sites WHERE project_id = ? AND status = 'active' ORDER BY created_at, seq"),
insert: db.prepare(`INSERT INTO host_sites (id, project_id, name, active_deploy_id, status, created_by, created_at, updated_at)
insert: db.prepare(`INSERT INTO host_sites (id, project_id, name, active_deploy_id, status, created_by, created_at, updated_at)
VALUES (?, ?, ?, NULL, 'active', ?, ?, ?)`),
VALUES (?, ?, ?, NULL, 'active', ?, ?, ?)`),
insertDefaultDomain: db.prepare(`INSERT INTO host_domains (id, project_id, site_id, hostname, kind, status, token, created_by, created_at, verified_at)
insertDefaultDomain: db.prepare(`INSERT INTO host_domains (id, project_id, site_id, hostname, kind, status, token, created_by, created_at, verified_at)
@@ -38,17 +38,17 @@ function createSites({ store, config, access, projects, takedowns }) {
@@ -38,17 +38,17 @@ function createSites({ store, config, access, projects, takedowns }) {
const defaultHostname = (name) => `${name}.${config.sitesDomain}`;
const defaultHostname = (name) => `${name}.${config.sitesDomain}`;
- function get(id) {
- return isId('site', id) ? q.byId.get(id) || null : null;
+ async function get(id) {
+ return isId('site', id) ? await q.byId.get(id) || null : null;
}
}
/** The site and its project, authorized at `need`; 404 for anyone without access. */
/** The site and its project, authorized at `need`; 404 for anyone without access. */
- function load(viewer, id, need) {
- const site = get(id);
+ async function load(viewer, id, need) {
+ const site = await get(id);
const notFound = new ApiError(404, 'site.not_found', 'no such site');
const notFound = new ApiError(404, 'site.not_found', 'no such site');
if (!site || site.status !== 'active') throw notFound;
if (!site || site.status !== 'active') throw notFound;
- const project = projects.get(site.project_id);
- const role = access.authorize(project, viewer, need, notFound);
+ const project = await projects.get(site.project_id);
+ const role = await access.authorize(project, viewer, need, notFound);
return { site, project, role };
return { site, project, role };
}
}
@@ -59,56 +59,56 @@ function createSites({ store, config, access, projects, takedowns }) {
@@ -59,56 +59,56 @@ function createSites({ store, config, access, projects, takedowns }) {
if (reserved.has(name) || /^xn--/.test(name) || /^openvibe/.test(name)) throw new ApiError(422, 'site.name_reserved', `"${name}" is reserved`);
if (reserved.has(name) || /^xn--/.test(name) || /^openvibe/.test(name)) throw new ApiError(422, 'site.name_reserved', `"${name}" is reserved`);
}
}
- function create(viewer, project, input = {}) {
- access.authorize(project, viewer, 'maintain');
+ async function create(viewer, project, input = {}) {
+ await access.authorize(project, viewer, 'maintain');
const name = String(input.name || '').trim().toLowerCase();
const name = String(input.name || '').trim().toLowerCase();
checkName(name);
checkName(name);
- const quota = projects.quotaOf(project);
- if (projects.usageOf(project).sites >= quota.sites) throw new ApiError(429, 'quota.sites', `this project may have ${quota.sites} sites`);
- if (q.byName.get(name)) throw new ApiError(409, 'site.name_taken', `${defaultHostname(name)} is taken`);
- const held = q.held.get(name, store.now() - HOLD_MS);
+ const quota = await projects.quotaOf(project);
+ if ((await projects.usageOf(project)).sites >= quota.sites) throw new ApiError(429, 'quota.sites', `this project may have ${quota.sites} sites`);
+ if (await q.byName.get(name)) throw new ApiError(409, 'site.name_taken', `${defaultHostname(name)} is taken`);
+ const held = await q.held.get(name, store.now() - HOLD_MS);
if (held && held.project_id !== project.id) throw new ApiError(409, 'site.name_held', `${defaultHostname(name)} was deleted recently and is held for 30 days`);
if (held && held.project_id !== project.id) throw new ApiError(409, 'site.name_held', `${defaultHostname(name)} was deleted recently and is held for 30 days`);
const id = newId('site', store.now());
const id = newId('site', store.now());
const now = store.now();
const now = store.now();
const who = principalOf(viewer);
const who = principalOf(viewer);
try {
try {
- store.tx(() => {
- q.insert.run(id, project.id, name, who, now, now);
- q.insertDefaultDomain.run(newId('domain', now), project.id, id, defaultHostname(name), who, now, now);
+ await store.tx(async () => {
+ await q.insert.run(id, project.id, name, who, now, now);
+ await q.insertDefaultDomain.run(newId('domain', now), project.id, id, defaultHostname(name), who, now, now);
});
});
} catch (err) {
} catch (err) {
- if (/UNIQUE/.test(err.message)) throw new ApiError(409, 'site.name_taken', `${defaultHostname(name)} is taken`);
+ if (err.code === '23505') throw new ApiError(409, 'site.name_taken', `${defaultHostname(name)} is taken`);
throw err;
throw err;
}
}
- return get(id);
+ return await get(id);
}
}
- function listForProject(projectId) {
- return q.forProject.all(projectId);
+ async function listForProject(projectId) {
+ return await q.forProject.all(projectId);
}
}
/** Inside a transaction: the site stops serving immediately; deploys and domains go with it. */
/** Inside a transaction: the site stops serving immediately; deploys and domains go with it. */
- function markDeleted(site, now) {
- q.markDeleted.run(now, now, site.id);
- db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE site_id = ? AND state <> 'deleted'").run(now, site.id);
- db.prepare('DELETE FROM host_deploy_files WHERE deploy_id IN (SELECT id FROM host_deploys WHERE site_id = ?)').run(site.id);
- db.prepare('DELETE FROM host_domains WHERE site_id = ?').run(site.id);
+ async function markDeleted(site, now) {
+ await q.markDeleted.run(now, now, site.id);
+ await db.prepare("UPDATE host_deploys SET state = 'deleted', deleted_at = ? WHERE site_id = ? AND state <> 'deleted'").run(now, site.id);
+ await db.prepare('DELETE FROM host_deploy_files WHERE deploy_id IN (SELECT id FROM host_deploys WHERE site_id = ?)').run(site.id);
+ await db.prepare('DELETE FROM host_domains WHERE site_id = ?').run(site.id);
}
}
- function remove(viewer, id, { deploys }) {
- const { site, project } = load(viewer, id, 'maintain');
- takedowns.assertDeletable(viewer, { site });
+ async function remove(viewer, id, { deploys }) {
+ const { site, project } = await load(viewer, id, 'maintain');
+ await takedowns.assertDeletable(viewer, { site });
const now = store.now();
const now = store.now();
let freed = [];
let freed = [];
- store.tx(() => {
- markDeleted(site, now);
- freed = deploys.collectGarbage(project.id);
+ await store.tx(async () => {
+ await markDeleted(site, now);
+ freed = await deploys.collectGarbage(project.id);
});
});
deploys.unlinkBlobs(project.id, freed);
deploys.unlinkBlobs(project.id, freed);
return { deleted: true };
return { deleted: true };
}
}
- return { get, load, create, listForProject, markDeleted, remove, defaultHostname, checkName, byName: (n) => q.byName.get(n) || null };
+ return { get, load, create, listForProject, markDeleted, remove, defaultHostname, checkName, byName: async (n) => await q.byName.get(n) || null };
}
}
module.exports = { createSites, RESERVED, NAME_RE };
module.exports = { createSites, RESERVED, NAME_RE };
@@ -34,20 +34,20 @@ function createTakedowns({ store }) {
@@ -34,20 +34,20 @@ function createTakedowns({ store }) {
const shape = (row) => (row ? { scope: row.target_kind, reason: row.reason, since: row.created_at } : null);
const shape = (row) => (row ? { scope: row.target_kind, reason: row.reason, since: row.created_at } : null);
/** The takedown that covers a site (its own, or its project's), or null. */
/** The takedown that covers a site (its own, or its project's), or null. */
- const ofSite = (site) => shape(q.forSite.get(site.id, site.project_id));
+ const ofSite = async (site) => shape(await q.forSite.get(site.id, site.project_id));
/** The project-wide takedown, or null. */
/** The project-wide takedown, or null. */
- const ofProject = (projectId) => shape(q.forProject.get(projectId));
+ const ofProject = async (projectId) => shape(await q.forProject.get(projectId));
/** Members may not publish into, or change, content that is taken down. */
/** Members may not publish into, or change, content that is taken down. */
- function assertOpen(site) {
- const t = ofSite(site);
+ async function assertOpen(site) {
+ const t = await ofSite(site);
if (t) throw new ApiError(403, 'site.taken_down', `this ${t.scope} was taken down by OpenVibe staff (${t.reason}); nothing can be published or changed until staff lift it`);
if (t) throw new ApiError(403, 'site.taken_down', `this ${t.scope} was taken down by OpenVibe staff (${t.reason}); nothing can be published or changed until staff lift it`);
}
}
/** Deleting taken-down content would destroy what staff are reviewing: staff only. */
/** Deleting taken-down content would destroy what staff are reviewing: staff only. */
- function assertDeletable(viewer, { site = null, projectId = null }) {
+ async function assertDeletable(viewer, { site = null, projectId = null }) {
if (viewer && viewer.staff) return;
if (viewer && viewer.staff) return;
- const row = site ? q.forSite.get(site.id, site.project_id) : q.anyInProject.get({ p: projectId });
+ const row = site ? await q.forSite.get(site.id, site.project_id) : await q.anyInProject.get({ p: projectId });
if (row) throw new ApiError(409, 'site.taken_down', `this ${row.target_kind} was taken down by OpenVibe staff and is kept for review; it cannot be deleted until staff lift the takedown`);
if (row) throw new ApiError(409, 'site.taken_down', `this ${row.target_kind} was taken down by OpenVibe staff and is kept for review; it cannot be deleted until staff lift the takedown`);
}
}
@@ -56,22 +56,22 @@ function createTakedowns({ store }) {
@@ -56,22 +56,22 @@ function createTakedowns({ store }) {
}
}
/** kind: 'site' | 'project'. -> the takedown */
/** kind: 'site' | 'project'. -> the takedown */
- function takeDown(viewer, kind, targetId, reasonRaw) {
+ async function takeDown(viewer, kind, targetId, reasonRaw) {
requireStaff(viewer);
requireStaff(viewer);
const reason = String(reasonRaw || '').trim();
const reason = String(reasonRaw || '').trim();
if (!reason || reason.length > REASON_MAX) throw new ApiError(422, 'takedown.reason_required', `a takedown needs a reason of 1–${REASON_MAX} characters (members see it)`);
if (!reason || reason.length > REASON_MAX) throw new ApiError(422, 'takedown.reason_required', `a takedown needs a reason of 1–${REASON_MAX} characters (members see it)`);
try {
try {
- q.insert.run(kind, targetId, reason, principalOf(viewer), store.now());
+ await q.insert.run(kind, targetId, reason, principalOf(viewer), store.now());
} catch (err) {
} catch (err) {
- if (/UNIQUE/.test(err.message)) throw new ApiError(409, 'takedown.exists', `this ${kind} is already taken down`);
+ if (err.code === '23505') throw new ApiError(409, 'takedown.exists', `this ${kind} is already taken down`);
throw err;
throw err;
}
}
return { scope: kind, reason, since: store.now() };
return { scope: kind, reason, since: store.now() };
}
}
- function lift(viewer, kind, targetId, note) {
+ async function lift(viewer, kind, targetId, note) {
requireStaff(viewer);
requireStaff(viewer);
- const r = q.lift.run(principalOf(viewer), store.now(), note == null ? null : String(note).slice(0, REASON_MAX), kind, targetId);
+ const r = await q.lift.run(principalOf(viewer), store.now(), note == null ? null : String(note).slice(0, REASON_MAX), kind, targetId);
if (r.changes !== 1) throw new ApiError(404, 'takedown.not_found', `this ${kind} is not taken down`);
if (r.changes !== 1) throw new ApiError(404, 'takedown.not_found', `this ${kind} is not taken down`);
return { lifted: true };
return { lifted: true };
}
}
@@ -15,7 +15,7 @@
@@ -15,7 +15,7 @@
*/
*/
const { createClient } = require('openvibe-sdk/core');
const { createClient } = require('openvibe-sdk/core');
const { createServiceTokenClient } = require('openvibe-sdk/auth');
const { createServiceTokenClient } = require('openvibe-sdk/auth');
-const { createEventsClient, createOutbox } = require('openvibe-sdk/events');
+const { createEventsClient, createPgOutbox } = require('openvibe-sdk/events');
const EVENT_TYPES = Object.freeze(['host.deploy.created', 'host.deploy.activated', 'host.deploy.failed', 'host.domain.verified']);
const EVENT_TYPES = Object.freeze(['host.deploy.created', 'host.deploy.activated', 'host.deploy.failed', 'host.domain.verified']);
@@ -33,7 +33,9 @@ function createHostOutbox({ db, config, fetchImpl, now, log = console }) {
@@ -33,7 +33,9 @@ function createHostOutbox({ db, config, fetchImpl, now, log = console }) {
}
}
const events = createEventsClient(createClient(clientOpts), { source: 'host' });
const events = createEventsClient(createClient(clientOpts), { source: 'host' });
let lastError = null;
let lastError = null;
- const outbox = createOutbox(db, {
+ // The PostgreSQL outbox: rows are written in the change's own transaction (enqueue(db, …) joins the ambient
+ // transaction); several processes relay one table safely (leases).
+ const outbox = createPgOutbox(db, {
events,
events,
intervalMs: config.events.intervalMs,
intervalMs: config.events.intervalMs,
now,
now,
@@ -43,12 +45,11 @@ function createHostOutbox({ db, config, fetchImpl, now, log = console }) {
@@ -43,12 +45,11 @@ function createHostOutbox({ db, config, fetchImpl, now, log = console }) {
lastError = msg;
lastError = msg;
},
},
});
});
- outbox.ensureSchema();
/** Inside the caller's transaction. Returns the complete envelope (with its event_id). */
/** Inside the caller's transaction. Returns the complete envelope (with its event_id). */
- function emit(envelope, { traceparent } = {}) {
+ async function emit(envelope, { traceparent } = {}) {
if (!EVENT_TYPES.includes(envelope.event_type)) throw new Error(`undeclared event type ${envelope.event_type}`);
if (!EVENT_TYPES.includes(envelope.event_type)) throw new Error(`undeclared event type ${envelope.event_type}`);
- return outbox.enqueue(envelope, { traceparent });
+ return await outbox.enqueue(db, envelope, { traceparent });
}
}
return {
return {
@@ -58,7 +59,7 @@ function createHostOutbox({ db, config, fetchImpl, now, log = console }) {
@@ -58,7 +59,7 @@ function createHostOutbox({ db, config, fetchImpl, now, log = console }) {
start() { if (enabled) outbox.start(); },
start() { if (enabled) outbox.start(); },
stop: () => outbox.stop(),
stop: () => outbox.stop(),
kick() { if (enabled) outbox.kick(); },
kick() { if (enabled) outbox.kick(); },
- status: () => ({ enabled, pending: outbox.pending(), rejected: outbox.rejected(), last_error: lastError }),
+ status: async () => ({ enabled, pending: await outbox.pending(), rejected: await outbox.rejected(), last_error: lastError }),
};
};
}
}
@@ -10,7 +10,7 @@
@@ -10,7 +10,7 @@
*/
*/
'use strict';
'use strict';
-const { createActorLimiter } = require('openvibe-sdk/limits');
+const { createActorLimiter, createValkeyLimitStore } = require('openvibe-sdk/limits');
const num = (v, d) => { const n = parseInt(v, 10); return Number.isFinite(n) && n > 0 ? n : d; };
const num = (v, d) => { const n = parseInt(v, 10); return Number.isFinite(n) && n > 0 ? n : d; };
@@ -21,7 +21,7 @@ const ROUTES = [
@@ -21,7 +21,7 @@ const ROUTES = [
['host.project.create', /^POST$/, /^\/projects\/?$/, { minute: 5, hour: 30 }],
['host.project.create', /^POST$/, /^\/projects\/?$/, { minute: 5, hour: 30 }],
];
];
-function createHostActorLimits({ env = process.env, registry = null, now } = {}) {
+function createHostActorLimits({ env = process.env, registry = null, now, valkey = null } = {}) {
// HOST_ACTOR_LIMITS=off: a rollback lever (and what the Stage B suites use: they deploy dozens of times a minute).
// HOST_ACTOR_LIMITS=off: a rollback lever (and what the Stage B suites use: they deploy dozens of times a minute).
if (/^(off|0|false)$/i.test(String(env.HOST_ACTOR_LIMITS || ''))) return function noLimits(req, res, next) { next(); };
if (/^(off|0|false)$/i.test(String(env.HOST_ACTOR_LIMITS || ''))) return function noLimits(req, res, next) { next(); };
const refused = registry && typeof registry.counter === 'function'
const refused = registry && typeof registry.counter === 'function'
@@ -38,6 +38,8 @@ function createHostActorLimits({ env = process.env, registry = null, now } = {})
@@ -38,6 +38,8 @@ function createHostActorLimits({ env = process.env, registry = null, now } = {})
limits: { minute: num(env.HOST_LIMITS_MINUTE, 120), hour: num(env.HOST_LIMITS_HOUR, 3000) },
limits: { minute: num(env.HOST_LIMITS_MINUTE, 120), hour: num(env.HOST_LIMITS_HOUR, 3000) },
actor,
actor,
...(now ? { now } : {}),
...(now ? { now } : {}),
+ // Shared across processes on Valkey (ADR-035) when VALKEY_URL is set; in-process otherwise.
+ ...(valkey ? { store: createValkeyLimitStore(valkey) } : {}),
onLimited(e) {
onLimited(e) {
console.warn(`[Limits] ${e.name}: ${e.actor} refused, over ${e.limit} per ${e.window}`);
console.warn(`[Limits] ${e.name}: ${e.actor} refused, over ${e.limit} per ${e.window}`);
if (refused) refused.inc({ limit: e.name, window: e.window });
if (refused) refused.inc({ limit: e.name, window: e.window });
@@ -49,103 +49,103 @@ function createApi(ctx) {
@@ -49,103 +49,103 @@ function createApi(ctx) {
const u = new URL(config.baseUrl);
const u = new URL(config.baseUrl);
return `${u.protocol}//${hostname}${u.port ? `:${u.port}` : ''}`;
return `${u.protocol}//${hostname}${u.port ? `:${u.port}` : ''}`;
};
};
- const siteOut = (s) => { const hostname = sites.defaultHostname(s.name); return out.site(s, { hostname, url: siteUrl(hostname), takedown: takedowns.ofSite(s) }); };
- const projectOut = (p, role) => out.project(p, { role, quota: projects.quotaOf(p), usage: projects.usageOf(p), takedown: takedowns.ofProject(p.id) });
+ const siteOut = async (s) => { const hostname = sites.defaultHostname(s.name); return out.site(s, { hostname, url: siteUrl(hostname), takedown: await takedowns.ofSite(s) }); };
+ const projectOut = async (p, role) => out.project(p, { role, quota: await projects.quotaOf(p), usage: await projects.usageOf(p), takedown: await takedowns.ofProject(p.id) });
const tp = (req) => (req.ov ? req.ov.traceparent : undefined);
const tp = (req) => (req.ov ? req.ov.traceparent : undefined);
- const loadProject = (req, need) => {
- const project = projects.get(req.params.id);
- const role = ctx.access.authorize(project, req.viewer, need);
+ const loadProject = async (req, need) => {
+ const project = await projects.get(req.params.id);
+ const role = await ctx.access.authorize(project, req.viewer, need);
return { project, role };
return { project, role };
};
};
// ── Projects ────────────────────────────────────────────
// ── Projects ────────────────────────────────────────────
- router.get('/projects', guard('host.site.manage'), run((req) => {
+ router.get('/projects', guard('host.site.manage'), run(async (req) => {
if (req.viewer.kind === 'anonymous') throw new ApiError(401, 'auth.required', 'sign in with OpenVibe, or present a service token');
if (req.viewer.kind === 'anonymous') throw new ApiError(401, 'auth.required', 'sign in with OpenVibe, or present a service token');
- return { projects: projects.listFor(req.viewer).map((p) => out.project(p, { takedown: takedowns.ofProject(p.id) })) };
+ return { projects: (await Promise.all((await projects.listFor(req.viewer)).map(async (p) => out.project(p, { takedown: await takedowns.ofProject(p.id) })))) };
}));
}));
- router.post('/projects', guard('host.site.manage'), jsonBody, run((req) => {
+ router.post('/projects', guard('host.site.manage'), jsonBody, run(async (req) => {
if (req.viewer.kind === 'anonymous') throw new ApiError(401, 'auth.required', 'sign in with OpenVibe, or present a service token');
if (req.viewer.kind === 'anonymous') throw new ApiError(401, 'auth.required', 'sign in with OpenVibe, or present a service token');
- const p = projects.create(req.viewer, req.body || {});
- return { project: projectOut(p, 'owner') };
+ const p = await projects.create(req.viewer, req.body || {});
+ return { project: await projectOut(p, 'owner') };
}, 201));
}, 201));
- router.get('/projects/:id', guard('host.site.manage'), run((req) => {
- const { project, role } = loadProject(req, 'read');
- return { project: projectOut(project, role), sites: sites.listForProject(project.id).map(siteOut), members: projects.members(project).map((m) => ({ principal: m.principal, role: m.role, added_at: out.iso(m.created_at) })) };
- }));
- router.delete('/projects/:id', guard('host.site.manage'), run((req) => {
- const { project } = loadProject(req, 'read');
- return projects.remove(req.viewer, project, { sites });
- }));
- router.get('/projects/:id/quota', guard('host.site.manage'), run((req) => {
- const { project } = loadProject(req, 'read');
- return { quota: out.quotaOut(projects.quotaOf(project)), usage: out.usageOut(projects.usageOf(project)) };
- }));
- router.put('/projects/:id/quota', guard('host.site.manage'), jsonBody, run((req) => {
- const project = projects.get(req.params.id);
- const q = projects.setQuota(req.viewer, project, req.body || {});
- return { quota: out.quotaOut(q), usage: out.usageOut(projects.usageOf(project)) };
- }));
- router.put('/projects/:id/members/:principal', guard('host.site.manage'), jsonBody, run((req) => {
- const { project } = loadProject(req, 'read');
- const list = projects.setMember(req.viewer, project, req.params.principal, String((req.body || {}).role || ''));
+ router.get('/projects/:id', guard('host.site.manage'), run(async (req) => {
+ const { project, role } = await loadProject(req, 'read');
+ return { project: await projectOut(project, role), sites: (await Promise.all((await sites.listForProject(project.id)).map(siteOut))), members: (await projects.members(project)).map((m) => ({ principal: m.principal, role: m.role, added_at: out.iso(m.created_at) })) };
+ }));
+ router.delete('/projects/:id', guard('host.site.manage'), run(async (req) => {
+ const { project } = await loadProject(req, 'read');
+ return await projects.remove(req.viewer, project, { sites });
+ }));
+ router.get('/projects/:id/quota', guard('host.site.manage'), run(async (req) => {
+ const { project } = await loadProject(req, 'read');
+ return { quota: out.quotaOut(await projects.quotaOf(project)), usage: out.usageOut(await projects.usageOf(project)) };
+ }));
+ router.put('/projects/:id/quota', guard('host.site.manage'), jsonBody, run(async (req) => {
+ const project = await projects.get(req.params.id);
+ const q = await projects.setQuota(req.viewer, project, req.body || {});
+ return { quota: out.quotaOut(q), usage: out.usageOut(await projects.usageOf(project)) };
+ }));
+ router.put('/projects/:id/members/:principal', guard('host.site.manage'), jsonBody, run(async (req) => {
+ const { project } = await loadProject(req, 'read');
+ const list = await projects.setMember(req.viewer, project, req.params.principal, String((req.body || {}).role || ''));
return { members: list.map((m) => ({ principal: m.principal, role: m.role, added_at: out.iso(m.created_at) })) };
return { members: list.map((m) => ({ principal: m.principal, role: m.role, added_at: out.iso(m.created_at) })) };
}));
}));
- router.delete('/projects/:id/members/:principal', guard('host.site.manage'), run((req) => {
- const { project } = loadProject(req, 'read');
- const list = projects.removeMember(req.viewer, project, req.params.principal);
+ router.delete('/projects/:id/members/:principal', guard('host.site.manage'), run(async (req) => {
+ const { project } = await loadProject(req, 'read');
+ const list = await projects.removeMember(req.viewer, project, req.params.principal);
return { members: list.map((m) => ({ principal: m.principal, role: m.role, added_at: out.iso(m.created_at) })) };
return { members: list.map((m) => ({ principal: m.principal, role: m.role, added_at: out.iso(m.created_at) })) };
}));
}));
// ── Takedowns (staff only; members see them on the project and site) ──
// ── Takedowns (staff only; members see them on the project and site) ──
// Serving stops at once and the content is kept for review (domain/takedowns.js).
// Serving stops at once and the content is kept for review (domain/takedowns.js).
- router.post('/projects/:id/takedown', guard('host.site.manage'), jsonBody, run((req) => {
- const { project } = loadProject(req, 'read');
- return { takedown: out.takedownOut(takedowns.takeDown(req.viewer, 'project', project.id, (req.body || {}).reason)) };
+ router.post('/projects/:id/takedown', guard('host.site.manage'), jsonBody, run(async (req) => {
+ const { project } = await loadProject(req, 'read');
+ return { takedown: out.takedownOut(await takedowns.takeDown(req.viewer, 'project', project.id, (req.body || {}).reason)) };
}, 201));
}, 201));
- router.delete('/projects/:id/takedown', guard('host.site.manage'), jsonBody, run((req) => {
- const { project } = loadProject(req, 'read');
- return takedowns.lift(req.viewer, 'project', project.id, (req.body || {}).note);
+ router.delete('/projects/:id/takedown', guard('host.site.manage'), jsonBody, run(async (req) => {
+ const { project } = await loadProject(req, 'read');
+ return await takedowns.lift(req.viewer, 'project', project.id, (req.body || {}).note);
}));
}));
- router.post('/sites/:id/takedown', guard('host.site.manage'), jsonBody, run((req) => {
- const { site } = sites.load(req.viewer, req.params.id, 'read');
- return { takedown: out.takedownOut(takedowns.takeDown(req.viewer, 'site', site.id, (req.body || {}).reason)) };
+ router.post('/sites/:id/takedown', guard('host.site.manage'), jsonBody, run(async (req) => {
+ const { site } = await sites.load(req.viewer, req.params.id, 'read');
+ return { takedown: out.takedownOut(await takedowns.takeDown(req.viewer, 'site', site.id, (req.body || {}).reason)) };
}, 201));
}, 201));
- router.delete('/sites/:id/takedown', guard('host.site.manage'), jsonBody, run((req) => {
- const { site } = sites.load(req.viewer, req.params.id, 'read');
- return takedowns.lift(req.viewer, 'site', site.id, (req.body || {}).note);
+ router.delete('/sites/:id/takedown', guard('host.site.manage'), jsonBody, run(async (req) => {
+ const { site } = await sites.load(req.viewer, req.params.id, 'read');
+ return await takedowns.lift(req.viewer, 'site', site.id, (req.body || {}).note);
}));
}));
// ── Sites ───────────────────────────────────────────────
// ── Sites ───────────────────────────────────────────────
- router.get('/projects/:id/sites', guard('host.site.manage'), run((req) => {
- const { project } = loadProject(req, 'read');
- return { sites: sites.listForProject(project.id).map(siteOut) };
- }));
- router.post('/projects/:id/sites', guard('host.site.manage'), jsonBody, run((req) => {
- const project = projects.get(req.params.id);
- ctx.access.authorize(project, req.viewer, 'read');
- return { site: siteOut(sites.create(req.viewer, project, req.body || {})) };
+ router.get('/projects/:id/sites', guard('host.site.manage'), run(async (req) => {
+ const { project } = await loadProject(req, 'read');
+ return { sites: (await Promise.all((await sites.listForProject(project.id)).map(siteOut))) };
+ }));
+ router.post('/projects/:id/sites', guard('host.site.manage'), jsonBody, run(async (req) => {
+ const project = await projects.get(req.params.id);
+ await ctx.access.authorize(project, req.viewer, 'read');
+ return { site: await siteOut(await sites.create(req.viewer, project, req.body || {})) };
}, 201));
}, 201));
- router.get('/sites/:id', guard('host.site.manage'), run((req) => {
- const { site } = sites.load(req.viewer, req.params.id, 'read');
- return { site: siteOut(site), domains: domains.listForSite(site.id).map((d) => out.domain(d, domains.instructions(d, site))) };
+ router.get('/sites/:id', guard('host.site.manage'), run(async (req) => {
+ const { site } = await sites.load(req.viewer, req.params.id, 'read');
+ return { site: await siteOut(site), domains: (await domains.listForSite(site.id)).map((d) => out.domain(d, domains.instructions(d, site))) };
}));
}));
- router.delete('/sites/:id', guard('host.site.manage'), run((req) => sites.remove(req.viewer, req.params.id, { deploys })));
+ router.delete('/sites/:id', guard('host.site.manage'), run(async (req) => await sites.remove(req.viewer, req.params.id, { deploys })));
// ── Deploys ─────────────────────────────────────────────
// ── Deploys ─────────────────────────────────────────────
- router.get('/sites/:id/deploys', guard('host.deploy.create'), run((req) => {
+ router.get('/sites/:id/deploys', guard('host.deploy.create'), run(async (req) => {
// Reads need the least member role (deployer); staff may read them too, e.g. to review a takedown.
// Reads need the least member role (deployer); staff may read them too, e.g. to review a takedown.
- const { site } = sites.load(req.viewer, req.params.id, 'read');
+ const { site } = await sites.load(req.viewer, req.params.id, 'read');
return {
return {
active_deploy_id: site.active_deploy_id || null,
active_deploy_id: site.active_deploy_id || null,
- deploys: deploys.list(site.id, Number(req.query.limit) || 50).map((d) => out.deploy(d, { active: d.id === site.active_deploy_id })),
- activations: deploys.activationsOf(site.id, 20).map(out.activation),
+ deploys: (await deploys.list(site.id, Number(req.query.limit) || 50)).map((d) => out.deploy(d, { active: d.id === site.active_deploy_id })),
+ activations: (await deploys.activationsOf(site.id, 20)).map(out.activation),
};
};
}));
}));
router.post('/sites/:id/deploys', guard('host.deploy.create'), async (req, res) => {
router.post('/sites/:id/deploys', guard('host.deploy.create'), async (req, res) => {
let release = null;
let release = null;
try {
try {
- const pre = deploys.precheck(req.viewer, req.params.id);
+ const pre = await deploys.precheck(req.viewer, req.params.id);
release = ctx.uploadGate.enter();
release = ctx.uploadGate.enter();
if (!release) { res.set('Retry-After', '30'); throw new ApiError(503, 'upload.busy', 'Host is validating other uploads right now; try again in 30 seconds'); }
if (!release) { res.set('Retry-After', '30'); throw new ApiError(503, 'upload.busy', 'Host is validating other uploads right now; try again in 30 seconds'); }
let up;
let up;
@@ -155,12 +155,12 @@ function createApi(ctx) {
@@ -155,12 +155,12 @@ function createApi(ctx) {
if (!(err instanceof UploadError)) throw err;
if (!(err instanceof UploadError)) throw err;
if (err.closeConnection) res.set('Connection', 'close');
if (err.closeConnection) res.set('Connection', 'close');
const multipart = String(req.headers['content-type'] || '').startsWith('multipart/');
const multipart = String(req.headers['content-type'] || '').startsWith('multipart/');
- const r = deploys.recordFailure(req.viewer, pre, { source: err.source || (multipart ? 'files' : 'archive'), code: err.code, problems: [{ code: err.code, message: err.message }], traceparent: tp(req) });
+ const r = await deploys.recordFailure(req.viewer, pre, { source: err.source || (multipart ? 'files' : 'archive'), code: err.code, problems: [{ code: err.code, message: err.message }], traceparent: tp(req) });
return contracts.http.sendProblem(res, err.status, err.code, { detail: err.message, ctx: req.ov, extra: { deploy_id: r.deploy.id, log: r.log.map((l) => `${l.level}: ${l.message}`) } });
return contracts.http.sendProblem(res, err.status, err.code, { detail: err.message, ctx: req.ov, extra: { deploy_id: r.deploy.id, log: r.log.map((l) => `${l.level}: ${l.message}`) } });
}
}
const activate = truthy(up.fields.activate != null ? up.fields.activate : req.query.activate);
const activate = truthy(up.fields.activate != null ? up.fields.activate : req.query.activate);
- const r = deploys.create(req.viewer, pre, up.entries, { source: up.source, activate, notes: up.notes, traceparent: tp(req) });
- const site = sites.get(pre.site.id);
+ const r = await deploys.create(req.viewer, pre, up.entries, { source: up.source, activate, notes: up.notes, traceparent: tp(req) });
+ const site = await sites.get(pre.site.id);
res.status(201).json({ deploy: out.deploy(r.deploy, { active: site.active_deploy_id === r.deploy.id, log: r.log }), activated: Boolean(r.activated && r.activated.changed), url: siteUrl(sites.defaultHostname(site.name)) });
res.status(201).json({ deploy: out.deploy(r.deploy, { active: site.active_deploy_id === r.deploy.id, log: r.log }), activated: Boolean(r.activated && r.activated.changed), url: siteUrl(sites.defaultHostname(site.name)) });
} catch (err) {
} catch (err) {
// Refused before the body was read (404, 429, 503): close instead of draining a large upload.
// Refused before the body was read (404, 429, 503): close instead of draining a large upload.
@@ -171,41 +171,41 @@ function createApi(ctx) {
@@ -171,41 +171,41 @@ function createApi(ctx) {
}
}
});
});
- router.get('/deploys/:id', guard('host.deploy.create'), run((req) => {
- const { deploy, site } = deploys.load(req.viewer, req.params.id, 'read');
- return { deploy: out.deploy(deploy, { active: site.active_deploy_id === deploy.id, files: deploys.filesOf(deploy.id) }) };
+ router.get('/deploys/:id', guard('host.deploy.create'), run(async (req) => {
+ const { deploy, site } = await deploys.load(req.viewer, req.params.id, 'read');
+ return { deploy: out.deploy(deploy, { active: site.active_deploy_id === deploy.id, files: await deploys.filesOf(deploy.id) }) };
}));
}));
- router.get('/deploys/:id/log', guard('host.deploy.create'), run((req) => {
- const { deploy } = deploys.load(req.viewer, req.params.id, 'read');
- return { deploy_id: deploy.id, state: deploy.state, log: deploys.logOf(deploy.id).map(out.logLine) };
+ router.get('/deploys/:id/log', guard('host.deploy.create'), run(async (req) => {
+ const { deploy } = await deploys.load(req.viewer, req.params.id, 'read');
+ return { deploy_id: deploy.id, state: deploy.state, log: (await deploys.logOf(deploy.id)).map(out.logLine) };
}));
}));
- router.post('/deploys/:id/activate', guard('host.deploy.create'), jsonBody, run((req) => {
+ router.post('/deploys/:id/activate', guard('host.deploy.create'), jsonBody, run(async (req) => {
const body = req.body || {};
const body = req.body || {};
- const r = deploys.activate(req.viewer, req.params.id, { expectedActive: 'expected_active' in body ? body.expected_active : undefined, traceparent: tp(req) });
+ const r = await deploys.activate(req.viewer, req.params.id, { expectedActive: 'expected_active' in body ? body.expected_active : undefined, traceparent: tp(req) });
return { active_deploy_id: r.deploy_id, previous_deploy_id: r.previous_deploy_id, changed: r.changed };
return { active_deploy_id: r.deploy_id, previous_deploy_id: r.previous_deploy_id, changed: r.changed };
}));
}));
- router.post('/sites/:id/rollback', guard('host.deploy.create'), jsonBody, run((req) => {
+ router.post('/sites/:id/rollback', guard('host.deploy.create'), jsonBody, run(async (req) => {
const body = req.body || {};
const body = req.body || {};
- const r = deploys.rollback(req.viewer, req.params.id, { deployId: body.deploy_id || null, expectedActive: 'expected_active' in body ? body.expected_active : undefined, traceparent: tp(req) });
+ const r = await deploys.rollback(req.viewer, req.params.id, { deployId: body.deploy_id || null, expectedActive: 'expected_active' in body ? body.expected_active : undefined, traceparent: tp(req) });
return { active_deploy_id: r.deploy_id, previous_deploy_id: r.previous_deploy_id, changed: r.changed };
return { active_deploy_id: r.deploy_id, previous_deploy_id: r.previous_deploy_id, changed: r.changed };
}));
}));
- router.delete('/deploys/:id', guard('host.site.manage'), run((req) => deploys.remove(req.viewer, req.params.id)));
+ router.delete('/deploys/:id', guard('host.site.manage'), run(async (req) => await deploys.remove(req.viewer, req.params.id)));
// ── Domains ─────────────────────────────────────────────
// ── Domains ─────────────────────────────────────────────
- router.get('/sites/:id/domains', guard('host.domain.manage'), run((req) => {
- const { site } = sites.load(req.viewer, req.params.id, 'read');
- return { domains: domains.listForSite(site.id).map((d) => out.domain(d, domains.instructions(d, site))) };
+ router.get('/sites/:id/domains', guard('host.domain.manage'), run(async (req) => {
+ const { site } = await sites.load(req.viewer, req.params.id, 'read');
+ return { domains: (await domains.listForSite(site.id)).map((d) => out.domain(d, domains.instructions(d, site))) };
}));
}));
- router.post('/sites/:id/domains', guard('host.domain.manage'), jsonBody, run((req) => {
- const { domain, site } = domains.add(req.viewer, req.params.id, req.body || {});
+ router.post('/sites/:id/domains', guard('host.domain.manage'), jsonBody, run(async (req) => {
+ const { domain, site } = await domains.add(req.viewer, req.params.id, req.body || {});
return { domain: out.domain(domain, domains.instructions(domain, site)) };
return { domain: out.domain(domain, domains.instructions(domain, site)) };
}, 201));
}, 201));
router.post('/domains/:id/verify', guard('host.domain.manage'), run(async (req) => {
router.post('/domains/:id/verify', guard('host.domain.manage'), run(async (req) => {
const d = await domains.verify(req.viewer, req.params.id, { traceparent: tp(req) });
const d = await domains.verify(req.viewer, req.params.id, { traceparent: tp(req) });
- const site = sites.get(d.site_id);
+ const site = await sites.get(d.site_id);
return { domain: out.domain(d, domains.instructions(d, site)) };
return { domain: out.domain(d, domains.instructions(d, site)) };
}));
}));
- router.delete('/domains/:id', guard('host.domain.manage'), run((req) => domains.remove(req.viewer, req.params.id)));
+ router.delete('/domains/:id', guard('host.domain.manage'), run(async (req) => await domains.remove(req.viewer, req.params.id)));
router.use((req, res) => contracts.http.sendProblem(res, 404, 'route.not_found', { detail: `no route ${req.method} ${req.baseUrl}${req.path}`, ctx: req.ov }));
router.use((req, res) => contracts.http.sendProblem(res, 404, 'route.not_found', { detail: `no route ${req.method} ${req.baseUrl}${req.path}`, ctx: req.ov }));
return router;
return router;
@@ -66,102 +66,102 @@ function createDashboard(ctx) {
@@ -66,102 +66,102 @@ function createDashboard(ctx) {
// ── Pages ───────────────────────────────────────────────
// ── Pages ───────────────────────────────────────────────
// What shipped on OpenVibe.Host: the shared update log every OpenVibe site has.
// What shipped on OpenVibe.Host: the shared update log every OpenVibe site has.
router.get('/updates', view((req, res) => page(req, res, 200, 'What shipped on OpenVibe.Host', frame.updatesBody({ service: 'host', siteName: 'OpenVibe.Host' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>`, null, { indexable: true })));
router.get('/updates', view((req, res) => page(req, res, 200, 'What shipped on OpenVibe.Host', frame.updatesBody({ service: 'host', siteName: 'OpenVibe.Host' }) + `<script src="${ovServe.url('shipped.js')}" defer></script>`, null, { indexable: true })));
- router.get('/', view((req, res) => {
+ router.get('/', view(async (req, res) => {
// The signed-out front page is the public page of openvibe.host (sitemap.xml): indexable.
// The signed-out front page is the public page of openvibe.host (sitemap.xml): indexable.
// Everything behind sign-in stays noindex and private.
// Everything behind sign-in stays noindex and private.
if (req.viewer.kind !== 'user') return page(req, res, 200, null, pages.signedOut(), null, { indexable: !req.query || !Object.keys(req.query).length });
if (req.viewer.kind !== 'user') return page(req, res, 200, null, pages.signedOut(), null, { indexable: !req.query || !Object.keys(req.query).length });
- page(req, res, 200, 'Projects', pages.home({ projects: projects.listFor(req.viewer), csrf: csrfToken(config, req.viewer) }), notice(req));
+ page(req, res, 200, 'Projects', pages.home({ projects: await projects.listFor(req.viewer), csrf: csrfToken(config, req.viewer) }), notice(req));
}));
}));
- router.get('/projects/:id', view((req, res) => {
+ router.get('/projects/:id', view(async (req, res) => {
if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');
if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');
- const project = projects.get(req.params.id);
- const role = access.authorize(project, req.viewer, 'read');
+ const project = await projects.get(req.params.id);
+ const role = await access.authorize(project, req.viewer, 'read');
page(req, res, 200, project.name, pages.project({
page(req, res, 200, project.name, pages.project({
- project, role, sites: sites.listForProject(project.id), quota: projects.quotaOf(project), usage: projects.usageOf(project),
- members: projects.members(project), csrf: csrfToken(config, req.viewer), siteUrl,
- }), takedownNotice(ctx.takedowns.ofProject(project.id)) || notice(req));
+ project, role, sites: await sites.listForProject(project.id), quota: await projects.quotaOf(project), usage: await projects.usageOf(project),
+ members: await projects.members(project), csrf: csrfToken(config, req.viewer), siteUrl,
+ }), takedownNotice(await ctx.takedowns.ofProject(project.id)) || notice(req));
}));
}));
- router.get('/sites/:id', view((req, res) => {
+ router.get('/sites/:id', view(async (req, res) => {
if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');
if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');
- const { site, project, role } = sites.load(req.viewer, req.params.id, 'read');
+ const { site, project, role } = await sites.load(req.viewer, req.params.id, 'read');
page(req, res, 200, site.name, pages.site({
page(req, res, 200, site.name, pages.site({
site, project, role, url: siteUrl(site), csrf: csrfToken(config, req.viewer),
site, project, role, url: siteUrl(site), csrf: csrfToken(config, req.viewer),
- deploys: deploys.list(site.id, 50), activations: deploys.activationsOf(site.id, 10),
- domains: domains.listForSite(site.id).map((d) => ({ domain: d, instructions: domains.instructions(d, site) })),
- }), takedownNotice(ctx.takedowns.ofSite(site)) || notice(req));
+ deploys: await deploys.list(site.id, 50), activations: await deploys.activationsOf(site.id, 10),
+ domains: (await domains.listForSite(site.id)).map((d) => ({ domain: d, instructions: domains.instructions(d, site) })),
+ }), takedownNotice(await ctx.takedowns.ofSite(site)) || notice(req));
}));
}));
- router.get('/deploys/:id', view((req, res) => {
+ router.get('/deploys/:id', view(async (req, res) => {
if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');
if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');
- const { deploy, site, project } = deploys.load(req.viewer, req.params.id, 'read');
+ const { deploy, site, project } = await deploys.load(req.viewer, req.params.id, 'read');
page(req, res, 200, `Deploy ${deploy.id.slice(0, 12)}`, pages.deploy({
page(req, res, 200, `Deploy ${deploy.id.slice(0, 12)}`, pages.deploy({
- deploy, site, project, active: site.active_deploy_id === deploy.id, files: deploys.filesOf(deploy.id), log: deploys.logOf(deploy.id),
+ deploy, site, project, active: site.active_deploy_id === deploy.id, files: await deploys.filesOf(deploy.id), log: await deploys.logOf(deploy.id),
}), notice(req));
}), notice(req));
}));
}));
// ── Actions ─────────────────────────────────────────────
// ── Actions ─────────────────────────────────────────────
- router.post('/projects', action((req) => {
- const p = projects.create(req.viewer, { name: req.body.name, environment: req.body.environment });
+ router.post('/projects', action(async (req) => {
+ const p = await projects.create(req.viewer, { name: req.body.name, environment: req.body.environment });
return back(`/projects/${p.id}`, 'Project created.');
return back(`/projects/${p.id}`, 'Project created.');
}));
}));
- router.post('/projects/:id/sites', action((req) => {
- const project = projects.get(req.params.id);
- access.authorize(project, req.viewer, 'read');
- const s = sites.create(req.viewer, project, { name: req.body.name });
+ router.post('/projects/:id/sites', action(async (req) => {
+ const project = await projects.get(req.params.id);
+ await access.authorize(project, req.viewer, 'read');
+ const s = await sites.create(req.viewer, project, { name: req.body.name });
return back(`/sites/${s.id}`, `Site created: ${siteUrl(s)}`);
return back(`/sites/${s.id}`, `Site created: ${siteUrl(s)}`);
}));
}));
- router.post('/projects/:id/members', action((req) => {
- const project = projects.get(req.params.id);
- access.authorize(project, req.viewer, 'read');
- projects.setMember(req.viewer, project, String(req.body.principal || '').trim(), String(req.body.role || ''));
+ router.post('/projects/:id/members', action(async (req) => {
+ const project = await projects.get(req.params.id);
+ await access.authorize(project, req.viewer, 'read');
+ await projects.setMember(req.viewer, project, String(req.body.principal || '').trim(), String(req.body.role || ''));
return back(`/projects/${project.id}`, 'Member saved.');
return back(`/projects/${project.id}`, 'Member saved.');
}));
}));
- router.post('/projects/:id/members/remove', action((req) => {
- const project = projects.get(req.params.id);
- access.authorize(project, req.viewer, 'read');
- projects.removeMember(req.viewer, project, String(req.body.principal || ''));
+ router.post('/projects/:id/members/remove', action(async (req) => {
+ const project = await projects.get(req.params.id);
+ await access.authorize(project, req.viewer, 'read');
+ await projects.removeMember(req.viewer, project, String(req.body.principal || ''));
return back(`/projects/${project.id}`, 'Member removed.');
return back(`/projects/${project.id}`, 'Member removed.');
}));
}));
- router.post('/projects/:id/delete', action((req) => {
- const project = projects.get(req.params.id);
- access.authorize(project, req.viewer, 'read');
+ router.post('/projects/:id/delete', action(async (req) => {
+ const project = await projects.get(req.params.id);
+ await access.authorize(project, req.viewer, 'read');
if (req.body.confirm !== 'yes') throw new ApiError(422, 'form.confirm', 'tick the confirmation box to delete');
if (req.body.confirm !== 'yes') throw new ApiError(422, 'form.confirm', 'tick the confirmation box to delete');
- projects.remove(req.viewer, project, { sites });
+ await projects.remove(req.viewer, project, { sites });
return back('/', 'Project deleted.');
return back('/', 'Project deleted.');
}));
}));
- router.post('/sites/:id/domains', action((req) => {
- const { domain } = domains.add(req.viewer, req.params.id, { hostname: req.body.hostname });
+ router.post('/sites/:id/domains', action(async (req) => {
+ const { domain } = await domains.add(req.viewer, req.params.id, { hostname: req.body.hostname });
return back(`/sites/${domain.site_id}`, `Added ${domain.hostname}: publish the DNS records below, then check.`);
return back(`/sites/${domain.site_id}`, `Added ${domain.hostname}: publish the DNS records below, then check.`);
}));
}));
- router.post('/sites/:id/rollback', action((req) => {
- const r = deploys.rollback(req.viewer, req.params.id, { expectedActive: req.body.expected_active || null, traceparent: req.ov && req.ov.traceparent });
+ router.post('/sites/:id/rollback', action(async (req) => {
+ const r = await deploys.rollback(req.viewer, req.params.id, { expectedActive: req.body.expected_active || null, traceparent: req.ov && req.ov.traceparent });
return back(`/sites/${req.params.id}`, r.changed ? `Rolled back: ${r.deploy_id} is active.` : 'Nothing changed.');
return back(`/sites/${req.params.id}`, r.changed ? `Rolled back: ${r.deploy_id} is active.` : 'Nothing changed.');
}));
}));
- router.post('/sites/:id/delete', action((req) => {
- const { site } = sites.load(req.viewer, req.params.id, 'maintain');
+ router.post('/sites/:id/delete', action(async (req) => {
+ const { site } = await sites.load(req.viewer, req.params.id, 'maintain');
if (req.body.confirm !== 'yes') throw new ApiError(422, 'form.confirm', 'tick the confirmation box to delete');
if (req.body.confirm !== 'yes') throw new ApiError(422, 'form.confirm', 'tick the confirmation box to delete');
- sites.remove(req.viewer, site.id, { deploys });
+ await sites.remove(req.viewer, site.id, { deploys });
return back(`/projects/${site.project_id}`, `${site.name} deleted.`);
return back(`/projects/${site.project_id}`, `${site.name} deleted.`);
}));
}));
- router.post('/deploys/:id/activate', action((req) => {
- const { site } = deploys.load(req.viewer, req.params.id, 'deploy');
- const r = deploys.activate(req.viewer, req.params.id, { expectedActive: req.body.expected_active || null, traceparent: req.ov && req.ov.traceparent });
+ router.post('/deploys/:id/activate', action(async (req) => {
+ const { site } = await deploys.load(req.viewer, req.params.id, 'deploy');
+ const r = await deploys.activate(req.viewer, req.params.id, { expectedActive: req.body.expected_active || null, traceparent: req.ov && req.ov.traceparent });
return back(`/sites/${site.id}`, r.changed ? `${r.deploy_id} is active.` : 'That deploy was already active.');
return back(`/sites/${site.id}`, r.changed ? `${r.deploy_id} is active.` : 'That deploy was already active.');
}));
}));
- router.post('/deploys/:id/delete', action((req) => {
- const { site } = deploys.load(req.viewer, req.params.id, 'maintain');
- deploys.remove(req.viewer, req.params.id);
+ router.post('/deploys/:id/delete', action(async (req) => {
+ const { site } = await deploys.load(req.viewer, req.params.id, 'maintain');
+ await deploys.remove(req.viewer, req.params.id);
return back(`/sites/${site.id}`, 'Deploy deleted.');
return back(`/sites/${site.id}`, 'Deploy deleted.');
}));
}));
router.post('/domains/:id/verify', action(async (req) => {
router.post('/domains/:id/verify', action(async (req) => {
const d = await domains.verify(req.viewer, req.params.id, { traceparent: req.ov && req.ov.traceparent });
const d = await domains.verify(req.viewer, req.params.id, { traceparent: req.ov && req.ov.traceparent });
return back(`/sites/${d.site_id}`, d.status === 'verified' ? `${d.hostname} is verified and served.` : `${d.hostname}: ${d.last_error || d.status}`, d.status === 'verified' ? null : 'error');
return back(`/sites/${d.site_id}`, d.status === 'verified' ? `${d.hostname} is verified and served.` : `${d.hostname}: ${d.last_error || d.status}`, d.status === 'verified' ? null : 'error');
}));
}));
- router.post('/domains/:id/delete', action((req) => {
- const { domain } = domains.load(req.viewer, req.params.id, 'maintain');
- domains.remove(req.viewer, domain.id);
+ router.post('/domains/:id/delete', action(async (req) => {
+ const { domain } = await domains.load(req.viewer, req.params.id, 'maintain');
+ await domains.remove(req.viewer, domain.id);
return back(`/sites/${domain.site_id}`, `${domain.hostname} removed.`);
return back(`/sites/${domain.site_id}`, `${domain.hostname} removed.`);
}));
}));
@@ -171,7 +171,7 @@ function createDashboard(ctx) {
@@ -171,7 +171,7 @@ function createDashboard(ctx) {
try {
try {
if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');
if (req.viewer.kind !== 'user') throw new ApiError(401, 'auth.required', 'sign in first');
if (!sameOrigin(config, req)) throw new ApiError(403, 'form.invalid', 'this form did not come from this dashboard');
if (!sameOrigin(config, req)) throw new ApiError(403, 'form.invalid', 'this form did not come from this dashboard');
- const pre = deploys.precheck(req.viewer, req.params.id);
+ const pre = await deploys.precheck(req.viewer, req.params.id);
release = ctx.uploadGate.enter();
release = ctx.uploadGate.enter();
if (!release) { res.set('Retry-After', '30'); throw new ApiError(503, 'upload.busy', 'Host is validating other uploads right now; try again in 30 seconds'); }
if (!release) { res.set('Retry-After', '30'); throw new ApiError(503, 'upload.busy', 'Host is validating other uploads right now; try again in 30 seconds'); }
let up;
let up;
@@ -181,12 +181,12 @@ function createDashboard(ctx) {
@@ -181,12 +181,12 @@ function createDashboard(ctx) {
if (!(err instanceof UploadError)) throw err;
if (!(err instanceof UploadError)) throw err;
if (err.closeConnection) res.set('Connection', 'close');
if (err.closeConnection) res.set('Connection', 'close');
if (!checkCsrf(config, req.viewer, err.fields && err.fields.csrf) && !err.closeConnection) throw new ApiError(403, 'form.invalid', 'this form expired; reload the page and try again');
if (!checkCsrf(config, req.viewer, err.fields && err.fields.csrf) && !err.closeConnection) throw new ApiError(403, 'form.invalid', 'this form expired; reload the page and try again');
- const r = deploys.recordFailure(req.viewer, pre, { source: err.source || 'files', code: err.code, problems: [{ code: err.code, message: err.message }], traceparent: req.ov && req.ov.traceparent });
+ const r = await deploys.recordFailure(req.viewer, pre, { source: err.source || 'files', code: err.code, problems: [{ code: err.code, message: err.message }], traceparent: req.ov && req.ov.traceparent });
return res.redirect(303, back(`/deploys/${r.deploy.id}`, `Upload refused: ${err.message}`, 'error'));
return res.redirect(303, back(`/deploys/${r.deploy.id}`, `Upload refused: ${err.message}`, 'error'));
}
}
if (!checkCsrf(config, req.viewer, up.fields.csrf)) throw new ApiError(403, 'form.invalid', 'this form expired; reload the page and try again');
if (!checkCsrf(config, req.viewer, up.fields.csrf)) throw new ApiError(403, 'form.invalid', 'this form expired; reload the page and try again');
try {
try {
- const r = deploys.create(req.viewer, pre, up.entries, { source: up.source, activate: up.fields.activate === '1', notes: up.notes, traceparent: req.ov && req.ov.traceparent });
+ const r = await deploys.create(req.viewer, pre, up.entries, { source: up.source, activate: up.fields.activate === '1', notes: up.notes, traceparent: req.ov && req.ov.traceparent });
return res.redirect(303, back(`/deploys/${r.deploy.id}`, r.activated && r.activated.changed ? 'Deployed and activated.' : 'Deployed. Activate it from the site page.'));
return res.redirect(303, back(`/deploys/${r.deploy.id}`, r.activated && r.activated.changed ? 'Deployed and activated.' : 'Deployed. Activate it from the site page.'));
} catch (err) {
} catch (err) {
if (err instanceof ApiError && err.extra && err.extra.deploy_id) return res.redirect(303, back(`/deploys/${err.extra.deploy_id}`, `Upload refused: ${err.message}`, 'error'));
if (err instanceof ApiError && err.extra && err.extra.deploy_id) return res.redirect(303, back(`/deploys/${err.extra.deploy_id}`, `Upload refused: ${err.message}`, 'error'));
@@ -89,18 +89,18 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons
@@ -89,18 +89,18 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons
/**
/**
* -> { kind: 'dashboard' } | { kind: 'site', site } | { kind: 'unknown', host }
* -> { kind: 'dashboard' } | { kind: 'site', site } | { kind: 'unknown', host }
*/
*/
- function resolve(rawHost) {
+ async function resolve(rawHost) {
const host = normaliseHost(rawHost);
const host = normaliseHost(rawHost);
if (!host) return { kind: 'unknown', host: null };
if (!host) return { kind: 'unknown', host: null };
if (host === config.dashboardHost || LOOPBACK_HOSTS.has(host)) return { kind: 'dashboard' };
if (host === config.dashboardHost || LOOPBACK_HOSTS.has(host)) return { kind: 'dashboard' };
if (host.endsWith(suffix)) {
if (host.endsWith(suffix)) {
const label = host.slice(0, -suffix.length);
const label = host.slice(0, -suffix.length);
if (!label || label.includes('.')) return { kind: 'unknown', host };
if (!label || label.includes('.')) return { kind: 'unknown', host };
- const site = q.siteByName.get(label);
+ const site = await q.siteByName.get(label);
return site ? { kind: 'site', site, host } : { kind: 'unknown', host };
return site ? { kind: 'site', site, host } : { kind: 'unknown', host };
}
}
if (!isHostname(host)) return { kind: 'unknown', host };
if (!isHostname(host)) return { kind: 'unknown', host };
- const site = q.siteByCustom.get(host);
+ const site = await q.siteByCustom.get(host);
return site ? { kind: 'site', site, host } : { kind: 'unknown', host };
return site ? { kind: 'site', site, host } : { kind: 'unknown', host };
}
}
@@ -160,8 +160,8 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons
@@ -160,8 +160,8 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons
stream.pipe(res);
stream.pipe(res);
}
}
- function notFound(req, res, site, reason) {
- const page = site.active_deploy_id ? q.file.get(site.active_deploy_id, '404.html') : null;
+ async function notFound(req, res, site, reason) {
+ const page = site.active_deploy_id ? await q.file.get(site.active_deploy_id, '404.html') : null;
if (page && page.project_id === site.project_id) return send(req, res, site, page, { status: 404, cache: 'no-cache' });
if (page && page.project_id === site.project_id) return send(req, res, site, page, { status: 404, cache: 'no-cache' });
res.statusCode = 404;
res.statusCode = 404;
res.setHeader('Content-Type', 'text/html; charset=utf-8');
res.setHeader('Content-Type', 'text/html; charset=utf-8');
@@ -174,13 +174,13 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons
@@ -174,13 +174,13 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons
}
}
/** Serves one request for a resolved site. */
/** Serves one request for a resolved site. */
- function handle(req, res, site) {
+ async function handle(req, res, site) {
baseHeaders(res, { sandbox: site.environment === 'sandbox' });
baseHeaders(res, { sandbox: site.environment === 'sandbox' });
if (req.method !== 'GET' && req.method !== 'HEAD') {
if (req.method !== 'GET' && req.method !== 'HEAD') {
return plain(res, 405, 'Static site: only GET and HEAD are supported.', { Allow: 'GET, HEAD' });
return plain(res, 405, 'Static site: only GET and HEAD are supported.', { Allow: 'GET, HEAD' });
}
}
// Taken down by staff: nothing of the tenant's is served, on any of its hosts, until lifted.
// Taken down by staff: nothing of the tenant's is served, on any of its hosts, until lifted.
- if (takedowns && takedowns.ofSite(site)) {
+ if (takedowns && await takedowns.ofSite(site)) {
// Clear-Site-Data asks the browser to drop what the site left behind for this visitor
// Clear-Site-Data asks the browser to drop what the site left behind for this visitor
// (caches, storage and service workers a phishing page may have installed).
// (caches, storage and service workers a phishing page may have installed).
return plain(res, 451, 'This site is unavailable: OpenVibe.Host staff took it down after a report.', { 'Clear-Site-Data': '"cache", "storage"' });
return plain(res, 451, 'This site is unavailable: OpenVibe.Host staff took it down after a report.', { 'Clear-Site-Data': '"cache", "storage"' });
@@ -196,12 +196,12 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons
@@ -196,12 +196,12 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons
const q0 = target.indexOf('?');
const q0 = target.indexOf('?');
const rawPath = q0 >= 0 ? target.slice(0, q0) : target;
const rawPath = q0 >= 0 ? target.slice(0, q0) : target;
const search = q0 >= 0 ? target.slice(q0) : '';
const search = q0 >= 0 ? target.slice(q0) : '';
- if (!site.active_deploy_id) return notFound(req, res, site, 'no-deploy');
+ if (!site.active_deploy_id) return await notFound(req, res, site, 'no-deploy');
const c = candidates(rawPath);
const c = candidates(rawPath);
- if (!c) return notFound(req, res, site, 'invalid');
+ if (!c) return await notFound(req, res, site, 'invalid');
const deployId = site.active_deploy_id;
const deployId = site.active_deploy_id;
for (const p of c.list) {
for (const p of c.list) {
- const row = q.file.get(deployId, p);
+ const row = await q.file.get(deployId, p);
if (row && row.project_id === site.project_id) {
if (row && row.project_id === site.project_id) {
const hashed = isHashedAsset(p);
const hashed = isHashedAsset(p);
// Browsers keep fingerprinted assets for a year; a shared CDN in front of the tenant
// Browsers keep fingerprinted assets for a year; a shared CDN in front of the tenant
@@ -211,14 +211,14 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons
@@ -211,14 +211,14 @@ function createTenantServer({ store, config, blobs, takedowns = null, log = cons
return send(req, res, site, row, { cache: hashed ? 'public, max-age=31536000, immutable' : 'public, max-age=0, must-revalidate' });
return send(req, res, site, row, { cache: hashed ? 'public, max-age=31536000, immutable' : 'public, max-age=0, must-revalidate' });
}
}
}
}
- if (!c.dir && c.dirIndex && q.file.get(deployId, c.dirIndex)) {
+ if (!c.dir && c.dirIndex && await q.file.get(deployId, c.dirIndex)) {
// /docs → /docs/ so relative links inside docs/index.html resolve; built from the raw path.
// /docs → /docs/ so relative links inside docs/index.html resolve; built from the raw path.
res.statusCode = 301;
res.statusCode = 301;
res.setHeader('Location', `${rawPath}/${search}`);
res.setHeader('Location', `${rawPath}/${search}`);
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Cache-Control', 'no-cache');
return res.end();
return res.end();
}
}
- return notFound(req, res, site, 'missing');
+ return await notFound(req, res, site, 'missing');
}
}
function unknownHost(req, res) {
function unknownHost(req, res) {
@@ -9,7 +9,8 @@
@@ -9,7 +9,8 @@
*/
*/
const { createApp } = require('./app');
const { createApp } = require('./app');
-const { app, ctx } = createApp();
+(async () => {
+const { app, ctx } = await createApp();
const { config } = ctx;
const { config } = ctx;
const server = app.listen(config.port, config.host, () => {
const server = app.listen(config.port, config.host, () => {
@@ -27,10 +28,11 @@ function shutdown(signal) {
@@ -27,10 +28,11 @@ function shutdown(signal) {
server.close(async () => {
server.close(async () => {
try { await ctx.outbox.stop(); } catch { /* best effort */ }
try { await ctx.outbox.stop(); } catch { /* best effort */ }
try { ctx.stopMetrics(); } catch { /* best effort */ }
try { ctx.stopMetrics(); } catch { /* best effort */ }
- try { ctx.store.close(); } catch { /* already closed */ }
+ try { await ctx.store.close(); } catch { /* already closed */ }
process.exit(0);
process.exit(0);
});
});
setTimeout(() => process.exit(0), 5000).unref();
setTimeout(() => process.exit(0), 5000).unref();
}
}
process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));
process.on('SIGINT', () => shutdown('SIGINT'));
+})().catch((err) => { console.error('[Host] failed to start:', err); process.exit(1); });
@@ -12,7 +12,7 @@
@@ -12,7 +12,7 @@
const { createReadiness } = require('openvibe-shared/ready');
const { createReadiness } = require('openvibe-shared/ready');
const { CHARTER_TABLES } = require('./db');
const { CHARTER_TABLES } = require('./db');
-function createHostReadiness({ store, blobs, auth, outbox, release = null, minFreeBytes = () => 0 }) {
+function createHostReadiness({ store, blobs, auth, outbox, release = null, minFreeBytes = () => 0, valkey = null }) {
const { db } = store;
const { db } = store;
return createReadiness({
return createReadiness({
service: 'host',
service: 'host',
@@ -20,8 +20,8 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFr
@@ -20,8 +20,8 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFr
checks: [
checks: [
{
{
name: 'db', required: true,
name: 'db', required: true,
- check: () => {
- const names = new Set(db.prepare("SELECT name FROM sqlite_master WHERE type = 'table'").all().map((r) => r.name));
+ check: async () => {
+ const names = new Set((await db.prepare("SELECT table_name AS name FROM information_schema.tables WHERE table_schema = current_schema()").all()).map((r) => r.name));
const missing = CHARTER_TABLES.filter((t) => !names.has(t));
const missing = CHARTER_TABLES.filter((t) => !names.has(t));
return missing.length ? `missing ${missing.join(', ')}` : true;
return missing.length ? `missing ${missing.join(', ')}` : true;
},
},
@@ -36,6 +36,7 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFr
@@ -36,6 +36,7 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFr
return free >= floor ? { ok: true, detail: { free_bytes: free } } : `${free} bytes free, below HOST_MIN_FREE_BYTES (${floor}): new deploys are refused`;
return free >= floor ? { ok: true, detail: { free_bytes: free } } : `${free} bytes free, below HOST_MIN_FREE_BYTES (${floor}): new deploys are refused`;
},
},
},
},
+ { name: 'valkey', required: false, check: async () => (valkey ? valkey.ready() : { skipped: 'VALKEY_URL not set: per-actor limits count in this process only' }) },
{
{
name: 'network_jwks', required: false,
name: 'network_jwks', required: false,
check: () => {
check: () => {
@@ -46,8 +47,8 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFr
@@ -46,8 +47,8 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFr
},
},
{
{
name: 'events_relay', required: false,
name: 'events_relay', required: false,
- check: () => {
- const s = outbox.status();
+ check: async () => {
+ const s = await outbox.status();
if (!s.enabled) return `relay off (Events URL or service credentials not configured); ${s.pending} events waiting`;
if (!s.enabled) return `relay off (Events URL or service credentials not configured); ${s.pending} events waiting`;
if (s.rejected) return `${s.rejected} events rejected by OpenVibe.Events`;
if (s.rejected) return `${s.rejected} events rejected by OpenVibe.Events`;
return { ok: true, detail: { pending: s.pending } };
return { ok: true, detail: { pending: s.pending } };
@@ -55,8 +56,8 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFr
@@ -55,8 +56,8 @@ function createHostReadiness({ store, blobs, auth, outbox, release = null, minFr
},
},
{
{
name: 'domain_checks', required: false,
name: 'domain_checks', required: false,
- check: () => {
- const stale = db.prepare("SELECT COUNT(*) AS n FROM host_domains WHERE kind = 'custom' AND status = 'pending' AND COALESCE(last_checked_at, created_at) < ?").get(store.now() - 3600 * 1000).n;
+ check: async () => {
+ const stale = (await db.prepare("SELECT COUNT(*) AS n FROM host_domains WHERE kind = 'custom' AND status = 'pending' AND COALESCE(last_checked_at, created_at) < ?").get(store.now() - 3600 * 1000)).n;
return stale ? `${stale} pending custom domains not checked in the last hour (is the worker on?)` : true;
return stale ? `${stale} pending custom domains not checked in the last hour (is the worker on?)` : true;
},
},
},
},
@@ -29,11 +29,11 @@ function createWorker({ config, store, domains, blobs, outbox, log = console })
@@ -29,11 +29,11 @@ function createWorker({ config, store, domains, blobs, outbox, log = console })
}
}
/** Remove objects no row references, once they are older than `graceMs`. */
/** Remove objects no row references, once they are older than `graceMs`. */
- function sweepObjects({ graceMs = 3600 * 1000 } = {}) {
+ async function sweepObjects({ graceMs = 3600 * 1000 } = {}) {
const known = store.db.prepare('SELECT 1 FROM host_blobs WHERE project_id = ? AND sha256 = ?');
const known = store.db.prepare('SELECT 1 FROM host_blobs WHERE project_id = ? AND sha256 = ?');
let removed = 0;
let removed = 0;
for (const o of blobs.list()) {
for (const o of blobs.list()) {
- if (known.get(o.projectId, o.sha256)) continue;
+ if (await known.get(o.projectId, o.sha256)) continue;
try {
try {
if (Date.now() - fs.statSync(o.file).mtimeMs < graceMs) continue;
if (Date.now() - fs.statSync(o.file).mtimeMs < graceMs) continue;
fs.unlinkSync(o.file);
fs.unlinkSync(o.file);
@@ -51,8 +51,8 @@ function createWorker({ config, store, domains, blobs, outbox, log = console })
@@ -51,8 +51,8 @@ function createWorker({ config, store, domains, blobs, outbox, log = console })
if (!config.worker.enabled) return;
if (!config.worker.enabled) return;
const every = (ms, fn) => { const t = setInterval(fn, ms); t.unref(); timers.push(t); };
const every = (ms, fn) => { const t = setInterval(fn, ms); t.unref(); timers.push(t); };
every(config.domains.recheckIntervalMs, domainTick);
every(config.domains.recheckIntervalMs, domainTick);
- every(3600 * 1000, () => { try { sweepObjects(); } catch (err) { log.warn('[Host] object sweep failed:', err.message); } });
- every(24 * 3600 * 1000, () => { try { outbox.outbox.prune(); } catch (err) { log.warn('[Host] outbox prune failed:', err.message); } });
+ every(3600 * 1000, async () => { try { await sweepObjects(); } catch (err) { log.warn('[Host] object sweep failed:', err.message); } });
+ every(24 * 3600 * 1000, async () => { try { await outbox.outbox.prune(); } catch (err) { log.warn('[Host] outbox prune failed:', err.message); } });
setTimeout(domainTick, 5000).unref();
setTimeout(domainTick, 5000).unref();
},
},
stop() { for (const t of timers) clearInterval(t); timers.length = 0; },
stop() { for (const t of timers) clearInterval(t); timers.length = 0; },
@@ -149,7 +149,7 @@ async function runProof(host, args) {
@@ -149,7 +149,7 @@ async function runProof(host, args) {
for (const t of ['ovhost', 'systemctl', 'runuser']) { fs.writeFileSync(path.join(bin, t), STUB); fs.chmodSync(path.join(bin, t), 0o755); }
for (const t of ['ovhost', 'systemctl', 'runuser']) { fs.writeFileSync(path.join(bin, t), STUB); fs.chmodSync(path.join(bin, t), 0o755); }
fs.symlinkSync(process.execPath, path.join(bin, 'node'));
fs.symlinkSync(process.execPath, path.join(bin, 'node'));
const state = path.join(dir, 'd41');
const state = path.join(dir, 'd41');
- return withRpc(host, (rpc) => new Promise((resolve) => {
+ return await withRpc(host, (rpc) => new Promise((resolve) => {
const child = spawn('bash', [SCRIPT, ...args], {
const child = spawn('bash', [SCRIPT, ...args], {
env: {
env: {
PATH: `${bin}:${process.env.PATH}`, HOME: dir, D41_RPC: rpc, D41_REQUIRE_ROOT: '0', D41_STATE_DIR: state,
PATH: `${bin}:${process.env.PATH}`, HOME: dir, D41_RPC: rpc, D41_REQUIRE_ROOT: '0', D41_STATE_DIR: state,
new file mode 100644
new file mode 100644
@@ -0,0 +1,11 @@
@@ -0,0 +1,11 @@
+'use strict';
+/**
+ * A migrated database for one test run (ADR-035), from openvibe-sdk/testing: PGlite by default; with
+ * HOST_TEST_STORE=pg (npm run test:pg) the PostgreSQL + PgBouncer containers, with roles and a schema of this run's own.
+ */
+const { createTestDb, pgAvailable } = require('openvibe-sdk/testing');
+const { MIGRATIONS } = require('../../server/db');
+
+const testDb = ({ store = process.env.HOST_TEST_STORE || 'pglite', max = 4 } = {}) => createTestDb({ migrations: MIGRATIONS, store, service: 'host', max });
+
+module.exports = { testDb, pgAvailable };
@@ -97,7 +97,7 @@ const { site: siteTar } = require('./stageb/tar');
@@ -97,7 +97,7 @@ const { site: siteTar } = require('./stageb/tar');
assert.strictEqual((await t.get('alpha.openvibe.host', '/')).text, 'ALPHA TWO');
assert.strictEqual((await t.get('alpha.openvibe.host', '/')).text, 'ALPHA TWO');
assert.strictEqual((await t.get('www.alpha-example.org', '/')).text, 'ALPHA TWO');
assert.strictEqual((await t.get('www.alpha-example.org', '/')).text, 'ALPHA TWO');
assert.strictEqual((await t.api('DELETE', `/api/v1/sites/${alpha.id}/takedown`, { as: staff })).status, 404);
assert.strictEqual((await t.api('DELETE', `/api/v1/sites/${alpha.id}/takedown`, { as: staff })).status, 404);
- const rows = t.ctx.store.db.prepare('SELECT target_kind, reason, lifted_at IS NOT NULL AS lifted, lift_note FROM host_takedowns').all();
+ const rows = await t.ctx.store.db.prepare('SELECT target_kind, reason, (lifted_at IS NOT NULL)::int AS lifted, lift_note FROM host_takedowns').all();
assert.deepStrictEqual(rows, [{ target_kind: 'site', reason: 'phishing report #42', lifted: 1, lift_note: 'reviewed: false positive' }], 'the history is kept');
assert.deepStrictEqual(rows, [{ target_kind: 'site', reason: 'phishing report #42', lifted: 1, lift_note: 'reviewed: false positive' }], 'the history is kept');
});
});
@@ -134,7 +134,7 @@ const { site: siteTar } = require('./stageb/tar');
@@ -134,7 +134,7 @@ const { site: siteTar } = require('./stageb/tar');
const pc = await t.project(alice, 'C');
const pc = await t.project(alice, 'C');
const gamma = await t.site(alice, pc.id, 'gamma');
const gamma = await t.site(alice, pc.id, 'gamma');
const body = siteTar({ 'index.html': 'GAMMA' });
const body = siteTar({ 'index.html': 'GAMMA' });
- const failedBefore = t.ctx.store.db.prepare("SELECT COUNT(*) AS n FROM host_deploys WHERE state = 'failed'").get().n;
+ const failedBefore = (await t.ctx.store.db.prepare("SELECT COUNT(*) AS n FROM host_deploys WHERE state = 'failed'").get()).n;
// The first upload sends its headers and half its body, then waits.
// The first upload sends its headers and half its body, then waits.
let firstResponse;
let firstResponse;
let finish;
let finish;
@@ -157,7 +157,7 @@ const { site: siteTar } = require('./stageb/tar');
@@ -157,7 +157,7 @@ const { site: siteTar } = require('./stageb/tar');
await first;
await first;
assert.strictEqual(firstResponse.status, 201, firstResponse.text);
assert.strictEqual(firstResponse.status, 201, firstResponse.text);
assert.strictEqual(t.ctx.uploadGate.inFlight, 0, 'the slot is released');
assert.strictEqual(t.ctx.uploadGate.inFlight, 0, 'the slot is released');
- assert.strictEqual(t.ctx.store.db.prepare("SELECT COUNT(*) AS n FROM host_deploys WHERE state = 'failed'").get().n, failedBefore, 'a busy refusal is not the tenant\'s failed deploy');
+ assert.strictEqual((await t.ctx.store.db.prepare("SELECT COUNT(*) AS n FROM host_deploys WHERE state = 'failed'").get()).n, failedBefore, 'a busy refusal is not the tenant\'s failed deploy');
const third = await t.upload(alice, gamma.id, { 'index.html': 'THIRD' });
const third = await t.upload(alice, gamma.id, { 'index.html': 'THIRD' });
assert.strictEqual(third.status, 201, third.text);
assert.strictEqual(third.status, 201, third.text);
// A refused upload releases its slot too.
// A refused upload releases its slot too.
@@ -79,7 +79,7 @@ const DIR = path.join(__dirname, '..', 'docs', 'capabilities-proposal');
@@ -79,7 +79,7 @@ const DIR = path.join(__dirname, '..', 'docs', 'capabilities-proposal');
await t.ctx.outbox.outbox.flush();
await t.ctx.outbox.outbox.flush();
assert.deepStrictEqual(received.map((e) => e.event_type).sort(), ['host.deploy.activated', 'host.deploy.created', 'host.deploy.failed']);
assert.deepStrictEqual(received.map((e) => e.event_type).sort(), ['host.deploy.activated', 'host.deploy.created', 'host.deploy.failed']);
assert.ok(received.every((e) => e.source === 'host'));
assert.ok(received.every((e) => e.source === 'host'));
- assert.strictEqual(t.ctx.outbox.status().pending, 0);
+ assert.strictEqual((await t.ctx.outbox.status()).pending, 0);
await t.close();
await t.close();
await events.close();
await events.close();
});
});
@@ -63,7 +63,7 @@ const DAY = 24 * 3600 * 1000;
@@ -63,7 +63,7 @@ const DAY = 24 * 3600 * 1000;
const r = await t.api('POST', `/api/v1/domains/${dom.id}/verify`, { as: alice });
const r = await t.api('POST', `/api/v1/domains/${dom.id}/verify`, { as: alice });
assert.strictEqual(r.json().domain.status, 'verified');
assert.strictEqual(r.json().domain.status, 'verified');
assert.strictEqual(r.json().domain.served, true);
assert.strictEqual(r.json().domain.served, true);
- const ev = t.events('host.domain.verified');
+ const ev = await t.events('host.domain.verified');
assert.strictEqual(ev.length, 1);
assert.strictEqual(ev.length, 1);
assert.deepStrictEqual(ev[0].payload, { project_id: pa.id, site_id: sa.id, site: 'alice-site', hostname: HOST });
assert.deepStrictEqual(ev[0].payload, { project_id: pa.id, site_id: sa.id, site: 'alice-site', hostname: HOST });
const g = await t.get(HOST, '/');
const g = await t.get(HOST, '/');
@@ -115,7 +115,7 @@ const DAY = 24 * 3600 * 1000;
@@ -115,7 +115,7 @@ const DAY = 24 * 3600 * 1000;
const never = (await t.api('POST', `/api/v1/sites/${sb.id}/domains`, { as: bob, json: { hostname: 'never.example.org' } })).json().domain;
const never = (await t.api('POST', `/api/v1/sites/${sb.id}/domains`, { as: bob, json: { hostname: 'never.example.org' } })).json().domain;
t.clock.advance(8 * DAY);
t.clock.advance(8 * DAY);
await t.ctx.worker.domainTick();
await t.ctx.worker.domainTick();
- const after = t.ctx.domains.get(never.id);
+ const after = await t.ctx.domains.get(never.id);
assert.strictEqual(after.status, 'failed');
assert.strictEqual(after.status, 'failed');
});
});
@@ -123,10 +123,10 @@ const DAY = 24 * 3600 * 1000;
@@ -123,10 +123,10 @@ const DAY = 24 * 3600 * 1000;
t.dns.delete(`_openvibe-host.${HOST}`);
t.dns.delete(`_openvibe-host.${HOST}`);
t.clock.advance(DAY + 1000);
t.clock.advance(DAY + 1000);
await t.ctx.worker.domainTick();
await t.ctx.worker.domainTick();
- assert.strictEqual(t.ctx.domains.get(dom.id).status, 'verified', 'still served during the grace period');
- assert.ok(t.ctx.domains.get(dom.id).record_missing_since);
+ assert.strictEqual((await t.ctx.domains.get(dom.id)).status, 'verified', 'still served during the grace period');
+ assert.ok((await t.ctx.domains.get(dom.id)).record_missing_since);
for (let i = 0; i < 8; i++) { t.clock.advance(DAY + 1000); await t.ctx.worker.domainTick(); }
for (let i = 0; i < 8; i++) { t.clock.advance(DAY + 1000); await t.ctx.worker.domainTick(); }
- assert.strictEqual(t.ctx.domains.get(dom.id).status, 'lapsed');
+ assert.strictEqual((await t.ctx.domains.get(dom.id)).status, 'lapsed');
assert.strictEqual((await t.get(HOST, '/')).status, 404);
assert.strictEqual((await t.get(HOST, '/')).status, 404);
});
});
@@ -142,8 +142,8 @@ const { boot, check, done } = require('./stageb/boot');
@@ -142,8 +142,8 @@ const { boot, check, done } = require('./stageb/boot');
await check('events: created/activated in the outbox with valid envelopes and no file contents', async () => {
await check('events: created/activated in the outbox with valid envelopes and no file contents', async () => {
const contracts = require('openvibe-contracts');
const contracts = require('openvibe-contracts');
- const created = t.events('host.deploy.created');
- const activated = t.events('host.deploy.activated');
+ const created = await t.events('host.deploy.created');
+ const activated = await t.events('host.deploy.activated');
assert.ok(created.length >= 4);
assert.ok(created.length >= 4);
assert.ok(activated.some((e) => e.payload.rollback === true));
assert.ok(activated.some((e) => e.payload.rollback === true));
for (const e of [...created, ...activated]) {
for (const e of [...created, ...activated]) {
@@ -163,7 +163,7 @@ const DAY = 24 * 3600 * 1000;
@@ -163,7 +163,7 @@ const DAY = 24 * 3600 * 1000;
const up = await t.api('POST', `/api/v1/sites/${site.id}/deploys?activate=1`, { as: ci, body: siteTar({ 'index.html': 'from CI' }), headers: { 'content-type': 'application/gzip' } });
const up = await t.api('POST', `/api/v1/sites/${site.id}/deploys?activate=1`, { as: ci, body: siteTar({ 'index.html': 'from CI' }), headers: { 'content-type': 'application/gzip' } });
assert.strictEqual(up.status, 201, up.text);
assert.strictEqual(up.status, 201, up.text);
assert.strictEqual(up.json().deploy.created_by, `app:${appId}`);
assert.strictEqual(up.json().deploy.created_by, `app:${appId}`);
- assert.deepStrictEqual(t.events('host.deploy.activated').pop().actor, { type: 'app', id: appId });
+ assert.deepStrictEqual((await t.events('host.deploy.activated')).pop().actor, { type: 'app', id: appId });
assert.strictEqual((await t.get('quota-site.openvibe.host', '/')).text, 'from CI');
assert.strictEqual((await t.get('quota-site.openvibe.host', '/')).text, 'from CI');
const mk = await t.api('POST', `/api/v1/projects/${project.id}/sites`, { as: ci, json: { name: 'ci-site' } });
const mk = await t.api('POST', `/api/v1/projects/${project.id}/sites`, { as: ci, json: { name: 'ci-site' } });
assert.strictEqual(mk.status, 403);
assert.strictEqual(mk.status, 403);
@@ -49,15 +49,15 @@ const { boot, check, done } = require('./stageb/boot');
@@ -49,15 +49,15 @@ const { boot, check, done } = require('./stageb/boot');
await check('a failure inside the switch leaves the pointer, history and outbox untouched', async () => {
await check('a failure inside the switch leaves the pointer, history and outbox untouched', async () => {
await t.api('POST', `/api/v1/deploys/${v1.id}/activate`, { as: alice, json: {} });
await t.api('POST', `/api/v1/deploys/${v1.id}/activate`, { as: alice, json: {} });
const db = t.ctx.store.db;
const db = t.ctx.store.db;
- const before = { active: db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id).active_deploy_id, acts: db.prepare('SELECT COUNT(*) AS n FROM host_activations').get().n, events: db.prepare('SELECT COUNT(*) AS n FROM event_outbox').get().n };
+ const before = { active: (await db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id)).active_deploy_id, acts: (await db.prepare('SELECT COUNT(*) AS n FROM host_activations').get()).n, events: (await db.prepare('SELECT COUNT(*) AS n FROM event_outbox').get()).n };
const emit = t.ctx.outbox.emit;
const emit = t.ctx.outbox.emit;
t.ctx.outbox.emit = () => { throw new Error('simulated crash after the pointer moved'); };
t.ctx.outbox.emit = () => { throw new Error('simulated crash after the pointer moved'); };
const r = await t.api('POST', `/api/v1/deploys/${v2.id}/activate`, { as: alice, json: {} });
const r = await t.api('POST', `/api/v1/deploys/${v2.id}/activate`, { as: alice, json: {} });
t.ctx.outbox.emit = emit;
t.ctx.outbox.emit = emit;
assert.strictEqual(r.status, 500);
assert.strictEqual(r.status, 500);
- assert.strictEqual(db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id).active_deploy_id, before.active);
- assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM host_activations').get().n, before.acts);
- assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM event_outbox').get().n, before.events);
+ assert.strictEqual((await db.prepare('SELECT active_deploy_id FROM host_sites WHERE id = ?').get(site.id)).active_deploy_id, before.active);
+ assert.strictEqual((await db.prepare('SELECT COUNT(*) AS n FROM host_activations').get()).n, before.acts);
+ assert.strictEqual((await db.prepare('SELECT COUNT(*) AS n FROM event_outbox').get()).n, before.events);
assert.strictEqual((await t.get('flip.openvibe.host', '/')).text, 'version-1');
assert.strictEqual((await t.get('flip.openvibe.host', '/')).text, 'version-1');
});
});
@@ -86,12 +86,12 @@ const { boot, check, done } = require('./stageb/boot');
@@ -86,12 +86,12 @@ const { boot, check, done } = require('./stageb/boot');
await check('artifacts are immutable in the database itself', async () => {
await check('artifacts are immutable in the database itself', async () => {
const db = t.ctx.store.db;
const db = t.ctx.store.db;
- assert.throws(() => db.prepare("UPDATE host_deploys SET manifest = '{}' WHERE id = ?").run(v1.id), /immutable/);
- assert.throws(() => db.prepare('UPDATE host_deploys SET total_bytes = 1 WHERE id = ?').run(v1.id), /immutable/);
- assert.throws(() => db.prepare("UPDATE host_deploy_files SET sha256 = 'x' WHERE deploy_id = ?").run(v1.id), /immutable/);
- assert.throws(() => db.prepare('DELETE FROM host_deploy_files WHERE deploy_id = ?').run(v1.id), /live deploy/);
- db.prepare("UPDATE host_deploys SET state = 'deleted' WHERE id = (SELECT id FROM host_deploys WHERE state = 'failed' LIMIT 1)").run();
- assert.throws(() => db.prepare("UPDATE host_deploys SET state = 'ready' WHERE state = 'deleted'").run(), /cannot become ready/);
+ await assert.rejects(db.prepare("UPDATE host_deploys SET manifest = '{}' WHERE id = ?").run(v1.id), /immutable/);
+ await assert.rejects(db.prepare('UPDATE host_deploys SET total_bytes = 1 WHERE id = ?').run(v1.id), /immutable/);
+ await assert.rejects(db.prepare("UPDATE host_deploy_files SET sha256 = 'x' WHERE deploy_id = ?").run(v1.id), /immutable/);
+ await assert.rejects(db.prepare('DELETE FROM host_deploy_files WHERE deploy_id = ?').run(v1.id), /live deploy/);
+ await db.prepare("UPDATE host_deploys SET state = 'deleted' WHERE id = (SELECT id FROM host_deploys WHERE state = 'failed' LIMIT 1)").run();
+ await assert.rejects(db.prepare("UPDATE host_deploys SET state = 'ready' WHERE state = 'deleted'").run(), /cannot become ready/);
});
});
await check('the active pointer survives a restart', async () => {
await check('the active pointer survives a restart', async () => {
@@ -42,7 +42,7 @@ const ORIGIN = 'https://openvibe.host';
@@ -42,7 +42,7 @@ const ORIGIN = 'https://openvibe.host';
}
}
const deployKeys = Object.keys(d).sort();
const deployKeys = Object.keys(d).sort();
assert.deepStrictEqual(deployKeys, ['active', 'created_at', 'created_by', 'failure_code', 'file_count', 'id', 'log', 'manifest_sha256', 'new_bytes', 'project_id', 'site_id', 'source', 'state', 'total_bytes'].sort());
assert.deepStrictEqual(deployKeys, ['active', 'created_at', 'created_by', 'failure_code', 'file_count', 'id', 'log', 'manifest_sha256', 'new_bytes', 'project_id', 'site_id', 'source', 'state', 'total_bytes'].sort());
- const events = JSON.stringify(t.events());
+ const events = JSON.stringify(await t.events());
assert.ok(!events.includes('sk_live') && !events.includes(SECRET));
assert.ok(!events.includes('sk_live') && !events.includes(SECRET));
});
});
@@ -22,7 +22,7 @@ const { tarball } = require('./stageb/tar');
@@ -22,7 +22,7 @@ const { tarball } = require('./stageb/tar');
let baseline = countObjects();
let baseline = countObjects();
async function refused(entries, code, { gzip = true, raw = null, contentType } = {}) {
async function refused(entries, code, { gzip = true, raw = null, contentType } = {}) {
- const before = t.events('host.deploy.failed').length;
+ const before = (await t.events('host.deploy.failed')).length;
const r = await t.api('POST', `/api/v1/sites/${site.id}/deploys?activate=1`, { as: alice, body: raw || tarball(entries, { gzip }), headers: { 'content-type': contentType || 'application/gzip' } });
const r = await t.api('POST', `/api/v1/sites/${site.id}/deploys?activate=1`, { as: alice, body: raw || tarball(entries, { gzip }), headers: { 'content-type': contentType || 'application/gzip' } });
assert.ok([413, 422].includes(r.status), `${code}: status ${r.status} ${r.text}`);
assert.ok([413, 422].includes(r.status), `${code}: status ${r.status} ${r.text}`);
const body = r.json();
const body = r.json();
@@ -33,7 +33,7 @@ const { tarball } = require('./stageb/tar');
@@ -33,7 +33,7 @@ const { tarball } = require('./stageb/tar');
assert.strictEqual(failed.state, 'failed');
assert.strictEqual(failed.state, 'failed');
assert.strictEqual(failed.failure_code, code);
assert.strictEqual(failed.failure_code, code);
assert.strictEqual(failed.file_count, 0);
assert.strictEqual(failed.file_count, 0);
- assert.strictEqual(t.events('host.deploy.failed').length, before + 1);
+ assert.strictEqual((await t.events('host.deploy.failed')).length, before + 1);
assert.strictEqual((await t.get('uploads.openvibe.host', '/')).text, 'GOOD', 'the active deploy is unchanged');
assert.strictEqual((await t.get('uploads.openvibe.host', '/')).text, 'GOOD', 'the active deploy is unchanged');
assert.strictEqual(countObjects(), baseline, 'nothing was stored');
assert.strictEqual(countObjects(), baseline, 'nothing was stored');
return body;
return body;
@@ -121,7 +121,7 @@ const { tarball } = require('./stageb/tar');
@@ -121,7 +121,7 @@ const { tarball } = require('./stageb/tar');
const one = list.deploys.find((d) => d.failure_code === 'deploy.link_refused');
const one = list.deploys.find((d) => d.failure_code === 'deploy.link_refused');
const log = (await t.api('GET', `/api/v1/deploys/${one.id}/log`, { as: alice })).json().log;
const log = (await t.api('GET', `/api/v1/deploys/${one.id}/log`, { as: alice })).json().log;
assert.ok(log.some((l) => l.level === 'error' && /link/.test(l.message)));
assert.ok(log.some((l) => l.level === 'error' && /link/.test(l.message)));
- for (const e of t.events('host.deploy.failed')) assert.ok(!JSON.stringify(e).includes('sk_live'), 'no file contents in events');
+ for (const e of await t.events('host.deploy.failed')) assert.ok(!JSON.stringify(e).includes('sk_live'), 'no file contents in events');
});
});
await t.close();
await t.close();
@@ -53,6 +53,9 @@ async function boot(opts = {}) {
@@ -53,6 +53,9 @@ async function boot(opts = {}) {
};
};
const configLib = require('../../server/config');
const configLib = require('../../server/config');
const { createApp } = require('../../server/app');
const { createApp } = require('../../server/app');
+ const { createStore } = require('../../server/db');
+ // One database per boot (PGlite, or HOST_TEST_STORE=pg: the containers); a restart keeps it, like a file did.
+ const testdb = await require('../helpers/db').testDb();
const quiet = { log() {}, warn() {}, error: (...a) => { if (process.env.VERBOSE) console.error(...a); } };
const quiet = { log() {}, warn() {}, error: (...a) => { if (process.env.VERBOSE) console.error(...a); } };
let server = null;
let server = null;
@@ -61,7 +64,7 @@ async function boot(opts = {}) {
@@ -61,7 +64,7 @@ async function boot(opts = {}) {
async function start() {
async function start() {
const config = configLib.load(env);
const config = configLib.load(env);
- built = createApp({ config, now: clock.now, log: quiet, resolver, fetchImpl: opts.fetchImpl });
+ built = await createApp({ config, store: createStore(testdb.db, { now: clock.now }), now: clock.now, log: quiet, resolver, fetchImpl: opts.fetchImpl });
await built.ctx.auth.ensureKey();
await built.ctx.auth.ensureKey();
server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
server = await new Promise((resolve) => { const s = http.createServer(built.app); s.listen(0, '127.0.0.1', () => resolve(s)); });
t.port = server.address().port;
t.port = server.address().port;
@@ -71,7 +74,7 @@ async function boot(opts = {}) {
@@ -71,7 +74,7 @@ async function boot(opts = {}) {
}
}
async function stop() {
async function stop() {
if (server) { server.closeAllConnections(); await new Promise((r) => server.close(r)); }
if (server) { server.closeAllConnections(); await new Promise((r) => server.close(r)); }
- if (built) { built.ctx.worker.stop(); await built.ctx.outbox.stop(); built.ctx.stopMetrics(); built.ctx.store.close(); }
+ if (built) { built.ctx.worker.stop(); await built.ctx.outbox.stop(); built.ctx.stopMetrics(); }
server = null; built = null;
server = null; built = null;
}
}
@@ -107,7 +110,7 @@ async function boot(opts = {}) {
@@ -107,7 +110,7 @@ async function boot(opts = {}) {
t.get = (host, p, o = {}) => request({ ...o, method: o.method || 'GET', host, path: p });
t.get = (host, p, o = {}) => request({ ...o, method: o.method || 'GET', host, path: p });
/** Outbox rows as parsed envelopes. */
/** Outbox rows as parsed envelopes. */
- t.events = (type = null) => t.ctx.store.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all().map((r) => JSON.parse(r.envelope)).filter((e) => !type || e.event_type === type);
+ t.events = async (type = null) => (await t.ctx.store.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all()).map((r) => (typeof r.envelope === 'string' ? (typeof r.envelope === 'string' ? (typeof r.envelope === 'string' ? JSON.parse(r.envelope) : r.envelope) : r.envelope) : r.envelope)).filter((e) => !type || e.event_type === type);
t.user = (name, extra) => network.addUser(name, extra);
t.user = (name, extra) => network.addUser(name, extra);
t.project = async (user, name = 'Project', extra = {}) => {
t.project = async (user, name = 'Project', extra = {}) => {
@@ -131,7 +134,7 @@ async function boot(opts = {}) {
@@ -131,7 +134,7 @@ async function boot(opts = {}) {
};
};
t.restart = async () => { await stop(); await start(); };
t.restart = async () => { await stop(); await start(); };
- t.close = async () => { await stop(); await network.close(); fs.rmSync(dir, { recursive: true, force: true }); };
+ t.close = async () => { await stop(); await testdb.close(); await network.close(); fs.rmSync(dir, { recursive: true, force: true }); };
await start();
await start();
return t;
return t;
}
}