openvibers/openvibe.ai Commits
AI moves to PostgreSQL (ADR-035, roadmap WS-X2, repository-pass group G2), async from end to end.
Changes
@@@ -11,6 +11,14 @@ NODE_ENV=developmentBASE_URL=http://localhost:4700# SQLite (production unit: /var/lib/openvibe-ai/ai.db via StateDirectory=openvibe-ai)++# PostgreSQL (ADR-035), written by OpenVibe.Host roles/data/add-service.sh ai: the runtime role through PgBouncer,++# the owner on a direct connection (migrations and the one-time import), and Valkey (shared limit counters).++# Without DATABASE_URL, development uses an embedded PGlite database in data/pglite.++DATABASE_URL=++DATABASE_DIRECT_URL=++VALKEY_URL=++VALKEY_PREFIX=++# The SQLite file of releases before PostgreSQL: read once by scripts/migrate-to-postgres.js.AI_DB_PATH=./data/ai.db# Identity: OpenVibe.Network signs the service tokens callers present (audience openvibe.ai).@@@ -9,6 +9,9 @@ jobs:uses: OpenVibers/OpenVibe.Shared/.github/workflows/test.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176with:contracts-service: ai++ # ADR-035: PostgreSQL 18 + PgBouncer (transaction mode) + Valkey 9 containers (openvibe-sdk++ # scripts/test-services.sh). npm test runs every file on PGlite; test:pg runs them all through PgBouncer.++ test: 'eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" && npm test && npm run test:pg'# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).security:uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176@@@ -21,7 +21,8 @@ owns publication truth: Wiki, Blog, News, Live, … decide what to publish.- the eleven record groups ([below](#the-eleven-record-groups)): providers, models, routing profiles,prompt templates, workflow definitions, runs, the request log (hashes, not prompts), citations, the-- cache, quotas and usage, and the audit log, in AI's own SQLite++ cache, quotas and usage, and the audit log, in AI's own PostgreSQL database (`ov_ai`, ADR-035; schema in++ [migrations/](migrations/))- people's own provider keys (`ai.credential.manage`), encrypted at rest, and per-streamerattribution quotas (`ai.quota.attribution.manage`)- the `ai.run.queued|succeeded|failed|cached` events and the `ai.preferences` / `ai.usage_summary`@@@ -36,13 +37,14 @@ owns publication truth: Wiki, Blog, News, Live, … decide what to publish.## Depends on++- PostgreSQL 18 (OpenVibe.Host `roles/data/`, ADR-035): every read and write is async through `openvibe-sdk/db`- OpenVibe.Network (JWKS, service tokens, SSO for the operator console, user modules)- OpenVibe.Events (the `ai.run.*` outbox relay; off without `EVENTS_URL`)- OpenVibe.Media (recordings `media.analyze` reads, over allow-listed https URLs or signed URLs)- provider APIs configured by the operator (OpenAI-compatible, Anthropic, the HTTP seam), whisper.cppand the local model server `openvibe-llm.service` on this host--- `openvibe-contracts` v0.75.0, `openvibe-sdk` v0.11.0 (service tokens, events outbox),-- `openvibe-shared` v1.25.0, pinned by release tarball++- `openvibe-contracts` v0.76.0, `openvibe-sdk` v0.20.3 (service tokens, events outbox),++ `openvibe-shared` v1.27.0, pinned by release tarball## Capabilities@@@ -300,9 +302,9 @@ Reporting a vulnerability: [SECURITY.md](SECURITY.md). The rules the code keeps:`docs/live-patch.diff` added `AI_SERVICE=remote` to Live (deployed as Live `fa22de5`; production runswith `AI_SERVICE=remote` since 2026-09-23): Live's shared-key AI calls become runs here, authenticatedwith `serviceHeaders('openvibe.ai')` from Live's `server/net/network-principal.js`. Without the flag--Live's behaviour is unchanged. `scripts/import-from-live.js`--moves the AI records that belong here (`--dry-run` first); it ran on production on 2026-09-23 at--18:29 UTC (79,657 ledger rows, 11 holds recorded with their reasons).++Live's behaviour is unchanged. The one-time `scripts/import-from-live.js` moved the AI records that belong here;++it ran on production on 2026-09-23 at 18:29 UTC (79,657 ledger rows, 11 holds recorded with their reasons, kept in++`import_ledger` and `import_holds`) and was retired with the move to PostgreSQL (it is in git history).Network's footer copy calls the `network.site_copy` workflow (Network `422f8e9`), but the deployed Network stillfalls back to Live's `/internal/ai/site-copy`, and Live still serves that route. The full plan, holdsand rollback are in `docs/migration.md`.@@@ -311,14 +313,17 @@ and rollback are in `docs/migration.md`.Production deploys with `sudo ovhost deploy ai` on the host (strategy `git-checkout`: fetch,fast-forward `/opt/openvibe.ai`, install on a lockfile change, restart, wait for `/api/ready`).--The unit is `openvibe-ai.service` on `127.0.0.1:4700`, the env file `/etc/openvibe/ai.env`. The local model++The unit is `openvibe-ai.service` on `127.0.0.1:4700`, the env file `/etc/openvibe/ai.env`. The database is++`ov_ai` on the host's data role (`sudo /opt/openvibe.host/roles/data/add-service.sh ai` writes its settings); the++release migrates it at boot. The one-time move from SQLite is `scripts/migrate-to-postgres.js` (openvibe-sdk++`runSqliteMigration`, with a `--pglite` rehearsal mode), run while the service is stopped; the old++`/var/lib/openvibe-ai/ai.db` stays read-only for 7 days as the rollback. The local modelruns as `openvibe-llm.service` on 127.0.0.1:8090 ([Local model](#local-model)).Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the--restart; afterwards `sudo ovhost rollback ai --to <sha>`. Nothing blocks a rollback: the schema--code only adds tables and columns.++restart; afterwards `sudo ovhost rollback ai --to <sha>`. Migrations only add tables and columns.Deployed: `/opt/openvibe.ai`, env `/etc/openvibe/ai.env` (0600), unit `deploy/systemd/openvibe-ai.service`--(`StateDirectory=openvibe-ai`, database `/var/lib/openvibe-ai/ai.db`), principal `ai`. The nginx vhost++(`StateDirectory=openvibe-ai`; database `ov_ai` on the host's data role, the SQLite file `/var/lib/openvibe-ai/ai.db` before the switch), principal `ai`. The nginx vhost`deploy/nginx/ai.openvibe.network.conf` (health/ready, `/`, `/robots.txt`, the operator console`/console` and its sign-in `/auth/` public; the API is host-local) is not installed:`ai.openvibe.network` still serves the Sites placeholder. The console also needs@@@ -7,10 +7,25 @@"runtime": "node 22, express, better-sqlite3","port": 4700,"domain": "ai.openvibe.network still serves the OpenVibe.Sites placeholder; the service runs on openvibe-ovh (/opt/openvibe.ai, unit openvibe-ai) on 127.0.0.1:4700, loopback only, used by OpenVibe.Live and OpenVibe.Network",-- "contracts": "openvibe-contracts v0.75.0",-- "capabilities": ["ai.run.create", "ai.run.read", "ai.workflow.manage", "ai.provider.manage", "ai.usage.read", "ai.credential.manage", "ai.quota.attribution.manage"],++ "contracts": "openvibe-contracts v0.76.0",++ "capabilities": [++ "ai.run.create",++ "ai.run.read",++ "ai.workflow.manage",++ "ai.provider.manage",++ "ai.usage.read",++ "ai.credential.manage",++ "ai.quota.attribution.manage"++ ],"capabilitiesRegistered": true,-- "providers": ["stub", "openai-compatible", "anthropic", "http-seam", "whisper.cpp", "local llama.cpp server (openvibe-llm.service)"],++ "providers": [++ "stub",++ "openai-compatible",++ "anthropic",++ "http-seam",++ "whisper.cpp",++ "local llama.cpp server (openvibe-llm.service)"++ ],"livePatch": "applied: Live fa22de5 runs AI_SERVICE=remote in production since 2026-09-23 ~15:34 UTC (309 runs by 18:30 UTC across 13 live.* workflows and 1 network.site_copy, 0 failed). As of 2026-09-23, not moved yet: VOD captions/whisper and streamers' own provider keys stayed in Live and most Live features were passthrough prompts built by Live (since 2026-09-27 AI has live.media.transcribe, stored provider keys and versioned templates for most live.* workflows); the deployed Network still falls back to Live's /internal/ai/site-copy; scripts/import-from-live.js ran on production at 2026-09-23 18:29 UTC (79,657 ledger rows: 79,647 live.ai_usage, 2 channel_ai_config, 8 site_settings; 11 holds: provider settings and the key stay env references, 1 streamer BYO key stays in Live, 3,021 translations are unrepresentable and stay in Live); every route has fallbacks=[]; 0 citation rows; no /metrics","deployed": true,"updated": "2026-09-28",@@@ -2,8 +2,8 @@Description=OpenVibe.AI — providers, routing, versioned templates and workflows, runs, citations, cache, quotas and audit# Network (openvibe-network) signs the service tokens callers present. The AI service starts without# it and keeps retrying the JWKS fetch; /api/ready stays 503 until the key has loaded.--After=network.target--Wants=openvibe-network.service++After=network.target postgresql.service pgbouncer.service valkey-server.service++Wants=openvibe-network.service postgresql.service pgbouncer.service valkey-server.service[Service]Type=simplennew file mode 100644@@@ -0,0 +1,336 @@++-- phase: expand++-- OpenVibe.AI on PostgreSQL (ADR-035, roadmap WS-X2): the tables as they were on SQLite (converted by openvibe-sdk++-- tools/asyncify/sqlite-schema-to-pg: text COLLATE "C" compares like SQLite, integers are bigint, identities keep their ids),++-- then the openvibe-publishing stores and the openvibe-sdk inbox and outbox. Generated once on 2026-09-28; never edited after it runs.++++CREATE TABLE providers (++ key text COLLATE "C" PRIMARY KEY,++ display_name text COLLATE "C" NOT NULL,++ kind text COLLATE "C" NOT NULL, -- stub | openai | anthropic | http | whisper++ status text COLLATE "C" NOT NULL DEFAULT 'active', -- active | disabled++ base_url text COLLATE "C",++ auth_mode text COLLATE "C" NOT NULL DEFAULT 'none', -- none | bearer | x-api-key++ secret_ref text COLLATE "C", -- 'env:NAME' — never a value++ default_model text COLLATE "C",++ capabilities text COLLATE "C" NOT NULL DEFAULT '[]', -- features supports() answers true for++ timeout_ms bigint NOT NULL DEFAULT 30000,++ priority bigint NOT NULL DEFAULT 100,++ metadata text COLLATE "C" NOT NULL DEFAULT '{}',++ origin text COLLATE "C" NOT NULL DEFAULT 'seed', -- seed (from env at boot) | admin | import++ created_at text COLLATE "C" NOT NULL,++ updated_at text COLLATE "C" NOT NULL++);++CREATE TABLE provider_health (++ provider_key text COLLATE "C" PRIMARY KEY,++ state text COLLATE "C" NOT NULL DEFAULT 'closed', -- closed (healthy) | open (skipped) | half_open (one probe)++ consecutive_failures bigint NOT NULL DEFAULT 0,++ opened_at bigint,++ last_error text COLLATE "C",++ last_success_at bigint,++ last_failure_at bigint++);++CREATE TABLE models (++ provider_key text COLLATE "C" NOT NULL,++ model_key text COLLATE "C" NOT NULL,++ display_name text COLLATE "C",++ type text COLLATE "C" NOT NULL DEFAULT 'chat', -- chat | vision | embedding | stt++ status text COLLATE "C" NOT NULL DEFAULT 'active',++ context_window bigint,++ max_output bigint,++ cost_in_per_mtok double precision,++ cost_out_per_mtok double precision,++ cost_cached_per_mtok double precision,++ supports_json bigint NOT NULL DEFAULT 1,++ supports_tools bigint NOT NULL DEFAULT 0,++ supports_streaming bigint NOT NULL DEFAULT 0,++ supports_vision bigint NOT NULL DEFAULT 0,++ metadata text COLLATE "C" NOT NULL DEFAULT '{}',++ created_at text COLLATE "C" NOT NULL,++ updated_at text COLLATE "C" NOT NULL,++ PRIMARY KEY (provider_key, model_key)++);++CREATE TABLE routes (++ key text COLLATE "C" NOT NULL,++ version bigint NOT NULL,++ status text COLLATE "C" NOT NULL DEFAULT 'active', -- active | disabled++ primary_provider text COLLATE "C" NOT NULL,++ primary_model text COLLATE "C",++ fallbacks text COLLATE "C" NOT NULL DEFAULT '[]', -- [{provider, model}]++ options text COLLATE "C" NOT NULL DEFAULT '{}', -- temperature and friends++ max_output_tokens bigint,++ response_format text COLLATE "C" NOT NULL DEFAULT 'text', -- text | json++ timeout_ms bigint,++ alias_of text COLLATE "C", -- historical key kept as an explicit alias++ created_by text COLLATE "C",++ created_at text COLLATE "C" NOT NULL,++ PRIMARY KEY (key, version)++);++CREATE TABLE templates (++ key text COLLATE "C" NOT NULL,++ version bigint NOT NULL,++ name text COLLATE "C" NOT NULL,++ description text COLLATE "C",++ input_schema text COLLATE "C" NOT NULL DEFAULT '{}',++ output_schema text COLLATE "C" NOT NULL DEFAULT '{}',++ system_prompt text COLLATE "C" NOT NULL DEFAULT '',++ user_prompt text COLLATE "C" NOT NULL DEFAULT '',++ default_route text COLLATE "C",++ owner text COLLATE "C" NOT NULL DEFAULT 'ai',++ visibility text COLLATE "C" NOT NULL DEFAULT 'internal', -- public | first-party | internal++ status text COLLATE "C" NOT NULL DEFAULT 'active', -- draft | active | deprecated | archived++ metadata text COLLATE "C" NOT NULL DEFAULT '{}',++ created_by text COLLATE "C",++ created_at text COLLATE "C" NOT NULL,++ PRIMARY KEY (key, version)++);++CREATE TABLE workflows (++ key text COLLATE "C" NOT NULL,++ version bigint NOT NULL,++ name text COLLATE "C" NOT NULL,++ description text COLLATE "C",++ namespace text COLLATE "C" NOT NULL, -- owning product: live, network, wiki, ...++ input_schema text COLLATE "C" NOT NULL,++ output_schema text COLLATE "C" NOT NULL,++ steps text COLLATE "C" NOT NULL, -- [{kind, template?, operation?, ...}]++ default_route text COLLATE "C",++ cache_mode text COLLATE "C" NOT NULL DEFAULT 'private', -- none | private | service++ cache_ttl_sec bigint,++ status text COLLATE "C" NOT NULL DEFAULT 'active', -- draft | active | deprecated | archived++ metadata text COLLATE "C" NOT NULL DEFAULT '{}',++ created_by text COLLATE "C",++ created_at text COLLATE "C" NOT NULL,++ PRIMARY KEY (key, version)++);++CREATE TABLE runs (++ id text COLLATE "C" PRIMARY KEY, -- run_<ULID>++ workflow_key text COLLATE "C" NOT NULL,++ workflow_version bigint NOT NULL,++ template_key text COLLATE "C",++ template_version bigint,++ route_key text COLLATE "C",++ route_version bigint,++ status text COLLATE "C" NOT NULL, -- queued | running | succeeded | failed | cancelled | cached++ requester_type text COLLATE "C" NOT NULL, -- service | app | mod (the token principal)++ requester_id text COLLATE "C" NOT NULL,++ on_behalf_of text COLLATE "C", -- SubjectRef JSON (optional)++ attribution text COLLATE "C", -- EntityRef JSON the spend is attributed to (optional)++ source_service text COLLATE "C",++ target text COLLATE "C", -- EntityRef JSON (optional)++ target_key text COLLATE "C", -- service:type:id for lookups++ input text COLLATE "C" NOT NULL,++ input_hash text COLLATE "C" NOT NULL,++ output text COLLATE "C",++ citations_count bigint NOT NULL DEFAULT 0,++ error_code text COLLATE "C",++ error_detail text COLLATE "C",++ synthetic bigint NOT NULL DEFAULT 0, -- produced by the stub provider++ provider_key text COLLATE "C",++ model_key text COLLATE "C",++ fallback_used bigint NOT NULL DEFAULT 0,++ attempts bigint NOT NULL DEFAULT 0,++ tokens_in bigint NOT NULL DEFAULT 0,++ tokens_out bigint NOT NULL DEFAULT 0,++ cost_usd double precision NOT NULL DEFAULT 0,++ cache_key text COLLATE "C",++ cached_from text COLLATE "C", -- run id whose output a cached run reused++ retry_of text COLLATE "C",++ idempotency_key text COLLATE "C",++ trace_id text COLLATE "C",++ request_id text COLLATE "C",++ options text COLLATE "C" NOT NULL DEFAULT '{}',++ created_at text COLLATE "C" NOT NULL,++ started_at text COLLATE "C",++ finished_at text COLLATE "C"++);++CREATE UNIQUE INDEX idx_runs_idem ON runs(requester_type, requester_id, idempotency_key) WHERE idempotency_key IS NOT NULL;++CREATE INDEX idx_runs_status ON runs(status, created_at);++CREATE INDEX idx_runs_workflow ON runs(workflow_key, created_at);++CREATE INDEX idx_runs_target ON runs(target_key, created_at);++CREATE INDEX idx_runs_requester ON runs(requester_type, requester_id, created_at);++CREATE TABLE requests (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ run_id text COLLATE "C",++ seq bigint NOT NULL DEFAULT 0,++ operation text COLLATE "C" NOT NULL, -- chat | generate | summarize | classify | extract | enrich | embed | transcribe++ provider_key text COLLATE "C" NOT NULL,++ model_key text COLLATE "C",++ route_key text COLLATE "C",++ route_version bigint,++ status text COLLATE "C" NOT NULL, -- ok | error | timeout | skipped | cancelled++ skip_reason text COLLATE "C", -- disabled | circuit_open | unsupported | no_credentials++ fallback bigint NOT NULL DEFAULT 0, -- 1 when this was not the route's primary++ prompt_hash text COLLATE "C",++ input_hash text COLLATE "C",++ output_hash text COLLATE "C",++ tokens_in bigint NOT NULL DEFAULT 0,++ tokens_out bigint NOT NULL DEFAULT 0,++ tokens_cached bigint NOT NULL DEFAULT 0,++ tokens_estimated bigint NOT NULL DEFAULT 0,++ cost_usd double precision NOT NULL DEFAULT 0,++ latency_ms bigint,++ error text COLLATE "C",++ debug_prompt text COLLATE "C", -- only with AI_DEBUG_RAW_LOG and a caller opt-in++ debug_response text COLLATE "C",++ created_at text COLLATE "C" NOT NULL++);++CREATE INDEX idx_requests_run ON requests(run_id, seq);++CREATE INDEX idx_requests_provider ON requests(provider_key, created_at);++CREATE TABLE citations (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ run_id text COLLATE "C" NOT NULL,++ ordinal bigint NOT NULL DEFAULT 0,++ source_type text COLLATE "C" NOT NULL,++ source_id text COLLATE "C",++ url text COLLATE "C",++ title text COLLATE "C",++ author text COLLATE "C",++ published_at text COLLATE "C",++ retrieved_at text COLLATE "C",++ snippet text COLLATE "C",++ content_hash text COLLATE "C",++ trust text COLLATE "C" NOT NULL DEFAULT '{}',++ provenance text COLLATE "C" NOT NULL DEFAULT '{}',++ attached_by text COLLATE "C", -- workflow | <principal sub>++ created_at text COLLATE "C" NOT NULL++);++CREATE INDEX idx_citations_run ON citations(run_id, ordinal);++CREATE TABLE cache_entries (++ cache_key text COLLATE "C" PRIMARY KEY, -- sha256 over everything below++ scope text COLLATE "C" NOT NULL, -- who may read it (see server/cache.js)++ workflow_key text COLLATE "C" NOT NULL,++ workflow_version bigint NOT NULL,++ template_version bigint,++ route_key text COLLATE "C",++ route_version bigint,++ model_key text COLLATE "C",++ input_hash text COLLATE "C" NOT NULL,++ privacy text COLLATE "C" NOT NULL, -- private | service++ output text COLLATE "C" NOT NULL,++ run_id text COLLATE "C" NOT NULL,++ hits bigint NOT NULL DEFAULT 0,++ created_at text COLLATE "C" NOT NULL,++ expires_at bigint NOT NULL++);++CREATE INDEX idx_cache_expiry ON cache_entries(expires_at);++CREATE INDEX idx_cache_workflow ON cache_entries(workflow_key);++CREATE TABLE quotas (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ scope_type text COLLATE "C" NOT NULL, -- global | service | actor | attribution++ scope_id text COLLATE "C" NOT NULL, -- '*' | live | user:usr_… | live:user:42++ "window" text COLLATE "C" NOT NULL, -- minute | hour | day++ max_requests bigint,++ max_tokens bigint,++ max_cost_usd double precision,++ workflow_prefix text COLLATE "C", -- optional: only runs of workflows with this prefix++ status text COLLATE "C" NOT NULL DEFAULT 'active',++ origin text COLLATE "C" NOT NULL DEFAULT 'admin', -- seed | admin | import++ created_at text COLLATE "C" NOT NULL,++ updated_at text COLLATE "C" NOT NULL++);++CREATE UNIQUE INDEX idx_quotas_unique ON quotas(scope_type, scope_id, "window", COALESCE(workflow_prefix, ''));++CREATE TABLE usage_counters (++ scope_type text COLLATE "C" NOT NULL,++ scope_id text COLLATE "C" NOT NULL,++ "window" text COLLATE "C" NOT NULL,++ window_start bigint NOT NULL, -- epoch seconds, UTC++ workflow_prefix text COLLATE "C" NOT NULL DEFAULT '',++ requests bigint NOT NULL DEFAULT 0,++ tokens bigint NOT NULL DEFAULT 0,++ cost_usd double precision NOT NULL DEFAULT 0,++ PRIMARY KEY (scope_type, scope_id, "window", window_start, workflow_prefix)++);++CREATE TABLE usage_daily (++ day text COLLATE "C" NOT NULL, -- YYYY-MM-DD (UTC)++ requester text COLLATE "C" NOT NULL, -- service:live++ attribution text COLLATE "C" NOT NULL DEFAULT '', -- live:user:42++ workflow_key text COLLATE "C" NOT NULL,++ provider_key text COLLATE "C" NOT NULL,++ model_key text COLLATE "C" NOT NULL DEFAULT '',++ requests bigint NOT NULL DEFAULT 0,++ tokens_in bigint NOT NULL DEFAULT 0,++ tokens_out bigint NOT NULL DEFAULT 0,++ tokens_cached bigint NOT NULL DEFAULT 0,++ cost_usd double precision NOT NULL DEFAULT 0,++ PRIMARY KEY (day, requester, attribution, workflow_key, provider_key, model_key)++);++CREATE TABLE audit_log (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ at text COLLATE "C" NOT NULL,++ actor text COLLATE "C" NOT NULL, -- principal sub (svc:live) or 'system'++ action text COLLATE "C" NOT NULL, -- run.create | template.version | provider.update | ...++ target_type text COLLATE "C",++ target_id text COLLATE "C",++ trace_id text COLLATE "C",++ metadata text COLLATE "C" NOT NULL DEFAULT '{}'++);++CREATE INDEX idx_audit_target ON audit_log(target_type, target_id, id);++CREATE INDEX idx_audit_action ON audit_log(action, id);++CREATE TABLE console_sessions (++ id_hash text COLLATE "C" PRIMARY KEY, -- sha256 of the session cookie; the cookie value is never stored++ subject text COLLATE "C" NOT NULL, -- the Network person (usr_…)++ username text COLLATE "C",++ staff text COLLATE "C" NOT NULL DEFAULT '{}', -- the sign-in token's staff claims (role, is_owner, staff_caps, staff_map)++ csrf text COLLATE "C" NOT NULL,++ created_at text COLLATE "C" NOT NULL,++ expires_at text COLLATE "C" NOT NULL,++ revoked_at text COLLATE "C",++ ip_hash text COLLATE "C" -- keyed hash of the client address++);++CREATE TABLE import_ledger (++ source text COLLATE "C" NOT NULL, -- live.ai_usage, live.translations, ...++ source_key text COLLATE "C" NOT NULL,++ target text COLLATE "C" NOT NULL, -- where it went (table:key)++ imported_at text COLLATE "C" NOT NULL,++ PRIMARY KEY (source, source_key)++);++CREATE TABLE import_holds (++ source text COLLATE "C" NOT NULL,++ source_key text COLLATE "C" NOT NULL,++ reason text COLLATE "C" NOT NULL,++ detail text COLLATE "C",++ held_at text COLLATE "C" NOT NULL,++ PRIMARY KEY (source, source_key)++);++++-- openvibe-sdk/events PostgreSQL outbox++CREATE TABLE IF NOT EXISTS event_outbox (++ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,++ event_id text NOT NULL UNIQUE,++ envelope jsonb NOT NULL,++ traceparent text,++ created_at bigint NOT NULL,++ attempts integer NOT NULL DEFAULT 0,++ next_attempt_at bigint NOT NULL DEFAULT 0,++ sent_at bigint,++ seq bigint,++ rejected_at bigint,++ last_error text++);++CREATE INDEX IF NOT EXISTS event_outbox_due ON event_outbox (next_attempt_at, id) WHERE sent_at IS NULL AND rejected_at IS NULL;++CREATE INDEX IF NOT EXISTS event_outbox_sent ON event_outbox (sent_at) WHERE sent_at IS NOT NULL;++++-- Added after the first SQLite schema: a run's grounding ({ cited, gaps }, WS-O task 3).++ALTER TABLE runs ADD COLUMN grounding text COLLATE "C";++++-- server/user-modules.js: the last preferences hash pushed to Network per person (was created on first use).++CREATE TABLE ai_module_pushes (++ subject_id text COLLATE "C" PRIMARY KEY,++ hash text COLLATE "C" NOT NULL,++ pushed_at text COLLATE "C" NOT NULL++);++++-- server/credentials.js: a person's own provider keys (WS-O task 2), encrypted (was created on first use).++CREATE TABLE subject_credentials (++ owner text COLLATE "C" NOT NULL,++ subject text COLLATE "C" NOT NULL,++ provider text COLLATE "C" NOT NULL CHECK (provider IN ('openai', 'anthropic')),++ base_url text COLLATE "C",++ key_enc text COLLATE "C" NOT NULL,++ key_hint text COLLATE "C" NOT NULL,++ models text COLLATE "C" NOT NULL DEFAULT '{}',++ budget_usd_day double precision,++ created_at text COLLATE "C" NOT NULL,++ updated_at text COLLATE "C" NOT NULL,++ last_used_at text COLLATE "C",++ PRIMARY KEY (owner, subject)++);@@@ -13,9 +13,13 @@"better-sqlite3": "^11.10.0","dotenv": "^16.4.7","express": "^4.21.2",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.75.0",-- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0",-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",++ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",++ "pg": "^8.23.0"++ },++ "devDependencies": {++ "@electric-sql/pglite": "^0.5.8"},"engines": {"node": ">=20.0.0"@@@ -24,6 +28,13 @@"sharp": "^0.33.5"}},++ "node_modules/@electric-sql/pglite": {++ "version": "0.5.8",++ "resolved": "https://registry.npmjs.org/@electric-sql/pglite/-/pglite-0.5.8.tgz",++ "integrity": "sha512-n9tsbUOhwx2epK1V0ZG9Ar4SHWUju04dhmzZXiSBXwBoleOvIfals33NAaWgagQVAL4Rbvx/Ptsu3P+pA09f6Q==",++ "devOptional": true,++ "license": "Apache-2.0"++ },"node_modules/@emnapi/runtime": {"version": "1.11.3","resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.11.3.tgz",@@@ -1395,9 +1406,9 @@}},"node_modules/openvibe-contracts": {-- "version": "0.75.0",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.75.0",-- "integrity": "sha512-l+VZGYvVpX2IGsEFPAFwJa8upFRBpZ0ufZxTSua72UeGSj7z2ceHxNjuua0oQ1fnsshl4hCvXHyen1eHjic/eA==",++ "version": "0.76.0",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",++ "integrity": "sha512-zTdx776KWrcJKm1JTnf+0+BnsvswdvXy1ldPyse1hSqhnW/SSr98sX696eRetFUhTl8qVM1xSRCykqIUbqItPw==","license": "AGPL-3.0-only","dependencies": {"ajv": "^8.17.1",@@@ -1411,18 +1422,38 @@}},"node_modules/openvibe-sdk": {-- "version": "0.11.0",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0",-- "integrity": "sha512-sY/0bl5bhixjxLv2ACeL/AKIE2QlnndiDqkNuA5YGo2pC7AMvPqRP05y4OAtuJ7JvWtaBZ9gETB03u07uYp3wQ==",++ "version": "0.20.3",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",++ "integrity": "sha512-+g+ySveItzOBnUsw9aZZIJVKRrGuj0aO328W4N25T0Z8mY4fE6++ImvLnZ55LH2BRh0WYozC1KAVIShACm2KGQ==","license": "MIT","engines": {"node": ">=20"++ },++ "peerDependencies": {++ "@electric-sql/pglite": "^0.5.8",++ "better-sqlite3": ">=11",++ "iovalkey": "^0.4.0",++ "pg": "^8.23.0"++ },++ "peerDependenciesMeta": {++ "@electric-sql/pglite": {++ "optional": true++ },++ "better-sqlite3": {++ "optional": true++ },++ "iovalkey": {++ "optional": true++ },++ "pg": {++ "optional": true++ }}},"node_modules/openvibe-shared": {-- "version": "1.25.0",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0",-- "integrity": "sha512-oqZ+PvljrWi8IEbtg6bDYai17ZV2sALFx/ezMptniELSr10TgFUiGUQt+b1IWwiHv0T2Y1U2iSrkirJxMNgjSw==",++ "version": "1.27.0",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",++ "integrity": "sha512-/H7dyFw+cURcdjyfbpyY7mNc6F+Pcb9TwkVwZHR+9d3M62q70KDhZLo/gZi71eaYxUjD8lO8tmbXu70EAa9paQ==","license": "MIT","dependencies": {"jsonwebtoken": "^9.0.0"@@@ -1443,6 +1474,134 @@"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==","license": "MIT"},++ "node_modules/pg": {++ "version": "8.23.0",++ "resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",++ "integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",++ "license": "MIT",++ "dependencies": {++ "pg-connection-string": "^2.14.0",++ "pg-pool": "^3.14.0",++ "pg-protocol": "^1.16.0",++ "pg-types": "2.2.0",++ "pgpass": "1.0.5"++ },++ "engines": {++ "node": ">= 16.0.0"++ },++ "optionalDependencies": {++ "pg-cloudflare": "^1.4.0"++ },++ "peerDependencies": {++ "pg-native": ">=3.0.1"++ },++ "peerDependenciesMeta": {++ "pg-native": {++ "optional": true++ }++ }++ },++ "node_modules/pg-cloudflare": {++ "version": "1.4.0",++ "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",++ "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",++ "license": "MIT",++ "optional": true++ },++ "node_modules/pg-connection-string": {++ "version": "2.14.0",++ "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",++ "integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",++ "license": "MIT"++ },++ "node_modules/pg-int8": {++ "version": "1.0.1",++ "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",++ "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",++ "license": "ISC",++ "engines": {++ "node": ">=4.0.0"++ }++ },++ "node_modules/pg-pool": {++ "version": "3.14.0",++ "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",++ "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",++ "license": "MIT",++ "peerDependencies": {++ "pg": ">=8.0"++ }++ },++ "node_modules/pg-protocol": {++ "version": "1.16.0",++ "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz",++ "integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==",++ "license": "MIT"++ },++ "node_modules/pg-types": {++ "version": "2.2.0",++ "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",++ "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",++ "license": "MIT",++ "dependencies": {++ "pg-int8": "1.0.1",++ "postgres-array": "~2.0.0",++ "postgres-bytea": "~1.0.0",++ "postgres-date": "~1.0.4",++ "postgres-interval": "^1.1.0"++ },++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/pgpass": {++ "version": "1.0.5",++ "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",++ "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",++ "license": "MIT",++ "dependencies": {++ "split2": "^4.1.0"++ }++ },++ "node_modules/postgres-array": {++ "version": "2.0.0",++ "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",++ "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",++ "license": "MIT",++ "engines": {++ "node": ">=4"++ }++ },++ "node_modules/postgres-bytea": {++ "version": "1.0.1",++ "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",++ "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",++ "license": "MIT",++ "engines": {++ "node": ">=0.10.0"++ }++ },++ "node_modules/postgres-date": {++ "version": "1.0.7",++ "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",++ "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",++ "license": "MIT",++ "engines": {++ "node": ">=0.10.0"++ }++ },++ "node_modules/postgres-interval": {++ "version": "1.2.0",++ "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",++ "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",++ "license": "MIT",++ "dependencies": {++ "xtend": "^4.0.0"++ },++ "engines": {++ "node": ">=0.10.0"++ }++ },"node_modules/prebuild-install": {"version": "7.1.3","resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",@@@ -1831,6 +1990,15 @@"is-arrayish": "^0.3.1"}},++ "node_modules/split2": {++ "version": "4.2.0",++ "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",++ "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",++ "license": "ISC",++ "engines": {++ "node": ">= 10.x"++ }++ },"node_modules/statuses": {"version": "2.0.2","resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",@@@ -1965,6 +2133,15 @@"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz","integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==","license": "ISC"++ },++ "node_modules/xtend": {++ "version": "4.0.2",++ "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",++ "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",++ "license": "MIT",++ "engines": {++ "node": ">=0.4"++ }}}}@@@ -8,9 +8,15 @@"start": "node server/index.js","dev": "NODE_ENV=development node server/index.js","test": "node test/run.js",-- "import-from-live": "node scripts/import-from-live.js"++ "test:pg": "AI_TEST_STORE=pg node test/run.js"},-- "keywords": ["openvibe", "ai", "llm", "workflows", "whisper"],++ "keywords": [++ "openvibe",++ "ai",++ "llm",++ "workflows",++ "whisper"++ ],"author": "OpenVibers","license": "AGPL-3.0-only","repository": "github:OpenVibers/OpenVibe.AI",@@@ -19,14 +25,18 @@"better-sqlite3": "^11.10.0","dotenv": "^16.4.7","express": "^4.21.2",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.75.0",-- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0",-- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",++ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",++ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",++ "pg": "^8.23.0"},"optionalDependencies": {"sharp": "^0.33.5"},"engines": {"node": ">=20.0.0"++ },++ "devDependencies": {++ "@electric-sql/pglite": "^0.5.8"}}ddeleted file mode 100644@@@ -1,224 +0,0 @@--#!/usr/bin/env node--'use strict';--/**-- * Import the AI records that belong to OpenVibe.AI from a snapshot of OpenVibe.Live's database.-- *-- * node scripts/import-from-live.js --live-db /path/to/live-snapshot.db [--ai-db ./data/ai.db] [--dry-run]-- *-- * Reads the Live snapshot READ-ONLY. Idempotent: every imported source row is recorded in-- * import_ledger (source, key, content hash), so a second run imports nothing new and a changed-- * setting is applied once. Rows this service cannot represent are recorded in import_holds with a-- * reason — never silently dropped, never guessed. --dry-run prints the plan and writes nothing.-- *-- * What moves (see docs/migration.md for the full table):-- * site_settings ai_model_<role> -> a new version of route live.<role> with that model (origin import)-- * site_settings ai_pricing_json -> model rows with cost metadata for provider `shared`-- * site_settings ai_max_cost_usd_per_day -> quota global/day max_cost_usd-- * site_settings ai_viewers_global_cap_usd_per_day -> quota global/day max_cost_usd, workflow_prefix live.viewers.-- * site_settings ai_provider / ai_base_url / ai_model / ai_api_key -> HELD: these are environment-- * (AI_PROVIDER, AI_BASE_URL, AI_MODEL, AI_API_KEY); the script prints which names to set, never values of secrets-- * ai_usage -> usage_daily (per day, requester service:live, attribution live:user:<owner>,-- * workflow by Live kind), plus today's quota windows-- * channel_ai_config -> attribution quota live:user:<id>/day for live.viewers.* (shared key budgets);-- * BYO keys are HELD (per-actor provider secrets are not stored here yet)-- * translations, ai_timeline_cache, ai_chatbot_configs -> HELD (one row each; reasons in the hold)-- * Everything else AI-shaped in Live (stream_memories, streamer_overviews, stream_recaps, vod/clip_ai_state,-- * chat_ai_summaries, channel_ai_bots, ai_viewer_*) is Live product state and stays in Live.-- */--const fs = require('fs');--const path = require('path');--const Database = require('better-sqlite3');--const { load } = require('../server/config');--const { openDb } = require('../server/db');--const { createRegistry } = require('../server/registry');--const { createQuotas } = require('../server/quota');--const { seed } = require('../server/workflows/seed');--const { KIND_TO_WORKFLOW } = require('../server/workflows/live');--const { sha256, iso } = require('../server/util');----/** Live ai_usage.kind -> the workflow that now does that job. */--const KIND_MAP = {-- ...KIND_TO_WORKFLOW,-- translate: 'live.translate', paste_image: 'live.paste.describe_image', moment_frame: 'live.paste.describe_image', paste_text: 'live.paste.summarize_text',-- stream_memory: 'live.stream.summarize', streamer_overview: 'live.streamer.overview', media_overview: 'live.media.overview', stream_recap: 'live.stream.recap',-- site_copy: 'network.site_copy',--};--const ROLES = ['chat', 'vision', 'director', 'summary'];----function parseArgs(argv) {-- const out = { dryRun: false, liveDb: null, aiDb: null, quiet: false };-- for (let i = 0; i < argv.length; i++) {-- const a = argv[i];-- if (a === '--dry-run') out.dryRun = true;-- else if (a === '--quiet') out.quiet = true;-- else if (a === '--live-db') out.liveDb = argv[++i];-- else if (a === '--ai-db') out.aiDb = argv[++i];-- else if (a === '-h' || a === '--help') out.help = true;-- else throw new Error(`unknown argument ${a}`);-- }-- return out;--}----function hasTable(db, name) {-- return Boolean(db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name));--}----function run({ liveDb, aiDb, dryRun = false, env = process.env, log = console.log, now = Date.now() } = {}) {-- if (!liveDb || !fs.existsSync(liveDb)) throw new Error('--live-db <snapshot> is required and must exist');-- const live = new Database(liveDb, { readonly: true, fileMustExist: true });-- const config = load({ ...env, AI_DB_PATH: aiDb || env.AI_DB_PATH || './data/ai.db' });-- const aiPath = path.resolve(config.dbPath);---- // Dry run: never create or write the AI database; read its ledger if it already exists.-- let db = null;-- let registry = null;-- let quotas = null;-- if (!dryRun) {-- db = openDb(aiPath);-- registry = createRegistry(db, { env });-- quotas = createQuotas(db, { registry });-- seed({ registry, quotas, config, env, db });-- } else if (fs.existsSync(aiPath)) {-- db = new Database(aiPath, { readonly: true, fileMustExist: true });-- }-- const ledgerGet = (source, key) => (db && hasTable(db, 'import_ledger') ? db.prepare('SELECT target FROM import_ledger WHERE source = ? AND source_key = ?').get(source, String(key)) : null);-- const holdGet = (source, key) => (db && hasTable(db, 'import_holds') ? db.prepare('SELECT 1 FROM import_holds WHERE source = ? AND source_key = ?').get(source, String(key)) : null);-- const stats = {};-- const bump = (source, what, n = 1) => { stats[source] = stats[source] || { imported: 0, unchanged: 0, held: 0 }; stats[source][what] += n; };-- const ledger = (source, key, target) => { if (!dryRun) db.prepare('INSERT OR REPLACE INTO import_ledger (source, source_key, target, imported_at) VALUES (?, ?, ?, ?)').run(source, String(key), target, iso(now)); };-- const hold = (source, key, reason, detail = null) => {-- if (holdGet(source, key)) return bump(source, 'unchanged');-- if (!dryRun) db.prepare('INSERT OR REPLACE INTO import_holds (source, source_key, reason, detail, held_at) VALUES (?, ?, ?, ?, ?)').run(source, String(key), reason, detail, iso(now));-- bump(source, 'held');-- };-- /** Apply once per (source, key, content): unchanged content is skipped on re-runs. */-- const once = (source, key, content, apply) => {-- const h = sha256(content);-- const prev = ledgerGet(source, key);-- if (prev && prev.target.endsWith(`#${h}`)) return bump(source, 'unchanged');-- const target = dryRun ? 'dry-run' : apply();-- ledger(source, key, `${target}#${h}`);-- bump(source, 'imported');-- };---- // ── site_settings ──-- const settings = hasTable(live, 'site_settings') ? Object.fromEntries(live.prepare("SELECT key, value FROM site_settings WHERE key LIKE 'ai_%'").all().map(r => [r.key, r.value])) : {};-- const envAdvice = [];-- for (const [k, envName] of [['ai_provider', 'AI_PROVIDER'], ['ai_base_url', 'AI_BASE_URL'], ['ai_model', 'AI_MODEL'], ['ai_enabled', 'AI_ENABLED'], ['ai_input_cost_per_mtok', 'AI_INPUT_COST_PER_MTOK'], ['ai_output_cost_per_mtok', 'AI_OUTPUT_COST_PER_MTOK']]) {-- if (settings[k] != null && settings[k] !== '') { envAdvice.push(`${envName}=${settings[k]}`); hold('live.site_settings', k, 'environment', `set ${envName} in /etc/openvibe/ai.env`); }-- }-- if (settings.ai_api_key) {-- envAdvice.push('AI_API_KEY=<copy the value of Live\'s ai_api_key setting; not printed>');-- hold('live.site_settings', 'ai_api_key', 'secret', 'secrets are environment references (env:AI_API_KEY); set it in /etc/openvibe/ai.env');-- }-- for (const role of ROLES) {-- const m = String(settings[`ai_model_${role}`] || '').trim();-- if (!m) continue;-- once('live.site_settings', `ai_model_${role}`, m, () => {-- const cur = registry.getRoute(`live.${role}`);-- const v = registry.createRouteVersion(`live.${role}`, { primary: { provider: 'shared', model: m }, fallbacks: cur ? cur.fallbacks : [] }, { actor: 'import' });-- return `route:live.${role}@${v.version}`;-- });-- }-- if (settings.ai_pricing_json) {-- let table = null;-- try { table = JSON.parse(settings.ai_pricing_json); } catch { table = null; }-- if (!table || typeof table !== 'object') hold('live.site_settings', 'ai_pricing_json', 'unparseable', 'not JSON');-- else {-- for (const [model, p] of Object.entries(table)) {-- if (model === 'default' || !p || typeof p !== 'object') { hold('live.site_settings', `ai_pricing_json.${model}`, 'environment', 'prefix/default pricing stays in AI_PRICING_JSON'); continue; }-- once('live.site_settings', `ai_pricing_json.${model}`, p, () => {-- registry.upsertModel({ provider_key: 'shared', model_key: model, type: 'chat', cost: { in_per_mtok: Number(p.in), out_per_mtok: Number(p.out), cached_per_mtok: p.cached != null ? Number(p.cached) : null }, metadata: { imported_from: 'live.ai_pricing_json' } }, { actor: 'import' });-- return `model:shared/${model}`;-- });-- }-- }-- }-- const cap = Number(settings.ai_max_cost_usd_per_day);-- if (cap > 0) once('live.site_settings', 'ai_max_cost_usd_per_day', cap, () => `quota:${quotas.upsert({ scope_type: 'global', window: 'day', max_cost_usd: cap }, { actor: 'import', origin: 'import' }).id}`);-- const vcap = Number(settings.ai_viewers_global_cap_usd_per_day);-- if (vcap > 0) once('live.site_settings', 'ai_viewers_global_cap_usd_per_day', vcap, () => `quota:${quotas.upsert({ scope_type: 'global', window: 'day', max_cost_usd: vcap, workflow_prefix: 'live.viewers.' }, { actor: 'import', origin: 'import' }).id}`);---- // ── ai_usage -> usage_daily (+ today's windows) ──-- if (hasTable(live, 'ai_usage')) {-- const cols = live.prepare('PRAGMA table_info(ai_usage)').all().map(c => c.name);-- const has = (c) => cols.includes(c);-- const rows = live.prepare(`SELECT id, kind, model, input_tokens, output_tokens, cost_usd, created_at,-- ${has('owner_user_id') ? 'owner_user_id' : 'NULL AS owner_user_id'}, ${has('cached_tokens') ? 'cached_tokens' : '0 AS cached_tokens'}, ${has('provider') ? 'provider' : "'shared' AS provider"}-- FROM ai_usage ORDER BY id`).all();-- const today = iso(now).slice(0, 10);-- const dayStart = Math.floor(now / 86400000) * 86400;-- const insDaily = db && !dryRun ? db.prepare(`INSERT INTO usage_daily (day, requester, attribution, workflow_key, provider_key, model_key, requests, tokens_in, tokens_out, tokens_cached, cost_usd)-- VALUES (?, 'service:live', ?, ?, ?, ?, 1, ?, ?, ?, ?)-- ON CONFLICT(day, requester, attribution, workflow_key, provider_key, model_key) DO UPDATE SET requests = requests + 1,-- tokens_in = tokens_in + excluded.tokens_in, tokens_out = tokens_out + excluded.tokens_out, tokens_cached = tokens_cached + excluded.tokens_cached, cost_usd = cost_usd + excluded.cost_usd`) : null;-- const insCounter = db && !dryRun ? db.prepare(`INSERT INTO usage_counters (scope_type, scope_id, window, window_start, workflow_prefix, requests, tokens, cost_usd) VALUES (?, ?, 'day', ?, '', 1, ?, ?)-- ON CONFLICT(scope_type, scope_id, window, window_start, workflow_prefix) DO UPDATE SET requests = requests + 1, tokens = tokens + excluded.tokens, cost_usd = cost_usd + excluded.cost_usd`) : null;-- const tx = (fn) => (db && !dryRun ? db.transaction(fn)() : fn());-- tx(() => {-- for (const r of rows) {-- if (ledgerGet('live.ai_usage', r.id)) { bump('live.ai_usage', 'unchanged'); continue; }-- const day = String(r.created_at || '').slice(0, 10);-- if (!/^\d{4}-\d{2}-\d{2}$/.test(day)) { hold('live.ai_usage', r.id, 'unrepresentable', 'no valid created_at'); continue; }-- const wf = KIND_MAP[r.kind] || 'live.complete';-- const attribution = r.owner_user_id ? `live:user:${r.owner_user_id}` : '';-- const provider = r.provider === 'byo' ? 'live-byo' : 'shared';-- const tokens = (Number(r.input_tokens) || 0) + (Number(r.output_tokens) || 0);-- if (!dryRun) {-- insDaily.run(day, attribution, wf, provider, r.model || '', Number(r.input_tokens) || 0, Number(r.output_tokens) || 0, Number(r.cached_tokens) || 0, Number(r.cost_usd) || 0);-- if (day === today && provider === 'shared') {-- insCounter.run('global', '*', dayStart, tokens, Number(r.cost_usd) || 0);-- insCounter.run('service', 'live', dayStart, tokens, Number(r.cost_usd) || 0);-- if (attribution) insCounter.run('attribution', attribution, dayStart, tokens, Number(r.cost_usd) || 0);-- }-- }-- ledger('live.ai_usage', r.id, `usage_daily:${day}/${wf}#`);-- bump('live.ai_usage', 'imported');-- }-- });-- }---- // ── channel_ai_config -> per-streamer AI-viewer budgets; BYO keys held ──-- if (hasTable(live, 'channel_ai_config')) {-- for (const c of live.prepare('SELECT user_id, use_shared_key, daily_budget_cents, byo_key, byo_base_url FROM channel_ai_config').all()) {-- if (Number(c.use_shared_key) === 1 || c.use_shared_key == null) {-- const usd = (Number(c.daily_budget_cents) || 0) / 100;-- if (usd <= 0) { bump('live.channel_ai_config', 'unchanged'); continue; }-- once('live.channel_ai_config', c.user_id, usd, () => `quota:${quotas.upsert({ scope_type: 'attribution', scope_id: `live:user:${c.user_id}`, window: 'day', max_cost_usd: usd, workflow_prefix: 'live.viewers.' }, { actor: 'import', origin: 'import' }).id}`);-- } else {-- let host = '';-- try { host = c.byo_base_url ? new URL(c.byo_base_url).host : ''; } catch { host = ''; }-- hold('live.channel_ai_config', c.user_id, 'byo_provider', `streamer ${c.user_id} uses their own ${c.byo_key ? 'key' : 'self-hosted endpoint'}${host ? ` (${host})` : ''}; BYO providers keep running in Live (llm.js provider override) until per-actor provider secrets exist here`);-- }-- }-- }---- // ── Whole-table holds ──-- const tableHold = (table, reason, detail) => {-- if (!hasTable(live, table)) return;-- const n = live.prepare(`SELECT COUNT(*) AS n FROM ${table}`).get().n;-- hold(`live.${table}`, '*', reason, `${n} row(s): ${detail}`);-- };-- tableHold('translations', 'unrepresentable', 'keyed by sha1(from|to|text) without the source text, so they cannot become input-hashed cache entries; Live keeps them as its local read-through cache and new translations are cached here per service');-- tableHold('ai_timeline_cache', 'stays_in_live', 'an assembled Live read model (no model output); Live keeps serving it');-- tableHold('ai_chatbot_configs', 'superseded', 'Live already migrated these rows into channel_ai_config');---- live.close();-- if (db) db.close();-- const report = { dry_run: dryRun, ai_db: aiPath, sources: stats, environment: envAdvice };-- log(JSON.stringify(report, null, 2));-- return report;--}----if (require.main === module) {-- require('dotenv').config();-- let args;-- try { args = parseArgs(process.argv.slice(2)); } catch (e) { console.error(e.message); process.exit(2); }-- if (args.help || !args.liveDb) {-- console.error('usage: node scripts/import-from-live.js --live-db <snapshot.db> [--ai-db <ai.db>] [--dry-run]');-- process.exit(args.help ? 0 : 2);-- }-- try { run({ liveDb: args.liveDb, aiDb: args.aiDb, dryRun: args.dryRun }); } catch (e) { console.error(`[import] ${e.message}`); process.exit(1); }--}----module.exports = { run, KIND_MAP };nnew file mode 100644@@@ -0,0 +1,26 @@++#!/usr/bin/env node++'use strict';++/**++ * The one-time move of AI's SQLite database (AI_DB_PATH) into its PostgreSQL schema (ADR-035; the procedure is++ * openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by openvibe-sdk/db runSqliteMigration).++ *++ * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]++ *++ * Applies migrations/ as the owner (DATABASE_DIRECT_URL), copies every table into emptied tables, verifies row counts++ * and checksums, and exits 1 unless everything verified. The SQLite file is opened read-only. Every table keeps its++ * name and columns.++ */++require('dotenv').config();++const { runSqliteMigration } = require('openvibe-sdk/db');++const configLib = require('../server/config');++const { MIGRATIONS } = require('../server/db');++++const TABLES = {};++++if (require.main === module) {++ const config = (configLib.load || configLib.loadConfig)();++ runSqliteMigration({ service: 'ai', sqlite: config.dbPath, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES })++ .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });++}++++module.exports = { TABLES };@@@ -33,43 +33,43 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = creaconst defRead = auth.requireCap(CAPS.workflowManage, CAPS.runCreate, CAPS.usageRead);const defWrite = auth.requireCap(CAPS.workflowManage);const usageRead = auth.requireCap(CAPS.usageRead);-- const wrap = (fn) => (req, res) => { try { const out = fn(req, res); if (out !== undefined) res.json(out); } catch (err) { sendError(res, err, req.ov, log); } };++ const wrap = (fn) => async (req, res) => { try { const out = await fn(req, res); if (out !== undefined) res.json(out); } catch (err) { sendError(res, err, req.ov, log); } };const actor = (req) => ({ actor: req.principal.sub, trace: req.ov.traceId });const view = ops.providerView;const body = (req) => (req.body && typeof req.body === 'object' ? req.body : {});-- r.get('/api/v1/status', usageRead, wrap(() => ops.status()));++ r.get('/api/v1/status', usageRead, wrap(async () => await ops.status()));// Providers-- r.get('/api/v1/providers', provRead, wrap(() => ({ providers: registry.listProviders().map(view) })));-- r.get('/api/v1/providers/:key', provRead, wrap((req) => {-- const p = registry.getProvider(req.params.key);++ r.get('/api/v1/providers', provRead, wrap(async () => ({ providers: (await Promise.all((await registry.listProviders()).map(view))) })));++ r.get('/api/v1/providers/:key', provRead, wrap(async (req) => {++ const p = await registry.getProvider(req.params.key);if (!p) throw new AiError(404, 'ai.not_found', 'no such provider');-- return { provider: view(p), models: registry.listModels({ provider: p.key }), stats: pool.stats[p.key] || null };++ return { provider: await view(p), models: await registry.listModels({ provider: p.key }), stats: pool.stats[p.key] || null };}));-- r.post('/api/v1/providers', provWrite, wrap((req, res) => {++ r.post('/api/v1/providers', provWrite, wrap(async (req, res) => {const b = body(req);if ('api_key' in b || 'secret' in b) throw new AiError(422, 'ai.invalid', 'send secret_ref (env:NAME), never a secret value');res.status(201);-- return { provider: view(registry.upsertProvider(b, { ...actor(req), origin: 'admin' })) };++ return { provider: await view(await registry.upsertProvider(b, { ...actor(req), origin: 'admin' })) };}));-- r.patch('/api/v1/providers/:key', provWrite, wrap((req) => {++ r.patch('/api/v1/providers/:key', provWrite, wrap(async (req) => {const b = body(req);if ('api_key' in b || 'secret' in b) throw new AiError(422, 'ai.invalid', 'send secret_ref (env:NAME), never a secret value');-- if (!registry.getProvider(req.params.key)) throw new AiError(404, 'ai.not_found', 'no such provider');-- return { provider: view(registry.upsertProvider({ ...b, key: req.params.key }, { ...actor(req), origin: 'admin' })) };++ if (!await registry.getProvider(req.params.key)) throw new AiError(404, 'ai.not_found', 'no such provider');++ return { provider: await view(await registry.upsertProvider({ ...b, key: req.params.key }, { ...actor(req), origin: 'admin' })) };}));for (const [action, status] of [['disable', 'disabled'], ['enable', 'active']]) {-- r.post(`/api/v1/providers/:key/${action}`, provWrite, wrap((req) => ({ provider: view(ops.setProviderStatus(req.params.key, status, actor(req))) })));++ r.post(`/api/v1/providers/:key/${action}`, provWrite, wrap(async (req) => ({ provider: await view(await ops.setProviderStatus(req.params.key, status, actor(req))) })));}-- r.post('/api/v1/providers/:key/reset', provWrite, wrap((req) => ({ health: ops.resetCircuit(req.params.key, actor(req)) })));++ r.post('/api/v1/providers/:key/reset', provWrite, wrap(async (req) => ({ health: await ops.resetCircuit(req.params.key, actor(req)) })));// Models-- r.get('/api/v1/models', provRead, wrap((req) => ({ models: registry.listModels({ provider: req.query.provider }) })));-- r.post('/api/v1/models', provWrite, wrap((req, res) => { res.status(201); return { model: registry.upsertModel(body(req), actor(req)) }; }));-- r.patch('/api/v1/models/:provider/:model', provWrite, wrap((req) => {-- if (!registry.getModel(req.params.provider, req.params.model)) throw new AiError(404, 'ai.not_found', 'no such model');-- return { model: registry.upsertModel({ ...body(req), provider_key: req.params.provider, model_key: req.params.model }, actor(req)) };++ r.get('/api/v1/models', provRead, wrap(async (req) => ({ models: await registry.listModels({ provider: req.query.provider }) })));++ r.post('/api/v1/models', provWrite, wrap(async (req, res) => { res.status(201); return { model: await registry.upsertModel(body(req), actor(req)) }; }));++ r.patch('/api/v1/models/:provider/:model', provWrite, wrap(async (req) => {++ if (!await registry.getModel(req.params.provider, req.params.model)) throw new AiError(404, 'ai.not_found', 'no such model');++ return { model: await registry.upsertModel({ ...body(req), provider_key: req.params.provider, model_key: req.params.model }, actor(req)) };}));// Versioned definitions@@@ -79,17 +79,17 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = creaworkflows: { list: registry.listWorkflows, get: registry.getWorkflow, create: registry.createWorkflowVersion, one: 'workflow' },};for (const [plural, k] of Object.entries(kinds)) {-- r.get(`/api/v1/${plural}`, defRead, wrap((req) => ({ [plural]: k.list({ history: req.query.history === '1', namespace: req.query.namespace }) })));-- r.get(`/api/v1/${plural}/:key`, defRead, wrap((req) => {-- const latest = k.get(req.params.key, req.query.version ? Number(req.query.version) : undefined);++ r.get(`/api/v1/${plural}`, defRead, wrap(async (req) => ({ [plural]: await k.list({ history: req.query.history === '1', namespace: req.query.namespace }) })));++ r.get(`/api/v1/${plural}/:key`, defRead, wrap(async (req) => {++ const latest = await k.get(req.params.key, req.query.version ? Number(req.query.version) : undefined);if (!latest) throw new AiError(404, 'ai.not_found', `no ${k.one} ${req.params.key}`);-- const versions = k.list({ history: true }).filter(x => x.key === req.params.key).map(x => ({ version: x.version, status: x.status, created_by: x.created_by, created_at: x.created_at }));++ const versions = (await k.list({ history: true })).filter(x => x.key === req.params.key).map(x => ({ version: x.version, status: x.status, created_by: x.created_by, created_at: x.created_at }));return { [k.one]: latest, versions };}));// An edit is always a new version; the previous one stays readable.-- r.post(`/api/v1/${plural}/:key/versions`, defWrite, wrap((req, res) => {++ r.post(`/api/v1/${plural}/:key/versions`, defWrite, wrap(async (req, res) => {try {-- const created = k.create(req.params.key, body(req), actor(req));++ const created = await k.create(req.params.key, body(req), actor(req));res.status(201);return { [k.one]: created };} catch (err) {@@@ -97,16 +97,16 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = creathrow new AiError(422, 'ai.invalid', err.message);}}));-- r.post(`/api/v1/${plural}/:key/versions/:version/status`, defWrite, wrap((req) => ({-- [k.one]: registry.setStatus(k.one, req.params.key, Number(req.params.version), body(req).status, actor(req)),++ r.post(`/api/v1/${plural}/:key/versions/:version/status`, defWrite, wrap(async (req) => ({++ [k.one]: await registry.setStatus(k.one, req.params.key, Number(req.params.version), body(req).status, actor(req)),})));}// Quotas, usage, logs, audit, cache-- r.get('/api/v1/quotas', usageRead, wrap(() => ({ quotas: quotas.list(), current: quotas.counters() })));-- r.post('/api/v1/quotas', provWrite, wrap((req, res) => { res.status(201); return { quota: quotas.upsert(body(req), { ...actor(req), origin: 'admin' }) }; }));-- r.get('/api/v1/usage', usageRead, wrap((req) => ({ usage: quotas.usage({ from: req.query.from, to: req.query.to, requester: req.query.requester }) })));-- r.get('/api/v1/requests', usageRead, wrap((req) => {++ r.get('/api/v1/quotas', usageRead, wrap(async () => ({ quotas: await quotas.list(), current: await quotas.counters() })));++ r.post('/api/v1/quotas', provWrite, wrap(async (req, res) => { res.status(201); return { quota: await quotas.upsert(body(req), { ...actor(req), origin: 'admin' }) }; }));++ r.get('/api/v1/usage', usageRead, wrap(async (req) => ({ usage: await quotas.usage({ from: req.query.from, to: req.query.to, requester: req.query.requester }) })));++ r.get('/api/v1/requests', usageRead, wrap(async (req) => {const where = [];const args = [];if (req.query.provider) { where.push('provider_key = ?'); args.push(req.query.provider); }@@@ -116,11 +116,11 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = crea// Raw debug fields only for provider managers who ask for them (and only exist when opted in).const debug = req.query.debug === '1' && auth.principalHas(req, CAPS.providerManage);const cols = debug ? '*' : 'id, run_id, seq, operation, provider_key, model_key, route_key, route_version, status, skip_reason, fallback, prompt_hash, input_hash, output_hash, tokens_in, tokens_out, tokens_cached, tokens_estimated, cost_usd, latency_ms, error, created_at';-- return { requests: db.prepare(`SELECT ${cols} FROM requests ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(500, Number(req.query.limit) || 100)) };++ return { requests: await db.prepare(`SELECT ${cols} FROM requests ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(500, Number(req.query.limit) || 100)) };}));-- r.get('/api/v1/audit', usageRead, wrap((req) => ({ audit: registry.listAudit({ action: req.query.action, targetType: req.query.target_type, targetId: req.query.target_id, limit: Number(req.query.limit) || 100, before: req.query.before }) })));-- r.get('/api/v1/cache', usageRead, wrap(() => ({ cache: cache.stats() })));-- r.delete('/api/v1/cache', provWrite, wrap((req) => ({ removed: ops.purgeCache(req.query.workflow, actor(req)) })));++ r.get('/api/v1/audit', usageRead, wrap(async (req) => ({ audit: await registry.listAudit({ action: req.query.action, targetType: req.query.target_type, targetId: req.query.target_id, limit: Number(req.query.limit) || 100, before: req.query.before }) })));++ r.get('/api/v1/cache', usageRead, wrap(async () => ({ cache: await cache.stats() })));++ r.delete('/api/v1/cache', provWrite, wrap(async (req) => ({ removed: await ops.purgeCache(req.query.workflow, actor(req)) })));return r;}@@@ -31,11 +31,11 @@ function attributionQuotasRouter({ db, quotas, auth, clock = { now: () => Date.nreturn m[0];}-- function view(row) {++ async function view(row) {const now = Math.floor(clock.now() / 1000);const len = WINDOW_S[row.window];const start = Math.floor(now / len) * len;-- const c = db.prepare('SELECT requests, cost_usd FROM usage_counters WHERE scope_type = ? AND scope_id = ? AND window = ? AND window_start = ? AND workflow_prefix = ?')++ const c = await db.prepare('SELECT requests, cost_usd FROM usage_counters WHERE scope_type = ? AND scope_id = ? AND "window" = ? AND window_start = ? AND workflow_prefix = ?').get('attribution', row.scope_id, row.window, start, row.workflow_prefix || '') || { requests: 0, cost_usd: 0 };return {attribution: row.scope_id, window: row.window, workflow_prefix: row.workflow_prefix || null,@@@ -44,26 +44,26 @@ function attributionQuotasRouter({ db, quotas, auth, clock = { now: () => Date.nwindow_resets_at: iso((start + len) * 1000), updated_at: row.updated_at,};}-- const rows = (attr) => db.prepare("SELECT * FROM quotas WHERE scope_type = 'attribution' AND scope_id = ? AND status = 'active' ORDER BY window, workflow_prefix").all(attr);++ const rows = async (attr) => await db.prepare("SELECT * FROM quotas WHERE scope_type = 'attribution' AND scope_id = ? AND status = 'active' ORDER BY \"window\", workflow_prefix").all(attr);-- r.put('/api/v1/attribution-quotas/:attribution', guard, (req, res) => {++ r.put('/api/v1/attribution-quotas/:attribution', guard, async (req, res) => {try {const attr = own(req);const body = req.body || {};const v = validate('ai.attribution-quota-put@1', body);if (!v.valid) throw new AiError(422, 'input.invalid', 'body does not match ai.attribution-quota-put@1', { errors: v.errors });-- const row = quotas.upsert({ scope_type: 'attribution', scope_id: attr, window: body.window, max_cost_usd: body.max_cost_usd, max_requests: body.max_requests, workflow_prefix: body.workflow_prefix || null, status: 'active' },++ const row = await quotas.upsert({ scope_type: 'attribution', scope_id: attr, window: body.window, max_cost_usd: body.max_cost_usd, max_requests: body.max_requests, workflow_prefix: body.workflow_prefix || null, status: 'active' },{ actor: req.principal.sub, origin: `service:${req.principal.subject.id}`, trace: req.ov && req.ov.traceId });-- res.set('Cache-Control', 'no-store').json(view(row));++ res.set('Cache-Control', 'no-store').json(await view(row));} catch (err) { sendError(res, err, req.ov); }});-- r.get('/api/v1/attribution-quotas/:attribution', guard, (req, res) => {-- try { res.set('Cache-Control', 'no-store').json({ quotas: rows(own(req)).map(view) }); } catch (err) { sendError(res, err, req.ov); }++ r.get('/api/v1/attribution-quotas/:attribution', guard, async (req, res) => {++ try { res.set('Cache-Control', 'no-store').json({ quotas: (await Promise.all((await rows(own(req))).map(view))) }); } catch (err) { sendError(res, err, req.ov); }});-- r.delete('/api/v1/attribution-quotas/:attribution', guard, (req, res) => {++ r.delete('/api/v1/attribution-quotas/:attribution', guard, async (req, res) => {try {const attr = own(req);-- const n = db.prepare("UPDATE quotas SET status = 'disabled', updated_at = ? WHERE scope_type = 'attribution' AND scope_id = ? AND status = 'active'").run(iso(clock.now()), attr).changes;++ const n = (await db.prepare("UPDATE quotas SET status = 'disabled', updated_at = ? WHERE scope_type = 'attribution' AND scope_id = ? AND status = 'active'").run(iso(clock.now()), attr)).changes;res.status(n ? 204 : 404).end();} catch (err) { sendError(res, err, req.ov); }});@@@ -46,7 +46,7 @@ function runsRouter({ runs, registry, auth, config, log = console }) {async function createAndRespond(req, res, body, waitMs) {checkNamespace(req, String(body.workflow || ''));const idem = body.idempotency_key || req.ov.idempotencyKey || undefined;-- const created = runs.create({ ...body, idempotency_key: idem }, req.principal, { trace: req.ov.traceId, requestId: req.ov.requestId });++ const created = await runs.create({ ...body, idempotency_key: idem }, req.principal, { trace: req.ov.traceId, requestId: req.ov.requestId });const run = await runs.wait(created, waitMs);const status = !created.created ? 200 : (runs.TERMINAL.has(run.status) ? 201 : 202);if (status === 202) res.setHeader('Location', `/api/v1/runs/${run.id}`);@@@ -60,13 +60,13 @@ function runsRouter({ runs, registry, auth, config, log = console }) {} catch (err) { sendError(res, err, req.ov, log); }});-- r.get('/api/v1/runs', read, (req, res) => {++ r.get('/api/v1/runs', read, async (req, res) => {const all = req.query.all === '1' && auth.principalHas(req, CAPS.usageRead);-- res.json({ runs: runs.list(req.principal, { status: req.query.status, workflow: req.query.workflow, target: req.query.target, limit: req.query.limit, before: req.query.before, all }) });++ res.json({ runs: await runs.list(req.principal, { status: req.query.status, workflow: req.query.workflow, target: req.query.target, limit: req.query.limit, before: req.query.before, all }) });});-- function owned(req) {-- const run = runs.get(req.params.id);++ async function owned(req) {++ const run = await runs.get(req.params.id);if (!run) throw new AiError(404, 'run.not_found', 'no such run');const s = req.principal.subject;const mine = run.requester.type === s.type && run.requester.id === s.id;@@@ -74,33 +74,33 @@ function runsRouter({ runs, registry, auth, config, log = console }) {return run;}-- r.get('/api/v1/runs/:id', read, (req, res) => {++ r.get('/api/v1/runs/:id', read, async (req, res) => {try {-- const run = owned(req);-- res.json({ run, citations: runs.citations(run.id), requests: runs.requestsFor(run.id) });++ const run = await owned(req);++ res.json({ run, citations: await runs.citations(run.id), requests: await runs.requestsFor(run.id) });} catch (err) { sendError(res, err, req.ov, log); }});-- r.post('/api/v1/runs/:id/cancel', create, (req, res) => {-- try { res.json({ run: runs.cancel(req.params.id, req.principal, { trace: req.ov.traceId, principalHas: has(req) }) }); } catch (err) { sendError(res, err, req.ov, log); }++ r.post('/api/v1/runs/:id/cancel', create, async (req, res) => {++ try { res.json({ run: await runs.cancel(req.params.id, req.principal, { trace: req.ov.traceId, principalHas: has(req) }) }); } catch (err) { sendError(res, err, req.ov, log); }});r.post('/api/v1/runs/:id/retry', create, async (req, res) => {try {-- const created = runs.retry(req.params.id, req.principal, { trace: req.ov.traceId, requestId: req.ov.requestId, principalHas: has(req) });++ const created = await runs.retry(req.params.id, req.principal, { trace: req.ov.traceId, requestId: req.ov.requestId, principalHas: has(req) });const wait = Math.max(0, Math.min(Number(req.query.wait) || 0, config.runs.maxWaitMs));const run = await runs.wait(created, wait);res.status(runs.TERMINAL.has(run.status) ? 201 : 202).json({ run });} catch (err) { sendError(res, err, req.ov, log); }});-- r.get('/api/v1/runs/:id/citations', read, (req, res) => {-- try { const run = owned(req); res.json({ citations: runs.citations(run.id) }); } catch (err) { sendError(res, err, req.ov, log); }++ r.get('/api/v1/runs/:id/citations', read, async (req, res) => {++ try { const run = await owned(req); res.json({ citations: await runs.citations(run.id) }); } catch (err) { sendError(res, err, req.ov, log); }});-- r.post('/api/v1/runs/:id/citations', create, (req, res) => {++ r.post('/api/v1/runs/:id/citations', create, async (req, res) => {try {-- const run = owned(req);++ const run = await owned(req);const s = req.principal.subject;if (run.requester.type !== s.type || run.requester.id !== s.id) throw new AiError(403, 'capability.denied', 'only the requester may attach citations');const list = Array.isArray(req.body && req.body.citations) ? req.body.citations : null;@@@ -109,9 +109,9 @@ function runsRouter({ runs, registry, auth, config, log = console }) {if (!c || typeof c.source_type !== 'string' || !/^[a-z][a-z0-9_.-]{1,63}$/.test(c.source_type)) throw new AiError(422, 'input.invalid', 'every citation needs a source_type');for (const k of ['url', 'title', 'author', 'snippet', 'source_id']) if (c[k] != null && String(c[k]).length > 4000) throw new AiError(422, 'input.invalid', `${k} too long`);}-- runs.addCitations(run.id, list.map(c => ({ ...c, ordinal: null, provenance: { ...(c.provenance || {}), attached: true } })), req.principal.sub);-- registry.audit(req.principal.sub, 'run.citations.attach', 'run', run.id, { trace: req.ov.traceId, metadata: { count: list.length } });-- res.status(201).json({ citations: runs.citations(run.id) });++ await runs.addCitations(run.id, list.map(c => ({ ...c, ordinal: null, provenance: { ...(c.provenance || {}), attached: true } })), req.principal.sub);++ await registry.audit(req.principal.sub, 'run.citations.attach', 'run', run.id, { trace: req.ov.traceId, metadata: { count: list.length } });++ res.status(201).json({ citations: await runs.citations(run.id) });} catch (err) { sendError(res, err, req.ov, log); }});@@@ -56,7 +56,7 @@ function createKeyStore({ urls = [], pem = null, fetchImpl = globalThis.fetch, lreturn null;}-- function start() {++ async function start() {if (pem) return Promise.resolve(key);const attempt = async () => {const k = await fetchOnce();@@@ -68,7 +68,7 @@ function createKeyStore({ urls = [], pem = null, fetchImpl = globalThis.fetch, l};refreshTimer = setInterval(() => { fetchOnce().catch(() => {}); }, 6 * 60 * 60 * 1000);refreshTimer.unref?.();-- return attempt();++ return await attempt();}function stop() { clearTimeout(retryTimer); clearInterval(refreshTimer); }@@@ -27,33 +27,33 @@ function createCache(db, { clock = { now: () => Date.now() } } = {}) {return sha256({ scope, w: workflowKey, wv: workflowVersion, tv: templateVersion || null, r: routeKey || null, rv: routeVersion || null, i: inputHash });}-- function get(key, scope) {-- const row = db.prepare('SELECT * FROM cache_entries WHERE cache_key = ?').get(key);++ async function get(key, scope) {++ const row = await db.prepare('SELECT * FROM cache_entries WHERE cache_key = ?').get(key);if (!row) return null;if (row.scope !== scope) return null; // belt and braces: never cross scope-- if (row.expires_at <= clock.now()) { db.prepare('DELETE FROM cache_entries WHERE cache_key = ?').run(key); return null; }-- db.prepare('UPDATE cache_entries SET hits = hits + 1 WHERE cache_key = ?').run(key);++ if (row.expires_at <= clock.now()) { await db.prepare('DELETE FROM cache_entries WHERE cache_key = ?').run(key); return null; }++ await db.prepare('UPDATE cache_entries SET hits = hits + 1 WHERE cache_key = ?').run(key);return { ...row, output: parseJson(row.output, null) };}-- function put({ key, scope, privacy, workflowKey, workflowVersion, templateVersion, routeKey, routeVersion, model, inputHash, output, runId, ttlSec }) {-- db.prepare(`INSERT INTO cache_entries (cache_key, scope, workflow_key, workflow_version, template_version, route_key, route_version, model_key, input_hash, privacy, output, run_id, hits, created_at, expires_at)++ async function put({ key, scope, privacy, workflowKey, workflowVersion, templateVersion, routeKey, routeVersion, model, inputHash, output, runId, ttlSec }) {++ await db.prepare(`INSERT INTO cache_entries (cache_key, scope, workflow_key, workflow_version, template_version, route_key, route_version, model_key, input_hash, privacy, output, run_id, hits, created_at, expires_at)VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0, ?, ?)ON CONFLICT(cache_key) DO UPDATE SET output = excluded.output, run_id = excluded.run_id, model_key = excluded.model_key, created_at = excluded.created_at, expires_at = excluded.expires_at`).run(key, scope, workflowKey, workflowVersion, templateVersion || null, routeKey || null, routeVersion || null, model || null, inputHash, privacy,JSON.stringify(output), runId, iso(clock.now()), clock.now() + ttlSec * 1000);}-- function stats() {-- return db.prepare('SELECT workflow_key, privacy, COUNT(*) AS entries, SUM(hits) AS hits FROM cache_entries WHERE expires_at > ? GROUP BY workflow_key, privacy ORDER BY entries DESC').all(clock.now());++ async function stats() {++ return await db.prepare('SELECT workflow_key, privacy, COUNT(*) AS entries, SUM(hits) AS hits FROM cache_entries WHERE expires_at > ? GROUP BY workflow_key, privacy ORDER BY entries DESC').all(clock.now());}-- function purge({ workflow } = {}) {-- const r = workflow ? db.prepare('DELETE FROM cache_entries WHERE workflow_key = ?').run(workflow) : db.prepare('DELETE FROM cache_entries').run();++ async function purge({ workflow } = {}) {++ const r = workflow ? await db.prepare('DELETE FROM cache_entries WHERE workflow_key = ?').run(workflow) : await db.prepare('DELETE FROM cache_entries').run();return r.changes;}-- function prune() { return db.prepare('DELETE FROM cache_entries WHERE expires_at <= ?').run(clock.now()).changes; }++ async function prune() { return (await db.prepare('DELETE FROM cache_entries WHERE expires_at <= ?').run(clock.now())).changes; }return { scopeFor, keyFor, get, put, stats, purge, prune };}@@@ -59,6 +59,9 @@ function load(env = process.env) {networkPublicKey: env.OV_NETWORK_PUBLIC_KEY ? env.OV_NETWORK_PUBLIC_KEY.replace(/\\n/g, '\n') : null,audience: 'openvibe.ai',++ // PostgreSQL (ADR-035): DATABASE_URL serves (PgBouncer), DATABASE_DIRECT_URL migrates (owner role).++ db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '' },++ // The SQLite file of releases before the switch: read once by scripts/migrate-to-postgres.js.dbPath: env.AI_DB_PATH || './data/ai.db',// ── Operator console (server/console): OpenVibe.Network SSO (authorization code + PKCE S256@@@ -109,20 +109,20 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,const sessions = createSessions({ db, config, clock, secret: secret || 'unavailable' });/** Every secret value this process can resolve, so that no page can ever carry one (belt and braces). */-- function secretValues() {++ async function secretValues() {const vals = new Set([clientSecret(), secret]);-- for (const p of registry.listProviders()) vals.add(resolveSecret(p.secret_ref, env));++ for (const p of await registry.listProviders()) vals.add(resolveSecret(p.secret_ref, env));return [...vals].filter((v) => v && v.length >= 8);}-- function scrub(text) {++ async function scrub(text) {let out = text;-- for (const v of secretValues()) for (const form of new Set([v, esc(v)])) out = out.split(form).join('[redacted]');++ for (const v of await secretValues()) for (const form of new Set([v, esc(v)])) out = out.split(form).join('[redacted]');return out;}-- const send = (res, status, text) => res.status(status).type('html').send(scrub(text));++ const send = async (res, status, text) => res.status(status).type('html').send(await scrub(text));const who = (req) => ({ actor: req.staff.subject, trace: req.ov ? req.ov.traceId : null });const common = (req) => ({ staff: req.staff, csrf: req.staff ? req.staff.csrf : '' });-- const audit = (actor, action, targetType, targetId, metadata, req) => registry.audit(actor || 'anonymous', action, targetType, targetId, { trace: req && req.ov ? req.ov.traceId : null, metadata });++ const audit = async (actor, action, targetType, targetId, metadata, req) => await registry.audit(actor || 'anonymous', action, targetType, targetId, { trace: req && req.ov ? req.ov.traceId : null, metadata });const notice = (req) => {const base = NOTICES[req.query.done];if (!base) return null;@@@ -130,28 +130,28 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,};// ── Every console and sign-in response ───────────────────-- r.use(scoped, (req, res, next) => {++ r.use(scoped, async (req, res, next) => {res.setHeader('Content-Security-Policy', CSP);res.setHeader('X-Frame-Options', 'DENY');res.setHeader('Referrer-Policy', 'no-referrer');res.setHeader('X-Robots-Tag', 'noindex, nofollow');res.setHeader('Cache-Control', 'no-store');-- if (unavailable) return send(res, 503, pages.message({ title: 'Console unavailable', text: 'The operator console is not configured on this host. The operator must set its environment (see .env.example, "Operator console").' }));++ if (unavailable) return await send(res, 503, pages.message({ title: 'Console unavailable', text: 'The operator console is not configured on this host. The operator must set its environment (see .env.example, "Operator console").' }));return next();});r.use(scoped, express.urlencoded({ extended: false, limit: '16kb', parameterLimit: 50 }));// ── Session -> req.staff ─────────────────────────────────-- r.use(scoped, (req, res, next) => {++ r.use(scoped, async (req, res, next) => {req.staff = null;-- const row = sessions.read(req);++ const row = await sessions.read(req);if (!row) return next();const caps = consoleCapabilities(row.staff);if (!caps.includes(CAPS.usageRead)) {-- sessions.revoke(row);++ await sessions.revoke(row);sessions.clear(res);-- audit(row.subject, 'console.request.refused', 'console', 'session', { reason: 'the session no longer maps to ai.usage.read' }, req);-- return send(res, 403, pages.message({ title: 'Not authorized', text: 'Your account is not AI operator staff any more. The session was ended.' }));++ await audit(row.subject, 'console.request.refused', 'console', 'session', { reason: 'the session no longer maps to ai.usage.read' }, req);++ return await send(res, 403, pages.message({ title: 'Not authorized', text: 'Your account is not AI operator staff any more. The session was ended.' }));}req.staff = {subject: row.subject, username: row.username, role: staffMap.effectiveRole(row.staff), caps, csrf: row.csrf, session: row,@@@ -162,18 +162,18 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,/** Staff only, holding `cap` through STAFF_TO_AI. POSTs also need the CSRF token. */function needs(cap) {-- return (req, res, next) => {++ return async (req, res, next) => {if (!req.staff) {-- if (req.method === 'GET') return send(res, 401, pages.signIn({ next: sanitizeNext(req.originalUrl) }));-- return send(res, 401, pages.signIn({ message: 'Your session has ended. Sign in again, then repeat the action.' }));++ if (req.method === 'GET') return await send(res, 401, pages.signIn({ next: sanitizeNext(req.originalUrl) }));++ return await send(res, 401, pages.signIn({ message: 'Your session has ended. Sign in again, then repeat the action.' }));}if (!req.staff.caps.includes(cap)) {-- if (req.method === 'POST') audit(req.staff.subject, 'console.request.refused', 'console', req.path, { reason: `needs ${cap}` }, req);-- return send(res, 403, pages.message({ title: 'Not authorized', text: `This needs ${cap}.`, ...common(req) }));++ if (req.method === 'POST') await audit(req.staff.subject, 'console.request.refused', 'console', req.path, { reason: `needs ${cap}` }, req);++ return await send(res, 403, pages.message({ title: 'Not authorized', text: `This needs ${cap}.`, ...common(req) }));}if (req.method === 'POST' && !csrfOk(req)) {-- audit(req.staff.subject, 'console.request.refused', 'console', req.path, { reason: 'missing or invalid CSRF token, or a cross-site request' }, req);-- return send(res, 403, pages.message({ title: 'Request refused', text: 'The form was stale or came from another site. Reload the page and try again.', ...common(req) }));++ await audit(req.staff.subject, 'console.request.refused', 'console', req.path, { reason: 'missing or invalid CSRF token, or a cross-site request' }, req);++ return await send(res, 403, pages.message({ title: 'Request refused', text: 'The form was stale or came from another site. Reload the page and try again.', ...common(req) }));}return next();};@@@ -185,10 +185,10 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,if (origin && origin !== baseOrigin) return false;return sameString((req.body || {})._csrf, req.staff.csrf);}-- const wrap = (fn) => (req, res, next) => Promise.resolve().then(() => fn(req, res, next)).catch((e) => {-- if (e instanceof AiError) return send(res, e.status >= 400 && e.status < 600 ? e.status : 400, pages.message({ title: 'Refused', text: `${e.detail || e.message} (${e.code})`, back: req.get('referer') && req.get('referer').startsWith(`${baseOrigin}/console`) ? req.get('referer') : '/console', ...common(req) }));++ const wrap = (fn) => (req, res, next) => Promise.resolve().then(async () => await fn(req, res, next)).catch(async (e) => {++ if (e instanceof AiError) return await send(res, e.status >= 400 && e.status < 600 ? e.status : 400, pages.message({ title: 'Refused', text: `${e.detail || e.message} (${e.code})`, back: req.get('referer') && req.get('referer').startsWith(`${baseOrigin}/console`) ? req.get('referer') : '/console', ...common(req) }));log.error(`[console] ${req.method} ${req.path}: ${e && e.stack ? e.stack : e}`);-- if (!res.headersSent) send(res, 500, pages.message({ title: 'Error', text: 'Something went wrong. Nothing was changed unless the page says so; check the audit log.', ...common(req) }));++ if (!res.headersSent) await send(res, 500, pages.message({ title: 'Error', text: 'Something went wrong. Nothing was changed unless the page says so; check the audit log.', ...common(req) }));return null;});const read = needs(CAPS.usageRead);@@@ -203,9 +203,9 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,r.get('/auth/callback', wrap(async (req, res) => {const flow = sessions.takeFlow(req, res);-- if (req.query.error) return send(res, 400, pages.signIn({ message: `Sign-in was not completed (${String(req.query.error).slice(0, 60)}).` }));++ if (req.query.error) return await send(res, 400, pages.signIn({ message: `Sign-in was not completed (${String(req.query.error).slice(0, 60)}).` }));if (!flow || !req.query.code || !sameString(req.query.state, flow.s)) {-- return send(res, 400, pages.signIn({ message: 'The sign-in could not be verified (expired or mismatched state). Please try again.' }));++ return await send(res, 400, pages.signIn({ message: 'The sign-in could not be verified (expired or mismatched state). Please try again.' }));}let person;try {@@@ -214,69 +214,69 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,person = await sso.exchange({ config, clientSecret: clientSecret(), code: req.query.code, verifier: flow.v, publicKey, now: clock.now(), fetchImpl });} catch (e) {log.warn(`[console] sign-in failed: ${e.message}`);-- return send(res, e.status && e.status < 500 ? 400 : 502, pages.signIn({ message: 'OpenVibe.Network did not confirm the sign-in. Please try again.' }));++ return await send(res, e.status && e.status < 500 ? 400 : 502, pages.signIn({ message: 'OpenVibe.Network did not confirm the sign-in. Please try again.' }));}const ipHash = sessions.ipHash(req.ip);if (person.error) {-- audit('anonymous', 'console.sign_in.refused', 'console', 'sign_in', { reason: person.error, ip_hash: ipHash }, req);-- return send(res, 403, pages.message({ title: 'Not authorized', text: 'This console is only for OpenVibe.Network staff. Your sign-in was recorded.' }));++ await audit('anonymous', 'console.sign_in.refused', 'console', 'sign_in', { reason: person.error, ip_hash: ipHash }, req);++ return await send(res, 403, pages.message({ title: 'Not authorized', text: 'This console is only for OpenVibe.Network staff. Your sign-in was recorded.' }));}const caps = consoleCapabilities(person.staff);const role = staffMap.effectiveRole(person.staff);if (!caps.includes(CAPS.usageRead)) {-- audit(person.subject, 'console.sign_in.refused', 'console', 'sign_in', { reason: `role ${role} does not hold staff.site.view`, role, ip_hash: ipHash }, req);-- return send(res, 403, pages.message({ title: 'Not authorized', text: 'This console is only for OpenVibe.Network staff (admins and the owner). Your sign-in was recorded.' }));++ await audit(person.subject, 'console.sign_in.refused', 'console', 'sign_in', { reason: `role ${role} does not hold staff.site.view`, role, ip_hash: ipHash }, req);++ return await send(res, 403, pages.message({ title: 'Not authorized', text: 'This console is only for OpenVibe.Network staff (admins and the owner). Your sign-in was recorded.' }));}-- db.transaction(() => {-- sessions.create(res, { subject: person.subject, username: person.username, staff: person.staff, ip: req.ip });-- audit(person.subject, 'console.sign_in', 'console', 'sign_in', { role, capabilities: caps, ip_hash: ipHash }, req);-- })();++ await db.tx(async () => {++ await sessions.create(res, { subject: person.subject, username: person.username, staff: person.staff, ip: req.ip });++ await audit(person.subject, 'console.sign_in', 'console', 'sign_in', { role, capabilities: caps, ip_hash: ipHash }, req);++ });return res.redirect(303, flow.n || '/console');}));-- r.post('/auth/logout', read, (req, res) => {-- db.transaction(() => {-- sessions.revoke(req.staff.session);-- audit(req.staff.subject, 'console.sign_out', 'console', 'sign_in', {}, req);-- })();++ r.post('/auth/logout', read, async (req, res) => {++ await db.tx(async () => {++ await sessions.revoke(req.staff.session);++ await audit(req.staff.subject, 'console.sign_out', 'console', 'sign_in', {}, req);++ });sessions.clear(res);res.redirect(303, '/console');});// ── Overview ─────────────────────────────────────────────-- r.get('/console', read, wrap((req, res) => {++ r.get('/console', read, wrap(async (req, res) => {const since = new Date(clock.now() - DAY_MS).toISOString();-- const st = ops.status();++ const st = await ops.status();const problems = [];-- for (const p of registry.listProviders().filter((x) => x.status === 'active' && x.kind !== 'stub')) {-- const v = ops.providerView(p);++ for (const p of (await registry.listProviders()).filter((x) => x.status === 'active' && x.kind !== 'stub')) {++ const v = await ops.providerView(p);if (v.credentials === 'missing') problems.push({ key: p.key, kind: p.kind, problem: `credentials missing (${p.secret_ref || 'no reference'})` });if (v.health && v.health.state !== 'closed') problems.push({ key: p.key, kind: p.kind, problem: `circuit ${v.health.state}` });}-- const stats = cache.stats();++ const stats = await cache.stats();const frac = (c) => {const x = c.quota;return Math.max(x.max_requests ? c.used.requests / x.max_requests : 0, x.max_tokens ? c.used.tokens / x.max_tokens : 0, x.max_cost_usd ? c.used.cost_usd / x.max_cost_usd : 0);};-- send(res, 200, pages.overview({-- ...common(req), st, counts24: q.statusCounts(db, since), failures24: q.failures(db, since), queue: runs.stats(), problems,++ await send(res, 200, pages.overview({++ ...common(req), st, counts24: await q.statusCounts(db, since), failures24: await q.failures(db, since), queue: runs.stats(), problems,cacheTotals: { entries: stats.reduce((a, s) => a + s.entries, 0), hits: stats.reduce((a, s) => a + (s.hits || 0), 0) },stubFallback: config.stubFallback, topQuotas: [...st.quotas].sort((a, b) => frac(b) - frac(a)).slice(0, 10),}));}));// ── Providers ────────────────────────────────────────────-- r.get('/console/providers', read, wrap((req, res) => send(res, 200, pages.providers({-- ...common(req), rows: registry.listProviders().map(ops.providerView), models: registry.listModels(), stubFallback: config.stubFallback,++ r.get('/console/providers', read, wrap(async (req, res) => await send(res, 200, pages.providers({++ ...common(req), rows: (await Promise.all((await registry.listProviders()).map(ops.providerView))), models: await registry.listModels(), stubFallback: config.stubFallback,canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req),}))));-- r.post('/console/providers/:key/status', needs(CAPS.providerManage), wrap((req, res) => {-- ops.setProviderStatus(req.params.key, String(req.body.status || ''), who(req));++ r.post('/console/providers/:key/status', needs(CAPS.providerManage), wrap(async (req, res) => {++ await ops.setProviderStatus(req.params.key, String(req.body.status || ''), who(req));res.redirect(303, '/console/providers?done=provider_status');}));-- r.post('/console/providers/:key/reset', needs(CAPS.providerManage), wrap((req, res) => {-- if (!registry.getProvider(req.params.key)) throw new AiError(404, 'ai.not_found', 'no such provider');-- ops.resetCircuit(req.params.key, who(req));++ r.post('/console/providers/:key/reset', needs(CAPS.providerManage), wrap(async (req, res) => {++ if (!await registry.getProvider(req.params.key)) throw new AiError(404, 'ai.not_found', 'no such provider');++ await ops.resetCircuit(req.params.key, who(req));res.redirect(303, '/console/providers?done=circuit_reset');}));@@@ -294,95 +294,95 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,return m;}for (const [kind, k] of Object.entries(DEF_KINDS)) {-- r.get(`/console/${kind}`, read, wrap((req, res) => send(res, 200, pages.definitions({-- ...common(req), kind, rows: registry[k.list](), inUse: inUseMap(kind, registry[k.list]({ history: true })),++ r.get(`/console/${kind}`, read, wrap(async (req, res) => await send(res, 200, pages.definitions({++ ...common(req), kind, rows: await registry[k.list](), inUse: inUseMap(kind, await registry[k.list]({ history: true })),}))));-- r.get(`/console/${kind}/:key`, read, wrap((req, res) => {-- const versions = registry[k.list]({ history: true }).filter((x) => x.key === req.params.key).sort((a, b) => b.version - a.version);-- if (!versions.length) return send(res, 404, pages.message({ title: 'Not found', text: `No ${k.one} ${req.params.key}.`, back: `/console/${kind}`, ...common(req) }));++ r.get(`/console/${kind}/:key`, read, wrap(async (req, res) => {++ const versions = (await registry[k.list]({ history: true })).filter((x) => x.key === req.params.key).sort((a, b) => b.version - a.version);++ if (!versions.length) return await send(res, 404, pages.message({ title: 'Not found', text: `No ${k.one} ${req.params.key}.`, back: `/console/${kind}`, ...common(req) }));const want = Number(req.query.version);const selected = (Number.isInteger(want) && versions.find((x) => x.version === want)) || versions[0];-- return send(res, 200, pages.definition({-- ...common(req), kind, key: req.params.key, versions, selected: registry[k.get](req.params.key, selected.version),++ return await send(res, 200, pages.definition({++ ...common(req), kind, key: req.params.key, versions, selected: await registry[k.get](req.params.key, selected.version),inUse: inUseMap(kind, versions).get(req.params.key), canManage: req.staff.caps.includes(CAPS.workflowManage), notice: notice(req),}));}));-- r.post(`/console/${kind}/:key/versions/:version/status`, needs(CAPS.workflowManage), wrap((req, res) => {++ r.post(`/console/${kind}/:key/versions/:version/status`, needs(CAPS.workflowManage), wrap(async (req, res) => {const version = Number(req.params.version);if (!Number.isInteger(version) || version < 1) throw new AiError(404, 'ai.not_found', `no ${k.one} ${req.params.key} v${req.params.version}`);-- registry.setStatus(k.one, req.params.key, version, String(req.body.status || ''), who(req));++ await registry.setStatus(k.one, req.params.key, version, String(req.body.status || ''), who(req));res.redirect(303, `/console/${kind}/${encodeURIComponent(req.params.key)}?version=${version}&done=version_status`);}));}// ── Runs ─────────────────────────────────────────────────-- r.get('/console/runs', read, wrap((req, res) => {++ r.get('/console/runs', read, wrap(async (req, res) => {const { filters, form, bad } = q.parseRunFilters(req.query);-- const list = q.listRuns(db, filters);-- send(res, bad.length ? 400 : 200, pages.runs({-- ...common(req), form, filters, bad, rows: list.rows, next: list.next, facets: q.runFacets(db),-- failures24: filters.status === 'failed' ? q.failures(db, new Date(clock.now() - DAY_MS).toISOString()) : [],++ const list = await q.listRuns(db, filters);++ await send(res, bad.length ? 400 : 200, pages.runs({++ ...common(req), form, filters, bad, rows: list.rows, next: list.next, facets: await q.runFacets(db),++ failures24: filters.status === 'failed' ? await q.failures(db, new Date(clock.now() - DAY_MS).toISOString()) : [],}));}));-- r.get('/console/runs/:id', read, wrap((req, res) => {-- const run = runs.get(req.params.id);-- if (!run) return send(res, 404, pages.message({ title: 'Not found', text: `No run ${req.params.id}.`, back: '/console/runs', ...common(req) }));-- return send(res, 200, pages.run({-- ...common(req), r: run, citations: runs.citations(run.id), requests: runs.requestsFor(run.id), input: q.preview(run.input), output: q.preview(run.output),++ r.get('/console/runs/:id', read, wrap(async (req, res) => {++ const run = await runs.get(req.params.id);++ if (!run) return await send(res, 404, pages.message({ title: 'Not found', text: `No run ${req.params.id}.`, back: '/console/runs', ...common(req) }));++ return await send(res, 200, pages.run({++ ...common(req), r: run, citations: await runs.citations(run.id), requests: await runs.requestsFor(run.id), input: q.preview(run.input), output: q.preview(run.output),canCancel: req.staff.caps.includes(CAPS.workflowManage), notice: notice(req),}));}));-- r.post('/console/runs/:id/cancel', needs(CAPS.workflowManage), wrap((req, res) => {-- runs.cancel(req.params.id, req.staff.principal, { trace: req.ov ? req.ov.traceId : null, principalHas: (cap) => req.staff.caps.includes(cap) });++ r.post('/console/runs/:id/cancel', needs(CAPS.workflowManage), wrap(async (req, res) => {++ await runs.cancel(req.params.id, req.staff.principal, { trace: req.ov ? req.ov.traceId : null, principalHas: (cap) => req.staff.caps.includes(cap) });res.redirect(303, `/console/runs/${encodeURIComponent(req.params.id)}?done=run_cancelled`);}));// ── Quotas and usage ─────────────────────────────────────-- const quotaPage = (req, res, status, extra = {}) => send(res, status, pages.quotas({-- ...common(req), counters: quotas.counters(), all: quotas.list(), canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req), ...extra,++ const quotaPage = async (req, res, status, extra = {}) => await send(res, status, pages.quotas({++ ...common(req), counters: await quotas.counters(), all: await quotas.list(), canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req), ...extra,}));-- r.get('/console/quotas', read, wrap((req, res) => quotaPage(req, res, 200)));-- r.post('/console/quotas', needs(CAPS.providerManage), wrap((req, res) => {++ r.get('/console/quotas', read, wrap(async (req, res) => await quotaPage(req, res, 200)));++ r.post('/console/quotas', needs(CAPS.providerManage), wrap(async (req, res) => {const b = req.body || {};const form = {};for (const f of ['scope_type', 'scope_id', 'window', 'max_requests', 'max_tokens', 'max_cost_usd', 'workflow_prefix', 'status']) form[f] = String(b[f] == null ? '' : b[f]).trim().slice(0, 200);try {-- quotas.upsert({ ...form, scope_id: form.scope_id || (form.scope_type === 'global' ? '*' : ''), workflow_prefix: form.workflow_prefix || null, status: form.status || 'active' }, { ...who(req), origin: 'admin' });++ await quotas.upsert({ ...form, scope_id: form.scope_id || (form.scope_type === 'global' ? '*' : ''), workflow_prefix: form.workflow_prefix || null, status: form.status || 'active' }, { ...who(req), origin: 'admin' });} catch (e) {if (!(e instanceof AiError)) throw e;-- return quotaPage(req, res, e.status, { error: `${e.detail || e.message} (${e.code})`, form, notice: null });++ return await quotaPage(req, res, e.status, { error: `${e.detail || e.message} (${e.code})`, form, notice: null });}return res.redirect(303, '/console/quotas?done=quota_saved');}));-- r.get('/console/usage', read, wrap((req, res) => {++ r.get('/console/usage', read, wrap(async (req, res) => {const { filters, form, bad } = q.parseUsageFilters(req.query, clock.now());-- send(res, bad.length ? 400 : 200, pages.usage({ ...common(req), form, bad, report: q.usageReport(quotas, filters) }));++ await send(res, bad.length ? 400 : 200, pages.usage({ ...common(req), form, bad, report: await q.usageReport(quotas, filters) }));}));// ── Cache ────────────────────────────────────────────────-- const cachePage = (req, res, status, extra = {}) => send(res, status, pages.cache({-- ...common(req), stats: cache.stats(), canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req), ...extra,++ const cachePage = async (req, res, status, extra = {}) => await send(res, status, pages.cache({++ ...common(req), stats: await cache.stats(), canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req), ...extra,}));-- r.get('/console/cache', read, wrap((req, res) => cachePage(req, res, 200)));-- r.post('/console/cache/purge', needs(CAPS.providerManage), wrap((req, res) => {++ r.get('/console/cache', read, wrap(async (req, res) => await cachePage(req, res, 200)));++ r.post('/console/cache/purge', needs(CAPS.providerManage), wrap(async (req, res) => {const workflow = String((req.body || {}).workflow || '').trim();-- if (workflow && !/^[a-z][a-z0-9_-]*(\.[a-z0-9_-]+)*$/.test(workflow)) return cachePage(req, res, 422, { error: 'that is not a workflow key', notice: null });-- if (!workflow && (req.body || {}).confirm_all !== 'yes') return cachePage(req, res, 422, { error: 'give a workflow key, or tick the box to purge every entry', notice: null });-- const removed = ops.purgeCache(workflow || null, who(req));++ if (workflow && !/^[a-z][a-z0-9_-]*(\.[a-z0-9_-]+)*$/.test(workflow)) return await cachePage(req, res, 422, { error: 'that is not a workflow key', notice: null });++ if (!workflow && (req.body || {}).confirm_all !== 'yes') return await cachePage(req, res, 422, { error: 'give a workflow key, or tick the box to purge every entry', notice: null });++ const removed = await ops.purgeCache(workflow || null, who(req));return res.redirect(303, `/console/cache?done=cache_purged&removed=${removed}`);}));// ── Audit ────────────────────────────────────────────────-- r.get('/console/audit', read, wrap((req, res) => {++ r.get('/console/audit', read, wrap(async (req, res) => {const { filters, form, bad } = q.parseAuditFilters(req.query);-- const page = q.listAudit(db, filters);-- send(res, bad.length ? 400 : 200, pages.audit({ ...common(req), form, bad, rows: page.rows, next: page.next, names: q.usernames(db), kinds: q.AUDIT_KINDS }));++ const page = await q.listAudit(db, filters);++ await send(res, bad.length ? 400 : 200, pages.audit({ ...common(req), form, bad, rows: page.rows, next: page.next, names: await q.usernames(db), kinds: q.AUDIT_KINDS }));}));// Anything else under /console or /auth: the sign-in page, or (signed in) the console's own not-found page.-- r.all(['/console/*', '/auth/*'], (req, res) => {-- if (!req.staff) return send(res, 401, pages.signIn({ next: req.method === 'GET' ? sanitizeNext(req.originalUrl) : '/console' }));-- return send(res, 404, pages.message({ title: 'Not found', text: 'There is no such console page.', ...common(req) }));++ r.all(['/console/*', '/auth/*'], async (req, res) => {++ if (!req.staff) return await send(res, 401, pages.signIn({ next: req.method === 'GET' ? sanitizeNext(req.originalUrl) : '/console' }));++ return await send(res, 404, pages.message({ title: 'Not found', text: 'There is no such console page.', ...common(req) }));});return r;@@@ -67,7 +67,7 @@ function parseRunFilters(q = {}) {return { filters, form, bad };}--function listRuns(db, f, limit = PAGE) {++async function listRuns(db, f, limit = PAGE) {const where = [];const args = [];if (f.status) { where.push('status = ?'); args.push(f.status); }@@@ -81,28 +81,28 @@ function listRuns(db, f, limit = PAGE) {if (f.fromIso) { where.push('created_at >= ?'); args.push(f.fromIso); }if (f.toIso) { where.push('created_at < ?'); args.push(f.toIso); }if (f.before) { where.push('id < ?'); args.push(f.before); }-- const rows = db.prepare(`SELECT id, status, workflow_key, workflow_version, requester_type, requester_id, provider_key, model_key, fallback_used, attempts,++ const rows = await db.prepare(`SELECT id, status, workflow_key, workflow_version, requester_type, requester_id, provider_key, model_key, fallback_used, attempts,tokens_in, tokens_out, cost_usd, error_code, synthetic, cached_from, retry_of, created_at, finished_atFROM runs ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, limit + 1);return { rows: rows.slice(0, limit), next: rows.length > limit ? rows[limit - 1].id : null };}/** Failed runs since `sinceIso`, grouped by error code. */--function failures(db, sinceIso) {-- return db.prepare(`SELECT error_code, COUNT(*) AS n, COUNT(DISTINCT workflow_key) AS workflows, MAX(created_at) AS last_at++async function failures(db, sinceIso) {++ return await db.prepare(`SELECT error_code, COUNT(*) AS n, COUNT(DISTINCT workflow_key) AS workflows, MAX(created_at) AS last_atFROM runs WHERE status = 'failed' AND created_at >= ? GROUP BY error_code ORDER BY n DESC LIMIT 20`).all(sinceIso);}--function statusCounts(db, sinceIso) {-- return Object.fromEntries(db.prepare('SELECT status, COUNT(*) AS n FROM runs WHERE created_at >= ? GROUP BY status').all(sinceIso).map(r => [r.status, r.n]));++async function statusCounts(db, sinceIso) {++ return Object.fromEntries((await db.prepare('SELECT status, COUNT(*) AS n FROM runs WHERE created_at >= ? GROUP BY status').all(sinceIso)).map(r => [r.status, r.n]));}/** Suggestions for the run filters (<datalist>): requesters and workflow keys that have runs. */--function runFacets(db) {++async function runFacets(db) {return {-- requesters: db.prepare("SELECT DISTINCT requester_type || ':' || requester_id AS r FROM runs ORDER BY r LIMIT 100").all().map(x => x.r),-- workflows: db.prepare('SELECT DISTINCT workflow_key AS w FROM runs ORDER BY w LIMIT 300').all().map(x => x.w),-- codes: db.prepare("SELECT DISTINCT error_code AS c FROM runs WHERE error_code IS NOT NULL ORDER BY c LIMIT 100").all().map(x => x.c),++ requesters: (await db.prepare("SELECT DISTINCT requester_type || ':' || requester_id AS r FROM runs ORDER BY r LIMIT 100").all()).map(x => x.r),++ workflows: (await db.prepare('SELECT DISTINCT workflow_key AS w FROM runs ORDER BY w LIMIT 300').all()).map(x => x.w),++ codes: (await db.prepare("SELECT DISTINCT error_code AS c FROM runs WHERE error_code IS NOT NULL ORDER BY c LIMIT 100").all()).map(x => x.c),};}@@@ -138,7 +138,7 @@ function preview(value) {const AUDIT_KINDS = Object.freeze({changes: { label: 'Configuration changes', where: "target_type IN ('provider', 'model', 'route', 'template', 'workflow', 'quota', 'cache')" },runs: { label: 'Runs', where: "target_type = 'run'" },-- console: { label: 'Console sign-ins and refusals', where: "action LIKE 'console.%'" },++ console: { label: 'Console sign-ins and refusals', where: "action ILIKE 'console.%'" },all: { label: 'Everything', where: null },});@@@ -156,7 +156,7 @@ function parseAuditFilters(q = {}) {return { filters: f, form, bad };}--function listAudit(db, f, limit = 100) {++async function listAudit(db, f, limit = 100) {const where = [];const args = [];const kind = AUDIT_KINDS[f.kind] || AUDIT_KINDS.changes;@@@ -166,16 +166,16 @@ function listAudit(db, f, limit = 100) {if (f.targetType) { where.push('target_type = ?'); args.push(f.targetType); }if (f.targetId) { where.push('target_id = ?'); args.push(f.targetId); }if (f.before) { where.push('id < ?'); args.push(f.before); }-- const rows = db.prepare(`SELECT * FROM audit_log ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, limit + 1);++ const rows = await db.prepare(`SELECT * FROM audit_log ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, limit + 1);const page = rows.slice(0, limit).map(r => ({ ...r, metadata: parseMeta(r.metadata) }));return { rows: page, next: rows.length > limit ? page[page.length - 1].id : null };}function parseMeta(s) { try { const v = JSON.parse(s || '{}'); return v && typeof v === 'object' ? v : {}; } catch { return {}; } }/** subject -> username, from console sign-ins (the audit log keeps only the subject). */--function usernames(db) {++async function usernames(db) {const map = new Map();-- for (const r of db.prepare('SELECT subject, username FROM console_sessions WHERE username IS NOT NULL ORDER BY created_at').all()) map.set(r.subject, r.username);++ for (const r of await db.prepare('SELECT subject, username FROM console_sessions WHERE username IS NOT NULL ORDER BY created_at').all()) map.set(r.subject, r.username);return map;}@@@ -194,8 +194,8 @@ function parseUsageFilters(q = {}, nowMs = Date.now()) {}/** usage_daily rows (quotas.usage, the API's own query) summed four ways. */--function usageReport(quotas, f) {-- const rows = quotas.usage({ from: f.from, to: f.to, requester: f.requester });++async function usageReport(quotas, f) {++ const rows = await quotas.usage({ from: f.from, to: f.to, requester: f.requester });const blank = () => ({ requests: 0, tokens_in: 0, tokens_out: 0, tokens_cached: 0, cost_usd: 0 });const add = (acc, r) => { acc.requests += r.requests; acc.tokens_in += r.tokens_in; acc.tokens_out += r.tokens_out; acc.tokens_cached += r.tokens_cached; acc.cost_usd += r.cost_usd; return acc; };const group = (keyOf) => {@@@ -80,27 +80,27 @@ function createSessions({ db, config, clock, secret }) {}// ── Sessions ─────────────────────────────────────────────-- function create(res, { subject, username, staff, ip }) {++ async function create(res, { subject, username, staff, ip }) {const t = now();-- db.prepare('DELETE FROM console_sessions WHERE expires_at < ?').run(iso(t - 24 * 3600 * 1000));++ await db.prepare('DELETE FROM console_sessions WHERE expires_at < ?').run(iso(t - 24 * 3600 * 1000));const id = random(32);const ttlMs = cc.sessionTtlMin * 60 * 1000;-- db.prepare(`INSERT INTO console_sessions (id_hash, subject, username, staff, csrf, created_at, expires_at, ip_hash)++ await db.prepare(`INSERT INTO console_sessions (id_hash, subject, username, staff, csrf, created_at, expires_at, ip_hash)VALUES (?, ?, ?, ?, ?, ?, ?, ?)`).run(sha256(id), subject, username || null, JSON.stringify(staff || {}), random(32), iso(t), iso(t + ttlMs), ipHash(ip));appendCookie(res, cookieHeader(sessionCookie, id, { maxAgeSec: ttlMs / 1000, secure: cc.cookieSecure }));}/** The live session of this request, or null (unknown, expired or revoked). */-- function read(req) {++ async function read(req) {const id = parseCookies(req)[sessionCookie];if (!id || !/^[A-Za-z0-9_-]{43}$/.test(id)) return null;-- const row = db.prepare('SELECT * FROM console_sessions WHERE id_hash = ?').get(sha256(id));++ const row = await db.prepare('SELECT * FROM console_sessions WHERE id_hash = ?').get(sha256(id));if (!row || row.revoked_at || Date.parse(row.expires_at) <= now()) return null;return { ...row, staff: parseJson(row.staff, {}) };}-- function revoke(row) {-- if (row) db.prepare('UPDATE console_sessions SET revoked_at = ? WHERE id_hash = ? AND revoked_at IS NULL').run(iso(now()), row.id_hash);++ async function revoke(row) {++ if (row) await db.prepare('UPDATE console_sessions SET revoked_at = ? WHERE id_hash = ? AND revoked_at IS NULL').run(iso(now()), row.id_hash);}function clear(res) {appendCookie(res, cookieHeader(sessionCookie, '', { maxAgeSec: 0, secure: cc.cookieSecure }));@@@ -24,20 +24,7 @@ const SUBJECT_RE = /^usr_[0-9A-HJKMNP-TV-Z]{26}$/;function providerKeyFor(owner, subject) { return `byo:${owner}:${subject}`; }function createCredentials({ db, config, clock = { now: () => Date.now() } }) {-- db.exec(`CREATE TABLE IF NOT EXISTS subject_credentials (-- owner TEXT NOT NULL,-- subject TEXT NOT NULL,-- provider TEXT NOT NULL CHECK (provider IN ('openai', 'anthropic')),-- base_url TEXT,-- key_enc TEXT NOT NULL,-- key_hint TEXT NOT NULL,-- models TEXT NOT NULL DEFAULT '{}',-- budget_usd_day REAL,-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL,-- last_used_at TEXT,-- PRIMARY KEY (owner, subject)-- )`);++ // subject_credentials is in migrations/0001_initial.sql (ADR-035).const iso = (ms) => new Date(ms).toISOString();function secretKey() {@@@ -61,69 +48,69 @@ function createCredentials({ db, config, clock = { now: () => Date.now() } }) {return Buffer.concat([d.update(Buffer.from(body, 'base64')), d.final()]).toString('utf8');}-- function spentToday(owner, subject) {++ async function spentToday(owner, subject) {const day = iso(clock.now()).slice(0, 10);-- const r = db.prepare("SELECT COALESCE(SUM(cost_usd), 0) AS c FROM requests WHERE provider_key = ? AND status = 'ok' AND created_at >= ?").get(providerKeyFor(owner, subject), `${day}T00:00:00.000Z`);++ const r = await db.prepare("SELECT COALESCE(SUM(cost_usd), 0) AS c FROM requests WHERE provider_key = ? AND status = 'ok' AND created_at >= ?").get(providerKeyFor(owner, subject), `${day}T00:00:00.000Z`);return r ? Number(r.c) || 0 : 0;}-- function view(row) {++ async function view(row) {return {subject: row.subject, owner: row.owner, provider: row.provider, base_url: row.base_url || null, key_hint: row.key_hint,models: JSON.parse(row.models || '{}'), budget_usd_per_day: row.budget_usd_day == null ? null : row.budget_usd_day,-- spent_usd_today: Math.round(spentToday(row.owner, row.subject) * 1e6) / 1e6,++ spent_usd_today: Math.round(await spentToday(row.owner, row.subject) * 1e6) / 1e6,created_at: row.created_at, updated_at: row.updated_at, last_used_at: row.last_used_at || null,};}-- const row = (owner, subject) => db.prepare('SELECT * FROM subject_credentials WHERE owner = ? AND subject = ?').get(owner, subject);++ const row = async (owner, subject) => await db.prepare('SELECT * FROM subject_credentials WHERE owner = ? AND subject = ?').get(owner, subject);-- function put(owner, subject, body) {++ async function put(owner, subject, body) {if (!SUBJECT_RE.test(String(subject))) throw new AiError(400, 'input.invalid', 'subject must be a usr_ subject');const v = validate('ai.credential-put@1', body);if (!v.valid) throw new AiError(422, 'input.invalid', 'body does not match ai.credential-put@1', { errors: v.errors });const at = iso(clock.now());-- const prev = row(owner, subject);++ const prev = await row(owner, subject);// The stored key stays for a change of models or budget; a new provider or endpoint needs the key again.const sameEndpoint = prev && prev.provider === body.provider && (prev.base_url || null) === (body.base_url || null);if (body.api_key == null && !sameEndpoint) {throw new AiError(400, 'credential.key_required', prev ? 'enter the key again to change the provider or its address' : 'api_key is required the first time');}const key = body.api_key != null ? String(body.api_key) : open(prev.key_enc, `${owner}|${subject}`);-- db.prepare(`INSERT INTO subject_credentials (owner, subject, provider, base_url, key_enc, key_hint, models, budget_usd_day, created_at, updated_at)++ await db.prepare(`INSERT INTO subject_credentials (owner, subject, provider, base_url, key_enc, key_hint, models, budget_usd_day, created_at, updated_at)VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)ON CONFLICT(owner, subject) DO UPDATE SET provider = excluded.provider, base_url = excluded.base_url, key_enc = excluded.key_enc,key_hint = excluded.key_hint, models = excluded.models, budget_usd_day = excluded.budget_usd_day, updated_at = excluded.updated_at`).run(owner, subject, body.provider, body.base_url || null, seal(key, `${owner}|${subject}`), `…${key.slice(-4)}`, JSON.stringify(body.models || {}),body.budget_usd_per_day ? Number(body.budget_usd_per_day) : null, prev ? prev.created_at : at, at);-- return view(row(owner, subject));++ return await view(await row(owner, subject));}-- function get(owner, subject) {-- const r = row(owner, subject);++ async function get(owner, subject) {++ const r = await row(owner, subject);if (!r) throw new AiError(404, 'credential.not_found', 'no credential stored for that subject');-- return view(r);++ return await view(r);}-- function remove(owner, subject) {-- return db.prepare('DELETE FROM subject_credentials WHERE owner = ? AND subject = ?').run(owner, subject).changes > 0;++ async function remove(owner, subject) {++ return (await db.prepare('DELETE FROM subject_credentials WHERE owner = ? AND subject = ?').run(owner, subject)).changes > 0;}/** Checks a run may use the credential (exists, within budget); throws AiError otherwise. */-- function admit(owner, subject) {-- const r = row(owner, subject);++ async function admit(owner, subject) {++ const r = await row(owner, subject);if (!r) throw new AiError(404, 'credential.not_found', 'no credential stored for that subject');secretKey();-- if (r.budget_usd_day != null && r.budget_usd_day > 0 && spentToday(owner, subject) >= r.budget_usd_day) {++ if (r.budget_usd_day != null && r.budget_usd_day > 0 && await spentToday(owner, subject) >= r.budget_usd_day) {const tomorrow = new Date(iso(clock.now()).slice(0, 10) + 'T00:00:00.000Z').getTime() + 86400000;throw new AiError(429, 'quota.exceeded', "this person's own daily budget for their key is spent", { retry_after_seconds: Math.max(60, Math.ceil((tomorrow - clock.now()) / 1000)), quota: { scope: 'credential' } });}}/** The decrypted credential for one execution: { providerKey, provider, base_url, apiKey, models }. */-- function forRun(owner, subject) {-- const r = row(owner, subject);++ async function forRun(owner, subject) {++ const r = await row(owner, subject);if (!r) throw new AiError(404, 'credential.not_found', 'the credential was deleted');-- db.prepare('UPDATE subject_credentials SET last_used_at = ? WHERE owner = ? AND subject = ?').run(iso(clock.now()), owner, subject);++ await db.prepare('UPDATE subject_credentials SET last_used_at = ? WHERE owner = ? AND subject = ?').run(iso(clock.now()), owner, subject);return { providerKey: providerKeyFor(owner, subject), provider: r.provider, base_url: r.base_url || null, apiKey: open(r.key_enc, `${owner}|${subject}`), models: JSON.parse(r.models || '{}') };}@@@ -139,21 +126,21 @@ function credentialsRouter({ credentials, auth, registry, sendError }) {if (!s || s.type !== 'service') throw new AiError(403, 'capability.denied', 'only a service stores credentials for the people it serves');return s.id;};-- r.put('/api/v1/credentials/:subject', guard, (req, res) => {++ r.put('/api/v1/credentials/:subject', guard, async (req, res) => {try {-- const out = credentials.put(owner(req), req.params.subject, req.body || {});-- registry.audit(req.principal.sub, 'credential.put', 'credential', `${out.owner}:${out.subject}`, { trace: req.ov && req.ov.traceId, metadata: { provider: out.provider, base_url: out.base_url } });++ const out = await credentials.put(owner(req), req.params.subject, req.body || {});++ await registry.audit(req.principal.sub, 'credential.put', 'credential', `${out.owner}:${out.subject}`, { trace: req.ov && req.ov.traceId, metadata: { provider: out.provider, base_url: out.base_url } });res.set('Cache-Control', 'no-store').json(out);} catch (err) { sendError(res, err, req.ov); }});-- r.get('/api/v1/credentials/:subject', guard, (req, res) => {-- try { res.set('Cache-Control', 'no-store').json(credentials.get(owner(req), req.params.subject)); } catch (err) { sendError(res, err, req.ov); }++ r.get('/api/v1/credentials/:subject', guard, async (req, res) => {++ try { res.set('Cache-Control', 'no-store').json(await credentials.get(owner(req), req.params.subject)); } catch (err) { sendError(res, err, req.ov); }});-- r.delete('/api/v1/credentials/:subject', guard, (req, res) => {++ r.delete('/api/v1/credentials/:subject', guard, async (req, res) => {try {const o = owner(req);-- const gone = credentials.remove(o, req.params.subject);-- if (gone) registry.audit(req.principal.sub, 'credential.delete', 'credential', `${o}:${req.params.subject}`, { trace: req.ov && req.ov.traceId });++ const gone = await credentials.remove(o, req.params.subject);++ if (gone) await registry.audit(req.principal.sub, 'credential.delete', 'credential', `${o}:${req.params.subject}`, { trace: req.ov && req.ov.traceId });res.status(gone ? 204 : 404).end();} catch (err) { sendError(res, err, req.ov); }});@@@ -1,6 +1,6 @@'use strict';/**-- * The AI authority's SQLite database (better-sqlite3). One table family per canonical record++ * The AI authority's PostgreSQL database (ADR-035; migrations/NNNN_*.sql). One table family per canonical record* group of the plan (§12.13 / §15.14):** 1 providers (+ provider_health) 6 runs@@@ -17,311 +17,28 @@*/const fs = require('fs');const path = require('path');--const Database = require('better-sqlite3');++const { createDb } = require('openvibe-sdk/db');--const SCHEMA = `--CREATE TABLE IF NOT EXISTS providers (-- key TEXT PRIMARY KEY,-- display_name TEXT NOT NULL,-- kind TEXT NOT NULL, -- stub | openai | anthropic | http | whisper-- status TEXT NOT NULL DEFAULT 'active', -- active | disabled-- base_url TEXT,-- auth_mode TEXT NOT NULL DEFAULT 'none', -- none | bearer | x-api-key-- secret_ref TEXT, -- 'env:NAME' — never a value-- default_model TEXT,-- capabilities TEXT NOT NULL DEFAULT '[]', -- features supports() answers true for-- timeout_ms INTEGER NOT NULL DEFAULT 30000,-- priority INTEGER NOT NULL DEFAULT 100,-- metadata TEXT NOT NULL DEFAULT '{}',-- origin TEXT NOT NULL DEFAULT 'seed', -- seed (from env at boot) | admin | import-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL--);--CREATE TABLE IF NOT EXISTS provider_health (-- provider_key TEXT PRIMARY KEY,-- state TEXT NOT NULL DEFAULT 'closed', -- closed (healthy) | open (skipped) | half_open (one probe)-- consecutive_failures INTEGER NOT NULL DEFAULT 0,-- opened_at INTEGER,-- last_error TEXT,-- last_success_at INTEGER,-- last_failure_at INTEGER--);--CREATE TABLE IF NOT EXISTS models (-- provider_key TEXT NOT NULL,-- model_key TEXT NOT NULL,-- display_name TEXT,-- type TEXT NOT NULL DEFAULT 'chat', -- chat | vision | embedding | stt-- status TEXT NOT NULL DEFAULT 'active',-- context_window INTEGER,-- max_output INTEGER,-- cost_in_per_mtok REAL,-- cost_out_per_mtok REAL,-- cost_cached_per_mtok REAL,-- supports_json INTEGER NOT NULL DEFAULT 1,-- supports_tools INTEGER NOT NULL DEFAULT 0,-- supports_streaming INTEGER NOT NULL DEFAULT 0,-- supports_vision INTEGER NOT NULL DEFAULT 0,-- metadata TEXT NOT NULL DEFAULT '{}',-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL,-- PRIMARY KEY (provider_key, model_key)--);--CREATE TABLE IF NOT EXISTS routes (-- key TEXT NOT NULL,-- version INTEGER NOT NULL,-- status TEXT NOT NULL DEFAULT 'active', -- active | disabled-- primary_provider TEXT NOT NULL,-- primary_model TEXT,-- fallbacks TEXT NOT NULL DEFAULT '[]', -- [{provider, model}]-- options TEXT NOT NULL DEFAULT '{}', -- temperature and friends-- max_output_tokens INTEGER,-- response_format TEXT NOT NULL DEFAULT 'text', -- text | json-- timeout_ms INTEGER,-- alias_of TEXT, -- historical key kept as an explicit alias-- created_by TEXT,-- created_at TEXT NOT NULL,-- PRIMARY KEY (key, version)--);--CREATE TABLE IF NOT EXISTS templates (-- key TEXT NOT NULL,-- version INTEGER NOT NULL,-- name TEXT NOT NULL,-- description TEXT,-- input_schema TEXT NOT NULL DEFAULT '{}',-- output_schema TEXT NOT NULL DEFAULT '{}',-- system_prompt TEXT NOT NULL DEFAULT '',-- user_prompt TEXT NOT NULL DEFAULT '',-- default_route TEXT,-- owner TEXT NOT NULL DEFAULT 'ai',-- visibility TEXT NOT NULL DEFAULT 'internal', -- public | first-party | internal-- status TEXT NOT NULL DEFAULT 'active', -- draft | active | deprecated | archived-- metadata TEXT NOT NULL DEFAULT '{}',-- created_by TEXT,-- created_at TEXT NOT NULL,-- PRIMARY KEY (key, version)--);--CREATE TABLE IF NOT EXISTS workflows (-- key TEXT NOT NULL,-- version INTEGER NOT NULL,-- name TEXT NOT NULL,-- description TEXT,-- namespace TEXT NOT NULL, -- owning product: live, network, wiki, ...-- input_schema TEXT NOT NULL,-- output_schema TEXT NOT NULL,-- steps TEXT NOT NULL, -- [{kind, template?, operation?, ...}]-- default_route TEXT,-- cache_mode TEXT NOT NULL DEFAULT 'private', -- none | private | service-- cache_ttl_sec INTEGER,-- status TEXT NOT NULL DEFAULT 'active', -- draft | active | deprecated | archived-- metadata TEXT NOT NULL DEFAULT '{}',-- created_by TEXT,-- created_at TEXT NOT NULL,-- PRIMARY KEY (key, version)--);--CREATE TABLE IF NOT EXISTS runs (-- id TEXT PRIMARY KEY, -- run_<ULID>-- workflow_key TEXT NOT NULL,-- workflow_version INTEGER NOT NULL,-- template_key TEXT,-- template_version INTEGER,-- route_key TEXT,-- route_version INTEGER,-- status TEXT NOT NULL, -- queued | running | succeeded | failed | cancelled | cached-- requester_type TEXT NOT NULL, -- service | app | mod (the token principal)-- requester_id TEXT NOT NULL,-- on_behalf_of TEXT, -- SubjectRef JSON (optional)-- attribution TEXT, -- EntityRef JSON the spend is attributed to (optional)-- source_service TEXT,-- target TEXT, -- EntityRef JSON (optional)-- target_key TEXT, -- service:type:id for lookups-- input TEXT NOT NULL,-- input_hash TEXT NOT NULL,-- output TEXT,-- citations_count INTEGER NOT NULL DEFAULT 0,-- error_code TEXT,-- error_detail TEXT,-- synthetic INTEGER NOT NULL DEFAULT 0, -- produced by the stub provider-- provider_key TEXT,-- model_key TEXT,-- fallback_used INTEGER NOT NULL DEFAULT 0,-- attempts INTEGER NOT NULL DEFAULT 0,-- tokens_in INTEGER NOT NULL DEFAULT 0,-- tokens_out INTEGER NOT NULL DEFAULT 0,-- cost_usd REAL NOT NULL DEFAULT 0,-- cache_key TEXT,-- cached_from TEXT, -- run id whose output a cached run reused-- retry_of TEXT,-- idempotency_key TEXT,-- trace_id TEXT,-- request_id TEXT,-- options TEXT NOT NULL DEFAULT '{}',-- created_at TEXT NOT NULL,-- started_at TEXT,-- finished_at TEXT--);--CREATE UNIQUE INDEX IF NOT EXISTS idx_runs_idem ON runs(requester_type, requester_id, idempotency_key) WHERE idempotency_key IS NOT NULL;--CREATE INDEX IF NOT EXISTS idx_runs_status ON runs(status, created_at);--CREATE INDEX IF NOT EXISTS idx_runs_workflow ON runs(workflow_key, created_at);--CREATE INDEX IF NOT EXISTS idx_runs_target ON runs(target_key, created_at);--CREATE INDEX IF NOT EXISTS idx_runs_requester ON runs(requester_type, requester_id, created_at);--CREATE TABLE IF NOT EXISTS requests (-- id INTEGER PRIMARY KEY AUTOINCREMENT,-- run_id TEXT,-- seq INTEGER NOT NULL DEFAULT 0,-- operation TEXT NOT NULL, -- chat | generate | summarize | classify | extract | enrich | embed | transcribe-- provider_key TEXT NOT NULL,-- model_key TEXT,-- route_key TEXT,-- route_version INTEGER,-- status TEXT NOT NULL, -- ok | error | timeout | skipped | cancelled-- skip_reason TEXT, -- disabled | circuit_open | unsupported | no_credentials-- fallback INTEGER NOT NULL DEFAULT 0, -- 1 when this was not the route's primary-- prompt_hash TEXT,-- input_hash TEXT,-- output_hash TEXT,-- tokens_in INTEGER NOT NULL DEFAULT 0,-- tokens_out INTEGER NOT NULL DEFAULT 0,-- tokens_cached INTEGER NOT NULL DEFAULT 0,-- tokens_estimated INTEGER NOT NULL DEFAULT 0,-- cost_usd REAL NOT NULL DEFAULT 0,-- latency_ms INTEGER,-- error TEXT,-- debug_prompt TEXT, -- only with AI_DEBUG_RAW_LOG and a caller opt-in-- debug_response TEXT,-- created_at TEXT NOT NULL--);--CREATE INDEX IF NOT EXISTS idx_requests_run ON requests(run_id, seq);--CREATE INDEX IF NOT EXISTS idx_requests_provider ON requests(provider_key, created_at);--CREATE TABLE IF NOT EXISTS citations (-- id INTEGER PRIMARY KEY AUTOINCREMENT,-- run_id TEXT NOT NULL,-- ordinal INTEGER NOT NULL DEFAULT 0,-- source_type TEXT NOT NULL,-- source_id TEXT,-- url TEXT,-- title TEXT,-- author TEXT,-- published_at TEXT,-- retrieved_at TEXT,-- snippet TEXT,-- content_hash TEXT,-- trust TEXT NOT NULL DEFAULT '{}',-- provenance TEXT NOT NULL DEFAULT '{}',-- attached_by TEXT, -- workflow | <principal sub>-- created_at TEXT NOT NULL--);--CREATE INDEX IF NOT EXISTS idx_citations_run ON citations(run_id, ordinal);--CREATE TABLE IF NOT EXISTS cache_entries (-- cache_key TEXT PRIMARY KEY, -- sha256 over everything below-- scope TEXT NOT NULL, -- who may read it (see server/cache.js)-- workflow_key TEXT NOT NULL,-- workflow_version INTEGER NOT NULL,-- template_version INTEGER,-- route_key TEXT,-- route_version INTEGER,-- model_key TEXT,-- input_hash TEXT NOT NULL,-- privacy TEXT NOT NULL, -- private | service-- output TEXT NOT NULL,-- run_id TEXT NOT NULL,-- hits INTEGER NOT NULL DEFAULT 0,-- created_at TEXT NOT NULL,-- expires_at INTEGER NOT NULL--);--CREATE INDEX IF NOT EXISTS idx_cache_expiry ON cache_entries(expires_at);--CREATE INDEX IF NOT EXISTS idx_cache_workflow ON cache_entries(workflow_key);--CREATE TABLE IF NOT EXISTS quotas (-- id INTEGER PRIMARY KEY AUTOINCREMENT,-- scope_type TEXT NOT NULL, -- global | service | actor | attribution-- scope_id TEXT NOT NULL, -- '*' | live | user:usr_… | live:user:42-- window TEXT NOT NULL, -- minute | hour | day-- max_requests INTEGER,-- max_tokens INTEGER,-- max_cost_usd REAL,-- workflow_prefix TEXT, -- optional: only runs of workflows with this prefix-- status TEXT NOT NULL DEFAULT 'active',-- origin TEXT NOT NULL DEFAULT 'admin', -- seed | admin | import-- created_at TEXT NOT NULL,-- updated_at TEXT NOT NULL--);--CREATE UNIQUE INDEX IF NOT EXISTS idx_quotas_unique ON quotas(scope_type, scope_id, window, COALESCE(workflow_prefix, ''));--CREATE TABLE IF NOT EXISTS usage_counters (-- scope_type TEXT NOT NULL,-- scope_id TEXT NOT NULL,-- window TEXT NOT NULL,-- window_start INTEGER NOT NULL, -- epoch seconds, UTC-- workflow_prefix TEXT NOT NULL DEFAULT '',-- requests INTEGER NOT NULL DEFAULT 0,-- tokens INTEGER NOT NULL DEFAULT 0,-- cost_usd REAL NOT NULL DEFAULT 0,-- PRIMARY KEY (scope_type, scope_id, window, window_start, workflow_prefix)--);--CREATE TABLE IF NOT EXISTS usage_daily (-- day TEXT NOT NULL, -- YYYY-MM-DD (UTC)-- requester TEXT NOT NULL, -- service:live-- attribution TEXT NOT NULL DEFAULT '', -- live:user:42-- workflow_key TEXT NOT NULL,-- provider_key TEXT NOT NULL,-- model_key TEXT NOT NULL DEFAULT '',-- requests INTEGER NOT NULL DEFAULT 0,-- tokens_in INTEGER NOT NULL DEFAULT 0,-- tokens_out INTEGER NOT NULL DEFAULT 0,-- tokens_cached INTEGER NOT NULL DEFAULT 0,-- cost_usd REAL NOT NULL DEFAULT 0,-- PRIMARY KEY (day, requester, attribution, workflow_key, provider_key, model_key)--);--CREATE TABLE IF NOT EXISTS audit_log (-- id INTEGER PRIMARY KEY AUTOINCREMENT,-- at TEXT NOT NULL,-- actor TEXT NOT NULL, -- principal sub (svc:live) or 'system'-- action TEXT NOT NULL, -- run.create | template.version | provider.update | ...-- target_type TEXT,-- target_id TEXT,-- trace_id TEXT,-- metadata TEXT NOT NULL DEFAULT '{}'--);--CREATE INDEX IF NOT EXISTS idx_audit_target ON audit_log(target_type, target_id, id);--CREATE INDEX IF NOT EXISTS idx_audit_action ON audit_log(action, id);--CREATE TABLE IF NOT EXISTS console_sessions (-- id_hash TEXT PRIMARY KEY, -- sha256 of the session cookie; the cookie value is never stored-- subject TEXT NOT NULL, -- the Network person (usr_…)-- username TEXT,-- staff TEXT NOT NULL DEFAULT '{}', -- the sign-in token's staff claims (role, is_owner, staff_caps, staff_map)-- csrf TEXT NOT NULL,-- created_at TEXT NOT NULL,-- expires_at TEXT NOT NULL,-- revoked_at TEXT,-- ip_hash TEXT -- keyed hash of the client address--);--CREATE TABLE IF NOT EXISTS import_ledger (-- source TEXT NOT NULL, -- live.ai_usage, live.translations, ...-- source_key TEXT NOT NULL,-- target TEXT NOT NULL, -- where it went (table:key)-- imported_at TEXT NOT NULL,-- PRIMARY KEY (source, source_key)--);--CREATE TABLE IF NOT EXISTS import_holds (-- source TEXT NOT NULL,-- source_key TEXT NOT NULL,-- reason TEXT NOT NULL,-- detail TEXT,-- held_at TEXT NOT NULL,-- PRIMARY KEY (source, source_key)--);--`;++const MIGRATIONS = path.join(__dirname, '..', 'migrations');++const DEV_PGLITE = path.join(__dirname, '..', 'data', 'pglite');--function openDb(dbPath) {-- if (dbPath !== ':memory:') fs.mkdirSync(path.dirname(path.resolve(dbPath)), { recursive: true });-- const db = new Database(dbPath);-- db.pragma('journal_mode = WAL');-- db.pragma('synchronous = NORMAL');-- db.pragma('foreign_keys = ON');-- db.pragma('busy_timeout = 5000');-- db.exec(SCHEMA);-- // Added after the first release: ADD COLUMN only, so an older build ignores it.-- const cols = new Set(db.prepare("SELECT name FROM pragma_table_info('runs')").all().map((c) => c.name));-- if (!cols.has('grounding')) db.exec('ALTER TABLE runs ADD COLUMN grounding TEXT'); // { cited, gaps } (WS-O task 3)-- return db;++/**++ * The serving handle (ADR-035): DATABASE_URL through PgBouncer; in development without it, an embedded PGlite database++ * in data/pglite. Migrations run first, as the owner (DATABASE_DIRECT_URL), or on the embedded handle.++ */++async function openDb(config, { log = console, registry } = {}) {++ if (!config.db.url) {++ if (config.isProduction) throw new Error('DATABASE_URL is not set: production serves from PostgreSQL (OpenVibe.Host roles/data add-service.sh ai)');++ log.warn(`[AI] DATABASE_URL unset: embedded PGlite database in ${DEV_PGLITE} (development only, one process)`);++ fs.mkdirSync(DEV_PGLITE, { recursive: true });++ const db = createDb({ pglite: DEV_PGLITE, service: 'ai', registry, log });++ await db.migrate({ dir: MIGRATIONS, log });++ return db;++ }++ if (!config.db.directUrl) throw new Error('DATABASE_DIRECT_URL is not set: migrations run with the owner role on a direct connection');++ const owner = createDb({ url: config.db.directUrl, service: 'ai-migrate', max: 1, log });++ try { await owner.migrate({ dir: MIGRATIONS, log }); } finally { await owner.close(); }++ return createDb({ url: config.db.url, service: 'ai', registry, log });}--module.exports = { openDb, SCHEMA };++module.exports = { openDb, MIGRATIONS };@@@ -13,9 +13,10 @@*/const { createClient } = require('openvibe-sdk/core');const { createServiceTokenClient } = require('openvibe-sdk/auth');--const { createEventsClient, createOutbox } = require('openvibe-sdk/events');++const { createEventsClient, createPgOutbox } = require('openvibe-sdk/events');let outbox = null;++let dbRef = null; // the handle enqueue joins the running transaction through (ADR-035)let pruneTimer = null;const stats = { queued: 0, lastError: null };const PRUNE_EVERY_MS = 6 * 60 * 60 * 1000;@@@ -23,15 +24,16 @@ const PRUNE_EVERY_MS = 6 * 60 * 60 * 1000;function init(db, { eventsUrl = process.env.EVENTS_URL, clientSecret = process.env.OV_OAUTH_CLIENT_SECRET, clientId = process.env.OV_OAUTH_CLIENT_ID || 'ai',networkUrl = process.env.OV_NETWORK_INTERNAL_URL || 'http://127.0.0.1:4000', fetchImpl, intervalMs, log = console } = {}) {if (outbox) return outbox;++ dbRef = db;if (process.env.EVENTS_PUBLISH === 'off' || !eventsUrl || !clientSecret) return null;const tokens = createServiceTokenClient({ tokenUrl: `${String(networkUrl).replace(/\/+$/, '')}/oauth/token`, clientId, clientSecret, fetch: fetchImpl });const client = createClient({ baseUrls: { events: String(eventsUrl).replace(/\/+$/, '') }, tokenProvider: tokens, fetch: fetchImpl, retries: 0 });-- outbox = createOutbox(db, {++ // The PostgreSQL outbox (ADR-035): rows are written in the change's own transaction (enqueue(db, …) joins it).++ outbox = createPgOutbox(db, {events: createEventsClient(client, { source: 'ai' }),intervalMs: intervalMs || 2000,onError: (err) => { const m = err && err.message; if (m !== stats.lastError) log.warn('[Events] publish failed (will retry):', m); stats.lastError = m; },});-- outbox.ensureSchema();outbox.start();pruneTimer = setInterval(() => { try { outbox.prune(); } catch { /* next time */ } }, PRUNE_EVERY_MS);if (pruneTimer.unref) pruneTimer.unref();@@@ -76,7 +78,7 @@ function payloadOf(row) {function runChanged(row) {if (!outbox || !row || !['queued', 'cached', 'succeeded', 'failed'].includes(row.status)) return null;const requester = { type: row.requester_type, id: row.requester_id };-- const env = outbox.enqueue({++ const env = outbox.enqueue(dbRef, {event_type: `ai.run.${row.status}`,actor: requester.type === 'user' || requester.type === 'service' ? requester : { type: 'service', id: 'ai' },subject: { type: 'run', id: row.id },@@@ -85,13 +87,14 @@ function runChanged(row) {payload: payloadOf(row),});stats.queued++;-- setImmediate(() => outbox && outbox.kick());++ // Relay once the change (and its event) committed.++ dbRef.afterCommit(() => outbox && outbox.kick());return env;}--function status() {++async function status() {if (!outbox) return { enabled: false };-- return { enabled: true, pending: outbox.pending(), rejected: outbox.rejected(), queued_since_boot: stats.queued, last_error: stats.lastError };++ return { enabled: true, pending: await outbox.pending(), rejected: await outbox.rejected(), queued_since_boot: stats.queued, last_error: stats.lastError };}/*** Graceful stop (start().close(), after runs.drain()): the relay stops and nothing more is queued; resolves@@@ -253,7 +253,7 @@ function createFetcher(config, { transport = null } = {}) {type = r.contentType.split(';')[0].trim();if (!/^image\//i.test(type)) throw new AiError(422, 'source.invalid', `expected an image, got ${type || 'unknown type'}`);}-- return downscale(buf, type, input.max_width || 1024);++ return await downscale(buf, type, input.max_width || 1024);}/** Media input -> a temp file path (caller deletes it). */@@@ -21,10 +21,11 @@ const { createKeyStore, createAuth } = require('./auth');const schemas = require('./schemas');const { createApp } = require('./app');--async function start({ config, clock = { now: () => Date.now() }, fetchImpl = globalThis.fetch, env = process.env, log = console, listen = true, credentialFetch = null } = {}) {++async function start({ config, db: givenDb = null, clock = { now: () => Date.now() }, fetchImpl = globalThis.fetch, env = process.env, log = console, listen = true, credentialFetch = null } = {}) {config = config || load(env);schemas.configure(config.schemaCache);-- const db = openDb(config.dbPath);++ // PostgreSQL (ADR-035): opened and migrated here unless the caller (a test) hands in a migrated handle.++ const db = givenDb || await openDb(config, { log });const registry = createRegistry(db, { clock, env });const quotas = createQuotas(db, { clock, registry });const cache = createCache(db, { clock });@@@ -39,8 +40,8 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = gl// A person's own provider keys (WS-O task 2): stored by the service holding their consent, used by their runs only.const credentials = require('./credentials').createCredentials({ db, config, clock });const runs = createRuns({ db, registry, engine, cache, quotas, config, clock, log, userModules, credentials });-- seed({ registry, quotas, config, env, db });-- const interrupted = runs.recoverInterrupted();++ await seed({ registry, quotas, config, env, db });++ const interrupted = await runs.recoverInterrupted();if (interrupted) log.warn(`[ai] marked ${interrupted} interrupted run(s) failed (run.interrupted); callers can retry them`);const keys = createKeyStore({ urls: [config.networkInternalUrl, config.networkUrl], pem: config.networkPublicKey, fetchImpl, log });@@@ -48,10 +49,10 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = glconst app = createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env, clock, fetchImpl, log, credentials });const keyLoaded = keys.start().catch(() => null);-- const housekeeping = setInterval(() => {++ const housekeeping = setInterval(async () => {try {-- const pruned = runs.prune();-- const expired = cache.prune();++ const pruned = await runs.prune();++ const expired = await cache.prune();if (pruned || expired) log.log(`[ai] retention: ${pruned} old run(s), ${expired} expired cache entr${expired === 1 ? 'y' : 'ies'}`);} catch (err) { log.error(`[ai] retention failed: ${err.message}`); }}, 60 * 60 * 1000);@@@ -68,7 +69,7 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = glserver.headersTimeout = 66000;server.requestTimeout = config.runs.maxWaitMs + 30000;log.log(`[ai] listening on http://${config.host}:${server.address().port} (${config.nodeEnv})`);-- const shared = registry.getProvider('shared');++ const shared = await registry.getProvider('shared');log.log(`[ai] shared provider: ${shared.kind} ${shared.status}, credentials ${registry.publicProvider(shared).credentials}; stub fallback ${config.stubFallback ? 'on' : 'off'}`);}@@@ -82,9 +83,9 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = gl}await runs.drain();await require('./events').stop(); // after the runs: their last ai.run.* rows are queued first-- const w = pool.adapter('whisper');++ const w = await pool.adapter('whisper');if (w && w.adapter.killActive) w.adapter.killActive();-- db.close();++ if (!givenDb) await db.close();}return { config, db, registry, pool, quotas, cache, fetcher, engine, runs, keys, keyLoaded, auth, app, server, close };@@@ -20,11 +20,11 @@ const { createReadiness } = require('openvibe-shared/ready');const CIRCUIT_STATES = ['closed', 'half_open', 'open'];/** Active providers with what they need to answer: credentials and circuit state. */--function providerView(registry, pool) {-- return registry.listProviders().filter(p => p.status === 'active').map((p) => {++async function providerView(registry, pool) {++ return (await Promise.all((await registry.listProviders()).filter(p => p.status === 'active').map(async (p) => {const pub = registry.publicProvider(p);-- return { key: p.key, kind: p.kind, credentials: pub.credentials, circuit: pool.health(p.key).state };-- });++ return { key: p.key, kind: p.kind, credentials: pub.credentials, circuit: (await pool.health(p.key)).state };++ })));}function createAiReadiness({ db, registry, pool, keys, runs, release = null }) {@@@ -32,19 +32,19 @@ function createAiReadiness({ db, registry, pool, keys, runs, release = null }) {service: 'ai',release,checks: [-- { name: 'db', required: true, check: () => (db.prepare('SELECT COUNT(*) AS n FROM providers').get().n > 0 ? true : 'the provider registry is empty') },++ { name: 'db', required: true, check: async () => ((await db.prepare('SELECT COUNT(*) AS n FROM providers').get()).n > 0 ? true : 'the provider registry is empty') },{name: 'workflows', required: true,-- check: () => {-- const n = db.prepare("SELECT COUNT(DISTINCT key) AS n FROM workflows WHERE status = 'active'").get().n;++ check: async () => {++ const n = (await db.prepare("SELECT COUNT(DISTINCT key) AS n FROM workflows WHERE status = 'active'").get()).n;return n > 0 ? { ok: true, detail: { active: n } } : 'no active workflow';},},{ name: 'network_jwks', required: true, check: () => keys.loaded() || 'Network signing key not loaded yet: no service token can be verified' },{name: 'providers', required: false,-- check: () => {-- const list = providerView(registry, pool);++ check: async () => {++ const list = await providerView(registry, pool);const real = list.filter(p => p.kind !== 'stub');const detail = { active: list.length, usable: real.filter(p => p.credentials !== 'missing' && p.circuit !== 'open').length };if (!real.length) return { ok: false, error: 'no real provider is active: only synthetic stub output is available', detail };@@@ -54,8 +54,8 @@ function createAiReadiness({ db, registry, pool, keys, runs, release = null }) {},},],-- details: (body) => ({-- providers: body.checks.db.status === 'ok' ? providerView(registry, pool) : null,++ details: async (body) => ({++ providers: body.checks.db.status === 'ok' ? await providerView(registry, pool) : null,runs: runs.stats(),}),});@@@ -74,11 +74,11 @@ function registerAiGauges(registry, { runs, registry: ai, pool }) {registry.gauge({ name: 'ai_runs_max_queued', help: 'Runs allowed to wait for a slot (AI_MAX_QUEUED_RUNS)', collect: () => runs.stats().max_queued });registry.gauge({name: 'ai_provider_circuit', help: 'Circuit-breaker state of each active provider (1 = the current state)', labelNames: ['provider', 'state'],-- collect: () => providerView(ai, pool).flatMap(p => CIRCUIT_STATES.map(state => ({ labels: { provider: p.key, state }, value: p.circuit === state ? 1 : 0 }))),++ collect: async () => (await providerView(ai, pool)).flatMap(p => CIRCUIT_STATES.map(state => ({ labels: { provider: p.key, state }, value: p.circuit === state ? 1 : 0 }))),});registry.gauge({name: 'ai_provider_credentials_missing', help: 'Active providers whose secret reference does not resolve (1 = missing)', labelNames: ['provider'],-- collect: () => providerView(ai, pool).filter(p => p.credentials !== 'not_required').map(p => ({ labels: { provider: p.key }, value: p.credentials === 'missing' ? 1 : 0 })),++ collect: async () => (await providerView(ai, pool)).filter(p => p.credentials !== 'not_required').map(p => ({ labels: { provider: p.key }, value: p.credentials === 'missing' ? 1 : 0 })),});}@@@ -13,39 +13,39 @@ const { AiError } = require('./util');function createOps({ db, registry, pool, quotas, cache, runs }) {/** A provider as every operator surface shows it: secret reference NAME + whether it resolves, health. */-- const providerView = (p) => registry.publicProvider(p, pool.health(p.key));++ const providerView = async (p) => registry.publicProvider(p, await pool.health(p.key));/** GET /api/v1/status: providers, run counts, today's spend, definitions, queue, quota counters. */-- function status() {-- const counts = Object.fromEntries(db.prepare('SELECT status, COUNT(*) n FROM runs GROUP BY status').all().map(x => [x.status, x.n]));++ async function status() {++ const counts = Object.fromEntries((await db.prepare('SELECT status, COUNT(*) n FROM runs GROUP BY status').all()).map(x => [x.status, x.n]));const today = new Date().toISOString().slice(0, 10);-- const spend = db.prepare('SELECT COALESCE(SUM(cost_usd),0) cost, COALESCE(SUM(requests),0) requests FROM usage_daily WHERE day = ?').get(today);++ const spend = await db.prepare('SELECT COALESCE(SUM(cost_usd),0) cost, COALESCE(SUM(requests),0) requests FROM usage_daily WHERE day = ?').get(today);return {-- providers: registry.listProviders().map(p => ({ key: p.key, kind: p.kind, status: p.status, credentials: providerView(p).credentials, health: pool.health(p.key).state })),++ providers: (await Promise.all((await registry.listProviders()).map(async p => ({ key: p.key, kind: p.kind, status: p.status, credentials: (await providerView(p)).credentials, health: (await pool.health(p.key)).state })))),runs: counts, today: { day: today, requests: spend.requests, cost_usd: spend.cost },-- workflows: registry.listWorkflows().length, templates: registry.listTemplates().length, routes: registry.listRoutes().length,-- inflight: runs.inflight.size, quotas: quotas.counters(),++ workflows: (await registry.listWorkflows()).length, templates: (await registry.listTemplates()).length, routes: (await registry.listRoutes()).length,++ inflight: runs.inflight.size, quotas: await quotas.counters(),};}/** Enable or disable a provider (a registry upsert of its status; audited as provider.update). */-- function setProviderStatus(key, status, { actor, trace = null }) {-- if (!registry.getProvider(key)) throw new AiError(404, 'ai.not_found', 'no such provider');++ async function setProviderStatus(key, status, { actor, trace = null }) {++ if (!await registry.getProvider(key)) throw new AiError(404, 'ai.not_found', 'no such provider');if (!['active', 'disabled'].includes(status)) throw new AiError(422, 'ai.invalid', 'provider status must be active or disabled');-- return registry.upsertProvider({ key, status }, { actor, trace, origin: 'admin' });++ return await registry.upsertProvider({ key, status }, { actor, trace, origin: 'admin' });}/** Close a provider's circuit (breaker reset); audited as provider.circuit_reset. */-- function resetCircuit(key, { actor, trace = null }) {-- pool.resetHealth(key);-- registry.audit(actor, 'provider.circuit_reset', 'provider', key, { trace });-- return pool.health(key);++ async function resetCircuit(key, { actor, trace = null }) {++ await pool.resetHealth(key);++ await registry.audit(actor, 'provider.circuit_reset', 'provider', key, { trace });++ return await pool.health(key);}/** Remove cache entries: one workflow's, or every entry when `workflow` is empty; audited as cache.purge. */-- function purgeCache(workflow, { actor, trace = null }) {-- const removed = cache.purge({ workflow: workflow || undefined });-- registry.audit(actor, 'cache.purge', 'cache', workflow || '*', { trace, metadata: { removed } });++ async function purgeCache(workflow, { actor, trace = null }) {++ const removed = await cache.purge({ workflow: workflow || undefined });++ await registry.audit(actor, 'cache.purge', 'cache', workflow || '*', { trace, metadata: { removed } });return removed;}@@@ -40,7 +40,7 @@ function createHttpSeamProvider(record, { apiKey = '', fetchImpl = globalThis.fe}const adapter = { key: record.key, kind: 'http', supports: (f) => caps.has(f) };-- for (const op of OPS) adapter[op] = (req) => call(op, req);++ for (const op of OPS) adapter[op] = async (req) => await call(op, req);return adapter;}@@@ -47,11 +47,11 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no}}-- function adapter(key) {-- const p = registry.getProvider(key);++ async function adapter(key) {++ const p = await registry.getProvider(key);if (!p) return null;const stamp = `${p.updated_at}|${p.secret_ref}|${resolveSecret(p.secret_ref, env) ? 1 : 0}`;-- const hit = cache.get(key);++ const hit = await cache.get(key);if (hit && hit.stamp === stamp) return { record: p, adapter: hit.adapter };const a = build(p);cache.set(key, { stamp, adapter: a });@@@ -64,30 +64,30 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.noVALUES (@provider_key, @state, @consecutive_failures, @opened_at, @last_error, @last_success_at, @last_failure_at)ON CONFLICT(provider_key) DO UPDATE SET state = excluded.state, consecutive_failures = excluded.consecutive_failures, opened_at = excluded.opened_at,last_error = excluded.last_error, last_success_at = excluded.last_success_at, last_failure_at = excluded.last_failure_at`);-- function health(key) {-- const h = getHealth.get(key) || { provider_key: key, state: 'closed', consecutive_failures: 0, opened_at: null, last_error: null, last_success_at: null, last_failure_at: null };++ async function health(key) {++ const h = await getHealth.get(key) || { provider_key: key, state: 'closed', consecutive_failures: 0, opened_at: null, last_error: null, last_success_at: null, last_failure_at: null };if (h.state === 'open' && clock.now() - (h.opened_at || 0) >= config.breaker.cooldownMs) return { ...h, state: 'half_open' };return h;}-- function recordSuccess(key) {-- const h = health(key);-- if (h.state !== 'closed') registry.audit('system', 'provider.circuit_closed', 'provider', key, { metadata: { after_failures: h.consecutive_failures } });-- putHealth.run({ ...h, provider_key: key, state: 'closed', consecutive_failures: 0, opened_at: null, last_success_at: clock.now() });++ async function recordSuccess(key) {++ const h = await health(key);++ if (h.state !== 'closed') await registry.audit('system', 'provider.circuit_closed', 'provider', key, { metadata: { after_failures: h.consecutive_failures } });++ await putHealth.run({ ...h, provider_key: key, state: 'closed', consecutive_failures: 0, opened_at: null, last_success_at: clock.now() });}-- function recordFailure(key, err) {-- const h = health(key);++ async function recordFailure(key, err) {++ const h = await health(key);const failures = (h.consecutive_failures || 0) + 1;const open = h.state === 'half_open' || failures >= config.breaker.failureThreshold;-- if (open && h.state !== 'open') registry.audit('system', 'provider.circuit_opened', 'provider', key, { metadata: { failures, error: String(err && err.message || err).slice(0, 200) } });-- putHealth.run({ ...h, provider_key: key, state: open ? 'open' : 'closed', consecutive_failures: failures, opened_at: open ? clock.now() : h.opened_at, last_error: String(err && err.message || err).slice(0, 500), last_failure_at: clock.now() });++ if (open && h.state !== 'open') await registry.audit('system', 'provider.circuit_opened', 'provider', key, { metadata: { failures, error: String(err && err.message || err).slice(0, 200) } });++ await putHealth.run({ ...h, provider_key: key, state: open ? 'open' : 'closed', consecutive_failures: failures, opened_at: open ? clock.now() : h.opened_at, last_error: String(err && err.message || err).slice(0, 500), last_failure_at: clock.now() });}-- function resetHealth(key) { db.prepare('DELETE FROM provider_health WHERE provider_key = ?').run(key); }++ async function resetHealth(key) { await db.prepare('DELETE FROM provider_health WHERE provider_key = ?').run(key); }// ── Pricing (Live's rules: model row -> AI_PRICING_JSON longest prefix -> flat rates) ──-- function priceFor(providerKey, model) {-- const p = registry.getProvider(providerKey);++ async function priceFor(providerKey, model) {++ const p = await registry.getProvider(providerKey);if (!p || p.kind === 'stub' || p.kind === 'whisper') return { in: 0, out: 0, cached: 0 };-- const row = model ? registry.getModel(providerKey, model) : null;++ const row = model ? await registry.getModel(providerKey, model) : null;if (row && row.cost && Number.isFinite(row.cost.in_per_mtok) && row.cost.in_per_mtok !== null) {const inRate = Number(row.cost.in_per_mtok);return { in: inRate, out: Number(row.cost.out_per_mtok) || 0, cached: row.cost.cached_per_mtok != null ? Number(row.cost.cached_per_mtok) : inRate * 0.1 };@@@ -106,25 +106,25 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.noconst cachedRate = best && Number.isFinite(Number(best.cached)) ? Number(best.cached) : inRate * 0.1;return { in: inRate, out: outRate, cached: cachedRate };}-- function costOf(providerKey, model, usage) {-- const pr = priceFor(providerKey, model);++ async function costOf(providerKey, model, usage) {++ const pr = await priceFor(providerKey, model);const input = Math.max(0, (usage.input || 0) - (usage.cached || 0));return (input / 1e6) * pr.in + ((usage.cached || 0) / 1e6) * pr.cached + ((usage.output || 0) / 1e6) * pr.out;}// ── Routing ────────────────────────────────────────────-- function candidates(route) {++ async function candidates(route) {const list = [route.primary, ...(route.fallbacks || [])].filter(c => c && c.provider);-- if (config.stubFallback && !list.some(c => c.provider === 'stub') && registry.getProvider('stub')) list.push({ provider: 'stub', model: null, auto: true });++ if (config.stubFallback && !list.some(c => c.provider === 'stub') && await registry.getProvider('stub')) list.push({ provider: 'stub', model: null, auto: true });return list;}-- function skipReason(p, a, features) {++ async function skipReason(p, a, features) {if (!p || !a) return 'unknown_provider';if (p.status !== 'active') return 'disabled';if (p.auth_mode !== 'none' && !resolveSecret(p.secret_ref, env)) return 'no_credentials';if (features.some(f => !a.supports(f))) return 'unsupported';-- if (health(p.key).state === 'open') return 'circuit_open';++ if ((await health(p.key)).state === 'open') return 'circuit_open';return null;}@@@ -186,21 +186,21 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no}async function execute(route, operation, req, ctx = {}) {-- if (ctx.credential) return executeWithCredential(ctx.credential, operation, req, ctx);++ if (ctx.credential) return await executeWithCredential(ctx.credential, operation, req, ctx);const features = [operation];if (req.image) features.push('vision');if (req.json) features.push('json');const tried = [];-- const list = candidates(route);++ const list = await candidates(route);for (let i = 0; i < list.length; i++) {const c = list[i];-- const entry = adapter(c.provider);++ const entry = await adapter(c.provider);const p = entry && entry.record;const a = entry && entry.adapter;const model = c.model || (p && p.default_model) || null;const fallback = i > 0;const base = { operation, provider_key: c.provider, model_key: model, route_key: ctx.routeKey || null, route_version: ctx.routeVersion || null, fallback: fallback ? 1 : 0, prompt_hash: ctx.promptHash || null, input_hash: ctx.inputHash || null };-- const why = skipReason(p, a, features);++ const why = await skipReason(p, a, features);if (why) {statFor(c.provider).skipped++;tried.push({ provider: c.provider, skipped: why });@@@ -221,9 +221,9 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.noconst result = await fn({ ...req, model, signal: ctx.signal, timeoutMs });const latency = Date.now() - t0;const usage = result.usage || { input: 0, output: 0, cached: 0 };-- const cost = costOf(c.provider, result.model || model, usage);++ const cost = await costOf(c.provider, result.model || model, usage);statFor(c.provider).ok++;-- recordSuccess(c.provider);++ await recordSuccess(c.provider);if (ctx.logRequest) {ctx.logRequest({...base, model_key: result.model || model, status: 'ok', output_hash: sha256(result.json || result.text || result.vectors || result.segments || ''),@@@ -248,7 +248,7 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no}// A request the provider rejected as malformed is the caller's fault, not a sign the// provider is unhealthy: it must not open the circuit other callers depend on.-- if (!callerFault(lastErr)) recordFailure(c.provider, lastErr);++ if (!callerFault(lastErr)) await recordFailure(c.provider, lastErr);tried.push({ provider: c.provider, error: String(lastErr && lastErr.message || lastErr).slice(0, 200) });log.warn(`[ai] ${operation} via ${c.provider}/${model || '-'} failed: ${lastErr && lastErr.message}`);}@@@ -146,10 +146,10 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nukind: 'stub',synthetic: true,supports: () => true,-- transcribe: (req) => run('transcribe', req),-- embed: (req) => run('embed', req),++ transcribe: async (req) => await run('transcribe', req),++ embed: async (req) => await run('embed', req),};-- for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich']) adapter[op] = (req) => run(op, req);++ for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich']) adapter[op] = async (req) => await run(op, req);return adapter;}@@@ -35,9 +35,9 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})* scope_id is '*' on a non-global scope applies to EACH principal of that type separately* (e.g. service '*' = every service gets its own window).*/-- function applicable(ctx) {++ async function applicable(ctx) {const scopes = scopesFor(ctx);-- const rows = db.prepare("SELECT * FROM quotas WHERE status = 'active'").all();++ const rows = await db.prepare("SELECT * FROM quotas WHERE status = 'active'").all();const out = [];for (const q of rows) {if (q.workflow_prefix && !String(ctx.workflowKey || '').startsWith(q.workflow_prefix)) continue;@@@ -47,19 +47,19 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})return out;}-- const getCounter = db.prepare('SELECT * FROM usage_counters WHERE scope_type = ? AND scope_id = ? AND window = ? AND window_start = ? AND workflow_prefix = ?');-- const bump = db.prepare(`INSERT INTO usage_counters (scope_type, scope_id, window, window_start, workflow_prefix, requests, tokens, cost_usd)++ const getCounter = db.prepare('SELECT * FROM usage_counters WHERE scope_type = ? AND scope_id = ? AND "window" = ? AND window_start = ? AND workflow_prefix = ?');++ const bump = db.prepare(`INSERT INTO usage_counters (scope_type, scope_id, "window", window_start, workflow_prefix, requests, tokens, cost_usd)VALUES (?, ?, ?, ?, ?, ?, ?, ?)-- ON CONFLICT(scope_type, scope_id, window, window_start, workflow_prefix)-- DO UPDATE SET requests = requests + excluded.requests, tokens = tokens + excluded.tokens, cost_usd = cost_usd + excluded.cost_usd`);++ ON CONFLICT(scope_type, scope_id, "window", window_start, workflow_prefix)++ DO UPDATE SET requests = usage_counters.requests + excluded.requests, tokens = usage_counters.tokens + excluded.tokens, cost_usd = usage_counters.cost_usd + excluded.cost_usd`);-- const reserveTx = db.transaction((ctx) => {-- const qs = applicable(ctx);++ const reserveTx = async (ctx) => await db.tx(async () => {++ const qs = await applicable(ctx);const t = nowSec();let worst = null;for (const { q, cid } of qs) {const ws = windowStart(q.window, t);-- const c = getCounter.get(q.scope_type, cid, q.window, ws, q.workflow_prefix || '') || { requests: 0, tokens: 0, cost_usd: 0 };++ const c = await getCounter.get(q.scope_type, cid, q.window, ws, q.workflow_prefix || '') || { requests: 0, tokens: 0, cost_usd: 0 };let reason = null;if (q.max_requests != null && c.requests + 1 > q.max_requests) reason = `${q.max_requests} requests per ${q.window}`;else if (q.max_tokens != null && c.tokens >= q.max_tokens) reason = `${q.max_tokens} tokens per ${q.window}`;@@@ -81,13 +81,13 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})if (q.workflow_prefix) keys.set(`${q.scope_type}|${cid}|${q.window}|${q.workflow_prefix}`, { scope_type: q.scope_type, scope_id: cid, window: q.window, workflow_prefix: q.workflow_prefix, ws: windowStart(q.window, t) });}const counted = [...keys.values()];-- for (const k of counted) bump.run(k.scope_type, k.scope_id, k.window, k.ws, k.workflow_prefix, 1, 0, 0);++ for (const k of counted) await bump.run(k.scope_type, k.scope_id, k.window, k.ws, k.workflow_prefix, 1, 0, 0);return { ok: true, quotas: counted };});/** Check + count one request. Throws 429 quota.exceeded before any provider is touched. */-- function reserve(ctx) {-- const r = reserveTx(ctx);++ async function reserve(ctx) {++ const r = await reserveTx(ctx);if (!r.ok) {throw new AiError(429, 'quota.exceeded', `quota exceeded for ${r.q.scope_type} ${r.cid}: ${r.reason}`, {retry_after_seconds: Math.max(1, r.retryAfter), quota: { id: r.q.id, scope_type: r.q.scope_type, scope_id: r.cid, window: r.q.window },@@@ -98,21 +98,21 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})const insDaily = db.prepare(`INSERT INTO usage_daily (day, requester, attribution, workflow_key, provider_key, model_key, requests, tokens_in, tokens_out, tokens_cached, cost_usd)VALUES (?, ?, ?, ?, ?, ?, 1, ?, ?, ?, ?)-- ON CONFLICT(day, requester, attribution, workflow_key, provider_key, model_key) DO UPDATE SET requests = requests + 1,-- tokens_in = tokens_in + excluded.tokens_in, tokens_out = tokens_out + excluded.tokens_out, tokens_cached = tokens_cached + excluded.tokens_cached, cost_usd = cost_usd + excluded.cost_usd`);++ ON CONFLICT(day, requester, attribution, workflow_key, provider_key, model_key) DO UPDATE SET requests = usage_daily.requests + 1,++ tokens_in = usage_daily.tokens_in + excluded.tokens_in, tokens_out = usage_daily.tokens_out + excluded.tokens_out, tokens_cached = usage_daily.tokens_cached + excluded.tokens_cached, cost_usd = usage_daily.cost_usd + excluded.cost_usd`);/** Add real usage to the reserved windows and the daily usage table. */-- function account(ctx, reserved, { provider, model, tokensIn = 0, tokensOut = 0, tokensCached = 0, cost = 0 }) {++ async function account(ctx, reserved, { provider, model, tokensIn = 0, tokensOut = 0, tokensCached = 0, cost = 0 }) {const tokens = tokensIn + tokensOut;-- db.transaction(() => {-- for (const r of reserved || []) bump.run(r.scope_type, r.scope_id, r.window, r.ws, r.workflow_prefix, 0, tokens, cost);-- insDaily.run(iso(clock.now()).slice(0, 10), `${ctx.requesterType}:${ctx.requesterId}`, ctx.attributionKey || '', ctx.workflowKey, provider || '', model || '', tokensIn, tokensOut, tokensCached, cost);-- })();++ await db.tx(async () => {++ for (const r of reserved || []) await bump.run(r.scope_type, r.scope_id, r.window, r.ws, r.workflow_prefix, 0, tokens, cost);++ await insDaily.run(iso(clock.now()).slice(0, 10), `${ctx.requesterType}:${ctx.requesterId}`, ctx.attributionKey || '', ctx.workflowKey, provider || '', model || '', tokensIn, tokensOut, tokensCached, cost);++ });}// ── Admin ──-- function list() { return db.prepare('SELECT * FROM quotas ORDER BY scope_type, scope_id, window').all(); }-- function upsert(q, { actor = 'system', origin = 'admin', trace = null } = {}) {++ async function list() { return await db.prepare('SELECT * FROM quotas ORDER BY scope_type, scope_id, "window"').all(); }++ async function upsert(q, { actor = 'system', origin = 'admin', trace = null } = {}) {if (!['global', 'service', 'actor', 'attribution'].includes(q.scope_type)) throw new AiError(422, 'ai.invalid', 'scope_type must be global, service, actor or attribution');if (!WINDOWS[q.window]) throw new AiError(422, 'ai.invalid', 'window must be minute, hour or day');const scopeId = q.scope_type === 'global' ? '*' : String(q.scope_id || ''); // '*' on other scopes = each one separately@@@ -123,36 +123,36 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})if (!['active', 'disabled'].includes(status)) throw new AiError(422, 'ai.invalid', 'status must be active or disabled');const t = iso(clock.now());const prefix = q.workflow_prefix || null;-- const prev = db.prepare("SELECT * FROM quotas WHERE scope_type = ? AND scope_id = ? AND window = ? AND COALESCE(workflow_prefix, '') = ?").get(q.scope_type, scopeId, q.window, prefix || '');++ const prev = await db.prepare("SELECT * FROM quotas WHERE scope_type = ? AND scope_id = ? AND \"window\" = ? AND COALESCE(workflow_prefix, '') = ?").get(q.scope_type, scopeId, q.window, prefix || '');if (prev) {-- db.prepare('UPDATE quotas SET max_requests = ?, max_tokens = ?, max_cost_usd = ?, status = ?, updated_at = ? WHERE id = ?')++ await db.prepare('UPDATE quotas SET max_requests = ?, max_tokens = ?, max_cost_usd = ?, status = ?, updated_at = ? WHERE id = ?').run(num(q.max_requests), num(q.max_tokens), num(q.max_cost_usd), status, t, prev.id);} else {-- db.prepare('INSERT INTO quotas (scope_type, scope_id, window, max_requests, max_tokens, max_cost_usd, workflow_prefix, status, origin, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)')++ await db.prepare('INSERT INTO quotas (scope_type, scope_id, "window", max_requests, max_tokens, max_cost_usd, workflow_prefix, status, origin, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)').run(q.scope_type, scopeId, q.window, num(q.max_requests), num(q.max_tokens), num(q.max_cost_usd), prefix, status, origin, t, t);}-- const row = db.prepare("SELECT * FROM quotas WHERE scope_type = ? AND scope_id = ? AND window = ? AND COALESCE(workflow_prefix, '') = ?").get(q.scope_type, scopeId, q.window, prefix || '');-- if (registry) registry.audit(actor, prev ? 'quota.update' : 'quota.create', 'quota', row.id, { trace, metadata: { scope_type: row.scope_type, scope_id: row.scope_id, window: row.window, max_requests: row.max_requests, max_tokens: row.max_tokens, max_cost_usd: row.max_cost_usd, status: row.status, origin } });++ const row = await db.prepare("SELECT * FROM quotas WHERE scope_type = ? AND scope_id = ? AND \"window\" = ? AND COALESCE(workflow_prefix, '') = ?").get(q.scope_type, scopeId, q.window, prefix || '');++ if (registry) await registry.audit(actor, prev ? 'quota.update' : 'quota.create', 'quota', row.id, { trace, metadata: { scope_type: row.scope_type, scope_id: row.scope_id, window: row.window, max_requests: row.max_requests, max_tokens: row.max_tokens, max_cost_usd: row.max_cost_usd, status: row.status, origin } });return row;}-- function usage({ from, to, requester } = {}) {++ async function usage({ from, to, requester } = {}) {const where = [];const args = [];if (from) { where.push('day >= ?'); args.push(from); }if (to) { where.push('day <= ?'); args.push(to); }if (requester) { where.push('requester = ?'); args.push(requester); }-- return db.prepare(`SELECT * FROM usage_daily ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY day DESC, cost_usd DESC LIMIT 1000`).all(...args);++ return await db.prepare(`SELECT * FROM usage_daily ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY day DESC, cost_usd DESC LIMIT 1000`).all(...args);}-- function counters(ctxLike) {-- const qs = ctxLike ? applicable(ctxLike) : list().filter(q => q.status === 'active' && (q.scope_type === 'global' || q.scope_id !== '*')).map(q => ({ q, cid: q.scope_id }));++ async function counters(ctxLike) {++ const qs = ctxLike ? await applicable(ctxLike) : (await list()).filter(q => q.status === 'active' && (q.scope_type === 'global' || q.scope_id !== '*')).map(q => ({ q, cid: q.scope_id }));const t = nowSec();-- return qs.map(({ q, cid }) => {++ return (await Promise.all(qs.map(async ({ q, cid }) => {const ws = windowStart(q.window, t);-- const c = getCounter.get(q.scope_type, cid, q.window, ws, q.workflow_prefix || '') || { requests: 0, tokens: 0, cost_usd: 0 };++ const c = await getCounter.get(q.scope_type, cid, q.window, ws, q.workflow_prefix || '') || { requests: 0, tokens: 0, cost_usd: 0 };return { quota: q, scope_id: cid, window_start: iso(ws * 1000), resets_in_seconds: ws + WINDOWS[q.window] - t, used: { requests: c.requests, tokens: c.tokens, cost_usd: c.cost_usd } };-- });++ })));}/**@@@ -161,11 +161,11 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})* applies them) and whose max_cost_usd is above 0. The tightest cap counts, against today's (UTC) spend of the* workflows starting with `spendPrefix`. null when there is none: no budget, no paid call.*/-- function paidBudget(prefix, spendPrefix) {-- const rows = db.prepare("SELECT max_cost_usd FROM quotas WHERE status = 'active' AND workflow_prefix = ? AND max_cost_usd > 0").all(prefix);++ async function paidBudget(prefix, spendPrefix) {++ const rows = await db.prepare("SELECT max_cost_usd FROM quotas WHERE status = 'active' AND workflow_prefix = ? AND max_cost_usd > 0").all(prefix);if (!rows.length) return null;const day = new Date(clock.now()).toISOString().slice(0, 10);-- const spent = db.prepare("SELECT COALESCE(SUM(cost_usd), 0) AS c FROM usage_daily WHERE day = ? AND substr(workflow_key, 1, ?) = ?").get(day, spendPrefix.length, spendPrefix).c;++ const spent = (await db.prepare("SELECT COALESCE(SUM(cost_usd), 0) AS c FROM usage_daily WHERE day = ? AND substr(workflow_key, 1, ?) = ?").get(day, spendPrefix.length, spendPrefix)).c;return { max_cost_usd: Math.min(...rows.map((r) => r.max_cost_usd)), spent, window: 'day' };}@@@ -22,17 +22,17 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e// ── Audit ──────────────────────────────────────────────const insAudit = db.prepare('INSERT INTO audit_log (at, actor, action, target_type, target_id, trace_id, metadata) VALUES (?, ?, ?, ?, ?, ?, ?)');-- function audit(actor, action, targetType, targetId, { trace = null, metadata = {} } = {}) {-- insAudit.run(now(), actor || 'system', action, targetType || null, targetId == null ? null : String(targetId), trace, JSON.stringify(metadata || {}));++ async function audit(actor, action, targetType, targetId, { trace = null, metadata = {} } = {}) {++ await insAudit.run(now(), actor || 'system', action, targetType || null, targetId == null ? null : String(targetId), trace, JSON.stringify(metadata || {}));}-- function listAudit({ action, targetType, targetId, limit = 100, before } = {}) {++ async function listAudit({ action, targetType, targetId, limit = 100, before } = {}) {const where = [];const args = [];if (action) { where.push('action = ?'); args.push(action); }if (targetType) { where.push('target_type = ?'); args.push(targetType); }if (targetId) { where.push('target_id = ?'); args.push(String(targetId)); }if (before) { where.push('id < ?'); args.push(Number(before)); }-- const rows = db.prepare(`SELECT * FROM audit_log ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(500, limit));++ const rows = await db.prepare(`SELECT * FROM audit_log ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(500, limit));return rows.map(r => ({ ...r, metadata: parseJson(r.metadata, {}) }));}@@@ -46,8 +46,8 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.emetadata: parseJson(r.metadata, {}), origin: r.origin, created_at: r.created_at, updated_at: r.updated_at,};}-- function getProvider(key) { return decodeProvider(db.prepare('SELECT * FROM providers WHERE key = ?').get(key)); }-- function listProviders() { return db.prepare('SELECT * FROM providers ORDER BY priority, key').all().map(decodeProvider); }++ async function getProvider(key) { return decodeProvider(await db.prepare('SELECT * FROM providers WHERE key = ?').get(key)); }++ async function listProviders() { return (await db.prepare('SELECT * FROM providers ORDER BY priority, key').all()).map(decodeProvider); }/** Public view: the secret reference name and whether it resolves — never the value. */function publicProvider(p, health) {@@@ -66,8 +66,8 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.eif (!Array.isArray(p.capabilities) || p.capabilities.some(f => !FEATURES.includes(f))) throw new AiError(422, 'ai.invalid', `capabilities must be a subset of ${FEATURES.join(', ')}`);}-- function upsertProvider(input, { actor = 'system', origin = 'admin', trace = null } = {}) {-- const prev = getProvider(input.key);++ async function upsertProvider(input, { actor = 'system', origin = 'admin', trace = null } = {}) {++ const prev = await getProvider(input.key);const p = {key: input.key,display_name: input.display_name || (prev && prev.display_name) || input.key,@@@ -84,14 +84,14 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e};checkProvider(p);const t = now();-- db.prepare(`INSERT INTO providers (key, display_name, kind, status, base_url, auth_mode, secret_ref, default_model, capabilities, timeout_ms, priority, metadata, origin, created_at, updated_at)++ await db.prepare(`INSERT INTO providers (key, display_name, kind, status, base_url, auth_mode, secret_ref, default_model, capabilities, timeout_ms, priority, metadata, origin, created_at, updated_at)VALUES (@key, @display_name, @kind, @status, @base_url, @auth_mode, @secret_ref, @default_model, @capabilities, @timeout_ms, @priority, @metadata, @origin, @t, @t)ON CONFLICT(key) DO UPDATE SET display_name = excluded.display_name, kind = excluded.kind, status = excluded.status, base_url = excluded.base_url,auth_mode = excluded.auth_mode, secret_ref = excluded.secret_ref, default_model = excluded.default_model, capabilities = excluded.capabilities,timeout_ms = excluded.timeout_ms, priority = excluded.priority, metadata = excluded.metadata, origin = excluded.origin, updated_at = excluded.updated_at`).run({ ...p, capabilities: JSON.stringify(p.capabilities), metadata: JSON.stringify(p.metadata), origin, t });-- audit(actor, prev ? 'provider.update' : 'provider.create', 'provider', p.key, { trace, metadata: { kind: p.kind, status: p.status, base_url: p.base_url, secret_ref: p.secret_ref, origin } });-- return getProvider(p.key);++ await audit(actor, prev ? 'provider.update' : 'provider.create', 'provider', p.key, { trace, metadata: { kind: p.kind, status: p.status, base_url: p.base_url, secret_ref: p.secret_ref, origin } });++ return await getProvider(p.key);}// ── Models ─────────────────────────────────────────────@@@ -105,15 +105,15 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.emetadata: parseJson(r.metadata, {}), created_at: r.created_at, updated_at: r.updated_at,};}-- function getModel(providerKey, modelKey) { return decodeModel(db.prepare('SELECT * FROM models WHERE provider_key = ? AND model_key = ?').get(providerKey, modelKey)); }-- function listModels({ provider } = {}) {-- const rows = provider ? db.prepare('SELECT * FROM models WHERE provider_key = ? ORDER BY model_key').all(provider) : db.prepare('SELECT * FROM models ORDER BY provider_key, model_key').all();++ async function getModel(providerKey, modelKey) { return decodeModel(await db.prepare('SELECT * FROM models WHERE provider_key = ? AND model_key = ?').get(providerKey, modelKey)); }++ async function listModels({ provider } = {}) {++ const rows = provider ? await db.prepare('SELECT * FROM models WHERE provider_key = ? ORDER BY model_key').all(provider) : await db.prepare('SELECT * FROM models ORDER BY provider_key, model_key').all();return rows.map(decodeModel);}-- function upsertModel(input, { actor = 'system', trace = null } = {}) {-- if (!getProvider(input.provider_key)) throw new AiError(404, 'ai.not_found', `no provider ${input.provider_key}`);++ async function upsertModel(input, { actor = 'system', trace = null } = {}) {++ if (!await getProvider(input.provider_key)) throw new AiError(404, 'ai.not_found', `no provider ${input.provider_key}`);if (!input.model_key || String(input.model_key).length > 200) throw new AiError(422, 'ai.invalid', 'model_key required');-- const prev = getModel(input.provider_key, input.model_key);++ const prev = await getModel(input.provider_key, input.model_key);const type = input.type || (prev && prev.type) || 'chat';if (!['chat', 'vision', 'embedding', 'stt'].includes(type)) throw new AiError(422, 'ai.invalid', 'model type must be chat, vision, embedding or stt');const status = input.status || (prev && prev.status) || 'active';@@@ -121,7 +121,7 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.econst cost = input.cost || (prev && prev.cost) || {};const sup = input.supports || (prev && prev.supports) || {};const t = now();-- db.prepare(`INSERT INTO models (provider_key, model_key, display_name, type, status, context_window, max_output, cost_in_per_mtok, cost_out_per_mtok, cost_cached_per_mtok,++ await db.prepare(`INSERT INTO models (provider_key, model_key, display_name, type, status, context_window, max_output, cost_in_per_mtok, cost_out_per_mtok, cost_cached_per_mtok,supports_json, supports_tools, supports_streaming, supports_vision, metadata, created_at, updated_at)VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)ON CONFLICT(provider_key, model_key) DO UPDATE SET display_name = excluded.display_name, type = excluded.type, status = excluded.status,@@@ -134,20 +134,20 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.ecost.in_per_mtok ?? null, cost.out_per_mtok ?? null, cost.cached_per_mtok ?? null,sup.json === false ? 0 : 1, sup.tools ? 1 : 0, sup.streaming ? 1 : 0, sup.vision ? 1 : 0,JSON.stringify(input.metadata || (prev && prev.metadata) || {}), t, t);-- audit(actor, prev ? 'model.update' : 'model.create', 'model', `${input.provider_key}/${input.model_key}`, { trace, metadata: { status, type } });-- return getModel(input.provider_key, input.model_key);++ await audit(actor, prev ? 'model.update' : 'model.create', 'model', `${input.provider_key}/${input.model_key}`, { trace, metadata: { status, type } });++ return await getModel(input.provider_key, input.model_key);}// ── Versioned records (routes, templates, workflows) ───-- function nextVersion(table, key) {-- const r = db.prepare(`SELECT MAX(version) AS v FROM ${table} WHERE key = ?`).get(key);++ async function nextVersion(table, key) {++ const r = await db.prepare(`SELECT MAX(version) AS v FROM ${table} WHERE key = ?`).get(key);return (r && r.v ? r.v : 0) + 1;}-- function latestAny(table, key) {-- return db.prepare(`SELECT * FROM ${table} WHERE key = ? ORDER BY version DESC LIMIT 1`).get(key);++ async function latestAny(table, key) {++ return await db.prepare(`SELECT * FROM ${table} WHERE key = ? ORDER BY version DESC LIMIT 1`).get(key);}-- function latestActive(table, key) {-- return db.prepare(`SELECT * FROM ${table} WHERE key = ? AND status = 'active' ORDER BY version DESC LIMIT 1`).get(key);++ async function latestActive(table, key) {++ return await db.prepare(`SELECT * FROM ${table} WHERE key = ? AND status = 'active' ORDER BY version DESC LIMIT 1`).get(key);}// Routes@@@ -160,9 +160,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e};}const routeContent = (r) => sha256({ p: r.primary, f: r.fallbacks, o: r.options, m: r.max_output_tokens, rf: r.response_format, t: r.timeout_ms, a: r.alias_of, s: r.status });-- function createRouteVersion(key, input, { actor = 'system', trace = null } = {}) {++ async function createRouteVersion(key, input, { actor = 'system', trace = null } = {}) {if (!KEY_RE.test(key || '')) throw new AiError(422, 'ai.invalid', 'route key must be lowercase dotted');-- const prev = decodeRoute(latestAny('routes', key));++ const prev = decodeRoute(await latestAny('routes', key));const r = {primary: input.primary || (prev && prev.primary),fallbacks: input.fallbacks !== undefined ? input.fallbacks : (prev ? prev.fallbacks : []),@@@ -176,28 +176,28 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.eif (!r.alias_of) {if (!r.primary || !r.primary.provider) throw new AiError(422, 'ai.invalid', 'route needs primary.provider');for (const c of [r.primary, ...(r.fallbacks || [])]) {-- if (!c || !getProvider(c.provider)) throw new AiError(422, 'ai.invalid', `route references unknown provider ${c && c.provider}`);++ if (!c || !await getProvider(c.provider)) throw new AiError(422, 'ai.invalid', `route references unknown provider ${c && c.provider}`);}-- } else if (!latestAny('routes', r.alias_of)) throw new AiError(422, 'ai.invalid', `alias target ${r.alias_of} does not exist`);++ } else if (!await latestAny('routes', r.alias_of)) throw new AiError(422, 'ai.invalid', `alias target ${r.alias_of} does not exist`);if (!Array.isArray(r.fallbacks)) throw new AiError(422, 'ai.invalid', 'fallbacks must be an array');if (!['text', 'json'].includes(r.response_format)) throw new AiError(422, 'ai.invalid', 'response_format must be text or json');if (!['active', 'disabled'].includes(r.status)) throw new AiError(422, 'ai.invalid', 'route status must be active or disabled');-- const version = nextVersion('routes', key);-- db.prepare(`INSERT INTO routes (key, version, status, primary_provider, primary_model, fallbacks, options, max_output_tokens, response_format, timeout_ms, alias_of, created_by, created_at)++ const version = await nextVersion('routes', key);++ await db.prepare(`INSERT INTO routes (key, version, status, primary_provider, primary_model, fallbacks, options, max_output_tokens, response_format, timeout_ms, alias_of, created_by, created_at)VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(key, version, r.status, r.alias_of ? (r.primary && r.primary.provider) || 'stub' : r.primary.provider, r.primary ? r.primary.model || null : null,JSON.stringify(r.fallbacks || []), JSON.stringify(r.options || {}), r.max_output_tokens, r.response_format, r.timeout_ms, r.alias_of, actor, now());-- audit(actor, 'route.version', 'route', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: r.status, alias_of: r.alias_of } });-- return getRoute(key, version);++ await audit(actor, 'route.version', 'route', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: r.status, alias_of: r.alias_of } });++ return await getRoute(key, version);}-- function getRoute(key, version) {-- return decodeRoute(version ? db.prepare('SELECT * FROM routes WHERE key = ? AND version = ?').get(key, version) : latestAny('routes', key));++ async function getRoute(key, version) {++ return decodeRoute(version ? await db.prepare('SELECT * FROM routes WHERE key = ? AND version = ?').get(key, version) : await latestAny('routes', key));}/** The route a run uses: newest version, which must be active; aliases are followed (max 3 hops). */-- function resolveRoute(key) {++ async function resolveRoute(key) {let k = key;for (let hop = 0; hop < 4; hop++) {-- const r = decodeRoute(latestAny('routes', k));++ const r = decodeRoute(await latestAny('routes', k));if (!r) return null;if (r.status !== 'active') return { ...r, disabled: true };if (!r.alias_of) return r;@@@ -205,9 +205,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e}return null;}-- function listRoutes({ history = false } = {}) {-- const rows = history ? db.prepare('SELECT * FROM routes ORDER BY key, version').all()-- : db.prepare('SELECT r.* FROM routes r JOIN (SELECT key, MAX(version) v FROM routes GROUP BY key) m ON m.key = r.key AND m.v = r.version ORDER BY r.key').all();++ async function listRoutes({ history = false } = {}) {++ const rows = history ? await db.prepare('SELECT * FROM routes ORDER BY key, version').all()++ : await db.prepare('SELECT r.* FROM routes r JOIN (SELECT key, MAX(version) v FROM routes GROUP BY key) m ON m.key = r.key AND m.v = r.version ORDER BY r.key').all();return rows.map(decodeRoute);}@@@ -223,9 +223,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e};}const templateContent = (t) => sha256({ n: t.name, d: t.description, i: t.input_schema, o: t.output_schema, s: t.system_prompt, u: t.user_prompt, r: t.default_route, v: t.visibility, m: t.metadata });-- function createTemplateVersion(key, input, { actor = 'system', trace = null } = {}) {++ async function createTemplateVersion(key, input, { actor = 'system', trace = null } = {}) {if (!KEY_RE.test(key || '')) throw new AiError(422, 'ai.invalid', 'template key must be lowercase dotted');-- const prev = decodeTemplate(latestAny('templates', key));++ const prev = decodeTemplate(await latestAny('templates', key));const t = {name: input.name || (prev && prev.name) || key,description: input.description !== undefined ? input.description : (prev ? prev.description : null),@@@ -243,21 +243,21 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.eschemas.assertSchema(t.output_schema, 'output_schema');if (!LIFECYCLE.includes(t.status)) throw new AiError(422, 'ai.invalid', `template status must be one of ${LIFECYCLE.join(', ')}`);if (!['public', 'first-party', 'internal'].includes(t.visibility)) throw new AiError(422, 'ai.invalid', 'visibility must be public, first-party or internal');-- const version = nextVersion('templates', key);-- db.prepare(`INSERT INTO templates (key, version, name, description, input_schema, output_schema, system_prompt, user_prompt, default_route, owner, visibility, status, metadata, created_by, created_at)++ const version = await nextVersion('templates', key);++ await db.prepare(`INSERT INTO templates (key, version, name, description, input_schema, output_schema, system_prompt, user_prompt, default_route, owner, visibility, status, metadata, created_by, created_at)VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(key, version, t.name, t.description, JSON.stringify(t.input_schema), JSON.stringify(t.output_schema), t.system_prompt, t.user_prompt,t.default_route, t.owner, t.visibility, t.status, JSON.stringify(t.metadata), actor, now());-- audit(actor, 'template.version', 'template', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: t.status, content_hash: templateContent(t) } });-- return getTemplate(key, version);++ await audit(actor, 'template.version', 'template', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: t.status, content_hash: templateContent(t) } });++ return await getTemplate(key, version);}-- function getTemplate(key, version) {-- return decodeTemplate(version ? db.prepare('SELECT * FROM templates WHERE key = ? AND version = ?').get(key, version) : latestAny('templates', key));++ async function getTemplate(key, version) {++ return decodeTemplate(version ? await db.prepare('SELECT * FROM templates WHERE key = ? AND version = ?').get(key, version) : await latestAny('templates', key));}-- function activeTemplate(key) { return decodeTemplate(latestActive('templates', key)); }-- function listTemplates({ history = false } = {}) {-- const rows = history ? db.prepare('SELECT * FROM templates ORDER BY key, version').all()-- : db.prepare('SELECT t.* FROM templates t JOIN (SELECT key, MAX(version) v FROM templates GROUP BY key) m ON m.key = t.key AND m.v = t.version ORDER BY t.key').all();++ async function activeTemplate(key) { return decodeTemplate(await latestActive('templates', key)); }++ async function listTemplates({ history = false } = {}) {++ const rows = history ? await db.prepare('SELECT * FROM templates ORDER BY key, version').all()++ : await db.prepare('SELECT t.* FROM templates t JOIN (SELECT key, MAX(version) v FROM templates GROUP BY key) m ON m.key = t.key AND m.v = t.version ORDER BY t.key').all();return rows.map(decodeTemplate);}@@@ -272,9 +272,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e};}const workflowContent = (w) => sha256({ n: w.name, d: w.description, ns: w.namespace, i: w.input_schema, o: w.output_schema, s: w.steps, r: w.default_route, c: w.cache_mode, ttl: w.cache_ttl_sec, m: w.metadata });-- function createWorkflowVersion(key, input, { actor = 'system', trace = null } = {}) {++ async function createWorkflowVersion(key, input, { actor = 'system', trace = null } = {}) {if (!KEY_RE.test(key || '') || key.split('.').length < 2) throw new AiError(422, 'ai.invalid', 'workflow key must be <namespace>.<name>');-- const prev = decodeWorkflow(latestAny('workflows', key));++ const prev = decodeWorkflow(await latestAny('workflows', key));const w = {name: input.name || (prev && prev.name) || key,description: input.description !== undefined ? input.description : (prev ? prev.description : null),@@@ -293,39 +293,39 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.eif (!Array.isArray(w.steps) || !w.steps.length) throw new AiError(422, 'ai.invalid', 'workflow needs at least one step');for (const s of w.steps) {if (!s || !['llm', 'passthrough', 'transcribe', 'embed', 'media_analysis'].includes(s.kind)) throw new AiError(422, 'ai.invalid', 'step kind must be llm, passthrough, transcribe, embed or media_analysis');-- if ((s.kind === 'llm' || (s.kind === 'media_analysis' && s.template)) && !(s.template && latestAny('templates', s.template))) throw new AiError(422, 'ai.invalid', `step references unknown template ${s.template}`);++ if ((s.kind === 'llm' || (s.kind === 'media_analysis' && s.template)) && !(s.template && await latestAny('templates', s.template))) throw new AiError(422, 'ai.invalid', `step references unknown template ${s.template}`);}if (!['none', 'private', 'service'].includes(w.cache_mode)) throw new AiError(422, 'ai.invalid', 'cache_mode must be none, private or service');if (!LIFECYCLE.includes(w.status)) throw new AiError(422, 'ai.invalid', `workflow status must be one of ${LIFECYCLE.join(', ')}`);-- const version = nextVersion('workflows', key);-- db.prepare(`INSERT INTO workflows (key, version, name, description, namespace, input_schema, output_schema, steps, default_route, cache_mode, cache_ttl_sec, status, metadata, created_by, created_at)++ const version = await nextVersion('workflows', key);++ await db.prepare(`INSERT INTO workflows (key, version, name, description, namespace, input_schema, output_schema, steps, default_route, cache_mode, cache_ttl_sec, status, metadata, created_by, created_at)VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(key, version, w.name, w.description, w.namespace, JSON.stringify(w.input_schema), JSON.stringify(w.output_schema), JSON.stringify(w.steps),w.default_route, w.cache_mode, w.cache_ttl_sec, w.status, JSON.stringify(w.metadata), actor, now());-- audit(actor, 'workflow.version', 'workflow', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: w.status, content_hash: workflowContent(w) } });-- return getWorkflow(key, version);++ await audit(actor, 'workflow.version', 'workflow', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: w.status, content_hash: workflowContent(w) } });++ return await getWorkflow(key, version);}-- function getWorkflow(key, version) {-- return decodeWorkflow(version ? db.prepare('SELECT * FROM workflows WHERE key = ? AND version = ?').get(key, version) : latestAny('workflows', key));++ async function getWorkflow(key, version) {++ return decodeWorkflow(version ? await db.prepare('SELECT * FROM workflows WHERE key = ? AND version = ?').get(key, version) : await latestAny('workflows', key));}-- function activeWorkflow(key) { return decodeWorkflow(latestActive('workflows', key)); }-- function listWorkflows({ history = false, namespace } = {}) {-- let rows = history ? db.prepare('SELECT * FROM workflows ORDER BY key, version').all()-- : db.prepare('SELECT w.* FROM workflows w JOIN (SELECT key, MAX(version) v FROM workflows GROUP BY key) m ON m.key = w.key AND m.v = w.version ORDER BY w.key').all();++ async function activeWorkflow(key) { return decodeWorkflow(await latestActive('workflows', key)); }++ async function listWorkflows({ history = false, namespace } = {}) {++ let rows = history ? await db.prepare('SELECT * FROM workflows ORDER BY key, version').all()++ : await db.prepare('SELECT w.* FROM workflows w JOIN (SELECT key, MAX(version) v FROM workflows GROUP BY key) m ON m.key = w.key AND m.v = w.version ORDER BY w.key').all();if (namespace) rows = rows.filter(r => r.namespace === namespace);return rows.map(decodeWorkflow);}/** Lifecycle change on one version of a template/workflow/route (status only; audited). */-- function setStatus(kind, key, version, status, { actor = 'system', trace = null } = {}) {++ async function setStatus(kind, key, version, status, { actor = 'system', trace = null } = {}) {const table = { template: 'templates', workflow: 'workflows', route: 'routes' }[kind];if (!table) throw new AiError(400, 'ai.invalid', 'unknown kind');const allowed = kind === 'route' ? ['active', 'disabled'] : LIFECYCLE;if (!allowed.includes(status)) throw new AiError(422, 'ai.invalid', `status must be one of ${allowed.join(', ')}`);-- const row = db.prepare(`SELECT status FROM ${table} WHERE key = ? AND version = ?`).get(key, version);++ const row = await db.prepare(`SELECT status FROM ${table} WHERE key = ? AND version = ?`).get(key, version);if (!row) throw new AiError(404, 'ai.not_found', `no ${kind} ${key} v${version}`);-- db.prepare(`UPDATE ${table} SET status = ? WHERE key = ? AND version = ?`).run(status, key, version);-- audit(actor, `${kind}.status`, kind, key, { trace, metadata: { version, from: row.status, to: status } });++ await db.prepare(`UPDATE ${table} SET status = ? WHERE key = ? AND version = ?`).run(status, key, version);++ await audit(actor, `${kind}.status`, kind, key, { trace, metadata: { version, from: row.status, to: status } });return { key, version, status };}@@@ -333,13 +333,13 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e* Seeding: insert a code-defined record when it is missing, or add a new version when the code* changed it and nobody edited it through the admin API since (admin edits always win).*/-- function seedVersioned(kind, key, def) {++ async function seedVersioned(kind, key, def) {const [table, create, decode, content] = {route: ['routes', createRouteVersion, decodeRoute, routeContent],template: ['templates', createTemplateVersion, decodeTemplate, templateContent],workflow: ['workflows', createWorkflowVersion, decodeWorkflow, workflowContent],}[kind];-- const latest = decode(latestAny(table, key));++ const latest = decode(await latestAny(table, key));if (!latest) return create(key, def, { actor: 'seed' });if (latest.created_by !== 'seed') return latest;const merged = { ...latest, ...def };@@@ -93,22 +93,22 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { nowcreated_at: r.created_at, started_at: r.started_at, finished_at: r.finished_at,};}-- const getRow = (id) => db.prepare('SELECT * FROM runs WHERE id = ?').get(id);-- const get = (id) => decode(getRow(id));++ const getRow = async (id) => await db.prepare('SELECT * FROM runs WHERE id = ?').get(id);++ const get = async (id) => decode(await getRow(id));-- function citations(runId) {-- return db.prepare('SELECT * FROM citations WHERE run_id = ? ORDER BY ordinal, id').all(runId)++ async function citations(runId) {++ return (await db.prepare('SELECT * FROM citations WHERE run_id = ? ORDER BY ordinal, id').all(runId)).map(c => ({ ...c, trust: parseJson(c.trust, {}), provenance: parseJson(c.provenance, {}) }));}-- function addCitations(runId, list, attachedBy) {++ async function addCitations(runId, list, attachedBy) {const ins = db.prepare(`INSERT INTO citations (run_id, ordinal, source_type, source_id, url, title, author, published_at, retrieved_at, snippet, content_hash, trust, provenance, attached_by, created_at)VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`);-- const base = db.prepare('SELECT COUNT(*) AS n FROM citations WHERE run_id = ?').get(runId).n;-- db.transaction(() => {-- list.forEach((c, i) => ins.run(runId, c.ordinal != null ? c.ordinal : base + i, c.source_type, c.source_id || null, c.url || null, c.title || null, c.author || null,-- c.published_at || null, c.retrieved_at || null, c.snippet || null, c.content_hash || null, JSON.stringify(c.trust || {}), JSON.stringify(c.provenance || {}), attachedBy, iso(clock.now())));-- db.prepare('UPDATE runs SET citations_count = (SELECT COUNT(*) FROM citations WHERE run_id = ?) WHERE id = ?').run(runId, runId);-- })();++ const base = (await db.prepare('SELECT COUNT(*) AS n FROM citations WHERE run_id = ?').get(runId)).n;++ await db.tx(async () => {++ for (const [i, c] of list.entries()) await ins.run(runId, c.ordinal != null ? c.ordinal : base + i, c.source_type, c.source_id || null, c.url || null, c.title || null, c.author || null,++ c.published_at || null, c.retrieved_at || null, c.snippet || null, c.content_hash || null, JSON.stringify(c.trust || {}), JSON.stringify(c.provenance || {}), attachedBy, iso(clock.now()));++ await db.prepare('UPDATE runs SET citations_count = (SELECT COUNT(*) FROM citations WHERE run_id = ?) WHERE id = ?').run(runId, runId);++ });}const insReq = db.prepare(`INSERT INTO requests (run_id, seq, operation, provider_key, model_key, route_key, route_version, status, skip_reason, fallback, prompt_hash, input_hash, output_hash,@@@ -138,10 +138,10 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now* principal: { sub, subject: {type, id} }* Returns { run, created, promise? }.*/-- function create(body, principal, { trace = null, requestId = null, retryOf = null } = {}) {++ async function create(body, principal, { trace = null, requestId = null, retryOf = null } = {}) {if (!body || typeof body !== 'object') throw new AiError(400, 'input.invalid', 'JSON body required');const wfKey = String(body.workflow || '');-- const wf = body.version ? registry.getWorkflow(wfKey, Number(body.version)) : registry.activeWorkflow(wfKey);++ const wf = body.version ? await registry.getWorkflow(wfKey, Number(body.version)) : await registry.activeWorkflow(wfKey);if (!wf) throw new AiError(404, 'workflow.not_found', `no active workflow ${wfKey}${body.version ? ` v${body.version}` : ''}`);if (body.version && !['active', 'deprecated'].includes(wf.status)) throw new AiError(409, 'workflow.inactive', `${wfKey} v${wf.version} is ${wf.status}`);validateRefs(body);@@@ -156,7 +156,7 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { nowif (idem !== null && !/^[A-Za-z0-9._:-]{8,128}$/.test(idem)) throw new AiError(422, 'input.invalid', 'idempotency_key must be 8-128 of [A-Za-z0-9._:-]');if (idem) {-- const prev = db.prepare('SELECT * FROM runs WHERE requester_type = ? AND requester_id = ? AND idempotency_key = ?').get(requester.type, requester.id, idem);++ const prev = await db.prepare('SELECT * FROM runs WHERE requester_type = ? AND requester_id = ? AND idempotency_key = ?').get(requester.type, requester.id, idem);if (prev) {if (prev.input_hash !== inputHash || prev.workflow_key !== wf.key) throw new AiError(409, 'idempotency.conflict', 'this idempotency_key was used with a different workflow or input');return { run: decode(prev), created: false, promise: inflight.get(prev.id) ? inflight.get(prev.id).promise : null };@@@ -164,16 +164,16 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now}const llmStep = wf.steps.find(s => s.kind === 'llm');-- const tpl = llmStep ? registry.activeTemplate(llmStep.template) : null;++ const tpl = llmStep ? await registry.activeTemplate(llmStep.template) : null;const routeKey = (wf.steps[0] && wf.steps[0].route) || (tpl && tpl.default_route) || wf.default_route || null;-- const route = routeKey && !String(routeKey).includes('{') ? registry.resolveRoute(routeKey) : null;++ const route = routeKey && !String(routeKey).includes('{') ? await registry.resolveRoute(routeKey) : null;const options = body.options && typeof body.options === 'object' ? body.options : {};// A person's own key (WS-O task 2): only the service that stored it, within its daily budget; never cached.const credentialSubject = body.credential && typeof body.credential === 'object' ? String(body.credential.subject || '') : null;if (credentialSubject !== null) {if (!credentials) throw new AiError(503, 'credentials.unavailable', 'person credentials are not configured on this service');if (requester.type !== 'service') throw new AiError(403, 'capability.denied', 'only the service that stored a credential runs with it');-- credentials.admit(requester.id, credentialSubject);++ await credentials.admit(requester.id, credentialSubject);}const ctx = {requesterType: requester.type, requesterId: requester.id,@@@ -200,22 +200,22 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { nowlet cacheKey = null;if (scope) {cacheKey = cache.keyFor({ scope, workflowKey: wf.key, workflowVersion: wf.version, templateVersion: base.template_version, routeKey: base.route_key, routeVersion: base.route_version, inputHash });-- const hit = cache.get(cacheKey, scope);++ const hit = await cache.get(cacheKey, scope);if (hit) {-- db.transaction(() => {-- db.prepare(`INSERT INTO runs (id, workflow_key, workflow_version, template_key, template_version, route_key, route_version, status, requester_type, requester_id, on_behalf_of, attribution,++ await db.tx(async () => {++ await db.prepare(`INSERT INTO runs (id, workflow_key, workflow_version, template_key, template_version, route_key, route_version, status, requester_type, requester_id, on_behalf_of, attribution,source_service, target, target_key, input, input_hash, output, model_key, cache_key, cached_from, retry_of, idempotency_key, trace_id, request_id, options, created_at, started_at, finished_at, citations_count)VALUES (@id, @workflow_key, @workflow_version, @template_key, @template_version, @route_key, @route_version, 'cached', @requester_type, @requester_id, @on_behalf_of, @attribution,@source_service, @target, @target_key, @input, @input_hash, @output, @model_key, @cache_key, @cached_from, @retry_of, @idempotency_key, @trace_id, @request_id, @options, @created_at, @created_at, @created_at, 0)`).run({ ...base, output: JSON.stringify(hit.output), model_key: hit.model_key, cache_key: cacheKey, cached_from: hit.run_id });-- const src = citations(hit.run_id);-- if (src.length) addCitations(id, src.map(c => ({ ...c, provenance: { ...c.provenance, via_cache: hit.run_id } })), 'cache');++ const src = await citations(hit.run_id);++ if (src.length) await addCitations(id, src.map(c => ({ ...c, provenance: { ...c.provenance, via_cache: hit.run_id } })), 'cache');// The reused output carries the grounding it was produced with.-- db.prepare('UPDATE runs SET grounding = (SELECT grounding FROM runs WHERE id = ?) WHERE id = ?').run(hit.run_id, id);-- events.runChanged(getRow(id));-- })();-- registry.audit(principal.sub, 'run.create', 'run', id, { trace, metadata: { workflow: wf.key, version: wf.version, status: 'cached', cached_from: hit.run_id } });-- return { run: get(id), created: true, promise: null };++ await db.prepare('UPDATE runs SET grounding = (SELECT grounding FROM runs WHERE id = ?) WHERE id = ?').run(hit.run_id, id);++ events.runChanged(await getRow(id));++ });++ await registry.audit(principal.sub, 'run.create', 'run', id, { trace, metadata: { workflow: wf.key, version: wf.version, status: 'cached', cached_from: hit.run_id } });++ return { run: await get(id), created: true, promise: null };}}@@@ -223,26 +223,26 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { nowconst caller = `${requester.type}:${requester.id}`;admit(caller);// A run with a person's own key spends their money, not the shared quotas (its own budget was checked above).-- const reserved = credentialSubject !== null ? null : quotas.reserve(ctx);++ const reserved = credentialSubject !== null ? null : await quotas.reserve(ctx);-- db.transaction(() => {-- db.prepare(`INSERT INTO runs (id, workflow_key, workflow_version, template_key, template_version, route_key, route_version, status, requester_type, requester_id, on_behalf_of, attribution,++ await db.tx(async () => {++ await db.prepare(`INSERT INTO runs (id, workflow_key, workflow_version, template_key, template_version, route_key, route_version, status, requester_type, requester_id, on_behalf_of, attribution,source_service, target, target_key, input, input_hash, cache_key, retry_of, idempotency_key, trace_id, request_id, options, created_at)VALUES (@id, @workflow_key, @workflow_version, @template_key, @template_version, @route_key, @route_version, 'queued', @requester_type, @requester_id, @on_behalf_of, @attribution,@source_service, @target, @target_key, @input, @input_hash, @cache_key, @retry_of, @idempotency_key, @trace_id, @request_id, @options, @created_at)`).run({ ...base, cache_key: cacheKey });-- events.runChanged(getRow(id));-- })();-- registry.audit(principal.sub, retryOf ? 'run.retry' : 'run.create', 'run', id, { trace, metadata: { workflow: wf.key, version: wf.version, target: ctx.targetKey, retry_of: retryOf } });++ events.runChanged(await getRow(id));++ });++ await registry.audit(principal.sub, retryOf ? 'run.retry' : 'run.create', 'run', id, { trace, metadata: { workflow: wf.key, version: wf.version, target: ctx.targetKey, retry_of: retryOf } });const controller = new AbortController();let release;const gate = new Promise((r) => { release = r; });-- const promise = gate.then(() => execute(id, { controller, reserved, scope, ctx, input })).catch((err) => { log.error(`[runs] ${id}: ${err.stack || err}`); });++ const promise = gate.then(async () => await execute(id, { controller, reserved, scope, ctx, input })).catch((err) => { log.error(`[runs] ${id}: ${err.stack || err}`); });inflight.set(id, { controller, promise, release, caller });queue.push(id);pump();-- return { run: get(id), created: true, promise };++ return { run: await get(id), created: true, promise };}/** Refuse a run that would wait in a full queue, globally or for this caller. */@@@ -267,16 +267,16 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now}async function execute(id, { controller, reserved, scope, ctx, input }) {-- const row = getRow(id);++ const row = await getRow(id);if (!row || row.status !== 'queued') { inflight.delete(id); return; }-- db.prepare("UPDATE runs SET status = 'running', started_at = ? WHERE id = ? AND status = 'queued'").run(iso(clock.now()), id);-- const wf = registry.getWorkflow(row.workflow_key, row.workflow_version);++ await db.prepare("UPDATE runs SET status = 'running', started_at = ? WHERE id = ? AND status = 'queued'").run(iso(clock.now()), id);++ const wf = await registry.getWorkflow(row.workflow_key, row.workflow_version);const opts = parseJson(row.options, {});let seq = 0;let attempts = 0;-- const logRequest = (e) => {++ const logRequest = async (e) => {if (e.status !== 'skipped') attempts++;-- insReq.run({++ await insReq.run({run_id: id, seq: seq++, operation: e.operation, provider_key: e.provider_key, model_key: e.model_key || null, route_key: e.route_key || null, route_version: e.route_version || null,status: e.status, skip_reason: e.skip_reason || null, fallback: e.fallback || 0, prompt_hash: e.prompt_hash || null, input_hash: row.input_hash, output_hash: e.output_hash || null,tokens_in: e.tokens_in || 0, tokens_out: e.tokens_out || 0, tokens_cached: e.tokens_cached || 0, tokens_estimated: e.tokens_estimated || 0, cost_usd: e.cost_usd || 0,@@@ -290,29 +290,29 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now// perspective shape the prompt; history: false keeps neither their input nor a cache entry.const preferences = userModules && row.on_behalf_of ? await userModules.preferencesFor(row.on_behalf_of) : {};if (preferences.history === false) {-- db.prepare("UPDATE runs SET input = 'null', cache_key = NULL, options = json_set(options, '$.input_retained', json('false'), '$.history', json('false')) WHERE id = ?").run(id);++ await db.prepare("UPDATE runs SET input = 'null', cache_key = NULL, options = (COALESCE(options, '{}')::jsonb || jsonb_build_object('input_retained', false, 'history', false))::text WHERE id = ?").run(id);}-- const credential = opts.credential_subject ? credentials.forRun(row.requester_id, opts.credential_subject) : null;++ const credential = opts.credential_subject ? await credentials.forRun(row.requester_id, opts.credential_subject) : null;const r = await engine.execute(wf, row, input, { signal: controller.signal, logRequest, debugRaw: Boolean(config.debugRawLog && opts.debug), inputHash: row.input_hash, preferences,...(credential ? { credential, role: input && typeof input.role === 'string' ? input.role : null } : {}) });if (controller.signal.aborted) throw new AiError(409, 'run.cancelled', 'cancelled');// The status, its citations and the ai.run.succeeded event commit together.-- const done = db.transaction(() => {-- const d = db.prepare(`UPDATE runs SET status = 'succeeded', output = ?, grounding = ?, synthetic = ?, provider_key = ?, model_key = ?, fallback_used = ?, attempts = ?, tokens_in = ?, tokens_out = ?, cost_usd = ?,++ const done = await db.tx(async () => {++ const d = await db.prepare(`UPDATE runs SET status = 'succeeded', output = ?, grounding = ?, synthetic = ?, provider_key = ?, model_key = ?, fallback_used = ?, attempts = ?, tokens_in = ?, tokens_out = ?, cost_usd = ?,route_key = COALESCE(?, route_key), route_version = COALESCE(?, route_version), finished_at = ? WHERE id = ? AND status = 'running'`).run(JSON.stringify(r.output), JSON.stringify(r.grounding || null), r.synthetic ? 1 : 0, r.provider, r.model, r.fallbackUsed ? 1 : 0, attempts, r.usage.input, r.usage.output, r.cost,r.route ? r.route.key : null, r.route ? r.route.version : null, iso(clock.now()), id);if (!d.changes) return d;-- if (r.citations.length) addCitations(id, r.citations, 'workflow');-- events.runChanged(getRow(id));++ if (r.citations.length) await addCitations(id, r.citations, 'workflow');++ events.runChanged(await getRow(id));return d;-- })();++ });if (!done.changes) return; // cancelled while finishing-- if (r.provider && reserved) quotas.account(ctx, reserved, { provider: r.provider, model: r.model, tokensIn: r.usage.input, tokensOut: r.usage.output, tokensCached: r.usage.cached, cost: r.cost });-- if (r.fallbackUsed) registry.audit('system', 'run.fallback', 'run', id, { trace: row.trace_id, metadata: { workflow: row.workflow_key, provider: r.provider, route: r.route && r.route.key } });++ if (r.provider && reserved) await quotas.account(ctx, reserved, { provider: r.provider, model: r.model, tokensIn: r.usage.input, tokensOut: r.usage.output, tokensCached: r.usage.cached, cost: r.cost });++ if (r.fallbackUsed) await registry.audit('system', 'run.fallback', 'run', id, { trace: row.trace_id, metadata: { workflow: row.workflow_key, provider: r.provider, route: r.route && r.route.key } });// Output shaped by someone's preferences, or kept for nobody (history off), is never cached for reuse.if (scope && row.cache_key && !r.synthetic && wf.cache_mode !== 'none' && preferences.history !== false && !preferenceLines(preferences)) {-- cache.put({ key: row.cache_key, scope, privacy: wf.cache_mode, workflowKey: wf.key, workflowVersion: wf.version, templateVersion: row.template_version, routeKey: row.route_key, routeVersion: row.route_version, model: r.model, inputHash: row.input_hash, output: r.output, runId: id, ttlSec: wf.cache_ttl_sec || config.runs.defaultCacheTtlSec });++ await cache.put({ key: row.cache_key, scope, privacy: wf.cache_mode, workflowKey: wf.key, workflowVersion: wf.version, templateVersion: row.template_version, routeKey: row.route_key, routeVersion: row.route_version, model: r.model, inputHash: row.input_hash, output: r.output, runId: id, ttlSec: wf.cache_ttl_sec || config.runs.defaultCacheTtlSec });}} catch (err) {const interrupted = shuttingDown && controller.signal.aborted;@@@ -321,13 +321,13 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { nowconst detail = interrupted ? 'the service shut down while this run was in progress' : cancelled ? 'cancelled' : (err instanceof AiError ? err.detail : 'internal error');if (!(err instanceof AiError) && !cancelled) log.error(`[runs] ${id} crashed: ${err && err.stack || err}`);const extra = err instanceof AiError && err.extra ? JSON.stringify(err.extra).slice(0, 2000) : null;-- db.transaction(() => {-- const f = db.prepare(`UPDATE runs SET status = ?, error_code = ?, error_detail = ?, attempts = ?, finished_at = ? WHERE id = ? AND status IN ('queued', 'running')`)++ await db.tx(async () => {++ const f = await db.prepare(`UPDATE runs SET status = ?, error_code = ?, error_detail = ?, attempts = ?, finished_at = ? WHERE id = ? AND status IN ('queued', 'running')`).run(cancelled ? 'cancelled' : 'failed', code, extra ? `${detail} ${extra}` : detail, attempts, iso(clock.now()), id);-- if (f.changes) events.runChanged(getRow(id));-- })();-- const spent = db.prepare("SELECT COALESCE(SUM(tokens_in),0) ti, COALESCE(SUM(tokens_out),0) tout, COALESCE(SUM(cost_usd),0) c FROM requests WHERE run_id = ? AND status = 'ok'").get(id);-- if (reserved && spent && (spent.ti || spent.tout)) quotas.account(ctx, reserved, { provider: 'mixed', model: '', tokensIn: spent.ti, tokensOut: spent.tout, cost: spent.c });++ if (f.changes) events.runChanged(await getRow(id));++ });++ const spent = await db.prepare("SELECT COALESCE(SUM(tokens_in),0) ti, COALESCE(SUM(tokens_out),0) tout, COALESCE(SUM(cost_usd),0) c FROM requests WHERE run_id = ? AND status = 'ok'").get(id);++ if (reserved && spent && (spent.ti || spent.tout)) await quotas.account(ctx, reserved, { provider: 'mixed', model: '', tokensIn: spent.ti, tokensOut: spent.tout, cost: spent.c });} finally {inflight.delete(id);}@@@ -335,11 +335,11 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now/** Wait for a run to reach a terminal state, up to ms. */async function wait(created, ms) {-- if (!created.promise || ms <= 0) return get(created.run.id);++ if (!created.promise || ms <= 0) return await get(created.run.id);let timer;await Promise.race([created.promise, new Promise((r) => { timer = setTimeout(r, Math.min(ms, config.runs.maxWaitMs)); timer.unref?.(); })]);clearTimeout(timer);-- return get(created.run.id);++ return await get(created.run.id);}function assertOwner(row, principal, adminCap, principalHas) {@@@ -349,36 +349,36 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { nowthrow new AiError(404, 'run.not_found', 'no such run'); // do not reveal other callers' runs}-- function cancel(id, principal, { trace = null, principalHas } = {}) {-- const row = getRow(id);++ async function cancel(id, principal, { trace = null, principalHas } = {}) {++ const row = await getRow(id);if (!row) throw new AiError(404, 'run.not_found', 'no such run');assertOwner(row, principal, 'ai.workflow.manage', principalHas);if (TERMINAL.has(row.status)) throw new AiError(409, 'run.terminal', `run is already ${row.status}`);const h = inflight.get(id);-- const r = db.prepare("UPDATE runs SET status = 'cancelled', error_code = 'run.cancelled', error_detail = 'cancelled by caller', finished_at = ? WHERE id = ? AND status IN ('queued', 'running')").run(iso(clock.now()), id);++ const r = await db.prepare("UPDATE runs SET status = 'cancelled', error_code = 'run.cancelled', error_detail = 'cancelled by caller', finished_at = ? WHERE id = ? AND status IN ('queued', 'running')").run(iso(clock.now()), id);if (h) {h.controller.abort();const i = queue.indexOf(id);if (i >= 0) { queue.splice(i, 1); h.release(); }}-- registry.audit(principal.sub, 'run.cancel', 'run', id, { trace, metadata: { from: row.status, changed: r.changes } });-- return get(id);++ await registry.audit(principal.sub, 'run.cancel', 'run', id, { trace, metadata: { from: row.status, changed: r.changes } });++ return await get(id);}-- function retry(id, principal, { trace = null, requestId = null, principalHas } = {}) {-- const row = getRow(id);++ async function retry(id, principal, { trace = null, requestId = null, principalHas } = {}) {++ const row = await getRow(id);if (!row) throw new AiError(404, 'run.not_found', 'no such run');assertOwner(row, principal, 'ai.workflow.manage', principalHas);if (!['failed', 'cancelled'].includes(row.status)) throw new AiError(409, 'run.not_retryable', `only failed or cancelled runs can be retried (this one is ${row.status})`);if (parseJson(row.options, {}).input_retained === false) throw new AiError(409, 'run.input_not_retained', 'this run did not keep its input (inline media or a passthrough prompt); submit it again');-- return create({++ return await create({workflow: row.workflow_key, input: parseJson(row.input, {}), target: parseJson(row.target, null) || undefined,attribution: parseJson(row.attribution, null) || undefined, on_behalf_of: parseJson(row.on_behalf_of, null) || undefined,options: { ...parseJson(row.options, {}), cache: false },}, principal, { trace, requestId, retryOf: id });}-- function list(principal, { status, workflow, target, limit = 50, before, all = false } = {}) {++ async function list(principal, { status, workflow, target, limit = 50, before, all = false } = {}) {const where = [];const args = [];if (!all) { where.push('requester_type = ? AND requester_id = ?'); args.push(principal.subject.type, principal.subject.id); }@@@ -386,33 +386,33 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { nowif (workflow) { where.push('workflow_key = ?'); args.push(workflow); }if (target) { where.push('target_key = ?'); args.push(target); }if (before) { where.push('id < ?'); args.push(before); }-- return db.prepare(`SELECT * FROM runs ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(200, Number(limit) || 50)).map(decode);++ return (await db.prepare(`SELECT * FROM runs ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(200, Number(limit) || 50))).map(decode);}-- function requestsFor(runId) {-- return db.prepare('SELECT id, seq, operation, provider_key, model_key, route_key, route_version, status, skip_reason, fallback, prompt_hash, input_hash, output_hash, tokens_in, tokens_out, tokens_cached, tokens_estimated, cost_usd, latency_ms, error, created_at FROM requests WHERE run_id = ? ORDER BY seq').all(runId);++ async function requestsFor(runId) {++ return await db.prepare('SELECT id, seq, operation, provider_key, model_key, route_key, route_version, status, skip_reason, fallback, prompt_hash, input_hash, output_hash, tokens_in, tokens_out, tokens_cached, tokens_estimated, cost_usd, latency_ms, error, created_at FROM requests WHERE run_id = ? ORDER BY seq').all(runId);}/** Boot: runs interrupted by a restart are failed explicitly (retry is one call away). */-- function recoverInterrupted() {-- return db.transaction(() => {-- const open = db.prepare("SELECT id FROM runs WHERE status IN ('queued', 'running')").all();-- const r = db.prepare("UPDATE runs SET status = 'failed', error_code = 'run.interrupted', error_detail = 'the service restarted while this run was in progress', finished_at = ? WHERE status IN ('queued', 'running')").run(iso(clock.now()));-- for (const { id } of open) events.runChanged(getRow(id));++ async function recoverInterrupted() {++ return await db.tx(async () => {++ const open = await db.prepare("SELECT id FROM runs WHERE status IN ('queued', 'running')").all();++ const r = await db.prepare("UPDATE runs SET status = 'failed', error_code = 'run.interrupted', error_detail = 'the service restarted while this run was in progress', finished_at = ? WHERE status IN ('queued', 'running')").run(iso(clock.now()));++ for (const { id } of open) events.runChanged(await getRow(id));return r.changes;-- })();++ });}-- function prune(days = config.runs.retentionDays) {++ async function prune(days = config.runs.retentionDays) {const cutoff = iso(clock.now() - days * 86400000);-- const old = db.prepare('SELECT id FROM runs WHERE finished_at IS NOT NULL AND finished_at < ?').all(cutoff).map(r => r.id);-- db.transaction(() => {++ const old = (await db.prepare('SELECT id FROM runs WHERE finished_at IS NOT NULL AND finished_at < ?').all(cutoff)).map(r => r.id);++ await db.tx(async () => {for (const id of old) {-- db.prepare('DELETE FROM requests WHERE run_id = ?').run(id);-- db.prepare('DELETE FROM citations WHERE run_id = ?').run(id);-- db.prepare('DELETE FROM runs WHERE id = ?').run(id);++ await db.prepare('DELETE FROM requests WHERE run_id = ?').run(id);++ await db.prepare('DELETE FROM citations WHERE run_id = ?').run(id);++ await db.prepare('DELETE FROM runs WHERE id = ?').run(id);}-- })();++ });return old.length;}@@@ -67,16 +67,16 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalTh}function ensureSchema() {-- db.exec(`CREATE TABLE IF NOT EXISTS ai_module_pushes (subject_id TEXT PRIMARY KEY, hash TEXT NOT NULL, pushed_at TEXT NOT NULL)`);++ // ai_module_pushes is in migrations/0001_initial.sql (ADR-035).}/** ai.usage_summary for one person over the 30 days before now, or null when there were no runs. */-- function summarize(subject, now = clock.now()) {++ async function summarize(subject, now = clock.now()) {const since = new Date(now - 30 * DAY_MS).toISOString();-- const rows = db.prepare(`SELECT COALESCE(source_service, requester_id) AS svc, COUNT(*) AS n, SUM(tokens_in + tokens_out) AS tokens, MAX(created_at) AS last-- FROM runs WHERE json_extract(on_behalf_of, '$.type') = 'user' AND json_extract(on_behalf_of, '$.id') = ? AND created_at >= ?++ const rows = await db.prepare(`SELECT COALESCE(source_service, requester_id) AS svc, COUNT(*) AS n, SUM(tokens_in + tokens_out) AS tokens, MAX(created_at) AS last++ FROM runs WHERE (on_behalf_of::jsonb ->> 'type') = 'user' AND (on_behalf_of::jsonb ->> 'id') = ? AND created_at >= ?GROUP BY svc`).all(subject, since);-- const lastEver = db.prepare("SELECT MAX(created_at) AS last FROM runs WHERE json_extract(on_behalf_of, '$.type') = 'user' AND json_extract(on_behalf_of, '$.id') = ?").get(subject).last;++ const lastEver = (await db.prepare("SELECT MAX(created_at) AS last FROM runs WHERE (on_behalf_of::jsonb ->> 'type') = 'user' AND (on_behalf_of::jsonb ->> 'id') = ?").get(subject)).last;if (!lastEver) return null;const by = {};for (const r of rows.slice(0, 30)) if (/^[a-z][a-z0-9-]{1,39}$/.test(String(r.svc || ''))) by[r.svc] = r.n;@@@ -90,29 +90,29 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalThasync function push(subject, now = clock.now()) {if (!enabled || !USR.test(String(subject || ''))) return false;-- const data = summarize(subject, now);++ const data = await summarize(subject, now);if (!data) return false;const hash = crypto.createHash('sha256').update(JSON.stringify(data)).digest('hex').slice(0, 32);-- const last = db.prepare('SELECT hash FROM ai_module_pushes WHERE subject_id = ?').get(subject);++ const last = await db.prepare('SELECT hash FROM ai_module_pushes WHERE subject_id = ?').get(subject);if (last && last.hash === hash) { stats.unchanged++; return false; }try {const r = await call('PUT', 'ai.usage_summary', subject, { ...data, computed_at: new Date(now).toISOString() });if (r.status >= 300) throw new Error(`Network answered ${r.status} ${(r.body && r.body.code) || ''}`);} catch (err) { stats.failed++; stats.lastError = err.message; return false; }-- db.prepare(`INSERT INTO ai_module_pushes (subject_id, hash, pushed_at) VALUES (?, ?, ?)++ await db.prepare(`INSERT INTO ai_module_pushes (subject_id, hash, pushed_at) VALUES (?, ?, ?)ON CONFLICT(subject_id) DO UPDATE SET hash = excluded.hash, pushed_at = excluded.pushed_at`).run(subject, hash, new Date(now).toISOString());stats.written++;return true;}-- const people = (sql, ...args) => db.prepare(sql).all(...args).map((r) => r.s).filter((s) => USR.test(String(s || '')));++ const people = async (sql, ...args) => (await db.prepare(sql).all(...args)).map((r) => r.s).filter((s) => USR.test(String(s || '')));/** People with a run created since the previous scan. */async function scan(now = clock.now()) {if (!enabled) return 0;const from = lastScan || new Date(now - 10 * 60 * 1000).toISOString();const to = new Date(now).toISOString();-- const subjects = people("SELECT DISTINCT json_extract(on_behalf_of, '$.id') AS s FROM runs WHERE on_behalf_of IS NOT NULL AND created_at >= ? AND created_at < ?", from, to);++ const subjects = await people("SELECT DISTINCT (on_behalf_of::jsonb ->> 'id') AS s FROM runs WHERE on_behalf_of IS NOT NULL AND created_at >= ? AND created_at < ?", from, to);for (const s of subjects) await push(s, now);lastScan = to;return subjects.length;@@@ -122,7 +122,7 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalThasync function refresh(now = clock.now()) {if (!enabled) return 0;const since = new Date(now - 31 * DAY_MS).toISOString();-- const subjects = people(`SELECT DISTINCT json_extract(on_behalf_of, '$.id') AS s FROM runs WHERE on_behalf_of IS NOT NULL AND created_at >= ?++ const subjects = await people(`SELECT DISTINCT (on_behalf_of::jsonb ->> 'id') AS s FROM runs WHERE on_behalf_of IS NOT NULL AND created_at >= ?UNION SELECT subject_id AS s FROM ai_module_pushes`, since);for (const s of subjects) await push(s, now);return subjects.length;@@@ -132,7 +132,7 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalThif (!enabled || timers.length) return false;ensureSchema();const safe = (fn) => () => { fn().catch((err) => { stats.lastError = err.message; }); };-- timers = [setInterval(safe(() => scan()), 5 * 60 * 1000), setInterval(safe(() => refresh()), DAY_MS), setTimeout(safe(() => refresh()), 3 * 60 * 1000)];++ timers = [setInterval(safe(async () => await scan()), 5 * 60 * 1000), setInterval(safe(async () => await refresh()), DAY_MS), setTimeout(safe(async () => await refresh()), 3 * 60 * 1000)];for (const t of timers) t.unref?.();return true;}@@@ -101,8 +101,8 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spalet proxy = null;const reader = () => proxy || (proxy = require('../media-proxy').createMediaProxy({ fetcher, log }));const mediaAnalysis = require('./media-analysis').createMediaAnalysis({ registry, pool, fetcher, quotas, config, spawnImpl, log, reader });-- function routeFor(key) {-- const r = registry.resolveRoute(key);++ async function routeFor(key) {++ const r = await registry.resolveRoute(key);if (!r) throw new AiError(503, 'route.unavailable', `no route ${key}`);if (r.disabled) throw new AiError(503, 'route.unavailable', `route ${key} is disabled`);return r;@@@ -111,8 +111,8 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spaasync function llmStep(step, input, wf, run, ctx) {// The run pinned its template version when it was created; later steps use the active one.const template = run.template_key === step.template && run.template_version-- ? registry.getTemplate(step.template, run.template_version)-- : registry.activeTemplate(step.template);++ ? await registry.getTemplate(step.template, run.template_version)++ : await registry.activeTemplate(step.template);if (!template) throw new AiError(500, 'workflow.broken', `template ${step.template} is missing`);const prep = step.prepare ? PREPARE[step.prepare](input) : { vars: input };if (!prep) throw new AiError(500, 'workflow.broken', `prepare hook ${step.prepare} is missing`);@@@ -126,7 +126,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spaconst system = [render(template.system_prompt, vars).trim(), asked].filter(Boolean).join('\n\n');const user = render(template.user_prompt, vars);const routeKey = step.route || template.default_route || wf.default_route || 'default.chat';-- const route = routeFor(routeKey);++ const route = await routeFor(routeKey);const sources = Array.isArray(input.sources) ? input.sources : [];let jsonSchema = null;if (step.output === 'json') {@@@ -168,7 +168,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spaasync function passthroughStep(step, input, wf, run, ctx) {const routeKey = step.route || `${step.route_prefix || 'live'}.${input.role || 'legacy'}`;-- const route = routeFor(routeKey);++ const route = await routeFor(routeKey);const sys = typeof input.system === 'string' ? (input.system.trim() ? [{ text: input.system, cache: false }] : []): Array.isArray(input.system) ? input.system.filter(x => x && String(x.text || '').trim()).map(x => ({ text: String(x.text), cache: Boolean(x.cache) })) : [];const messages = normMessages(input.messages, input.user);@@@ -188,7 +188,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa}async function transcribeStep(step, input, wf, run, ctx) {-- const route = routeFor(step.route || 'live.stt');++ const route = await routeFor(step.route || 'live.stt');// Read where it lies (the loopback reader), one window at a time: start_sec seeks with a range request, and// timestamps are offset by start_sec unless offset_sec says otherwise. AI_MEDIA_STREAM=0 downloads first.const streamed = !config.mediaAnalysis || config.mediaAnalysis.stream !== false;@@@ -207,7 +207,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa}async function embedStep(step, input, wf, run, ctx) {-- const route = routeFor(step.route || 'default.embedding');++ const route = await routeFor(step.route || 'default.embedding');const inputs = Array.isArray(input.input) ? input.input : [input.input];const exec = await pool.execute(route, 'embed', { input: inputs, timeoutMs: route.timeout_ms || 30000 }, { ...ctx, routeKey: route.key, routeVersion: route.version, promptHash: null });const vectors = exec.result.vectors || [];@@@ -258,8 +258,8 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa}/** A quota with workflow_prefix `media.paid` and a cost cap: today's media.* spend must be under it. */-- function paidBudget() {-- return quotas && quotas.paidBudget ? quotas.paidBudget('media.paid', 'media.') : null;++ async function paidBudget() {++ return quotas && quotas.paidBudget ? await quotas.paidBudget('media.paid', 'media.') : null;}/** Refuse before downloading when the temp disk could not hold the file (a full disk stops every service). */@@@ -273,14 +273,14 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa}async function overview({ step, input, wf, run: runRow, ctx, facts, highlights, transcript, gaps }) {-- const template = registry.activeTemplate(step.template || 'media.analyze.overview');++ const template = await registry.activeTemplate(step.template || 'media.analyze.overview');const vars = {facts: facts.join('\n'),highlights: highlights.map((h) => `${clock(h.start)}–${clock(h.end)} [${h.reasons.join(', ')}] ${h.excerpt}`).join('\n') || '(none)',transcript: (transcript.text || '').slice(0, 8000) || '(no transcript)',};const call = async (routeKey) => {-- const route = registry.resolveRoute(routeKey);++ const route = await registry.resolveRoute(routeKey);if (!route || route.disabled) return { missing: true };const system = render(template.system_prompt, vars).trim();const req = { system: [{ text: system, cache: true }], messages: [{ role: 'user', content: render(template.user_prompt, vars) }], image: null, json: null, maxTokens: 250, temperature: 0.3, timeoutMs: route.timeout_ms || 120000, cacheKey: `${wf.key}:${wf.version}` };@@@ -298,7 +298,7 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spagaps.push(`The local model did not answer (${e.code || e.message}).`);}if (input.allow_paid) {-- const budget = paidBudget();++ const budget = await paidBudget();if (!budget) gaps.push('Paid providers are off for media analysis: no quota gives media.paid a cost budget.');else if (budget.spent >= budget.max_cost_usd) gaps.push(`Today's paid budget for media analysis ($${budget.max_cost_usd}) is spent.`);else {@@@ -314,7 +314,7 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa}async function step(stepDef, input, wf, runRow, ctx) {-- const mine = turn.then(() => analyse(stepDef, input, wf, runRow, ctx));++ const mine = turn.then(async () => await analyse(stepDef, input, wf, runRow, ctx));turn = mine.catch(() => {});return mine;}@@@ -346,7 +346,7 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spalet transcript = { available: false, language: input.language || 'en', text: '', segments: [] };let sttExec = null;if (streams.audio) {-- const route = registry.resolveRoute(stepDef.stt_route || 'live.stt');++ const route = await registry.resolveRoute(stepDef.stt_route || 'live.stt');if (!route || route.disabled) gaps.push('No speech-to-text route (live.stt): no transcript.');else {// In windows (sttWindowSec, 10 min): each call stays inside the provider's timeout, and one@@@ -64,59 +64,59 @@ function providerRecords(config, env) {return out;}--function seed({ registry, quotas, config, env = process.env, db }) {++async function seed({ registry, quotas, config, env = process.env, db }) {// Providersfor (const p of providerRecords(config, env)) {-- const prev = registry.getProvider(p.key);++ const prev = await registry.getProvider(p.key);if (!prev || prev.origin === 'seed') {const same = prev && Object.keys(p).every(k => JSON.stringify(p[k] ?? null) === JSON.stringify(prev[k] ?? null));-- if (!same) registry.upsertProvider({ base_url: null, secret_ref: null, ...p }, { actor: 'seed', origin: 'seed' });++ if (!same) await registry.upsertProvider({ base_url: null, secret_ref: null, ...p }, { actor: 'seed', origin: 'seed' });}}-- const shared = registry.getProvider('shared');-- const hasFallback = Boolean(registry.getProvider('fallback'));++ const shared = await registry.getProvider('shared');++ const hasFallback = Boolean(await registry.getProvider('fallback'));const fallbacks = hasFallback ? [{ provider: 'fallback', model: null }] : [];// Modelsconst models = new Set([shared.default_model, ...Object.values(config.shared.roleModels)]);-- for (const m of models) if (m && !registry.getModel('shared', m)) registry.upsertModel({ provider_key: 'shared', model_key: m, type: 'chat', supports: { json: true, vision: true } }, { actor: 'seed' });-- if (!registry.getModel('stub', 'stub-1')) registry.upsertModel({ provider_key: 'stub', model_key: 'stub-1', type: 'chat', supports: { json: true, vision: true }, metadata: { synthetic: true } }, { actor: 'seed' });-- if (!registry.getModel('whisper', 'whisper.cpp')) registry.upsertModel({ provider_key: 'whisper', model_key: 'whisper.cpp', type: 'stt', supports: { json: false } }, { actor: 'seed' });++ for (const m of models) if (m && !await registry.getModel('shared', m)) await registry.upsertModel({ provider_key: 'shared', model_key: m, type: 'chat', supports: { json: true, vision: true } }, { actor: 'seed' });++ if (!await registry.getModel('stub', 'stub-1')) await registry.upsertModel({ provider_key: 'stub', model_key: 'stub-1', type: 'chat', supports: { json: true, vision: true }, metadata: { synthetic: true } }, { actor: 'seed' });++ if (!await registry.getModel('whisper', 'whisper.cpp')) await registry.upsertModel({ provider_key: 'whisper', model_key: 'whisper.cpp', type: 'stt', supports: { json: false } }, { actor: 'seed' });// Routesfor (const role of live.ROLES) {-- registry.seedVersioned('route', `live.${role}`, {++ await registry.seedVersioned('route', `live.${role}`, {primary: { provider: 'shared', model: config.shared.roleModels[role] || null }, fallbacks,options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: ROLE_TIMEOUT[role], alias_of: null,});}-- registry.seedVersioned('route', 'live.stt', { primary: { provider: 'whisper', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: 600000, alias_of: null });-- registry.seedVersioned('route', 'default.chat', { primary: { provider: 'shared', model: null }, fallbacks, options: {}, max_output_tokens: 800, response_format: 'text', timeout_ms: 30000, alias_of: null });-- registry.seedVersioned('route', 'default.json', { primary: { provider: 'shared', model: null }, fallbacks, options: { temperature: 0.3 }, max_output_tokens: 2400, response_format: 'json', timeout_ms: 60000, alias_of: null });-- registry.seedVersioned('route', 'default.embedding', { primary: { provider: 'shared', model: env.AI_EMBEDDING_MODEL || 'text-embedding-3-small' }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: 30000, alias_of: null });++ await registry.seedVersioned('route', 'live.stt', { primary: { provider: 'whisper', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: 600000, alias_of: null });++ await registry.seedVersioned('route', 'default.chat', { primary: { provider: 'shared', model: null }, fallbacks, options: {}, max_output_tokens: 800, response_format: 'text', timeout_ms: 30000, alias_of: null });++ await registry.seedVersioned('route', 'default.json', { primary: { provider: 'shared', model: null }, fallbacks, options: { temperature: 0.3 }, max_output_tokens: 2400, response_format: 'json', timeout_ms: 60000, alias_of: null });++ await registry.seedVersioned('route', 'default.embedding', { primary: { provider: 'shared', model: env.AI_EMBEDDING_MODEL || 'text-embedding-3-small' }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: 30000, alias_of: null });// media.analyze (WS-O task 5): the local model when there is one; the paid route is used only under a media.paid budget.-- if (config.localLlm && config.localLlm.url) registry.seedVersioned('route', 'media.local', { primary: { provider: 'local', model: config.localLlm.model || null }, fallbacks: [], options: { temperature: 0.3 }, max_output_tokens: 350, response_format: 'text', timeout_ms: config.localLlm.timeoutMs, alias_of: null });-- registry.seedVersioned('route', 'media.paid', { primary: { provider: 'shared', model: config.shared.roleModels.summary || null }, fallbacks, options: { temperature: 0.3 }, max_output_tokens: 350, response_format: 'text', timeout_ms: 60000, alias_of: null });-- for (const key of HISTORICAL_ROUTES) registry.seedVersioned('route', key, { primary: { provider: 'shared', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'json', timeout_ms: null, alias_of: 'default.json' });++ if (config.localLlm && config.localLlm.url) await registry.seedVersioned('route', 'media.local', { primary: { provider: 'local', model: config.localLlm.model || null }, fallbacks: [], options: { temperature: 0.3 }, max_output_tokens: 350, response_format: 'text', timeout_ms: config.localLlm.timeoutMs, alias_of: null });++ await registry.seedVersioned('route', 'media.paid', { primary: { provider: 'shared', model: config.shared.roleModels.summary || null }, fallbacks, options: { temperature: 0.3 }, max_output_tokens: 350, response_format: 'text', timeout_ms: 60000, alias_of: null });++ for (const key of HISTORICAL_ROUTES) await registry.seedVersioned('route', key, { primary: { provider: 'shared', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'json', timeout_ms: null, alias_of: 'default.json' });// Templates, then workflows (workflows reference templates)for (const t of [...live.templates, ...core.templates, ...products.templates, ...media.templates]) {const { key, ...def } = t;-- registry.seedVersioned('template', key, { description: null, default_route: null, owner: 'ai', visibility: 'internal', metadata: {}, ...def });++ await registry.seedVersioned('template', key, { description: null, default_route: null, owner: 'ai', visibility: 'internal', metadata: {}, ...def });}for (const w of [...live.workflows, ...core.workflows, ...products.workflows, ...media.workflows]) {const { key, ...def } = w;-- registry.seedVersioned('workflow', key, { description: null, default_route: null, cache_ttl_sec: null, metadata: {}, ...def });++ await registry.seedVersioned('workflow', key, { description: null, default_route: null, cache_ttl_sec: null, metadata: {}, ...def });}// Quotas-- const seedQuota = (q) => {-- const prev = db.prepare("SELECT origin FROM quotas WHERE scope_type = ? AND scope_id = ? AND window = ? AND COALESCE(workflow_prefix, '') = ''").get(q.scope_type, q.scope_type === 'global' ? '*' : q.scope_id, q.window);-- if (!prev || prev.origin === 'seed') quotas.upsert(q, { actor: 'seed', origin: 'seed' });++ const seedQuota = async (q) => {++ const prev = await db.prepare("SELECT origin FROM quotas WHERE scope_type = ? AND scope_id = ? AND \"window\" = ? AND COALESCE(workflow_prefix, '') = ''").get(q.scope_type, q.scope_type === 'global' ? '*' : q.scope_id, q.window);++ if (!prev || prev.origin === 'seed') await quotas.upsert(q, { actor: 'seed', origin: 'seed' });};-- if (config.quotas.globalCostPerDay > 0) seedQuota({ scope_type: 'global', window: 'day', max_cost_usd: config.quotas.globalCostPerDay });-- if (config.quotas.serviceRequestsPerMinute > 0) seedQuota({ scope_type: 'service', scope_id: '*', window: 'minute', max_requests: config.quotas.serviceRequestsPerMinute });-- if (config.quotas.serviceRequestsPerDay > 0) seedQuota({ scope_type: 'service', scope_id: '*', window: 'day', max_requests: config.quotas.serviceRequestsPerDay });++ if (config.quotas.globalCostPerDay > 0) await seedQuota({ scope_type: 'global', window: 'day', max_cost_usd: config.quotas.globalCostPerDay });++ if (config.quotas.serviceRequestsPerMinute > 0) await seedQuota({ scope_type: 'service', scope_id: '*', window: 'minute', max_requests: config.quotas.serviceRequestsPerMinute });++ if (config.quotas.serviceRequestsPerDay > 0) await seedQuota({ scope_type: 'service', scope_id: '*', window: 'day', max_requests: config.quotas.serviceRequestsPerDay });}module.exports = { seed, providerRecords, HISTORICAL_ROUTES };@@@ -12,7 +12,7 @@ const live = token('live', [...ALL, 'ai.quota.attribution.manage']);const tools = token('tools', [...ALL, 'ai.quota.attribution.manage']);const plain = token('live', ALL);let h;--const run = (workflow, userId) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow, input: { role: 'chat', user: 'hi' }, attribution: { service: 'live', type: 'user', id: String(userId) } } });++const run = async (workflow, userId) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow, input: { role: 'chat', user: 'hi' }, attribution: { service: 'live', type: 'user', id: String(userId) } } });t.test('boot', async () => { h = await boot(); });@@@ -64,7 +64,7 @@ t.test('a namespace-limited token may only run workflows in its namespaces', asy});t.test('namespaces fail closed: a token without ns runs nothing (C-22/C-23 retired)', async () => {-- const run = (tok, workflow, input) => request(h.base, 'POST', '/api/v1/runs', { tok, body: { workflow, input } });++ const run = async (tok, workflow, input) => await request(h.base, 'POST', '/api/v1/runs', { tok, body: { workflow, input } });const denied = (r, what) => { assert.strictEqual(r.status, 403, what); assert.strictEqual(r.body.code, 'capability.namespace_denied', what); };// No ns claim, or an empty one: nothing, not even the service's own namespace.denied(await run(token('games', ['ai.run.create'], { ns: null }), 'games.generate_lore', {}), 'svc:games without ns -> games.*');@@@ -31,7 +31,7 @@ t.test('bad requests from one caller fail that caller only; the circuit stays clconst bad = await request(h.base, 'POST', '/api/v1/generate', { tok: attacker, body: { prompt: `BAD ${i}`, options: { cache: false } } });assert.strictEqual(bad.body.run.status, 'failed', bad.text);}-- assert.strictEqual(h.pool.health('seam').state, 'closed', 'client errors must not open the circuit');++ assert.strictEqual((await h.pool.health('seam')).state, 'closed', 'client errors must not open the circuit');const good = await request(h.base, 'POST', '/api/v1/generate', { tok: admin, body: { prompt: 'hello', options: { cache: false } } });assert.strictEqual(good.body.run.status, 'succeeded', good.text);assert.strictEqual(good.body.run.output.text, 'fine');@@@ -40,7 +40,7 @@ t.test('bad requests from one caller fail that caller only; the circuit stays clt.test('server errors still open it', async () => {seam.handler = () => ({ status: 503, body: { error: { message: 'overloaded' } } });for (let i = 0; i < 2; i++) await request(h.base, 'POST', '/api/v1/generate', { tok: admin, body: { prompt: `x ${i}`, options: { cache: false } } });-- assert.strictEqual(h.pool.health('seam').state, 'open');++ assert.strictEqual((await h.pool.health('seam')).state, 'open');});t.test('shutdown', async () => { await h.stop(); await seam.close(); });@@@ -15,7 +15,7 @@ const B = { type: 'user', id: 'usr_01JAB2C3D4E5F6G7H8J9K0MNPB' };let h;let seam;--const summarize = (tok, extra = {}) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'ai.summarize', input: { text: 'The same private text for everyone.' }, ...extra } });++const summarize = async (tok, extra = {}) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'ai.summarize', input: { text: 'The same private text for everyone.' }, ...extra } });t.test('boot with a real (non-synthetic) seam provider', async () => {seam = await seamServer((b) => ({ text: `summary #${seam.calls}`, usage: { input: 5, output: 2 } }));@@@ -62,7 +62,7 @@ t.test('another service never reads this service\'s entries', async () => {});t.test('service-scoped workflows (translation) share within the service only', async () => {-- const tr = (tok, obo) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'live.translate', input: { text: 'こんにちは、みなさん', from: 'ja', to: 'en' }, on_behalf_of: obo } });++ const tr = async (tok, obo) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'live.translate', input: { text: 'こんにちは、みなさん', from: 'ja', to: 'en' }, on_behalf_of: obo } });assert.strictEqual((await tr(live, A)).body.run.status, 'succeeded');assert.strictEqual((await tr(live, B)).body.run.status, 'cached', 'public text translation is shared inside Live');assert.strictEqual((await tr(other, A)).body.run.status, 'succeeded', 'but not with another service');@@@ -95,8 +95,8 @@ async function signIn(subject, { role = 'admin', is_owner = false, username = 'sreturn { cb, setCookie: sess || null, cookie: sess ? sess.split(';')[0] : null };}const csrfOf = (page) => (page.text.match(/name="_csrf" value="([^"]+)"/) || [])[1];--const auditRows = (where, ...args) => h.db.prepare(`SELECT * FROM audit_log WHERE ${where} ORDER BY id`).all(...args);--const sessionCount = () => h.db.prepare('SELECT COUNT(*) AS n FROM console_sessions').get().n;++const auditRows = async (where, ...args) => await h.db.prepare(`SELECT * FROM audit_log WHERE ${where} ORDER BY id`).all(...args);++const sessionCount = async () => (await h.db.prepare('SELECT COUNT(*) AS n FROM console_sessions').get()).n;/** Controls that are neither hidden nor inside a <label>. */function unlabelled(text) {@@@ -155,7 +155,7 @@ t.test('boot with a provider secret, a seam provider, a failing stub and a few rconst ok = await gen(live, `${'long prompt text '.repeat(40)}END-OF-PROMPT`);assert.strictEqual(ok.status, 'succeeded', JSON.stringify(ok.error));runIds.ok = ok.id;-- assert.ok(h.cache.stats().length >= 1, 'a real answer was cached');++ assert.ok((await h.cache.stats()).length >= 1, 'a real answer was cached');});t.test('the staff map: admin reads and manages workflows, only the owner manages providers, a global_mod gets nothing', () => {@@@ -176,11 +176,11 @@ t.test('anonymous: console pages answer the sign-in page (401), never data; writassert.strictEqual(r.headers.get('x-robots-tag'), 'noindex, nofollow');}assert.ok((await get('/console/runs?status=failed')).text.includes(`href="/auth/login?next=${encodeURIComponent('/console/runs?status=failed')}"`));-- const before = h.quotas.list().length;++ const before = (await h.quotas.list()).length;assert.strictEqual((await post('/console/quotas', null, { scope_type: 'service', scope_id: 'live', window: 'day', max_requests: '1' })).status, 401);assert.strictEqual((await post('/console/providers/okseam/status', null, { status: 'disabled' })).status, 401);-- assert.strictEqual(h.quotas.list().length, before);-- assert.strictEqual(h.registry.getProvider('okseam').status, 'active');++ assert.strictEqual((await h.quotas.list()).length, before);++ assert.strictEqual((await h.registry.getProvider('okseam')).status, 'active');});t.test('sign-in redirects to the Network with PKCE S256 as client ai, with a signed host-only flow cookie', async () => {@@@ -212,7 +212,7 @@ t.test('the callback refuses a missing or mismatched state, a forged flow cookieassert.strictEqual((await get(`/auth/callback?code=${code}&state=${loc.searchParams.get('state')}`, `ovai_flow=${body}.forged`)).status, 400, 'forged signature');code = authorize({ subject_id: people.admin, role: 'admin', challenge: 'A'.repeat(43), redirect_uri: redirect });assert.strictEqual((await get(`/auth/callback?code=${code}&state=${encodeURIComponent(loc.searchParams.get('state'))}`, flow)).status, 400, 'PKCE mismatch');-- assert.strictEqual(sessionCount(), 0);++ assert.strictEqual(await sessionCount(), 0);});t.test('non-staff get 403 and no session: a user, a global_mod, a token without a subject; refusals are audited', async () => {@@@ -222,8 +222,8 @@ t.test('non-staff get 403 and no session: a user, a global_mod, a token withoutassert.strictEqual(r.cookie, null);assert.ok(r.cb.text.includes('only for OpenVibe.Network staff'));}-- assert.strictEqual(sessionCount(), 0);-- const refused = auditRows("action = 'console.sign_in.refused'");++ assert.strictEqual(await sessionCount(), 0);++ const refused = await auditRows("action = 'console.sign_in.refused'");assert.deepStrictEqual(refused.map((x) => x.actor).sort(), ['anonymous', people.mod, people.user].sort());assert.ok(refused.every((x) => /^[0-9a-f]{32}$/.test(JSON.parse(x.metadata).ip_hash)), 'the client address is kept as a keyed hash');assert.ok(revoked.length >= 3, 'the Network refresh token is revoked straight away');@@@ -238,10 +238,10 @@ t.test('an admin signs in: host-only, HttpOnly, Secure, SameSite=Strict, short-lconst maxAge = Number((r.setCookie.match(/Max-Age=(\d+)/) || [])[1]);assert.ok(maxAge > 0 && maxAge <= 3600, `max-age ${maxAge}`);cookies.admin = r.cookie;-- const row = h.db.prepare('SELECT * FROM console_sessions WHERE subject = ?').get(people.admin);++ const row = await h.db.prepare('SELECT * FROM console_sessions WHERE subject = ?').get(people.admin);assert.notStrictEqual(row.id_hash, r.cookie.split('=')[1], 'only a hash of the session id is stored');assert.strictEqual(JSON.parse(row.staff).role, 'admin');-- assert.strictEqual(auditRows("action = 'console.sign_in' AND actor = ?", people.admin).length, 1);++ assert.strictEqual((await auditRows("action = 'console.sign_in' AND actor = ?", people.admin)).length, 1);const g = grants.filter((x) => x.grant_type === 'authorization_code').pop();assert.strictEqual(g.client_id, 'ai');assert.strictEqual(g.redirect_uri, 'https://ai.test/auth/callback');@@@ -281,9 +281,9 @@ t.test('an admin cannot use owner-only actions (ai.provider.manage): 403, auditeassert.strictEqual(r.status, 403, p);assert.ok(r.text.includes('This needs ai.provider.manage'), p);}-- assert.strictEqual(h.registry.getProvider('okseam').status, 'active');-- assert.ok(h.cache.stats().length >= 1);-- assert.strictEqual(auditRows("action = 'console.request.refused' AND actor = ?", people.admin).length, 4);++ assert.strictEqual((await h.registry.getProvider('okseam')).status, 'active');++ assert.ok((await h.cache.stats()).length >= 1);++ assert.strictEqual((await auditRows("action = 'console.request.refused' AND actor = ?", people.admin)).length, 4);});t.test('CSRF: a write needs the session token and a same-origin request', async () => {@@@ -291,7 +291,7 @@ t.test('CSRF: a write needs the session token and a same-origin request', asyncassert.strictEqual(draft.status, 201, draft.text);const v = draft.body.workflow.version;const p = `/console/workflows/ai.generate/versions/${v}/status`;-- const refusedBefore = auditRows("action = 'console.request.refused'").length;++ const refusedBefore = (await auditRows("action = 'console.request.refused'")).length;for (const [form, headers, why] of [[{ status: 'archived' }, {}, 'no token'],[{ status: 'archived', _csrf: 'x'.repeat(43) }, {}, 'wrong token'],@@@ -300,14 +300,14 @@ t.test('CSRF: a write needs the session token and a same-origin request', async]) {const r = await post(p, cookies.admin, form, headers);assert.strictEqual(r.status, 403, why);-- assert.strictEqual(h.registry.getWorkflow('ai.generate', v).status, 'draft', why);++ assert.strictEqual((await h.registry.getWorkflow('ai.generate', v)).status, 'draft', why);}-- assert.strictEqual(auditRows("action = 'console.request.refused'").length, refusedBefore + 4);++ assert.strictEqual((await auditRows("action = 'console.request.refused'")).length, refusedBefore + 4);const ok = await post(p, cookies.admin, { status: 'archived', _csrf: csrf.admin }, { Origin: 'https://ai.test', 'Sec-Fetch-Site': 'same-origin' });assert.strictEqual(ok.status, 303, ok.text);assert.strictEqual(ok.headers.get('location'), `/console/workflows/ai.generate?version=${v}&done=version_status`);-- assert.strictEqual(h.registry.getWorkflow('ai.generate', v).status, 'archived');-- const row = auditRows("action = 'workflow.status' AND target_id = 'ai.generate'").pop();++ assert.strictEqual((await h.registry.getWorkflow('ai.generate', v)).status, 'archived');++ const row = (await auditRows("action = 'workflow.status' AND target_id = 'ai.generate'")).pop();assert.strictEqual(row.actor, people.admin);assert.deepStrictEqual(JSON.parse(row.metadata), { version: v, from: 'draft', to: 'archived' });const bad = await post(p, cookies.admin, { status: 'published', _csrf: csrf.admin });@@@ -324,11 +324,11 @@ t.test('owner: providers, circuits, quotas and cache purge through the console,csrf.owner = csrfOf(page);let x = await post('/console/providers/badstub/status', cookies.owner, { status: 'disabled', _csrf: csrf.owner });assert.strictEqual(x.status, 303);-- assert.strictEqual(h.registry.getProvider('badstub').status, 'disabled');-- assert.strictEqual(auditRows("action = 'provider.update' AND target_id = 'badstub' AND actor = ?", people.owner).length, 1);++ assert.strictEqual((await h.registry.getProvider('badstub')).status, 'disabled');++ assert.strictEqual((await auditRows("action = 'provider.update' AND target_id = 'badstub' AND actor = ?", people.owner)).length, 1);x = await post('/console/providers/badstub/reset', cookies.owner, { _csrf: csrf.owner });assert.strictEqual(x.status, 303);-- assert.strictEqual(auditRows("action = 'provider.circuit_reset' AND target_id = 'badstub' AND actor = ?", people.owner).length, 1);++ assert.strictEqual((await auditRows("action = 'provider.circuit_reset' AND target_id = 'badstub' AND actor = ?", people.owner)).length, 1);assert.strictEqual((await post('/console/providers/nope/reset', cookies.owner, { _csrf: csrf.owner })).status, 404);x = await post('/console/quotas', cookies.owner, { scope_type: 'service', scope_id: 'live', window: 'fortnight', max_requests: '5', _csrf: csrf.owner });@@@ -336,20 +336,20 @@ t.test('owner: providers, circuits, quotas and cache purge through the console,assert.ok(x.text.includes('window must be minute, hour or day'));x = await post('/console/quotas', cookies.owner, { scope_type: 'service', scope_id: 'live', window: 'day', max_requests: '500', max_tokens: '', max_cost_usd: '2.5', _csrf: csrf.owner });assert.strictEqual(x.status, 303);-- const q = h.quotas.list().find((z) => z.scope_type === 'service' && z.scope_id === 'live' && z.window === 'day');++ const q = (await h.quotas.list()).find((z) => z.scope_type === 'service' && z.scope_id === 'live' && z.window === 'day');assert.strictEqual(q.max_requests, 500);assert.strictEqual(q.max_tokens, null);assert.strictEqual(q.max_cost_usd, 2.5);-- assert.strictEqual(auditRows("action = 'quota.create' AND actor = ?", people.owner).length, 1);++ assert.strictEqual((await auditRows("action = 'quota.create' AND actor = ?", people.owner)).length, 1);x = await post('/console/cache/purge', cookies.owner, { _csrf: csrf.owner });assert.strictEqual(x.status, 422, 'purging everything needs the confirmation box');-- assert.ok(h.cache.stats().length >= 1);++ assert.ok((await h.cache.stats()).length >= 1);x = await post('/console/cache/purge', cookies.owner, { workflow: 'ai.generate', _csrf: csrf.owner });assert.strictEqual(x.status, 303);assert.match(x.headers.get('location'), /done=cache_purged&removed=1$/);-- assert.strictEqual(h.cache.stats().length, 0);-- const purge = auditRows("action = 'cache.purge' AND actor = ?", people.owner).pop();++ assert.strictEqual((await h.cache.stats()).length, 0);++ const purge = (await auditRows("action = 'cache.purge' AND actor = ?", people.owner)).pop();assert.strictEqual(purge.target_id, 'ai.generate');assert.strictEqual(JSON.parse(purge.metadata).removed, 1);});@@@ -360,7 +360,7 @@ t.test('admin API changes write the same audit rows, and the audit page shows whassert.strictEqual((await request(h.base, 'POST', '/api/v1/providers/badstub/reset', { tok: ops })).status, 200);assert.strictEqual((await request(h.base, 'DELETE', '/api/v1/cache?workflow=ai.generate', { tok: ops })).status, 200);for (const action of ['provider.update', 'quota.create', 'provider.circuit_reset', 'cache.purge', 'route.version', 'workflow.version']) {-- assert.ok(auditRows('action = ? AND actor = ?', action, 'svc:ops').length >= 1, `${action} by svc:ops`);++ assert.ok((await auditRows('action = ? AND actor = ?', action, 'svc:ops')).length >= 1, `${action} by svc:ops`);}const a = await get('/console/audit', cookies.admin);assertPage(a, '/console/audit');@@@ -430,8 +430,8 @@ t.test('cancel: an admin cancels a running run (ai.workflow.manage), audited witassert.ok(page.text.includes(`action="/console/runs/${id}/cancel"`));const x = await post(`/console/runs/${id}/cancel`, cookies.admin, { _csrf: csrf.admin });assert.strictEqual(x.status, 303);-- assert.strictEqual(h.runs.get(id).status, 'cancelled');-- assert.strictEqual(auditRows("action = 'run.cancel' AND target_id = ? AND actor = ?", id, people.admin).length, 1);++ assert.strictEqual((await h.runs.get(id)).status, 'cancelled');++ assert.strictEqual((await auditRows("action = 'run.cancel' AND target_id = ? AND actor = ?", id, people.admin)).length, 1);const again = await post(`/console/runs/${id}/cancel`, cookies.admin, { _csrf: csrf.admin });assert.strictEqual(again.status, 409, 'a finished run cannot be cancelled');await request(h.base, 'POST', '/api/v1/routes/default.chat/versions', { tok: ops, body: { primary: { provider: 'okseam' }, fallbacks: [] } });@@@ -441,7 +441,7 @@ t.test('the staff mapping is applied on every request: a session whose claims noconst r = await signIn(people.mod, { role: 'admin', username: 'demoted' });assert.strictEqual(r.cb.status, 303);assert.strictEqual((await get('/console', r.cookie)).status, 200);-- h.db.prepare("UPDATE console_sessions SET staff = ? WHERE subject = ? AND revoked_at IS NULL").run(JSON.stringify({ role: 'global_mod' }), people.mod);++ await h.db.prepare("UPDATE console_sessions SET staff = ? WHERE subject = ? AND revoked_at IS NULL").run(JSON.stringify({ role: 'global_mod' }), people.mod);const x = await get('/console', r.cookie);assert.strictEqual(x.status, 403);assert.ok(x.cookies.some((c) => c.startsWith('__Host-ovai_staff=;')), 'the cookie is cleared');@@@ -453,8 +453,8 @@ t.test('sign out revokes the session', async () => {assert.strictEqual(x.status, 303);assert.ok(x.cookies.some((c) => c.startsWith('__Host-ovai_staff=;')));assert.strictEqual((await get('/console', cookies.admin)).status, 401);-- assert.strictEqual(auditRows("action = 'console.sign_out' AND actor = ?", people.admin).length, 1);-- for (const s of SECRETS) assert.ok(!JSON.stringify(h.db.prepare('SELECT * FROM audit_log').all()).includes(s), 'no secret in the audit log');++ assert.strictEqual((await auditRows("action = 'console.sign_out' AND actor = ?", people.admin)).length, 1);++ for (const s of SECRETS) assert.ok(!JSON.stringify(await h.db.prepare('SELECT * FROM audit_log').all()).includes(s), 'no secret in the audit log');});t.test('robots.txt keeps crawlers out of the console and sign-in; there is no sitemap', async () => {@@@ -40,7 +40,7 @@ t.test('store, read without the key, refuse other services', async () => {assert.ok(!r.text.includes(API_KEY), 'the key never comes back');const { validate } = require('openvibe-contracts');assert.ok(validate('ai.credential@1', r.body).valid, JSON.stringify(validate('ai.credential@1', r.body).errors));-- const row = h.db.prepare('SELECT key_enc FROM subject_credentials WHERE owner = ? AND subject = ?').get('live', DANA);++ const row = await h.db.prepare('SELECT key_enc FROM subject_credentials WHERE owner = ? AND subject = ?').get('live', DANA);assert.ok(row.key_enc.startsWith('v1:') && !row.key_enc.includes(API_KEY), 'encrypted at rest');// A models change keeps the stored key; a new endpoint needs it again.r = await request(h.base, 'PUT', `/api/v1/credentials/${DANA}`, { tok: live, body: { provider: 'openai', base_url: 'https://llm.example.com/v1', models: { chat: 'gpt-4o-mini', vision: 'gpt-4o' }, budget_usd_per_day: 0.003 } });@@@ -50,14 +50,14 @@ t.test('store, read without the key, refuse other services', async () => {r = await request(h.base, 'GET', `/api/v1/credentials/${DANA}`, { tok: tools });assert.strictEqual(r.status, 404, 'another service does not see it');// A row moved to another owner does not decrypt: the key is bound to owner and subject.-- h.db.prepare("INSERT INTO subject_credentials (owner, subject, provider, base_url, key_enc, key_hint, models, created_at, updated_at) SELECT 'tools', subject, provider, base_url, key_enc, key_hint, models, created_at, updated_at FROM subject_credentials WHERE owner = 'live'").run();++ await h.db.prepare("INSERT INTO subject_credentials (owner, subject, provider, base_url, key_enc, key_hint, models, created_at, updated_at) SELECT 'tools', subject, provider, base_url, key_enc, key_hint, models, created_at, updated_at FROM subject_credentials WHERE owner = 'live'").run();const creds = require('../server/credentials').createCredentials({ db: h.db, config: h.config });-- assert.throws(() => creds.forRun('tools', DANA));-- h.db.prepare("DELETE FROM subject_credentials WHERE owner = 'tools'").run();++ await assert.rejects(creds.forRun('tools', DANA));++ await h.db.prepare("DELETE FROM subject_credentials WHERE owner = 'tools'").run();});t.test('a run with the credential uses that key only, uncached, and its budget stops it', async () => {-- const run = (tok = live) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'live.viewers.line', input: { role: 'chat', user: 'say hi' }, credential: { subject: DANA } } });++ const run = async (tok = live) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'live.viewers.line', input: { role: 'chat', user: 'say hi' }, credential: { subject: DANA } } });let r = await run();assert.strictEqual(r.status, 201, r.text);assert.strictEqual(r.body.run.status, 'succeeded', JSON.stringify(r.body.run));@@@ -65,7 +65,7 @@ t.test('a run with the credential uses that key only, uncached, and its budget sassert.strictEqual(calls.length, 1);assert.strictEqual(calls[0].url, 'https://llm.example.com/v1/chat/completions');assert.strictEqual(calls[0].auth, `Bearer ${API_KEY}`);-- const reqs = h.db.prepare('SELECT provider_key, status, cost_usd FROM requests WHERE run_id = ?').all(r.body.run.id);++ const reqs = await h.db.prepare('SELECT provider_key, status, cost_usd FROM requests WHERE run_id = ?').all(r.body.run.id);assert.deepStrictEqual(reqs.map((x) => [x.provider_key, x.status]), [[`byo:live:${DANA}`, 'ok']]);assert.ok(Math.abs(reqs[0].cost_usd - 0.002) < 1e-9, 'priced at list price');r = await run();nnew file mode 100644@@@ -0,0 +1,11 @@++'use strict';++/**++ * A migrated database for one test boot (ADR-035), from openvibe-sdk/testing: PGlite by default; with++ * AI_TEST_STORE=pg (npm run test:pg) the PostgreSQL + PgBouncer containers, with roles and a schema of its own.++ */++const { createTestDb, pgAvailable } = require('openvibe-sdk/testing');++const { MIGRATIONS } = require('../server/db');++++const testDb = ({ store = process.env.AI_TEST_STORE || 'pglite', max = 4 } = {}) => createTestDb({ migrations: MIGRATIONS, store, service: 'ai', max });++++module.exports = { testDb, pgAvailable };@@@ -7,23 +7,23 @@ const contracts = require('openvibe-contracts');process.env.EVENTS_URL = 'http://127.0.0.1:9'; // unreachable: rows stay in the outboxprocess.env.OV_OAUTH_CLIENT_SECRET = 'ai-secret-for-tests';process.env.OV_NETWORK_INTERNAL_URL = 'http://127.0.0.1:9';--const { boot, request, token, suite, ALL } = require('./helpers');++const { boot, request, token, suite, tmpDir, ALL } = require('./helpers');const t = suite('events');const live = token('live', ALL);let h;--const queued = () => h.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all().map((r) => JSON.parse(r.envelope));++const queued = async () => (await h.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all()).map((r) => (typeof r.envelope === 'string' ? (typeof r.envelope === 'string' ? (typeof r.envelope === 'string' ? JSON.parse(r.envelope) : r.envelope) : r.envelope) : r.envelope));const valid = (env) => { const r = contracts.validate(`${env.event_type}@1`, env.payload); assert.ok(r.valid, `${env.event_type}: ${JSON.stringify(r.errors)}`); };t.test('boot with events on', async () => {-- h = await boot({ env: { AI_STUB_FALLBACK: 'true' } });-- assert.strictEqual(require('../server/events').status().enabled, true);++ h = await boot({ dir: tmpDir(), env: { AI_STUB_FALLBACK: 'true' } }); // a dir: the database outlives h.stop()++ assert.strictEqual((await require('../server/events').status()).enabled, true);});t.test('a run queues ai.run.queued then ai.run.succeeded, valid and without the prompt or the text', async () => {const r = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'ai.generate', input: { prompt: 'secret prompt words' } } });assert.strictEqual(r.status, 201, r.text);-- const evs = queued().filter((e) => e.payload.run_id === r.body.run.id);++ const evs = (await queued()).filter((e) => e.payload.run_id === r.body.run.id);assert.deepStrictEqual(evs.map((e) => e.event_type), ['ai.run.queued', 'ai.run.succeeded']);evs.forEach(valid);assert.strictEqual(evs[0].visibility, 'internal');@@@ -37,7 +37,7 @@ t.test('a cache hit queues ai.run.cached with cached_from', async () => {const body = { workflow: 'ai.generate', input: { prompt: 'cache me' } };const a = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body });const b = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body });-- const ev = queued().find((e) => e.payload.run_id === b.body.run.id);++ const ev = (await queued()).find((e) => e.payload.run_id === b.body.run.id);if (b.body.run.status !== 'cached') { assert.ok(ev, 'an event for the second run'); return; } // this workflow may not cacheassert.strictEqual(ev.event_type, 'ai.run.cached');assert.strictEqual(ev.payload.cached_from, a.body.run.id);@@@ -47,13 +47,13 @@ t.test('a cache hit queues ai.run.cached with cached_from', async () => {t.test('a failed run queues ai.run.failed with its error code; a restart announces interrupted runs', async () => {const r = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'no.such.workflow', input: {} } });assert.ok(r.status >= 400, 'refused before a run exists: no event');-- const before = queued().length;++ const before = (await queued()).length;// A queued run left behind by a restart: recoverInterrupted fails it and announces it.-- const row = h.db.prepare("SELECT * FROM runs WHERE status = 'succeeded' LIMIT 1").get();-- h.db.prepare("INSERT INTO runs (id, workflow_key, workflow_version, status, requester_type, requester_id, input, input_hash, created_at) VALUES ('run_01JAB2C3D4E5F6G7H8J9K0MNPQ', ?, ?, 'running', 'service', 'live', '{}', 'x', ?)")++ const row = await h.db.prepare("SELECT * FROM runs WHERE status = 'succeeded' LIMIT 1").get();++ await h.db.prepare("INSERT INTO runs (id, workflow_key, workflow_version, status, requester_type, requester_id, input, input_hash, created_at) VALUES ('run_01JAB2C3D4E5F6G7H8J9K0MNPQ', ?, ?, 'running', 'service', 'live', '{}', 'x', ?)").run(row.workflow_key, row.workflow_version, new Date().toISOString());-- assert.strictEqual(h.runs.recoverInterrupted(), 1);-- const ev = queued().slice(before).find((e) => e.payload.run_id === 'run_01JAB2C3D4E5F6G7H8J9K0MNPQ');++ assert.strictEqual(await h.runs.recoverInterrupted(), 1);++ const ev = (await queued()).slice(before).find((e) => e.payload.run_id === 'run_01JAB2C3D4E5F6G7H8J9K0MNPQ');assert.strictEqual(ev.event_type, 'ai.run.failed');assert.strictEqual(ev.payload.error.code, 'run.interrupted');assert.strictEqual(ev.priority, 'important');@@@ -62,14 +62,13 @@ t.test('a failed run queues ai.run.failed with its error code; a restart announct.test('close() stops the ai.run.* relay (unsent rows stay in the outbox)', async () => {const events = require('../server/events');-- const pending = events.status().pending;++ const pending = (await events.status()).pending;assert.ok(pending > 0, 'rows are waiting (Events is unreachable)');await h.stop();-- assert.strictEqual(events.status().enabled, false, 'the relay is stopped');-- const Database = require('better-sqlite3');-- const db = new Database(require('path').join(h.dir, 'ai.db'), { readonly: true });-- assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM event_outbox WHERE sent_at IS NULL AND rejected_at IS NULL').get().n, pending, 'and its rows wait for the next start');-- db.close();++ assert.strictEqual((await events.status()).enabled, false, 'the relay is stopped');++ const db = h.testdb.db;++ assert.strictEqual((await db.prepare('SELECT COUNT(*) AS n FROM event_outbox WHERE sent_at IS NULL AND rejected_at IS NULL').get()).n, pending, 'and its rows wait for the next start');++ await h.testdb.close();events._reset();});t.run();@@@ -22,7 +22,7 @@ async function provider(body) {const r = await request(h.base, 'POST', '/api/v1/providers', { tok, body: { kind: 'http', auth_mode: 'none', capabilities: ['chat', 'generate', 'json'], timeout_ms: 5000, ...body } });assert.strictEqual(r.status, 201, r.text);}--const generate = (prompt) => request(h.base, 'POST', '/api/v1/generate', { tok, body: { prompt, options: { cache: false } } });++const generate = async (prompt) => await request(h.base, 'POST', '/api/v1/generate', { tok, body: { prompt, options: { cache: false } } });t.test('boot with stub fallback off and two seam providers', async () => {ok = await seamServer(() => ({ text: 'a real answer', model: 'seam-model', usage: { input: 12, output: 4 } }));@@@ -61,7 +61,7 @@ t.test('a failing primary falls back, and the run, request log and audit all sayt.test('the circuit opens after repeated failures and the open provider is skipped', async () => {await generate('second'); // second consecutive failure of `dead` opens it-- assert.strictEqual(h.pool.health('dead').state, 'open');++ assert.strictEqual((await h.pool.health('dead')).state, 'open');const r = await generate('third');const d = await request(h.base, 'GET', `/api/v1/runs/${r.body.run.id}`, { tok });const skip = d.body.requests.find(x => x.provider_key === 'dead');@@@ -42,16 +42,24 @@ function tmpDir() {}/** Boot the service. `env` goes through config.load() and is also what secrets resolve against. */--async function boot({ env = {}, dir = tmpDir(), clock, fetchImpl, credentialFetch } = {}) {++// A boot on a dir used before is a restart on the same database (as the SQLite file was): its database stays open.++const restartable = new Map();++++async function boot({ env = {}, dir, clock, fetchImpl, credentialFetch } = {}) {++ const reuse = dir !== undefined;++ if (dir === undefined) dir = tmpDir();const fullEnv = {NODE_ENV: 'test', PORT: '0', AI_DB_PATH: path.join(dir, 'ai.db'), OV_NETWORK_PUBLIC_KEY: publicKey,AI_PROVIDER_RETRY_DELAY_MS: '5', AI_QUOTA_SERVICE_RPM: '0', AI_QUOTA_SERVICE_RPD: '0',...env,};const config = load(fullEnv);-- const h = await start({ config, env: fullEnv, log: silent, clock, fetchImpl, credentialFetch });++ // One database per boot (PGlite, or AI_TEST_STORE=pg: the containers), dropped when the boot stops.++ const testdb = (reuse && restartable.get(dir)) || await require('./db').testDb();++ if (reuse) restartable.set(dir, testdb);++ const h = await start({ config, db: testdb.db, env: fullEnv, log: silent, clock, fetchImpl, credentialFetch });const base = `http://127.0.0.1:${h.server.address().port}`;-- return { ...h, base, dir, env: fullEnv, stop: () => h.close() };++ return { ...h, base, dir, env: fullEnv, testdb, stop: async () => { await h.close(); if (!reuse) await testdb.close(); } };}async function request(base, method, p, { tok, body, headers = {} } = {}) {ddeleted file mode 100644@@@ -1,113 +0,0 @@--'use strict';--// scripts/import-from-live.js against a synthetic Live snapshot: --dry-run writes nothing; a real--// run imports usage, budgets, per-role models and pricing, and holds what cannot be represented--// (secrets, BYO keys, text-less translation cache); a second run imports nothing; a new Live row--// is picked up alone. Secret values never appear in the report or the holds.----const assert = require('assert');--const fs = require('fs');--const path = require('path');--const Database = require('better-sqlite3');--const { suite, tmpDir } = require('./helpers');--const { run } = require('../scripts/import-from-live');----const t = suite('import');--const dir = tmpDir();--const liveDb = path.join(dir, 'live-snapshot.db');--const aiDb = path.join(dir, 'ai.db');--const SECRET = 'sk-live-snapshot-secret-DO-NOT-PRINT';--const env = { NODE_ENV: 'test', AI_QUOTA_SERVICE_RPM: '0', AI_QUOTA_SERVICE_RPD: '0' };--const quiet = () => {};--const today = new Date().toISOString().slice(0, 10);----t.test('build a Live snapshot', () => {-- const d = new Database(liveDb);-- d.exec(`CREATE TABLE site_settings (key TEXT PRIMARY KEY, value TEXT NOT NULL DEFAULT '', description TEXT, type TEXT, updated_at DATETIME);-- CREATE TABLE ai_usage (id INTEGER PRIMARY KEY AUTOINCREMENT, kind TEXT, model TEXT, input_tokens INTEGER, output_tokens INTEGER, cost_usd REAL, owner_user_id INTEGER, source TEXT, created_at DATETIME, cached_tokens INTEGER, role TEXT, provider TEXT, latency_ms INTEGER);-- CREATE TABLE channel_ai_config (user_id INTEGER PRIMARY KEY, enabled INTEGER, use_shared_key INTEGER, daily_budget_cents INTEGER, byo_key TEXT, byo_base_url TEXT, byo_model TEXT, settings_json TEXT);-- CREATE TABLE translations (key TEXT PRIMARY KEY, src TEXT, dst TEXT, text TEXT NOT NULL, created_at DATETIME);-- CREATE TABLE ai_timeline_cache (user_id INTEGER PRIMARY KEY, payload TEXT, generated_at DATETIME);-- CREATE TABLE ai_chatbot_configs (user_id INTEGER PRIMARY KEY, api_token TEXT);`);-- const set = d.prepare('INSERT INTO site_settings (key, value) VALUES (?, ?)');-- for (const [k, v] of [['ai_enabled', 'true'], ['ai_provider', 'openai'], ['ai_api_key', SECRET], ['ai_model', 'gpt-5-mini'], ['ai_model_director', 'gpt-5'], ['ai_model_vision', ''],-- ['ai_pricing_json', '{"gpt-5-nano":{"in":0.05,"out":0.4,"cached":0.005},"default":{"in":1,"out":2}}'], ['ai_max_cost_usd_per_day', '5'], ['ai_viewers_global_cap_usd_per_day', '2']]) set.run(k, v);-- const use = d.prepare('INSERT INTO ai_usage (kind, model, input_tokens, output_tokens, cost_usd, owner_user_id, created_at, cached_tokens, provider) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)');-- use.run('translate', 'gpt-5-nano', 100, 20, 0.01, null, `${today} 10:00:00`, 0, 'shared');-- use.run('ai_viewers_director', 'gpt-5', 1000, 200, 0.2, 42, `${today} 10:01:00`, 500, 'shared');-- use.run('ai_viewers', 'llama3', 50, 10, 0, 7, '2026-09-01 08:00:00', 0, 'byo');-- use.run('weird_kind', 'gpt-5-mini', 10, 10, 0.001, null, '2026-09-02 08:00:00', 0, 'shared');-- use.run('paste_text', 'gpt-5-mini', 10, 10, 0.001, null, 'garbage', 0, 'shared');-- const ch = d.prepare('INSERT INTO channel_ai_config (user_id, use_shared_key, daily_budget_cents, byo_key, byo_base_url) VALUES (?, ?, ?, ?, ?)');-- ch.run(42, 1, 20, '', '');-- ch.run(7, 0, 20, 'sk-byo-streamer-secret', 'https://llm.example.org/v1');-- d.prepare("INSERT INTO translations (key, src, dst, text) VALUES ('abc', 'ja', 'en', 'hello')").run();-- d.close();--});----t.test('--dry-run reports the plan and writes nothing', () => {-- const r = run({ liveDb, aiDb, dryRun: true, env, log: quiet });-- assert.strictEqual(r.dry_run, true);-- assert.strictEqual(fs.existsSync(aiDb), false, 'no AI database created');-- assert.strictEqual(r.sources['live.ai_usage'].imported, 4);-- assert.strictEqual(r.sources['live.ai_usage'].held, 1);-- assert.ok(!JSON.stringify(r).includes(SECRET));--});----t.test('a real run imports and holds, without leaking secrets', () => {-- const r = run({ liveDb, aiDb, env, log: quiet });-- assert.ok(!JSON.stringify(r).includes(SECRET) && !JSON.stringify(r).includes('sk-byo'));-- assert.ok(r.environment.some(l => l.startsWith('AI_API_KEY=<')), 'tells the operator which secret NAME to set');-- const d = new Database(aiDb, { readonly: true });-- const quotas = d.prepare('SELECT * FROM quotas WHERE origin = ?').all('import');-- assert.ok(quotas.some(q => q.scope_type === 'global' && q.max_cost_usd === 5 && !q.workflow_prefix));-- assert.ok(quotas.some(q => q.scope_type === 'global' && q.max_cost_usd === 2 && q.workflow_prefix === 'live.viewers.'));-- assert.ok(quotas.some(q => q.scope_type === 'attribution' && q.scope_id === 'live:user:42' && q.max_cost_usd === 0.2));-- const route = d.prepare("SELECT * FROM routes WHERE key = 'live.director' ORDER BY version DESC LIMIT 1").get();-- assert.strictEqual(route.primary_model, 'gpt-5');-- assert.strictEqual(route.created_by, 'import');-- const model = d.prepare("SELECT * FROM models WHERE provider_key = 'shared' AND model_key = 'gpt-5-nano'").get();-- assert.strictEqual(model.cost_in_per_mtok, 0.05);-- const usage = d.prepare('SELECT * FROM usage_daily').all();-- assert.ok(usage.some(u => u.workflow_key === 'live.translate' && u.day === today));-- assert.ok(usage.some(u => u.workflow_key === 'live.viewers.plan' && u.attribution === 'live:user:42' && u.tokens_cached === 500));-- assert.ok(usage.some(u => u.provider_key === 'live-byo'));-- assert.ok(usage.some(u => u.workflow_key === 'live.complete'), 'unknown kinds land on live.complete');-- const counter = d.prepare("SELECT * FROM usage_counters WHERE scope_type = 'global' AND window = 'day'").get();-- assert.ok(Math.abs(counter.cost_usd - 0.21) < 1e-9, "today's spend counts toward today's cap");-- const holds = d.prepare('SELECT * FROM import_holds').all();-- const reasons = Object.fromEntries(holds.map(h => [`${h.source}:${h.source_key}`, h.reason]));-- assert.strictEqual(reasons['live.site_settings:ai_api_key'], 'secret');-- assert.strictEqual(reasons['live.channel_ai_config:7'], 'byo_provider');-- assert.strictEqual(reasons['live.translations:*'], 'unrepresentable');-- assert.strictEqual(reasons['live.ai_usage:5'], 'unrepresentable');-- assert.ok(!JSON.stringify(holds).includes(SECRET) && !JSON.stringify(holds).includes('sk-byo'));-- d.close();--});----t.test('a second run imports nothing and changes no totals', () => {-- const before = new Database(aiDb, { readonly: true });-- const sum = (x) => x.prepare('SELECT SUM(requests) r, SUM(cost_usd) c FROM usage_daily').get();-- const s1 = sum(before);-- before.close();-- const r = run({ liveDb, aiDb, env, log: quiet });-- for (const [src, s] of Object.entries(r.sources)) assert.strictEqual(s.imported + s.held, 0, `${src} imported again: ${JSON.stringify(s)}`);-- const after = new Database(aiDb, { readonly: true });-- assert.deepStrictEqual(sum(after), s1);-- assert.strictEqual(after.prepare("SELECT COUNT(*) n FROM routes WHERE key = 'live.director'").get().n, 2, 'no extra route version');-- after.close();--});----t.test('new Live rows and changed settings are picked up alone', () => {-- const d = new Database(liveDb);-- d.prepare("INSERT INTO ai_usage (kind, model, input_tokens, output_tokens, cost_usd, created_at, provider) VALUES ('hero_slogans', 'gpt-5-mini', 5, 5, 0.002, ?, 'shared')").run(`${today} 11:00:00`);-- d.prepare("UPDATE site_settings SET value = '8' WHERE key = 'ai_max_cost_usd_per_day'").run();-- d.close();-- const r = run({ liveDb, aiDb, env, log: quiet });-- assert.strictEqual(r.sources['live.ai_usage'].imported, 1);-- assert.strictEqual(r.sources['live.site_settings'].imported, 1);-- const a = new Database(aiDb, { readonly: true });-- assert.strictEqual(a.prepare("SELECT max_cost_usd FROM quotas WHERE scope_type = 'global' AND workflow_prefix IS NULL").get().max_cost_usd, 8);-- a.close();--});----t.run();@@@ -40,7 +40,7 @@ const LOG = [const t = suite('media-analysis');let media; let mediaBase;const tok = token('live', ALL);--const run = (h, input) => request(h.base, 'POST', '/api/v1/runs?wait=60000', { tok, body: { workflow: 'media.analyze', input: { media_url: `${mediaBase}/fixture.mp4`, language: 'en', ...input } } });++const run = async (h, input) => await request(h.base, 'POST', '/api/v1/runs?wait=60000', { tok, body: { workflow: 'media.analyze', input: { media_url: `${mediaBase}/fixture.mp4`, language: 'en', ...input } } });const completion = (content) => ({ id: 'x', object: 'chat.completion', model: 'fake-model', choices: [{ index: 0, message: { role: 'assistant', content }, finish_reason: 'stop' }], usage: { prompt_tokens: 50, completion_tokens: 20 } });t.test('the parser reads every filter\'s lines; loud means well above the typical second', async () => {@@@ -151,7 +151,7 @@ t.test('local model down: extractive, and it says so; paid only with allow_paidassert.ok(out.gaps.some((g) => /no quota gives media\.paid a cost budget/.test(g)));assert.strictEqual(paid.calls, 0, 'no paid call without a budget');-- h.quotas.upsert({ scope_type: 'global', scope_id: '*', window: 'day', max_cost_usd: 0.5, workflow_prefix: 'media.paid' }, { actor: 'test' });++ await h.quotas.upsert({ scope_type: 'global', scope_id: '*', window: 'day', max_cost_usd: 0.5, workflow_prefix: 'media.paid' }, { actor: 'test' });r = await run(h, { allow_paid: true });out = r.body.run.output;assert.deepStrictEqual([out.overview.source, out.overview.text], ['paid', 'Paid overview of the recording.'], JSON.stringify(out.gaps));@@@ -163,7 +163,7 @@ t.test('local model down: extractive, and it says so; paid only with allow_paidconst day = new Date().toISOString().slice(0, 10);const callsBefore = paid.calls;-- h.db.prepare("INSERT INTO usage_daily (day, requester, attribution, workflow_key, provider_key, model_key, requests, cost_usd) VALUES (?, 'service:live', '', 'media.analyze', 'shared', 'paid-model', 1, 0.75)").run(day);++ await h.db.prepare("INSERT INTO usage_daily (day, requester, attribution, workflow_key, provider_key, model_key, requests, cost_usd) VALUES (?, 'service:live', '', 'media.analyze', 'shared', 'paid-model', 1, 0.75)").run(day);r = await run(h, { allow_paid: true });out = r.body.run.output;assert.strictEqual(r.body.run.status, 'succeeded', 'a spent budget refuses the paid call, not the analysis');@@@ -85,7 +85,7 @@ t.test('/metrics: 404 through a proxy; queued and running runs, route templates,});t.test('an open circuit on a real provider degrades /api/ready (still 200) and shows on /metrics', async () => {-- h.db.prepare("INSERT INTO provider_health (provider_key, state, consecutive_failures, opened_at, last_error) VALUES ('shared', 'open', 3, ?, 'test')").run(Date.now());++ await h.db.prepare("INSERT INTO provider_health (provider_key, state, consecutive_failures, opened_at, last_error) VALUES ('shared', 'open', 3, ?, 'test')").run(Date.now());const r = await request(h.base, 'GET', '/api/ready');assert.strictEqual(r.status, 200, r.text);assert.strictEqual(r.body.status, 'degraded');@@@ -94,7 +94,7 @@ t.test('an open circuit on a real provider degrades /api/ready (still 200) and sconst text = (await get(h.base, '/metrics')).body;assert.ok(/ai_provider_circuit\{provider="shared",state="open"\} 1\n/.test(text));assert.ok(/ai_provider_circuit\{provider="shared",state="closed"\} 0\n/.test(text));-- h.pool.resetHealth('shared');++ await h.pool.resetHealth('shared');});t.test('a broken database makes the service unready (503); /metrics still answers', async () => {@@@ -14,8 +14,8 @@ const b = token('tools', ['ai.run.create', 'ai.run.read']);let h;const sleep = (ms) => new Promise(r => setTimeout(r, ms));--const create = (tok, i) => request(h.base, 'POST', '/api/v1/runs', { tok, body: { workflow: 'ai.generate', input: { prompt: `queued ${i}` }, options: { cache: false } } });--const count = (who) => h.db.prepare('SELECT COUNT(*) AS n FROM runs WHERE requester_id = ?').get(who).n;++const create = async (tok, i) => await request(h.base, 'POST', '/api/v1/runs', { tok, body: { workflow: 'ai.generate', input: { prompt: `queued ${i}` }, options: { cache: false } } });++const count = async (who) => (await h.db.prepare('SELECT COUNT(*) AS n FROM runs WHERE requester_id = ?').get(who)).n;function assertFull(r, scope) {assert.strictEqual(r.status, 429, r.text);@@@ -39,9 +39,9 @@ t.test('one caller is held to its per-caller cap; others still get in', async ()assert.strictEqual(r.status, 202, r.text);mine.push(r.body.run.id);}-- const before = count('live');++ const before = await count('live');assertFull(await create(a, 'over'), 'caller');-- assert.strictEqual(count('live'), before, 'a refused run leaves no row');++ assert.strictEqual(await count('live'), before, 'a refused run leaves no row');const other = await create(b, 0);assert.strictEqual(other.status, 202, 'another caller still queues');h.mine = mine;@@@ -12,8 +12,8 @@ const tools = token('tools', ['ai.run.create', 'ai.run.read']);let h;let seam;--const runCount = () => h.db.prepare('SELECT COUNT(*) n FROM runs').get().n;--const gen = (tok, prompt, extra = {}) => request(h.base, 'POST', '/api/v1/generate', { tok, body: { prompt, options: { cache: false }, ...extra } });++const runCount = async () => (await h.db.prepare('SELECT COUNT(*) n FROM runs').get()).n;++const gen = async (tok, prompt, extra = {}) => await request(h.base, 'POST', '/api/v1/generate', { tok, body: { prompt, options: { cache: false }, ...extra } });t.test('boot with a counting seam provider as the only route target', async () => {seam = await seamServer(() => ({ text: 'ok', usage: { input: 1000, output: 1000 } }));@@@ -27,7 +27,7 @@ t.test('boot with a counting seam provider as the only route target', async () =t.test('the per-service rate refuses the 4th request with 429 before the provider is called', async () => {for (let i = 0; i < 3; i++) assert.strictEqual((await gen(live, `p${i}`)).status, 201);const callsBefore = seam.calls;-- const runsBefore = runCount();++ const runsBefore = await runCount();const r = await gen(live, 'one too many');assert.strictEqual(r.status, 429);assert.strictEqual(r.body.code, 'quota.exceeded');@@@ -36,7 +36,7 @@ t.test('the per-service rate refuses the 4th request with 429 before the provideassert.strictEqual(r.body.quota.scope_type, 'service');assert.strictEqual(r.body.quota.scope_id, 'live');assert.strictEqual(seam.calls, callsBefore, 'the provider was never called');-- assert.strictEqual(runCount(), runsBefore, 'no run was created');++ assert.strictEqual(await runCount(), runsBefore, 'no run was created');});t.test("'*' service quotas give each service its own window", async () => {@@@ -99,12 +99,12 @@ t.test('passthrough prompts and inline images are not kept on the run record', aawait request(h.base, 'POST', '/api/v1/providers', { tok: live, body: { key: 'slowstub', kind: 'stub', capabilities: ['chat', 'generate', 'summarize', 'json', 'vision'], metadata: { delay_ms: 10 } } });const p = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'live.chat.insight', input: { role: 'chat', kind: 'chat_user', user: 'PRIVATE CHAT LOG: alice said hi' } } });assert.strictEqual(p.body.run.status, 'succeeded');-- const row = h.db.prepare('SELECT input, input_hash FROM runs WHERE id = ?').get(p.body.run.id);++ const row = await h.db.prepare('SELECT input, input_hash FROM runs WHERE id = ?').get(p.body.run.id);assert.ok(!row.input.includes('PRIVATE CHAT LOG'), 'raw prompt not stored');assert.ok(row.input_hash);const png = `data:image/png;base64,${Buffer.alloc(600, 7).toString('base64')}`;const img = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'live.stream.describe_frame', input: { image: { data_url: png } } } });-- const irow = h.db.prepare('SELECT input FROM runs WHERE id = ?').get(img.body.run.id);++ const irow = await h.db.prepare('SELECT input FROM runs WHERE id = ?').get(img.body.run.id);assert.ok(!irow.input.includes(png.slice(40, 120)) && irow.input.includes('data_url_sha256'));await request(h.base, 'POST', '/api/v1/providers/slowstub/disable', { tok: live });const failed = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'live.chat.insight', input: { role: 'chat', user: 'again' } } });@@@ -138,11 +138,11 @@ t.test('runs still in flight at shutdown end as run.interrupted, retryable afterconst d = await getRun(r.body.run.id);assert.strictEqual(d.run.status, 'failed');assert.strictEqual(d.run.error.code, 'run.interrupted');-- h.db.prepare("UPDATE runs SET status = 'running', error_code = NULL, finished_at = NULL WHERE id = ?").run(r.body.run.id);++ await h.db.prepare("UPDATE runs SET status = 'running', error_code = NULL, finished_at = NULL WHERE id = ?").run(r.body.run.id);await h.stop();h = await boot({ dir, env: { AI_MAX_CONCURRENT_RUNS: '1', AI_STUB_FALLBACK: 'false' } });assert.strictEqual((await getRun(r.body.run.id)).run.error.code, 'run.interrupted', 'boot recovery marks orphans too');-- assert.strictEqual(h.registry.getWorkflow('ai.generate').version, 2, 'a reboot does not clobber admin versions with the seed');++ assert.strictEqual((await h.registry.getWorkflow('ai.generate')).version, 2, 'a reboot does not clobber admin versions with the seed');});t.test('shutdown', async () => { await h.stop(); });@@@ -28,7 +28,7 @@ t.test('address policy: internal, loopback, link-local, mapped IPv6 are not publfor (const ip of ['1.1.1.1', '93.184.216.34', '2606:4700:4700::1111']) assert.strictEqual(isPublicAddress(ip), true, ip);});--t.test('object storage is followed only as a redirect from Media', () => {++t.test('object storage is followed only as a redirect from Media', async () => {const f = createFetcher(load({ NODE_ENV: 'test', OV_MEDIA_INTERNAL_URL: 'http://127.0.0.1:4100' }));const media = { url: new URL('https://openvibe.media/v/1'), internal: false };const internal = { url: new URL('http://127.0.0.1:4100/v/1'), internal: true };@@@ -37,11 +37,11 @@ t.test('object storage is followed only as a redirect from Media', () => {assert.strictEqual(f.follow(media, b2).url.hostname, 's3.us-west-004.backblazeb2.com', 'Media -> B2');assert.strictEqual(f.follow(internal, 'https://acct.r2.cloudflarestorage.com/b/x').internal, false, 'internal Media -> R2, as a public target');const refused = (fn, why) => { try { fn(); assert.fail(why); } catch (e) { assert.strictEqual(e.code, 'fetch.refused', why); } };-- refused(() => f.follow(other, b2), 'another host may not send it to storage');-- refused(() => f.judge(b2), 'storage is never a URL a caller names');-- refused(() => f.follow(media, 'http://s3.us-west-004.backblazeb2.com/x'), 'https only');-- refused(() => f.follow(media, 'https://evil.example/x'), 'not storage');-- refused(() => f.follow(media, 'https://user:[email protected]/x'), 'no credentials in the URL');++ await refused(() => f.follow(other, b2), 'another host may not send it to storage');++ await refused(() => f.judge(b2), 'storage is never a URL a caller names');++ await refused(() => f.follow(media, 'http://s3.us-west-004.backblazeb2.com/x'), 'https only');++ await refused(() => f.follow(media, 'https://evil.example/x'), 'not storage');++ await refused(() => f.follow(media, 'https://user:[email protected]/x'), 'no credentials in the URL');});t.test('URL rules: https + allow-listed host only', async () => {@@@ -20,7 +20,7 @@ t.test('boot with an internal Media stand-in', async () => {});t.test('a window starts at start_sec and its timestamps follow', async () => {-- const run = (input) => request(h.base, 'POST', '/api/v1/runs?wait=20000', { tok, body: { workflow: 'live.media.transcribe', input: { media_url: `${base}/v/7`, language: 'en', ...input } } });++ const run = async (input) => await request(h.base, 'POST', '/api/v1/runs?wait=20000', { tok, body: { workflow: 'live.media.transcribe', input: { media_url: `${base}/v/7`, language: 'en', ...input } } });let r = await run({ start_sec: 300, seconds: 60, low_power: true });assert.strictEqual(r.body.run.status, 'succeeded', JSON.stringify(r.body.run.error || r.body));const segs = r.body.run.output.segments;@@@ -18,7 +18,7 @@ let h, seam, network;const prefs = { [A.id]: { style: 'casual', length: 'short', perspective: 'a night-shift worker', history: false } };const puts = [];const systemOf = () => (seam.last.system || []).map((s) => s.text || s).join('\n');--const summarize = (extra = {}) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'ai.summarize', input: { text: 'A long day at the plant, told briefly.' }, ...extra } });++const summarize = async (extra = {}) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'ai.summarize', input: { text: 'A long day at the plant, told briefly.' }, ...extra } });t.test('boot with a stub Network and a seam provider', async () => {network = http.createServer((req, res) => {@@@ -56,7 +56,7 @@ t.test("a run on A's behalf follows A's preferences; history off keeps no inputassert.strictEqual(r.body.run.status, 'succeeded', r.text);const sys = systemOf();assert.ok(sys.includes('casual style') && sys.includes('Keep it short.') && sys.includes('a night-shift worker'), sys);-- const row = h.db.prepare('SELECT input, cache_key, options FROM runs WHERE id = ?').get(r.body.run.id);++ const row = await h.db.prepare('SELECT input, cache_key, options FROM runs WHERE id = ?').get(r.body.run.id);assert.strictEqual(row.input, 'null'); assert.strictEqual(row.cache_key, null);assert.strictEqual(JSON.parse(row.options).history, false);const calls = seam.calls;@@@ -70,13 +70,13 @@ t.test('runs for someone without preferences, or for nobody, are untouched', asyassert.ok(!systemOf().includes('The person this is for asked'));const r = await summarize();assert.ok(!systemOf().includes('The person this is for asked'));-- assert.notStrictEqual(h.db.prepare('SELECT input FROM runs WHERE id = ?').get(r.body.run.id).input, 'null');++ assert.notStrictEqual((await h.db.prepare('SELECT input FROM runs WHERE id = ?').get(r.body.run.id)).input, 'null');});t.test('ai.usage_summary: 30-day runs and tokens per person, written only when changed', async () => {const mods = require('../server/user-modules').createUserModules({ db: h.db, config: h.config || { networkInternalUrl: h.env.OV_NETWORK_INTERNAL_URL }, env: h.env, log: {} });mods.ensureSchema();-- const s = mods.summarize(A.id);++ const s = await mods.summarize(A.id);assert.strictEqual(s.runs_30d, 2); assert.strictEqual(s.tokens_30d, 80);assert.deepStrictEqual(s.by_service, { live: 2 });assert.ok(await mods.push(A.id));@@@ -87,7 +87,7 @@ t.test('ai.usage_summary: 30-day runs and tokens per person, written only when cassert.strictEqual(await mods.push(A.id), false, 'unchanged: not written again');assert.strictEqual(await mods.scan(Date.now() + 1000), 2, 'the scan sees A and B (their runs were just made)');assert.deepStrictEqual(puts.map((p) => p.subject).sort(), [A.id, B.id].sort());-- assert.strictEqual(mods.summarize('usr_01JAB2C3D4E5F6G7H8J9K0MNPC'), null);++ assert.strictEqual(await mods.summarize('usr_01JAB2C3D4E5F6G7H8J9K0MNPC'), null);});t.test('shutdown', async () => { await h.stop(); await seam.close(); network.close(); });@@@ -10,7 +10,7 @@ const { boot, request, token, suite, ALL } = require('./helpers');const t = suite('versioning');const tok = token('live', ALL);let h;--const run = (body) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body });++const run = async (body) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body });t.test('boot', async () => { h = await boot(); });@@@ -34,7 +34,7 @@ t.test('an invalid edit is refused and creates no version', async () => {assert.strictEqual(bad.status, 422);const bad2 = await request(h.base, 'POST', '/api/v1/workflows/ai.summarize/versions', { tok, body: { steps: [{ kind: 'llm', template: 'no.such.template' }] } });assert.strictEqual(bad2.status, 422);-- assert.strictEqual(h.registry.getWorkflow('ai.summarize').version, 1);++ assert.strictEqual((await h.registry.getWorkflow('ai.summarize')).version, 1);});t.test('deprecating the newest workflow version sends new runs to the newest active one', async () => {@@@ -72,11 +72,11 @@ t.test('historical route keys are explicit aliases of default.json', async () =>});t.test('seed: a code change becomes a new version, unless an admin versioned the key', async () => {-- const before = h.registry.getTemplate('ai.generate').version;-- h.registry.seedVersioned('template', 'ai.generate', { name: 'Generate text', system_prompt: '{{system}}', user_prompt: 'CHANGED {{prompt}}', default_route: 'default.chat', input_schema: { type: 'object' }, output_schema: { type: 'object' } });-- assert.strictEqual(h.registry.getTemplate('ai.generate').version, before + 1);-- assert.strictEqual(h.registry.getTemplate('ai.generate').created_by, 'seed');-- const again = h.registry.seedVersioned('template', 'ai.summarize', { name: 'x', user_prompt: 'seed wants this' });++ const before = (await h.registry.getTemplate('ai.generate')).version;++ await h.registry.seedVersioned('template', 'ai.generate', { name: 'Generate text', system_prompt: '{{system}}', user_prompt: 'CHANGED {{prompt}}', default_route: 'default.chat', input_schema: { type: 'object' }, output_schema: { type: 'object' } });++ assert.strictEqual((await h.registry.getTemplate('ai.generate')).version, before + 1);++ assert.strictEqual((await h.registry.getTemplate('ai.generate')).created_by, 'seed');++ const again = await h.registry.seedVersioned('template', 'ai.summarize', { name: 'x', user_prompt: 'seed wants this' });assert.strictEqual(again.created_by, 'svc:live', 'admin version kept');});@@@ -82,19 +82,19 @@ t.test('boot seeds every required workflow key', async () => {EXAMPLES['live.media.transcribe'].media_url = `${mediaBase}/v/1`;EXAMPLES['media.analyze'].media_url = `${mediaBase}/fixture.mp4`;h = await boot({ env: { OV_MEDIA_INTERNAL_URL: mediaBase } });-- const keys = h.registry.listWorkflows().map(w => w.key);++ const keys = (await h.registry.listWorkflows()).map(w => w.key);for (const k of ['wiki.generate_space', 'wiki.generate_page', 'blog.draft_post', 'news.summarize_story', 'news.compare_perspectives', 'reviews.summarize_entity', 'deals.enrich_deal','coupons.extract_coupon', 'trade.summarize_market_context', 'codes.generate_docs', 'games.generate_lore', 'moderation.classify', 'tools.describe','live.translate', 'live.stream.describe_frame', 'live.stream.summarize', 'live.viewers.reply', 'live.chat.insight', 'live.hero.slogans', 'live.stream.recap', 'network.site_copy']) {assert.ok(keys.includes(k), `missing ${k}`);}for (const k of ['default.chat', 'default.json', 'default.embedding', 'wiki.generate', 'blog.draft', 'news.summarize', 'reviews.summarize', 'deals.enrich', 'coupons.extract', 'trade.summarize', 'codes.generate_docs', 'games.generate_lore', 'tools.describe', 'moderation.classify']) {-- assert.ok(h.registry.getRoute(k), `missing historical route ${k}`);++ assert.ok(await h.registry.getRoute(k), `missing historical route ${k}`);}});t.test('every workflow has compiling, versioned input and output schemas and an example', async () => {-- for (const w of h.registry.listWorkflows()) {++ for (const w of await h.registry.listWorkflows()) {assert.strictEqual(w.version, 1);schemas.assertSchema(w.input_schema, `${w.key} input`);schemas.assertSchema(w.output_schema, `${w.key} output`);@@@ -105,7 +105,7 @@ t.test('every workflow has compiling, versioned input and output schemas and an});t.test('every workflow runs end to end on the stub and returns schema-valid output', async () => {-- for (const w of h.registry.listWorkflows()) {++ for (const w of await h.registry.listWorkflows()) {if (w.key === 'media.analyze' && !HAS_FFMPEG) { console.log('workflow media.analyze end to end: skipped (no ffprobe)'); continue; }const r = await request(h.base, 'POST', '/api/v1/runs?wait=10000', { tok, body: { workflow: w.key, input: EXAMPLES[w.key] } });assert.strictEqual(r.status, 201, `${w.key}: ${r.status} ${r.text.slice(0, 300)}`);
@@ -11,6 +11,14 @@ NODE_ENV=development
@@ -11,6 +11,14 @@ NODE_ENV=development
BASE_URL=http://localhost:4700
BASE_URL=http://localhost:4700
# SQLite (production unit: /var/lib/openvibe-ai/ai.db via StateDirectory=openvibe-ai)
# SQLite (production unit: /var/lib/openvibe-ai/ai.db via StateDirectory=openvibe-ai)
+# PostgreSQL (ADR-035), written by OpenVibe.Host roles/data/add-service.sh ai: the runtime role through PgBouncer,
+# the owner on a direct connection (migrations and the one-time import), and Valkey (shared limit counters).
+# Without DATABASE_URL, development uses an embedded PGlite database in data/pglite.
+DATABASE_URL=
+DATABASE_DIRECT_URL=
+VALKEY_URL=
+VALKEY_PREFIX=
+# The SQLite file of releases before PostgreSQL: read once by scripts/migrate-to-postgres.js.
AI_DB_PATH=./data/ai.db
AI_DB_PATH=./data/ai.db
# Identity: OpenVibe.Network signs the service tokens callers present (audience openvibe.ai).
# Identity: OpenVibe.Network signs the service tokens callers present (audience openvibe.ai).
@@ -9,6 +9,9 @@ jobs:
@@ -9,6 +9,9 @@ jobs:
uses: OpenVibers/OpenVibe.Shared/.github/workflows/test.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176
uses: OpenVibers/OpenVibe.Shared/.github/workflows/test.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176
with:
with:
contracts-service: ai
contracts-service: ai
+ # ADR-035: PostgreSQL 18 + PgBouncer (transaction mode) + Valkey 9 containers (openvibe-sdk
+ # scripts/test-services.sh). npm test runs every file on PGlite; test:pg runs them all through PgBouncer.
+ test: 'eval "$(node_modules/openvibe-sdk/scripts/test-services.sh up)" && npm test && npm run test:pg'
# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).
# Secret scan (gitleaks) and dependency audit, shared by every OpenVibers repo (Track Q).
security:
security:
uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176
uses: OpenVibers/OpenVibe.Shared/.github/workflows/security.yml@423fe519b5fe9594cc81b670e5ba5cb9f21f6176
@@ -21,7 +21,8 @@ owns publication truth: Wiki, Blog, News, Live, … decide what to publish.
@@ -21,7 +21,8 @@ owns publication truth: Wiki, Blog, News, Live, … decide what to publish.
- the eleven record groups ([below](#the-eleven-record-groups)): providers, models, routing profiles,
- the eleven record groups ([below](#the-eleven-record-groups)): providers, models, routing profiles,
prompt templates, workflow definitions, runs, the request log (hashes, not prompts), citations, the
prompt templates, workflow definitions, runs, the request log (hashes, not prompts), citations, the
- cache, quotas and usage, and the audit log, in AI's own SQLite
+ cache, quotas and usage, and the audit log, in AI's own PostgreSQL database (`ov_ai`, ADR-035; schema in
+ [migrations/](migrations/))
- people's own provider keys (`ai.credential.manage`), encrypted at rest, and per-streamer
- people's own provider keys (`ai.credential.manage`), encrypted at rest, and per-streamer
attribution quotas (`ai.quota.attribution.manage`)
attribution quotas (`ai.quota.attribution.manage`)
- the `ai.run.queued|succeeded|failed|cached` events and the `ai.preferences` / `ai.usage_summary`
- the `ai.run.queued|succeeded|failed|cached` events and the `ai.preferences` / `ai.usage_summary`
@@ -36,13 +37,14 @@ owns publication truth: Wiki, Blog, News, Live, … decide what to publish.
@@ -36,13 +37,14 @@ owns publication truth: Wiki, Blog, News, Live, … decide what to publish.
## Depends on
## Depends on
+- PostgreSQL 18 (OpenVibe.Host `roles/data/`, ADR-035): every read and write is async through `openvibe-sdk/db`
- OpenVibe.Network (JWKS, service tokens, SSO for the operator console, user modules)
- OpenVibe.Network (JWKS, service tokens, SSO for the operator console, user modules)
- OpenVibe.Events (the `ai.run.*` outbox relay; off without `EVENTS_URL`)
- OpenVibe.Events (the `ai.run.*` outbox relay; off without `EVENTS_URL`)
- OpenVibe.Media (recordings `media.analyze` reads, over allow-listed https URLs or signed URLs)
- OpenVibe.Media (recordings `media.analyze` reads, over allow-listed https URLs or signed URLs)
- provider APIs configured by the operator (OpenAI-compatible, Anthropic, the HTTP seam), whisper.cpp
- provider APIs configured by the operator (OpenAI-compatible, Anthropic, the HTTP seam), whisper.cpp
and the local model server `openvibe-llm.service` on this host
and the local model server `openvibe-llm.service` on this host
-- `openvibe-contracts` v0.75.0, `openvibe-sdk` v0.11.0 (service tokens, events outbox),
- `openvibe-shared` v1.25.0, pinned by release tarball
+- `openvibe-contracts` v0.76.0, `openvibe-sdk` v0.20.3 (service tokens, events outbox),
+ `openvibe-shared` v1.27.0, pinned by release tarball
## Capabilities
## Capabilities
@@ -300,9 +302,9 @@ Reporting a vulnerability: [SECURITY.md](SECURITY.md). The rules the code keeps:
@@ -300,9 +302,9 @@ Reporting a vulnerability: [SECURITY.md](SECURITY.md). The rules the code keeps:
`docs/live-patch.diff` added `AI_SERVICE=remote` to Live (deployed as Live `fa22de5`; production runs
`docs/live-patch.diff` added `AI_SERVICE=remote` to Live (deployed as Live `fa22de5`; production runs
with `AI_SERVICE=remote` since 2026-09-23): Live's shared-key AI calls become runs here, authenticated
with `AI_SERVICE=remote` since 2026-09-23): Live's shared-key AI calls become runs here, authenticated
with `serviceHeaders('openvibe.ai')` from Live's `server/net/network-principal.js`. Without the flag
with `serviceHeaders('openvibe.ai')` from Live's `server/net/network-principal.js`. Without the flag
-Live's behaviour is unchanged. `scripts/import-from-live.js`
-moves the AI records that belong here (`--dry-run` first); it ran on production on 2026-09-23 at
-18:29 UTC (79,657 ledger rows, 11 holds recorded with their reasons).
+Live's behaviour is unchanged. The one-time `scripts/import-from-live.js` moved the AI records that belong here;
+it ran on production on 2026-09-23 at 18:29 UTC (79,657 ledger rows, 11 holds recorded with their reasons, kept in
+`import_ledger` and `import_holds`) and was retired with the move to PostgreSQL (it is in git history).
Network's footer copy calls the `network.site_copy` workflow (Network `422f8e9`), but the deployed Network still
Network's footer copy calls the `network.site_copy` workflow (Network `422f8e9`), but the deployed Network still
falls back to Live's `/internal/ai/site-copy`, and Live still serves that route. The full plan, holds
falls back to Live's `/internal/ai/site-copy`, and Live still serves that route. The full plan, holds
and rollback are in `docs/migration.md`.
and rollback are in `docs/migration.md`.
@@ -311,14 +313,17 @@ and rollback are in `docs/migration.md`.
@@ -311,14 +313,17 @@ and rollback are in `docs/migration.md`.
Production deploys with `sudo ovhost deploy ai` on the host (strategy `git-checkout`: fetch,
Production deploys with `sudo ovhost deploy ai` on the host (strategy `git-checkout`: fetch,
fast-forward `/opt/openvibe.ai`, install on a lockfile change, restart, wait for `/api/ready`).
fast-forward `/opt/openvibe.ai`, install on a lockfile change, restart, wait for `/api/ready`).
-The unit is `openvibe-ai.service` on `127.0.0.1:4700`, the env file `/etc/openvibe/ai.env`. The local model
+The unit is `openvibe-ai.service` on `127.0.0.1:4700`, the env file `/etc/openvibe/ai.env`. The database is
+`ov_ai` on the host's data role (`sudo /opt/openvibe.host/roles/data/add-service.sh ai` writes its settings); the
+release migrates it at boot. The one-time move from SQLite is `scripts/migrate-to-postgres.js` (openvibe-sdk
+`runSqliteMigration`, with a `--pglite` rehearsal mode), run while the service is stopped; the old
+`/var/lib/openvibe-ai/ai.db` stays read-only for 7 days as the rollback. The local model
runs as `openvibe-llm.service` on 127.0.0.1:8090 ([Local model](#local-model)).
runs as `openvibe-llm.service` on 127.0.0.1:8090 ([Local model](#local-model)).
Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the
Rollback: ovhost puts the previous sha back by itself when `/api/ready` does not answer 2xx after the
-restart; afterwards `sudo ovhost rollback ai --to <sha>`. Nothing blocks a rollback: the schema
-code only adds tables and columns.
+restart; afterwards `sudo ovhost rollback ai --to <sha>`. Migrations only add tables and columns.
Deployed: `/opt/openvibe.ai`, env `/etc/openvibe/ai.env` (0600), unit `deploy/systemd/openvibe-ai.service`
Deployed: `/opt/openvibe.ai`, env `/etc/openvibe/ai.env` (0600), unit `deploy/systemd/openvibe-ai.service`
-(`StateDirectory=openvibe-ai`, database `/var/lib/openvibe-ai/ai.db`), principal `ai`. The nginx vhost
+(`StateDirectory=openvibe-ai`; database `ov_ai` on the host's data role, the SQLite file `/var/lib/openvibe-ai/ai.db` before the switch), principal `ai`. The nginx vhost
`deploy/nginx/ai.openvibe.network.conf` (health/ready, `/`, `/robots.txt`, the operator console
`deploy/nginx/ai.openvibe.network.conf` (health/ready, `/`, `/robots.txt`, the operator console
`/console` and its sign-in `/auth/` public; the API is host-local) is not installed:
`/console` and its sign-in `/auth/` public; the API is host-local) is not installed:
`ai.openvibe.network` still serves the Sites placeholder. The console also needs
`ai.openvibe.network` still serves the Sites placeholder. The console also needs
@@ -7,10 +7,25 @@
@@ -7,10 +7,25 @@
"runtime": "node 22, express, better-sqlite3",
"runtime": "node 22, express, better-sqlite3",
"port": 4700,
"port": 4700,
"domain": "ai.openvibe.network still serves the OpenVibe.Sites placeholder; the service runs on openvibe-ovh (/opt/openvibe.ai, unit openvibe-ai) on 127.0.0.1:4700, loopback only, used by OpenVibe.Live and OpenVibe.Network",
"domain": "ai.openvibe.network still serves the OpenVibe.Sites placeholder; the service runs on openvibe-ovh (/opt/openvibe.ai, unit openvibe-ai) on 127.0.0.1:4700, loopback only, used by OpenVibe.Live and OpenVibe.Network",
- "contracts": "openvibe-contracts v0.75.0",
- "capabilities": ["ai.run.create", "ai.run.read", "ai.workflow.manage", "ai.provider.manage", "ai.usage.read", "ai.credential.manage", "ai.quota.attribution.manage"],
+ "contracts": "openvibe-contracts v0.76.0",
+ "capabilities": [
+ "ai.run.create",
+ "ai.run.read",
+ "ai.workflow.manage",
+ "ai.provider.manage",
+ "ai.usage.read",
+ "ai.credential.manage",
+ "ai.quota.attribution.manage"
+ ],
"capabilitiesRegistered": true,
"capabilitiesRegistered": true,
- "providers": ["stub", "openai-compatible", "anthropic", "http-seam", "whisper.cpp", "local llama.cpp server (openvibe-llm.service)"],
+ "providers": [
+ "stub",
+ "openai-compatible",
+ "anthropic",
+ "http-seam",
+ "whisper.cpp",
+ "local llama.cpp server (openvibe-llm.service)"
+ ],
"livePatch": "applied: Live fa22de5 runs AI_SERVICE=remote in production since 2026-09-23 ~15:34 UTC (309 runs by 18:30 UTC across 13 live.* workflows and 1 network.site_copy, 0 failed). As of 2026-09-23, not moved yet: VOD captions/whisper and streamers' own provider keys stayed in Live and most Live features were passthrough prompts built by Live (since 2026-09-27 AI has live.media.transcribe, stored provider keys and versioned templates for most live.* workflows); the deployed Network still falls back to Live's /internal/ai/site-copy; scripts/import-from-live.js ran on production at 2026-09-23 18:29 UTC (79,657 ledger rows: 79,647 live.ai_usage, 2 channel_ai_config, 8 site_settings; 11 holds: provider settings and the key stay env references, 1 streamer BYO key stays in Live, 3,021 translations are unrepresentable and stay in Live); every route has fallbacks=[]; 0 citation rows; no /metrics",
"livePatch": "applied: Live fa22de5 runs AI_SERVICE=remote in production since 2026-09-23 ~15:34 UTC (309 runs by 18:30 UTC across 13 live.* workflows and 1 network.site_copy, 0 failed). As of 2026-09-23, not moved yet: VOD captions/whisper and streamers' own provider keys stayed in Live and most Live features were passthrough prompts built by Live (since 2026-09-27 AI has live.media.transcribe, stored provider keys and versioned templates for most live.* workflows); the deployed Network still falls back to Live's /internal/ai/site-copy; scripts/import-from-live.js ran on production at 2026-09-23 18:29 UTC (79,657 ledger rows: 79,647 live.ai_usage, 2 channel_ai_config, 8 site_settings; 11 holds: provider settings and the key stay env references, 1 streamer BYO key stays in Live, 3,021 translations are unrepresentable and stay in Live); every route has fallbacks=[]; 0 citation rows; no /metrics",
"deployed": true,
"deployed": true,
"updated": "2026-09-28",
"updated": "2026-09-28",
@@ -2,8 +2,8 @@
@@ -2,8 +2,8 @@
Description=OpenVibe.AI — providers, routing, versioned templates and workflows, runs, citations, cache, quotas and audit
Description=OpenVibe.AI — providers, routing, versioned templates and workflows, runs, citations, cache, quotas and audit
# Network (openvibe-network) signs the service tokens callers present. The AI service starts without
# Network (openvibe-network) signs the service tokens callers present. The AI service starts without
# it and keeps retrying the JWKS fetch; /api/ready stays 503 until the key has loaded.
# it and keeps retrying the JWKS fetch; /api/ready stays 503 until the key has loaded.
-After=network.target
-Wants=openvibe-network.service
+After=network.target postgresql.service pgbouncer.service valkey-server.service
+Wants=openvibe-network.service postgresql.service pgbouncer.service valkey-server.service
[Service]
[Service]
Type=simple
Type=simple
new file mode 100644
new file mode 100644
@@ -0,0 +1,336 @@
@@ -0,0 +1,336 @@
+-- phase: expand
+-- OpenVibe.AI on PostgreSQL (ADR-035, roadmap WS-X2): the tables as they were on SQLite (converted by openvibe-sdk
+-- tools/asyncify/sqlite-schema-to-pg: text COLLATE "C" compares like SQLite, integers are bigint, identities keep their ids),
+-- then the openvibe-publishing stores and the openvibe-sdk inbox and outbox. Generated once on 2026-09-28; never edited after it runs.
+
+CREATE TABLE providers (
+ key text COLLATE "C" PRIMARY KEY,
+ display_name text COLLATE "C" NOT NULL,
+ kind text COLLATE "C" NOT NULL, -- stub | openai | anthropic | http | whisper
+ status text COLLATE "C" NOT NULL DEFAULT 'active', -- active | disabled
+ base_url text COLLATE "C",
+ auth_mode text COLLATE "C" NOT NULL DEFAULT 'none', -- none | bearer | x-api-key
+ secret_ref text COLLATE "C", -- 'env:NAME' — never a value
+ default_model text COLLATE "C",
+ capabilities text COLLATE "C" NOT NULL DEFAULT '[]', -- features supports() answers true for
+ timeout_ms bigint NOT NULL DEFAULT 30000,
+ priority bigint NOT NULL DEFAULT 100,
+ metadata text COLLATE "C" NOT NULL DEFAULT '{}',
+ origin text COLLATE "C" NOT NULL DEFAULT 'seed', -- seed (from env at boot) | admin | import
+ created_at text COLLATE "C" NOT NULL,
+ updated_at text COLLATE "C" NOT NULL
+);
+CREATE TABLE provider_health (
+ provider_key text COLLATE "C" PRIMARY KEY,
+ state text COLLATE "C" NOT NULL DEFAULT 'closed', -- closed (healthy) | open (skipped) | half_open (one probe)
+ consecutive_failures bigint NOT NULL DEFAULT 0,
+ opened_at bigint,
+ last_error text COLLATE "C",
+ last_success_at bigint,
+ last_failure_at bigint
+);
+CREATE TABLE models (
+ provider_key text COLLATE "C" NOT NULL,
+ model_key text COLLATE "C" NOT NULL,
+ display_name text COLLATE "C",
+ type text COLLATE "C" NOT NULL DEFAULT 'chat', -- chat | vision | embedding | stt
+ status text COLLATE "C" NOT NULL DEFAULT 'active',
+ context_window bigint,
+ max_output bigint,
+ cost_in_per_mtok double precision,
+ cost_out_per_mtok double precision,
+ cost_cached_per_mtok double precision,
+ supports_json bigint NOT NULL DEFAULT 1,
+ supports_tools bigint NOT NULL DEFAULT 0,
+ supports_streaming bigint NOT NULL DEFAULT 0,
+ supports_vision bigint NOT NULL DEFAULT 0,
+ metadata text COLLATE "C" NOT NULL DEFAULT '{}',
+ created_at text COLLATE "C" NOT NULL,
+ updated_at text COLLATE "C" NOT NULL,
+ PRIMARY KEY (provider_key, model_key)
+);
+CREATE TABLE routes (
+ key text COLLATE "C" NOT NULL,
+ version bigint NOT NULL,
+ status text COLLATE "C" NOT NULL DEFAULT 'active', -- active | disabled
+ primary_provider text COLLATE "C" NOT NULL,
+ primary_model text COLLATE "C",
+ fallbacks text COLLATE "C" NOT NULL DEFAULT '[]', -- [{provider, model}]
+ options text COLLATE "C" NOT NULL DEFAULT '{}', -- temperature and friends
+ max_output_tokens bigint,
+ response_format text COLLATE "C" NOT NULL DEFAULT 'text', -- text | json
+ timeout_ms bigint,
+ alias_of text COLLATE "C", -- historical key kept as an explicit alias
+ created_by text COLLATE "C",
+ created_at text COLLATE "C" NOT NULL,
+ PRIMARY KEY (key, version)
+);
+CREATE TABLE templates (
+ key text COLLATE "C" NOT NULL,
+ version bigint NOT NULL,
+ name text COLLATE "C" NOT NULL,
+ description text COLLATE "C",
+ input_schema text COLLATE "C" NOT NULL DEFAULT '{}',
+ output_schema text COLLATE "C" NOT NULL DEFAULT '{}',
+ system_prompt text COLLATE "C" NOT NULL DEFAULT '',
+ user_prompt text COLLATE "C" NOT NULL DEFAULT '',
+ default_route text COLLATE "C",
+ owner text COLLATE "C" NOT NULL DEFAULT 'ai',
+ visibility text COLLATE "C" NOT NULL DEFAULT 'internal', -- public | first-party | internal
+ status text COLLATE "C" NOT NULL DEFAULT 'active', -- draft | active | deprecated | archived
+ metadata text COLLATE "C" NOT NULL DEFAULT '{}',
+ created_by text COLLATE "C",
+ created_at text COLLATE "C" NOT NULL,
+ PRIMARY KEY (key, version)
+);
+CREATE TABLE workflows (
+ key text COLLATE "C" NOT NULL,
+ version bigint NOT NULL,
+ name text COLLATE "C" NOT NULL,
+ description text COLLATE "C",
+ namespace text COLLATE "C" NOT NULL, -- owning product: live, network, wiki, ...
+ input_schema text COLLATE "C" NOT NULL,
+ output_schema text COLLATE "C" NOT NULL,
+ steps text COLLATE "C" NOT NULL, -- [{kind, template?, operation?, ...}]
+ default_route text COLLATE "C",
+ cache_mode text COLLATE "C" NOT NULL DEFAULT 'private', -- none | private | service
+ cache_ttl_sec bigint,
+ status text COLLATE "C" NOT NULL DEFAULT 'active', -- draft | active | deprecated | archived
+ metadata text COLLATE "C" NOT NULL DEFAULT '{}',
+ created_by text COLLATE "C",
+ created_at text COLLATE "C" NOT NULL,
+ PRIMARY KEY (key, version)
+);
+CREATE TABLE runs (
+ id text COLLATE "C" PRIMARY KEY, -- run_<ULID>
+ workflow_key text COLLATE "C" NOT NULL,
+ workflow_version bigint NOT NULL,
+ template_key text COLLATE "C",
+ template_version bigint,
+ route_key text COLLATE "C",
+ route_version bigint,
+ status text COLLATE "C" NOT NULL, -- queued | running | succeeded | failed | cancelled | cached
+ requester_type text COLLATE "C" NOT NULL, -- service | app | mod (the token principal)
+ requester_id text COLLATE "C" NOT NULL,
+ on_behalf_of text COLLATE "C", -- SubjectRef JSON (optional)
+ attribution text COLLATE "C", -- EntityRef JSON the spend is attributed to (optional)
+ source_service text COLLATE "C",
+ target text COLLATE "C", -- EntityRef JSON (optional)
+ target_key text COLLATE "C", -- service:type:id for lookups
+ input text COLLATE "C" NOT NULL,
+ input_hash text COLLATE "C" NOT NULL,
+ output text COLLATE "C",
+ citations_count bigint NOT NULL DEFAULT 0,
+ error_code text COLLATE "C",
+ error_detail text COLLATE "C",
+ synthetic bigint NOT NULL DEFAULT 0, -- produced by the stub provider
+ provider_key text COLLATE "C",
+ model_key text COLLATE "C",
+ fallback_used bigint NOT NULL DEFAULT 0,
+ attempts bigint NOT NULL DEFAULT 0,
+ tokens_in bigint NOT NULL DEFAULT 0,
+ tokens_out bigint NOT NULL DEFAULT 0,
+ cost_usd double precision NOT NULL DEFAULT 0,
+ cache_key text COLLATE "C",
+ cached_from text COLLATE "C", -- run id whose output a cached run reused
+ retry_of text COLLATE "C",
+ idempotency_key text COLLATE "C",
+ trace_id text COLLATE "C",
+ request_id text COLLATE "C",
+ options text COLLATE "C" NOT NULL DEFAULT '{}',
+ created_at text COLLATE "C" NOT NULL,
+ started_at text COLLATE "C",
+ finished_at text COLLATE "C"
+);
+CREATE UNIQUE INDEX idx_runs_idem ON runs(requester_type, requester_id, idempotency_key) WHERE idempotency_key IS NOT NULL;
+CREATE INDEX idx_runs_status ON runs(status, created_at);
+CREATE INDEX idx_runs_workflow ON runs(workflow_key, created_at);
+CREATE INDEX idx_runs_target ON runs(target_key, created_at);
+CREATE INDEX idx_runs_requester ON runs(requester_type, requester_id, created_at);
+CREATE TABLE requests (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ run_id text COLLATE "C",
+ seq bigint NOT NULL DEFAULT 0,
+ operation text COLLATE "C" NOT NULL, -- chat | generate | summarize | classify | extract | enrich | embed | transcribe
+ provider_key text COLLATE "C" NOT NULL,
+ model_key text COLLATE "C",
+ route_key text COLLATE "C",
+ route_version bigint,
+ status text COLLATE "C" NOT NULL, -- ok | error | timeout | skipped | cancelled
+ skip_reason text COLLATE "C", -- disabled | circuit_open | unsupported | no_credentials
+ fallback bigint NOT NULL DEFAULT 0, -- 1 when this was not the route's primary
+ prompt_hash text COLLATE "C",
+ input_hash text COLLATE "C",
+ output_hash text COLLATE "C",
+ tokens_in bigint NOT NULL DEFAULT 0,
+ tokens_out bigint NOT NULL DEFAULT 0,
+ tokens_cached bigint NOT NULL DEFAULT 0,
+ tokens_estimated bigint NOT NULL DEFAULT 0,
+ cost_usd double precision NOT NULL DEFAULT 0,
+ latency_ms bigint,
+ error text COLLATE "C",
+ debug_prompt text COLLATE "C", -- only with AI_DEBUG_RAW_LOG and a caller opt-in
+ debug_response text COLLATE "C",
+ created_at text COLLATE "C" NOT NULL
+);
+CREATE INDEX idx_requests_run ON requests(run_id, seq);
+CREATE INDEX idx_requests_provider ON requests(provider_key, created_at);
+CREATE TABLE citations (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ run_id text COLLATE "C" NOT NULL,
+ ordinal bigint NOT NULL DEFAULT 0,
+ source_type text COLLATE "C" NOT NULL,
+ source_id text COLLATE "C",
+ url text COLLATE "C",
+ title text COLLATE "C",
+ author text COLLATE "C",
+ published_at text COLLATE "C",
+ retrieved_at text COLLATE "C",
+ snippet text COLLATE "C",
+ content_hash text COLLATE "C",
+ trust text COLLATE "C" NOT NULL DEFAULT '{}',
+ provenance text COLLATE "C" NOT NULL DEFAULT '{}',
+ attached_by text COLLATE "C", -- workflow | <principal sub>
+ created_at text COLLATE "C" NOT NULL
+);
+CREATE INDEX idx_citations_run ON citations(run_id, ordinal);
+CREATE TABLE cache_entries (
+ cache_key text COLLATE "C" PRIMARY KEY, -- sha256 over everything below
+ scope text COLLATE "C" NOT NULL, -- who may read it (see server/cache.js)
+ workflow_key text COLLATE "C" NOT NULL,
+ workflow_version bigint NOT NULL,
+ template_version bigint,
+ route_key text COLLATE "C",
+ route_version bigint,
+ model_key text COLLATE "C",
+ input_hash text COLLATE "C" NOT NULL,
+ privacy text COLLATE "C" NOT NULL, -- private | service
+ output text COLLATE "C" NOT NULL,
+ run_id text COLLATE "C" NOT NULL,
+ hits bigint NOT NULL DEFAULT 0,
+ created_at text COLLATE "C" NOT NULL,
+ expires_at bigint NOT NULL
+);
+CREATE INDEX idx_cache_expiry ON cache_entries(expires_at);
+CREATE INDEX idx_cache_workflow ON cache_entries(workflow_key);
+CREATE TABLE quotas (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ scope_type text COLLATE "C" NOT NULL, -- global | service | actor | attribution
+ scope_id text COLLATE "C" NOT NULL, -- '*' | live | user:usr_… | live:user:42
+ "window" text COLLATE "C" NOT NULL, -- minute | hour | day
+ max_requests bigint,
+ max_tokens bigint,
+ max_cost_usd double precision,
+ workflow_prefix text COLLATE "C", -- optional: only runs of workflows with this prefix
+ status text COLLATE "C" NOT NULL DEFAULT 'active',
+ origin text COLLATE "C" NOT NULL DEFAULT 'admin', -- seed | admin | import
+ created_at text COLLATE "C" NOT NULL,
+ updated_at text COLLATE "C" NOT NULL
+);
+CREATE UNIQUE INDEX idx_quotas_unique ON quotas(scope_type, scope_id, "window", COALESCE(workflow_prefix, ''));
+CREATE TABLE usage_counters (
+ scope_type text COLLATE "C" NOT NULL,
+ scope_id text COLLATE "C" NOT NULL,
+ "window" text COLLATE "C" NOT NULL,
+ window_start bigint NOT NULL, -- epoch seconds, UTC
+ workflow_prefix text COLLATE "C" NOT NULL DEFAULT '',
+ requests bigint NOT NULL DEFAULT 0,
+ tokens bigint NOT NULL DEFAULT 0,
+ cost_usd double precision NOT NULL DEFAULT 0,
+ PRIMARY KEY (scope_type, scope_id, "window", window_start, workflow_prefix)
+);
+CREATE TABLE usage_daily (
+ day text COLLATE "C" NOT NULL, -- YYYY-MM-DD (UTC)
+ requester text COLLATE "C" NOT NULL, -- service:live
+ attribution text COLLATE "C" NOT NULL DEFAULT '', -- live:user:42
+ workflow_key text COLLATE "C" NOT NULL,
+ provider_key text COLLATE "C" NOT NULL,
+ model_key text COLLATE "C" NOT NULL DEFAULT '',
+ requests bigint NOT NULL DEFAULT 0,
+ tokens_in bigint NOT NULL DEFAULT 0,
+ tokens_out bigint NOT NULL DEFAULT 0,
+ tokens_cached bigint NOT NULL DEFAULT 0,
+ cost_usd double precision NOT NULL DEFAULT 0,
+ PRIMARY KEY (day, requester, attribution, workflow_key, provider_key, model_key)
+);
+CREATE TABLE audit_log (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ at text COLLATE "C" NOT NULL,
+ actor text COLLATE "C" NOT NULL, -- principal sub (svc:live) or 'system'
+ action text COLLATE "C" NOT NULL, -- run.create | template.version | provider.update | ...
+ target_type text COLLATE "C",
+ target_id text COLLATE "C",
+ trace_id text COLLATE "C",
+ metadata text COLLATE "C" NOT NULL DEFAULT '{}'
+);
+CREATE INDEX idx_audit_target ON audit_log(target_type, target_id, id);
+CREATE INDEX idx_audit_action ON audit_log(action, id);
+CREATE TABLE console_sessions (
+ id_hash text COLLATE "C" PRIMARY KEY, -- sha256 of the session cookie; the cookie value is never stored
+ subject text COLLATE "C" NOT NULL, -- the Network person (usr_…)
+ username text COLLATE "C",
+ staff text COLLATE "C" NOT NULL DEFAULT '{}', -- the sign-in token's staff claims (role, is_owner, staff_caps, staff_map)
+ csrf text COLLATE "C" NOT NULL,
+ created_at text COLLATE "C" NOT NULL,
+ expires_at text COLLATE "C" NOT NULL,
+ revoked_at text COLLATE "C",
+ ip_hash text COLLATE "C" -- keyed hash of the client address
+);
+CREATE TABLE import_ledger (
+ source text COLLATE "C" NOT NULL, -- live.ai_usage, live.translations, ...
+ source_key text COLLATE "C" NOT NULL,
+ target text COLLATE "C" NOT NULL, -- where it went (table:key)
+ imported_at text COLLATE "C" NOT NULL,
+ PRIMARY KEY (source, source_key)
+);
+CREATE TABLE import_holds (
+ source text COLLATE "C" NOT NULL,
+ source_key text COLLATE "C" NOT NULL,
+ reason text COLLATE "C" NOT NULL,
+ detail text COLLATE "C",
+ held_at text COLLATE "C" NOT NULL,
+ PRIMARY KEY (source, source_key)
+);
+
+-- openvibe-sdk/events PostgreSQL outbox
+CREATE TABLE IF NOT EXISTS event_outbox (
+ id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+ event_id text NOT NULL UNIQUE,
+ envelope jsonb NOT NULL,
+ traceparent text,
+ created_at bigint NOT NULL,
+ attempts integer NOT NULL DEFAULT 0,
+ next_attempt_at bigint NOT NULL DEFAULT 0,
+ sent_at bigint,
+ seq bigint,
+ rejected_at bigint,
+ last_error text
+);
+CREATE INDEX IF NOT EXISTS event_outbox_due ON event_outbox (next_attempt_at, id) WHERE sent_at IS NULL AND rejected_at IS NULL;
+CREATE INDEX IF NOT EXISTS event_outbox_sent ON event_outbox (sent_at) WHERE sent_at IS NOT NULL;
+
+-- Added after the first SQLite schema: a run's grounding ({ cited, gaps }, WS-O task 3).
+ALTER TABLE runs ADD COLUMN grounding text COLLATE "C";
+
+-- server/user-modules.js: the last preferences hash pushed to Network per person (was created on first use).
+CREATE TABLE ai_module_pushes (
+ subject_id text COLLATE "C" PRIMARY KEY,
+ hash text COLLATE "C" NOT NULL,
+ pushed_at text COLLATE "C" NOT NULL
+);
+
+-- server/credentials.js: a person's own provider keys (WS-O task 2), encrypted (was created on first use).
+CREATE TABLE subject_credentials (
+ owner text COLLATE "C" NOT NULL,
+ subject text COLLATE "C" NOT NULL,
+ provider text COLLATE "C" NOT NULL CHECK (provider IN ('openai', 'anthropic')),
+ base_url text COLLATE "C",
+ key_enc text COLLATE "C" NOT NULL,
+ key_hint text COLLATE "C" NOT NULL,
+ models text COLLATE "C" NOT NULL DEFAULT '{}',
+ budget_usd_day double precision,
+ created_at text COLLATE "C" NOT NULL,
+ updated_at text COLLATE "C" NOT NULL,
+ last_used_at text COLLATE "C",
+ PRIMARY KEY (owner, subject)
+);
@@ -13,9 +13,13 @@
@@ -13,9 +13,13 @@
"better-sqlite3": "^11.10.0",
"better-sqlite3": "^11.10.0",
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"express": "^4.21.2",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.75.0",
- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0",
- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",
+ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",
+ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",
+ "pg": "^8.23.0"
+ },
+ "devDependencies": {
+ "@electric-sql/pglite": "^0.5.8"
},
},
"engines": {
"engines": {
"node": ">=20.0.0"
"node": ">=20.0.0"
@@ -24,6 +28,13 @@
@@ -24,6 +28,13 @@
"sharp": "^0.33.5"
"sharp": "^0.33.5"
}
}
},
},
+ "node_modules/@electric-sql/pglite": {
+ "version": "0.5.8",
+ "resolved": "https://registry.npmjs.org/@electric-sql/pglite/-/pglite-0.5.8.tgz",
+ "integrity": "sha512-n9tsbUOhwx2epK1V0ZG9Ar4SHWUju04dhmzZXiSBXwBoleOvIfals33NAaWgagQVAL4Rbvx/Ptsu3P+pA09f6Q==",
+ "devOptional": true,
+ "license": "Apache-2.0"
+ },
"node_modules/@emnapi/runtime": {
"node_modules/@emnapi/runtime": {
"version": "1.11.3",
"version": "1.11.3",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.11.3.tgz",
"resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.11.3.tgz",
@@ -1395,9 +1406,9 @@
@@ -1395,9 +1406,9 @@
}
}
},
},
"node_modules/openvibe-contracts": {
"node_modules/openvibe-contracts": {
- "version": "0.75.0",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.75.0",
- "integrity": "sha512-l+VZGYvVpX2IGsEFPAFwJa8upFRBpZ0ufZxTSua72UeGSj7z2ceHxNjuua0oQ1fnsshl4hCvXHyen1eHjic/eA==",
+ "version": "0.76.0",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",
+ "integrity": "sha512-zTdx776KWrcJKm1JTnf+0+BnsvswdvXy1ldPyse1hSqhnW/SSr98sX696eRetFUhTl8qVM1xSRCykqIUbqItPw==",
"license": "AGPL-3.0-only",
"license": "AGPL-3.0-only",
"dependencies": {
"dependencies": {
"ajv": "^8.17.1",
"ajv": "^8.17.1",
@@ -1411,18 +1422,38 @@
@@ -1411,18 +1422,38 @@
}
}
},
},
"node_modules/openvibe-sdk": {
"node_modules/openvibe-sdk": {
- "version": "0.11.0",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0",
- "integrity": "sha512-sY/0bl5bhixjxLv2ACeL/AKIE2QlnndiDqkNuA5YGo2pC7AMvPqRP05y4OAtuJ7JvWtaBZ9gETB03u07uYp3wQ==",
+ "version": "0.20.3",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",
+ "integrity": "sha512-+g+ySveItzOBnUsw9aZZIJVKRrGuj0aO328W4N25T0Z8mY4fE6++ImvLnZ55LH2BRh0WYozC1KAVIShACm2KGQ==",
"license": "MIT",
"license": "MIT",
"engines": {
"engines": {
"node": ">=20"
"node": ">=20"
+ },
+ "peerDependencies": {
+ "@electric-sql/pglite": "^0.5.8",
+ "better-sqlite3": ">=11",
+ "iovalkey": "^0.4.0",
+ "pg": "^8.23.0"
+ },
+ "peerDependenciesMeta": {
+ "@electric-sql/pglite": {
+ "optional": true
+ },
+ "better-sqlite3": {
+ "optional": true
+ },
+ "iovalkey": {
+ "optional": true
+ },
+ "pg": {
+ "optional": true
+ }
}
}
},
},
"node_modules/openvibe-shared": {
"node_modules/openvibe-shared": {
- "version": "1.25.0",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0",
- "integrity": "sha512-oqZ+PvljrWi8IEbtg6bDYai17ZV2sALFx/ezMptniELSr10TgFUiGUQt+b1IWwiHv0T2Y1U2iSrkirJxMNgjSw==",
+ "version": "1.27.0",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",
+ "integrity": "sha512-/H7dyFw+cURcdjyfbpyY7mNc6F+Pcb9TwkVwZHR+9d3M62q70KDhZLo/gZi71eaYxUjD8lO8tmbXu70EAa9paQ==",
"license": "MIT",
"license": "MIT",
"dependencies": {
"dependencies": {
"jsonwebtoken": "^9.0.0"
"jsonwebtoken": "^9.0.0"
@@ -1443,6 +1474,134 @@
@@ -1443,6 +1474,134 @@
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"license": "MIT"
"license": "MIT"
},
},
+ "node_modules/pg": {
+ "version": "8.23.0",
+ "resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",
+ "integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",
+ "license": "MIT",
+ "dependencies": {
+ "pg-connection-string": "^2.14.0",
+ "pg-pool": "^3.14.0",
+ "pg-protocol": "^1.16.0",
+ "pg-types": "2.2.0",
+ "pgpass": "1.0.5"
+ },
+ "engines": {
+ "node": ">= 16.0.0"
+ },
+ "optionalDependencies": {
+ "pg-cloudflare": "^1.4.0"
+ },
+ "peerDependencies": {
+ "pg-native": ">=3.0.1"
+ },
+ "peerDependenciesMeta": {
+ "pg-native": {
+ "optional": true
+ }
+ }
+ },
+ "node_modules/pg-cloudflare": {
+ "version": "1.4.0",
+ "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",
+ "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",
+ "license": "MIT",
+ "optional": true
+ },
+ "node_modules/pg-connection-string": {
+ "version": "2.14.0",
+ "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",
+ "integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",
+ "license": "MIT"
+ },
+ "node_modules/pg-int8": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
+ "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
+ "license": "ISC",
+ "engines": {
+ "node": ">=4.0.0"
+ }
+ },
+ "node_modules/pg-pool": {
+ "version": "3.14.0",
+ "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",
+ "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",
+ "license": "MIT",
+ "peerDependencies": {
+ "pg": ">=8.0"
+ }
+ },
+ "node_modules/pg-protocol": {
+ "version": "1.16.0",
+ "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz",
+ "integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==",
+ "license": "MIT"
+ },
+ "node_modules/pg-types": {
+ "version": "2.2.0",
+ "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
+ "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
+ "license": "MIT",
+ "dependencies": {
+ "pg-int8": "1.0.1",
+ "postgres-array": "~2.0.0",
+ "postgres-bytea": "~1.0.0",
+ "postgres-date": "~1.0.4",
+ "postgres-interval": "^1.1.0"
+ },
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/pgpass": {
+ "version": "1.0.5",
+ "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
+ "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
+ "license": "MIT",
+ "dependencies": {
+ "split2": "^4.1.0"
+ }
+ },
+ "node_modules/postgres-array": {
+ "version": "2.0.0",
+ "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
+ "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=4"
+ }
+ },
+ "node_modules/postgres-bytea": {
+ "version": "1.0.1",
+ "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
+ "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/postgres-date": {
+ "version": "1.0.7",
+ "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
+ "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
+ "node_modules/postgres-interval": {
+ "version": "1.2.0",
+ "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
+ "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
+ "license": "MIT",
+ "dependencies": {
+ "xtend": "^4.0.0"
+ },
+ "engines": {
+ "node": ">=0.10.0"
+ }
+ },
"node_modules/prebuild-install": {
"node_modules/prebuild-install": {
"version": "7.1.3",
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
"resolved": "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz",
@@ -1831,6 +1990,15 @@
@@ -1831,6 +1990,15 @@
"is-arrayish": "^0.3.1"
"is-arrayish": "^0.3.1"
}
}
},
},
+ "node_modules/split2": {
+ "version": "4.2.0",
+ "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
+ "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
+ "license": "ISC",
+ "engines": {
+ "node": ">= 10.x"
+ }
+ },
"node_modules/statuses": {
"node_modules/statuses": {
"version": "2.0.2",
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
@@ -1965,6 +2133,15 @@
@@ -1965,6 +2133,15 @@
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC"
"license": "ISC"
+ },
+ "node_modules/xtend": {
+ "version": "4.0.2",
+ "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
+ "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=0.4"
+ }
}
}
}
}
}
}
@@ -8,9 +8,15 @@
@@ -8,9 +8,15 @@
"start": "node server/index.js",
"start": "node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
"test": "node test/run.js",
"test": "node test/run.js",
- "import-from-live": "node scripts/import-from-live.js"
+ "test:pg": "AI_TEST_STORE=pg node test/run.js"
},
},
- "keywords": ["openvibe", "ai", "llm", "workflows", "whisper"],
+ "keywords": [
+ "openvibe",
+ "ai",
+ "llm",
+ "workflows",
+ "whisper"
+ ],
"author": "OpenVibers",
"author": "OpenVibers",
"license": "AGPL-3.0-only",
"license": "AGPL-3.0-only",
"repository": "github:OpenVibers/OpenVibe.AI",
"repository": "github:OpenVibers/OpenVibe.AI",
@@ -19,14 +25,18 @@
@@ -19,14 +25,18 @@
"better-sqlite3": "^11.10.0",
"better-sqlite3": "^11.10.0",
"dotenv": "^16.4.7",
"dotenv": "^16.4.7",
"express": "^4.21.2",
"express": "^4.21.2",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.75.0",
- "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0",
- "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.25.0"
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.76.0",
+ "openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.20.3",
+ "openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.27.0",
+ "pg": "^8.23.0"
},
},
"optionalDependencies": {
"optionalDependencies": {
"sharp": "^0.33.5"
"sharp": "^0.33.5"
},
},
"engines": {
"engines": {
"node": ">=20.0.0"
"node": ">=20.0.0"
+ },
+ "devDependencies": {
+ "@electric-sql/pglite": "^0.5.8"
}
}
}
}
deleted file mode 100644
deleted file mode 100644
@@ -1,224 +0,0 @@
@@ -1,224 +0,0 @@
-#!/usr/bin/env node
-'use strict';
-/**
- * Import the AI records that belong to OpenVibe.AI from a snapshot of OpenVibe.Live's database.
- *
- * node scripts/import-from-live.js --live-db /path/to/live-snapshot.db [--ai-db ./data/ai.db] [--dry-run]
- *
- * Reads the Live snapshot READ-ONLY. Idempotent: every imported source row is recorded in
- * import_ledger (source, key, content hash), so a second run imports nothing new and a changed
- * setting is applied once. Rows this service cannot represent are recorded in import_holds with a
- * reason — never silently dropped, never guessed. --dry-run prints the plan and writes nothing.
- *
- * What moves (see docs/migration.md for the full table):
- * site_settings ai_model_<role> -> a new version of route live.<role> with that model (origin import)
- * site_settings ai_pricing_json -> model rows with cost metadata for provider `shared`
- * site_settings ai_max_cost_usd_per_day -> quota global/day max_cost_usd
- * site_settings ai_viewers_global_cap_usd_per_day -> quota global/day max_cost_usd, workflow_prefix live.viewers.
- * site_settings ai_provider / ai_base_url / ai_model / ai_api_key -> HELD: these are environment
- * (AI_PROVIDER, AI_BASE_URL, AI_MODEL, AI_API_KEY); the script prints which names to set, never values of secrets
- * ai_usage -> usage_daily (per day, requester service:live, attribution live:user:<owner>,
- * workflow by Live kind), plus today's quota windows
- * channel_ai_config -> attribution quota live:user:<id>/day for live.viewers.* (shared key budgets);
- * BYO keys are HELD (per-actor provider secrets are not stored here yet)
- * translations, ai_timeline_cache, ai_chatbot_configs -> HELD (one row each; reasons in the hold)
- * Everything else AI-shaped in Live (stream_memories, streamer_overviews, stream_recaps, vod/clip_ai_state,
- * chat_ai_summaries, channel_ai_bots, ai_viewer_*) is Live product state and stays in Live.
- */
-const fs = require('fs');
-const path = require('path');
-const Database = require('better-sqlite3');
-const { load } = require('../server/config');
-const { openDb } = require('../server/db');
-const { createRegistry } = require('../server/registry');
-const { createQuotas } = require('../server/quota');
-const { seed } = require('../server/workflows/seed');
-const { KIND_TO_WORKFLOW } = require('../server/workflows/live');
-const { sha256, iso } = require('../server/util');
-
-/** Live ai_usage.kind -> the workflow that now does that job. */
-const KIND_MAP = {
- ...KIND_TO_WORKFLOW,
- translate: 'live.translate', paste_image: 'live.paste.describe_image', moment_frame: 'live.paste.describe_image', paste_text: 'live.paste.summarize_text',
- stream_memory: 'live.stream.summarize', streamer_overview: 'live.streamer.overview', media_overview: 'live.media.overview', stream_recap: 'live.stream.recap',
- site_copy: 'network.site_copy',
-};
-const ROLES = ['chat', 'vision', 'director', 'summary'];
-
-function parseArgs(argv) {
- const out = { dryRun: false, liveDb: null, aiDb: null, quiet: false };
- for (let i = 0; i < argv.length; i++) {
- const a = argv[i];
- if (a === '--dry-run') out.dryRun = true;
- else if (a === '--quiet') out.quiet = true;
- else if (a === '--live-db') out.liveDb = argv[++i];
- else if (a === '--ai-db') out.aiDb = argv[++i];
- else if (a === '-h' || a === '--help') out.help = true;
- else throw new Error(`unknown argument ${a}`);
- }
- return out;
-}
-
-function hasTable(db, name) {
- return Boolean(db.prepare("SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = ?").get(name));
-}
-
-function run({ liveDb, aiDb, dryRun = false, env = process.env, log = console.log, now = Date.now() } = {}) {
- if (!liveDb || !fs.existsSync(liveDb)) throw new Error('--live-db <snapshot> is required and must exist');
- const live = new Database(liveDb, { readonly: true, fileMustExist: true });
- const config = load({ ...env, AI_DB_PATH: aiDb || env.AI_DB_PATH || './data/ai.db' });
- const aiPath = path.resolve(config.dbPath);
-
- // Dry run: never create or write the AI database; read its ledger if it already exists.
- let db = null;
- let registry = null;
- let quotas = null;
- if (!dryRun) {
- db = openDb(aiPath);
- registry = createRegistry(db, { env });
- quotas = createQuotas(db, { registry });
- seed({ registry, quotas, config, env, db });
- } else if (fs.existsSync(aiPath)) {
- db = new Database(aiPath, { readonly: true, fileMustExist: true });
- }
- const ledgerGet = (source, key) => (db && hasTable(db, 'import_ledger') ? db.prepare('SELECT target FROM import_ledger WHERE source = ? AND source_key = ?').get(source, String(key)) : null);
- const holdGet = (source, key) => (db && hasTable(db, 'import_holds') ? db.prepare('SELECT 1 FROM import_holds WHERE source = ? AND source_key = ?').get(source, String(key)) : null);
- const stats = {};
- const bump = (source, what, n = 1) => { stats[source] = stats[source] || { imported: 0, unchanged: 0, held: 0 }; stats[source][what] += n; };
- const ledger = (source, key, target) => { if (!dryRun) db.prepare('INSERT OR REPLACE INTO import_ledger (source, source_key, target, imported_at) VALUES (?, ?, ?, ?)').run(source, String(key), target, iso(now)); };
- const hold = (source, key, reason, detail = null) => {
- if (holdGet(source, key)) return bump(source, 'unchanged');
- if (!dryRun) db.prepare('INSERT OR REPLACE INTO import_holds (source, source_key, reason, detail, held_at) VALUES (?, ?, ?, ?, ?)').run(source, String(key), reason, detail, iso(now));
- bump(source, 'held');
- };
- /** Apply once per (source, key, content): unchanged content is skipped on re-runs. */
- const once = (source, key, content, apply) => {
- const h = sha256(content);
- const prev = ledgerGet(source, key);
- if (prev && prev.target.endsWith(`#${h}`)) return bump(source, 'unchanged');
- const target = dryRun ? 'dry-run' : apply();
- ledger(source, key, `${target}#${h}`);
- bump(source, 'imported');
- };
-
- // ── site_settings ──
- const settings = hasTable(live, 'site_settings') ? Object.fromEntries(live.prepare("SELECT key, value FROM site_settings WHERE key LIKE 'ai_%'").all().map(r => [r.key, r.value])) : {};
- const envAdvice = [];
- for (const [k, envName] of [['ai_provider', 'AI_PROVIDER'], ['ai_base_url', 'AI_BASE_URL'], ['ai_model', 'AI_MODEL'], ['ai_enabled', 'AI_ENABLED'], ['ai_input_cost_per_mtok', 'AI_INPUT_COST_PER_MTOK'], ['ai_output_cost_per_mtok', 'AI_OUTPUT_COST_PER_MTOK']]) {
- if (settings[k] != null && settings[k] !== '') { envAdvice.push(`${envName}=${settings[k]}`); hold('live.site_settings', k, 'environment', `set ${envName} in /etc/openvibe/ai.env`); }
- }
- if (settings.ai_api_key) {
- envAdvice.push('AI_API_KEY=<copy the value of Live\'s ai_api_key setting; not printed>');
- hold('live.site_settings', 'ai_api_key', 'secret', 'secrets are environment references (env:AI_API_KEY); set it in /etc/openvibe/ai.env');
- }
- for (const role of ROLES) {
- const m = String(settings[`ai_model_${role}`] || '').trim();
- if (!m) continue;
- once('live.site_settings', `ai_model_${role}`, m, () => {
- const cur = registry.getRoute(`live.${role}`);
- const v = registry.createRouteVersion(`live.${role}`, { primary: { provider: 'shared', model: m }, fallbacks: cur ? cur.fallbacks : [] }, { actor: 'import' });
- return `route:live.${role}@${v.version}`;
- });
- }
- if (settings.ai_pricing_json) {
- let table = null;
- try { table = JSON.parse(settings.ai_pricing_json); } catch { table = null; }
- if (!table || typeof table !== 'object') hold('live.site_settings', 'ai_pricing_json', 'unparseable', 'not JSON');
- else {
- for (const [model, p] of Object.entries(table)) {
- if (model === 'default' || !p || typeof p !== 'object') { hold('live.site_settings', `ai_pricing_json.${model}`, 'environment', 'prefix/default pricing stays in AI_PRICING_JSON'); continue; }
- once('live.site_settings', `ai_pricing_json.${model}`, p, () => {
- registry.upsertModel({ provider_key: 'shared', model_key: model, type: 'chat', cost: { in_per_mtok: Number(p.in), out_per_mtok: Number(p.out), cached_per_mtok: p.cached != null ? Number(p.cached) : null }, metadata: { imported_from: 'live.ai_pricing_json' } }, { actor: 'import' });
- return `model:shared/${model}`;
- });
- }
- }
- }
- const cap = Number(settings.ai_max_cost_usd_per_day);
- if (cap > 0) once('live.site_settings', 'ai_max_cost_usd_per_day', cap, () => `quota:${quotas.upsert({ scope_type: 'global', window: 'day', max_cost_usd: cap }, { actor: 'import', origin: 'import' }).id}`);
- const vcap = Number(settings.ai_viewers_global_cap_usd_per_day);
- if (vcap > 0) once('live.site_settings', 'ai_viewers_global_cap_usd_per_day', vcap, () => `quota:${quotas.upsert({ scope_type: 'global', window: 'day', max_cost_usd: vcap, workflow_prefix: 'live.viewers.' }, { actor: 'import', origin: 'import' }).id}`);
-
- // ── ai_usage -> usage_daily (+ today's windows) ──
- if (hasTable(live, 'ai_usage')) {
- const cols = live.prepare('PRAGMA table_info(ai_usage)').all().map(c => c.name);
- const has = (c) => cols.includes(c);
- const rows = live.prepare(`SELECT id, kind, model, input_tokens, output_tokens, cost_usd, created_at,
- ${has('owner_user_id') ? 'owner_user_id' : 'NULL AS owner_user_id'}, ${has('cached_tokens') ? 'cached_tokens' : '0 AS cached_tokens'}, ${has('provider') ? 'provider' : "'shared' AS provider"}
- FROM ai_usage ORDER BY id`).all();
- const today = iso(now).slice(0, 10);
- const dayStart = Math.floor(now / 86400000) * 86400;
- const insDaily = db && !dryRun ? db.prepare(`INSERT INTO usage_daily (day, requester, attribution, workflow_key, provider_key, model_key, requests, tokens_in, tokens_out, tokens_cached, cost_usd)
- VALUES (?, 'service:live', ?, ?, ?, ?, 1, ?, ?, ?, ?)
- ON CONFLICT(day, requester, attribution, workflow_key, provider_key, model_key) DO UPDATE SET requests = requests + 1,
- tokens_in = tokens_in + excluded.tokens_in, tokens_out = tokens_out + excluded.tokens_out, tokens_cached = tokens_cached + excluded.tokens_cached, cost_usd = cost_usd + excluded.cost_usd`) : null;
- const insCounter = db && !dryRun ? db.prepare(`INSERT INTO usage_counters (scope_type, scope_id, window, window_start, workflow_prefix, requests, tokens, cost_usd) VALUES (?, ?, 'day', ?, '', 1, ?, ?)
- ON CONFLICT(scope_type, scope_id, window, window_start, workflow_prefix) DO UPDATE SET requests = requests + 1, tokens = tokens + excluded.tokens, cost_usd = cost_usd + excluded.cost_usd`) : null;
- const tx = (fn) => (db && !dryRun ? db.transaction(fn)() : fn());
- tx(() => {
- for (const r of rows) {
- if (ledgerGet('live.ai_usage', r.id)) { bump('live.ai_usage', 'unchanged'); continue; }
- const day = String(r.created_at || '').slice(0, 10);
- if (!/^\d{4}-\d{2}-\d{2}$/.test(day)) { hold('live.ai_usage', r.id, 'unrepresentable', 'no valid created_at'); continue; }
- const wf = KIND_MAP[r.kind] || 'live.complete';
- const attribution = r.owner_user_id ? `live:user:${r.owner_user_id}` : '';
- const provider = r.provider === 'byo' ? 'live-byo' : 'shared';
- const tokens = (Number(r.input_tokens) || 0) + (Number(r.output_tokens) || 0);
- if (!dryRun) {
- insDaily.run(day, attribution, wf, provider, r.model || '', Number(r.input_tokens) || 0, Number(r.output_tokens) || 0, Number(r.cached_tokens) || 0, Number(r.cost_usd) || 0);
- if (day === today && provider === 'shared') {
- insCounter.run('global', '*', dayStart, tokens, Number(r.cost_usd) || 0);
- insCounter.run('service', 'live', dayStart, tokens, Number(r.cost_usd) || 0);
- if (attribution) insCounter.run('attribution', attribution, dayStart, tokens, Number(r.cost_usd) || 0);
- }
- }
- ledger('live.ai_usage', r.id, `usage_daily:${day}/${wf}#`);
- bump('live.ai_usage', 'imported');
- }
- });
- }
-
- // ── channel_ai_config -> per-streamer AI-viewer budgets; BYO keys held ──
- if (hasTable(live, 'channel_ai_config')) {
- for (const c of live.prepare('SELECT user_id, use_shared_key, daily_budget_cents, byo_key, byo_base_url FROM channel_ai_config').all()) {
- if (Number(c.use_shared_key) === 1 || c.use_shared_key == null) {
- const usd = (Number(c.daily_budget_cents) || 0) / 100;
- if (usd <= 0) { bump('live.channel_ai_config', 'unchanged'); continue; }
- once('live.channel_ai_config', c.user_id, usd, () => `quota:${quotas.upsert({ scope_type: 'attribution', scope_id: `live:user:${c.user_id}`, window: 'day', max_cost_usd: usd, workflow_prefix: 'live.viewers.' }, { actor: 'import', origin: 'import' }).id}`);
- } else {
- let host = '';
- try { host = c.byo_base_url ? new URL(c.byo_base_url).host : ''; } catch { host = ''; }
- hold('live.channel_ai_config', c.user_id, 'byo_provider', `streamer ${c.user_id} uses their own ${c.byo_key ? 'key' : 'self-hosted endpoint'}${host ? ` (${host})` : ''}; BYO providers keep running in Live (llm.js provider override) until per-actor provider secrets exist here`);
- }
- }
- }
-
- // ── Whole-table holds ──
- const tableHold = (table, reason, detail) => {
- if (!hasTable(live, table)) return;
- const n = live.prepare(`SELECT COUNT(*) AS n FROM ${table}`).get().n;
- hold(`live.${table}`, '*', reason, `${n} row(s): ${detail}`);
- };
- tableHold('translations', 'unrepresentable', 'keyed by sha1(from|to|text) without the source text, so they cannot become input-hashed cache entries; Live keeps them as its local read-through cache and new translations are cached here per service');
- tableHold('ai_timeline_cache', 'stays_in_live', 'an assembled Live read model (no model output); Live keeps serving it');
- tableHold('ai_chatbot_configs', 'superseded', 'Live already migrated these rows into channel_ai_config');
-
- live.close();
- if (db) db.close();
- const report = { dry_run: dryRun, ai_db: aiPath, sources: stats, environment: envAdvice };
- log(JSON.stringify(report, null, 2));
- return report;
-}
-
-if (require.main === module) {
- require('dotenv').config();
- let args;
- try { args = parseArgs(process.argv.slice(2)); } catch (e) { console.error(e.message); process.exit(2); }
- if (args.help || !args.liveDb) {
- console.error('usage: node scripts/import-from-live.js --live-db <snapshot.db> [--ai-db <ai.db>] [--dry-run]');
- process.exit(args.help ? 0 : 2);
- }
- try { run({ liveDb: args.liveDb, aiDb: args.aiDb, dryRun: args.dryRun }); } catch (e) { console.error(`[import] ${e.message}`); process.exit(1); }
-}
-
-module.exports = { run, KIND_MAP };
new file mode 100644
new file mode 100644
@@ -0,0 +1,26 @@
@@ -0,0 +1,26 @@
+#!/usr/bin/env node
+'use strict';
+/**
+ * The one-time move of AI's SQLite database (AI_DB_PATH) into its PostgreSQL schema (ADR-035; the procedure is
+ * openvibe-sdk docs/migrating-to-postgresql.md, section 6, carried out by openvibe-sdk/db runSqliteMigration).
+ *
+ * node scripts/migrate-to-postgres.js [--sqlite <file>] [--pglite] [--json]
+ *
+ * Applies migrations/ as the owner (DATABASE_DIRECT_URL), copies every table into emptied tables, verifies row counts
+ * and checksums, and exits 1 unless everything verified. The SQLite file is opened read-only. Every table keeps its
+ * name and columns.
+ */
+require('dotenv').config();
+const { runSqliteMigration } = require('openvibe-sdk/db');
+const configLib = require('../server/config');
+const { MIGRATIONS } = require('../server/db');
+
+const TABLES = {};
+
+if (require.main === module) {
+ const config = (configLib.load || configLib.loadConfig)();
+ runSqliteMigration({ service: 'ai', sqlite: config.dbPath, directUrl: config.db.directUrl, migrations: MIGRATIONS, tables: TABLES })
+ .then((code) => process.exit(code), (err) => { console.error(`migrate-to-postgres failed: ${err.message}`); process.exit(1); });
+}
+
+module.exports = { TABLES };
@@ -33,43 +33,43 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = crea
@@ -33,43 +33,43 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = crea
const defRead = auth.requireCap(CAPS.workflowManage, CAPS.runCreate, CAPS.usageRead);
const defRead = auth.requireCap(CAPS.workflowManage, CAPS.runCreate, CAPS.usageRead);
const defWrite = auth.requireCap(CAPS.workflowManage);
const defWrite = auth.requireCap(CAPS.workflowManage);
const usageRead = auth.requireCap(CAPS.usageRead);
const usageRead = auth.requireCap(CAPS.usageRead);
- const wrap = (fn) => (req, res) => { try { const out = fn(req, res); if (out !== undefined) res.json(out); } catch (err) { sendError(res, err, req.ov, log); } };
+ const wrap = (fn) => async (req, res) => { try { const out = await fn(req, res); if (out !== undefined) res.json(out); } catch (err) { sendError(res, err, req.ov, log); } };
const actor = (req) => ({ actor: req.principal.sub, trace: req.ov.traceId });
const actor = (req) => ({ actor: req.principal.sub, trace: req.ov.traceId });
const view = ops.providerView;
const view = ops.providerView;
const body = (req) => (req.body && typeof req.body === 'object' ? req.body : {});
const body = (req) => (req.body && typeof req.body === 'object' ? req.body : {});
- r.get('/api/v1/status', usageRead, wrap(() => ops.status()));
+ r.get('/api/v1/status', usageRead, wrap(async () => await ops.status()));
// Providers
// Providers
- r.get('/api/v1/providers', provRead, wrap(() => ({ providers: registry.listProviders().map(view) })));
- r.get('/api/v1/providers/:key', provRead, wrap((req) => {
- const p = registry.getProvider(req.params.key);
+ r.get('/api/v1/providers', provRead, wrap(async () => ({ providers: (await Promise.all((await registry.listProviders()).map(view))) })));
+ r.get('/api/v1/providers/:key', provRead, wrap(async (req) => {
+ const p = await registry.getProvider(req.params.key);
if (!p) throw new AiError(404, 'ai.not_found', 'no such provider');
if (!p) throw new AiError(404, 'ai.not_found', 'no such provider');
- return { provider: view(p), models: registry.listModels({ provider: p.key }), stats: pool.stats[p.key] || null };
+ return { provider: await view(p), models: await registry.listModels({ provider: p.key }), stats: pool.stats[p.key] || null };
}));
}));
- r.post('/api/v1/providers', provWrite, wrap((req, res) => {
+ r.post('/api/v1/providers', provWrite, wrap(async (req, res) => {
const b = body(req);
const b = body(req);
if ('api_key' in b || 'secret' in b) throw new AiError(422, 'ai.invalid', 'send secret_ref (env:NAME), never a secret value');
if ('api_key' in b || 'secret' in b) throw new AiError(422, 'ai.invalid', 'send secret_ref (env:NAME), never a secret value');
res.status(201);
res.status(201);
- return { provider: view(registry.upsertProvider(b, { ...actor(req), origin: 'admin' })) };
+ return { provider: await view(await registry.upsertProvider(b, { ...actor(req), origin: 'admin' })) };
}));
}));
- r.patch('/api/v1/providers/:key', provWrite, wrap((req) => {
+ r.patch('/api/v1/providers/:key', provWrite, wrap(async (req) => {
const b = body(req);
const b = body(req);
if ('api_key' in b || 'secret' in b) throw new AiError(422, 'ai.invalid', 'send secret_ref (env:NAME), never a secret value');
if ('api_key' in b || 'secret' in b) throw new AiError(422, 'ai.invalid', 'send secret_ref (env:NAME), never a secret value');
- if (!registry.getProvider(req.params.key)) throw new AiError(404, 'ai.not_found', 'no such provider');
- return { provider: view(registry.upsertProvider({ ...b, key: req.params.key }, { ...actor(req), origin: 'admin' })) };
+ if (!await registry.getProvider(req.params.key)) throw new AiError(404, 'ai.not_found', 'no such provider');
+ return { provider: await view(await registry.upsertProvider({ ...b, key: req.params.key }, { ...actor(req), origin: 'admin' })) };
}));
}));
for (const [action, status] of [['disable', 'disabled'], ['enable', 'active']]) {
for (const [action, status] of [['disable', 'disabled'], ['enable', 'active']]) {
- r.post(`/api/v1/providers/:key/${action}`, provWrite, wrap((req) => ({ provider: view(ops.setProviderStatus(req.params.key, status, actor(req))) })));
+ r.post(`/api/v1/providers/:key/${action}`, provWrite, wrap(async (req) => ({ provider: await view(await ops.setProviderStatus(req.params.key, status, actor(req))) })));
}
}
- r.post('/api/v1/providers/:key/reset', provWrite, wrap((req) => ({ health: ops.resetCircuit(req.params.key, actor(req)) })));
+ r.post('/api/v1/providers/:key/reset', provWrite, wrap(async (req) => ({ health: await ops.resetCircuit(req.params.key, actor(req)) })));
// Models
// Models
- r.get('/api/v1/models', provRead, wrap((req) => ({ models: registry.listModels({ provider: req.query.provider }) })));
- r.post('/api/v1/models', provWrite, wrap((req, res) => { res.status(201); return { model: registry.upsertModel(body(req), actor(req)) }; }));
- r.patch('/api/v1/models/:provider/:model', provWrite, wrap((req) => {
- if (!registry.getModel(req.params.provider, req.params.model)) throw new AiError(404, 'ai.not_found', 'no such model');
- return { model: registry.upsertModel({ ...body(req), provider_key: req.params.provider, model_key: req.params.model }, actor(req)) };
+ r.get('/api/v1/models', provRead, wrap(async (req) => ({ models: await registry.listModels({ provider: req.query.provider }) })));
+ r.post('/api/v1/models', provWrite, wrap(async (req, res) => { res.status(201); return { model: await registry.upsertModel(body(req), actor(req)) }; }));
+ r.patch('/api/v1/models/:provider/:model', provWrite, wrap(async (req) => {
+ if (!await registry.getModel(req.params.provider, req.params.model)) throw new AiError(404, 'ai.not_found', 'no such model');
+ return { model: await registry.upsertModel({ ...body(req), provider_key: req.params.provider, model_key: req.params.model }, actor(req)) };
}));
}));
// Versioned definitions
// Versioned definitions
@@ -79,17 +79,17 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = crea
@@ -79,17 +79,17 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = crea
workflows: { list: registry.listWorkflows, get: registry.getWorkflow, create: registry.createWorkflowVersion, one: 'workflow' },
workflows: { list: registry.listWorkflows, get: registry.getWorkflow, create: registry.createWorkflowVersion, one: 'workflow' },
};
};
for (const [plural, k] of Object.entries(kinds)) {
for (const [plural, k] of Object.entries(kinds)) {
- r.get(`/api/v1/${plural}`, defRead, wrap((req) => ({ [plural]: k.list({ history: req.query.history === '1', namespace: req.query.namespace }) })));
- r.get(`/api/v1/${plural}/:key`, defRead, wrap((req) => {
- const latest = k.get(req.params.key, req.query.version ? Number(req.query.version) : undefined);
+ r.get(`/api/v1/${plural}`, defRead, wrap(async (req) => ({ [plural]: await k.list({ history: req.query.history === '1', namespace: req.query.namespace }) })));
+ r.get(`/api/v1/${plural}/:key`, defRead, wrap(async (req) => {
+ const latest = await k.get(req.params.key, req.query.version ? Number(req.query.version) : undefined);
if (!latest) throw new AiError(404, 'ai.not_found', `no ${k.one} ${req.params.key}`);
if (!latest) throw new AiError(404, 'ai.not_found', `no ${k.one} ${req.params.key}`);
- const versions = k.list({ history: true }).filter(x => x.key === req.params.key).map(x => ({ version: x.version, status: x.status, created_by: x.created_by, created_at: x.created_at }));
+ const versions = (await k.list({ history: true })).filter(x => x.key === req.params.key).map(x => ({ version: x.version, status: x.status, created_by: x.created_by, created_at: x.created_at }));
return { [k.one]: latest, versions };
return { [k.one]: latest, versions };
}));
}));
// An edit is always a new version; the previous one stays readable.
// An edit is always a new version; the previous one stays readable.
- r.post(`/api/v1/${plural}/:key/versions`, defWrite, wrap((req, res) => {
+ r.post(`/api/v1/${plural}/:key/versions`, defWrite, wrap(async (req, res) => {
try {
try {
- const created = k.create(req.params.key, body(req), actor(req));
+ const created = await k.create(req.params.key, body(req), actor(req));
res.status(201);
res.status(201);
return { [k.one]: created };
return { [k.one]: created };
} catch (err) {
} catch (err) {
@@ -97,16 +97,16 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = crea
@@ -97,16 +97,16 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = crea
throw new AiError(422, 'ai.invalid', err.message);
throw new AiError(422, 'ai.invalid', err.message);
}
}
}));
}));
- r.post(`/api/v1/${plural}/:key/versions/:version/status`, defWrite, wrap((req) => ({
- [k.one]: registry.setStatus(k.one, req.params.key, Number(req.params.version), body(req).status, actor(req)),
+ r.post(`/api/v1/${plural}/:key/versions/:version/status`, defWrite, wrap(async (req) => ({
+ [k.one]: await registry.setStatus(k.one, req.params.key, Number(req.params.version), body(req).status, actor(req)),
})));
})));
}
}
// Quotas, usage, logs, audit, cache
// Quotas, usage, logs, audit, cache
- r.get('/api/v1/quotas', usageRead, wrap(() => ({ quotas: quotas.list(), current: quotas.counters() })));
- r.post('/api/v1/quotas', provWrite, wrap((req, res) => { res.status(201); return { quota: quotas.upsert(body(req), { ...actor(req), origin: 'admin' }) }; }));
- r.get('/api/v1/usage', usageRead, wrap((req) => ({ usage: quotas.usage({ from: req.query.from, to: req.query.to, requester: req.query.requester }) })));
- r.get('/api/v1/requests', usageRead, wrap((req) => {
+ r.get('/api/v1/quotas', usageRead, wrap(async () => ({ quotas: await quotas.list(), current: await quotas.counters() })));
+ r.post('/api/v1/quotas', provWrite, wrap(async (req, res) => { res.status(201); return { quota: await quotas.upsert(body(req), { ...actor(req), origin: 'admin' }) }; }));
+ r.get('/api/v1/usage', usageRead, wrap(async (req) => ({ usage: await quotas.usage({ from: req.query.from, to: req.query.to, requester: req.query.requester }) })));
+ r.get('/api/v1/requests', usageRead, wrap(async (req) => {
const where = [];
const where = [];
const args = [];
const args = [];
if (req.query.provider) { where.push('provider_key = ?'); args.push(req.query.provider); }
if (req.query.provider) { where.push('provider_key = ?'); args.push(req.query.provider); }
@@ -116,11 +116,11 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = crea
@@ -116,11 +116,11 @@ function adminRouter({ db, registry, pool, quotas, cache, runs, auth, ops = crea
// Raw debug fields only for provider managers who ask for them (and only exist when opted in).
// Raw debug fields only for provider managers who ask for them (and only exist when opted in).
const debug = req.query.debug === '1' && auth.principalHas(req, CAPS.providerManage);
const debug = req.query.debug === '1' && auth.principalHas(req, CAPS.providerManage);
const cols = debug ? '*' : 'id, run_id, seq, operation, provider_key, model_key, route_key, route_version, status, skip_reason, fallback, prompt_hash, input_hash, output_hash, tokens_in, tokens_out, tokens_cached, tokens_estimated, cost_usd, latency_ms, error, created_at';
const cols = debug ? '*' : 'id, run_id, seq, operation, provider_key, model_key, route_key, route_version, status, skip_reason, fallback, prompt_hash, input_hash, output_hash, tokens_in, tokens_out, tokens_cached, tokens_estimated, cost_usd, latency_ms, error, created_at';
- return { requests: db.prepare(`SELECT ${cols} FROM requests ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(500, Number(req.query.limit) || 100)) };
+ return { requests: await db.prepare(`SELECT ${cols} FROM requests ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(500, Number(req.query.limit) || 100)) };
}));
}));
- r.get('/api/v1/audit', usageRead, wrap((req) => ({ audit: registry.listAudit({ action: req.query.action, targetType: req.query.target_type, targetId: req.query.target_id, limit: Number(req.query.limit) || 100, before: req.query.before }) })));
- r.get('/api/v1/cache', usageRead, wrap(() => ({ cache: cache.stats() })));
- r.delete('/api/v1/cache', provWrite, wrap((req) => ({ removed: ops.purgeCache(req.query.workflow, actor(req)) })));
+ r.get('/api/v1/audit', usageRead, wrap(async (req) => ({ audit: await registry.listAudit({ action: req.query.action, targetType: req.query.target_type, targetId: req.query.target_id, limit: Number(req.query.limit) || 100, before: req.query.before }) })));
+ r.get('/api/v1/cache', usageRead, wrap(async () => ({ cache: await cache.stats() })));
+ r.delete('/api/v1/cache', provWrite, wrap(async (req) => ({ removed: await ops.purgeCache(req.query.workflow, actor(req)) })));
return r;
return r;
}
}
@@ -31,11 +31,11 @@ function attributionQuotasRouter({ db, quotas, auth, clock = { now: () => Date.n
@@ -31,11 +31,11 @@ function attributionQuotasRouter({ db, quotas, auth, clock = { now: () => Date.n
return m[0];
return m[0];
}
}
- function view(row) {
+ async function view(row) {
const now = Math.floor(clock.now() / 1000);
const now = Math.floor(clock.now() / 1000);
const len = WINDOW_S[row.window];
const len = WINDOW_S[row.window];
const start = Math.floor(now / len) * len;
const start = Math.floor(now / len) * len;
- const c = db.prepare('SELECT requests, cost_usd FROM usage_counters WHERE scope_type = ? AND scope_id = ? AND window = ? AND window_start = ? AND workflow_prefix = ?')
+ const c = await db.prepare('SELECT requests, cost_usd FROM usage_counters WHERE scope_type = ? AND scope_id = ? AND "window" = ? AND window_start = ? AND workflow_prefix = ?')
.get('attribution', row.scope_id, row.window, start, row.workflow_prefix || '') || { requests: 0, cost_usd: 0 };
.get('attribution', row.scope_id, row.window, start, row.workflow_prefix || '') || { requests: 0, cost_usd: 0 };
return {
return {
attribution: row.scope_id, window: row.window, workflow_prefix: row.workflow_prefix || null,
attribution: row.scope_id, window: row.window, workflow_prefix: row.workflow_prefix || null,
@@ -44,26 +44,26 @@ function attributionQuotasRouter({ db, quotas, auth, clock = { now: () => Date.n
@@ -44,26 +44,26 @@ function attributionQuotasRouter({ db, quotas, auth, clock = { now: () => Date.n
window_resets_at: iso((start + len) * 1000), updated_at: row.updated_at,
window_resets_at: iso((start + len) * 1000), updated_at: row.updated_at,
};
};
}
}
- const rows = (attr) => db.prepare("SELECT * FROM quotas WHERE scope_type = 'attribution' AND scope_id = ? AND status = 'active' ORDER BY window, workflow_prefix").all(attr);
+ const rows = async (attr) => await db.prepare("SELECT * FROM quotas WHERE scope_type = 'attribution' AND scope_id = ? AND status = 'active' ORDER BY \"window\", workflow_prefix").all(attr);
- r.put('/api/v1/attribution-quotas/:attribution', guard, (req, res) => {
+ r.put('/api/v1/attribution-quotas/:attribution', guard, async (req, res) => {
try {
try {
const attr = own(req);
const attr = own(req);
const body = req.body || {};
const body = req.body || {};
const v = validate('ai.attribution-quota-put@1', body);
const v = validate('ai.attribution-quota-put@1', body);
if (!v.valid) throw new AiError(422, 'input.invalid', 'body does not match ai.attribution-quota-put@1', { errors: v.errors });
if (!v.valid) throw new AiError(422, 'input.invalid', 'body does not match ai.attribution-quota-put@1', { errors: v.errors });
- const row = quotas.upsert({ scope_type: 'attribution', scope_id: attr, window: body.window, max_cost_usd: body.max_cost_usd, max_requests: body.max_requests, workflow_prefix: body.workflow_prefix || null, status: 'active' },
+ const row = await quotas.upsert({ scope_type: 'attribution', scope_id: attr, window: body.window, max_cost_usd: body.max_cost_usd, max_requests: body.max_requests, workflow_prefix: body.workflow_prefix || null, status: 'active' },
{ actor: req.principal.sub, origin: `service:${req.principal.subject.id}`, trace: req.ov && req.ov.traceId });
{ actor: req.principal.sub, origin: `service:${req.principal.subject.id}`, trace: req.ov && req.ov.traceId });
- res.set('Cache-Control', 'no-store').json(view(row));
+ res.set('Cache-Control', 'no-store').json(await view(row));
} catch (err) { sendError(res, err, req.ov); }
} catch (err) { sendError(res, err, req.ov); }
});
});
- r.get('/api/v1/attribution-quotas/:attribution', guard, (req, res) => {
- try { res.set('Cache-Control', 'no-store').json({ quotas: rows(own(req)).map(view) }); } catch (err) { sendError(res, err, req.ov); }
+ r.get('/api/v1/attribution-quotas/:attribution', guard, async (req, res) => {
+ try { res.set('Cache-Control', 'no-store').json({ quotas: (await Promise.all((await rows(own(req))).map(view))) }); } catch (err) { sendError(res, err, req.ov); }
});
});
- r.delete('/api/v1/attribution-quotas/:attribution', guard, (req, res) => {
+ r.delete('/api/v1/attribution-quotas/:attribution', guard, async (req, res) => {
try {
try {
const attr = own(req);
const attr = own(req);
- const n = db.prepare("UPDATE quotas SET status = 'disabled', updated_at = ? WHERE scope_type = 'attribution' AND scope_id = ? AND status = 'active'").run(iso(clock.now()), attr).changes;
+ const n = (await db.prepare("UPDATE quotas SET status = 'disabled', updated_at = ? WHERE scope_type = 'attribution' AND scope_id = ? AND status = 'active'").run(iso(clock.now()), attr)).changes;
res.status(n ? 204 : 404).end();
res.status(n ? 204 : 404).end();
} catch (err) { sendError(res, err, req.ov); }
} catch (err) { sendError(res, err, req.ov); }
});
});
@@ -46,7 +46,7 @@ function runsRouter({ runs, registry, auth, config, log = console }) {
@@ -46,7 +46,7 @@ function runsRouter({ runs, registry, auth, config, log = console }) {
async function createAndRespond(req, res, body, waitMs) {
async function createAndRespond(req, res, body, waitMs) {
checkNamespace(req, String(body.workflow || ''));
checkNamespace(req, String(body.workflow || ''));
const idem = body.idempotency_key || req.ov.idempotencyKey || undefined;
const idem = body.idempotency_key || req.ov.idempotencyKey || undefined;
- const created = runs.create({ ...body, idempotency_key: idem }, req.principal, { trace: req.ov.traceId, requestId: req.ov.requestId });
+ const created = await runs.create({ ...body, idempotency_key: idem }, req.principal, { trace: req.ov.traceId, requestId: req.ov.requestId });
const run = await runs.wait(created, waitMs);
const run = await runs.wait(created, waitMs);
const status = !created.created ? 200 : (runs.TERMINAL.has(run.status) ? 201 : 202);
const status = !created.created ? 200 : (runs.TERMINAL.has(run.status) ? 201 : 202);
if (status === 202) res.setHeader('Location', `/api/v1/runs/${run.id}`);
if (status === 202) res.setHeader('Location', `/api/v1/runs/${run.id}`);
@@ -60,13 +60,13 @@ function runsRouter({ runs, registry, auth, config, log = console }) {
@@ -60,13 +60,13 @@ function runsRouter({ runs, registry, auth, config, log = console }) {
} catch (err) { sendError(res, err, req.ov, log); }
} catch (err) { sendError(res, err, req.ov, log); }
});
});
- r.get('/api/v1/runs', read, (req, res) => {
+ r.get('/api/v1/runs', read, async (req, res) => {
const all = req.query.all === '1' && auth.principalHas(req, CAPS.usageRead);
const all = req.query.all === '1' && auth.principalHas(req, CAPS.usageRead);
- res.json({ runs: runs.list(req.principal, { status: req.query.status, workflow: req.query.workflow, target: req.query.target, limit: req.query.limit, before: req.query.before, all }) });
+ res.json({ runs: await runs.list(req.principal, { status: req.query.status, workflow: req.query.workflow, target: req.query.target, limit: req.query.limit, before: req.query.before, all }) });
});
});
- function owned(req) {
- const run = runs.get(req.params.id);
+ async function owned(req) {
+ const run = await runs.get(req.params.id);
if (!run) throw new AiError(404, 'run.not_found', 'no such run');
if (!run) throw new AiError(404, 'run.not_found', 'no such run');
const s = req.principal.subject;
const s = req.principal.subject;
const mine = run.requester.type === s.type && run.requester.id === s.id;
const mine = run.requester.type === s.type && run.requester.id === s.id;
@@ -74,33 +74,33 @@ function runsRouter({ runs, registry, auth, config, log = console }) {
@@ -74,33 +74,33 @@ function runsRouter({ runs, registry, auth, config, log = console }) {
return run;
return run;
}
}
- r.get('/api/v1/runs/:id', read, (req, res) => {
+ r.get('/api/v1/runs/:id', read, async (req, res) => {
try {
try {
- const run = owned(req);
- res.json({ run, citations: runs.citations(run.id), requests: runs.requestsFor(run.id) });
+ const run = await owned(req);
+ res.json({ run, citations: await runs.citations(run.id), requests: await runs.requestsFor(run.id) });
} catch (err) { sendError(res, err, req.ov, log); }
} catch (err) { sendError(res, err, req.ov, log); }
});
});
- r.post('/api/v1/runs/:id/cancel', create, (req, res) => {
- try { res.json({ run: runs.cancel(req.params.id, req.principal, { trace: req.ov.traceId, principalHas: has(req) }) }); } catch (err) { sendError(res, err, req.ov, log); }
+ r.post('/api/v1/runs/:id/cancel', create, async (req, res) => {
+ try { res.json({ run: await runs.cancel(req.params.id, req.principal, { trace: req.ov.traceId, principalHas: has(req) }) }); } catch (err) { sendError(res, err, req.ov, log); }
});
});
r.post('/api/v1/runs/:id/retry', create, async (req, res) => {
r.post('/api/v1/runs/:id/retry', create, async (req, res) => {
try {
try {
- const created = runs.retry(req.params.id, req.principal, { trace: req.ov.traceId, requestId: req.ov.requestId, principalHas: has(req) });
+ const created = await runs.retry(req.params.id, req.principal, { trace: req.ov.traceId, requestId: req.ov.requestId, principalHas: has(req) });
const wait = Math.max(0, Math.min(Number(req.query.wait) || 0, config.runs.maxWaitMs));
const wait = Math.max(0, Math.min(Number(req.query.wait) || 0, config.runs.maxWaitMs));
const run = await runs.wait(created, wait);
const run = await runs.wait(created, wait);
res.status(runs.TERMINAL.has(run.status) ? 201 : 202).json({ run });
res.status(runs.TERMINAL.has(run.status) ? 201 : 202).json({ run });
} catch (err) { sendError(res, err, req.ov, log); }
} catch (err) { sendError(res, err, req.ov, log); }
});
});
- r.get('/api/v1/runs/:id/citations', read, (req, res) => {
- try { const run = owned(req); res.json({ citations: runs.citations(run.id) }); } catch (err) { sendError(res, err, req.ov, log); }
+ r.get('/api/v1/runs/:id/citations', read, async (req, res) => {
+ try { const run = await owned(req); res.json({ citations: await runs.citations(run.id) }); } catch (err) { sendError(res, err, req.ov, log); }
});
});
- r.post('/api/v1/runs/:id/citations', create, (req, res) => {
+ r.post('/api/v1/runs/:id/citations', create, async (req, res) => {
try {
try {
- const run = owned(req);
+ const run = await owned(req);
const s = req.principal.subject;
const s = req.principal.subject;
if (run.requester.type !== s.type || run.requester.id !== s.id) throw new AiError(403, 'capability.denied', 'only the requester may attach citations');
if (run.requester.type !== s.type || run.requester.id !== s.id) throw new AiError(403, 'capability.denied', 'only the requester may attach citations');
const list = Array.isArray(req.body && req.body.citations) ? req.body.citations : null;
const list = Array.isArray(req.body && req.body.citations) ? req.body.citations : null;
@@ -109,9 +109,9 @@ function runsRouter({ runs, registry, auth, config, log = console }) {
@@ -109,9 +109,9 @@ function runsRouter({ runs, registry, auth, config, log = console }) {
if (!c || typeof c.source_type !== 'string' || !/^[a-z][a-z0-9_.-]{1,63}$/.test(c.source_type)) throw new AiError(422, 'input.invalid', 'every citation needs a source_type');
if (!c || typeof c.source_type !== 'string' || !/^[a-z][a-z0-9_.-]{1,63}$/.test(c.source_type)) throw new AiError(422, 'input.invalid', 'every citation needs a source_type');
for (const k of ['url', 'title', 'author', 'snippet', 'source_id']) if (c[k] != null && String(c[k]).length > 4000) throw new AiError(422, 'input.invalid', `${k} too long`);
for (const k of ['url', 'title', 'author', 'snippet', 'source_id']) if (c[k] != null && String(c[k]).length > 4000) throw new AiError(422, 'input.invalid', `${k} too long`);
}
}
- runs.addCitations(run.id, list.map(c => ({ ...c, ordinal: null, provenance: { ...(c.provenance || {}), attached: true } })), req.principal.sub);
- registry.audit(req.principal.sub, 'run.citations.attach', 'run', run.id, { trace: req.ov.traceId, metadata: { count: list.length } });
- res.status(201).json({ citations: runs.citations(run.id) });
+ await runs.addCitations(run.id, list.map(c => ({ ...c, ordinal: null, provenance: { ...(c.provenance || {}), attached: true } })), req.principal.sub);
+ await registry.audit(req.principal.sub, 'run.citations.attach', 'run', run.id, { trace: req.ov.traceId, metadata: { count: list.length } });
+ res.status(201).json({ citations: await runs.citations(run.id) });
} catch (err) { sendError(res, err, req.ov, log); }
} catch (err) { sendError(res, err, req.ov, log); }
});
});
@@ -56,7 +56,7 @@ function createKeyStore({ urls = [], pem = null, fetchImpl = globalThis.fetch, l
@@ -56,7 +56,7 @@ function createKeyStore({ urls = [], pem = null, fetchImpl = globalThis.fetch, l
return null;
return null;
}
}
- function start() {
+ async function start() {
if (pem) return Promise.resolve(key);
if (pem) return Promise.resolve(key);
const attempt = async () => {
const attempt = async () => {
const k = await fetchOnce();
const k = await fetchOnce();
@@ -68,7 +68,7 @@ function createKeyStore({ urls = [], pem = null, fetchImpl = globalThis.fetch, l
@@ -68,7 +68,7 @@ function createKeyStore({ urls = [], pem = null, fetchImpl = globalThis.fetch, l
};
};
refreshTimer = setInterval(() => { fetchOnce().catch(() => {}); }, 6 * 60 * 60 * 1000);
refreshTimer = setInterval(() => { fetchOnce().catch(() => {}); }, 6 * 60 * 60 * 1000);
refreshTimer.unref?.();
refreshTimer.unref?.();
- return attempt();
+ return await attempt();
}
}
function stop() { clearTimeout(retryTimer); clearInterval(refreshTimer); }
function stop() { clearTimeout(retryTimer); clearInterval(refreshTimer); }
@@ -27,33 +27,33 @@ function createCache(db, { clock = { now: () => Date.now() } } = {}) {
@@ -27,33 +27,33 @@ function createCache(db, { clock = { now: () => Date.now() } } = {}) {
return sha256({ scope, w: workflowKey, wv: workflowVersion, tv: templateVersion || null, r: routeKey || null, rv: routeVersion || null, i: inputHash });
return sha256({ scope, w: workflowKey, wv: workflowVersion, tv: templateVersion || null, r: routeKey || null, rv: routeVersion || null, i: inputHash });
}
}
- function get(key, scope) {
- const row = db.prepare('SELECT * FROM cache_entries WHERE cache_key = ?').get(key);
+ async function get(key, scope) {
+ const row = await db.prepare('SELECT * FROM cache_entries WHERE cache_key = ?').get(key);
if (!row) return null;
if (!row) return null;
if (row.scope !== scope) return null; // belt and braces: never cross scope
if (row.scope !== scope) return null; // belt and braces: never cross scope
- if (row.expires_at <= clock.now()) { db.prepare('DELETE FROM cache_entries WHERE cache_key = ?').run(key); return null; }
- db.prepare('UPDATE cache_entries SET hits = hits + 1 WHERE cache_key = ?').run(key);
+ if (row.expires_at <= clock.now()) { await db.prepare('DELETE FROM cache_entries WHERE cache_key = ?').run(key); return null; }
+ await db.prepare('UPDATE cache_entries SET hits = hits + 1 WHERE cache_key = ?').run(key);
return { ...row, output: parseJson(row.output, null) };
return { ...row, output: parseJson(row.output, null) };
}
}
- function put({ key, scope, privacy, workflowKey, workflowVersion, templateVersion, routeKey, routeVersion, model, inputHash, output, runId, ttlSec }) {
- db.prepare(`INSERT INTO cache_entries (cache_key, scope, workflow_key, workflow_version, template_version, route_key, route_version, model_key, input_hash, privacy, output, run_id, hits, created_at, expires_at)
+ async function put({ key, scope, privacy, workflowKey, workflowVersion, templateVersion, routeKey, routeVersion, model, inputHash, output, runId, ttlSec }) {
+ await db.prepare(`INSERT INTO cache_entries (cache_key, scope, workflow_key, workflow_version, template_version, route_key, route_version, model_key, input_hash, privacy, output, run_id, hits, created_at, expires_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0, ?, ?)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0, ?, ?)
ON CONFLICT(cache_key) DO UPDATE SET output = excluded.output, run_id = excluded.run_id, model_key = excluded.model_key, created_at = excluded.created_at, expires_at = excluded.expires_at`)
ON CONFLICT(cache_key) DO UPDATE SET output = excluded.output, run_id = excluded.run_id, model_key = excluded.model_key, created_at = excluded.created_at, expires_at = excluded.expires_at`)
.run(key, scope, workflowKey, workflowVersion, templateVersion || null, routeKey || null, routeVersion || null, model || null, inputHash, privacy,
.run(key, scope, workflowKey, workflowVersion, templateVersion || null, routeKey || null, routeVersion || null, model || null, inputHash, privacy,
JSON.stringify(output), runId, iso(clock.now()), clock.now() + ttlSec * 1000);
JSON.stringify(output), runId, iso(clock.now()), clock.now() + ttlSec * 1000);
}
}
- function stats() {
- return db.prepare('SELECT workflow_key, privacy, COUNT(*) AS entries, SUM(hits) AS hits FROM cache_entries WHERE expires_at > ? GROUP BY workflow_key, privacy ORDER BY entries DESC').all(clock.now());
+ async function stats() {
+ return await db.prepare('SELECT workflow_key, privacy, COUNT(*) AS entries, SUM(hits) AS hits FROM cache_entries WHERE expires_at > ? GROUP BY workflow_key, privacy ORDER BY entries DESC').all(clock.now());
}
}
- function purge({ workflow } = {}) {
- const r = workflow ? db.prepare('DELETE FROM cache_entries WHERE workflow_key = ?').run(workflow) : db.prepare('DELETE FROM cache_entries').run();
+ async function purge({ workflow } = {}) {
+ const r = workflow ? await db.prepare('DELETE FROM cache_entries WHERE workflow_key = ?').run(workflow) : await db.prepare('DELETE FROM cache_entries').run();
return r.changes;
return r.changes;
}
}
- function prune() { return db.prepare('DELETE FROM cache_entries WHERE expires_at <= ?').run(clock.now()).changes; }
+ async function prune() { return (await db.prepare('DELETE FROM cache_entries WHERE expires_at <= ?').run(clock.now())).changes; }
return { scopeFor, keyFor, get, put, stats, purge, prune };
return { scopeFor, keyFor, get, put, stats, purge, prune };
}
}
@@ -59,6 +59,9 @@ function load(env = process.env) {
@@ -59,6 +59,9 @@ function load(env = process.env) {
networkPublicKey: env.OV_NETWORK_PUBLIC_KEY ? env.OV_NETWORK_PUBLIC_KEY.replace(/\\n/g, '\n') : null,
networkPublicKey: env.OV_NETWORK_PUBLIC_KEY ? env.OV_NETWORK_PUBLIC_KEY.replace(/\\n/g, '\n') : null,
audience: 'openvibe.ai',
audience: 'openvibe.ai',
+ // PostgreSQL (ADR-035): DATABASE_URL serves (PgBouncer), DATABASE_DIRECT_URL migrates (owner role).
+ db: { url: env.DATABASE_URL || '', directUrl: env.DATABASE_DIRECT_URL || '' },
+ // The SQLite file of releases before the switch: read once by scripts/migrate-to-postgres.js.
dbPath: env.AI_DB_PATH || './data/ai.db',
dbPath: env.AI_DB_PATH || './data/ai.db',
// ── Operator console (server/console): OpenVibe.Network SSO (authorization code + PKCE S256
// ── Operator console (server/console): OpenVibe.Network SSO (authorization code + PKCE S256
@@ -109,20 +109,20 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
@@ -109,20 +109,20 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
const sessions = createSessions({ db, config, clock, secret: secret || 'unavailable' });
const sessions = createSessions({ db, config, clock, secret: secret || 'unavailable' });
/** Every secret value this process can resolve, so that no page can ever carry one (belt and braces). */
/** Every secret value this process can resolve, so that no page can ever carry one (belt and braces). */
- function secretValues() {
+ async function secretValues() {
const vals = new Set([clientSecret(), secret]);
const vals = new Set([clientSecret(), secret]);
- for (const p of registry.listProviders()) vals.add(resolveSecret(p.secret_ref, env));
+ for (const p of await registry.listProviders()) vals.add(resolveSecret(p.secret_ref, env));
return [...vals].filter((v) => v && v.length >= 8);
return [...vals].filter((v) => v && v.length >= 8);
}
}
- function scrub(text) {
+ async function scrub(text) {
let out = text;
let out = text;
- for (const v of secretValues()) for (const form of new Set([v, esc(v)])) out = out.split(form).join('[redacted]');
+ for (const v of await secretValues()) for (const form of new Set([v, esc(v)])) out = out.split(form).join('[redacted]');
return out;
return out;
}
}
- const send = (res, status, text) => res.status(status).type('html').send(scrub(text));
+ const send = async (res, status, text) => res.status(status).type('html').send(await scrub(text));
const who = (req) => ({ actor: req.staff.subject, trace: req.ov ? req.ov.traceId : null });
const who = (req) => ({ actor: req.staff.subject, trace: req.ov ? req.ov.traceId : null });
const common = (req) => ({ staff: req.staff, csrf: req.staff ? req.staff.csrf : '' });
const common = (req) => ({ staff: req.staff, csrf: req.staff ? req.staff.csrf : '' });
- const audit = (actor, action, targetType, targetId, metadata, req) => registry.audit(actor || 'anonymous', action, targetType, targetId, { trace: req && req.ov ? req.ov.traceId : null, metadata });
+ const audit = async (actor, action, targetType, targetId, metadata, req) => await registry.audit(actor || 'anonymous', action, targetType, targetId, { trace: req && req.ov ? req.ov.traceId : null, metadata });
const notice = (req) => {
const notice = (req) => {
const base = NOTICES[req.query.done];
const base = NOTICES[req.query.done];
if (!base) return null;
if (!base) return null;
@@ -130,28 +130,28 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
@@ -130,28 +130,28 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
};
};
// ── Every console and sign-in response ───────────────────
// ── Every console and sign-in response ───────────────────
- r.use(scoped, (req, res, next) => {
+ r.use(scoped, async (req, res, next) => {
res.setHeader('Content-Security-Policy', CSP);
res.setHeader('Content-Security-Policy', CSP);
res.setHeader('X-Frame-Options', 'DENY');
res.setHeader('X-Frame-Options', 'DENY');
res.setHeader('Referrer-Policy', 'no-referrer');
res.setHeader('Referrer-Policy', 'no-referrer');
res.setHeader('X-Robots-Tag', 'noindex, nofollow');
res.setHeader('X-Robots-Tag', 'noindex, nofollow');
res.setHeader('Cache-Control', 'no-store');
res.setHeader('Cache-Control', 'no-store');
- if (unavailable) return send(res, 503, pages.message({ title: 'Console unavailable', text: 'The operator console is not configured on this host. The operator must set its environment (see .env.example, "Operator console").' }));
+ if (unavailable) return await send(res, 503, pages.message({ title: 'Console unavailable', text: 'The operator console is not configured on this host. The operator must set its environment (see .env.example, "Operator console").' }));
return next();
return next();
});
});
r.use(scoped, express.urlencoded({ extended: false, limit: '16kb', parameterLimit: 50 }));
r.use(scoped, express.urlencoded({ extended: false, limit: '16kb', parameterLimit: 50 }));
// ── Session -> req.staff ─────────────────────────────────
// ── Session -> req.staff ─────────────────────────────────
- r.use(scoped, (req, res, next) => {
+ r.use(scoped, async (req, res, next) => {
req.staff = null;
req.staff = null;
- const row = sessions.read(req);
+ const row = await sessions.read(req);
if (!row) return next();
if (!row) return next();
const caps = consoleCapabilities(row.staff);
const caps = consoleCapabilities(row.staff);
if (!caps.includes(CAPS.usageRead)) {
if (!caps.includes(CAPS.usageRead)) {
- sessions.revoke(row);
+ await sessions.revoke(row);
sessions.clear(res);
sessions.clear(res);
- audit(row.subject, 'console.request.refused', 'console', 'session', { reason: 'the session no longer maps to ai.usage.read' }, req);
- return send(res, 403, pages.message({ title: 'Not authorized', text: 'Your account is not AI operator staff any more. The session was ended.' }));
+ await audit(row.subject, 'console.request.refused', 'console', 'session', { reason: 'the session no longer maps to ai.usage.read' }, req);
+ return await send(res, 403, pages.message({ title: 'Not authorized', text: 'Your account is not AI operator staff any more. The session was ended.' }));
}
}
req.staff = {
req.staff = {
subject: row.subject, username: row.username, role: staffMap.effectiveRole(row.staff), caps, csrf: row.csrf, session: row,
subject: row.subject, username: row.username, role: staffMap.effectiveRole(row.staff), caps, csrf: row.csrf, session: row,
@@ -162,18 +162,18 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
@@ -162,18 +162,18 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
/** Staff only, holding `cap` through STAFF_TO_AI. POSTs also need the CSRF token. */
/** Staff only, holding `cap` through STAFF_TO_AI. POSTs also need the CSRF token. */
function needs(cap) {
function needs(cap) {
- return (req, res, next) => {
+ return async (req, res, next) => {
if (!req.staff) {
if (!req.staff) {
- if (req.method === 'GET') return send(res, 401, pages.signIn({ next: sanitizeNext(req.originalUrl) }));
- return send(res, 401, pages.signIn({ message: 'Your session has ended. Sign in again, then repeat the action.' }));
+ if (req.method === 'GET') return await send(res, 401, pages.signIn({ next: sanitizeNext(req.originalUrl) }));
+ return await send(res, 401, pages.signIn({ message: 'Your session has ended. Sign in again, then repeat the action.' }));
}
}
if (!req.staff.caps.includes(cap)) {
if (!req.staff.caps.includes(cap)) {
- if (req.method === 'POST') audit(req.staff.subject, 'console.request.refused', 'console', req.path, { reason: `needs ${cap}` }, req);
- return send(res, 403, pages.message({ title: 'Not authorized', text: `This needs ${cap}.`, ...common(req) }));
+ if (req.method === 'POST') await audit(req.staff.subject, 'console.request.refused', 'console', req.path, { reason: `needs ${cap}` }, req);
+ return await send(res, 403, pages.message({ title: 'Not authorized', text: `This needs ${cap}.`, ...common(req) }));
}
}
if (req.method === 'POST' && !csrfOk(req)) {
if (req.method === 'POST' && !csrfOk(req)) {
- audit(req.staff.subject, 'console.request.refused', 'console', req.path, { reason: 'missing or invalid CSRF token, or a cross-site request' }, req);
- return send(res, 403, pages.message({ title: 'Request refused', text: 'The form was stale or came from another site. Reload the page and try again.', ...common(req) }));
+ await audit(req.staff.subject, 'console.request.refused', 'console', req.path, { reason: 'missing or invalid CSRF token, or a cross-site request' }, req);
+ return await send(res, 403, pages.message({ title: 'Request refused', text: 'The form was stale or came from another site. Reload the page and try again.', ...common(req) }));
}
}
return next();
return next();
};
};
@@ -185,10 +185,10 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
@@ -185,10 +185,10 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
if (origin && origin !== baseOrigin) return false;
if (origin && origin !== baseOrigin) return false;
return sameString((req.body || {})._csrf, req.staff.csrf);
return sameString((req.body || {})._csrf, req.staff.csrf);
}
}
- const wrap = (fn) => (req, res, next) => Promise.resolve().then(() => fn(req, res, next)).catch((e) => {
- if (e instanceof AiError) return send(res, e.status >= 400 && e.status < 600 ? e.status : 400, pages.message({ title: 'Refused', text: `${e.detail || e.message} (${e.code})`, back: req.get('referer') && req.get('referer').startsWith(`${baseOrigin}/console`) ? req.get('referer') : '/console', ...common(req) }));
+ const wrap = (fn) => (req, res, next) => Promise.resolve().then(async () => await fn(req, res, next)).catch(async (e) => {
+ if (e instanceof AiError) return await send(res, e.status >= 400 && e.status < 600 ? e.status : 400, pages.message({ title: 'Refused', text: `${e.detail || e.message} (${e.code})`, back: req.get('referer') && req.get('referer').startsWith(`${baseOrigin}/console`) ? req.get('referer') : '/console', ...common(req) }));
log.error(`[console] ${req.method} ${req.path}: ${e && e.stack ? e.stack : e}`);
log.error(`[console] ${req.method} ${req.path}: ${e && e.stack ? e.stack : e}`);
- if (!res.headersSent) send(res, 500, pages.message({ title: 'Error', text: 'Something went wrong. Nothing was changed unless the page says so; check the audit log.', ...common(req) }));
+ if (!res.headersSent) await send(res, 500, pages.message({ title: 'Error', text: 'Something went wrong. Nothing was changed unless the page says so; check the audit log.', ...common(req) }));
return null;
return null;
});
});
const read = needs(CAPS.usageRead);
const read = needs(CAPS.usageRead);
@@ -203,9 +203,9 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
@@ -203,9 +203,9 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
r.get('/auth/callback', wrap(async (req, res) => {
r.get('/auth/callback', wrap(async (req, res) => {
const flow = sessions.takeFlow(req, res);
const flow = sessions.takeFlow(req, res);
- if (req.query.error) return send(res, 400, pages.signIn({ message: `Sign-in was not completed (${String(req.query.error).slice(0, 60)}).` }));
+ if (req.query.error) return await send(res, 400, pages.signIn({ message: `Sign-in was not completed (${String(req.query.error).slice(0, 60)}).` }));
if (!flow || !req.query.code || !sameString(req.query.state, flow.s)) {
if (!flow || !req.query.code || !sameString(req.query.state, flow.s)) {
- return send(res, 400, pages.signIn({ message: 'The sign-in could not be verified (expired or mismatched state). Please try again.' }));
+ return await send(res, 400, pages.signIn({ message: 'The sign-in could not be verified (expired or mismatched state). Please try again.' }));
}
}
let person;
let person;
try {
try {
@@ -214,69 +214,69 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
@@ -214,69 +214,69 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
person = await sso.exchange({ config, clientSecret: clientSecret(), code: req.query.code, verifier: flow.v, publicKey, now: clock.now(), fetchImpl });
person = await sso.exchange({ config, clientSecret: clientSecret(), code: req.query.code, verifier: flow.v, publicKey, now: clock.now(), fetchImpl });
} catch (e) {
} catch (e) {
log.warn(`[console] sign-in failed: ${e.message}`);
log.warn(`[console] sign-in failed: ${e.message}`);
- return send(res, e.status && e.status < 500 ? 400 : 502, pages.signIn({ message: 'OpenVibe.Network did not confirm the sign-in. Please try again.' }));
+ return await send(res, e.status && e.status < 500 ? 400 : 502, pages.signIn({ message: 'OpenVibe.Network did not confirm the sign-in. Please try again.' }));
}
}
const ipHash = sessions.ipHash(req.ip);
const ipHash = sessions.ipHash(req.ip);
if (person.error) {
if (person.error) {
- audit('anonymous', 'console.sign_in.refused', 'console', 'sign_in', { reason: person.error, ip_hash: ipHash }, req);
- return send(res, 403, pages.message({ title: 'Not authorized', text: 'This console is only for OpenVibe.Network staff. Your sign-in was recorded.' }));
+ await audit('anonymous', 'console.sign_in.refused', 'console', 'sign_in', { reason: person.error, ip_hash: ipHash }, req);
+ return await send(res, 403, pages.message({ title: 'Not authorized', text: 'This console is only for OpenVibe.Network staff. Your sign-in was recorded.' }));
}
}
const caps = consoleCapabilities(person.staff);
const caps = consoleCapabilities(person.staff);
const role = staffMap.effectiveRole(person.staff);
const role = staffMap.effectiveRole(person.staff);
if (!caps.includes(CAPS.usageRead)) {
if (!caps.includes(CAPS.usageRead)) {
- audit(person.subject, 'console.sign_in.refused', 'console', 'sign_in', { reason: `role ${role} does not hold staff.site.view`, role, ip_hash: ipHash }, req);
- return send(res, 403, pages.message({ title: 'Not authorized', text: 'This console is only for OpenVibe.Network staff (admins and the owner). Your sign-in was recorded.' }));
+ await audit(person.subject, 'console.sign_in.refused', 'console', 'sign_in', { reason: `role ${role} does not hold staff.site.view`, role, ip_hash: ipHash }, req);
+ return await send(res, 403, pages.message({ title: 'Not authorized', text: 'This console is only for OpenVibe.Network staff (admins and the owner). Your sign-in was recorded.' }));
}
}
- db.transaction(() => {
- sessions.create(res, { subject: person.subject, username: person.username, staff: person.staff, ip: req.ip });
- audit(person.subject, 'console.sign_in', 'console', 'sign_in', { role, capabilities: caps, ip_hash: ipHash }, req);
- })();
+ await db.tx(async () => {
+ await sessions.create(res, { subject: person.subject, username: person.username, staff: person.staff, ip: req.ip });
+ await audit(person.subject, 'console.sign_in', 'console', 'sign_in', { role, capabilities: caps, ip_hash: ipHash }, req);
+ });
return res.redirect(303, flow.n || '/console');
return res.redirect(303, flow.n || '/console');
}));
}));
- r.post('/auth/logout', read, (req, res) => {
- db.transaction(() => {
- sessions.revoke(req.staff.session);
- audit(req.staff.subject, 'console.sign_out', 'console', 'sign_in', {}, req);
- })();
+ r.post('/auth/logout', read, async (req, res) => {
+ await db.tx(async () => {
+ await sessions.revoke(req.staff.session);
+ await audit(req.staff.subject, 'console.sign_out', 'console', 'sign_in', {}, req);
+ });
sessions.clear(res);
sessions.clear(res);
res.redirect(303, '/console');
res.redirect(303, '/console');
});
});
// ── Overview ─────────────────────────────────────────────
// ── Overview ─────────────────────────────────────────────
- r.get('/console', read, wrap((req, res) => {
+ r.get('/console', read, wrap(async (req, res) => {
const since = new Date(clock.now() - DAY_MS).toISOString();
const since = new Date(clock.now() - DAY_MS).toISOString();
- const st = ops.status();
+ const st = await ops.status();
const problems = [];
const problems = [];
- for (const p of registry.listProviders().filter((x) => x.status === 'active' && x.kind !== 'stub')) {
- const v = ops.providerView(p);
+ for (const p of (await registry.listProviders()).filter((x) => x.status === 'active' && x.kind !== 'stub')) {
+ const v = await ops.providerView(p);
if (v.credentials === 'missing') problems.push({ key: p.key, kind: p.kind, problem: `credentials missing (${p.secret_ref || 'no reference'})` });
if (v.credentials === 'missing') problems.push({ key: p.key, kind: p.kind, problem: `credentials missing (${p.secret_ref || 'no reference'})` });
if (v.health && v.health.state !== 'closed') problems.push({ key: p.key, kind: p.kind, problem: `circuit ${v.health.state}` });
if (v.health && v.health.state !== 'closed') problems.push({ key: p.key, kind: p.kind, problem: `circuit ${v.health.state}` });
}
}
- const stats = cache.stats();
+ const stats = await cache.stats();
const frac = (c) => {
const frac = (c) => {
const x = c.quota;
const x = c.quota;
return Math.max(x.max_requests ? c.used.requests / x.max_requests : 0, x.max_tokens ? c.used.tokens / x.max_tokens : 0, x.max_cost_usd ? c.used.cost_usd / x.max_cost_usd : 0);
return Math.max(x.max_requests ? c.used.requests / x.max_requests : 0, x.max_tokens ? c.used.tokens / x.max_tokens : 0, x.max_cost_usd ? c.used.cost_usd / x.max_cost_usd : 0);
};
};
- send(res, 200, pages.overview({
- ...common(req), st, counts24: q.statusCounts(db, since), failures24: q.failures(db, since), queue: runs.stats(), problems,
+ await send(res, 200, pages.overview({
+ ...common(req), st, counts24: await q.statusCounts(db, since), failures24: await q.failures(db, since), queue: runs.stats(), problems,
cacheTotals: { entries: stats.reduce((a, s) => a + s.entries, 0), hits: stats.reduce((a, s) => a + (s.hits || 0), 0) },
cacheTotals: { entries: stats.reduce((a, s) => a + s.entries, 0), hits: stats.reduce((a, s) => a + (s.hits || 0), 0) },
stubFallback: config.stubFallback, topQuotas: [...st.quotas].sort((a, b) => frac(b) - frac(a)).slice(0, 10),
stubFallback: config.stubFallback, topQuotas: [...st.quotas].sort((a, b) => frac(b) - frac(a)).slice(0, 10),
}));
}));
}));
}));
// ── Providers ────────────────────────────────────────────
// ── Providers ────────────────────────────────────────────
- r.get('/console/providers', read, wrap((req, res) => send(res, 200, pages.providers({
- ...common(req), rows: registry.listProviders().map(ops.providerView), models: registry.listModels(), stubFallback: config.stubFallback,
+ r.get('/console/providers', read, wrap(async (req, res) => await send(res, 200, pages.providers({
+ ...common(req), rows: (await Promise.all((await registry.listProviders()).map(ops.providerView))), models: await registry.listModels(), stubFallback: config.stubFallback,
canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req),
canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req),
}))));
}))));
- r.post('/console/providers/:key/status', needs(CAPS.providerManage), wrap((req, res) => {
- ops.setProviderStatus(req.params.key, String(req.body.status || ''), who(req));
+ r.post('/console/providers/:key/status', needs(CAPS.providerManage), wrap(async (req, res) => {
+ await ops.setProviderStatus(req.params.key, String(req.body.status || ''), who(req));
res.redirect(303, '/console/providers?done=provider_status');
res.redirect(303, '/console/providers?done=provider_status');
}));
}));
- r.post('/console/providers/:key/reset', needs(CAPS.providerManage), wrap((req, res) => {
- if (!registry.getProvider(req.params.key)) throw new AiError(404, 'ai.not_found', 'no such provider');
- ops.resetCircuit(req.params.key, who(req));
+ r.post('/console/providers/:key/reset', needs(CAPS.providerManage), wrap(async (req, res) => {
+ if (!await registry.getProvider(req.params.key)) throw new AiError(404, 'ai.not_found', 'no such provider');
+ await ops.resetCircuit(req.params.key, who(req));
res.redirect(303, '/console/providers?done=circuit_reset');
res.redirect(303, '/console/providers?done=circuit_reset');
}));
}));
@@ -294,95 +294,95 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
@@ -294,95 +294,95 @@ function consoleRouter({ config, db, registry, pool, quotas, cache, runs, keys,
return m;
return m;
}
}
for (const [kind, k] of Object.entries(DEF_KINDS)) {
for (const [kind, k] of Object.entries(DEF_KINDS)) {
- r.get(`/console/${kind}`, read, wrap((req, res) => send(res, 200, pages.definitions({
- ...common(req), kind, rows: registry[k.list](), inUse: inUseMap(kind, registry[k.list]({ history: true })),
+ r.get(`/console/${kind}`, read, wrap(async (req, res) => await send(res, 200, pages.definitions({
+ ...common(req), kind, rows: await registry[k.list](), inUse: inUseMap(kind, await registry[k.list]({ history: true })),
}))));
}))));
- r.get(`/console/${kind}/:key`, read, wrap((req, res) => {
- const versions = registry[k.list]({ history: true }).filter((x) => x.key === req.params.key).sort((a, b) => b.version - a.version);
- if (!versions.length) return send(res, 404, pages.message({ title: 'Not found', text: `No ${k.one} ${req.params.key}.`, back: `/console/${kind}`, ...common(req) }));
+ r.get(`/console/${kind}/:key`, read, wrap(async (req, res) => {
+ const versions = (await registry[k.list]({ history: true })).filter((x) => x.key === req.params.key).sort((a, b) => b.version - a.version);
+ if (!versions.length) return await send(res, 404, pages.message({ title: 'Not found', text: `No ${k.one} ${req.params.key}.`, back: `/console/${kind}`, ...common(req) }));
const want = Number(req.query.version);
const want = Number(req.query.version);
const selected = (Number.isInteger(want) && versions.find((x) => x.version === want)) || versions[0];
const selected = (Number.isInteger(want) && versions.find((x) => x.version === want)) || versions[0];
- return send(res, 200, pages.definition({
- ...common(req), kind, key: req.params.key, versions, selected: registry[k.get](req.params.key, selected.version),
+ return await send(res, 200, pages.definition({
+ ...common(req), kind, key: req.params.key, versions, selected: await registry[k.get](req.params.key, selected.version),
inUse: inUseMap(kind, versions).get(req.params.key), canManage: req.staff.caps.includes(CAPS.workflowManage), notice: notice(req),
inUse: inUseMap(kind, versions).get(req.params.key), canManage: req.staff.caps.includes(CAPS.workflowManage), notice: notice(req),
}));
}));
}));
}));
- r.post(`/console/${kind}/:key/versions/:version/status`, needs(CAPS.workflowManage), wrap((req, res) => {
+ r.post(`/console/${kind}/:key/versions/:version/status`, needs(CAPS.workflowManage), wrap(async (req, res) => {
const version = Number(req.params.version);
const version = Number(req.params.version);
if (!Number.isInteger(version) || version < 1) throw new AiError(404, 'ai.not_found', `no ${k.one} ${req.params.key} v${req.params.version}`);
if (!Number.isInteger(version) || version < 1) throw new AiError(404, 'ai.not_found', `no ${k.one} ${req.params.key} v${req.params.version}`);
- registry.setStatus(k.one, req.params.key, version, String(req.body.status || ''), who(req));
+ await registry.setStatus(k.one, req.params.key, version, String(req.body.status || ''), who(req));
res.redirect(303, `/console/${kind}/${encodeURIComponent(req.params.key)}?version=${version}&done=version_status`);
res.redirect(303, `/console/${kind}/${encodeURIComponent(req.params.key)}?version=${version}&done=version_status`);
}));
}));
}
}
// ── Runs ─────────────────────────────────────────────────
// ── Runs ─────────────────────────────────────────────────
- r.get('/console/runs', read, wrap((req, res) => {
+ r.get('/console/runs', read, wrap(async (req, res) => {
const { filters, form, bad } = q.parseRunFilters(req.query);
const { filters, form, bad } = q.parseRunFilters(req.query);
- const list = q.listRuns(db, filters);
- send(res, bad.length ? 400 : 200, pages.runs({
- ...common(req), form, filters, bad, rows: list.rows, next: list.next, facets: q.runFacets(db),
- failures24: filters.status === 'failed' ? q.failures(db, new Date(clock.now() - DAY_MS).toISOString()) : [],
+ const list = await q.listRuns(db, filters);
+ await send(res, bad.length ? 400 : 200, pages.runs({
+ ...common(req), form, filters, bad, rows: list.rows, next: list.next, facets: await q.runFacets(db),
+ failures24: filters.status === 'failed' ? await q.failures(db, new Date(clock.now() - DAY_MS).toISOString()) : [],
}));
}));
}));
}));
- r.get('/console/runs/:id', read, wrap((req, res) => {
- const run = runs.get(req.params.id);
- if (!run) return send(res, 404, pages.message({ title: 'Not found', text: `No run ${req.params.id}.`, back: '/console/runs', ...common(req) }));
- return send(res, 200, pages.run({
- ...common(req), r: run, citations: runs.citations(run.id), requests: runs.requestsFor(run.id), input: q.preview(run.input), output: q.preview(run.output),
+ r.get('/console/runs/:id', read, wrap(async (req, res) => {
+ const run = await runs.get(req.params.id);
+ if (!run) return await send(res, 404, pages.message({ title: 'Not found', text: `No run ${req.params.id}.`, back: '/console/runs', ...common(req) }));
+ return await send(res, 200, pages.run({
+ ...common(req), r: run, citations: await runs.citations(run.id), requests: await runs.requestsFor(run.id), input: q.preview(run.input), output: q.preview(run.output),
canCancel: req.staff.caps.includes(CAPS.workflowManage), notice: notice(req),
canCancel: req.staff.caps.includes(CAPS.workflowManage), notice: notice(req),
}));
}));
}));
}));
- r.post('/console/runs/:id/cancel', needs(CAPS.workflowManage), wrap((req, res) => {
- runs.cancel(req.params.id, req.staff.principal, { trace: req.ov ? req.ov.traceId : null, principalHas: (cap) => req.staff.caps.includes(cap) });
+ r.post('/console/runs/:id/cancel', needs(CAPS.workflowManage), wrap(async (req, res) => {
+ await runs.cancel(req.params.id, req.staff.principal, { trace: req.ov ? req.ov.traceId : null, principalHas: (cap) => req.staff.caps.includes(cap) });
res.redirect(303, `/console/runs/${encodeURIComponent(req.params.id)}?done=run_cancelled`);
res.redirect(303, `/console/runs/${encodeURIComponent(req.params.id)}?done=run_cancelled`);
}));
}));
// ── Quotas and usage ─────────────────────────────────────
// ── Quotas and usage ─────────────────────────────────────
- const quotaPage = (req, res, status, extra = {}) => send(res, status, pages.quotas({
- ...common(req), counters: quotas.counters(), all: quotas.list(), canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req), ...extra,
+ const quotaPage = async (req, res, status, extra = {}) => await send(res, status, pages.quotas({
+ ...common(req), counters: await quotas.counters(), all: await quotas.list(), canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req), ...extra,
}));
}));
- r.get('/console/quotas', read, wrap((req, res) => quotaPage(req, res, 200)));
- r.post('/console/quotas', needs(CAPS.providerManage), wrap((req, res) => {
+ r.get('/console/quotas', read, wrap(async (req, res) => await quotaPage(req, res, 200)));
+ r.post('/console/quotas', needs(CAPS.providerManage), wrap(async (req, res) => {
const b = req.body || {};
const b = req.body || {};
const form = {};
const form = {};
for (const f of ['scope_type', 'scope_id', 'window', 'max_requests', 'max_tokens', 'max_cost_usd', 'workflow_prefix', 'status']) form[f] = String(b[f] == null ? '' : b[f]).trim().slice(0, 200);
for (const f of ['scope_type', 'scope_id', 'window', 'max_requests', 'max_tokens', 'max_cost_usd', 'workflow_prefix', 'status']) form[f] = String(b[f] == null ? '' : b[f]).trim().slice(0, 200);
try {
try {
- quotas.upsert({ ...form, scope_id: form.scope_id || (form.scope_type === 'global' ? '*' : ''), workflow_prefix: form.workflow_prefix || null, status: form.status || 'active' }, { ...who(req), origin: 'admin' });
+ await quotas.upsert({ ...form, scope_id: form.scope_id || (form.scope_type === 'global' ? '*' : ''), workflow_prefix: form.workflow_prefix || null, status: form.status || 'active' }, { ...who(req), origin: 'admin' });
} catch (e) {
} catch (e) {
if (!(e instanceof AiError)) throw e;
if (!(e instanceof AiError)) throw e;
- return quotaPage(req, res, e.status, { error: `${e.detail || e.message} (${e.code})`, form, notice: null });
+ return await quotaPage(req, res, e.status, { error: `${e.detail || e.message} (${e.code})`, form, notice: null });
}
}
return res.redirect(303, '/console/quotas?done=quota_saved');
return res.redirect(303, '/console/quotas?done=quota_saved');
}));
}));
- r.get('/console/usage', read, wrap((req, res) => {
+ r.get('/console/usage', read, wrap(async (req, res) => {
const { filters, form, bad } = q.parseUsageFilters(req.query, clock.now());
const { filters, form, bad } = q.parseUsageFilters(req.query, clock.now());
- send(res, bad.length ? 400 : 200, pages.usage({ ...common(req), form, bad, report: q.usageReport(quotas, filters) }));
+ await send(res, bad.length ? 400 : 200, pages.usage({ ...common(req), form, bad, report: await q.usageReport(quotas, filters) }));
}));
}));
// ── Cache ────────────────────────────────────────────────
// ── Cache ────────────────────────────────────────────────
- const cachePage = (req, res, status, extra = {}) => send(res, status, pages.cache({
- ...common(req), stats: cache.stats(), canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req), ...extra,
+ const cachePage = async (req, res, status, extra = {}) => await send(res, status, pages.cache({
+ ...common(req), stats: await cache.stats(), canManage: req.staff.caps.includes(CAPS.providerManage), notice: notice(req), ...extra,
}));
}));
- r.get('/console/cache', read, wrap((req, res) => cachePage(req, res, 200)));
- r.post('/console/cache/purge', needs(CAPS.providerManage), wrap((req, res) => {
+ r.get('/console/cache', read, wrap(async (req, res) => await cachePage(req, res, 200)));
+ r.post('/console/cache/purge', needs(CAPS.providerManage), wrap(async (req, res) => {
const workflow = String((req.body || {}).workflow || '').trim();
const workflow = String((req.body || {}).workflow || '').trim();
- if (workflow && !/^[a-z][a-z0-9_-]*(\.[a-z0-9_-]+)*$/.test(workflow)) return cachePage(req, res, 422, { error: 'that is not a workflow key', notice: null });
- if (!workflow && (req.body || {}).confirm_all !== 'yes') return cachePage(req, res, 422, { error: 'give a workflow key, or tick the box to purge every entry', notice: null });
- const removed = ops.purgeCache(workflow || null, who(req));
+ if (workflow && !/^[a-z][a-z0-9_-]*(\.[a-z0-9_-]+)*$/.test(workflow)) return await cachePage(req, res, 422, { error: 'that is not a workflow key', notice: null });
+ if (!workflow && (req.body || {}).confirm_all !== 'yes') return await cachePage(req, res, 422, { error: 'give a workflow key, or tick the box to purge every entry', notice: null });
+ const removed = await ops.purgeCache(workflow || null, who(req));
return res.redirect(303, `/console/cache?done=cache_purged&removed=${removed}`);
return res.redirect(303, `/console/cache?done=cache_purged&removed=${removed}`);
}));
}));
// ── Audit ────────────────────────────────────────────────
// ── Audit ────────────────────────────────────────────────
- r.get('/console/audit', read, wrap((req, res) => {
+ r.get('/console/audit', read, wrap(async (req, res) => {
const { filters, form, bad } = q.parseAuditFilters(req.query);
const { filters, form, bad } = q.parseAuditFilters(req.query);
- const page = q.listAudit(db, filters);
- send(res, bad.length ? 400 : 200, pages.audit({ ...common(req), form, bad, rows: page.rows, next: page.next, names: q.usernames(db), kinds: q.AUDIT_KINDS }));
+ const page = await q.listAudit(db, filters);
+ await send(res, bad.length ? 400 : 200, pages.audit({ ...common(req), form, bad, rows: page.rows, next: page.next, names: await q.usernames(db), kinds: q.AUDIT_KINDS }));
}));
}));
// Anything else under /console or /auth: the sign-in page, or (signed in) the console's own not-found page.
// Anything else under /console or /auth: the sign-in page, or (signed in) the console's own not-found page.
- r.all(['/console/*', '/auth/*'], (req, res) => {
- if (!req.staff) return send(res, 401, pages.signIn({ next: req.method === 'GET' ? sanitizeNext(req.originalUrl) : '/console' }));
- return send(res, 404, pages.message({ title: 'Not found', text: 'There is no such console page.', ...common(req) }));
+ r.all(['/console/*', '/auth/*'], async (req, res) => {
+ if (!req.staff) return await send(res, 401, pages.signIn({ next: req.method === 'GET' ? sanitizeNext(req.originalUrl) : '/console' }));
+ return await send(res, 404, pages.message({ title: 'Not found', text: 'There is no such console page.', ...common(req) }));
});
});
return r;
return r;
@@ -67,7 +67,7 @@ function parseRunFilters(q = {}) {
@@ -67,7 +67,7 @@ function parseRunFilters(q = {}) {
return { filters, form, bad };
return { filters, form, bad };
}
}
-function listRuns(db, f, limit = PAGE) {
+async function listRuns(db, f, limit = PAGE) {
const where = [];
const where = [];
const args = [];
const args = [];
if (f.status) { where.push('status = ?'); args.push(f.status); }
if (f.status) { where.push('status = ?'); args.push(f.status); }
@@ -81,28 +81,28 @@ function listRuns(db, f, limit = PAGE) {
@@ -81,28 +81,28 @@ function listRuns(db, f, limit = PAGE) {
if (f.fromIso) { where.push('created_at >= ?'); args.push(f.fromIso); }
if (f.fromIso) { where.push('created_at >= ?'); args.push(f.fromIso); }
if (f.toIso) { where.push('created_at < ?'); args.push(f.toIso); }
if (f.toIso) { where.push('created_at < ?'); args.push(f.toIso); }
if (f.before) { where.push('id < ?'); args.push(f.before); }
if (f.before) { where.push('id < ?'); args.push(f.before); }
- const rows = db.prepare(`SELECT id, status, workflow_key, workflow_version, requester_type, requester_id, provider_key, model_key, fallback_used, attempts,
+ const rows = await db.prepare(`SELECT id, status, workflow_key, workflow_version, requester_type, requester_id, provider_key, model_key, fallback_used, attempts,
tokens_in, tokens_out, cost_usd, error_code, synthetic, cached_from, retry_of, created_at, finished_at
tokens_in, tokens_out, cost_usd, error_code, synthetic, cached_from, retry_of, created_at, finished_at
FROM runs ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, limit + 1);
FROM runs ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, limit + 1);
return { rows: rows.slice(0, limit), next: rows.length > limit ? rows[limit - 1].id : null };
return { rows: rows.slice(0, limit), next: rows.length > limit ? rows[limit - 1].id : null };
}
}
/** Failed runs since `sinceIso`, grouped by error code. */
/** Failed runs since `sinceIso`, grouped by error code. */
-function failures(db, sinceIso) {
- return db.prepare(`SELECT error_code, COUNT(*) AS n, COUNT(DISTINCT workflow_key) AS workflows, MAX(created_at) AS last_at
+async function failures(db, sinceIso) {
+ return await db.prepare(`SELECT error_code, COUNT(*) AS n, COUNT(DISTINCT workflow_key) AS workflows, MAX(created_at) AS last_at
FROM runs WHERE status = 'failed' AND created_at >= ? GROUP BY error_code ORDER BY n DESC LIMIT 20`).all(sinceIso);
FROM runs WHERE status = 'failed' AND created_at >= ? GROUP BY error_code ORDER BY n DESC LIMIT 20`).all(sinceIso);
}
}
-function statusCounts(db, sinceIso) {
- return Object.fromEntries(db.prepare('SELECT status, COUNT(*) AS n FROM runs WHERE created_at >= ? GROUP BY status').all(sinceIso).map(r => [r.status, r.n]));
+async function statusCounts(db, sinceIso) {
+ return Object.fromEntries((await db.prepare('SELECT status, COUNT(*) AS n FROM runs WHERE created_at >= ? GROUP BY status').all(sinceIso)).map(r => [r.status, r.n]));
}
}
/** Suggestions for the run filters (<datalist>): requesters and workflow keys that have runs. */
/** Suggestions for the run filters (<datalist>): requesters and workflow keys that have runs. */
-function runFacets(db) {
+async function runFacets(db) {
return {
return {
- requesters: db.prepare("SELECT DISTINCT requester_type || ':' || requester_id AS r FROM runs ORDER BY r LIMIT 100").all().map(x => x.r),
- workflows: db.prepare('SELECT DISTINCT workflow_key AS w FROM runs ORDER BY w LIMIT 300').all().map(x => x.w),
- codes: db.prepare("SELECT DISTINCT error_code AS c FROM runs WHERE error_code IS NOT NULL ORDER BY c LIMIT 100").all().map(x => x.c),
+ requesters: (await db.prepare("SELECT DISTINCT requester_type || ':' || requester_id AS r FROM runs ORDER BY r LIMIT 100").all()).map(x => x.r),
+ workflows: (await db.prepare('SELECT DISTINCT workflow_key AS w FROM runs ORDER BY w LIMIT 300').all()).map(x => x.w),
+ codes: (await db.prepare("SELECT DISTINCT error_code AS c FROM runs WHERE error_code IS NOT NULL ORDER BY c LIMIT 100").all()).map(x => x.c),
};
};
}
}
@@ -138,7 +138,7 @@ function preview(value) {
@@ -138,7 +138,7 @@ function preview(value) {
const AUDIT_KINDS = Object.freeze({
const AUDIT_KINDS = Object.freeze({
changes: { label: 'Configuration changes', where: "target_type IN ('provider', 'model', 'route', 'template', 'workflow', 'quota', 'cache')" },
changes: { label: 'Configuration changes', where: "target_type IN ('provider', 'model', 'route', 'template', 'workflow', 'quota', 'cache')" },
runs: { label: 'Runs', where: "target_type = 'run'" },
runs: { label: 'Runs', where: "target_type = 'run'" },
- console: { label: 'Console sign-ins and refusals', where: "action LIKE 'console.%'" },
+ console: { label: 'Console sign-ins and refusals', where: "action ILIKE 'console.%'" },
all: { label: 'Everything', where: null },
all: { label: 'Everything', where: null },
});
});
@@ -156,7 +156,7 @@ function parseAuditFilters(q = {}) {
@@ -156,7 +156,7 @@ function parseAuditFilters(q = {}) {
return { filters: f, form, bad };
return { filters: f, form, bad };
}
}
-function listAudit(db, f, limit = 100) {
+async function listAudit(db, f, limit = 100) {
const where = [];
const where = [];
const args = [];
const args = [];
const kind = AUDIT_KINDS[f.kind] || AUDIT_KINDS.changes;
const kind = AUDIT_KINDS[f.kind] || AUDIT_KINDS.changes;
@@ -166,16 +166,16 @@ function listAudit(db, f, limit = 100) {
@@ -166,16 +166,16 @@ function listAudit(db, f, limit = 100) {
if (f.targetType) { where.push('target_type = ?'); args.push(f.targetType); }
if (f.targetType) { where.push('target_type = ?'); args.push(f.targetType); }
if (f.targetId) { where.push('target_id = ?'); args.push(f.targetId); }
if (f.targetId) { where.push('target_id = ?'); args.push(f.targetId); }
if (f.before) { where.push('id < ?'); args.push(f.before); }
if (f.before) { where.push('id < ?'); args.push(f.before); }
- const rows = db.prepare(`SELECT * FROM audit_log ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, limit + 1);
+ const rows = await db.prepare(`SELECT * FROM audit_log ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, limit + 1);
const page = rows.slice(0, limit).map(r => ({ ...r, metadata: parseMeta(r.metadata) }));
const page = rows.slice(0, limit).map(r => ({ ...r, metadata: parseMeta(r.metadata) }));
return { rows: page, next: rows.length > limit ? page[page.length - 1].id : null };
return { rows: page, next: rows.length > limit ? page[page.length - 1].id : null };
}
}
function parseMeta(s) { try { const v = JSON.parse(s || '{}'); return v && typeof v === 'object' ? v : {}; } catch { return {}; } }
function parseMeta(s) { try { const v = JSON.parse(s || '{}'); return v && typeof v === 'object' ? v : {}; } catch { return {}; } }
/** subject -> username, from console sign-ins (the audit log keeps only the subject). */
/** subject -> username, from console sign-ins (the audit log keeps only the subject). */
-function usernames(db) {
+async function usernames(db) {
const map = new Map();
const map = new Map();
- for (const r of db.prepare('SELECT subject, username FROM console_sessions WHERE username IS NOT NULL ORDER BY created_at').all()) map.set(r.subject, r.username);
+ for (const r of await db.prepare('SELECT subject, username FROM console_sessions WHERE username IS NOT NULL ORDER BY created_at').all()) map.set(r.subject, r.username);
return map;
return map;
}
}
@@ -194,8 +194,8 @@ function parseUsageFilters(q = {}, nowMs = Date.now()) {
@@ -194,8 +194,8 @@ function parseUsageFilters(q = {}, nowMs = Date.now()) {
}
}
/** usage_daily rows (quotas.usage, the API's own query) summed four ways. */
/** usage_daily rows (quotas.usage, the API's own query) summed four ways. */
-function usageReport(quotas, f) {
- const rows = quotas.usage({ from: f.from, to: f.to, requester: f.requester });
+async function usageReport(quotas, f) {
+ const rows = await quotas.usage({ from: f.from, to: f.to, requester: f.requester });
const blank = () => ({ requests: 0, tokens_in: 0, tokens_out: 0, tokens_cached: 0, cost_usd: 0 });
const blank = () => ({ requests: 0, tokens_in: 0, tokens_out: 0, tokens_cached: 0, cost_usd: 0 });
const add = (acc, r) => { acc.requests += r.requests; acc.tokens_in += r.tokens_in; acc.tokens_out += r.tokens_out; acc.tokens_cached += r.tokens_cached; acc.cost_usd += r.cost_usd; return acc; };
const add = (acc, r) => { acc.requests += r.requests; acc.tokens_in += r.tokens_in; acc.tokens_out += r.tokens_out; acc.tokens_cached += r.tokens_cached; acc.cost_usd += r.cost_usd; return acc; };
const group = (keyOf) => {
const group = (keyOf) => {
@@ -80,27 +80,27 @@ function createSessions({ db, config, clock, secret }) {
@@ -80,27 +80,27 @@ function createSessions({ db, config, clock, secret }) {
}
}
// ── Sessions ─────────────────────────────────────────────
// ── Sessions ─────────────────────────────────────────────
- function create(res, { subject, username, staff, ip }) {
+ async function create(res, { subject, username, staff, ip }) {
const t = now();
const t = now();
- db.prepare('DELETE FROM console_sessions WHERE expires_at < ?').run(iso(t - 24 * 3600 * 1000));
+ await db.prepare('DELETE FROM console_sessions WHERE expires_at < ?').run(iso(t - 24 * 3600 * 1000));
const id = random(32);
const id = random(32);
const ttlMs = cc.sessionTtlMin * 60 * 1000;
const ttlMs = cc.sessionTtlMin * 60 * 1000;
- db.prepare(`INSERT INTO console_sessions (id_hash, subject, username, staff, csrf, created_at, expires_at, ip_hash)
+ await db.prepare(`INSERT INTO console_sessions (id_hash, subject, username, staff, csrf, created_at, expires_at, ip_hash)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`).run(sha256(id), subject, username || null, JSON.stringify(staff || {}), random(32), iso(t), iso(t + ttlMs), ipHash(ip));
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`).run(sha256(id), subject, username || null, JSON.stringify(staff || {}), random(32), iso(t), iso(t + ttlMs), ipHash(ip));
appendCookie(res, cookieHeader(sessionCookie, id, { maxAgeSec: ttlMs / 1000, secure: cc.cookieSecure }));
appendCookie(res, cookieHeader(sessionCookie, id, { maxAgeSec: ttlMs / 1000, secure: cc.cookieSecure }));
}
}
/** The live session of this request, or null (unknown, expired or revoked). */
/** The live session of this request, or null (unknown, expired or revoked). */
- function read(req) {
+ async function read(req) {
const id = parseCookies(req)[sessionCookie];
const id = parseCookies(req)[sessionCookie];
if (!id || !/^[A-Za-z0-9_-]{43}$/.test(id)) return null;
if (!id || !/^[A-Za-z0-9_-]{43}$/.test(id)) return null;
- const row = db.prepare('SELECT * FROM console_sessions WHERE id_hash = ?').get(sha256(id));
+ const row = await db.prepare('SELECT * FROM console_sessions WHERE id_hash = ?').get(sha256(id));
if (!row || row.revoked_at || Date.parse(row.expires_at) <= now()) return null;
if (!row || row.revoked_at || Date.parse(row.expires_at) <= now()) return null;
return { ...row, staff: parseJson(row.staff, {}) };
return { ...row, staff: parseJson(row.staff, {}) };
}
}
- function revoke(row) {
- if (row) db.prepare('UPDATE console_sessions SET revoked_at = ? WHERE id_hash = ? AND revoked_at IS NULL').run(iso(now()), row.id_hash);
+ async function revoke(row) {
+ if (row) await db.prepare('UPDATE console_sessions SET revoked_at = ? WHERE id_hash = ? AND revoked_at IS NULL').run(iso(now()), row.id_hash);
}
}
function clear(res) {
function clear(res) {
appendCookie(res, cookieHeader(sessionCookie, '', { maxAgeSec: 0, secure: cc.cookieSecure }));
appendCookie(res, cookieHeader(sessionCookie, '', { maxAgeSec: 0, secure: cc.cookieSecure }));
@@ -24,20 +24,7 @@ const SUBJECT_RE = /^usr_[0-9A-HJKMNP-TV-Z]{26}$/;
@@ -24,20 +24,7 @@ const SUBJECT_RE = /^usr_[0-9A-HJKMNP-TV-Z]{26}$/;
function providerKeyFor(owner, subject) { return `byo:${owner}:${subject}`; }
function providerKeyFor(owner, subject) { return `byo:${owner}:${subject}`; }
function createCredentials({ db, config, clock = { now: () => Date.now() } }) {
function createCredentials({ db, config, clock = { now: () => Date.now() } }) {
- db.exec(`CREATE TABLE IF NOT EXISTS subject_credentials (
- owner TEXT NOT NULL,
- subject TEXT NOT NULL,
- provider TEXT NOT NULL CHECK (provider IN ('openai', 'anthropic')),
- base_url TEXT,
- key_enc TEXT NOT NULL,
- key_hint TEXT NOT NULL,
- models TEXT NOT NULL DEFAULT '{}',
- budget_usd_day REAL,
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL,
- last_used_at TEXT,
- PRIMARY KEY (owner, subject)
- )`);
+ // subject_credentials is in migrations/0001_initial.sql (ADR-035).
const iso = (ms) => new Date(ms).toISOString();
const iso = (ms) => new Date(ms).toISOString();
function secretKey() {
function secretKey() {
@@ -61,69 +48,69 @@ function createCredentials({ db, config, clock = { now: () => Date.now() } }) {
@@ -61,69 +48,69 @@ function createCredentials({ db, config, clock = { now: () => Date.now() } }) {
return Buffer.concat([d.update(Buffer.from(body, 'base64')), d.final()]).toString('utf8');
return Buffer.concat([d.update(Buffer.from(body, 'base64')), d.final()]).toString('utf8');
}
}
- function spentToday(owner, subject) {
+ async function spentToday(owner, subject) {
const day = iso(clock.now()).slice(0, 10);
const day = iso(clock.now()).slice(0, 10);
- const r = db.prepare("SELECT COALESCE(SUM(cost_usd), 0) AS c FROM requests WHERE provider_key = ? AND status = 'ok' AND created_at >= ?").get(providerKeyFor(owner, subject), `${day}T00:00:00.000Z`);
+ const r = await db.prepare("SELECT COALESCE(SUM(cost_usd), 0) AS c FROM requests WHERE provider_key = ? AND status = 'ok' AND created_at >= ?").get(providerKeyFor(owner, subject), `${day}T00:00:00.000Z`);
return r ? Number(r.c) || 0 : 0;
return r ? Number(r.c) || 0 : 0;
}
}
- function view(row) {
+ async function view(row) {
return {
return {
subject: row.subject, owner: row.owner, provider: row.provider, base_url: row.base_url || null, key_hint: row.key_hint,
subject: row.subject, owner: row.owner, provider: row.provider, base_url: row.base_url || null, key_hint: row.key_hint,
models: JSON.parse(row.models || '{}'), budget_usd_per_day: row.budget_usd_day == null ? null : row.budget_usd_day,
models: JSON.parse(row.models || '{}'), budget_usd_per_day: row.budget_usd_day == null ? null : row.budget_usd_day,
- spent_usd_today: Math.round(spentToday(row.owner, row.subject) * 1e6) / 1e6,
+ spent_usd_today: Math.round(await spentToday(row.owner, row.subject) * 1e6) / 1e6,
created_at: row.created_at, updated_at: row.updated_at, last_used_at: row.last_used_at || null,
created_at: row.created_at, updated_at: row.updated_at, last_used_at: row.last_used_at || null,
};
};
}
}
- const row = (owner, subject) => db.prepare('SELECT * FROM subject_credentials WHERE owner = ? AND subject = ?').get(owner, subject);
+ const row = async (owner, subject) => await db.prepare('SELECT * FROM subject_credentials WHERE owner = ? AND subject = ?').get(owner, subject);
- function put(owner, subject, body) {
+ async function put(owner, subject, body) {
if (!SUBJECT_RE.test(String(subject))) throw new AiError(400, 'input.invalid', 'subject must be a usr_ subject');
if (!SUBJECT_RE.test(String(subject))) throw new AiError(400, 'input.invalid', 'subject must be a usr_ subject');
const v = validate('ai.credential-put@1', body);
const v = validate('ai.credential-put@1', body);
if (!v.valid) throw new AiError(422, 'input.invalid', 'body does not match ai.credential-put@1', { errors: v.errors });
if (!v.valid) throw new AiError(422, 'input.invalid', 'body does not match ai.credential-put@1', { errors: v.errors });
const at = iso(clock.now());
const at = iso(clock.now());
- const prev = row(owner, subject);
+ const prev = await row(owner, subject);
// The stored key stays for a change of models or budget; a new provider or endpoint needs the key again.
// The stored key stays for a change of models or budget; a new provider or endpoint needs the key again.
const sameEndpoint = prev && prev.provider === body.provider && (prev.base_url || null) === (body.base_url || null);
const sameEndpoint = prev && prev.provider === body.provider && (prev.base_url || null) === (body.base_url || null);
if (body.api_key == null && !sameEndpoint) {
if (body.api_key == null && !sameEndpoint) {
throw new AiError(400, 'credential.key_required', prev ? 'enter the key again to change the provider or its address' : 'api_key is required the first time');
throw new AiError(400, 'credential.key_required', prev ? 'enter the key again to change the provider or its address' : 'api_key is required the first time');
}
}
const key = body.api_key != null ? String(body.api_key) : open(prev.key_enc, `${owner}|${subject}`);
const key = body.api_key != null ? String(body.api_key) : open(prev.key_enc, `${owner}|${subject}`);
- db.prepare(`INSERT INTO subject_credentials (owner, subject, provider, base_url, key_enc, key_hint, models, budget_usd_day, created_at, updated_at)
+ await db.prepare(`INSERT INTO subject_credentials (owner, subject, provider, base_url, key_enc, key_hint, models, budget_usd_day, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(owner, subject) DO UPDATE SET provider = excluded.provider, base_url = excluded.base_url, key_enc = excluded.key_enc,
ON CONFLICT(owner, subject) DO UPDATE SET provider = excluded.provider, base_url = excluded.base_url, key_enc = excluded.key_enc,
key_hint = excluded.key_hint, models = excluded.models, budget_usd_day = excluded.budget_usd_day, updated_at = excluded.updated_at`)
key_hint = excluded.key_hint, models = excluded.models, budget_usd_day = excluded.budget_usd_day, updated_at = excluded.updated_at`)
.run(owner, subject, body.provider, body.base_url || null, seal(key, `${owner}|${subject}`), `…${key.slice(-4)}`, JSON.stringify(body.models || {}),
.run(owner, subject, body.provider, body.base_url || null, seal(key, `${owner}|${subject}`), `…${key.slice(-4)}`, JSON.stringify(body.models || {}),
body.budget_usd_per_day ? Number(body.budget_usd_per_day) : null, prev ? prev.created_at : at, at);
body.budget_usd_per_day ? Number(body.budget_usd_per_day) : null, prev ? prev.created_at : at, at);
- return view(row(owner, subject));
+ return await view(await row(owner, subject));
}
}
- function get(owner, subject) {
- const r = row(owner, subject);
+ async function get(owner, subject) {
+ const r = await row(owner, subject);
if (!r) throw new AiError(404, 'credential.not_found', 'no credential stored for that subject');
if (!r) throw new AiError(404, 'credential.not_found', 'no credential stored for that subject');
- return view(r);
+ return await view(r);
}
}
- function remove(owner, subject) {
- return db.prepare('DELETE FROM subject_credentials WHERE owner = ? AND subject = ?').run(owner, subject).changes > 0;
+ async function remove(owner, subject) {
+ return (await db.prepare('DELETE FROM subject_credentials WHERE owner = ? AND subject = ?').run(owner, subject)).changes > 0;
}
}
/** Checks a run may use the credential (exists, within budget); throws AiError otherwise. */
/** Checks a run may use the credential (exists, within budget); throws AiError otherwise. */
- function admit(owner, subject) {
- const r = row(owner, subject);
+ async function admit(owner, subject) {
+ const r = await row(owner, subject);
if (!r) throw new AiError(404, 'credential.not_found', 'no credential stored for that subject');
if (!r) throw new AiError(404, 'credential.not_found', 'no credential stored for that subject');
secretKey();
secretKey();
- if (r.budget_usd_day != null && r.budget_usd_day > 0 && spentToday(owner, subject) >= r.budget_usd_day) {
+ if (r.budget_usd_day != null && r.budget_usd_day > 0 && await spentToday(owner, subject) >= r.budget_usd_day) {
const tomorrow = new Date(iso(clock.now()).slice(0, 10) + 'T00:00:00.000Z').getTime() + 86400000;
const tomorrow = new Date(iso(clock.now()).slice(0, 10) + 'T00:00:00.000Z').getTime() + 86400000;
throw new AiError(429, 'quota.exceeded', "this person's own daily budget for their key is spent", { retry_after_seconds: Math.max(60, Math.ceil((tomorrow - clock.now()) / 1000)), quota: { scope: 'credential' } });
throw new AiError(429, 'quota.exceeded', "this person's own daily budget for their key is spent", { retry_after_seconds: Math.max(60, Math.ceil((tomorrow - clock.now()) / 1000)), quota: { scope: 'credential' } });
}
}
}
}
/** The decrypted credential for one execution: { providerKey, provider, base_url, apiKey, models }. */
/** The decrypted credential for one execution: { providerKey, provider, base_url, apiKey, models }. */
- function forRun(owner, subject) {
- const r = row(owner, subject);
+ async function forRun(owner, subject) {
+ const r = await row(owner, subject);
if (!r) throw new AiError(404, 'credential.not_found', 'the credential was deleted');
if (!r) throw new AiError(404, 'credential.not_found', 'the credential was deleted');
- db.prepare('UPDATE subject_credentials SET last_used_at = ? WHERE owner = ? AND subject = ?').run(iso(clock.now()), owner, subject);
+ await db.prepare('UPDATE subject_credentials SET last_used_at = ? WHERE owner = ? AND subject = ?').run(iso(clock.now()), owner, subject);
return { providerKey: providerKeyFor(owner, subject), provider: r.provider, base_url: r.base_url || null, apiKey: open(r.key_enc, `${owner}|${subject}`), models: JSON.parse(r.models || '{}') };
return { providerKey: providerKeyFor(owner, subject), provider: r.provider, base_url: r.base_url || null, apiKey: open(r.key_enc, `${owner}|${subject}`), models: JSON.parse(r.models || '{}') };
}
}
@@ -139,21 +126,21 @@ function credentialsRouter({ credentials, auth, registry, sendError }) {
@@ -139,21 +126,21 @@ function credentialsRouter({ credentials, auth, registry, sendError }) {
if (!s || s.type !== 'service') throw new AiError(403, 'capability.denied', 'only a service stores credentials for the people it serves');
if (!s || s.type !== 'service') throw new AiError(403, 'capability.denied', 'only a service stores credentials for the people it serves');
return s.id;
return s.id;
};
};
- r.put('/api/v1/credentials/:subject', guard, (req, res) => {
+ r.put('/api/v1/credentials/:subject', guard, async (req, res) => {
try {
try {
- const out = credentials.put(owner(req), req.params.subject, req.body || {});
- registry.audit(req.principal.sub, 'credential.put', 'credential', `${out.owner}:${out.subject}`, { trace: req.ov && req.ov.traceId, metadata: { provider: out.provider, base_url: out.base_url } });
+ const out = await credentials.put(owner(req), req.params.subject, req.body || {});
+ await registry.audit(req.principal.sub, 'credential.put', 'credential', `${out.owner}:${out.subject}`, { trace: req.ov && req.ov.traceId, metadata: { provider: out.provider, base_url: out.base_url } });
res.set('Cache-Control', 'no-store').json(out);
res.set('Cache-Control', 'no-store').json(out);
} catch (err) { sendError(res, err, req.ov); }
} catch (err) { sendError(res, err, req.ov); }
});
});
- r.get('/api/v1/credentials/:subject', guard, (req, res) => {
- try { res.set('Cache-Control', 'no-store').json(credentials.get(owner(req), req.params.subject)); } catch (err) { sendError(res, err, req.ov); }
+ r.get('/api/v1/credentials/:subject', guard, async (req, res) => {
+ try { res.set('Cache-Control', 'no-store').json(await credentials.get(owner(req), req.params.subject)); } catch (err) { sendError(res, err, req.ov); }
});
});
- r.delete('/api/v1/credentials/:subject', guard, (req, res) => {
+ r.delete('/api/v1/credentials/:subject', guard, async (req, res) => {
try {
try {
const o = owner(req);
const o = owner(req);
- const gone = credentials.remove(o, req.params.subject);
- if (gone) registry.audit(req.principal.sub, 'credential.delete', 'credential', `${o}:${req.params.subject}`, { trace: req.ov && req.ov.traceId });
+ const gone = await credentials.remove(o, req.params.subject);
+ if (gone) await registry.audit(req.principal.sub, 'credential.delete', 'credential', `${o}:${req.params.subject}`, { trace: req.ov && req.ov.traceId });
res.status(gone ? 204 : 404).end();
res.status(gone ? 204 : 404).end();
} catch (err) { sendError(res, err, req.ov); }
} catch (err) { sendError(res, err, req.ov); }
});
});
@@ -1,6 +1,6 @@
@@ -1,6 +1,6 @@
'use strict';
'use strict';
/**
/**
- * The AI authority's SQLite database (better-sqlite3). One table family per canonical record
+ * The AI authority's PostgreSQL database (ADR-035; migrations/NNNN_*.sql). One table family per canonical record
* group of the plan (§12.13 / §15.14):
* group of the plan (§12.13 / §15.14):
*
*
* 1 providers (+ provider_health) 6 runs
* 1 providers (+ provider_health) 6 runs
@@ -17,311 +17,28 @@
@@ -17,311 +17,28 @@
*/
*/
const fs = require('fs');
const fs = require('fs');
const path = require('path');
const path = require('path');
-const Database = require('better-sqlite3');
+const { createDb } = require('openvibe-sdk/db');
-const SCHEMA = `
-CREATE TABLE IF NOT EXISTS providers (
- key TEXT PRIMARY KEY,
- display_name TEXT NOT NULL,
- kind TEXT NOT NULL, -- stub | openai | anthropic | http | whisper
- status TEXT NOT NULL DEFAULT 'active', -- active | disabled
- base_url TEXT,
- auth_mode TEXT NOT NULL DEFAULT 'none', -- none | bearer | x-api-key
- secret_ref TEXT, -- 'env:NAME' — never a value
- default_model TEXT,
- capabilities TEXT NOT NULL DEFAULT '[]', -- features supports() answers true for
- timeout_ms INTEGER NOT NULL DEFAULT 30000,
- priority INTEGER NOT NULL DEFAULT 100,
- metadata TEXT NOT NULL DEFAULT '{}',
- origin TEXT NOT NULL DEFAULT 'seed', -- seed (from env at boot) | admin | import
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL
-);
-CREATE TABLE IF NOT EXISTS provider_health (
- provider_key TEXT PRIMARY KEY,
- state TEXT NOT NULL DEFAULT 'closed', -- closed (healthy) | open (skipped) | half_open (one probe)
- consecutive_failures INTEGER NOT NULL DEFAULT 0,
- opened_at INTEGER,
- last_error TEXT,
- last_success_at INTEGER,
- last_failure_at INTEGER
-);
-CREATE TABLE IF NOT EXISTS models (
- provider_key TEXT NOT NULL,
- model_key TEXT NOT NULL,
- display_name TEXT,
- type TEXT NOT NULL DEFAULT 'chat', -- chat | vision | embedding | stt
- status TEXT NOT NULL DEFAULT 'active',
- context_window INTEGER,
- max_output INTEGER,
- cost_in_per_mtok REAL,
- cost_out_per_mtok REAL,
- cost_cached_per_mtok REAL,
- supports_json INTEGER NOT NULL DEFAULT 1,
- supports_tools INTEGER NOT NULL DEFAULT 0,
- supports_streaming INTEGER NOT NULL DEFAULT 0,
- supports_vision INTEGER NOT NULL DEFAULT 0,
- metadata TEXT NOT NULL DEFAULT '{}',
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL,
- PRIMARY KEY (provider_key, model_key)
-);
-CREATE TABLE IF NOT EXISTS routes (
- key TEXT NOT NULL,
- version INTEGER NOT NULL,
- status TEXT NOT NULL DEFAULT 'active', -- active | disabled
- primary_provider TEXT NOT NULL,
- primary_model TEXT,
- fallbacks TEXT NOT NULL DEFAULT '[]', -- [{provider, model}]
- options TEXT NOT NULL DEFAULT '{}', -- temperature and friends
- max_output_tokens INTEGER,
- response_format TEXT NOT NULL DEFAULT 'text', -- text | json
- timeout_ms INTEGER,
- alias_of TEXT, -- historical key kept as an explicit alias
- created_by TEXT,
- created_at TEXT NOT NULL,
- PRIMARY KEY (key, version)
-);
-CREATE TABLE IF NOT EXISTS templates (
- key TEXT NOT NULL,
- version INTEGER NOT NULL,
- name TEXT NOT NULL,
- description TEXT,
- input_schema TEXT NOT NULL DEFAULT '{}',
- output_schema TEXT NOT NULL DEFAULT '{}',
- system_prompt TEXT NOT NULL DEFAULT '',
- user_prompt TEXT NOT NULL DEFAULT '',
- default_route TEXT,
- owner TEXT NOT NULL DEFAULT 'ai',
- visibility TEXT NOT NULL DEFAULT 'internal', -- public | first-party | internal
- status TEXT NOT NULL DEFAULT 'active', -- draft | active | deprecated | archived
- metadata TEXT NOT NULL DEFAULT '{}',
- created_by TEXT,
- created_at TEXT NOT NULL,
- PRIMARY KEY (key, version)
-);
-CREATE TABLE IF NOT EXISTS workflows (
- key TEXT NOT NULL,
- version INTEGER NOT NULL,
- name TEXT NOT NULL,
- description TEXT,
- namespace TEXT NOT NULL, -- owning product: live, network, wiki, ...
- input_schema TEXT NOT NULL,
- output_schema TEXT NOT NULL,
- steps TEXT NOT NULL, -- [{kind, template?, operation?, ...}]
- default_route TEXT,
- cache_mode TEXT NOT NULL DEFAULT 'private', -- none | private | service
- cache_ttl_sec INTEGER,
- status TEXT NOT NULL DEFAULT 'active', -- draft | active | deprecated | archived
- metadata TEXT NOT NULL DEFAULT '{}',
- created_by TEXT,
- created_at TEXT NOT NULL,
- PRIMARY KEY (key, version)
-);
-CREATE TABLE IF NOT EXISTS runs (
- id TEXT PRIMARY KEY, -- run_<ULID>
- workflow_key TEXT NOT NULL,
- workflow_version INTEGER NOT NULL,
- template_key TEXT,
- template_version INTEGER,
- route_key TEXT,
- route_version INTEGER,
- status TEXT NOT NULL, -- queued | running | succeeded | failed | cancelled | cached
- requester_type TEXT NOT NULL, -- service | app | mod (the token principal)
- requester_id TEXT NOT NULL,
- on_behalf_of TEXT, -- SubjectRef JSON (optional)
- attribution TEXT, -- EntityRef JSON the spend is attributed to (optional)
- source_service TEXT,
- target TEXT, -- EntityRef JSON (optional)
- target_key TEXT, -- service:type:id for lookups
- input TEXT NOT NULL,
- input_hash TEXT NOT NULL,
- output TEXT,
- citations_count INTEGER NOT NULL DEFAULT 0,
- error_code TEXT,
- error_detail TEXT,
- synthetic INTEGER NOT NULL DEFAULT 0, -- produced by the stub provider
- provider_key TEXT,
- model_key TEXT,
- fallback_used INTEGER NOT NULL DEFAULT 0,
- attempts INTEGER NOT NULL DEFAULT 0,
- tokens_in INTEGER NOT NULL DEFAULT 0,
- tokens_out INTEGER NOT NULL DEFAULT 0,
- cost_usd REAL NOT NULL DEFAULT 0,
- cache_key TEXT,
- cached_from TEXT, -- run id whose output a cached run reused
- retry_of TEXT,
- idempotency_key TEXT,
- trace_id TEXT,
- request_id TEXT,
- options TEXT NOT NULL DEFAULT '{}',
- created_at TEXT NOT NULL,
- started_at TEXT,
- finished_at TEXT
-);
-CREATE UNIQUE INDEX IF NOT EXISTS idx_runs_idem ON runs(requester_type, requester_id, idempotency_key) WHERE idempotency_key IS NOT NULL;
-CREATE INDEX IF NOT EXISTS idx_runs_status ON runs(status, created_at);
-CREATE INDEX IF NOT EXISTS idx_runs_workflow ON runs(workflow_key, created_at);
-CREATE INDEX IF NOT EXISTS idx_runs_target ON runs(target_key, created_at);
-CREATE INDEX IF NOT EXISTS idx_runs_requester ON runs(requester_type, requester_id, created_at);
-CREATE TABLE IF NOT EXISTS requests (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- run_id TEXT,
- seq INTEGER NOT NULL DEFAULT 0,
- operation TEXT NOT NULL, -- chat | generate | summarize | classify | extract | enrich | embed | transcribe
- provider_key TEXT NOT NULL,
- model_key TEXT,
- route_key TEXT,
- route_version INTEGER,
- status TEXT NOT NULL, -- ok | error | timeout | skipped | cancelled
- skip_reason TEXT, -- disabled | circuit_open | unsupported | no_credentials
- fallback INTEGER NOT NULL DEFAULT 0, -- 1 when this was not the route's primary
- prompt_hash TEXT,
- input_hash TEXT,
- output_hash TEXT,
- tokens_in INTEGER NOT NULL DEFAULT 0,
- tokens_out INTEGER NOT NULL DEFAULT 0,
- tokens_cached INTEGER NOT NULL DEFAULT 0,
- tokens_estimated INTEGER NOT NULL DEFAULT 0,
- cost_usd REAL NOT NULL DEFAULT 0,
- latency_ms INTEGER,
- error TEXT,
- debug_prompt TEXT, -- only with AI_DEBUG_RAW_LOG and a caller opt-in
- debug_response TEXT,
- created_at TEXT NOT NULL
-);
-CREATE INDEX IF NOT EXISTS idx_requests_run ON requests(run_id, seq);
-CREATE INDEX IF NOT EXISTS idx_requests_provider ON requests(provider_key, created_at);
-CREATE TABLE IF NOT EXISTS citations (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- run_id TEXT NOT NULL,
- ordinal INTEGER NOT NULL DEFAULT 0,
- source_type TEXT NOT NULL,
- source_id TEXT,
- url TEXT,
- title TEXT,
- author TEXT,
- published_at TEXT,
- retrieved_at TEXT,
- snippet TEXT,
- content_hash TEXT,
- trust TEXT NOT NULL DEFAULT '{}',
- provenance TEXT NOT NULL DEFAULT '{}',
- attached_by TEXT, -- workflow | <principal sub>
- created_at TEXT NOT NULL
-);
-CREATE INDEX IF NOT EXISTS idx_citations_run ON citations(run_id, ordinal);
-CREATE TABLE IF NOT EXISTS cache_entries (
- cache_key TEXT PRIMARY KEY, -- sha256 over everything below
- scope TEXT NOT NULL, -- who may read it (see server/cache.js)
- workflow_key TEXT NOT NULL,
- workflow_version INTEGER NOT NULL,
- template_version INTEGER,
- route_key TEXT,
- route_version INTEGER,
- model_key TEXT,
- input_hash TEXT NOT NULL,
- privacy TEXT NOT NULL, -- private | service
- output TEXT NOT NULL,
- run_id TEXT NOT NULL,
- hits INTEGER NOT NULL DEFAULT 0,
- created_at TEXT NOT NULL,
- expires_at INTEGER NOT NULL
-);
-CREATE INDEX IF NOT EXISTS idx_cache_expiry ON cache_entries(expires_at);
-CREATE INDEX IF NOT EXISTS idx_cache_workflow ON cache_entries(workflow_key);
-CREATE TABLE IF NOT EXISTS quotas (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- scope_type TEXT NOT NULL, -- global | service | actor | attribution
- scope_id TEXT NOT NULL, -- '*' | live | user:usr_… | live:user:42
- window TEXT NOT NULL, -- minute | hour | day
- max_requests INTEGER,
- max_tokens INTEGER,
- max_cost_usd REAL,
- workflow_prefix TEXT, -- optional: only runs of workflows with this prefix
- status TEXT NOT NULL DEFAULT 'active',
- origin TEXT NOT NULL DEFAULT 'admin', -- seed | admin | import
- created_at TEXT NOT NULL,
- updated_at TEXT NOT NULL
-);
-CREATE UNIQUE INDEX IF NOT EXISTS idx_quotas_unique ON quotas(scope_type, scope_id, window, COALESCE(workflow_prefix, ''));
-CREATE TABLE IF NOT EXISTS usage_counters (
- scope_type TEXT NOT NULL,
- scope_id TEXT NOT NULL,
- window TEXT NOT NULL,
- window_start INTEGER NOT NULL, -- epoch seconds, UTC
- workflow_prefix TEXT NOT NULL DEFAULT '',
- requests INTEGER NOT NULL DEFAULT 0,
- tokens INTEGER NOT NULL DEFAULT 0,
- cost_usd REAL NOT NULL DEFAULT 0,
- PRIMARY KEY (scope_type, scope_id, window, window_start, workflow_prefix)
-);
-CREATE TABLE IF NOT EXISTS usage_daily (
- day TEXT NOT NULL, -- YYYY-MM-DD (UTC)
- requester TEXT NOT NULL, -- service:live
- attribution TEXT NOT NULL DEFAULT '', -- live:user:42
- workflow_key TEXT NOT NULL,
- provider_key TEXT NOT NULL,
- model_key TEXT NOT NULL DEFAULT '',
- requests INTEGER NOT NULL DEFAULT 0,
- tokens_in INTEGER NOT NULL DEFAULT 0,
- tokens_out INTEGER NOT NULL DEFAULT 0,
- tokens_cached INTEGER NOT NULL DEFAULT 0,
- cost_usd REAL NOT NULL DEFAULT 0,
- PRIMARY KEY (day, requester, attribution, workflow_key, provider_key, model_key)
-);
-CREATE TABLE IF NOT EXISTS audit_log (
- id INTEGER PRIMARY KEY AUTOINCREMENT,
- at TEXT NOT NULL,
- actor TEXT NOT NULL, -- principal sub (svc:live) or 'system'
- action TEXT NOT NULL, -- run.create | template.version | provider.update | ...
- target_type TEXT,
- target_id TEXT,
- trace_id TEXT,
- metadata TEXT NOT NULL DEFAULT '{}'
-);
-CREATE INDEX IF NOT EXISTS idx_audit_target ON audit_log(target_type, target_id, id);
-CREATE INDEX IF NOT EXISTS idx_audit_action ON audit_log(action, id);
-CREATE TABLE IF NOT EXISTS console_sessions (
- id_hash TEXT PRIMARY KEY, -- sha256 of the session cookie; the cookie value is never stored
- subject TEXT NOT NULL, -- the Network person (usr_…)
- username TEXT,
- staff TEXT NOT NULL DEFAULT '{}', -- the sign-in token's staff claims (role, is_owner, staff_caps, staff_map)
- csrf TEXT NOT NULL,
- created_at TEXT NOT NULL,
- expires_at TEXT NOT NULL,
- revoked_at TEXT,
- ip_hash TEXT -- keyed hash of the client address
-);
-CREATE TABLE IF NOT EXISTS import_ledger (
- source TEXT NOT NULL, -- live.ai_usage, live.translations, ...
- source_key TEXT NOT NULL,
- target TEXT NOT NULL, -- where it went (table:key)
- imported_at TEXT NOT NULL,
- PRIMARY KEY (source, source_key)
-);
-CREATE TABLE IF NOT EXISTS import_holds (
- source TEXT NOT NULL,
- source_key TEXT NOT NULL,
- reason TEXT NOT NULL,
- detail TEXT,
- held_at TEXT NOT NULL,
- PRIMARY KEY (source, source_key)
-);
-`;
+const MIGRATIONS = path.join(__dirname, '..', 'migrations');
+const DEV_PGLITE = path.join(__dirname, '..', 'data', 'pglite');
-function openDb(dbPath) {
- if (dbPath !== ':memory:') fs.mkdirSync(path.dirname(path.resolve(dbPath)), { recursive: true });
- const db = new Database(dbPath);
- db.pragma('journal_mode = WAL');
- db.pragma('synchronous = NORMAL');
- db.pragma('foreign_keys = ON');
- db.pragma('busy_timeout = 5000');
- db.exec(SCHEMA);
- // Added after the first release: ADD COLUMN only, so an older build ignores it.
- const cols = new Set(db.prepare("SELECT name FROM pragma_table_info('runs')").all().map((c) => c.name));
- if (!cols.has('grounding')) db.exec('ALTER TABLE runs ADD COLUMN grounding TEXT'); // { cited, gaps } (WS-O task 3)
- return db;
+/**
+ * The serving handle (ADR-035): DATABASE_URL through PgBouncer; in development without it, an embedded PGlite database
+ * in data/pglite. Migrations run first, as the owner (DATABASE_DIRECT_URL), or on the embedded handle.
+ */
+async function openDb(config, { log = console, registry } = {}) {
+ if (!config.db.url) {
+ if (config.isProduction) throw new Error('DATABASE_URL is not set: production serves from PostgreSQL (OpenVibe.Host roles/data add-service.sh ai)');
+ log.warn(`[AI] DATABASE_URL unset: embedded PGlite database in ${DEV_PGLITE} (development only, one process)`);
+ fs.mkdirSync(DEV_PGLITE, { recursive: true });
+ const db = createDb({ pglite: DEV_PGLITE, service: 'ai', registry, log });
+ await db.migrate({ dir: MIGRATIONS, log });
+ return db;
+ }
+ if (!config.db.directUrl) throw new Error('DATABASE_DIRECT_URL is not set: migrations run with the owner role on a direct connection');
+ const owner = createDb({ url: config.db.directUrl, service: 'ai-migrate', max: 1, log });
+ try { await owner.migrate({ dir: MIGRATIONS, log }); } finally { await owner.close(); }
+ return createDb({ url: config.db.url, service: 'ai', registry, log });
}
}
-module.exports = { openDb, SCHEMA };
+module.exports = { openDb, MIGRATIONS };
@@ -13,9 +13,10 @@
@@ -13,9 +13,10 @@
*/
*/
const { createClient } = require('openvibe-sdk/core');
const { createClient } = require('openvibe-sdk/core');
const { createServiceTokenClient } = require('openvibe-sdk/auth');
const { createServiceTokenClient } = require('openvibe-sdk/auth');
-const { createEventsClient, createOutbox } = require('openvibe-sdk/events');
+const { createEventsClient, createPgOutbox } = require('openvibe-sdk/events');
let outbox = null;
let outbox = null;
+let dbRef = null; // the handle enqueue joins the running transaction through (ADR-035)
let pruneTimer = null;
let pruneTimer = null;
const stats = { queued: 0, lastError: null };
const stats = { queued: 0, lastError: null };
const PRUNE_EVERY_MS = 6 * 60 * 60 * 1000;
const PRUNE_EVERY_MS = 6 * 60 * 60 * 1000;
@@ -23,15 +24,16 @@ const PRUNE_EVERY_MS = 6 * 60 * 60 * 1000;
@@ -23,15 +24,16 @@ const PRUNE_EVERY_MS = 6 * 60 * 60 * 1000;
function init(db, { eventsUrl = process.env.EVENTS_URL, clientSecret = process.env.OV_OAUTH_CLIENT_SECRET, clientId = process.env.OV_OAUTH_CLIENT_ID || 'ai',
function init(db, { eventsUrl = process.env.EVENTS_URL, clientSecret = process.env.OV_OAUTH_CLIENT_SECRET, clientId = process.env.OV_OAUTH_CLIENT_ID || 'ai',
networkUrl = process.env.OV_NETWORK_INTERNAL_URL || 'http://127.0.0.1:4000', fetchImpl, intervalMs, log = console } = {}) {
networkUrl = process.env.OV_NETWORK_INTERNAL_URL || 'http://127.0.0.1:4000', fetchImpl, intervalMs, log = console } = {}) {
if (outbox) return outbox;
if (outbox) return outbox;
+ dbRef = db;
if (process.env.EVENTS_PUBLISH === 'off' || !eventsUrl || !clientSecret) return null;
if (process.env.EVENTS_PUBLISH === 'off' || !eventsUrl || !clientSecret) return null;
const tokens = createServiceTokenClient({ tokenUrl: `${String(networkUrl).replace(/\/+$/, '')}/oauth/token`, clientId, clientSecret, fetch: fetchImpl });
const tokens = createServiceTokenClient({ tokenUrl: `${String(networkUrl).replace(/\/+$/, '')}/oauth/token`, clientId, clientSecret, fetch: fetchImpl });
const client = createClient({ baseUrls: { events: String(eventsUrl).replace(/\/+$/, '') }, tokenProvider: tokens, fetch: fetchImpl, retries: 0 });
const client = createClient({ baseUrls: { events: String(eventsUrl).replace(/\/+$/, '') }, tokenProvider: tokens, fetch: fetchImpl, retries: 0 });
- outbox = createOutbox(db, {
+ // The PostgreSQL outbox (ADR-035): rows are written in the change's own transaction (enqueue(db, …) joins it).
+ outbox = createPgOutbox(db, {
events: createEventsClient(client, { source: 'ai' }),
events: createEventsClient(client, { source: 'ai' }),
intervalMs: intervalMs || 2000,
intervalMs: intervalMs || 2000,
onError: (err) => { const m = err && err.message; if (m !== stats.lastError) log.warn('[Events] publish failed (will retry):', m); stats.lastError = m; },
onError: (err) => { const m = err && err.message; if (m !== stats.lastError) log.warn('[Events] publish failed (will retry):', m); stats.lastError = m; },
});
});
- outbox.ensureSchema();
outbox.start();
outbox.start();
pruneTimer = setInterval(() => { try { outbox.prune(); } catch { /* next time */ } }, PRUNE_EVERY_MS);
pruneTimer = setInterval(() => { try { outbox.prune(); } catch { /* next time */ } }, PRUNE_EVERY_MS);
if (pruneTimer.unref) pruneTimer.unref();
if (pruneTimer.unref) pruneTimer.unref();
@@ -76,7 +78,7 @@ function payloadOf(row) {
@@ -76,7 +78,7 @@ function payloadOf(row) {
function runChanged(row) {
function runChanged(row) {
if (!outbox || !row || !['queued', 'cached', 'succeeded', 'failed'].includes(row.status)) return null;
if (!outbox || !row || !['queued', 'cached', 'succeeded', 'failed'].includes(row.status)) return null;
const requester = { type: row.requester_type, id: row.requester_id };
const requester = { type: row.requester_type, id: row.requester_id };
- const env = outbox.enqueue({
+ const env = outbox.enqueue(dbRef, {
event_type: `ai.run.${row.status}`,
event_type: `ai.run.${row.status}`,
actor: requester.type === 'user' || requester.type === 'service' ? requester : { type: 'service', id: 'ai' },
actor: requester.type === 'user' || requester.type === 'service' ? requester : { type: 'service', id: 'ai' },
subject: { type: 'run', id: row.id },
subject: { type: 'run', id: row.id },
@@ -85,13 +87,14 @@ function runChanged(row) {
@@ -85,13 +87,14 @@ function runChanged(row) {
payload: payloadOf(row),
payload: payloadOf(row),
});
});
stats.queued++;
stats.queued++;
- setImmediate(() => outbox && outbox.kick());
+ // Relay once the change (and its event) committed.
+ dbRef.afterCommit(() => outbox && outbox.kick());
return env;
return env;
}
}
-function status() {
+async function status() {
if (!outbox) return { enabled: false };
if (!outbox) return { enabled: false };
- return { enabled: true, pending: outbox.pending(), rejected: outbox.rejected(), queued_since_boot: stats.queued, last_error: stats.lastError };
+ return { enabled: true, pending: await outbox.pending(), rejected: await outbox.rejected(), queued_since_boot: stats.queued, last_error: stats.lastError };
}
}
/**
/**
* Graceful stop (start().close(), after runs.drain()): the relay stops and nothing more is queued; resolves
* Graceful stop (start().close(), after runs.drain()): the relay stops and nothing more is queued; resolves
@@ -253,7 +253,7 @@ function createFetcher(config, { transport = null } = {}) {
@@ -253,7 +253,7 @@ function createFetcher(config, { transport = null } = {}) {
type = r.contentType.split(';')[0].trim();
type = r.contentType.split(';')[0].trim();
if (!/^image\//i.test(type)) throw new AiError(422, 'source.invalid', `expected an image, got ${type || 'unknown type'}`);
if (!/^image\//i.test(type)) throw new AiError(422, 'source.invalid', `expected an image, got ${type || 'unknown type'}`);
}
}
- return downscale(buf, type, input.max_width || 1024);
+ return await downscale(buf, type, input.max_width || 1024);
}
}
/** Media input -> a temp file path (caller deletes it). */
/** Media input -> a temp file path (caller deletes it). */
@@ -21,10 +21,11 @@ const { createKeyStore, createAuth } = require('./auth');
@@ -21,10 +21,11 @@ const { createKeyStore, createAuth } = require('./auth');
const schemas = require('./schemas');
const schemas = require('./schemas');
const { createApp } = require('./app');
const { createApp } = require('./app');
-async function start({ config, clock = { now: () => Date.now() }, fetchImpl = globalThis.fetch, env = process.env, log = console, listen = true, credentialFetch = null } = {}) {
+async function start({ config, db: givenDb = null, clock = { now: () => Date.now() }, fetchImpl = globalThis.fetch, env = process.env, log = console, listen = true, credentialFetch = null } = {}) {
config = config || load(env);
config = config || load(env);
schemas.configure(config.schemaCache);
schemas.configure(config.schemaCache);
- const db = openDb(config.dbPath);
+ // PostgreSQL (ADR-035): opened and migrated here unless the caller (a test) hands in a migrated handle.
+ const db = givenDb || await openDb(config, { log });
const registry = createRegistry(db, { clock, env });
const registry = createRegistry(db, { clock, env });
const quotas = createQuotas(db, { clock, registry });
const quotas = createQuotas(db, { clock, registry });
const cache = createCache(db, { clock });
const cache = createCache(db, { clock });
@@ -39,8 +40,8 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = gl
@@ -39,8 +40,8 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = gl
// A person's own provider keys (WS-O task 2): stored by the service holding their consent, used by their runs only.
// A person's own provider keys (WS-O task 2): stored by the service holding their consent, used by their runs only.
const credentials = require('./credentials').createCredentials({ db, config, clock });
const credentials = require('./credentials').createCredentials({ db, config, clock });
const runs = createRuns({ db, registry, engine, cache, quotas, config, clock, log, userModules, credentials });
const runs = createRuns({ db, registry, engine, cache, quotas, config, clock, log, userModules, credentials });
- seed({ registry, quotas, config, env, db });
- const interrupted = runs.recoverInterrupted();
+ await seed({ registry, quotas, config, env, db });
+ const interrupted = await runs.recoverInterrupted();
if (interrupted) log.warn(`[ai] marked ${interrupted} interrupted run(s) failed (run.interrupted); callers can retry them`);
if (interrupted) log.warn(`[ai] marked ${interrupted} interrupted run(s) failed (run.interrupted); callers can retry them`);
const keys = createKeyStore({ urls: [config.networkInternalUrl, config.networkUrl], pem: config.networkPublicKey, fetchImpl, log });
const keys = createKeyStore({ urls: [config.networkInternalUrl, config.networkUrl], pem: config.networkPublicKey, fetchImpl, log });
@@ -48,10 +49,10 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = gl
@@ -48,10 +49,10 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = gl
const app = createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env, clock, fetchImpl, log, credentials });
const app = createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env, clock, fetchImpl, log, credentials });
const keyLoaded = keys.start().catch(() => null);
const keyLoaded = keys.start().catch(() => null);
- const housekeeping = setInterval(() => {
+ const housekeeping = setInterval(async () => {
try {
try {
- const pruned = runs.prune();
- const expired = cache.prune();
+ const pruned = await runs.prune();
+ const expired = await cache.prune();
if (pruned || expired) log.log(`[ai] retention: ${pruned} old run(s), ${expired} expired cache entr${expired === 1 ? 'y' : 'ies'}`);
if (pruned || expired) log.log(`[ai] retention: ${pruned} old run(s), ${expired} expired cache entr${expired === 1 ? 'y' : 'ies'}`);
} catch (err) { log.error(`[ai] retention failed: ${err.message}`); }
} catch (err) { log.error(`[ai] retention failed: ${err.message}`); }
}, 60 * 60 * 1000);
}, 60 * 60 * 1000);
@@ -68,7 +69,7 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = gl
@@ -68,7 +69,7 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = gl
server.headersTimeout = 66000;
server.headersTimeout = 66000;
server.requestTimeout = config.runs.maxWaitMs + 30000;
server.requestTimeout = config.runs.maxWaitMs + 30000;
log.log(`[ai] listening on http://${config.host}:${server.address().port} (${config.nodeEnv})`);
log.log(`[ai] listening on http://${config.host}:${server.address().port} (${config.nodeEnv})`);
- const shared = registry.getProvider('shared');
+ const shared = await registry.getProvider('shared');
log.log(`[ai] shared provider: ${shared.kind} ${shared.status}, credentials ${registry.publicProvider(shared).credentials}; stub fallback ${config.stubFallback ? 'on' : 'off'}`);
log.log(`[ai] shared provider: ${shared.kind} ${shared.status}, credentials ${registry.publicProvider(shared).credentials}; stub fallback ${config.stubFallback ? 'on' : 'off'}`);
}
}
@@ -82,9 +83,9 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = gl
@@ -82,9 +83,9 @@ async function start({ config, clock = { now: () => Date.now() }, fetchImpl = gl
}
}
await runs.drain();
await runs.drain();
await require('./events').stop(); // after the runs: their last ai.run.* rows are queued first
await require('./events').stop(); // after the runs: their last ai.run.* rows are queued first
- const w = pool.adapter('whisper');
+ const w = await pool.adapter('whisper');
if (w && w.adapter.killActive) w.adapter.killActive();
if (w && w.adapter.killActive) w.adapter.killActive();
- db.close();
+ if (!givenDb) await db.close();
}
}
return { config, db, registry, pool, quotas, cache, fetcher, engine, runs, keys, keyLoaded, auth, app, server, close };
return { config, db, registry, pool, quotas, cache, fetcher, engine, runs, keys, keyLoaded, auth, app, server, close };
@@ -20,11 +20,11 @@ const { createReadiness } = require('openvibe-shared/ready');
@@ -20,11 +20,11 @@ const { createReadiness } = require('openvibe-shared/ready');
const CIRCUIT_STATES = ['closed', 'half_open', 'open'];
const CIRCUIT_STATES = ['closed', 'half_open', 'open'];
/** Active providers with what they need to answer: credentials and circuit state. */
/** Active providers with what they need to answer: credentials and circuit state. */
-function providerView(registry, pool) {
- return registry.listProviders().filter(p => p.status === 'active').map((p) => {
+async function providerView(registry, pool) {
+ return (await Promise.all((await registry.listProviders()).filter(p => p.status === 'active').map(async (p) => {
const pub = registry.publicProvider(p);
const pub = registry.publicProvider(p);
- return { key: p.key, kind: p.kind, credentials: pub.credentials, circuit: pool.health(p.key).state };
- });
+ return { key: p.key, kind: p.kind, credentials: pub.credentials, circuit: (await pool.health(p.key)).state };
+ })));
}
}
function createAiReadiness({ db, registry, pool, keys, runs, release = null }) {
function createAiReadiness({ db, registry, pool, keys, runs, release = null }) {
@@ -32,19 +32,19 @@ function createAiReadiness({ db, registry, pool, keys, runs, release = null }) {
@@ -32,19 +32,19 @@ function createAiReadiness({ db, registry, pool, keys, runs, release = null }) {
service: 'ai',
service: 'ai',
release,
release,
checks: [
checks: [
- { name: 'db', required: true, check: () => (db.prepare('SELECT COUNT(*) AS n FROM providers').get().n > 0 ? true : 'the provider registry is empty') },
+ { name: 'db', required: true, check: async () => ((await db.prepare('SELECT COUNT(*) AS n FROM providers').get()).n > 0 ? true : 'the provider registry is empty') },
{
{
name: 'workflows', required: true,
name: 'workflows', required: true,
- check: () => {
- const n = db.prepare("SELECT COUNT(DISTINCT key) AS n FROM workflows WHERE status = 'active'").get().n;
+ check: async () => {
+ const n = (await db.prepare("SELECT COUNT(DISTINCT key) AS n FROM workflows WHERE status = 'active'").get()).n;
return n > 0 ? { ok: true, detail: { active: n } } : 'no active workflow';
return n > 0 ? { ok: true, detail: { active: n } } : 'no active workflow';
},
},
},
},
{ name: 'network_jwks', required: true, check: () => keys.loaded() || 'Network signing key not loaded yet: no service token can be verified' },
{ name: 'network_jwks', required: true, check: () => keys.loaded() || 'Network signing key not loaded yet: no service token can be verified' },
{
{
name: 'providers', required: false,
name: 'providers', required: false,
- check: () => {
- const list = providerView(registry, pool);
+ check: async () => {
+ const list = await providerView(registry, pool);
const real = list.filter(p => p.kind !== 'stub');
const real = list.filter(p => p.kind !== 'stub');
const detail = { active: list.length, usable: real.filter(p => p.credentials !== 'missing' && p.circuit !== 'open').length };
const detail = { active: list.length, usable: real.filter(p => p.credentials !== 'missing' && p.circuit !== 'open').length };
if (!real.length) return { ok: false, error: 'no real provider is active: only synthetic stub output is available', detail };
if (!real.length) return { ok: false, error: 'no real provider is active: only synthetic stub output is available', detail };
@@ -54,8 +54,8 @@ function createAiReadiness({ db, registry, pool, keys, runs, release = null }) {
@@ -54,8 +54,8 @@ function createAiReadiness({ db, registry, pool, keys, runs, release = null }) {
},
},
},
},
],
],
- details: (body) => ({
- providers: body.checks.db.status === 'ok' ? providerView(registry, pool) : null,
+ details: async (body) => ({
+ providers: body.checks.db.status === 'ok' ? await providerView(registry, pool) : null,
runs: runs.stats(),
runs: runs.stats(),
}),
}),
});
});
@@ -74,11 +74,11 @@ function registerAiGauges(registry, { runs, registry: ai, pool }) {
@@ -74,11 +74,11 @@ function registerAiGauges(registry, { runs, registry: ai, pool }) {
registry.gauge({ name: 'ai_runs_max_queued', help: 'Runs allowed to wait for a slot (AI_MAX_QUEUED_RUNS)', collect: () => runs.stats().max_queued });
registry.gauge({ name: 'ai_runs_max_queued', help: 'Runs allowed to wait for a slot (AI_MAX_QUEUED_RUNS)', collect: () => runs.stats().max_queued });
registry.gauge({
registry.gauge({
name: 'ai_provider_circuit', help: 'Circuit-breaker state of each active provider (1 = the current state)', labelNames: ['provider', 'state'],
name: 'ai_provider_circuit', help: 'Circuit-breaker state of each active provider (1 = the current state)', labelNames: ['provider', 'state'],
- collect: () => providerView(ai, pool).flatMap(p => CIRCUIT_STATES.map(state => ({ labels: { provider: p.key, state }, value: p.circuit === state ? 1 : 0 }))),
+ collect: async () => (await providerView(ai, pool)).flatMap(p => CIRCUIT_STATES.map(state => ({ labels: { provider: p.key, state }, value: p.circuit === state ? 1 : 0 }))),
});
});
registry.gauge({
registry.gauge({
name: 'ai_provider_credentials_missing', help: 'Active providers whose secret reference does not resolve (1 = missing)', labelNames: ['provider'],
name: 'ai_provider_credentials_missing', help: 'Active providers whose secret reference does not resolve (1 = missing)', labelNames: ['provider'],
- collect: () => providerView(ai, pool).filter(p => p.credentials !== 'not_required').map(p => ({ labels: { provider: p.key }, value: p.credentials === 'missing' ? 1 : 0 })),
+ collect: async () => (await providerView(ai, pool)).filter(p => p.credentials !== 'not_required').map(p => ({ labels: { provider: p.key }, value: p.credentials === 'missing' ? 1 : 0 })),
});
});
}
}
@@ -13,39 +13,39 @@ const { AiError } = require('./util');
@@ -13,39 +13,39 @@ const { AiError } = require('./util');
function createOps({ db, registry, pool, quotas, cache, runs }) {
function createOps({ db, registry, pool, quotas, cache, runs }) {
/** A provider as every operator surface shows it: secret reference NAME + whether it resolves, health. */
/** A provider as every operator surface shows it: secret reference NAME + whether it resolves, health. */
- const providerView = (p) => registry.publicProvider(p, pool.health(p.key));
+ const providerView = async (p) => registry.publicProvider(p, await pool.health(p.key));
/** GET /api/v1/status: providers, run counts, today's spend, definitions, queue, quota counters. */
/** GET /api/v1/status: providers, run counts, today's spend, definitions, queue, quota counters. */
- function status() {
- const counts = Object.fromEntries(db.prepare('SELECT status, COUNT(*) n FROM runs GROUP BY status').all().map(x => [x.status, x.n]));
+ async function status() {
+ const counts = Object.fromEntries((await db.prepare('SELECT status, COUNT(*) n FROM runs GROUP BY status').all()).map(x => [x.status, x.n]));
const today = new Date().toISOString().slice(0, 10);
const today = new Date().toISOString().slice(0, 10);
- const spend = db.prepare('SELECT COALESCE(SUM(cost_usd),0) cost, COALESCE(SUM(requests),0) requests FROM usage_daily WHERE day = ?').get(today);
+ const spend = await db.prepare('SELECT COALESCE(SUM(cost_usd),0) cost, COALESCE(SUM(requests),0) requests FROM usage_daily WHERE day = ?').get(today);
return {
return {
- providers: registry.listProviders().map(p => ({ key: p.key, kind: p.kind, status: p.status, credentials: providerView(p).credentials, health: pool.health(p.key).state })),
+ providers: (await Promise.all((await registry.listProviders()).map(async p => ({ key: p.key, kind: p.kind, status: p.status, credentials: (await providerView(p)).credentials, health: (await pool.health(p.key)).state })))),
runs: counts, today: { day: today, requests: spend.requests, cost_usd: spend.cost },
runs: counts, today: { day: today, requests: spend.requests, cost_usd: spend.cost },
- workflows: registry.listWorkflows().length, templates: registry.listTemplates().length, routes: registry.listRoutes().length,
- inflight: runs.inflight.size, quotas: quotas.counters(),
+ workflows: (await registry.listWorkflows()).length, templates: (await registry.listTemplates()).length, routes: (await registry.listRoutes()).length,
+ inflight: runs.inflight.size, quotas: await quotas.counters(),
};
};
}
}
/** Enable or disable a provider (a registry upsert of its status; audited as provider.update). */
/** Enable or disable a provider (a registry upsert of its status; audited as provider.update). */
- function setProviderStatus(key, status, { actor, trace = null }) {
- if (!registry.getProvider(key)) throw new AiError(404, 'ai.not_found', 'no such provider');
+ async function setProviderStatus(key, status, { actor, trace = null }) {
+ if (!await registry.getProvider(key)) throw new AiError(404, 'ai.not_found', 'no such provider');
if (!['active', 'disabled'].includes(status)) throw new AiError(422, 'ai.invalid', 'provider status must be active or disabled');
if (!['active', 'disabled'].includes(status)) throw new AiError(422, 'ai.invalid', 'provider status must be active or disabled');
- return registry.upsertProvider({ key, status }, { actor, trace, origin: 'admin' });
+ return await registry.upsertProvider({ key, status }, { actor, trace, origin: 'admin' });
}
}
/** Close a provider's circuit (breaker reset); audited as provider.circuit_reset. */
/** Close a provider's circuit (breaker reset); audited as provider.circuit_reset. */
- function resetCircuit(key, { actor, trace = null }) {
- pool.resetHealth(key);
- registry.audit(actor, 'provider.circuit_reset', 'provider', key, { trace });
- return pool.health(key);
+ async function resetCircuit(key, { actor, trace = null }) {
+ await pool.resetHealth(key);
+ await registry.audit(actor, 'provider.circuit_reset', 'provider', key, { trace });
+ return await pool.health(key);
}
}
/** Remove cache entries: one workflow's, or every entry when `workflow` is empty; audited as cache.purge. */
/** Remove cache entries: one workflow's, or every entry when `workflow` is empty; audited as cache.purge. */
- function purgeCache(workflow, { actor, trace = null }) {
- const removed = cache.purge({ workflow: workflow || undefined });
- registry.audit(actor, 'cache.purge', 'cache', workflow || '*', { trace, metadata: { removed } });
+ async function purgeCache(workflow, { actor, trace = null }) {
+ const removed = await cache.purge({ workflow: workflow || undefined });
+ await registry.audit(actor, 'cache.purge', 'cache', workflow || '*', { trace, metadata: { removed } });
return removed;
return removed;
}
}
@@ -40,7 +40,7 @@ function createHttpSeamProvider(record, { apiKey = '', fetchImpl = globalThis.fe
@@ -40,7 +40,7 @@ function createHttpSeamProvider(record, { apiKey = '', fetchImpl = globalThis.fe
}
}
const adapter = { key: record.key, kind: 'http', supports: (f) => caps.has(f) };
const adapter = { key: record.key, kind: 'http', supports: (f) => caps.has(f) };
- for (const op of OPS) adapter[op] = (req) => call(op, req);
+ for (const op of OPS) adapter[op] = async (req) => await call(op, req);
return adapter;
return adapter;
}
}
@@ -47,11 +47,11 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
@@ -47,11 +47,11 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
}
}
}
}
- function adapter(key) {
- const p = registry.getProvider(key);
+ async function adapter(key) {
+ const p = await registry.getProvider(key);
if (!p) return null;
if (!p) return null;
const stamp = `${p.updated_at}|${p.secret_ref}|${resolveSecret(p.secret_ref, env) ? 1 : 0}`;
const stamp = `${p.updated_at}|${p.secret_ref}|${resolveSecret(p.secret_ref, env) ? 1 : 0}`;
- const hit = cache.get(key);
+ const hit = await cache.get(key);
if (hit && hit.stamp === stamp) return { record: p, adapter: hit.adapter };
if (hit && hit.stamp === stamp) return { record: p, adapter: hit.adapter };
const a = build(p);
const a = build(p);
cache.set(key, { stamp, adapter: a });
cache.set(key, { stamp, adapter: a });
@@ -64,30 +64,30 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
@@ -64,30 +64,30 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
VALUES (@provider_key, @state, @consecutive_failures, @opened_at, @last_error, @last_success_at, @last_failure_at)
VALUES (@provider_key, @state, @consecutive_failures, @opened_at, @last_error, @last_success_at, @last_failure_at)
ON CONFLICT(provider_key) DO UPDATE SET state = excluded.state, consecutive_failures = excluded.consecutive_failures, opened_at = excluded.opened_at,
ON CONFLICT(provider_key) DO UPDATE SET state = excluded.state, consecutive_failures = excluded.consecutive_failures, opened_at = excluded.opened_at,
last_error = excluded.last_error, last_success_at = excluded.last_success_at, last_failure_at = excluded.last_failure_at`);
last_error = excluded.last_error, last_success_at = excluded.last_success_at, last_failure_at = excluded.last_failure_at`);
- function health(key) {
- const h = getHealth.get(key) || { provider_key: key, state: 'closed', consecutive_failures: 0, opened_at: null, last_error: null, last_success_at: null, last_failure_at: null };
+ async function health(key) {
+ const h = await getHealth.get(key) || { provider_key: key, state: 'closed', consecutive_failures: 0, opened_at: null, last_error: null, last_success_at: null, last_failure_at: null };
if (h.state === 'open' && clock.now() - (h.opened_at || 0) >= config.breaker.cooldownMs) return { ...h, state: 'half_open' };
if (h.state === 'open' && clock.now() - (h.opened_at || 0) >= config.breaker.cooldownMs) return { ...h, state: 'half_open' };
return h;
return h;
}
}
- function recordSuccess(key) {
- const h = health(key);
- if (h.state !== 'closed') registry.audit('system', 'provider.circuit_closed', 'provider', key, { metadata: { after_failures: h.consecutive_failures } });
- putHealth.run({ ...h, provider_key: key, state: 'closed', consecutive_failures: 0, opened_at: null, last_success_at: clock.now() });
+ async function recordSuccess(key) {
+ const h = await health(key);
+ if (h.state !== 'closed') await registry.audit('system', 'provider.circuit_closed', 'provider', key, { metadata: { after_failures: h.consecutive_failures } });
+ await putHealth.run({ ...h, provider_key: key, state: 'closed', consecutive_failures: 0, opened_at: null, last_success_at: clock.now() });
}
}
- function recordFailure(key, err) {
- const h = health(key);
+ async function recordFailure(key, err) {
+ const h = await health(key);
const failures = (h.consecutive_failures || 0) + 1;
const failures = (h.consecutive_failures || 0) + 1;
const open = h.state === 'half_open' || failures >= config.breaker.failureThreshold;
const open = h.state === 'half_open' || failures >= config.breaker.failureThreshold;
- if (open && h.state !== 'open') registry.audit('system', 'provider.circuit_opened', 'provider', key, { metadata: { failures, error: String(err && err.message || err).slice(0, 200) } });
- putHealth.run({ ...h, provider_key: key, state: open ? 'open' : 'closed', consecutive_failures: failures, opened_at: open ? clock.now() : h.opened_at, last_error: String(err && err.message || err).slice(0, 500), last_failure_at: clock.now() });
+ if (open && h.state !== 'open') await registry.audit('system', 'provider.circuit_opened', 'provider', key, { metadata: { failures, error: String(err && err.message || err).slice(0, 200) } });
+ await putHealth.run({ ...h, provider_key: key, state: open ? 'open' : 'closed', consecutive_failures: failures, opened_at: open ? clock.now() : h.opened_at, last_error: String(err && err.message || err).slice(0, 500), last_failure_at: clock.now() });
}
}
- function resetHealth(key) { db.prepare('DELETE FROM provider_health WHERE provider_key = ?').run(key); }
+ async function resetHealth(key) { await db.prepare('DELETE FROM provider_health WHERE provider_key = ?').run(key); }
// ── Pricing (Live's rules: model row -> AI_PRICING_JSON longest prefix -> flat rates) ──
// ── Pricing (Live's rules: model row -> AI_PRICING_JSON longest prefix -> flat rates) ──
- function priceFor(providerKey, model) {
- const p = registry.getProvider(providerKey);
+ async function priceFor(providerKey, model) {
+ const p = await registry.getProvider(providerKey);
if (!p || p.kind === 'stub' || p.kind === 'whisper') return { in: 0, out: 0, cached: 0 };
if (!p || p.kind === 'stub' || p.kind === 'whisper') return { in: 0, out: 0, cached: 0 };
- const row = model ? registry.getModel(providerKey, model) : null;
+ const row = model ? await registry.getModel(providerKey, model) : null;
if (row && row.cost && Number.isFinite(row.cost.in_per_mtok) && row.cost.in_per_mtok !== null) {
if (row && row.cost && Number.isFinite(row.cost.in_per_mtok) && row.cost.in_per_mtok !== null) {
const inRate = Number(row.cost.in_per_mtok);
const inRate = Number(row.cost.in_per_mtok);
return { in: inRate, out: Number(row.cost.out_per_mtok) || 0, cached: row.cost.cached_per_mtok != null ? Number(row.cost.cached_per_mtok) : inRate * 0.1 };
return { in: inRate, out: Number(row.cost.out_per_mtok) || 0, cached: row.cost.cached_per_mtok != null ? Number(row.cost.cached_per_mtok) : inRate * 0.1 };
@@ -106,25 +106,25 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
@@ -106,25 +106,25 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
const cachedRate = best && Number.isFinite(Number(best.cached)) ? Number(best.cached) : inRate * 0.1;
const cachedRate = best && Number.isFinite(Number(best.cached)) ? Number(best.cached) : inRate * 0.1;
return { in: inRate, out: outRate, cached: cachedRate };
return { in: inRate, out: outRate, cached: cachedRate };
}
}
- function costOf(providerKey, model, usage) {
- const pr = priceFor(providerKey, model);
+ async function costOf(providerKey, model, usage) {
+ const pr = await priceFor(providerKey, model);
const input = Math.max(0, (usage.input || 0) - (usage.cached || 0));
const input = Math.max(0, (usage.input || 0) - (usage.cached || 0));
return (input / 1e6) * pr.in + ((usage.cached || 0) / 1e6) * pr.cached + ((usage.output || 0) / 1e6) * pr.out;
return (input / 1e6) * pr.in + ((usage.cached || 0) / 1e6) * pr.cached + ((usage.output || 0) / 1e6) * pr.out;
}
}
// ── Routing ────────────────────────────────────────────
// ── Routing ────────────────────────────────────────────
- function candidates(route) {
+ async function candidates(route) {
const list = [route.primary, ...(route.fallbacks || [])].filter(c => c && c.provider);
const list = [route.primary, ...(route.fallbacks || [])].filter(c => c && c.provider);
- if (config.stubFallback && !list.some(c => c.provider === 'stub') && registry.getProvider('stub')) list.push({ provider: 'stub', model: null, auto: true });
+ if (config.stubFallback && !list.some(c => c.provider === 'stub') && await registry.getProvider('stub')) list.push({ provider: 'stub', model: null, auto: true });
return list;
return list;
}
}
- function skipReason(p, a, features) {
+ async function skipReason(p, a, features) {
if (!p || !a) return 'unknown_provider';
if (!p || !a) return 'unknown_provider';
if (p.status !== 'active') return 'disabled';
if (p.status !== 'active') return 'disabled';
if (p.auth_mode !== 'none' && !resolveSecret(p.secret_ref, env)) return 'no_credentials';
if (p.auth_mode !== 'none' && !resolveSecret(p.secret_ref, env)) return 'no_credentials';
if (features.some(f => !a.supports(f))) return 'unsupported';
if (features.some(f => !a.supports(f))) return 'unsupported';
- if (health(p.key).state === 'open') return 'circuit_open';
+ if ((await health(p.key)).state === 'open') return 'circuit_open';
return null;
return null;
}
}
@@ -186,21 +186,21 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
@@ -186,21 +186,21 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
}
}
async function execute(route, operation, req, ctx = {}) {
async function execute(route, operation, req, ctx = {}) {
- if (ctx.credential) return executeWithCredential(ctx.credential, operation, req, ctx);
+ if (ctx.credential) return await executeWithCredential(ctx.credential, operation, req, ctx);
const features = [operation];
const features = [operation];
if (req.image) features.push('vision');
if (req.image) features.push('vision');
if (req.json) features.push('json');
if (req.json) features.push('json');
const tried = [];
const tried = [];
- const list = candidates(route);
+ const list = await candidates(route);
for (let i = 0; i < list.length; i++) {
for (let i = 0; i < list.length; i++) {
const c = list[i];
const c = list[i];
- const entry = adapter(c.provider);
+ const entry = await adapter(c.provider);
const p = entry && entry.record;
const p = entry && entry.record;
const a = entry && entry.adapter;
const a = entry && entry.adapter;
const model = c.model || (p && p.default_model) || null;
const model = c.model || (p && p.default_model) || null;
const fallback = i > 0;
const fallback = i > 0;
const base = { operation, provider_key: c.provider, model_key: model, route_key: ctx.routeKey || null, route_version: ctx.routeVersion || null, fallback: fallback ? 1 : 0, prompt_hash: ctx.promptHash || null, input_hash: ctx.inputHash || null };
const base = { operation, provider_key: c.provider, model_key: model, route_key: ctx.routeKey || null, route_version: ctx.routeVersion || null, fallback: fallback ? 1 : 0, prompt_hash: ctx.promptHash || null, input_hash: ctx.inputHash || null };
- const why = skipReason(p, a, features);
+ const why = await skipReason(p, a, features);
if (why) {
if (why) {
statFor(c.provider).skipped++;
statFor(c.provider).skipped++;
tried.push({ provider: c.provider, skipped: why });
tried.push({ provider: c.provider, skipped: why });
@@ -221,9 +221,9 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
@@ -221,9 +221,9 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
const result = await fn({ ...req, model, signal: ctx.signal, timeoutMs });
const result = await fn({ ...req, model, signal: ctx.signal, timeoutMs });
const latency = Date.now() - t0;
const latency = Date.now() - t0;
const usage = result.usage || { input: 0, output: 0, cached: 0 };
const usage = result.usage || { input: 0, output: 0, cached: 0 };
- const cost = costOf(c.provider, result.model || model, usage);
+ const cost = await costOf(c.provider, result.model || model, usage);
statFor(c.provider).ok++;
statFor(c.provider).ok++;
- recordSuccess(c.provider);
+ await recordSuccess(c.provider);
if (ctx.logRequest) {
if (ctx.logRequest) {
ctx.logRequest({
ctx.logRequest({
...base, model_key: result.model || model, status: 'ok', output_hash: sha256(result.json || result.text || result.vectors || result.segments || ''),
...base, model_key: result.model || model, status: 'ok', output_hash: sha256(result.json || result.text || result.vectors || result.segments || ''),
@@ -248,7 +248,7 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
@@ -248,7 +248,7 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
}
}
// A request the provider rejected as malformed is the caller's fault, not a sign the
// A request the provider rejected as malformed is the caller's fault, not a sign the
// provider is unhealthy: it must not open the circuit other callers depend on.
// provider is unhealthy: it must not open the circuit other callers depend on.
- if (!callerFault(lastErr)) recordFailure(c.provider, lastErr);
+ if (!callerFault(lastErr)) await recordFailure(c.provider, lastErr);
tried.push({ provider: c.provider, error: String(lastErr && lastErr.message || lastErr).slice(0, 200) });
tried.push({ provider: c.provider, error: String(lastErr && lastErr.message || lastErr).slice(0, 200) });
log.warn(`[ai] ${operation} via ${c.provider}/${model || '-'} failed: ${lastErr && lastErr.message}`);
log.warn(`[ai] ${operation} via ${c.provider}/${model || '-'} failed: ${lastErr && lastErr.message}`);
}
}
@@ -146,10 +146,10 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nu
@@ -146,10 +146,10 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nu
kind: 'stub',
kind: 'stub',
synthetic: true,
synthetic: true,
supports: () => true,
supports: () => true,
- transcribe: (req) => run('transcribe', req),
- embed: (req) => run('embed', req),
+ transcribe: async (req) => await run('transcribe', req),
+ embed: async (req) => await run('embed', req),
};
};
- for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich']) adapter[op] = (req) => run(op, req);
+ for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich']) adapter[op] = async (req) => await run(op, req);
return adapter;
return adapter;
}
}
@@ -35,9 +35,9 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
@@ -35,9 +35,9 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
* scope_id is '*' on a non-global scope applies to EACH principal of that type separately
* scope_id is '*' on a non-global scope applies to EACH principal of that type separately
* (e.g. service '*' = every service gets its own window).
* (e.g. service '*' = every service gets its own window).
*/
*/
- function applicable(ctx) {
+ async function applicable(ctx) {
const scopes = scopesFor(ctx);
const scopes = scopesFor(ctx);
- const rows = db.prepare("SELECT * FROM quotas WHERE status = 'active'").all();
+ const rows = await db.prepare("SELECT * FROM quotas WHERE status = 'active'").all();
const out = [];
const out = [];
for (const q of rows) {
for (const q of rows) {
if (q.workflow_prefix && !String(ctx.workflowKey || '').startsWith(q.workflow_prefix)) continue;
if (q.workflow_prefix && !String(ctx.workflowKey || '').startsWith(q.workflow_prefix)) continue;
@@ -47,19 +47,19 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
@@ -47,19 +47,19 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
return out;
return out;
}
}
- const getCounter = db.prepare('SELECT * FROM usage_counters WHERE scope_type = ? AND scope_id = ? AND window = ? AND window_start = ? AND workflow_prefix = ?');
- const bump = db.prepare(`INSERT INTO usage_counters (scope_type, scope_id, window, window_start, workflow_prefix, requests, tokens, cost_usd)
+ const getCounter = db.prepare('SELECT * FROM usage_counters WHERE scope_type = ? AND scope_id = ? AND "window" = ? AND window_start = ? AND workflow_prefix = ?');
+ const bump = db.prepare(`INSERT INTO usage_counters (scope_type, scope_id, "window", window_start, workflow_prefix, requests, tokens, cost_usd)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
- ON CONFLICT(scope_type, scope_id, window, window_start, workflow_prefix)
- DO UPDATE SET requests = requests + excluded.requests, tokens = tokens + excluded.tokens, cost_usd = cost_usd + excluded.cost_usd`);
+ ON CONFLICT(scope_type, scope_id, "window", window_start, workflow_prefix)
+ DO UPDATE SET requests = usage_counters.requests + excluded.requests, tokens = usage_counters.tokens + excluded.tokens, cost_usd = usage_counters.cost_usd + excluded.cost_usd`);
- const reserveTx = db.transaction((ctx) => {
- const qs = applicable(ctx);
+ const reserveTx = async (ctx) => await db.tx(async () => {
+ const qs = await applicable(ctx);
const t = nowSec();
const t = nowSec();
let worst = null;
let worst = null;
for (const { q, cid } of qs) {
for (const { q, cid } of qs) {
const ws = windowStart(q.window, t);
const ws = windowStart(q.window, t);
- const c = getCounter.get(q.scope_type, cid, q.window, ws, q.workflow_prefix || '') || { requests: 0, tokens: 0, cost_usd: 0 };
+ const c = await getCounter.get(q.scope_type, cid, q.window, ws, q.workflow_prefix || '') || { requests: 0, tokens: 0, cost_usd: 0 };
let reason = null;
let reason = null;
if (q.max_requests != null && c.requests + 1 > q.max_requests) reason = `${q.max_requests} requests per ${q.window}`;
if (q.max_requests != null && c.requests + 1 > q.max_requests) reason = `${q.max_requests} requests per ${q.window}`;
else if (q.max_tokens != null && c.tokens >= q.max_tokens) reason = `${q.max_tokens} tokens per ${q.window}`;
else if (q.max_tokens != null && c.tokens >= q.max_tokens) reason = `${q.max_tokens} tokens per ${q.window}`;
@@ -81,13 +81,13 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
@@ -81,13 +81,13 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
if (q.workflow_prefix) keys.set(`${q.scope_type}|${cid}|${q.window}|${q.workflow_prefix}`, { scope_type: q.scope_type, scope_id: cid, window: q.window, workflow_prefix: q.workflow_prefix, ws: windowStart(q.window, t) });
if (q.workflow_prefix) keys.set(`${q.scope_type}|${cid}|${q.window}|${q.workflow_prefix}`, { scope_type: q.scope_type, scope_id: cid, window: q.window, workflow_prefix: q.workflow_prefix, ws: windowStart(q.window, t) });
}
}
const counted = [...keys.values()];
const counted = [...keys.values()];
- for (const k of counted) bump.run(k.scope_type, k.scope_id, k.window, k.ws, k.workflow_prefix, 1, 0, 0);
+ for (const k of counted) await bump.run(k.scope_type, k.scope_id, k.window, k.ws, k.workflow_prefix, 1, 0, 0);
return { ok: true, quotas: counted };
return { ok: true, quotas: counted };
});
});
/** Check + count one request. Throws 429 quota.exceeded before any provider is touched. */
/** Check + count one request. Throws 429 quota.exceeded before any provider is touched. */
- function reserve(ctx) {
- const r = reserveTx(ctx);
+ async function reserve(ctx) {
+ const r = await reserveTx(ctx);
if (!r.ok) {
if (!r.ok) {
throw new AiError(429, 'quota.exceeded', `quota exceeded for ${r.q.scope_type} ${r.cid}: ${r.reason}`, {
throw new AiError(429, 'quota.exceeded', `quota exceeded for ${r.q.scope_type} ${r.cid}: ${r.reason}`, {
retry_after_seconds: Math.max(1, r.retryAfter), quota: { id: r.q.id, scope_type: r.q.scope_type, scope_id: r.cid, window: r.q.window },
retry_after_seconds: Math.max(1, r.retryAfter), quota: { id: r.q.id, scope_type: r.q.scope_type, scope_id: r.cid, window: r.q.window },
@@ -98,21 +98,21 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
@@ -98,21 +98,21 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
const insDaily = db.prepare(`INSERT INTO usage_daily (day, requester, attribution, workflow_key, provider_key, model_key, requests, tokens_in, tokens_out, tokens_cached, cost_usd)
const insDaily = db.prepare(`INSERT INTO usage_daily (day, requester, attribution, workflow_key, provider_key, model_key, requests, tokens_in, tokens_out, tokens_cached, cost_usd)
VALUES (?, ?, ?, ?, ?, ?, 1, ?, ?, ?, ?)
VALUES (?, ?, ?, ?, ?, ?, 1, ?, ?, ?, ?)
- ON CONFLICT(day, requester, attribution, workflow_key, provider_key, model_key) DO UPDATE SET requests = requests + 1,
- tokens_in = tokens_in + excluded.tokens_in, tokens_out = tokens_out + excluded.tokens_out, tokens_cached = tokens_cached + excluded.tokens_cached, cost_usd = cost_usd + excluded.cost_usd`);
+ ON CONFLICT(day, requester, attribution, workflow_key, provider_key, model_key) DO UPDATE SET requests = usage_daily.requests + 1,
+ tokens_in = usage_daily.tokens_in + excluded.tokens_in, tokens_out = usage_daily.tokens_out + excluded.tokens_out, tokens_cached = usage_daily.tokens_cached + excluded.tokens_cached, cost_usd = usage_daily.cost_usd + excluded.cost_usd`);
/** Add real usage to the reserved windows and the daily usage table. */
/** Add real usage to the reserved windows and the daily usage table. */
- function account(ctx, reserved, { provider, model, tokensIn = 0, tokensOut = 0, tokensCached = 0, cost = 0 }) {
+ async function account(ctx, reserved, { provider, model, tokensIn = 0, tokensOut = 0, tokensCached = 0, cost = 0 }) {
const tokens = tokensIn + tokensOut;
const tokens = tokensIn + tokensOut;
- db.transaction(() => {
- for (const r of reserved || []) bump.run(r.scope_type, r.scope_id, r.window, r.ws, r.workflow_prefix, 0, tokens, cost);
- insDaily.run(iso(clock.now()).slice(0, 10), `${ctx.requesterType}:${ctx.requesterId}`, ctx.attributionKey || '', ctx.workflowKey, provider || '', model || '', tokensIn, tokensOut, tokensCached, cost);
- })();
+ await db.tx(async () => {
+ for (const r of reserved || []) await bump.run(r.scope_type, r.scope_id, r.window, r.ws, r.workflow_prefix, 0, tokens, cost);
+ await insDaily.run(iso(clock.now()).slice(0, 10), `${ctx.requesterType}:${ctx.requesterId}`, ctx.attributionKey || '', ctx.workflowKey, provider || '', model || '', tokensIn, tokensOut, tokensCached, cost);
+ });
}
}
// ── Admin ──
// ── Admin ──
- function list() { return db.prepare('SELECT * FROM quotas ORDER BY scope_type, scope_id, window').all(); }
- function upsert(q, { actor = 'system', origin = 'admin', trace = null } = {}) {
+ async function list() { return await db.prepare('SELECT * FROM quotas ORDER BY scope_type, scope_id, "window"').all(); }
+ async function upsert(q, { actor = 'system', origin = 'admin', trace = null } = {}) {
if (!['global', 'service', 'actor', 'attribution'].includes(q.scope_type)) throw new AiError(422, 'ai.invalid', 'scope_type must be global, service, actor or attribution');
if (!['global', 'service', 'actor', 'attribution'].includes(q.scope_type)) throw new AiError(422, 'ai.invalid', 'scope_type must be global, service, actor or attribution');
if (!WINDOWS[q.window]) throw new AiError(422, 'ai.invalid', 'window must be minute, hour or day');
if (!WINDOWS[q.window]) throw new AiError(422, 'ai.invalid', 'window must be minute, hour or day');
const scopeId = q.scope_type === 'global' ? '*' : String(q.scope_id || ''); // '*' on other scopes = each one separately
const scopeId = q.scope_type === 'global' ? '*' : String(q.scope_id || ''); // '*' on other scopes = each one separately
@@ -123,36 +123,36 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
@@ -123,36 +123,36 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
if (!['active', 'disabled'].includes(status)) throw new AiError(422, 'ai.invalid', 'status must be active or disabled');
if (!['active', 'disabled'].includes(status)) throw new AiError(422, 'ai.invalid', 'status must be active or disabled');
const t = iso(clock.now());
const t = iso(clock.now());
const prefix = q.workflow_prefix || null;
const prefix = q.workflow_prefix || null;
- const prev = db.prepare("SELECT * FROM quotas WHERE scope_type = ? AND scope_id = ? AND window = ? AND COALESCE(workflow_prefix, '') = ?").get(q.scope_type, scopeId, q.window, prefix || '');
+ const prev = await db.prepare("SELECT * FROM quotas WHERE scope_type = ? AND scope_id = ? AND \"window\" = ? AND COALESCE(workflow_prefix, '') = ?").get(q.scope_type, scopeId, q.window, prefix || '');
if (prev) {
if (prev) {
- db.prepare('UPDATE quotas SET max_requests = ?, max_tokens = ?, max_cost_usd = ?, status = ?, updated_at = ? WHERE id = ?')
+ await db.prepare('UPDATE quotas SET max_requests = ?, max_tokens = ?, max_cost_usd = ?, status = ?, updated_at = ? WHERE id = ?')
.run(num(q.max_requests), num(q.max_tokens), num(q.max_cost_usd), status, t, prev.id);
.run(num(q.max_requests), num(q.max_tokens), num(q.max_cost_usd), status, t, prev.id);
} else {
} else {
- db.prepare('INSERT INTO quotas (scope_type, scope_id, window, max_requests, max_tokens, max_cost_usd, workflow_prefix, status, origin, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)')
+ await db.prepare('INSERT INTO quotas (scope_type, scope_id, "window", max_requests, max_tokens, max_cost_usd, workflow_prefix, status, origin, created_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)')
.run(q.scope_type, scopeId, q.window, num(q.max_requests), num(q.max_tokens), num(q.max_cost_usd), prefix, status, origin, t, t);
.run(q.scope_type, scopeId, q.window, num(q.max_requests), num(q.max_tokens), num(q.max_cost_usd), prefix, status, origin, t, t);
}
}
- const row = db.prepare("SELECT * FROM quotas WHERE scope_type = ? AND scope_id = ? AND window = ? AND COALESCE(workflow_prefix, '') = ?").get(q.scope_type, scopeId, q.window, prefix || '');
- if (registry) registry.audit(actor, prev ? 'quota.update' : 'quota.create', 'quota', row.id, { trace, metadata: { scope_type: row.scope_type, scope_id: row.scope_id, window: row.window, max_requests: row.max_requests, max_tokens: row.max_tokens, max_cost_usd: row.max_cost_usd, status: row.status, origin } });
+ const row = await db.prepare("SELECT * FROM quotas WHERE scope_type = ? AND scope_id = ? AND \"window\" = ? AND COALESCE(workflow_prefix, '') = ?").get(q.scope_type, scopeId, q.window, prefix || '');
+ if (registry) await registry.audit(actor, prev ? 'quota.update' : 'quota.create', 'quota', row.id, { trace, metadata: { scope_type: row.scope_type, scope_id: row.scope_id, window: row.window, max_requests: row.max_requests, max_tokens: row.max_tokens, max_cost_usd: row.max_cost_usd, status: row.status, origin } });
return row;
return row;
}
}
- function usage({ from, to, requester } = {}) {
+ async function usage({ from, to, requester } = {}) {
const where = [];
const where = [];
const args = [];
const args = [];
if (from) { where.push('day >= ?'); args.push(from); }
if (from) { where.push('day >= ?'); args.push(from); }
if (to) { where.push('day <= ?'); args.push(to); }
if (to) { where.push('day <= ?'); args.push(to); }
if (requester) { where.push('requester = ?'); args.push(requester); }
if (requester) { where.push('requester = ?'); args.push(requester); }
- return db.prepare(`SELECT * FROM usage_daily ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY day DESC, cost_usd DESC LIMIT 1000`).all(...args);
+ return await db.prepare(`SELECT * FROM usage_daily ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY day DESC, cost_usd DESC LIMIT 1000`).all(...args);
}
}
- function counters(ctxLike) {
- const qs = ctxLike ? applicable(ctxLike) : list().filter(q => q.status === 'active' && (q.scope_type === 'global' || q.scope_id !== '*')).map(q => ({ q, cid: q.scope_id }));
+ async function counters(ctxLike) {
+ const qs = ctxLike ? await applicable(ctxLike) : (await list()).filter(q => q.status === 'active' && (q.scope_type === 'global' || q.scope_id !== '*')).map(q => ({ q, cid: q.scope_id }));
const t = nowSec();
const t = nowSec();
- return qs.map(({ q, cid }) => {
+ return (await Promise.all(qs.map(async ({ q, cid }) => {
const ws = windowStart(q.window, t);
const ws = windowStart(q.window, t);
- const c = getCounter.get(q.scope_type, cid, q.window, ws, q.workflow_prefix || '') || { requests: 0, tokens: 0, cost_usd: 0 };
+ const c = await getCounter.get(q.scope_type, cid, q.window, ws, q.workflow_prefix || '') || { requests: 0, tokens: 0, cost_usd: 0 };
return { quota: q, scope_id: cid, window_start: iso(ws * 1000), resets_in_seconds: ws + WINDOWS[q.window] - t, used: { requests: c.requests, tokens: c.tokens, cost_usd: c.cost_usd } };
return { quota: q, scope_id: cid, window_start: iso(ws * 1000), resets_in_seconds: ws + WINDOWS[q.window] - t, used: { requests: c.requests, tokens: c.tokens, cost_usd: c.cost_usd } };
- });
+ })));
}
}
/**
/**
@@ -161,11 +161,11 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
@@ -161,11 +161,11 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry } = {})
* applies them) and whose max_cost_usd is above 0. The tightest cap counts, against today's (UTC) spend of the
* applies them) and whose max_cost_usd is above 0. The tightest cap counts, against today's (UTC) spend of the
* workflows starting with `spendPrefix`. null when there is none: no budget, no paid call.
* workflows starting with `spendPrefix`. null when there is none: no budget, no paid call.
*/
*/
- function paidBudget(prefix, spendPrefix) {
- const rows = db.prepare("SELECT max_cost_usd FROM quotas WHERE status = 'active' AND workflow_prefix = ? AND max_cost_usd > 0").all(prefix);
+ async function paidBudget(prefix, spendPrefix) {
+ const rows = await db.prepare("SELECT max_cost_usd FROM quotas WHERE status = 'active' AND workflow_prefix = ? AND max_cost_usd > 0").all(prefix);
if (!rows.length) return null;
if (!rows.length) return null;
const day = new Date(clock.now()).toISOString().slice(0, 10);
const day = new Date(clock.now()).toISOString().slice(0, 10);
- const spent = db.prepare("SELECT COALESCE(SUM(cost_usd), 0) AS c FROM usage_daily WHERE day = ? AND substr(workflow_key, 1, ?) = ?").get(day, spendPrefix.length, spendPrefix).c;
+ const spent = (await db.prepare("SELECT COALESCE(SUM(cost_usd), 0) AS c FROM usage_daily WHERE day = ? AND substr(workflow_key, 1, ?) = ?").get(day, spendPrefix.length, spendPrefix)).c;
return { max_cost_usd: Math.min(...rows.map((r) => r.max_cost_usd)), spent, window: 'day' };
return { max_cost_usd: Math.min(...rows.map((r) => r.max_cost_usd)), spent, window: 'day' };
}
}
@@ -22,17 +22,17 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -22,17 +22,17 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
// ── Audit ──────────────────────────────────────────────
// ── Audit ──────────────────────────────────────────────
const insAudit = db.prepare('INSERT INTO audit_log (at, actor, action, target_type, target_id, trace_id, metadata) VALUES (?, ?, ?, ?, ?, ?, ?)');
const insAudit = db.prepare('INSERT INTO audit_log (at, actor, action, target_type, target_id, trace_id, metadata) VALUES (?, ?, ?, ?, ?, ?, ?)');
- function audit(actor, action, targetType, targetId, { trace = null, metadata = {} } = {}) {
- insAudit.run(now(), actor || 'system', action, targetType || null, targetId == null ? null : String(targetId), trace, JSON.stringify(metadata || {}));
+ async function audit(actor, action, targetType, targetId, { trace = null, metadata = {} } = {}) {
+ await insAudit.run(now(), actor || 'system', action, targetType || null, targetId == null ? null : String(targetId), trace, JSON.stringify(metadata || {}));
}
}
- function listAudit({ action, targetType, targetId, limit = 100, before } = {}) {
+ async function listAudit({ action, targetType, targetId, limit = 100, before } = {}) {
const where = [];
const where = [];
const args = [];
const args = [];
if (action) { where.push('action = ?'); args.push(action); }
if (action) { where.push('action = ?'); args.push(action); }
if (targetType) { where.push('target_type = ?'); args.push(targetType); }
if (targetType) { where.push('target_type = ?'); args.push(targetType); }
if (targetId) { where.push('target_id = ?'); args.push(String(targetId)); }
if (targetId) { where.push('target_id = ?'); args.push(String(targetId)); }
if (before) { where.push('id < ?'); args.push(Number(before)); }
if (before) { where.push('id < ?'); args.push(Number(before)); }
- const rows = db.prepare(`SELECT * FROM audit_log ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(500, limit));
+ const rows = await db.prepare(`SELECT * FROM audit_log ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(500, limit));
return rows.map(r => ({ ...r, metadata: parseJson(r.metadata, {}) }));
return rows.map(r => ({ ...r, metadata: parseJson(r.metadata, {}) }));
}
}
@@ -46,8 +46,8 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -46,8 +46,8 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
metadata: parseJson(r.metadata, {}), origin: r.origin, created_at: r.created_at, updated_at: r.updated_at,
metadata: parseJson(r.metadata, {}), origin: r.origin, created_at: r.created_at, updated_at: r.updated_at,
};
};
}
}
- function getProvider(key) { return decodeProvider(db.prepare('SELECT * FROM providers WHERE key = ?').get(key)); }
- function listProviders() { return db.prepare('SELECT * FROM providers ORDER BY priority, key').all().map(decodeProvider); }
+ async function getProvider(key) { return decodeProvider(await db.prepare('SELECT * FROM providers WHERE key = ?').get(key)); }
+ async function listProviders() { return (await db.prepare('SELECT * FROM providers ORDER BY priority, key').all()).map(decodeProvider); }
/** Public view: the secret reference name and whether it resolves — never the value. */
/** Public view: the secret reference name and whether it resolves — never the value. */
function publicProvider(p, health) {
function publicProvider(p, health) {
@@ -66,8 +66,8 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -66,8 +66,8 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
if (!Array.isArray(p.capabilities) || p.capabilities.some(f => !FEATURES.includes(f))) throw new AiError(422, 'ai.invalid', `capabilities must be a subset of ${FEATURES.join(', ')}`);
if (!Array.isArray(p.capabilities) || p.capabilities.some(f => !FEATURES.includes(f))) throw new AiError(422, 'ai.invalid', `capabilities must be a subset of ${FEATURES.join(', ')}`);
}
}
- function upsertProvider(input, { actor = 'system', origin = 'admin', trace = null } = {}) {
- const prev = getProvider(input.key);
+ async function upsertProvider(input, { actor = 'system', origin = 'admin', trace = null } = {}) {
+ const prev = await getProvider(input.key);
const p = {
const p = {
key: input.key,
key: input.key,
display_name: input.display_name || (prev && prev.display_name) || input.key,
display_name: input.display_name || (prev && prev.display_name) || input.key,
@@ -84,14 +84,14 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -84,14 +84,14 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
};
};
checkProvider(p);
checkProvider(p);
const t = now();
const t = now();
- db.prepare(`INSERT INTO providers (key, display_name, kind, status, base_url, auth_mode, secret_ref, default_model, capabilities, timeout_ms, priority, metadata, origin, created_at, updated_at)
+ await db.prepare(`INSERT INTO providers (key, display_name, kind, status, base_url, auth_mode, secret_ref, default_model, capabilities, timeout_ms, priority, metadata, origin, created_at, updated_at)
VALUES (@key, @display_name, @kind, @status, @base_url, @auth_mode, @secret_ref, @default_model, @capabilities, @timeout_ms, @priority, @metadata, @origin, @t, @t)
VALUES (@key, @display_name, @kind, @status, @base_url, @auth_mode, @secret_ref, @default_model, @capabilities, @timeout_ms, @priority, @metadata, @origin, @t, @t)
ON CONFLICT(key) DO UPDATE SET display_name = excluded.display_name, kind = excluded.kind, status = excluded.status, base_url = excluded.base_url,
ON CONFLICT(key) DO UPDATE SET display_name = excluded.display_name, kind = excluded.kind, status = excluded.status, base_url = excluded.base_url,
auth_mode = excluded.auth_mode, secret_ref = excluded.secret_ref, default_model = excluded.default_model, capabilities = excluded.capabilities,
auth_mode = excluded.auth_mode, secret_ref = excluded.secret_ref, default_model = excluded.default_model, capabilities = excluded.capabilities,
timeout_ms = excluded.timeout_ms, priority = excluded.priority, metadata = excluded.metadata, origin = excluded.origin, updated_at = excluded.updated_at`)
timeout_ms = excluded.timeout_ms, priority = excluded.priority, metadata = excluded.metadata, origin = excluded.origin, updated_at = excluded.updated_at`)
.run({ ...p, capabilities: JSON.stringify(p.capabilities), metadata: JSON.stringify(p.metadata), origin, t });
.run({ ...p, capabilities: JSON.stringify(p.capabilities), metadata: JSON.stringify(p.metadata), origin, t });
- audit(actor, prev ? 'provider.update' : 'provider.create', 'provider', p.key, { trace, metadata: { kind: p.kind, status: p.status, base_url: p.base_url, secret_ref: p.secret_ref, origin } });
- return getProvider(p.key);
+ await audit(actor, prev ? 'provider.update' : 'provider.create', 'provider', p.key, { trace, metadata: { kind: p.kind, status: p.status, base_url: p.base_url, secret_ref: p.secret_ref, origin } });
+ return await getProvider(p.key);
}
}
// ── Models ─────────────────────────────────────────────
// ── Models ─────────────────────────────────────────────
@@ -105,15 +105,15 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -105,15 +105,15 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
metadata: parseJson(r.metadata, {}), created_at: r.created_at, updated_at: r.updated_at,
metadata: parseJson(r.metadata, {}), created_at: r.created_at, updated_at: r.updated_at,
};
};
}
}
- function getModel(providerKey, modelKey) { return decodeModel(db.prepare('SELECT * FROM models WHERE provider_key = ? AND model_key = ?').get(providerKey, modelKey)); }
- function listModels({ provider } = {}) {
- const rows = provider ? db.prepare('SELECT * FROM models WHERE provider_key = ? ORDER BY model_key').all(provider) : db.prepare('SELECT * FROM models ORDER BY provider_key, model_key').all();
+ async function getModel(providerKey, modelKey) { return decodeModel(await db.prepare('SELECT * FROM models WHERE provider_key = ? AND model_key = ?').get(providerKey, modelKey)); }
+ async function listModels({ provider } = {}) {
+ const rows = provider ? await db.prepare('SELECT * FROM models WHERE provider_key = ? ORDER BY model_key').all(provider) : await db.prepare('SELECT * FROM models ORDER BY provider_key, model_key').all();
return rows.map(decodeModel);
return rows.map(decodeModel);
}
}
- function upsertModel(input, { actor = 'system', trace = null } = {}) {
- if (!getProvider(input.provider_key)) throw new AiError(404, 'ai.not_found', `no provider ${input.provider_key}`);
+ async function upsertModel(input, { actor = 'system', trace = null } = {}) {
+ if (!await getProvider(input.provider_key)) throw new AiError(404, 'ai.not_found', `no provider ${input.provider_key}`);
if (!input.model_key || String(input.model_key).length > 200) throw new AiError(422, 'ai.invalid', 'model_key required');
if (!input.model_key || String(input.model_key).length > 200) throw new AiError(422, 'ai.invalid', 'model_key required');
- const prev = getModel(input.provider_key, input.model_key);
+ const prev = await getModel(input.provider_key, input.model_key);
const type = input.type || (prev && prev.type) || 'chat';
const type = input.type || (prev && prev.type) || 'chat';
if (!['chat', 'vision', 'embedding', 'stt'].includes(type)) throw new AiError(422, 'ai.invalid', 'model type must be chat, vision, embedding or stt');
if (!['chat', 'vision', 'embedding', 'stt'].includes(type)) throw new AiError(422, 'ai.invalid', 'model type must be chat, vision, embedding or stt');
const status = input.status || (prev && prev.status) || 'active';
const status = input.status || (prev && prev.status) || 'active';
@@ -121,7 +121,7 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -121,7 +121,7 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
const cost = input.cost || (prev && prev.cost) || {};
const cost = input.cost || (prev && prev.cost) || {};
const sup = input.supports || (prev && prev.supports) || {};
const sup = input.supports || (prev && prev.supports) || {};
const t = now();
const t = now();
- db.prepare(`INSERT INTO models (provider_key, model_key, display_name, type, status, context_window, max_output, cost_in_per_mtok, cost_out_per_mtok, cost_cached_per_mtok,
+ await db.prepare(`INSERT INTO models (provider_key, model_key, display_name, type, status, context_window, max_output, cost_in_per_mtok, cost_out_per_mtok, cost_cached_per_mtok,
supports_json, supports_tools, supports_streaming, supports_vision, metadata, created_at, updated_at)
supports_json, supports_tools, supports_streaming, supports_vision, metadata, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(provider_key, model_key) DO UPDATE SET display_name = excluded.display_name, type = excluded.type, status = excluded.status,
ON CONFLICT(provider_key, model_key) DO UPDATE SET display_name = excluded.display_name, type = excluded.type, status = excluded.status,
@@ -134,20 +134,20 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -134,20 +134,20 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
cost.in_per_mtok ?? null, cost.out_per_mtok ?? null, cost.cached_per_mtok ?? null,
cost.in_per_mtok ?? null, cost.out_per_mtok ?? null, cost.cached_per_mtok ?? null,
sup.json === false ? 0 : 1, sup.tools ? 1 : 0, sup.streaming ? 1 : 0, sup.vision ? 1 : 0,
sup.json === false ? 0 : 1, sup.tools ? 1 : 0, sup.streaming ? 1 : 0, sup.vision ? 1 : 0,
JSON.stringify(input.metadata || (prev && prev.metadata) || {}), t, t);
JSON.stringify(input.metadata || (prev && prev.metadata) || {}), t, t);
- audit(actor, prev ? 'model.update' : 'model.create', 'model', `${input.provider_key}/${input.model_key}`, { trace, metadata: { status, type } });
- return getModel(input.provider_key, input.model_key);
+ await audit(actor, prev ? 'model.update' : 'model.create', 'model', `${input.provider_key}/${input.model_key}`, { trace, metadata: { status, type } });
+ return await getModel(input.provider_key, input.model_key);
}
}
// ── Versioned records (routes, templates, workflows) ───
// ── Versioned records (routes, templates, workflows) ───
- function nextVersion(table, key) {
- const r = db.prepare(`SELECT MAX(version) AS v FROM ${table} WHERE key = ?`).get(key);
+ async function nextVersion(table, key) {
+ const r = await db.prepare(`SELECT MAX(version) AS v FROM ${table} WHERE key = ?`).get(key);
return (r && r.v ? r.v : 0) + 1;
return (r && r.v ? r.v : 0) + 1;
}
}
- function latestAny(table, key) {
- return db.prepare(`SELECT * FROM ${table} WHERE key = ? ORDER BY version DESC LIMIT 1`).get(key);
+ async function latestAny(table, key) {
+ return await db.prepare(`SELECT * FROM ${table} WHERE key = ? ORDER BY version DESC LIMIT 1`).get(key);
}
}
- function latestActive(table, key) {
- return db.prepare(`SELECT * FROM ${table} WHERE key = ? AND status = 'active' ORDER BY version DESC LIMIT 1`).get(key);
+ async function latestActive(table, key) {
+ return await db.prepare(`SELECT * FROM ${table} WHERE key = ? AND status = 'active' ORDER BY version DESC LIMIT 1`).get(key);
}
}
// Routes
// Routes
@@ -160,9 +160,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -160,9 +160,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
};
};
}
}
const routeContent = (r) => sha256({ p: r.primary, f: r.fallbacks, o: r.options, m: r.max_output_tokens, rf: r.response_format, t: r.timeout_ms, a: r.alias_of, s: r.status });
const routeContent = (r) => sha256({ p: r.primary, f: r.fallbacks, o: r.options, m: r.max_output_tokens, rf: r.response_format, t: r.timeout_ms, a: r.alias_of, s: r.status });
- function createRouteVersion(key, input, { actor = 'system', trace = null } = {}) {
+ async function createRouteVersion(key, input, { actor = 'system', trace = null } = {}) {
if (!KEY_RE.test(key || '')) throw new AiError(422, 'ai.invalid', 'route key must be lowercase dotted');
if (!KEY_RE.test(key || '')) throw new AiError(422, 'ai.invalid', 'route key must be lowercase dotted');
- const prev = decodeRoute(latestAny('routes', key));
+ const prev = decodeRoute(await latestAny('routes', key));
const r = {
const r = {
primary: input.primary || (prev && prev.primary),
primary: input.primary || (prev && prev.primary),
fallbacks: input.fallbacks !== undefined ? input.fallbacks : (prev ? prev.fallbacks : []),
fallbacks: input.fallbacks !== undefined ? input.fallbacks : (prev ? prev.fallbacks : []),
@@ -176,28 +176,28 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -176,28 +176,28 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
if (!r.alias_of) {
if (!r.alias_of) {
if (!r.primary || !r.primary.provider) throw new AiError(422, 'ai.invalid', 'route needs primary.provider');
if (!r.primary || !r.primary.provider) throw new AiError(422, 'ai.invalid', 'route needs primary.provider');
for (const c of [r.primary, ...(r.fallbacks || [])]) {
for (const c of [r.primary, ...(r.fallbacks || [])]) {
- if (!c || !getProvider(c.provider)) throw new AiError(422, 'ai.invalid', `route references unknown provider ${c && c.provider}`);
+ if (!c || !await getProvider(c.provider)) throw new AiError(422, 'ai.invalid', `route references unknown provider ${c && c.provider}`);
}
}
- } else if (!latestAny('routes', r.alias_of)) throw new AiError(422, 'ai.invalid', `alias target ${r.alias_of} does not exist`);
+ } else if (!await latestAny('routes', r.alias_of)) throw new AiError(422, 'ai.invalid', `alias target ${r.alias_of} does not exist`);
if (!Array.isArray(r.fallbacks)) throw new AiError(422, 'ai.invalid', 'fallbacks must be an array');
if (!Array.isArray(r.fallbacks)) throw new AiError(422, 'ai.invalid', 'fallbacks must be an array');
if (!['text', 'json'].includes(r.response_format)) throw new AiError(422, 'ai.invalid', 'response_format must be text or json');
if (!['text', 'json'].includes(r.response_format)) throw new AiError(422, 'ai.invalid', 'response_format must be text or json');
if (!['active', 'disabled'].includes(r.status)) throw new AiError(422, 'ai.invalid', 'route status must be active or disabled');
if (!['active', 'disabled'].includes(r.status)) throw new AiError(422, 'ai.invalid', 'route status must be active or disabled');
- const version = nextVersion('routes', key);
- db.prepare(`INSERT INTO routes (key, version, status, primary_provider, primary_model, fallbacks, options, max_output_tokens, response_format, timeout_ms, alias_of, created_by, created_at)
+ const version = await nextVersion('routes', key);
+ await db.prepare(`INSERT INTO routes (key, version, status, primary_provider, primary_model, fallbacks, options, max_output_tokens, response_format, timeout_ms, alias_of, created_by, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`)
.run(key, version, r.status, r.alias_of ? (r.primary && r.primary.provider) || 'stub' : r.primary.provider, r.primary ? r.primary.model || null : null,
.run(key, version, r.status, r.alias_of ? (r.primary && r.primary.provider) || 'stub' : r.primary.provider, r.primary ? r.primary.model || null : null,
JSON.stringify(r.fallbacks || []), JSON.stringify(r.options || {}), r.max_output_tokens, r.response_format, r.timeout_ms, r.alias_of, actor, now());
JSON.stringify(r.fallbacks || []), JSON.stringify(r.options || {}), r.max_output_tokens, r.response_format, r.timeout_ms, r.alias_of, actor, now());
- audit(actor, 'route.version', 'route', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: r.status, alias_of: r.alias_of } });
- return getRoute(key, version);
+ await audit(actor, 'route.version', 'route', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: r.status, alias_of: r.alias_of } });
+ return await getRoute(key, version);
}
}
- function getRoute(key, version) {
- return decodeRoute(version ? db.prepare('SELECT * FROM routes WHERE key = ? AND version = ?').get(key, version) : latestAny('routes', key));
+ async function getRoute(key, version) {
+ return decodeRoute(version ? await db.prepare('SELECT * FROM routes WHERE key = ? AND version = ?').get(key, version) : await latestAny('routes', key));
}
}
/** The route a run uses: newest version, which must be active; aliases are followed (max 3 hops). */
/** The route a run uses: newest version, which must be active; aliases are followed (max 3 hops). */
- function resolveRoute(key) {
+ async function resolveRoute(key) {
let k = key;
let k = key;
for (let hop = 0; hop < 4; hop++) {
for (let hop = 0; hop < 4; hop++) {
- const r = decodeRoute(latestAny('routes', k));
+ const r = decodeRoute(await latestAny('routes', k));
if (!r) return null;
if (!r) return null;
if (r.status !== 'active') return { ...r, disabled: true };
if (r.status !== 'active') return { ...r, disabled: true };
if (!r.alias_of) return r;
if (!r.alias_of) return r;
@@ -205,9 +205,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -205,9 +205,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
}
}
return null;
return null;
}
}
- function listRoutes({ history = false } = {}) {
- const rows = history ? db.prepare('SELECT * FROM routes ORDER BY key, version').all()
- : db.prepare('SELECT r.* FROM routes r JOIN (SELECT key, MAX(version) v FROM routes GROUP BY key) m ON m.key = r.key AND m.v = r.version ORDER BY r.key').all();
+ async function listRoutes({ history = false } = {}) {
+ const rows = history ? await db.prepare('SELECT * FROM routes ORDER BY key, version').all()
+ : await db.prepare('SELECT r.* FROM routes r JOIN (SELECT key, MAX(version) v FROM routes GROUP BY key) m ON m.key = r.key AND m.v = r.version ORDER BY r.key').all();
return rows.map(decodeRoute);
return rows.map(decodeRoute);
}
}
@@ -223,9 +223,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -223,9 +223,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
};
};
}
}
const templateContent = (t) => sha256({ n: t.name, d: t.description, i: t.input_schema, o: t.output_schema, s: t.system_prompt, u: t.user_prompt, r: t.default_route, v: t.visibility, m: t.metadata });
const templateContent = (t) => sha256({ n: t.name, d: t.description, i: t.input_schema, o: t.output_schema, s: t.system_prompt, u: t.user_prompt, r: t.default_route, v: t.visibility, m: t.metadata });
- function createTemplateVersion(key, input, { actor = 'system', trace = null } = {}) {
+ async function createTemplateVersion(key, input, { actor = 'system', trace = null } = {}) {
if (!KEY_RE.test(key || '')) throw new AiError(422, 'ai.invalid', 'template key must be lowercase dotted');
if (!KEY_RE.test(key || '')) throw new AiError(422, 'ai.invalid', 'template key must be lowercase dotted');
- const prev = decodeTemplate(latestAny('templates', key));
+ const prev = decodeTemplate(await latestAny('templates', key));
const t = {
const t = {
name: input.name || (prev && prev.name) || key,
name: input.name || (prev && prev.name) || key,
description: input.description !== undefined ? input.description : (prev ? prev.description : null),
description: input.description !== undefined ? input.description : (prev ? prev.description : null),
@@ -243,21 +243,21 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -243,21 +243,21 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
schemas.assertSchema(t.output_schema, 'output_schema');
schemas.assertSchema(t.output_schema, 'output_schema');
if (!LIFECYCLE.includes(t.status)) throw new AiError(422, 'ai.invalid', `template status must be one of ${LIFECYCLE.join(', ')}`);
if (!LIFECYCLE.includes(t.status)) throw new AiError(422, 'ai.invalid', `template status must be one of ${LIFECYCLE.join(', ')}`);
if (!['public', 'first-party', 'internal'].includes(t.visibility)) throw new AiError(422, 'ai.invalid', 'visibility must be public, first-party or internal');
if (!['public', 'first-party', 'internal'].includes(t.visibility)) throw new AiError(422, 'ai.invalid', 'visibility must be public, first-party or internal');
- const version = nextVersion('templates', key);
- db.prepare(`INSERT INTO templates (key, version, name, description, input_schema, output_schema, system_prompt, user_prompt, default_route, owner, visibility, status, metadata, created_by, created_at)
+ const version = await nextVersion('templates', key);
+ await db.prepare(`INSERT INTO templates (key, version, name, description, input_schema, output_schema, system_prompt, user_prompt, default_route, owner, visibility, status, metadata, created_by, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`)
.run(key, version, t.name, t.description, JSON.stringify(t.input_schema), JSON.stringify(t.output_schema), t.system_prompt, t.user_prompt,
.run(key, version, t.name, t.description, JSON.stringify(t.input_schema), JSON.stringify(t.output_schema), t.system_prompt, t.user_prompt,
t.default_route, t.owner, t.visibility, t.status, JSON.stringify(t.metadata), actor, now());
t.default_route, t.owner, t.visibility, t.status, JSON.stringify(t.metadata), actor, now());
- audit(actor, 'template.version', 'template', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: t.status, content_hash: templateContent(t) } });
- return getTemplate(key, version);
+ await audit(actor, 'template.version', 'template', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: t.status, content_hash: templateContent(t) } });
+ return await getTemplate(key, version);
}
}
- function getTemplate(key, version) {
- return decodeTemplate(version ? db.prepare('SELECT * FROM templates WHERE key = ? AND version = ?').get(key, version) : latestAny('templates', key));
+ async function getTemplate(key, version) {
+ return decodeTemplate(version ? await db.prepare('SELECT * FROM templates WHERE key = ? AND version = ?').get(key, version) : await latestAny('templates', key));
}
}
- function activeTemplate(key) { return decodeTemplate(latestActive('templates', key)); }
- function listTemplates({ history = false } = {}) {
- const rows = history ? db.prepare('SELECT * FROM templates ORDER BY key, version').all()
- : db.prepare('SELECT t.* FROM templates t JOIN (SELECT key, MAX(version) v FROM templates GROUP BY key) m ON m.key = t.key AND m.v = t.version ORDER BY t.key').all();
+ async function activeTemplate(key) { return decodeTemplate(await latestActive('templates', key)); }
+ async function listTemplates({ history = false } = {}) {
+ const rows = history ? await db.prepare('SELECT * FROM templates ORDER BY key, version').all()
+ : await db.prepare('SELECT t.* FROM templates t JOIN (SELECT key, MAX(version) v FROM templates GROUP BY key) m ON m.key = t.key AND m.v = t.version ORDER BY t.key').all();
return rows.map(decodeTemplate);
return rows.map(decodeTemplate);
}
}
@@ -272,9 +272,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -272,9 +272,9 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
};
};
}
}
const workflowContent = (w) => sha256({ n: w.name, d: w.description, ns: w.namespace, i: w.input_schema, o: w.output_schema, s: w.steps, r: w.default_route, c: w.cache_mode, ttl: w.cache_ttl_sec, m: w.metadata });
const workflowContent = (w) => sha256({ n: w.name, d: w.description, ns: w.namespace, i: w.input_schema, o: w.output_schema, s: w.steps, r: w.default_route, c: w.cache_mode, ttl: w.cache_ttl_sec, m: w.metadata });
- function createWorkflowVersion(key, input, { actor = 'system', trace = null } = {}) {
+ async function createWorkflowVersion(key, input, { actor = 'system', trace = null } = {}) {
if (!KEY_RE.test(key || '') || key.split('.').length < 2) throw new AiError(422, 'ai.invalid', 'workflow key must be <namespace>.<name>');
if (!KEY_RE.test(key || '') || key.split('.').length < 2) throw new AiError(422, 'ai.invalid', 'workflow key must be <namespace>.<name>');
- const prev = decodeWorkflow(latestAny('workflows', key));
+ const prev = decodeWorkflow(await latestAny('workflows', key));
const w = {
const w = {
name: input.name || (prev && prev.name) || key,
name: input.name || (prev && prev.name) || key,
description: input.description !== undefined ? input.description : (prev ? prev.description : null),
description: input.description !== undefined ? input.description : (prev ? prev.description : null),
@@ -293,39 +293,39 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -293,39 +293,39 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
if (!Array.isArray(w.steps) || !w.steps.length) throw new AiError(422, 'ai.invalid', 'workflow needs at least one step');
if (!Array.isArray(w.steps) || !w.steps.length) throw new AiError(422, 'ai.invalid', 'workflow needs at least one step');
for (const s of w.steps) {
for (const s of w.steps) {
if (!s || !['llm', 'passthrough', 'transcribe', 'embed', 'media_analysis'].includes(s.kind)) throw new AiError(422, 'ai.invalid', 'step kind must be llm, passthrough, transcribe, embed or media_analysis');
if (!s || !['llm', 'passthrough', 'transcribe', 'embed', 'media_analysis'].includes(s.kind)) throw new AiError(422, 'ai.invalid', 'step kind must be llm, passthrough, transcribe, embed or media_analysis');
- if ((s.kind === 'llm' || (s.kind === 'media_analysis' && s.template)) && !(s.template && latestAny('templates', s.template))) throw new AiError(422, 'ai.invalid', `step references unknown template ${s.template}`);
+ if ((s.kind === 'llm' || (s.kind === 'media_analysis' && s.template)) && !(s.template && await latestAny('templates', s.template))) throw new AiError(422, 'ai.invalid', `step references unknown template ${s.template}`);
}
}
if (!['none', 'private', 'service'].includes(w.cache_mode)) throw new AiError(422, 'ai.invalid', 'cache_mode must be none, private or service');
if (!['none', 'private', 'service'].includes(w.cache_mode)) throw new AiError(422, 'ai.invalid', 'cache_mode must be none, private or service');
if (!LIFECYCLE.includes(w.status)) throw new AiError(422, 'ai.invalid', `workflow status must be one of ${LIFECYCLE.join(', ')}`);
if (!LIFECYCLE.includes(w.status)) throw new AiError(422, 'ai.invalid', `workflow status must be one of ${LIFECYCLE.join(', ')}`);
- const version = nextVersion('workflows', key);
- db.prepare(`INSERT INTO workflows (key, version, name, description, namespace, input_schema, output_schema, steps, default_route, cache_mode, cache_ttl_sec, status, metadata, created_by, created_at)
+ const version = await nextVersion('workflows', key);
+ await db.prepare(`INSERT INTO workflows (key, version, name, description, namespace, input_schema, output_schema, steps, default_route, cache_mode, cache_ttl_sec, status, metadata, created_by, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`)
.run(key, version, w.name, w.description, w.namespace, JSON.stringify(w.input_schema), JSON.stringify(w.output_schema), JSON.stringify(w.steps),
.run(key, version, w.name, w.description, w.namespace, JSON.stringify(w.input_schema), JSON.stringify(w.output_schema), JSON.stringify(w.steps),
w.default_route, w.cache_mode, w.cache_ttl_sec, w.status, JSON.stringify(w.metadata), actor, now());
w.default_route, w.cache_mode, w.cache_ttl_sec, w.status, JSON.stringify(w.metadata), actor, now());
- audit(actor, 'workflow.version', 'workflow', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: w.status, content_hash: workflowContent(w) } });
- return getWorkflow(key, version);
+ await audit(actor, 'workflow.version', 'workflow', key, { trace, metadata: { version, previous: prev ? prev.version : null, status: w.status, content_hash: workflowContent(w) } });
+ return await getWorkflow(key, version);
}
}
- function getWorkflow(key, version) {
- return decodeWorkflow(version ? db.prepare('SELECT * FROM workflows WHERE key = ? AND version = ?').get(key, version) : latestAny('workflows', key));
+ async function getWorkflow(key, version) {
+ return decodeWorkflow(version ? await db.prepare('SELECT * FROM workflows WHERE key = ? AND version = ?').get(key, version) : await latestAny('workflows', key));
}
}
- function activeWorkflow(key) { return decodeWorkflow(latestActive('workflows', key)); }
- function listWorkflows({ history = false, namespace } = {}) {
- let rows = history ? db.prepare('SELECT * FROM workflows ORDER BY key, version').all()
- : db.prepare('SELECT w.* FROM workflows w JOIN (SELECT key, MAX(version) v FROM workflows GROUP BY key) m ON m.key = w.key AND m.v = w.version ORDER BY w.key').all();
+ async function activeWorkflow(key) { return decodeWorkflow(await latestActive('workflows', key)); }
+ async function listWorkflows({ history = false, namespace } = {}) {
+ let rows = history ? await db.prepare('SELECT * FROM workflows ORDER BY key, version').all()
+ : await db.prepare('SELECT w.* FROM workflows w JOIN (SELECT key, MAX(version) v FROM workflows GROUP BY key) m ON m.key = w.key AND m.v = w.version ORDER BY w.key').all();
if (namespace) rows = rows.filter(r => r.namespace === namespace);
if (namespace) rows = rows.filter(r => r.namespace === namespace);
return rows.map(decodeWorkflow);
return rows.map(decodeWorkflow);
}
}
/** Lifecycle change on one version of a template/workflow/route (status only; audited). */
/** Lifecycle change on one version of a template/workflow/route (status only; audited). */
- function setStatus(kind, key, version, status, { actor = 'system', trace = null } = {}) {
+ async function setStatus(kind, key, version, status, { actor = 'system', trace = null } = {}) {
const table = { template: 'templates', workflow: 'workflows', route: 'routes' }[kind];
const table = { template: 'templates', workflow: 'workflows', route: 'routes' }[kind];
if (!table) throw new AiError(400, 'ai.invalid', 'unknown kind');
if (!table) throw new AiError(400, 'ai.invalid', 'unknown kind');
const allowed = kind === 'route' ? ['active', 'disabled'] : LIFECYCLE;
const allowed = kind === 'route' ? ['active', 'disabled'] : LIFECYCLE;
if (!allowed.includes(status)) throw new AiError(422, 'ai.invalid', `status must be one of ${allowed.join(', ')}`);
if (!allowed.includes(status)) throw new AiError(422, 'ai.invalid', `status must be one of ${allowed.join(', ')}`);
- const row = db.prepare(`SELECT status FROM ${table} WHERE key = ? AND version = ?`).get(key, version);
+ const row = await db.prepare(`SELECT status FROM ${table} WHERE key = ? AND version = ?`).get(key, version);
if (!row) throw new AiError(404, 'ai.not_found', `no ${kind} ${key} v${version}`);
if (!row) throw new AiError(404, 'ai.not_found', `no ${kind} ${key} v${version}`);
- db.prepare(`UPDATE ${table} SET status = ? WHERE key = ? AND version = ?`).run(status, key, version);
- audit(actor, `${kind}.status`, kind, key, { trace, metadata: { version, from: row.status, to: status } });
+ await db.prepare(`UPDATE ${table} SET status = ? WHERE key = ? AND version = ?`).run(status, key, version);
+ await audit(actor, `${kind}.status`, kind, key, { trace, metadata: { version, from: row.status, to: status } });
return { key, version, status };
return { key, version, status };
}
}
@@ -333,13 +333,13 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -333,13 +333,13 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
* Seeding: insert a code-defined record when it is missing, or add a new version when the code
* Seeding: insert a code-defined record when it is missing, or add a new version when the code
* changed it and nobody edited it through the admin API since (admin edits always win).
* changed it and nobody edited it through the admin API since (admin edits always win).
*/
*/
- function seedVersioned(kind, key, def) {
+ async function seedVersioned(kind, key, def) {
const [table, create, decode, content] = {
const [table, create, decode, content] = {
route: ['routes', createRouteVersion, decodeRoute, routeContent],
route: ['routes', createRouteVersion, decodeRoute, routeContent],
template: ['templates', createTemplateVersion, decodeTemplate, templateContent],
template: ['templates', createTemplateVersion, decodeTemplate, templateContent],
workflow: ['workflows', createWorkflowVersion, decodeWorkflow, workflowContent],
workflow: ['workflows', createWorkflowVersion, decodeWorkflow, workflowContent],
}[kind];
}[kind];
- const latest = decode(latestAny(table, key));
+ const latest = decode(await latestAny(table, key));
if (!latest) return create(key, def, { actor: 'seed' });
if (!latest) return create(key, def, { actor: 'seed' });
if (latest.created_by !== 'seed') return latest;
if (latest.created_by !== 'seed') return latest;
const merged = { ...latest, ...def };
const merged = { ...latest, ...def };
@@ -93,22 +93,22 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -93,22 +93,22 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
created_at: r.created_at, started_at: r.started_at, finished_at: r.finished_at,
created_at: r.created_at, started_at: r.started_at, finished_at: r.finished_at,
};
};
}
}
- const getRow = (id) => db.prepare('SELECT * FROM runs WHERE id = ?').get(id);
- const get = (id) => decode(getRow(id));
+ const getRow = async (id) => await db.prepare('SELECT * FROM runs WHERE id = ?').get(id);
+ const get = async (id) => decode(await getRow(id));
- function citations(runId) {
- return db.prepare('SELECT * FROM citations WHERE run_id = ? ORDER BY ordinal, id').all(runId)
+ async function citations(runId) {
+ return (await db.prepare('SELECT * FROM citations WHERE run_id = ? ORDER BY ordinal, id').all(runId))
.map(c => ({ ...c, trust: parseJson(c.trust, {}), provenance: parseJson(c.provenance, {}) }));
.map(c => ({ ...c, trust: parseJson(c.trust, {}), provenance: parseJson(c.provenance, {}) }));
}
}
- function addCitations(runId, list, attachedBy) {
+ async function addCitations(runId, list, attachedBy) {
const ins = db.prepare(`INSERT INTO citations (run_id, ordinal, source_type, source_id, url, title, author, published_at, retrieved_at, snippet, content_hash, trust, provenance, attached_by, created_at)
const ins = db.prepare(`INSERT INTO citations (run_id, ordinal, source_type, source_id, url, title, author, published_at, retrieved_at, snippet, content_hash, trust, provenance, attached_by, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`);
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`);
- const base = db.prepare('SELECT COUNT(*) AS n FROM citations WHERE run_id = ?').get(runId).n;
- db.transaction(() => {
- list.forEach((c, i) => ins.run(runId, c.ordinal != null ? c.ordinal : base + i, c.source_type, c.source_id || null, c.url || null, c.title || null, c.author || null,
- c.published_at || null, c.retrieved_at || null, c.snippet || null, c.content_hash || null, JSON.stringify(c.trust || {}), JSON.stringify(c.provenance || {}), attachedBy, iso(clock.now())));
- db.prepare('UPDATE runs SET citations_count = (SELECT COUNT(*) FROM citations WHERE run_id = ?) WHERE id = ?').run(runId, runId);
- })();
+ const base = (await db.prepare('SELECT COUNT(*) AS n FROM citations WHERE run_id = ?').get(runId)).n;
+ await db.tx(async () => {
+ for (const [i, c] of list.entries()) await ins.run(runId, c.ordinal != null ? c.ordinal : base + i, c.source_type, c.source_id || null, c.url || null, c.title || null, c.author || null,
+ c.published_at || null, c.retrieved_at || null, c.snippet || null, c.content_hash || null, JSON.stringify(c.trust || {}), JSON.stringify(c.provenance || {}), attachedBy, iso(clock.now()));
+ await db.prepare('UPDATE runs SET citations_count = (SELECT COUNT(*) FROM citations WHERE run_id = ?) WHERE id = ?').run(runId, runId);
+ });
}
}
const insReq = db.prepare(`INSERT INTO requests (run_id, seq, operation, provider_key, model_key, route_key, route_version, status, skip_reason, fallback, prompt_hash, input_hash, output_hash,
const insReq = db.prepare(`INSERT INTO requests (run_id, seq, operation, provider_key, model_key, route_key, route_version, status, skip_reason, fallback, prompt_hash, input_hash, output_hash,
@@ -138,10 +138,10 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -138,10 +138,10 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
* principal: { sub, subject: {type, id} }
* principal: { sub, subject: {type, id} }
* Returns { run, created, promise? }.
* Returns { run, created, promise? }.
*/
*/
- function create(body, principal, { trace = null, requestId = null, retryOf = null } = {}) {
+ async function create(body, principal, { trace = null, requestId = null, retryOf = null } = {}) {
if (!body || typeof body !== 'object') throw new AiError(400, 'input.invalid', 'JSON body required');
if (!body || typeof body !== 'object') throw new AiError(400, 'input.invalid', 'JSON body required');
const wfKey = String(body.workflow || '');
const wfKey = String(body.workflow || '');
- const wf = body.version ? registry.getWorkflow(wfKey, Number(body.version)) : registry.activeWorkflow(wfKey);
+ const wf = body.version ? await registry.getWorkflow(wfKey, Number(body.version)) : await registry.activeWorkflow(wfKey);
if (!wf) throw new AiError(404, 'workflow.not_found', `no active workflow ${wfKey}${body.version ? ` v${body.version}` : ''}`);
if (!wf) throw new AiError(404, 'workflow.not_found', `no active workflow ${wfKey}${body.version ? ` v${body.version}` : ''}`);
if (body.version && !['active', 'deprecated'].includes(wf.status)) throw new AiError(409, 'workflow.inactive', `${wfKey} v${wf.version} is ${wf.status}`);
if (body.version && !['active', 'deprecated'].includes(wf.status)) throw new AiError(409, 'workflow.inactive', `${wfKey} v${wf.version} is ${wf.status}`);
validateRefs(body);
validateRefs(body);
@@ -156,7 +156,7 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -156,7 +156,7 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
if (idem !== null && !/^[A-Za-z0-9._:-]{8,128}$/.test(idem)) throw new AiError(422, 'input.invalid', 'idempotency_key must be 8-128 of [A-Za-z0-9._:-]');
if (idem !== null && !/^[A-Za-z0-9._:-]{8,128}$/.test(idem)) throw new AiError(422, 'input.invalid', 'idempotency_key must be 8-128 of [A-Za-z0-9._:-]');
if (idem) {
if (idem) {
- const prev = db.prepare('SELECT * FROM runs WHERE requester_type = ? AND requester_id = ? AND idempotency_key = ?').get(requester.type, requester.id, idem);
+ const prev = await db.prepare('SELECT * FROM runs WHERE requester_type = ? AND requester_id = ? AND idempotency_key = ?').get(requester.type, requester.id, idem);
if (prev) {
if (prev) {
if (prev.input_hash !== inputHash || prev.workflow_key !== wf.key) throw new AiError(409, 'idempotency.conflict', 'this idempotency_key was used with a different workflow or input');
if (prev.input_hash !== inputHash || prev.workflow_key !== wf.key) throw new AiError(409, 'idempotency.conflict', 'this idempotency_key was used with a different workflow or input');
return { run: decode(prev), created: false, promise: inflight.get(prev.id) ? inflight.get(prev.id).promise : null };
return { run: decode(prev), created: false, promise: inflight.get(prev.id) ? inflight.get(prev.id).promise : null };
@@ -164,16 +164,16 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -164,16 +164,16 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
}
}
const llmStep = wf.steps.find(s => s.kind === 'llm');
const llmStep = wf.steps.find(s => s.kind === 'llm');
- const tpl = llmStep ? registry.activeTemplate(llmStep.template) : null;
+ const tpl = llmStep ? await registry.activeTemplate(llmStep.template) : null;
const routeKey = (wf.steps[0] && wf.steps[0].route) || (tpl && tpl.default_route) || wf.default_route || null;
const routeKey = (wf.steps[0] && wf.steps[0].route) || (tpl && tpl.default_route) || wf.default_route || null;
- const route = routeKey && !String(routeKey).includes('{') ? registry.resolveRoute(routeKey) : null;
+ const route = routeKey && !String(routeKey).includes('{') ? await registry.resolveRoute(routeKey) : null;
const options = body.options && typeof body.options === 'object' ? body.options : {};
const options = body.options && typeof body.options === 'object' ? body.options : {};
// A person's own key (WS-O task 2): only the service that stored it, within its daily budget; never cached.
// A person's own key (WS-O task 2): only the service that stored it, within its daily budget; never cached.
const credentialSubject = body.credential && typeof body.credential === 'object' ? String(body.credential.subject || '') : null;
const credentialSubject = body.credential && typeof body.credential === 'object' ? String(body.credential.subject || '') : null;
if (credentialSubject !== null) {
if (credentialSubject !== null) {
if (!credentials) throw new AiError(503, 'credentials.unavailable', 'person credentials are not configured on this service');
if (!credentials) throw new AiError(503, 'credentials.unavailable', 'person credentials are not configured on this service');
if (requester.type !== 'service') throw new AiError(403, 'capability.denied', 'only the service that stored a credential runs with it');
if (requester.type !== 'service') throw new AiError(403, 'capability.denied', 'only the service that stored a credential runs with it');
- credentials.admit(requester.id, credentialSubject);
+ await credentials.admit(requester.id, credentialSubject);
}
}
const ctx = {
const ctx = {
requesterType: requester.type, requesterId: requester.id,
requesterType: requester.type, requesterId: requester.id,
@@ -200,22 +200,22 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -200,22 +200,22 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
let cacheKey = null;
let cacheKey = null;
if (scope) {
if (scope) {
cacheKey = cache.keyFor({ scope, workflowKey: wf.key, workflowVersion: wf.version, templateVersion: base.template_version, routeKey: base.route_key, routeVersion: base.route_version, inputHash });
cacheKey = cache.keyFor({ scope, workflowKey: wf.key, workflowVersion: wf.version, templateVersion: base.template_version, routeKey: base.route_key, routeVersion: base.route_version, inputHash });
- const hit = cache.get(cacheKey, scope);
+ const hit = await cache.get(cacheKey, scope);
if (hit) {
if (hit) {
- db.transaction(() => {
- db.prepare(`INSERT INTO runs (id, workflow_key, workflow_version, template_key, template_version, route_key, route_version, status, requester_type, requester_id, on_behalf_of, attribution,
+ await db.tx(async () => {
+ await db.prepare(`INSERT INTO runs (id, workflow_key, workflow_version, template_key, template_version, route_key, route_version, status, requester_type, requester_id, on_behalf_of, attribution,
source_service, target, target_key, input, input_hash, output, model_key, cache_key, cached_from, retry_of, idempotency_key, trace_id, request_id, options, created_at, started_at, finished_at, citations_count)
source_service, target, target_key, input, input_hash, output, model_key, cache_key, cached_from, retry_of, idempotency_key, trace_id, request_id, options, created_at, started_at, finished_at, citations_count)
VALUES (@id, @workflow_key, @workflow_version, @template_key, @template_version, @route_key, @route_version, 'cached', @requester_type, @requester_id, @on_behalf_of, @attribution,
VALUES (@id, @workflow_key, @workflow_version, @template_key, @template_version, @route_key, @route_version, 'cached', @requester_type, @requester_id, @on_behalf_of, @attribution,
@source_service, @target, @target_key, @input, @input_hash, @output, @model_key, @cache_key, @cached_from, @retry_of, @idempotency_key, @trace_id, @request_id, @options, @created_at, @created_at, @created_at, 0)`)
@source_service, @target, @target_key, @input, @input_hash, @output, @model_key, @cache_key, @cached_from, @retry_of, @idempotency_key, @trace_id, @request_id, @options, @created_at, @created_at, @created_at, 0)`)
.run({ ...base, output: JSON.stringify(hit.output), model_key: hit.model_key, cache_key: cacheKey, cached_from: hit.run_id });
.run({ ...base, output: JSON.stringify(hit.output), model_key: hit.model_key, cache_key: cacheKey, cached_from: hit.run_id });
- const src = citations(hit.run_id);
- if (src.length) addCitations(id, src.map(c => ({ ...c, provenance: { ...c.provenance, via_cache: hit.run_id } })), 'cache');
+ const src = await citations(hit.run_id);
+ if (src.length) await addCitations(id, src.map(c => ({ ...c, provenance: { ...c.provenance, via_cache: hit.run_id } })), 'cache');
// The reused output carries the grounding it was produced with.
// The reused output carries the grounding it was produced with.
- db.prepare('UPDATE runs SET grounding = (SELECT grounding FROM runs WHERE id = ?) WHERE id = ?').run(hit.run_id, id);
- events.runChanged(getRow(id));
- })();
- registry.audit(principal.sub, 'run.create', 'run', id, { trace, metadata: { workflow: wf.key, version: wf.version, status: 'cached', cached_from: hit.run_id } });
- return { run: get(id), created: true, promise: null };
+ await db.prepare('UPDATE runs SET grounding = (SELECT grounding FROM runs WHERE id = ?) WHERE id = ?').run(hit.run_id, id);
+ events.runChanged(await getRow(id));
+ });
+ await registry.audit(principal.sub, 'run.create', 'run', id, { trace, metadata: { workflow: wf.key, version: wf.version, status: 'cached', cached_from: hit.run_id } });
+ return { run: await get(id), created: true, promise: null };
}
}
}
}
@@ -223,26 +223,26 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -223,26 +223,26 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
const caller = `${requester.type}:${requester.id}`;
const caller = `${requester.type}:${requester.id}`;
admit(caller);
admit(caller);
// A run with a person's own key spends their money, not the shared quotas (its own budget was checked above).
// A run with a person's own key spends their money, not the shared quotas (its own budget was checked above).
- const reserved = credentialSubject !== null ? null : quotas.reserve(ctx);
+ const reserved = credentialSubject !== null ? null : await quotas.reserve(ctx);
- db.transaction(() => {
- db.prepare(`INSERT INTO runs (id, workflow_key, workflow_version, template_key, template_version, route_key, route_version, status, requester_type, requester_id, on_behalf_of, attribution,
+ await db.tx(async () => {
+ await db.prepare(`INSERT INTO runs (id, workflow_key, workflow_version, template_key, template_version, route_key, route_version, status, requester_type, requester_id, on_behalf_of, attribution,
source_service, target, target_key, input, input_hash, cache_key, retry_of, idempotency_key, trace_id, request_id, options, created_at)
source_service, target, target_key, input, input_hash, cache_key, retry_of, idempotency_key, trace_id, request_id, options, created_at)
VALUES (@id, @workflow_key, @workflow_version, @template_key, @template_version, @route_key, @route_version, 'queued', @requester_type, @requester_id, @on_behalf_of, @attribution,
VALUES (@id, @workflow_key, @workflow_version, @template_key, @template_version, @route_key, @route_version, 'queued', @requester_type, @requester_id, @on_behalf_of, @attribution,
@source_service, @target, @target_key, @input, @input_hash, @cache_key, @retry_of, @idempotency_key, @trace_id, @request_id, @options, @created_at)`)
@source_service, @target, @target_key, @input, @input_hash, @cache_key, @retry_of, @idempotency_key, @trace_id, @request_id, @options, @created_at)`)
.run({ ...base, cache_key: cacheKey });
.run({ ...base, cache_key: cacheKey });
- events.runChanged(getRow(id));
- })();
- registry.audit(principal.sub, retryOf ? 'run.retry' : 'run.create', 'run', id, { trace, metadata: { workflow: wf.key, version: wf.version, target: ctx.targetKey, retry_of: retryOf } });
+ events.runChanged(await getRow(id));
+ });
+ await registry.audit(principal.sub, retryOf ? 'run.retry' : 'run.create', 'run', id, { trace, metadata: { workflow: wf.key, version: wf.version, target: ctx.targetKey, retry_of: retryOf } });
const controller = new AbortController();
const controller = new AbortController();
let release;
let release;
const gate = new Promise((r) => { release = r; });
const gate = new Promise((r) => { release = r; });
- const promise = gate.then(() => execute(id, { controller, reserved, scope, ctx, input })).catch((err) => { log.error(`[runs] ${id}: ${err.stack || err}`); });
+ const promise = gate.then(async () => await execute(id, { controller, reserved, scope, ctx, input })).catch((err) => { log.error(`[runs] ${id}: ${err.stack || err}`); });
inflight.set(id, { controller, promise, release, caller });
inflight.set(id, { controller, promise, release, caller });
queue.push(id);
queue.push(id);
pump();
pump();
- return { run: get(id), created: true, promise };
+ return { run: await get(id), created: true, promise };
}
}
/** Refuse a run that would wait in a full queue, globally or for this caller. */
/** Refuse a run that would wait in a full queue, globally or for this caller. */
@@ -267,16 +267,16 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -267,16 +267,16 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
}
}
async function execute(id, { controller, reserved, scope, ctx, input }) {
async function execute(id, { controller, reserved, scope, ctx, input }) {
- const row = getRow(id);
+ const row = await getRow(id);
if (!row || row.status !== 'queued') { inflight.delete(id); return; }
if (!row || row.status !== 'queued') { inflight.delete(id); return; }
- db.prepare("UPDATE runs SET status = 'running', started_at = ? WHERE id = ? AND status = 'queued'").run(iso(clock.now()), id);
- const wf = registry.getWorkflow(row.workflow_key, row.workflow_version);
+ await db.prepare("UPDATE runs SET status = 'running', started_at = ? WHERE id = ? AND status = 'queued'").run(iso(clock.now()), id);
+ const wf = await registry.getWorkflow(row.workflow_key, row.workflow_version);
const opts = parseJson(row.options, {});
const opts = parseJson(row.options, {});
let seq = 0;
let seq = 0;
let attempts = 0;
let attempts = 0;
- const logRequest = (e) => {
+ const logRequest = async (e) => {
if (e.status !== 'skipped') attempts++;
if (e.status !== 'skipped') attempts++;
- insReq.run({
+ await insReq.run({
run_id: id, seq: seq++, operation: e.operation, provider_key: e.provider_key, model_key: e.model_key || null, route_key: e.route_key || null, route_version: e.route_version || null,
run_id: id, seq: seq++, operation: e.operation, provider_key: e.provider_key, model_key: e.model_key || null, route_key: e.route_key || null, route_version: e.route_version || null,
status: e.status, skip_reason: e.skip_reason || null, fallback: e.fallback || 0, prompt_hash: e.prompt_hash || null, input_hash: row.input_hash, output_hash: e.output_hash || null,
status: e.status, skip_reason: e.skip_reason || null, fallback: e.fallback || 0, prompt_hash: e.prompt_hash || null, input_hash: row.input_hash, output_hash: e.output_hash || null,
tokens_in: e.tokens_in || 0, tokens_out: e.tokens_out || 0, tokens_cached: e.tokens_cached || 0, tokens_estimated: e.tokens_estimated || 0, cost_usd: e.cost_usd || 0,
tokens_in: e.tokens_in || 0, tokens_out: e.tokens_out || 0, tokens_cached: e.tokens_cached || 0, tokens_estimated: e.tokens_estimated || 0, cost_usd: e.cost_usd || 0,
@@ -290,29 +290,29 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -290,29 +290,29 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
// perspective shape the prompt; history: false keeps neither their input nor a cache entry.
// perspective shape the prompt; history: false keeps neither their input nor a cache entry.
const preferences = userModules && row.on_behalf_of ? await userModules.preferencesFor(row.on_behalf_of) : {};
const preferences = userModules && row.on_behalf_of ? await userModules.preferencesFor(row.on_behalf_of) : {};
if (preferences.history === false) {
if (preferences.history === false) {
- db.prepare("UPDATE runs SET input = 'null', cache_key = NULL, options = json_set(options, '$.input_retained', json('false'), '$.history', json('false')) WHERE id = ?").run(id);
+ await db.prepare("UPDATE runs SET input = 'null', cache_key = NULL, options = (COALESCE(options, '{}')::jsonb || jsonb_build_object('input_retained', false, 'history', false))::text WHERE id = ?").run(id);
}
}
- const credential = opts.credential_subject ? credentials.forRun(row.requester_id, opts.credential_subject) : null;
+ const credential = opts.credential_subject ? await credentials.forRun(row.requester_id, opts.credential_subject) : null;
const r = await engine.execute(wf, row, input, { signal: controller.signal, logRequest, debugRaw: Boolean(config.debugRawLog && opts.debug), inputHash: row.input_hash, preferences,
const r = await engine.execute(wf, row, input, { signal: controller.signal, logRequest, debugRaw: Boolean(config.debugRawLog && opts.debug), inputHash: row.input_hash, preferences,
...(credential ? { credential, role: input && typeof input.role === 'string' ? input.role : null } : {}) });
...(credential ? { credential, role: input && typeof input.role === 'string' ? input.role : null } : {}) });
if (controller.signal.aborted) throw new AiError(409, 'run.cancelled', 'cancelled');
if (controller.signal.aborted) throw new AiError(409, 'run.cancelled', 'cancelled');
// The status, its citations and the ai.run.succeeded event commit together.
// The status, its citations and the ai.run.succeeded event commit together.
- const done = db.transaction(() => {
- const d = db.prepare(`UPDATE runs SET status = 'succeeded', output = ?, grounding = ?, synthetic = ?, provider_key = ?, model_key = ?, fallback_used = ?, attempts = ?, tokens_in = ?, tokens_out = ?, cost_usd = ?,
+ const done = await db.tx(async () => {
+ const d = await db.prepare(`UPDATE runs SET status = 'succeeded', output = ?, grounding = ?, synthetic = ?, provider_key = ?, model_key = ?, fallback_used = ?, attempts = ?, tokens_in = ?, tokens_out = ?, cost_usd = ?,
route_key = COALESCE(?, route_key), route_version = COALESCE(?, route_version), finished_at = ? WHERE id = ? AND status = 'running'`)
route_key = COALESCE(?, route_key), route_version = COALESCE(?, route_version), finished_at = ? WHERE id = ? AND status = 'running'`)
.run(JSON.stringify(r.output), JSON.stringify(r.grounding || null), r.synthetic ? 1 : 0, r.provider, r.model, r.fallbackUsed ? 1 : 0, attempts, r.usage.input, r.usage.output, r.cost,
.run(JSON.stringify(r.output), JSON.stringify(r.grounding || null), r.synthetic ? 1 : 0, r.provider, r.model, r.fallbackUsed ? 1 : 0, attempts, r.usage.input, r.usage.output, r.cost,
r.route ? r.route.key : null, r.route ? r.route.version : null, iso(clock.now()), id);
r.route ? r.route.key : null, r.route ? r.route.version : null, iso(clock.now()), id);
if (!d.changes) return d;
if (!d.changes) return d;
- if (r.citations.length) addCitations(id, r.citations, 'workflow');
- events.runChanged(getRow(id));
+ if (r.citations.length) await addCitations(id, r.citations, 'workflow');
+ events.runChanged(await getRow(id));
return d;
return d;
- })();
+ });
if (!done.changes) return; // cancelled while finishing
if (!done.changes) return; // cancelled while finishing
- if (r.provider && reserved) quotas.account(ctx, reserved, { provider: r.provider, model: r.model, tokensIn: r.usage.input, tokensOut: r.usage.output, tokensCached: r.usage.cached, cost: r.cost });
- if (r.fallbackUsed) registry.audit('system', 'run.fallback', 'run', id, { trace: row.trace_id, metadata: { workflow: row.workflow_key, provider: r.provider, route: r.route && r.route.key } });
+ if (r.provider && reserved) await quotas.account(ctx, reserved, { provider: r.provider, model: r.model, tokensIn: r.usage.input, tokensOut: r.usage.output, tokensCached: r.usage.cached, cost: r.cost });
+ if (r.fallbackUsed) await registry.audit('system', 'run.fallback', 'run', id, { trace: row.trace_id, metadata: { workflow: row.workflow_key, provider: r.provider, route: r.route && r.route.key } });
// Output shaped by someone's preferences, or kept for nobody (history off), is never cached for reuse.
// Output shaped by someone's preferences, or kept for nobody (history off), is never cached for reuse.
if (scope && row.cache_key && !r.synthetic && wf.cache_mode !== 'none' && preferences.history !== false && !preferenceLines(preferences)) {
if (scope && row.cache_key && !r.synthetic && wf.cache_mode !== 'none' && preferences.history !== false && !preferenceLines(preferences)) {
- cache.put({ key: row.cache_key, scope, privacy: wf.cache_mode, workflowKey: wf.key, workflowVersion: wf.version, templateVersion: row.template_version, routeKey: row.route_key, routeVersion: row.route_version, model: r.model, inputHash: row.input_hash, output: r.output, runId: id, ttlSec: wf.cache_ttl_sec || config.runs.defaultCacheTtlSec });
+ await cache.put({ key: row.cache_key, scope, privacy: wf.cache_mode, workflowKey: wf.key, workflowVersion: wf.version, templateVersion: row.template_version, routeKey: row.route_key, routeVersion: row.route_version, model: r.model, inputHash: row.input_hash, output: r.output, runId: id, ttlSec: wf.cache_ttl_sec || config.runs.defaultCacheTtlSec });
}
}
} catch (err) {
} catch (err) {
const interrupted = shuttingDown && controller.signal.aborted;
const interrupted = shuttingDown && controller.signal.aborted;
@@ -321,13 +321,13 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -321,13 +321,13 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
const detail = interrupted ? 'the service shut down while this run was in progress' : cancelled ? 'cancelled' : (err instanceof AiError ? err.detail : 'internal error');
const detail = interrupted ? 'the service shut down while this run was in progress' : cancelled ? 'cancelled' : (err instanceof AiError ? err.detail : 'internal error');
if (!(err instanceof AiError) && !cancelled) log.error(`[runs] ${id} crashed: ${err && err.stack || err}`);
if (!(err instanceof AiError) && !cancelled) log.error(`[runs] ${id} crashed: ${err && err.stack || err}`);
const extra = err instanceof AiError && err.extra ? JSON.stringify(err.extra).slice(0, 2000) : null;
const extra = err instanceof AiError && err.extra ? JSON.stringify(err.extra).slice(0, 2000) : null;
- db.transaction(() => {
- const f = db.prepare(`UPDATE runs SET status = ?, error_code = ?, error_detail = ?, attempts = ?, finished_at = ? WHERE id = ? AND status IN ('queued', 'running')`)
+ await db.tx(async () => {
+ const f = await db.prepare(`UPDATE runs SET status = ?, error_code = ?, error_detail = ?, attempts = ?, finished_at = ? WHERE id = ? AND status IN ('queued', 'running')`)
.run(cancelled ? 'cancelled' : 'failed', code, extra ? `${detail} ${extra}` : detail, attempts, iso(clock.now()), id);
.run(cancelled ? 'cancelled' : 'failed', code, extra ? `${detail} ${extra}` : detail, attempts, iso(clock.now()), id);
- if (f.changes) events.runChanged(getRow(id));
- })();
- const spent = db.prepare("SELECT COALESCE(SUM(tokens_in),0) ti, COALESCE(SUM(tokens_out),0) tout, COALESCE(SUM(cost_usd),0) c FROM requests WHERE run_id = ? AND status = 'ok'").get(id);
- if (reserved && spent && (spent.ti || spent.tout)) quotas.account(ctx, reserved, { provider: 'mixed', model: '', tokensIn: spent.ti, tokensOut: spent.tout, cost: spent.c });
+ if (f.changes) events.runChanged(await getRow(id));
+ });
+ const spent = await db.prepare("SELECT COALESCE(SUM(tokens_in),0) ti, COALESCE(SUM(tokens_out),0) tout, COALESCE(SUM(cost_usd),0) c FROM requests WHERE run_id = ? AND status = 'ok'").get(id);
+ if (reserved && spent && (spent.ti || spent.tout)) await quotas.account(ctx, reserved, { provider: 'mixed', model: '', tokensIn: spent.ti, tokensOut: spent.tout, cost: spent.c });
} finally {
} finally {
inflight.delete(id);
inflight.delete(id);
}
}
@@ -335,11 +335,11 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -335,11 +335,11 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
/** Wait for a run to reach a terminal state, up to ms. */
/** Wait for a run to reach a terminal state, up to ms. */
async function wait(created, ms) {
async function wait(created, ms) {
- if (!created.promise || ms <= 0) return get(created.run.id);
+ if (!created.promise || ms <= 0) return await get(created.run.id);
let timer;
let timer;
await Promise.race([created.promise, new Promise((r) => { timer = setTimeout(r, Math.min(ms, config.runs.maxWaitMs)); timer.unref?.(); })]);
await Promise.race([created.promise, new Promise((r) => { timer = setTimeout(r, Math.min(ms, config.runs.maxWaitMs)); timer.unref?.(); })]);
clearTimeout(timer);
clearTimeout(timer);
- return get(created.run.id);
+ return await get(created.run.id);
}
}
function assertOwner(row, principal, adminCap, principalHas) {
function assertOwner(row, principal, adminCap, principalHas) {
@@ -349,36 +349,36 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -349,36 +349,36 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
throw new AiError(404, 'run.not_found', 'no such run'); // do not reveal other callers' runs
throw new AiError(404, 'run.not_found', 'no such run'); // do not reveal other callers' runs
}
}
- function cancel(id, principal, { trace = null, principalHas } = {}) {
- const row = getRow(id);
+ async function cancel(id, principal, { trace = null, principalHas } = {}) {
+ const row = await getRow(id);
if (!row) throw new AiError(404, 'run.not_found', 'no such run');
if (!row) throw new AiError(404, 'run.not_found', 'no such run');
assertOwner(row, principal, 'ai.workflow.manage', principalHas);
assertOwner(row, principal, 'ai.workflow.manage', principalHas);
if (TERMINAL.has(row.status)) throw new AiError(409, 'run.terminal', `run is already ${row.status}`);
if (TERMINAL.has(row.status)) throw new AiError(409, 'run.terminal', `run is already ${row.status}`);
const h = inflight.get(id);
const h = inflight.get(id);
- const r = db.prepare("UPDATE runs SET status = 'cancelled', error_code = 'run.cancelled', error_detail = 'cancelled by caller', finished_at = ? WHERE id = ? AND status IN ('queued', 'running')").run(iso(clock.now()), id);
+ const r = await db.prepare("UPDATE runs SET status = 'cancelled', error_code = 'run.cancelled', error_detail = 'cancelled by caller', finished_at = ? WHERE id = ? AND status IN ('queued', 'running')").run(iso(clock.now()), id);
if (h) {
if (h) {
h.controller.abort();
h.controller.abort();
const i = queue.indexOf(id);
const i = queue.indexOf(id);
if (i >= 0) { queue.splice(i, 1); h.release(); }
if (i >= 0) { queue.splice(i, 1); h.release(); }
}
}
- registry.audit(principal.sub, 'run.cancel', 'run', id, { trace, metadata: { from: row.status, changed: r.changes } });
- return get(id);
+ await registry.audit(principal.sub, 'run.cancel', 'run', id, { trace, metadata: { from: row.status, changed: r.changes } });
+ return await get(id);
}
}
- function retry(id, principal, { trace = null, requestId = null, principalHas } = {}) {
- const row = getRow(id);
+ async function retry(id, principal, { trace = null, requestId = null, principalHas } = {}) {
+ const row = await getRow(id);
if (!row) throw new AiError(404, 'run.not_found', 'no such run');
if (!row) throw new AiError(404, 'run.not_found', 'no such run');
assertOwner(row, principal, 'ai.workflow.manage', principalHas);
assertOwner(row, principal, 'ai.workflow.manage', principalHas);
if (!['failed', 'cancelled'].includes(row.status)) throw new AiError(409, 'run.not_retryable', `only failed or cancelled runs can be retried (this one is ${row.status})`);
if (!['failed', 'cancelled'].includes(row.status)) throw new AiError(409, 'run.not_retryable', `only failed or cancelled runs can be retried (this one is ${row.status})`);
if (parseJson(row.options, {}).input_retained === false) throw new AiError(409, 'run.input_not_retained', 'this run did not keep its input (inline media or a passthrough prompt); submit it again');
if (parseJson(row.options, {}).input_retained === false) throw new AiError(409, 'run.input_not_retained', 'this run did not keep its input (inline media or a passthrough prompt); submit it again');
- return create({
+ return await create({
workflow: row.workflow_key, input: parseJson(row.input, {}), target: parseJson(row.target, null) || undefined,
workflow: row.workflow_key, input: parseJson(row.input, {}), target: parseJson(row.target, null) || undefined,
attribution: parseJson(row.attribution, null) || undefined, on_behalf_of: parseJson(row.on_behalf_of, null) || undefined,
attribution: parseJson(row.attribution, null) || undefined, on_behalf_of: parseJson(row.on_behalf_of, null) || undefined,
options: { ...parseJson(row.options, {}), cache: false },
options: { ...parseJson(row.options, {}), cache: false },
}, principal, { trace, requestId, retryOf: id });
}, principal, { trace, requestId, retryOf: id });
}
}
- function list(principal, { status, workflow, target, limit = 50, before, all = false } = {}) {
+ async function list(principal, { status, workflow, target, limit = 50, before, all = false } = {}) {
const where = [];
const where = [];
const args = [];
const args = [];
if (!all) { where.push('requester_type = ? AND requester_id = ?'); args.push(principal.subject.type, principal.subject.id); }
if (!all) { where.push('requester_type = ? AND requester_id = ?'); args.push(principal.subject.type, principal.subject.id); }
@@ -386,33 +386,33 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
@@ -386,33 +386,33 @@ function createRuns({ db, registry, engine, cache, quotas, config, clock = { now
if (workflow) { where.push('workflow_key = ?'); args.push(workflow); }
if (workflow) { where.push('workflow_key = ?'); args.push(workflow); }
if (target) { where.push('target_key = ?'); args.push(target); }
if (target) { where.push('target_key = ?'); args.push(target); }
if (before) { where.push('id < ?'); args.push(before); }
if (before) { where.push('id < ?'); args.push(before); }
- return db.prepare(`SELECT * FROM runs ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(200, Number(limit) || 50)).map(decode);
+ return (await db.prepare(`SELECT * FROM runs ${where.length ? `WHERE ${where.join(' AND ')}` : ''} ORDER BY id DESC LIMIT ?`).all(...args, Math.min(200, Number(limit) || 50))).map(decode);
}
}
- function requestsFor(runId) {
- return db.prepare('SELECT id, seq, operation, provider_key, model_key, route_key, route_version, status, skip_reason, fallback, prompt_hash, input_hash, output_hash, tokens_in, tokens_out, tokens_cached, tokens_estimated, cost_usd, latency_ms, error, created_at FROM requests WHERE run_id = ? ORDER BY seq').all(runId);
+ async function requestsFor(runId) {
+ return await db.prepare('SELECT id, seq, operation, provider_key, model_key, route_key, route_version, status, skip_reason, fallback, prompt_hash, input_hash, output_hash, tokens_in, tokens_out, tokens_cached, tokens_estimated, cost_usd, latency_ms, error, created_at FROM requests WHERE run_id = ? ORDER BY seq').all(runId);
}
}
/** Boot: runs interrupted by a restart are failed explicitly (retry is one call away). */
/** Boot: runs interrupted by a restart are failed explicitly (retry is one call away). */
- function recoverInterrupted() {
- return db.transaction(() => {
- const open = db.prepare("SELECT id FROM runs WHERE status IN ('queued', 'running')").all();
- const r = db.prepare("UPDATE runs SET status = 'failed', error_code = 'run.interrupted', error_detail = 'the service restarted while this run was in progress', finished_at = ? WHERE status IN ('queued', 'running')").run(iso(clock.now()));
- for (const { id } of open) events.runChanged(getRow(id));
+ async function recoverInterrupted() {
+ return await db.tx(async () => {
+ const open = await db.prepare("SELECT id FROM runs WHERE status IN ('queued', 'running')").all();
+ const r = await db.prepare("UPDATE runs SET status = 'failed', error_code = 'run.interrupted', error_detail = 'the service restarted while this run was in progress', finished_at = ? WHERE status IN ('queued', 'running')").run(iso(clock.now()));
+ for (const { id } of open) events.runChanged(await getRow(id));
return r.changes;
return r.changes;
- })();
+ });
}
}
- function prune(days = config.runs.retentionDays) {
+ async function prune(days = config.runs.retentionDays) {
const cutoff = iso(clock.now() - days * 86400000);
const cutoff = iso(clock.now() - days * 86400000);
- const old = db.prepare('SELECT id FROM runs WHERE finished_at IS NOT NULL AND finished_at < ?').all(cutoff).map(r => r.id);
- db.transaction(() => {
+ const old = (await db.prepare('SELECT id FROM runs WHERE finished_at IS NOT NULL AND finished_at < ?').all(cutoff)).map(r => r.id);
+ await db.tx(async () => {
for (const id of old) {
for (const id of old) {
- db.prepare('DELETE FROM requests WHERE run_id = ?').run(id);
- db.prepare('DELETE FROM citations WHERE run_id = ?').run(id);
- db.prepare('DELETE FROM runs WHERE id = ?').run(id);
+ await db.prepare('DELETE FROM requests WHERE run_id = ?').run(id);
+ await db.prepare('DELETE FROM citations WHERE run_id = ?').run(id);
+ await db.prepare('DELETE FROM runs WHERE id = ?').run(id);
}
}
- })();
+ });
return old.length;
return old.length;
}
}
@@ -67,16 +67,16 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalTh
@@ -67,16 +67,16 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalTh
}
}
function ensureSchema() {
function ensureSchema() {
- db.exec(`CREATE TABLE IF NOT EXISTS ai_module_pushes (subject_id TEXT PRIMARY KEY, hash TEXT NOT NULL, pushed_at TEXT NOT NULL)`);
+ // ai_module_pushes is in migrations/0001_initial.sql (ADR-035).
}
}
/** ai.usage_summary for one person over the 30 days before now, or null when there were no runs. */
/** ai.usage_summary for one person over the 30 days before now, or null when there were no runs. */
- function summarize(subject, now = clock.now()) {
+ async function summarize(subject, now = clock.now()) {
const since = new Date(now - 30 * DAY_MS).toISOString();
const since = new Date(now - 30 * DAY_MS).toISOString();
- const rows = db.prepare(`SELECT COALESCE(source_service, requester_id) AS svc, COUNT(*) AS n, SUM(tokens_in + tokens_out) AS tokens, MAX(created_at) AS last
- FROM runs WHERE json_extract(on_behalf_of, '$.type') = 'user' AND json_extract(on_behalf_of, '$.id') = ? AND created_at >= ?
+ const rows = await db.prepare(`SELECT COALESCE(source_service, requester_id) AS svc, COUNT(*) AS n, SUM(tokens_in + tokens_out) AS tokens, MAX(created_at) AS last
+ FROM runs WHERE (on_behalf_of::jsonb ->> 'type') = 'user' AND (on_behalf_of::jsonb ->> 'id') = ? AND created_at >= ?
GROUP BY svc`).all(subject, since);
GROUP BY svc`).all(subject, since);
- const lastEver = db.prepare("SELECT MAX(created_at) AS last FROM runs WHERE json_extract(on_behalf_of, '$.type') = 'user' AND json_extract(on_behalf_of, '$.id') = ?").get(subject).last;
+ const lastEver = (await db.prepare("SELECT MAX(created_at) AS last FROM runs WHERE (on_behalf_of::jsonb ->> 'type') = 'user' AND (on_behalf_of::jsonb ->> 'id') = ?").get(subject)).last;
if (!lastEver) return null;
if (!lastEver) return null;
const by = {};
const by = {};
for (const r of rows.slice(0, 30)) if (/^[a-z][a-z0-9-]{1,39}$/.test(String(r.svc || ''))) by[r.svc] = r.n;
for (const r of rows.slice(0, 30)) if (/^[a-z][a-z0-9-]{1,39}$/.test(String(r.svc || ''))) by[r.svc] = r.n;
@@ -90,29 +90,29 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalTh
@@ -90,29 +90,29 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalTh
async function push(subject, now = clock.now()) {
async function push(subject, now = clock.now()) {
if (!enabled || !USR.test(String(subject || ''))) return false;
if (!enabled || !USR.test(String(subject || ''))) return false;
- const data = summarize(subject, now);
+ const data = await summarize(subject, now);
if (!data) return false;
if (!data) return false;
const hash = crypto.createHash('sha256').update(JSON.stringify(data)).digest('hex').slice(0, 32);
const hash = crypto.createHash('sha256').update(JSON.stringify(data)).digest('hex').slice(0, 32);
- const last = db.prepare('SELECT hash FROM ai_module_pushes WHERE subject_id = ?').get(subject);
+ const last = await db.prepare('SELECT hash FROM ai_module_pushes WHERE subject_id = ?').get(subject);
if (last && last.hash === hash) { stats.unchanged++; return false; }
if (last && last.hash === hash) { stats.unchanged++; return false; }
try {
try {
const r = await call('PUT', 'ai.usage_summary', subject, { ...data, computed_at: new Date(now).toISOString() });
const r = await call('PUT', 'ai.usage_summary', subject, { ...data, computed_at: new Date(now).toISOString() });
if (r.status >= 300) throw new Error(`Network answered ${r.status} ${(r.body && r.body.code) || ''}`);
if (r.status >= 300) throw new Error(`Network answered ${r.status} ${(r.body && r.body.code) || ''}`);
} catch (err) { stats.failed++; stats.lastError = err.message; return false; }
} catch (err) { stats.failed++; stats.lastError = err.message; return false; }
- db.prepare(`INSERT INTO ai_module_pushes (subject_id, hash, pushed_at) VALUES (?, ?, ?)
+ await db.prepare(`INSERT INTO ai_module_pushes (subject_id, hash, pushed_at) VALUES (?, ?, ?)
ON CONFLICT(subject_id) DO UPDATE SET hash = excluded.hash, pushed_at = excluded.pushed_at`).run(subject, hash, new Date(now).toISOString());
ON CONFLICT(subject_id) DO UPDATE SET hash = excluded.hash, pushed_at = excluded.pushed_at`).run(subject, hash, new Date(now).toISOString());
stats.written++;
stats.written++;
return true;
return true;
}
}
- const people = (sql, ...args) => db.prepare(sql).all(...args).map((r) => r.s).filter((s) => USR.test(String(s || '')));
+ const people = async (sql, ...args) => (await db.prepare(sql).all(...args)).map((r) => r.s).filter((s) => USR.test(String(s || '')));
/** People with a run created since the previous scan. */
/** People with a run created since the previous scan. */
async function scan(now = clock.now()) {
async function scan(now = clock.now()) {
if (!enabled) return 0;
if (!enabled) return 0;
const from = lastScan || new Date(now - 10 * 60 * 1000).toISOString();
const from = lastScan || new Date(now - 10 * 60 * 1000).toISOString();
const to = new Date(now).toISOString();
const to = new Date(now).toISOString();
- const subjects = people("SELECT DISTINCT json_extract(on_behalf_of, '$.id') AS s FROM runs WHERE on_behalf_of IS NOT NULL AND created_at >= ? AND created_at < ?", from, to);
+ const subjects = await people("SELECT DISTINCT (on_behalf_of::jsonb ->> 'id') AS s FROM runs WHERE on_behalf_of IS NOT NULL AND created_at >= ? AND created_at < ?", from, to);
for (const s of subjects) await push(s, now);
for (const s of subjects) await push(s, now);
lastScan = to;
lastScan = to;
return subjects.length;
return subjects.length;
@@ -122,7 +122,7 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalTh
@@ -122,7 +122,7 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalTh
async function refresh(now = clock.now()) {
async function refresh(now = clock.now()) {
if (!enabled) return 0;
if (!enabled) return 0;
const since = new Date(now - 31 * DAY_MS).toISOString();
const since = new Date(now - 31 * DAY_MS).toISOString();
- const subjects = people(`SELECT DISTINCT json_extract(on_behalf_of, '$.id') AS s FROM runs WHERE on_behalf_of IS NOT NULL AND created_at >= ?
+ const subjects = await people(`SELECT DISTINCT (on_behalf_of::jsonb ->> 'id') AS s FROM runs WHERE on_behalf_of IS NOT NULL AND created_at >= ?
UNION SELECT subject_id AS s FROM ai_module_pushes`, since);
UNION SELECT subject_id AS s FROM ai_module_pushes`, since);
for (const s of subjects) await push(s, now);
for (const s of subjects) await push(s, now);
return subjects.length;
return subjects.length;
@@ -132,7 +132,7 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalTh
@@ -132,7 +132,7 @@ function createUserModules({ db, config, env = process.env, fetchImpl = globalTh
if (!enabled || timers.length) return false;
if (!enabled || timers.length) return false;
ensureSchema();
ensureSchema();
const safe = (fn) => () => { fn().catch((err) => { stats.lastError = err.message; }); };
const safe = (fn) => () => { fn().catch((err) => { stats.lastError = err.message; }); };
- timers = [setInterval(safe(() => scan()), 5 * 60 * 1000), setInterval(safe(() => refresh()), DAY_MS), setTimeout(safe(() => refresh()), 3 * 60 * 1000)];
+ timers = [setInterval(safe(async () => await scan()), 5 * 60 * 1000), setInterval(safe(async () => await refresh()), DAY_MS), setTimeout(safe(async () => await refresh()), 3 * 60 * 1000)];
for (const t of timers) t.unref?.();
for (const t of timers) t.unref?.();
return true;
return true;
}
}
@@ -101,8 +101,8 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
@@ -101,8 +101,8 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
let proxy = null;
let proxy = null;
const reader = () => proxy || (proxy = require('../media-proxy').createMediaProxy({ fetcher, log }));
const reader = () => proxy || (proxy = require('../media-proxy').createMediaProxy({ fetcher, log }));
const mediaAnalysis = require('./media-analysis').createMediaAnalysis({ registry, pool, fetcher, quotas, config, spawnImpl, log, reader });
const mediaAnalysis = require('./media-analysis').createMediaAnalysis({ registry, pool, fetcher, quotas, config, spawnImpl, log, reader });
- function routeFor(key) {
- const r = registry.resolveRoute(key);
+ async function routeFor(key) {
+ const r = await registry.resolveRoute(key);
if (!r) throw new AiError(503, 'route.unavailable', `no route ${key}`);
if (!r) throw new AiError(503, 'route.unavailable', `no route ${key}`);
if (r.disabled) throw new AiError(503, 'route.unavailable', `route ${key} is disabled`);
if (r.disabled) throw new AiError(503, 'route.unavailable', `route ${key} is disabled`);
return r;
return r;
@@ -111,8 +111,8 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
@@ -111,8 +111,8 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
async function llmStep(step, input, wf, run, ctx) {
async function llmStep(step, input, wf, run, ctx) {
// The run pinned its template version when it was created; later steps use the active one.
// The run pinned its template version when it was created; later steps use the active one.
const template = run.template_key === step.template && run.template_version
const template = run.template_key === step.template && run.template_version
- ? registry.getTemplate(step.template, run.template_version)
- : registry.activeTemplate(step.template);
+ ? await registry.getTemplate(step.template, run.template_version)
+ : await registry.activeTemplate(step.template);
if (!template) throw new AiError(500, 'workflow.broken', `template ${step.template} is missing`);
if (!template) throw new AiError(500, 'workflow.broken', `template ${step.template} is missing`);
const prep = step.prepare ? PREPARE[step.prepare](input) : { vars: input };
const prep = step.prepare ? PREPARE[step.prepare](input) : { vars: input };
if (!prep) throw new AiError(500, 'workflow.broken', `prepare hook ${step.prepare} is missing`);
if (!prep) throw new AiError(500, 'workflow.broken', `prepare hook ${step.prepare} is missing`);
@@ -126,7 +126,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
@@ -126,7 +126,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
const system = [render(template.system_prompt, vars).trim(), asked].filter(Boolean).join('\n\n');
const system = [render(template.system_prompt, vars).trim(), asked].filter(Boolean).join('\n\n');
const user = render(template.user_prompt, vars);
const user = render(template.user_prompt, vars);
const routeKey = step.route || template.default_route || wf.default_route || 'default.chat';
const routeKey = step.route || template.default_route || wf.default_route || 'default.chat';
- const route = routeFor(routeKey);
+ const route = await routeFor(routeKey);
const sources = Array.isArray(input.sources) ? input.sources : [];
const sources = Array.isArray(input.sources) ? input.sources : [];
let jsonSchema = null;
let jsonSchema = null;
if (step.output === 'json') {
if (step.output === 'json') {
@@ -168,7 +168,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
@@ -168,7 +168,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
async function passthroughStep(step, input, wf, run, ctx) {
async function passthroughStep(step, input, wf, run, ctx) {
const routeKey = step.route || `${step.route_prefix || 'live'}.${input.role || 'legacy'}`;
const routeKey = step.route || `${step.route_prefix || 'live'}.${input.role || 'legacy'}`;
- const route = routeFor(routeKey);
+ const route = await routeFor(routeKey);
const sys = typeof input.system === 'string' ? (input.system.trim() ? [{ text: input.system, cache: false }] : [])
const sys = typeof input.system === 'string' ? (input.system.trim() ? [{ text: input.system, cache: false }] : [])
: Array.isArray(input.system) ? input.system.filter(x => x && String(x.text || '').trim()).map(x => ({ text: String(x.text), cache: Boolean(x.cache) })) : [];
: Array.isArray(input.system) ? input.system.filter(x => x && String(x.text || '').trim()).map(x => ({ text: String(x.text), cache: Boolean(x.cache) })) : [];
const messages = normMessages(input.messages, input.user);
const messages = normMessages(input.messages, input.user);
@@ -188,7 +188,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
@@ -188,7 +188,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
}
}
async function transcribeStep(step, input, wf, run, ctx) {
async function transcribeStep(step, input, wf, run, ctx) {
- const route = routeFor(step.route || 'live.stt');
+ const route = await routeFor(step.route || 'live.stt');
// Read where it lies (the loopback reader), one window at a time: start_sec seeks with a range request, and
// Read where it lies (the loopback reader), one window at a time: start_sec seeks with a range request, and
// timestamps are offset by start_sec unless offset_sec says otherwise. AI_MEDIA_STREAM=0 downloads first.
// timestamps are offset by start_sec unless offset_sec says otherwise. AI_MEDIA_STREAM=0 downloads first.
const streamed = !config.mediaAnalysis || config.mediaAnalysis.stream !== false;
const streamed = !config.mediaAnalysis || config.mediaAnalysis.stream !== false;
@@ -207,7 +207,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
@@ -207,7 +207,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
}
}
async function embedStep(step, input, wf, run, ctx) {
async function embedStep(step, input, wf, run, ctx) {
- const route = routeFor(step.route || 'default.embedding');
+ const route = await routeFor(step.route || 'default.embedding');
const inputs = Array.isArray(input.input) ? input.input : [input.input];
const inputs = Array.isArray(input.input) ? input.input : [input.input];
const exec = await pool.execute(route, 'embed', { input: inputs, timeoutMs: route.timeout_ms || 30000 }, { ...ctx, routeKey: route.key, routeVersion: route.version, promptHash: null });
const exec = await pool.execute(route, 'embed', { input: inputs, timeoutMs: route.timeout_ms || 30000 }, { ...ctx, routeKey: route.key, routeVersion: route.version, promptHash: null });
const vectors = exec.result.vectors || [];
const vectors = exec.result.vectors || [];
@@ -258,8 +258,8 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
@@ -258,8 +258,8 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
}
}
/** A quota with workflow_prefix `media.paid` and a cost cap: today's media.* spend must be under it. */
/** A quota with workflow_prefix `media.paid` and a cost cap: today's media.* spend must be under it. */
- function paidBudget() {
- return quotas && quotas.paidBudget ? quotas.paidBudget('media.paid', 'media.') : null;
+ async function paidBudget() {
+ return quotas && quotas.paidBudget ? await quotas.paidBudget('media.paid', 'media.') : null;
}
}
/** Refuse before downloading when the temp disk could not hold the file (a full disk stops every service). */
/** Refuse before downloading when the temp disk could not hold the file (a full disk stops every service). */
@@ -273,14 +273,14 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
@@ -273,14 +273,14 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
}
}
async function overview({ step, input, wf, run: runRow, ctx, facts, highlights, transcript, gaps }) {
async function overview({ step, input, wf, run: runRow, ctx, facts, highlights, transcript, gaps }) {
- const template = registry.activeTemplate(step.template || 'media.analyze.overview');
+ const template = await registry.activeTemplate(step.template || 'media.analyze.overview');
const vars = {
const vars = {
facts: facts.join('\n'),
facts: facts.join('\n'),
highlights: highlights.map((h) => `${clock(h.start)}–${clock(h.end)} [${h.reasons.join(', ')}] ${h.excerpt}`).join('\n') || '(none)',
highlights: highlights.map((h) => `${clock(h.start)}–${clock(h.end)} [${h.reasons.join(', ')}] ${h.excerpt}`).join('\n') || '(none)',
transcript: (transcript.text || '').slice(0, 8000) || '(no transcript)',
transcript: (transcript.text || '').slice(0, 8000) || '(no transcript)',
};
};
const call = async (routeKey) => {
const call = async (routeKey) => {
- const route = registry.resolveRoute(routeKey);
+ const route = await registry.resolveRoute(routeKey);
if (!route || route.disabled) return { missing: true };
if (!route || route.disabled) return { missing: true };
const system = render(template.system_prompt, vars).trim();
const system = render(template.system_prompt, vars).trim();
const req = { system: [{ text: system, cache: true }], messages: [{ role: 'user', content: render(template.user_prompt, vars) }], image: null, json: null, maxTokens: 250, temperature: 0.3, timeoutMs: route.timeout_ms || 120000, cacheKey: `${wf.key}:${wf.version}` };
const req = { system: [{ text: system, cache: true }], messages: [{ role: 'user', content: render(template.user_prompt, vars) }], image: null, json: null, maxTokens: 250, temperature: 0.3, timeoutMs: route.timeout_ms || 120000, cacheKey: `${wf.key}:${wf.version}` };
@@ -298,7 +298,7 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
@@ -298,7 +298,7 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
gaps.push(`The local model did not answer (${e.code || e.message}).`);
gaps.push(`The local model did not answer (${e.code || e.message}).`);
}
}
if (input.allow_paid) {
if (input.allow_paid) {
- const budget = paidBudget();
+ const budget = await paidBudget();
if (!budget) gaps.push('Paid providers are off for media analysis: no quota gives media.paid a cost budget.');
if (!budget) gaps.push('Paid providers are off for media analysis: no quota gives media.paid a cost budget.');
else if (budget.spent >= budget.max_cost_usd) gaps.push(`Today's paid budget for media analysis ($${budget.max_cost_usd}) is spent.`);
else if (budget.spent >= budget.max_cost_usd) gaps.push(`Today's paid budget for media analysis ($${budget.max_cost_usd}) is spent.`);
else {
else {
@@ -314,7 +314,7 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
@@ -314,7 +314,7 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
}
}
async function step(stepDef, input, wf, runRow, ctx) {
async function step(stepDef, input, wf, runRow, ctx) {
- const mine = turn.then(() => analyse(stepDef, input, wf, runRow, ctx));
+ const mine = turn.then(async () => await analyse(stepDef, input, wf, runRow, ctx));
turn = mine.catch(() => {});
turn = mine.catch(() => {});
return mine;
return mine;
}
}
@@ -346,7 +346,7 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
@@ -346,7 +346,7 @@ function createMediaAnalysis({ registry, pool, fetcher, quotas, config = {}, spa
let transcript = { available: false, language: input.language || 'en', text: '', segments: [] };
let transcript = { available: false, language: input.language || 'en', text: '', segments: [] };
let sttExec = null;
let sttExec = null;
if (streams.audio) {
if (streams.audio) {
- const route = registry.resolveRoute(stepDef.stt_route || 'live.stt');
+ const route = await registry.resolveRoute(stepDef.stt_route || 'live.stt');
if (!route || route.disabled) gaps.push('No speech-to-text route (live.stt): no transcript.');
if (!route || route.disabled) gaps.push('No speech-to-text route (live.stt): no transcript.');
else {
else {
// In windows (sttWindowSec, 10 min): each call stays inside the provider's timeout, and one
// In windows (sttWindowSec, 10 min): each call stays inside the provider's timeout, and one
@@ -64,59 +64,59 @@ function providerRecords(config, env) {
@@ -64,59 +64,59 @@ function providerRecords(config, env) {
return out;
return out;
}
}
-function seed({ registry, quotas, config, env = process.env, db }) {
+async function seed({ registry, quotas, config, env = process.env, db }) {
// Providers
// Providers
for (const p of providerRecords(config, env)) {
for (const p of providerRecords(config, env)) {
- const prev = registry.getProvider(p.key);
+ const prev = await registry.getProvider(p.key);
if (!prev || prev.origin === 'seed') {
if (!prev || prev.origin === 'seed') {
const same = prev && Object.keys(p).every(k => JSON.stringify(p[k] ?? null) === JSON.stringify(prev[k] ?? null));
const same = prev && Object.keys(p).every(k => JSON.stringify(p[k] ?? null) === JSON.stringify(prev[k] ?? null));
- if (!same) registry.upsertProvider({ base_url: null, secret_ref: null, ...p }, { actor: 'seed', origin: 'seed' });
+ if (!same) await registry.upsertProvider({ base_url: null, secret_ref: null, ...p }, { actor: 'seed', origin: 'seed' });
}
}
}
}
- const shared = registry.getProvider('shared');
- const hasFallback = Boolean(registry.getProvider('fallback'));
+ const shared = await registry.getProvider('shared');
+ const hasFallback = Boolean(await registry.getProvider('fallback'));
const fallbacks = hasFallback ? [{ provider: 'fallback', model: null }] : [];
const fallbacks = hasFallback ? [{ provider: 'fallback', model: null }] : [];
// Models
// Models
const models = new Set([shared.default_model, ...Object.values(config.shared.roleModels)]);
const models = new Set([shared.default_model, ...Object.values(config.shared.roleModels)]);
- for (const m of models) if (m && !registry.getModel('shared', m)) registry.upsertModel({ provider_key: 'shared', model_key: m, type: 'chat', supports: { json: true, vision: true } }, { actor: 'seed' });
- if (!registry.getModel('stub', 'stub-1')) registry.upsertModel({ provider_key: 'stub', model_key: 'stub-1', type: 'chat', supports: { json: true, vision: true }, metadata: { synthetic: true } }, { actor: 'seed' });
- if (!registry.getModel('whisper', 'whisper.cpp')) registry.upsertModel({ provider_key: 'whisper', model_key: 'whisper.cpp', type: 'stt', supports: { json: false } }, { actor: 'seed' });
+ for (const m of models) if (m && !await registry.getModel('shared', m)) await registry.upsertModel({ provider_key: 'shared', model_key: m, type: 'chat', supports: { json: true, vision: true } }, { actor: 'seed' });
+ if (!await registry.getModel('stub', 'stub-1')) await registry.upsertModel({ provider_key: 'stub', model_key: 'stub-1', type: 'chat', supports: { json: true, vision: true }, metadata: { synthetic: true } }, { actor: 'seed' });
+ if (!await registry.getModel('whisper', 'whisper.cpp')) await registry.upsertModel({ provider_key: 'whisper', model_key: 'whisper.cpp', type: 'stt', supports: { json: false } }, { actor: 'seed' });
// Routes
// Routes
for (const role of live.ROLES) {
for (const role of live.ROLES) {
- registry.seedVersioned('route', `live.${role}`, {
+ await registry.seedVersioned('route', `live.${role}`, {
primary: { provider: 'shared', model: config.shared.roleModels[role] || null }, fallbacks,
primary: { provider: 'shared', model: config.shared.roleModels[role] || null }, fallbacks,
options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: ROLE_TIMEOUT[role], alias_of: null,
options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: ROLE_TIMEOUT[role], alias_of: null,
});
});
}
}
- registry.seedVersioned('route', 'live.stt', { primary: { provider: 'whisper', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: 600000, alias_of: null });
- registry.seedVersioned('route', 'default.chat', { primary: { provider: 'shared', model: null }, fallbacks, options: {}, max_output_tokens: 800, response_format: 'text', timeout_ms: 30000, alias_of: null });
- registry.seedVersioned('route', 'default.json', { primary: { provider: 'shared', model: null }, fallbacks, options: { temperature: 0.3 }, max_output_tokens: 2400, response_format: 'json', timeout_ms: 60000, alias_of: null });
- registry.seedVersioned('route', 'default.embedding', { primary: { provider: 'shared', model: env.AI_EMBEDDING_MODEL || 'text-embedding-3-small' }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: 30000, alias_of: null });
+ await registry.seedVersioned('route', 'live.stt', { primary: { provider: 'whisper', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: 600000, alias_of: null });
+ await registry.seedVersioned('route', 'default.chat', { primary: { provider: 'shared', model: null }, fallbacks, options: {}, max_output_tokens: 800, response_format: 'text', timeout_ms: 30000, alias_of: null });
+ await registry.seedVersioned('route', 'default.json', { primary: { provider: 'shared', model: null }, fallbacks, options: { temperature: 0.3 }, max_output_tokens: 2400, response_format: 'json', timeout_ms: 60000, alias_of: null });
+ await registry.seedVersioned('route', 'default.embedding', { primary: { provider: 'shared', model: env.AI_EMBEDDING_MODEL || 'text-embedding-3-small' }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'text', timeout_ms: 30000, alias_of: null });
// media.analyze (WS-O task 5): the local model when there is one; the paid route is used only under a media.paid budget.
// media.analyze (WS-O task 5): the local model when there is one; the paid route is used only under a media.paid budget.
- if (config.localLlm && config.localLlm.url) registry.seedVersioned('route', 'media.local', { primary: { provider: 'local', model: config.localLlm.model || null }, fallbacks: [], options: { temperature: 0.3 }, max_output_tokens: 350, response_format: 'text', timeout_ms: config.localLlm.timeoutMs, alias_of: null });
- registry.seedVersioned('route', 'media.paid', { primary: { provider: 'shared', model: config.shared.roleModels.summary || null }, fallbacks, options: { temperature: 0.3 }, max_output_tokens: 350, response_format: 'text', timeout_ms: 60000, alias_of: null });
- for (const key of HISTORICAL_ROUTES) registry.seedVersioned('route', key, { primary: { provider: 'shared', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'json', timeout_ms: null, alias_of: 'default.json' });
+ if (config.localLlm && config.localLlm.url) await registry.seedVersioned('route', 'media.local', { primary: { provider: 'local', model: config.localLlm.model || null }, fallbacks: [], options: { temperature: 0.3 }, max_output_tokens: 350, response_format: 'text', timeout_ms: config.localLlm.timeoutMs, alias_of: null });
+ await registry.seedVersioned('route', 'media.paid', { primary: { provider: 'shared', model: config.shared.roleModels.summary || null }, fallbacks, options: { temperature: 0.3 }, max_output_tokens: 350, response_format: 'text', timeout_ms: 60000, alias_of: null });
+ for (const key of HISTORICAL_ROUTES) await registry.seedVersioned('route', key, { primary: { provider: 'shared', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'json', timeout_ms: null, alias_of: 'default.json' });
// Templates, then workflows (workflows reference templates)
// Templates, then workflows (workflows reference templates)
for (const t of [...live.templates, ...core.templates, ...products.templates, ...media.templates]) {
for (const t of [...live.templates, ...core.templates, ...products.templates, ...media.templates]) {
const { key, ...def } = t;
const { key, ...def } = t;
- registry.seedVersioned('template', key, { description: null, default_route: null, owner: 'ai', visibility: 'internal', metadata: {}, ...def });
+ await registry.seedVersioned('template', key, { description: null, default_route: null, owner: 'ai', visibility: 'internal', metadata: {}, ...def });
}
}
for (const w of [...live.workflows, ...core.workflows, ...products.workflows, ...media.workflows]) {
for (const w of [...live.workflows, ...core.workflows, ...products.workflows, ...media.workflows]) {
const { key, ...def } = w;
const { key, ...def } = w;
- registry.seedVersioned('workflow', key, { description: null, default_route: null, cache_ttl_sec: null, metadata: {}, ...def });
+ await registry.seedVersioned('workflow', key, { description: null, default_route: null, cache_ttl_sec: null, metadata: {}, ...def });
}
}
// Quotas
// Quotas
- const seedQuota = (q) => {
- const prev = db.prepare("SELECT origin FROM quotas WHERE scope_type = ? AND scope_id = ? AND window = ? AND COALESCE(workflow_prefix, '') = ''").get(q.scope_type, q.scope_type === 'global' ? '*' : q.scope_id, q.window);
- if (!prev || prev.origin === 'seed') quotas.upsert(q, { actor: 'seed', origin: 'seed' });
+ const seedQuota = async (q) => {
+ const prev = await db.prepare("SELECT origin FROM quotas WHERE scope_type = ? AND scope_id = ? AND \"window\" = ? AND COALESCE(workflow_prefix, '') = ''").get(q.scope_type, q.scope_type === 'global' ? '*' : q.scope_id, q.window);
+ if (!prev || prev.origin === 'seed') await quotas.upsert(q, { actor: 'seed', origin: 'seed' });
};
};
- if (config.quotas.globalCostPerDay > 0) seedQuota({ scope_type: 'global', window: 'day', max_cost_usd: config.quotas.globalCostPerDay });
- if (config.quotas.serviceRequestsPerMinute > 0) seedQuota({ scope_type: 'service', scope_id: '*', window: 'minute', max_requests: config.quotas.serviceRequestsPerMinute });
- if (config.quotas.serviceRequestsPerDay > 0) seedQuota({ scope_type: 'service', scope_id: '*', window: 'day', max_requests: config.quotas.serviceRequestsPerDay });
+ if (config.quotas.globalCostPerDay > 0) await seedQuota({ scope_type: 'global', window: 'day', max_cost_usd: config.quotas.globalCostPerDay });
+ if (config.quotas.serviceRequestsPerMinute > 0) await seedQuota({ scope_type: 'service', scope_id: '*', window: 'minute', max_requests: config.quotas.serviceRequestsPerMinute });
+ if (config.quotas.serviceRequestsPerDay > 0) await seedQuota({ scope_type: 'service', scope_id: '*', window: 'day', max_requests: config.quotas.serviceRequestsPerDay });
}
}
module.exports = { seed, providerRecords, HISTORICAL_ROUTES };
module.exports = { seed, providerRecords, HISTORICAL_ROUTES };
@@ -12,7 +12,7 @@ const live = token('live', [...ALL, 'ai.quota.attribution.manage']);
@@ -12,7 +12,7 @@ const live = token('live', [...ALL, 'ai.quota.attribution.manage']);
const tools = token('tools', [...ALL, 'ai.quota.attribution.manage']);
const tools = token('tools', [...ALL, 'ai.quota.attribution.manage']);
const plain = token('live', ALL);
const plain = token('live', ALL);
let h;
let h;
-const run = (workflow, userId) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow, input: { role: 'chat', user: 'hi' }, attribution: { service: 'live', type: 'user', id: String(userId) } } });
+const run = async (workflow, userId) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow, input: { role: 'chat', user: 'hi' }, attribution: { service: 'live', type: 'user', id: String(userId) } } });
t.test('boot', async () => { h = await boot(); });
t.test('boot', async () => { h = await boot(); });
@@ -64,7 +64,7 @@ t.test('a namespace-limited token may only run workflows in its namespaces', asy
@@ -64,7 +64,7 @@ t.test('a namespace-limited token may only run workflows in its namespaces', asy
});
});
t.test('namespaces fail closed: a token without ns runs nothing (C-22/C-23 retired)', async () => {
t.test('namespaces fail closed: a token without ns runs nothing (C-22/C-23 retired)', async () => {
- const run = (tok, workflow, input) => request(h.base, 'POST', '/api/v1/runs', { tok, body: { workflow, input } });
+ const run = async (tok, workflow, input) => await request(h.base, 'POST', '/api/v1/runs', { tok, body: { workflow, input } });
const denied = (r, what) => { assert.strictEqual(r.status, 403, what); assert.strictEqual(r.body.code, 'capability.namespace_denied', what); };
const denied = (r, what) => { assert.strictEqual(r.status, 403, what); assert.strictEqual(r.body.code, 'capability.namespace_denied', what); };
// No ns claim, or an empty one: nothing, not even the service's own namespace.
// No ns claim, or an empty one: nothing, not even the service's own namespace.
denied(await run(token('games', ['ai.run.create'], { ns: null }), 'games.generate_lore', {}), 'svc:games without ns -> games.*');
denied(await run(token('games', ['ai.run.create'], { ns: null }), 'games.generate_lore', {}), 'svc:games without ns -> games.*');
@@ -31,7 +31,7 @@ t.test('bad requests from one caller fail that caller only; the circuit stays cl
@@ -31,7 +31,7 @@ t.test('bad requests from one caller fail that caller only; the circuit stays cl
const bad = await request(h.base, 'POST', '/api/v1/generate', { tok: attacker, body: { prompt: `BAD ${i}`, options: { cache: false } } });
const bad = await request(h.base, 'POST', '/api/v1/generate', { tok: attacker, body: { prompt: `BAD ${i}`, options: { cache: false } } });
assert.strictEqual(bad.body.run.status, 'failed', bad.text);
assert.strictEqual(bad.body.run.status, 'failed', bad.text);
}
}
- assert.strictEqual(h.pool.health('seam').state, 'closed', 'client errors must not open the circuit');
+ assert.strictEqual((await h.pool.health('seam')).state, 'closed', 'client errors must not open the circuit');
const good = await request(h.base, 'POST', '/api/v1/generate', { tok: admin, body: { prompt: 'hello', options: { cache: false } } });
const good = await request(h.base, 'POST', '/api/v1/generate', { tok: admin, body: { prompt: 'hello', options: { cache: false } } });
assert.strictEqual(good.body.run.status, 'succeeded', good.text);
assert.strictEqual(good.body.run.status, 'succeeded', good.text);
assert.strictEqual(good.body.run.output.text, 'fine');
assert.strictEqual(good.body.run.output.text, 'fine');
@@ -40,7 +40,7 @@ t.test('bad requests from one caller fail that caller only; the circuit stays cl
@@ -40,7 +40,7 @@ t.test('bad requests from one caller fail that caller only; the circuit stays cl
t.test('server errors still open it', async () => {
t.test('server errors still open it', async () => {
seam.handler = () => ({ status: 503, body: { error: { message: 'overloaded' } } });
seam.handler = () => ({ status: 503, body: { error: { message: 'overloaded' } } });
for (let i = 0; i < 2; i++) await request(h.base, 'POST', '/api/v1/generate', { tok: admin, body: { prompt: `x ${i}`, options: { cache: false } } });
for (let i = 0; i < 2; i++) await request(h.base, 'POST', '/api/v1/generate', { tok: admin, body: { prompt: `x ${i}`, options: { cache: false } } });
- assert.strictEqual(h.pool.health('seam').state, 'open');
+ assert.strictEqual((await h.pool.health('seam')).state, 'open');
});
});
t.test('shutdown', async () => { await h.stop(); await seam.close(); });
t.test('shutdown', async () => { await h.stop(); await seam.close(); });
@@ -15,7 +15,7 @@ const B = { type: 'user', id: 'usr_01JAB2C3D4E5F6G7H8J9K0MNPB' };
@@ -15,7 +15,7 @@ const B = { type: 'user', id: 'usr_01JAB2C3D4E5F6G7H8J9K0MNPB' };
let h;
let h;
let seam;
let seam;
-const summarize = (tok, extra = {}) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'ai.summarize', input: { text: 'The same private text for everyone.' }, ...extra } });
+const summarize = async (tok, extra = {}) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'ai.summarize', input: { text: 'The same private text for everyone.' }, ...extra } });
t.test('boot with a real (non-synthetic) seam provider', async () => {
t.test('boot with a real (non-synthetic) seam provider', async () => {
seam = await seamServer((b) => ({ text: `summary #${seam.calls}`, usage: { input: 5, output: 2 } }));
seam = await seamServer((b) => ({ text: `summary #${seam.calls}`, usage: { input: 5, output: 2 } }));
@@ -62,7 +62,7 @@ t.test('another service never reads this service\'s entries', async () => {
@@ -62,7 +62,7 @@ t.test('another service never reads this service\'s entries', async () => {
});
});
t.test('service-scoped workflows (translation) share within the service only', async () => {
t.test('service-scoped workflows (translation) share within the service only', async () => {
- const tr = (tok, obo) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'live.translate', input: { text: 'こんにちは、みなさん', from: 'ja', to: 'en' }, on_behalf_of: obo } });
+ const tr = async (tok, obo) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'live.translate', input: { text: 'こんにちは、みなさん', from: 'ja', to: 'en' }, on_behalf_of: obo } });
assert.strictEqual((await tr(live, A)).body.run.status, 'succeeded');
assert.strictEqual((await tr(live, A)).body.run.status, 'succeeded');
assert.strictEqual((await tr(live, B)).body.run.status, 'cached', 'public text translation is shared inside Live');
assert.strictEqual((await tr(live, B)).body.run.status, 'cached', 'public text translation is shared inside Live');
assert.strictEqual((await tr(other, A)).body.run.status, 'succeeded', 'but not with another service');
assert.strictEqual((await tr(other, A)).body.run.status, 'succeeded', 'but not with another service');
@@ -95,8 +95,8 @@ async function signIn(subject, { role = 'admin', is_owner = false, username = 's
@@ -95,8 +95,8 @@ async function signIn(subject, { role = 'admin', is_owner = false, username = 's
return { cb, setCookie: sess || null, cookie: sess ? sess.split(';')[0] : null };
return { cb, setCookie: sess || null, cookie: sess ? sess.split(';')[0] : null };
}
}
const csrfOf = (page) => (page.text.match(/name="_csrf" value="([^"]+)"/) || [])[1];
const csrfOf = (page) => (page.text.match(/name="_csrf" value="([^"]+)"/) || [])[1];
-const auditRows = (where, ...args) => h.db.prepare(`SELECT * FROM audit_log WHERE ${where} ORDER BY id`).all(...args);
-const sessionCount = () => h.db.prepare('SELECT COUNT(*) AS n FROM console_sessions').get().n;
+const auditRows = async (where, ...args) => await h.db.prepare(`SELECT * FROM audit_log WHERE ${where} ORDER BY id`).all(...args);
+const sessionCount = async () => (await h.db.prepare('SELECT COUNT(*) AS n FROM console_sessions').get()).n;
/** Controls that are neither hidden nor inside a <label>. */
/** Controls that are neither hidden nor inside a <label>. */
function unlabelled(text) {
function unlabelled(text) {
@@ -155,7 +155,7 @@ t.test('boot with a provider secret, a seam provider, a failing stub and a few r
@@ -155,7 +155,7 @@ t.test('boot with a provider secret, a seam provider, a failing stub and a few r
const ok = await gen(live, `${'long prompt text '.repeat(40)}END-OF-PROMPT`);
const ok = await gen(live, `${'long prompt text '.repeat(40)}END-OF-PROMPT`);
assert.strictEqual(ok.status, 'succeeded', JSON.stringify(ok.error));
assert.strictEqual(ok.status, 'succeeded', JSON.stringify(ok.error));
runIds.ok = ok.id;
runIds.ok = ok.id;
- assert.ok(h.cache.stats().length >= 1, 'a real answer was cached');
+ assert.ok((await h.cache.stats()).length >= 1, 'a real answer was cached');
});
});
t.test('the staff map: admin reads and manages workflows, only the owner manages providers, a global_mod gets nothing', () => {
t.test('the staff map: admin reads and manages workflows, only the owner manages providers, a global_mod gets nothing', () => {
@@ -176,11 +176,11 @@ t.test('anonymous: console pages answer the sign-in page (401), never data; writ
@@ -176,11 +176,11 @@ t.test('anonymous: console pages answer the sign-in page (401), never data; writ
assert.strictEqual(r.headers.get('x-robots-tag'), 'noindex, nofollow');
assert.strictEqual(r.headers.get('x-robots-tag'), 'noindex, nofollow');
}
}
assert.ok((await get('/console/runs?status=failed')).text.includes(`href="/auth/login?next=${encodeURIComponent('/console/runs?status=failed')}"`));
assert.ok((await get('/console/runs?status=failed')).text.includes(`href="/auth/login?next=${encodeURIComponent('/console/runs?status=failed')}"`));
- const before = h.quotas.list().length;
+ const before = (await h.quotas.list()).length;
assert.strictEqual((await post('/console/quotas', null, { scope_type: 'service', scope_id: 'live', window: 'day', max_requests: '1' })).status, 401);
assert.strictEqual((await post('/console/quotas', null, { scope_type: 'service', scope_id: 'live', window: 'day', max_requests: '1' })).status, 401);
assert.strictEqual((await post('/console/providers/okseam/status', null, { status: 'disabled' })).status, 401);
assert.strictEqual((await post('/console/providers/okseam/status', null, { status: 'disabled' })).status, 401);
- assert.strictEqual(h.quotas.list().length, before);
- assert.strictEqual(h.registry.getProvider('okseam').status, 'active');
+ assert.strictEqual((await h.quotas.list()).length, before);
+ assert.strictEqual((await h.registry.getProvider('okseam')).status, 'active');
});
});
t.test('sign-in redirects to the Network with PKCE S256 as client ai, with a signed host-only flow cookie', async () => {
t.test('sign-in redirects to the Network with PKCE S256 as client ai, with a signed host-only flow cookie', async () => {
@@ -212,7 +212,7 @@ t.test('the callback refuses a missing or mismatched state, a forged flow cookie
@@ -212,7 +212,7 @@ t.test('the callback refuses a missing or mismatched state, a forged flow cookie
assert.strictEqual((await get(`/auth/callback?code=${code}&state=${loc.searchParams.get('state')}`, `ovai_flow=${body}.forged`)).status, 400, 'forged signature');
assert.strictEqual((await get(`/auth/callback?code=${code}&state=${loc.searchParams.get('state')}`, `ovai_flow=${body}.forged`)).status, 400, 'forged signature');
code = authorize({ subject_id: people.admin, role: 'admin', challenge: 'A'.repeat(43), redirect_uri: redirect });
code = authorize({ subject_id: people.admin, role: 'admin', challenge: 'A'.repeat(43), redirect_uri: redirect });
assert.strictEqual((await get(`/auth/callback?code=${code}&state=${encodeURIComponent(loc.searchParams.get('state'))}`, flow)).status, 400, 'PKCE mismatch');
assert.strictEqual((await get(`/auth/callback?code=${code}&state=${encodeURIComponent(loc.searchParams.get('state'))}`, flow)).status, 400, 'PKCE mismatch');
- assert.strictEqual(sessionCount(), 0);
+ assert.strictEqual(await sessionCount(), 0);
});
});
t.test('non-staff get 403 and no session: a user, a global_mod, a token without a subject; refusals are audited', async () => {
t.test('non-staff get 403 and no session: a user, a global_mod, a token without a subject; refusals are audited', async () => {
@@ -222,8 +222,8 @@ t.test('non-staff get 403 and no session: a user, a global_mod, a token without
@@ -222,8 +222,8 @@ t.test('non-staff get 403 and no session: a user, a global_mod, a token without
assert.strictEqual(r.cookie, null);
assert.strictEqual(r.cookie, null);
assert.ok(r.cb.text.includes('only for OpenVibe.Network staff'));
assert.ok(r.cb.text.includes('only for OpenVibe.Network staff'));
}
}
- assert.strictEqual(sessionCount(), 0);
- const refused = auditRows("action = 'console.sign_in.refused'");
+ assert.strictEqual(await sessionCount(), 0);
+ const refused = await auditRows("action = 'console.sign_in.refused'");
assert.deepStrictEqual(refused.map((x) => x.actor).sort(), ['anonymous', people.mod, people.user].sort());
assert.deepStrictEqual(refused.map((x) => x.actor).sort(), ['anonymous', people.mod, people.user].sort());
assert.ok(refused.every((x) => /^[0-9a-f]{32}$/.test(JSON.parse(x.metadata).ip_hash)), 'the client address is kept as a keyed hash');
assert.ok(refused.every((x) => /^[0-9a-f]{32}$/.test(JSON.parse(x.metadata).ip_hash)), 'the client address is kept as a keyed hash');
assert.ok(revoked.length >= 3, 'the Network refresh token is revoked straight away');
assert.ok(revoked.length >= 3, 'the Network refresh token is revoked straight away');
@@ -238,10 +238,10 @@ t.test('an admin signs in: host-only, HttpOnly, Secure, SameSite=Strict, short-l
@@ -238,10 +238,10 @@ t.test('an admin signs in: host-only, HttpOnly, Secure, SameSite=Strict, short-l
const maxAge = Number((r.setCookie.match(/Max-Age=(\d+)/) || [])[1]);
const maxAge = Number((r.setCookie.match(/Max-Age=(\d+)/) || [])[1]);
assert.ok(maxAge > 0 && maxAge <= 3600, `max-age ${maxAge}`);
assert.ok(maxAge > 0 && maxAge <= 3600, `max-age ${maxAge}`);
cookies.admin = r.cookie;
cookies.admin = r.cookie;
- const row = h.db.prepare('SELECT * FROM console_sessions WHERE subject = ?').get(people.admin);
+ const row = await h.db.prepare('SELECT * FROM console_sessions WHERE subject = ?').get(people.admin);
assert.notStrictEqual(row.id_hash, r.cookie.split('=')[1], 'only a hash of the session id is stored');
assert.notStrictEqual(row.id_hash, r.cookie.split('=')[1], 'only a hash of the session id is stored');
assert.strictEqual(JSON.parse(row.staff).role, 'admin');
assert.strictEqual(JSON.parse(row.staff).role, 'admin');
- assert.strictEqual(auditRows("action = 'console.sign_in' AND actor = ?", people.admin).length, 1);
+ assert.strictEqual((await auditRows("action = 'console.sign_in' AND actor = ?", people.admin)).length, 1);
const g = grants.filter((x) => x.grant_type === 'authorization_code').pop();
const g = grants.filter((x) => x.grant_type === 'authorization_code').pop();
assert.strictEqual(g.client_id, 'ai');
assert.strictEqual(g.client_id, 'ai');
assert.strictEqual(g.redirect_uri, 'https://ai.test/auth/callback');
assert.strictEqual(g.redirect_uri, 'https://ai.test/auth/callback');
@@ -281,9 +281,9 @@ t.test('an admin cannot use owner-only actions (ai.provider.manage): 403, audite
@@ -281,9 +281,9 @@ t.test('an admin cannot use owner-only actions (ai.provider.manage): 403, audite
assert.strictEqual(r.status, 403, p);
assert.strictEqual(r.status, 403, p);
assert.ok(r.text.includes('This needs ai.provider.manage'), p);
assert.ok(r.text.includes('This needs ai.provider.manage'), p);
}
}
- assert.strictEqual(h.registry.getProvider('okseam').status, 'active');
- assert.ok(h.cache.stats().length >= 1);
- assert.strictEqual(auditRows("action = 'console.request.refused' AND actor = ?", people.admin).length, 4);
+ assert.strictEqual((await h.registry.getProvider('okseam')).status, 'active');
+ assert.ok((await h.cache.stats()).length >= 1);
+ assert.strictEqual((await auditRows("action = 'console.request.refused' AND actor = ?", people.admin)).length, 4);
});
});
t.test('CSRF: a write needs the session token and a same-origin request', async () => {
t.test('CSRF: a write needs the session token and a same-origin request', async () => {
@@ -291,7 +291,7 @@ t.test('CSRF: a write needs the session token and a same-origin request', async
@@ -291,7 +291,7 @@ t.test('CSRF: a write needs the session token and a same-origin request', async
assert.strictEqual(draft.status, 201, draft.text);
assert.strictEqual(draft.status, 201, draft.text);
const v = draft.body.workflow.version;
const v = draft.body.workflow.version;
const p = `/console/workflows/ai.generate/versions/${v}/status`;
const p = `/console/workflows/ai.generate/versions/${v}/status`;
- const refusedBefore = auditRows("action = 'console.request.refused'").length;
+ const refusedBefore = (await auditRows("action = 'console.request.refused'")).length;
for (const [form, headers, why] of [
for (const [form, headers, why] of [
[{ status: 'archived' }, {}, 'no token'],
[{ status: 'archived' }, {}, 'no token'],
[{ status: 'archived', _csrf: 'x'.repeat(43) }, {}, 'wrong token'],
[{ status: 'archived', _csrf: 'x'.repeat(43) }, {}, 'wrong token'],
@@ -300,14 +300,14 @@ t.test('CSRF: a write needs the session token and a same-origin request', async
@@ -300,14 +300,14 @@ t.test('CSRF: a write needs the session token and a same-origin request', async
]) {
]) {
const r = await post(p, cookies.admin, form, headers);
const r = await post(p, cookies.admin, form, headers);
assert.strictEqual(r.status, 403, why);
assert.strictEqual(r.status, 403, why);
- assert.strictEqual(h.registry.getWorkflow('ai.generate', v).status, 'draft', why);
+ assert.strictEqual((await h.registry.getWorkflow('ai.generate', v)).status, 'draft', why);
}
}
- assert.strictEqual(auditRows("action = 'console.request.refused'").length, refusedBefore + 4);
+ assert.strictEqual((await auditRows("action = 'console.request.refused'")).length, refusedBefore + 4);
const ok = await post(p, cookies.admin, { status: 'archived', _csrf: csrf.admin }, { Origin: 'https://ai.test', 'Sec-Fetch-Site': 'same-origin' });
const ok = await post(p, cookies.admin, { status: 'archived', _csrf: csrf.admin }, { Origin: 'https://ai.test', 'Sec-Fetch-Site': 'same-origin' });
assert.strictEqual(ok.status, 303, ok.text);
assert.strictEqual(ok.status, 303, ok.text);
assert.strictEqual(ok.headers.get('location'), `/console/workflows/ai.generate?version=${v}&done=version_status`);
assert.strictEqual(ok.headers.get('location'), `/console/workflows/ai.generate?version=${v}&done=version_status`);
- assert.strictEqual(h.registry.getWorkflow('ai.generate', v).status, 'archived');
- const row = auditRows("action = 'workflow.status' AND target_id = 'ai.generate'").pop();
+ assert.strictEqual((await h.registry.getWorkflow('ai.generate', v)).status, 'archived');
+ const row = (await auditRows("action = 'workflow.status' AND target_id = 'ai.generate'")).pop();
assert.strictEqual(row.actor, people.admin);
assert.strictEqual(row.actor, people.admin);
assert.deepStrictEqual(JSON.parse(row.metadata), { version: v, from: 'draft', to: 'archived' });
assert.deepStrictEqual(JSON.parse(row.metadata), { version: v, from: 'draft', to: 'archived' });
const bad = await post(p, cookies.admin, { status: 'published', _csrf: csrf.admin });
const bad = await post(p, cookies.admin, { status: 'published', _csrf: csrf.admin });
@@ -324,11 +324,11 @@ t.test('owner: providers, circuits, quotas and cache purge through the console,
@@ -324,11 +324,11 @@ t.test('owner: providers, circuits, quotas and cache purge through the console,
csrf.owner = csrfOf(page);
csrf.owner = csrfOf(page);
let x = await post('/console/providers/badstub/status', cookies.owner, { status: 'disabled', _csrf: csrf.owner });
let x = await post('/console/providers/badstub/status', cookies.owner, { status: 'disabled', _csrf: csrf.owner });
assert.strictEqual(x.status, 303);
assert.strictEqual(x.status, 303);
- assert.strictEqual(h.registry.getProvider('badstub').status, 'disabled');
- assert.strictEqual(auditRows("action = 'provider.update' AND target_id = 'badstub' AND actor = ?", people.owner).length, 1);
+ assert.strictEqual((await h.registry.getProvider('badstub')).status, 'disabled');
+ assert.strictEqual((await auditRows("action = 'provider.update' AND target_id = 'badstub' AND actor = ?", people.owner)).length, 1);
x = await post('/console/providers/badstub/reset', cookies.owner, { _csrf: csrf.owner });
x = await post('/console/providers/badstub/reset', cookies.owner, { _csrf: csrf.owner });
assert.strictEqual(x.status, 303);
assert.strictEqual(x.status, 303);
- assert.strictEqual(auditRows("action = 'provider.circuit_reset' AND target_id = 'badstub' AND actor = ?", people.owner).length, 1);
+ assert.strictEqual((await auditRows("action = 'provider.circuit_reset' AND target_id = 'badstub' AND actor = ?", people.owner)).length, 1);
assert.strictEqual((await post('/console/providers/nope/reset', cookies.owner, { _csrf: csrf.owner })).status, 404);
assert.strictEqual((await post('/console/providers/nope/reset', cookies.owner, { _csrf: csrf.owner })).status, 404);
x = await post('/console/quotas', cookies.owner, { scope_type: 'service', scope_id: 'live', window: 'fortnight', max_requests: '5', _csrf: csrf.owner });
x = await post('/console/quotas', cookies.owner, { scope_type: 'service', scope_id: 'live', window: 'fortnight', max_requests: '5', _csrf: csrf.owner });
@@ -336,20 +336,20 @@ t.test('owner: providers, circuits, quotas and cache purge through the console,
@@ -336,20 +336,20 @@ t.test('owner: providers, circuits, quotas and cache purge through the console,
assert.ok(x.text.includes('window must be minute, hour or day'));
assert.ok(x.text.includes('window must be minute, hour or day'));
x = await post('/console/quotas', cookies.owner, { scope_type: 'service', scope_id: 'live', window: 'day', max_requests: '500', max_tokens: '', max_cost_usd: '2.5', _csrf: csrf.owner });
x = await post('/console/quotas', cookies.owner, { scope_type: 'service', scope_id: 'live', window: 'day', max_requests: '500', max_tokens: '', max_cost_usd: '2.5', _csrf: csrf.owner });
assert.strictEqual(x.status, 303);
assert.strictEqual(x.status, 303);
- const q = h.quotas.list().find((z) => z.scope_type === 'service' && z.scope_id === 'live' && z.window === 'day');
+ const q = (await h.quotas.list()).find((z) => z.scope_type === 'service' && z.scope_id === 'live' && z.window === 'day');
assert.strictEqual(q.max_requests, 500);
assert.strictEqual(q.max_requests, 500);
assert.strictEqual(q.max_tokens, null);
assert.strictEqual(q.max_tokens, null);
assert.strictEqual(q.max_cost_usd, 2.5);
assert.strictEqual(q.max_cost_usd, 2.5);
- assert.strictEqual(auditRows("action = 'quota.create' AND actor = ?", people.owner).length, 1);
+ assert.strictEqual((await auditRows("action = 'quota.create' AND actor = ?", people.owner)).length, 1);
x = await post('/console/cache/purge', cookies.owner, { _csrf: csrf.owner });
x = await post('/console/cache/purge', cookies.owner, { _csrf: csrf.owner });
assert.strictEqual(x.status, 422, 'purging everything needs the confirmation box');
assert.strictEqual(x.status, 422, 'purging everything needs the confirmation box');
- assert.ok(h.cache.stats().length >= 1);
+ assert.ok((await h.cache.stats()).length >= 1);
x = await post('/console/cache/purge', cookies.owner, { workflow: 'ai.generate', _csrf: csrf.owner });
x = await post('/console/cache/purge', cookies.owner, { workflow: 'ai.generate', _csrf: csrf.owner });
assert.strictEqual(x.status, 303);
assert.strictEqual(x.status, 303);
assert.match(x.headers.get('location'), /done=cache_purged&removed=1$/);
assert.match(x.headers.get('location'), /done=cache_purged&removed=1$/);
- assert.strictEqual(h.cache.stats().length, 0);
- const purge = auditRows("action = 'cache.purge' AND actor = ?", people.owner).pop();
+ assert.strictEqual((await h.cache.stats()).length, 0);
+ const purge = (await auditRows("action = 'cache.purge' AND actor = ?", people.owner)).pop();
assert.strictEqual(purge.target_id, 'ai.generate');
assert.strictEqual(purge.target_id, 'ai.generate');
assert.strictEqual(JSON.parse(purge.metadata).removed, 1);
assert.strictEqual(JSON.parse(purge.metadata).removed, 1);
});
});
@@ -360,7 +360,7 @@ t.test('admin API changes write the same audit rows, and the audit page shows wh
@@ -360,7 +360,7 @@ t.test('admin API changes write the same audit rows, and the audit page shows wh
assert.strictEqual((await request(h.base, 'POST', '/api/v1/providers/badstub/reset', { tok: ops })).status, 200);
assert.strictEqual((await request(h.base, 'POST', '/api/v1/providers/badstub/reset', { tok: ops })).status, 200);
assert.strictEqual((await request(h.base, 'DELETE', '/api/v1/cache?workflow=ai.generate', { tok: ops })).status, 200);
assert.strictEqual((await request(h.base, 'DELETE', '/api/v1/cache?workflow=ai.generate', { tok: ops })).status, 200);
for (const action of ['provider.update', 'quota.create', 'provider.circuit_reset', 'cache.purge', 'route.version', 'workflow.version']) {
for (const action of ['provider.update', 'quota.create', 'provider.circuit_reset', 'cache.purge', 'route.version', 'workflow.version']) {
- assert.ok(auditRows('action = ? AND actor = ?', action, 'svc:ops').length >= 1, `${action} by svc:ops`);
+ assert.ok((await auditRows('action = ? AND actor = ?', action, 'svc:ops')).length >= 1, `${action} by svc:ops`);
}
}
const a = await get('/console/audit', cookies.admin);
const a = await get('/console/audit', cookies.admin);
assertPage(a, '/console/audit');
assertPage(a, '/console/audit');
@@ -430,8 +430,8 @@ t.test('cancel: an admin cancels a running run (ai.workflow.manage), audited wit
@@ -430,8 +430,8 @@ t.test('cancel: an admin cancels a running run (ai.workflow.manage), audited wit
assert.ok(page.text.includes(`action="/console/runs/${id}/cancel"`));
assert.ok(page.text.includes(`action="/console/runs/${id}/cancel"`));
const x = await post(`/console/runs/${id}/cancel`, cookies.admin, { _csrf: csrf.admin });
const x = await post(`/console/runs/${id}/cancel`, cookies.admin, { _csrf: csrf.admin });
assert.strictEqual(x.status, 303);
assert.strictEqual(x.status, 303);
- assert.strictEqual(h.runs.get(id).status, 'cancelled');
- assert.strictEqual(auditRows("action = 'run.cancel' AND target_id = ? AND actor = ?", id, people.admin).length, 1);
+ assert.strictEqual((await h.runs.get(id)).status, 'cancelled');
+ assert.strictEqual((await auditRows("action = 'run.cancel' AND target_id = ? AND actor = ?", id, people.admin)).length, 1);
const again = await post(`/console/runs/${id}/cancel`, cookies.admin, { _csrf: csrf.admin });
const again = await post(`/console/runs/${id}/cancel`, cookies.admin, { _csrf: csrf.admin });
assert.strictEqual(again.status, 409, 'a finished run cannot be cancelled');
assert.strictEqual(again.status, 409, 'a finished run cannot be cancelled');
await request(h.base, 'POST', '/api/v1/routes/default.chat/versions', { tok: ops, body: { primary: { provider: 'okseam' }, fallbacks: [] } });
await request(h.base, 'POST', '/api/v1/routes/default.chat/versions', { tok: ops, body: { primary: { provider: 'okseam' }, fallbacks: [] } });
@@ -441,7 +441,7 @@ t.test('the staff mapping is applied on every request: a session whose claims no
@@ -441,7 +441,7 @@ t.test('the staff mapping is applied on every request: a session whose claims no
const r = await signIn(people.mod, { role: 'admin', username: 'demoted' });
const r = await signIn(people.mod, { role: 'admin', username: 'demoted' });
assert.strictEqual(r.cb.status, 303);
assert.strictEqual(r.cb.status, 303);
assert.strictEqual((await get('/console', r.cookie)).status, 200);
assert.strictEqual((await get('/console', r.cookie)).status, 200);
- h.db.prepare("UPDATE console_sessions SET staff = ? WHERE subject = ? AND revoked_at IS NULL").run(JSON.stringify({ role: 'global_mod' }), people.mod);
+ await h.db.prepare("UPDATE console_sessions SET staff = ? WHERE subject = ? AND revoked_at IS NULL").run(JSON.stringify({ role: 'global_mod' }), people.mod);
const x = await get('/console', r.cookie);
const x = await get('/console', r.cookie);
assert.strictEqual(x.status, 403);
assert.strictEqual(x.status, 403);
assert.ok(x.cookies.some((c) => c.startsWith('__Host-ovai_staff=;')), 'the cookie is cleared');
assert.ok(x.cookies.some((c) => c.startsWith('__Host-ovai_staff=;')), 'the cookie is cleared');
@@ -453,8 +453,8 @@ t.test('sign out revokes the session', async () => {
@@ -453,8 +453,8 @@ t.test('sign out revokes the session', async () => {
assert.strictEqual(x.status, 303);
assert.strictEqual(x.status, 303);
assert.ok(x.cookies.some((c) => c.startsWith('__Host-ovai_staff=;')));
assert.ok(x.cookies.some((c) => c.startsWith('__Host-ovai_staff=;')));
assert.strictEqual((await get('/console', cookies.admin)).status, 401);
assert.strictEqual((await get('/console', cookies.admin)).status, 401);
- assert.strictEqual(auditRows("action = 'console.sign_out' AND actor = ?", people.admin).length, 1);
- for (const s of SECRETS) assert.ok(!JSON.stringify(h.db.prepare('SELECT * FROM audit_log').all()).includes(s), 'no secret in the audit log');
+ assert.strictEqual((await auditRows("action = 'console.sign_out' AND actor = ?", people.admin)).length, 1);
+ for (const s of SECRETS) assert.ok(!JSON.stringify(await h.db.prepare('SELECT * FROM audit_log').all()).includes(s), 'no secret in the audit log');
});
});
t.test('robots.txt keeps crawlers out of the console and sign-in; there is no sitemap', async () => {
t.test('robots.txt keeps crawlers out of the console and sign-in; there is no sitemap', async () => {
@@ -40,7 +40,7 @@ t.test('store, read without the key, refuse other services', async () => {
@@ -40,7 +40,7 @@ t.test('store, read without the key, refuse other services', async () => {
assert.ok(!r.text.includes(API_KEY), 'the key never comes back');
assert.ok(!r.text.includes(API_KEY), 'the key never comes back');
const { validate } = require('openvibe-contracts');
const { validate } = require('openvibe-contracts');
assert.ok(validate('ai.credential@1', r.body).valid, JSON.stringify(validate('ai.credential@1', r.body).errors));
assert.ok(validate('ai.credential@1', r.body).valid, JSON.stringify(validate('ai.credential@1', r.body).errors));
- const row = h.db.prepare('SELECT key_enc FROM subject_credentials WHERE owner = ? AND subject = ?').get('live', DANA);
+ const row = await h.db.prepare('SELECT key_enc FROM subject_credentials WHERE owner = ? AND subject = ?').get('live', DANA);
assert.ok(row.key_enc.startsWith('v1:') && !row.key_enc.includes(API_KEY), 'encrypted at rest');
assert.ok(row.key_enc.startsWith('v1:') && !row.key_enc.includes(API_KEY), 'encrypted at rest');
// A models change keeps the stored key; a new endpoint needs it again.
// A models change keeps the stored key; a new endpoint needs it again.
r = await request(h.base, 'PUT', `/api/v1/credentials/${DANA}`, { tok: live, body: { provider: 'openai', base_url: 'https://llm.example.com/v1', models: { chat: 'gpt-4o-mini', vision: 'gpt-4o' }, budget_usd_per_day: 0.003 } });
r = await request(h.base, 'PUT', `/api/v1/credentials/${DANA}`, { tok: live, body: { provider: 'openai', base_url: 'https://llm.example.com/v1', models: { chat: 'gpt-4o-mini', vision: 'gpt-4o' }, budget_usd_per_day: 0.003 } });
@@ -50,14 +50,14 @@ t.test('store, read without the key, refuse other services', async () => {
@@ -50,14 +50,14 @@ t.test('store, read without the key, refuse other services', async () => {
r = await request(h.base, 'GET', `/api/v1/credentials/${DANA}`, { tok: tools });
r = await request(h.base, 'GET', `/api/v1/credentials/${DANA}`, { tok: tools });
assert.strictEqual(r.status, 404, 'another service does not see it');
assert.strictEqual(r.status, 404, 'another service does not see it');
// A row moved to another owner does not decrypt: the key is bound to owner and subject.
// A row moved to another owner does not decrypt: the key is bound to owner and subject.
- h.db.prepare("INSERT INTO subject_credentials (owner, subject, provider, base_url, key_enc, key_hint, models, created_at, updated_at) SELECT 'tools', subject, provider, base_url, key_enc, key_hint, models, created_at, updated_at FROM subject_credentials WHERE owner = 'live'").run();
+ await h.db.prepare("INSERT INTO subject_credentials (owner, subject, provider, base_url, key_enc, key_hint, models, created_at, updated_at) SELECT 'tools', subject, provider, base_url, key_enc, key_hint, models, created_at, updated_at FROM subject_credentials WHERE owner = 'live'").run();
const creds = require('../server/credentials').createCredentials({ db: h.db, config: h.config });
const creds = require('../server/credentials').createCredentials({ db: h.db, config: h.config });
- assert.throws(() => creds.forRun('tools', DANA));
- h.db.prepare("DELETE FROM subject_credentials WHERE owner = 'tools'").run();
+ await assert.rejects(creds.forRun('tools', DANA));
+ await h.db.prepare("DELETE FROM subject_credentials WHERE owner = 'tools'").run();
});
});
t.test('a run with the credential uses that key only, uncached, and its budget stops it', async () => {
t.test('a run with the credential uses that key only, uncached, and its budget stops it', async () => {
- const run = (tok = live) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'live.viewers.line', input: { role: 'chat', user: 'say hi' }, credential: { subject: DANA } } });
+ const run = async (tok = live) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body: { workflow: 'live.viewers.line', input: { role: 'chat', user: 'say hi' }, credential: { subject: DANA } } });
let r = await run();
let r = await run();
assert.strictEqual(r.status, 201, r.text);
assert.strictEqual(r.status, 201, r.text);
assert.strictEqual(r.body.run.status, 'succeeded', JSON.stringify(r.body.run));
assert.strictEqual(r.body.run.status, 'succeeded', JSON.stringify(r.body.run));
@@ -65,7 +65,7 @@ t.test('a run with the credential uses that key only, uncached, and its budget s
@@ -65,7 +65,7 @@ t.test('a run with the credential uses that key only, uncached, and its budget s
assert.strictEqual(calls.length, 1);
assert.strictEqual(calls.length, 1);
assert.strictEqual(calls[0].url, 'https://llm.example.com/v1/chat/completions');
assert.strictEqual(calls[0].url, 'https://llm.example.com/v1/chat/completions');
assert.strictEqual(calls[0].auth, `Bearer ${API_KEY}`);
assert.strictEqual(calls[0].auth, `Bearer ${API_KEY}`);
- const reqs = h.db.prepare('SELECT provider_key, status, cost_usd FROM requests WHERE run_id = ?').all(r.body.run.id);
+ const reqs = await h.db.prepare('SELECT provider_key, status, cost_usd FROM requests WHERE run_id = ?').all(r.body.run.id);
assert.deepStrictEqual(reqs.map((x) => [x.provider_key, x.status]), [[`byo:live:${DANA}`, 'ok']]);
assert.deepStrictEqual(reqs.map((x) => [x.provider_key, x.status]), [[`byo:live:${DANA}`, 'ok']]);
assert.ok(Math.abs(reqs[0].cost_usd - 0.002) < 1e-9, 'priced at list price');
assert.ok(Math.abs(reqs[0].cost_usd - 0.002) < 1e-9, 'priced at list price');
r = await run();
r = await run();
new file mode 100644
new file mode 100644
@@ -0,0 +1,11 @@
@@ -0,0 +1,11 @@
+'use strict';
+/**
+ * A migrated database for one test boot (ADR-035), from openvibe-sdk/testing: PGlite by default; with
+ * AI_TEST_STORE=pg (npm run test:pg) the PostgreSQL + PgBouncer containers, with roles and a schema of its own.
+ */
+const { createTestDb, pgAvailable } = require('openvibe-sdk/testing');
+const { MIGRATIONS } = require('../server/db');
+
+const testDb = ({ store = process.env.AI_TEST_STORE || 'pglite', max = 4 } = {}) => createTestDb({ migrations: MIGRATIONS, store, service: 'ai', max });
+
+module.exports = { testDb, pgAvailable };
@@ -7,23 +7,23 @@ const contracts = require('openvibe-contracts');
@@ -7,23 +7,23 @@ const contracts = require('openvibe-contracts');
process.env.EVENTS_URL = 'http://127.0.0.1:9'; // unreachable: rows stay in the outbox
process.env.EVENTS_URL = 'http://127.0.0.1:9'; // unreachable: rows stay in the outbox
process.env.OV_OAUTH_CLIENT_SECRET = 'ai-secret-for-tests';
process.env.OV_OAUTH_CLIENT_SECRET = 'ai-secret-for-tests';
process.env.OV_NETWORK_INTERNAL_URL = 'http://127.0.0.1:9';
process.env.OV_NETWORK_INTERNAL_URL = 'http://127.0.0.1:9';
-const { boot, request, token, suite, ALL } = require('./helpers');
+const { boot, request, token, suite, tmpDir, ALL } = require('./helpers');
const t = suite('events');
const t = suite('events');
const live = token('live', ALL);
const live = token('live', ALL);
let h;
let h;
-const queued = () => h.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all().map((r) => JSON.parse(r.envelope));
+const queued = async () => (await h.db.prepare('SELECT envelope FROM event_outbox ORDER BY id').all()).map((r) => (typeof r.envelope === 'string' ? (typeof r.envelope === 'string' ? (typeof r.envelope === 'string' ? JSON.parse(r.envelope) : r.envelope) : r.envelope) : r.envelope));
const valid = (env) => { const r = contracts.validate(`${env.event_type}@1`, env.payload); assert.ok(r.valid, `${env.event_type}: ${JSON.stringify(r.errors)}`); };
const valid = (env) => { const r = contracts.validate(`${env.event_type}@1`, env.payload); assert.ok(r.valid, `${env.event_type}: ${JSON.stringify(r.errors)}`); };
t.test('boot with events on', async () => {
t.test('boot with events on', async () => {
- h = await boot({ env: { AI_STUB_FALLBACK: 'true' } });
- assert.strictEqual(require('../server/events').status().enabled, true);
+ h = await boot({ dir: tmpDir(), env: { AI_STUB_FALLBACK: 'true' } }); // a dir: the database outlives h.stop()
+ assert.strictEqual((await require('../server/events').status()).enabled, true);
});
});
t.test('a run queues ai.run.queued then ai.run.succeeded, valid and without the prompt or the text', async () => {
t.test('a run queues ai.run.queued then ai.run.succeeded, valid and without the prompt or the text', async () => {
const r = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'ai.generate', input: { prompt: 'secret prompt words' } } });
const r = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'ai.generate', input: { prompt: 'secret prompt words' } } });
assert.strictEqual(r.status, 201, r.text);
assert.strictEqual(r.status, 201, r.text);
- const evs = queued().filter((e) => e.payload.run_id === r.body.run.id);
+ const evs = (await queued()).filter((e) => e.payload.run_id === r.body.run.id);
assert.deepStrictEqual(evs.map((e) => e.event_type), ['ai.run.queued', 'ai.run.succeeded']);
assert.deepStrictEqual(evs.map((e) => e.event_type), ['ai.run.queued', 'ai.run.succeeded']);
evs.forEach(valid);
evs.forEach(valid);
assert.strictEqual(evs[0].visibility, 'internal');
assert.strictEqual(evs[0].visibility, 'internal');
@@ -37,7 +37,7 @@ t.test('a cache hit queues ai.run.cached with cached_from', async () => {
@@ -37,7 +37,7 @@ t.test('a cache hit queues ai.run.cached with cached_from', async () => {
const body = { workflow: 'ai.generate', input: { prompt: 'cache me' } };
const body = { workflow: 'ai.generate', input: { prompt: 'cache me' } };
const a = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body });
const a = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body });
const b = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body });
const b = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body });
- const ev = queued().find((e) => e.payload.run_id === b.body.run.id);
+ const ev = (await queued()).find((e) => e.payload.run_id === b.body.run.id);
if (b.body.run.status !== 'cached') { assert.ok(ev, 'an event for the second run'); return; } // this workflow may not cache
if (b.body.run.status !== 'cached') { assert.ok(ev, 'an event for the second run'); return; } // this workflow may not cache
assert.strictEqual(ev.event_type, 'ai.run.cached');
assert.strictEqual(ev.event_type, 'ai.run.cached');
assert.strictEqual(ev.payload.cached_from, a.body.run.id);
assert.strictEqual(ev.payload.cached_from, a.body.run.id);
@@ -47,13 +47,13 @@ t.test('a cache hit queues ai.run.cached with cached_from', async () => {
@@ -47,13 +47,13 @@ t.test('a cache hit queues ai.run.cached with cached_from', async () => {
t.test('a failed run queues ai.run.failed with its error code; a restart announces interrupted runs', async () => {
t.test('a failed run queues ai.run.failed with its error code; a restart announces interrupted runs', async () => {
const r = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'no.such.workflow', input: {} } });
const r = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'no.such.workflow', input: {} } });
assert.ok(r.status >= 400, 'refused before a run exists: no event');
assert.ok(r.status >= 400, 'refused before a run exists: no event');
- const before = queued().length;
+ const before = (await queued()).length;
// A queued run left behind by a restart: recoverInterrupted fails it and announces it.
// A queued run left behind by a restart: recoverInterrupted fails it and announces it.
- const row = h.db.prepare("SELECT * FROM runs WHERE status = 'succeeded' LIMIT 1").get();
- h.db.prepare("INSERT INTO runs (id, workflow_key, workflow_version, status, requester_type, requester_id, input, input_hash, created_at) VALUES ('run_01JAB2C3D4E5F6G7H8J9K0MNPQ', ?, ?, 'running', 'service', 'live', '{}', 'x', ?)")
+ const row = await h.db.prepare("SELECT * FROM runs WHERE status = 'succeeded' LIMIT 1").get();
+ await h.db.prepare("INSERT INTO runs (id, workflow_key, workflow_version, status, requester_type, requester_id, input, input_hash, created_at) VALUES ('run_01JAB2C3D4E5F6G7H8J9K0MNPQ', ?, ?, 'running', 'service', 'live', '{}', 'x', ?)")
.run(row.workflow_key, row.workflow_version, new Date().toISOString());
.run(row.workflow_key, row.workflow_version, new Date().toISOString());
- assert.strictEqual(h.runs.recoverInterrupted(), 1);
- const ev = queued().slice(before).find((e) => e.payload.run_id === 'run_01JAB2C3D4E5F6G7H8J9K0MNPQ');
+ assert.strictEqual(await h.runs.recoverInterrupted(), 1);
+ const ev = (await queued()).slice(before).find((e) => e.payload.run_id === 'run_01JAB2C3D4E5F6G7H8J9K0MNPQ');
assert.strictEqual(ev.event_type, 'ai.run.failed');
assert.strictEqual(ev.event_type, 'ai.run.failed');
assert.strictEqual(ev.payload.error.code, 'run.interrupted');
assert.strictEqual(ev.payload.error.code, 'run.interrupted');
assert.strictEqual(ev.priority, 'important');
assert.strictEqual(ev.priority, 'important');
@@ -62,14 +62,13 @@ t.test('a failed run queues ai.run.failed with its error code; a restart announc
@@ -62,14 +62,13 @@ t.test('a failed run queues ai.run.failed with its error code; a restart announc
t.test('close() stops the ai.run.* relay (unsent rows stay in the outbox)', async () => {
t.test('close() stops the ai.run.* relay (unsent rows stay in the outbox)', async () => {
const events = require('../server/events');
const events = require('../server/events');
- const pending = events.status().pending;
+ const pending = (await events.status()).pending;
assert.ok(pending > 0, 'rows are waiting (Events is unreachable)');
assert.ok(pending > 0, 'rows are waiting (Events is unreachable)');
await h.stop();
await h.stop();
- assert.strictEqual(events.status().enabled, false, 'the relay is stopped');
- const Database = require('better-sqlite3');
- const db = new Database(require('path').join(h.dir, 'ai.db'), { readonly: true });
- assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM event_outbox WHERE sent_at IS NULL AND rejected_at IS NULL').get().n, pending, 'and its rows wait for the next start');
- db.close();
+ assert.strictEqual((await events.status()).enabled, false, 'the relay is stopped');
+ const db = h.testdb.db;
+ assert.strictEqual((await db.prepare('SELECT COUNT(*) AS n FROM event_outbox WHERE sent_at IS NULL AND rejected_at IS NULL').get()).n, pending, 'and its rows wait for the next start');
+ await h.testdb.close();
events._reset();
events._reset();
});
});
t.run();
t.run();
@@ -22,7 +22,7 @@ async function provider(body) {
@@ -22,7 +22,7 @@ async function provider(body) {
const r = await request(h.base, 'POST', '/api/v1/providers', { tok, body: { kind: 'http', auth_mode: 'none', capabilities: ['chat', 'generate', 'json'], timeout_ms: 5000, ...body } });
const r = await request(h.base, 'POST', '/api/v1/providers', { tok, body: { kind: 'http', auth_mode: 'none', capabilities: ['chat', 'generate', 'json'], timeout_ms: 5000, ...body } });
assert.strictEqual(r.status, 201, r.text);
assert.strictEqual(r.status, 201, r.text);
}
}
-const generate = (prompt) => request(h.base, 'POST', '/api/v1/generate', { tok, body: { prompt, options: { cache: false } } });
+const generate = async (prompt) => await request(h.base, 'POST', '/api/v1/generate', { tok, body: { prompt, options: { cache: false } } });
t.test('boot with stub fallback off and two seam providers', async () => {
t.test('boot with stub fallback off and two seam providers', async () => {
ok = await seamServer(() => ({ text: 'a real answer', model: 'seam-model', usage: { input: 12, output: 4 } }));
ok = await seamServer(() => ({ text: 'a real answer', model: 'seam-model', usage: { input: 12, output: 4 } }));
@@ -61,7 +61,7 @@ t.test('a failing primary falls back, and the run, request log and audit all say
@@ -61,7 +61,7 @@ t.test('a failing primary falls back, and the run, request log and audit all say
t.test('the circuit opens after repeated failures and the open provider is skipped', async () => {
t.test('the circuit opens after repeated failures and the open provider is skipped', async () => {
await generate('second'); // second consecutive failure of `dead` opens it
await generate('second'); // second consecutive failure of `dead` opens it
- assert.strictEqual(h.pool.health('dead').state, 'open');
+ assert.strictEqual((await h.pool.health('dead')).state, 'open');
const r = await generate('third');
const r = await generate('third');
const d = await request(h.base, 'GET', `/api/v1/runs/${r.body.run.id}`, { tok });
const d = await request(h.base, 'GET', `/api/v1/runs/${r.body.run.id}`, { tok });
const skip = d.body.requests.find(x => x.provider_key === 'dead');
const skip = d.body.requests.find(x => x.provider_key === 'dead');
@@ -42,16 +42,24 @@ function tmpDir() {
@@ -42,16 +42,24 @@ function tmpDir() {
}
}
/** Boot the service. `env` goes through config.load() and is also what secrets resolve against. */
/** Boot the service. `env` goes through config.load() and is also what secrets resolve against. */
-async function boot({ env = {}, dir = tmpDir(), clock, fetchImpl, credentialFetch } = {}) {
+// A boot on a dir used before is a restart on the same database (as the SQLite file was): its database stays open.
+const restartable = new Map();
+
+async function boot({ env = {}, dir, clock, fetchImpl, credentialFetch } = {}) {
+ const reuse = dir !== undefined;
+ if (dir === undefined) dir = tmpDir();
const fullEnv = {
const fullEnv = {
NODE_ENV: 'test', PORT: '0', AI_DB_PATH: path.join(dir, 'ai.db'), OV_NETWORK_PUBLIC_KEY: publicKey,
NODE_ENV: 'test', PORT: '0', AI_DB_PATH: path.join(dir, 'ai.db'), OV_NETWORK_PUBLIC_KEY: publicKey,
AI_PROVIDER_RETRY_DELAY_MS: '5', AI_QUOTA_SERVICE_RPM: '0', AI_QUOTA_SERVICE_RPD: '0',
AI_PROVIDER_RETRY_DELAY_MS: '5', AI_QUOTA_SERVICE_RPM: '0', AI_QUOTA_SERVICE_RPD: '0',
...env,
...env,
};
};
const config = load(fullEnv);
const config = load(fullEnv);
- const h = await start({ config, env: fullEnv, log: silent, clock, fetchImpl, credentialFetch });
+ // One database per boot (PGlite, or AI_TEST_STORE=pg: the containers), dropped when the boot stops.
+ const testdb = (reuse && restartable.get(dir)) || await require('./db').testDb();
+ if (reuse) restartable.set(dir, testdb);
+ const h = await start({ config, db: testdb.db, env: fullEnv, log: silent, clock, fetchImpl, credentialFetch });
const base = `http://127.0.0.1:${h.server.address().port}`;
const base = `http://127.0.0.1:${h.server.address().port}`;
- return { ...h, base, dir, env: fullEnv, stop: () => h.close() };
+ return { ...h, base, dir, env: fullEnv, testdb, stop: async () => { await h.close(); if (!reuse) await testdb.close(); } };
}
}
async function request(base, method, p, { tok, body, headers = {} } = {}) {
async function request(base, method, p, { tok, body, headers = {} } = {}) {
deleted file mode 100644
deleted file mode 100644
@@ -1,113 +0,0 @@
@@ -1,113 +0,0 @@
-'use strict';
-// scripts/import-from-live.js against a synthetic Live snapshot: --dry-run writes nothing; a real
-// run imports usage, budgets, per-role models and pricing, and holds what cannot be represented
-// (secrets, BYO keys, text-less translation cache); a second run imports nothing; a new Live row
-// is picked up alone. Secret values never appear in the report or the holds.
-
-const assert = require('assert');
-const fs = require('fs');
-const path = require('path');
-const Database = require('better-sqlite3');
-const { suite, tmpDir } = require('./helpers');
-const { run } = require('../scripts/import-from-live');
-
-const t = suite('import');
-const dir = tmpDir();
-const liveDb = path.join(dir, 'live-snapshot.db');
-const aiDb = path.join(dir, 'ai.db');
-const SECRET = 'sk-live-snapshot-secret-DO-NOT-PRINT';
-const env = { NODE_ENV: 'test', AI_QUOTA_SERVICE_RPM: '0', AI_QUOTA_SERVICE_RPD: '0' };
-const quiet = () => {};
-const today = new Date().toISOString().slice(0, 10);
-
-t.test('build a Live snapshot', () => {
- const d = new Database(liveDb);
- d.exec(`CREATE TABLE site_settings (key TEXT PRIMARY KEY, value TEXT NOT NULL DEFAULT '', description TEXT, type TEXT, updated_at DATETIME);
- CREATE TABLE ai_usage (id INTEGER PRIMARY KEY AUTOINCREMENT, kind TEXT, model TEXT, input_tokens INTEGER, output_tokens INTEGER, cost_usd REAL, owner_user_id INTEGER, source TEXT, created_at DATETIME, cached_tokens INTEGER, role TEXT, provider TEXT, latency_ms INTEGER);
- CREATE TABLE channel_ai_config (user_id INTEGER PRIMARY KEY, enabled INTEGER, use_shared_key INTEGER, daily_budget_cents INTEGER, byo_key TEXT, byo_base_url TEXT, byo_model TEXT, settings_json TEXT);
- CREATE TABLE translations (key TEXT PRIMARY KEY, src TEXT, dst TEXT, text TEXT NOT NULL, created_at DATETIME);
- CREATE TABLE ai_timeline_cache (user_id INTEGER PRIMARY KEY, payload TEXT, generated_at DATETIME);
- CREATE TABLE ai_chatbot_configs (user_id INTEGER PRIMARY KEY, api_token TEXT);`);
- const set = d.prepare('INSERT INTO site_settings (key, value) VALUES (?, ?)');
- for (const [k, v] of [['ai_enabled', 'true'], ['ai_provider', 'openai'], ['ai_api_key', SECRET], ['ai_model', 'gpt-5-mini'], ['ai_model_director', 'gpt-5'], ['ai_model_vision', ''],
- ['ai_pricing_json', '{"gpt-5-nano":{"in":0.05,"out":0.4,"cached":0.005},"default":{"in":1,"out":2}}'], ['ai_max_cost_usd_per_day', '5'], ['ai_viewers_global_cap_usd_per_day', '2']]) set.run(k, v);
- const use = d.prepare('INSERT INTO ai_usage (kind, model, input_tokens, output_tokens, cost_usd, owner_user_id, created_at, cached_tokens, provider) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)');
- use.run('translate', 'gpt-5-nano', 100, 20, 0.01, null, `${today} 10:00:00`, 0, 'shared');
- use.run('ai_viewers_director', 'gpt-5', 1000, 200, 0.2, 42, `${today} 10:01:00`, 500, 'shared');
- use.run('ai_viewers', 'llama3', 50, 10, 0, 7, '2026-09-01 08:00:00', 0, 'byo');
- use.run('weird_kind', 'gpt-5-mini', 10, 10, 0.001, null, '2026-09-02 08:00:00', 0, 'shared');
- use.run('paste_text', 'gpt-5-mini', 10, 10, 0.001, null, 'garbage', 0, 'shared');
- const ch = d.prepare('INSERT INTO channel_ai_config (user_id, use_shared_key, daily_budget_cents, byo_key, byo_base_url) VALUES (?, ?, ?, ?, ?)');
- ch.run(42, 1, 20, '', '');
- ch.run(7, 0, 20, 'sk-byo-streamer-secret', 'https://llm.example.org/v1');
- d.prepare("INSERT INTO translations (key, src, dst, text) VALUES ('abc', 'ja', 'en', 'hello')").run();
- d.close();
-});
-
-t.test('--dry-run reports the plan and writes nothing', () => {
- const r = run({ liveDb, aiDb, dryRun: true, env, log: quiet });
- assert.strictEqual(r.dry_run, true);
- assert.strictEqual(fs.existsSync(aiDb), false, 'no AI database created');
- assert.strictEqual(r.sources['live.ai_usage'].imported, 4);
- assert.strictEqual(r.sources['live.ai_usage'].held, 1);
- assert.ok(!JSON.stringify(r).includes(SECRET));
-});
-
-t.test('a real run imports and holds, without leaking secrets', () => {
- const r = run({ liveDb, aiDb, env, log: quiet });
- assert.ok(!JSON.stringify(r).includes(SECRET) && !JSON.stringify(r).includes('sk-byo'));
- assert.ok(r.environment.some(l => l.startsWith('AI_API_KEY=<')), 'tells the operator which secret NAME to set');
- const d = new Database(aiDb, { readonly: true });
- const quotas = d.prepare('SELECT * FROM quotas WHERE origin = ?').all('import');
- assert.ok(quotas.some(q => q.scope_type === 'global' && q.max_cost_usd === 5 && !q.workflow_prefix));
- assert.ok(quotas.some(q => q.scope_type === 'global' && q.max_cost_usd === 2 && q.workflow_prefix === 'live.viewers.'));
- assert.ok(quotas.some(q => q.scope_type === 'attribution' && q.scope_id === 'live:user:42' && q.max_cost_usd === 0.2));
- const route = d.prepare("SELECT * FROM routes WHERE key = 'live.director' ORDER BY version DESC LIMIT 1").get();
- assert.strictEqual(route.primary_model, 'gpt-5');
- assert.strictEqual(route.created_by, 'import');
- const model = d.prepare("SELECT * FROM models WHERE provider_key = 'shared' AND model_key = 'gpt-5-nano'").get();
- assert.strictEqual(model.cost_in_per_mtok, 0.05);
- const usage = d.prepare('SELECT * FROM usage_daily').all();
- assert.ok(usage.some(u => u.workflow_key === 'live.translate' && u.day === today));
- assert.ok(usage.some(u => u.workflow_key === 'live.viewers.plan' && u.attribution === 'live:user:42' && u.tokens_cached === 500));
- assert.ok(usage.some(u => u.provider_key === 'live-byo'));
- assert.ok(usage.some(u => u.workflow_key === 'live.complete'), 'unknown kinds land on live.complete');
- const counter = d.prepare("SELECT * FROM usage_counters WHERE scope_type = 'global' AND window = 'day'").get();
- assert.ok(Math.abs(counter.cost_usd - 0.21) < 1e-9, "today's spend counts toward today's cap");
- const holds = d.prepare('SELECT * FROM import_holds').all();
- const reasons = Object.fromEntries(holds.map(h => [`${h.source}:${h.source_key}`, h.reason]));
- assert.strictEqual(reasons['live.site_settings:ai_api_key'], 'secret');
- assert.strictEqual(reasons['live.channel_ai_config:7'], 'byo_provider');
- assert.strictEqual(reasons['live.translations:*'], 'unrepresentable');
- assert.strictEqual(reasons['live.ai_usage:5'], 'unrepresentable');
- assert.ok(!JSON.stringify(holds).includes(SECRET) && !JSON.stringify(holds).includes('sk-byo'));
- d.close();
-});
-
-t.test('a second run imports nothing and changes no totals', () => {
- const before = new Database(aiDb, { readonly: true });
- const sum = (x) => x.prepare('SELECT SUM(requests) r, SUM(cost_usd) c FROM usage_daily').get();
- const s1 = sum(before);
- before.close();
- const r = run({ liveDb, aiDb, env, log: quiet });
- for (const [src, s] of Object.entries(r.sources)) assert.strictEqual(s.imported + s.held, 0, `${src} imported again: ${JSON.stringify(s)}`);
- const after = new Database(aiDb, { readonly: true });
- assert.deepStrictEqual(sum(after), s1);
- assert.strictEqual(after.prepare("SELECT COUNT(*) n FROM routes WHERE key = 'live.director'").get().n, 2, 'no extra route version');
- after.close();
-});
-
-t.test('new Live rows and changed settings are picked up alone', () => {
- const d = new Database(liveDb);
- d.prepare("INSERT INTO ai_usage (kind, model, input_tokens, output_tokens, cost_usd, created_at, provider) VALUES ('hero_slogans', 'gpt-5-mini', 5, 5, 0.002, ?, 'shared')").run(`${today} 11:00:00`);
- d.prepare("UPDATE site_settings SET value = '8' WHERE key = 'ai_max_cost_usd_per_day'").run();
- d.close();
- const r = run({ liveDb, aiDb, env, log: quiet });
- assert.strictEqual(r.sources['live.ai_usage'].imported, 1);
- assert.strictEqual(r.sources['live.site_settings'].imported, 1);
- const a = new Database(aiDb, { readonly: true });
- assert.strictEqual(a.prepare("SELECT max_cost_usd FROM quotas WHERE scope_type = 'global' AND workflow_prefix IS NULL").get().max_cost_usd, 8);
- a.close();
-});
-
-t.run();
@@ -40,7 +40,7 @@ const LOG = [
@@ -40,7 +40,7 @@ const LOG = [
const t = suite('media-analysis');
const t = suite('media-analysis');
let media; let mediaBase;
let media; let mediaBase;
const tok = token('live', ALL);
const tok = token('live', ALL);
-const run = (h, input) => request(h.base, 'POST', '/api/v1/runs?wait=60000', { tok, body: { workflow: 'media.analyze', input: { media_url: `${mediaBase}/fixture.mp4`, language: 'en', ...input } } });
+const run = async (h, input) => await request(h.base, 'POST', '/api/v1/runs?wait=60000', { tok, body: { workflow: 'media.analyze', input: { media_url: `${mediaBase}/fixture.mp4`, language: 'en', ...input } } });
const completion = (content) => ({ id: 'x', object: 'chat.completion', model: 'fake-model', choices: [{ index: 0, message: { role: 'assistant', content }, finish_reason: 'stop' }], usage: { prompt_tokens: 50, completion_tokens: 20 } });
const completion = (content) => ({ id: 'x', object: 'chat.completion', model: 'fake-model', choices: [{ index: 0, message: { role: 'assistant', content }, finish_reason: 'stop' }], usage: { prompt_tokens: 50, completion_tokens: 20 } });
t.test('the parser reads every filter\'s lines; loud means well above the typical second', async () => {
t.test('the parser reads every filter\'s lines; loud means well above the typical second', async () => {
@@ -151,7 +151,7 @@ t.test('local model down: extractive, and it says so; paid only with allow_paid
@@ -151,7 +151,7 @@ t.test('local model down: extractive, and it says so; paid only with allow_paid
assert.ok(out.gaps.some((g) => /no quota gives media\.paid a cost budget/.test(g)));
assert.ok(out.gaps.some((g) => /no quota gives media\.paid a cost budget/.test(g)));
assert.strictEqual(paid.calls, 0, 'no paid call without a budget');
assert.strictEqual(paid.calls, 0, 'no paid call without a budget');
- h.quotas.upsert({ scope_type: 'global', scope_id: '*', window: 'day', max_cost_usd: 0.5, workflow_prefix: 'media.paid' }, { actor: 'test' });
+ await h.quotas.upsert({ scope_type: 'global', scope_id: '*', window: 'day', max_cost_usd: 0.5, workflow_prefix: 'media.paid' }, { actor: 'test' });
r = await run(h, { allow_paid: true });
r = await run(h, { allow_paid: true });
out = r.body.run.output;
out = r.body.run.output;
assert.deepStrictEqual([out.overview.source, out.overview.text], ['paid', 'Paid overview of the recording.'], JSON.stringify(out.gaps));
assert.deepStrictEqual([out.overview.source, out.overview.text], ['paid', 'Paid overview of the recording.'], JSON.stringify(out.gaps));
@@ -163,7 +163,7 @@ t.test('local model down: extractive, and it says so; paid only with allow_paid
@@ -163,7 +163,7 @@ t.test('local model down: extractive, and it says so; paid only with allow_paid
const day = new Date().toISOString().slice(0, 10);
const day = new Date().toISOString().slice(0, 10);
const callsBefore = paid.calls;
const callsBefore = paid.calls;
- h.db.prepare("INSERT INTO usage_daily (day, requester, attribution, workflow_key, provider_key, model_key, requests, cost_usd) VALUES (?, 'service:live', '', 'media.analyze', 'shared', 'paid-model', 1, 0.75)").run(day);
+ await h.db.prepare("INSERT INTO usage_daily (day, requester, attribution, workflow_key, provider_key, model_key, requests, cost_usd) VALUES (?, 'service:live', '', 'media.analyze', 'shared', 'paid-model', 1, 0.75)").run(day);
r = await run(h, { allow_paid: true });
r = await run(h, { allow_paid: true });
out = r.body.run.output;
out = r.body.run.output;
assert.strictEqual(r.body.run.status, 'succeeded', 'a spent budget refuses the paid call, not the analysis');
assert.strictEqual(r.body.run.status, 'succeeded', 'a spent budget refuses the paid call, not the analysis');
@@ -85,7 +85,7 @@ t.test('/metrics: 404 through a proxy; queued and running runs, route templates,
@@ -85,7 +85,7 @@ t.test('/metrics: 404 through a proxy; queued and running runs, route templates,
});
});
t.test('an open circuit on a real provider degrades /api/ready (still 200) and shows on /metrics', async () => {
t.test('an open circuit on a real provider degrades /api/ready (still 200) and shows on /metrics', async () => {
- h.db.prepare("INSERT INTO provider_health (provider_key, state, consecutive_failures, opened_at, last_error) VALUES ('shared', 'open', 3, ?, 'test')").run(Date.now());
+ await h.db.prepare("INSERT INTO provider_health (provider_key, state, consecutive_failures, opened_at, last_error) VALUES ('shared', 'open', 3, ?, 'test')").run(Date.now());
const r = await request(h.base, 'GET', '/api/ready');
const r = await request(h.base, 'GET', '/api/ready');
assert.strictEqual(r.status, 200, r.text);
assert.strictEqual(r.status, 200, r.text);
assert.strictEqual(r.body.status, 'degraded');
assert.strictEqual(r.body.status, 'degraded');
@@ -94,7 +94,7 @@ t.test('an open circuit on a real provider degrades /api/ready (still 200) and s
@@ -94,7 +94,7 @@ t.test('an open circuit on a real provider degrades /api/ready (still 200) and s
const text = (await get(h.base, '/metrics')).body;
const text = (await get(h.base, '/metrics')).body;
assert.ok(/ai_provider_circuit\{provider="shared",state="open"\} 1\n/.test(text));
assert.ok(/ai_provider_circuit\{provider="shared",state="open"\} 1\n/.test(text));
assert.ok(/ai_provider_circuit\{provider="shared",state="closed"\} 0\n/.test(text));
assert.ok(/ai_provider_circuit\{provider="shared",state="closed"\} 0\n/.test(text));
- h.pool.resetHealth('shared');
+ await h.pool.resetHealth('shared');
});
});
t.test('a broken database makes the service unready (503); /metrics still answers', async () => {
t.test('a broken database makes the service unready (503); /metrics still answers', async () => {
@@ -14,8 +14,8 @@ const b = token('tools', ['ai.run.create', 'ai.run.read']);
@@ -14,8 +14,8 @@ const b = token('tools', ['ai.run.create', 'ai.run.read']);
let h;
let h;
const sleep = (ms) => new Promise(r => setTimeout(r, ms));
const sleep = (ms) => new Promise(r => setTimeout(r, ms));
-const create = (tok, i) => request(h.base, 'POST', '/api/v1/runs', { tok, body: { workflow: 'ai.generate', input: { prompt: `queued ${i}` }, options: { cache: false } } });
-const count = (who) => h.db.prepare('SELECT COUNT(*) AS n FROM runs WHERE requester_id = ?').get(who).n;
+const create = async (tok, i) => await request(h.base, 'POST', '/api/v1/runs', { tok, body: { workflow: 'ai.generate', input: { prompt: `queued ${i}` }, options: { cache: false } } });
+const count = async (who) => (await h.db.prepare('SELECT COUNT(*) AS n FROM runs WHERE requester_id = ?').get(who)).n;
function assertFull(r, scope) {
function assertFull(r, scope) {
assert.strictEqual(r.status, 429, r.text);
assert.strictEqual(r.status, 429, r.text);
@@ -39,9 +39,9 @@ t.test('one caller is held to its per-caller cap; others still get in', async ()
@@ -39,9 +39,9 @@ t.test('one caller is held to its per-caller cap; others still get in', async ()
assert.strictEqual(r.status, 202, r.text);
assert.strictEqual(r.status, 202, r.text);
mine.push(r.body.run.id);
mine.push(r.body.run.id);
}
}
- const before = count('live');
+ const before = await count('live');
assertFull(await create(a, 'over'), 'caller');
assertFull(await create(a, 'over'), 'caller');
- assert.strictEqual(count('live'), before, 'a refused run leaves no row');
+ assert.strictEqual(await count('live'), before, 'a refused run leaves no row');
const other = await create(b, 0);
const other = await create(b, 0);
assert.strictEqual(other.status, 202, 'another caller still queues');
assert.strictEqual(other.status, 202, 'another caller still queues');
h.mine = mine;
h.mine = mine;
@@ -12,8 +12,8 @@ const tools = token('tools', ['ai.run.create', 'ai.run.read']);
@@ -12,8 +12,8 @@ const tools = token('tools', ['ai.run.create', 'ai.run.read']);
let h;
let h;
let seam;
let seam;
-const runCount = () => h.db.prepare('SELECT COUNT(*) n FROM runs').get().n;
-const gen = (tok, prompt, extra = {}) => request(h.base, 'POST', '/api/v1/generate', { tok, body: { prompt, options: { cache: false }, ...extra } });
+const runCount = async () => (await h.db.prepare('SELECT COUNT(*) n FROM runs').get()).n;
+const gen = async (tok, prompt, extra = {}) => await request(h.base, 'POST', '/api/v1/generate', { tok, body: { prompt, options: { cache: false }, ...extra } });
t.test('boot with a counting seam provider as the only route target', async () => {
t.test('boot with a counting seam provider as the only route target', async () => {
seam = await seamServer(() => ({ text: 'ok', usage: { input: 1000, output: 1000 } }));
seam = await seamServer(() => ({ text: 'ok', usage: { input: 1000, output: 1000 } }));
@@ -27,7 +27,7 @@ t.test('boot with a counting seam provider as the only route target', async () =
@@ -27,7 +27,7 @@ t.test('boot with a counting seam provider as the only route target', async () =
t.test('the per-service rate refuses the 4th request with 429 before the provider is called', async () => {
t.test('the per-service rate refuses the 4th request with 429 before the provider is called', async () => {
for (let i = 0; i < 3; i++) assert.strictEqual((await gen(live, `p${i}`)).status, 201);
for (let i = 0; i < 3; i++) assert.strictEqual((await gen(live, `p${i}`)).status, 201);
const callsBefore = seam.calls;
const callsBefore = seam.calls;
- const runsBefore = runCount();
+ const runsBefore = await runCount();
const r = await gen(live, 'one too many');
const r = await gen(live, 'one too many');
assert.strictEqual(r.status, 429);
assert.strictEqual(r.status, 429);
assert.strictEqual(r.body.code, 'quota.exceeded');
assert.strictEqual(r.body.code, 'quota.exceeded');
@@ -36,7 +36,7 @@ t.test('the per-service rate refuses the 4th request with 429 before the provide
@@ -36,7 +36,7 @@ t.test('the per-service rate refuses the 4th request with 429 before the provide
assert.strictEqual(r.body.quota.scope_type, 'service');
assert.strictEqual(r.body.quota.scope_type, 'service');
assert.strictEqual(r.body.quota.scope_id, 'live');
assert.strictEqual(r.body.quota.scope_id, 'live');
assert.strictEqual(seam.calls, callsBefore, 'the provider was never called');
assert.strictEqual(seam.calls, callsBefore, 'the provider was never called');
- assert.strictEqual(runCount(), runsBefore, 'no run was created');
+ assert.strictEqual(await runCount(), runsBefore, 'no run was created');
});
});
t.test("'*' service quotas give each service its own window", async () => {
t.test("'*' service quotas give each service its own window", async () => {
@@ -99,12 +99,12 @@ t.test('passthrough prompts and inline images are not kept on the run record', a
@@ -99,12 +99,12 @@ t.test('passthrough prompts and inline images are not kept on the run record', a
await request(h.base, 'POST', '/api/v1/providers', { tok: live, body: { key: 'slowstub', kind: 'stub', capabilities: ['chat', 'generate', 'summarize', 'json', 'vision'], metadata: { delay_ms: 10 } } });
await request(h.base, 'POST', '/api/v1/providers', { tok: live, body: { key: 'slowstub', kind: 'stub', capabilities: ['chat', 'generate', 'summarize', 'json', 'vision'], metadata: { delay_ms: 10 } } });
const p = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'live.chat.insight', input: { role: 'chat', kind: 'chat_user', user: 'PRIVATE CHAT LOG: alice said hi' } } });
const p = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'live.chat.insight', input: { role: 'chat', kind: 'chat_user', user: 'PRIVATE CHAT LOG: alice said hi' } } });
assert.strictEqual(p.body.run.status, 'succeeded');
assert.strictEqual(p.body.run.status, 'succeeded');
- const row = h.db.prepare('SELECT input, input_hash FROM runs WHERE id = ?').get(p.body.run.id);
+ const row = await h.db.prepare('SELECT input, input_hash FROM runs WHERE id = ?').get(p.body.run.id);
assert.ok(!row.input.includes('PRIVATE CHAT LOG'), 'raw prompt not stored');
assert.ok(!row.input.includes('PRIVATE CHAT LOG'), 'raw prompt not stored');
assert.ok(row.input_hash);
assert.ok(row.input_hash);
const png = `data:image/png;base64,${Buffer.alloc(600, 7).toString('base64')}`;
const png = `data:image/png;base64,${Buffer.alloc(600, 7).toString('base64')}`;
const img = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'live.stream.describe_frame', input: { image: { data_url: png } } } });
const img = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'live.stream.describe_frame', input: { image: { data_url: png } } } });
- const irow = h.db.prepare('SELECT input FROM runs WHERE id = ?').get(img.body.run.id);
+ const irow = await h.db.prepare('SELECT input FROM runs WHERE id = ?').get(img.body.run.id);
assert.ok(!irow.input.includes(png.slice(40, 120)) && irow.input.includes('data_url_sha256'));
assert.ok(!irow.input.includes(png.slice(40, 120)) && irow.input.includes('data_url_sha256'));
await request(h.base, 'POST', '/api/v1/providers/slowstub/disable', { tok: live });
await request(h.base, 'POST', '/api/v1/providers/slowstub/disable', { tok: live });
const failed = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'live.chat.insight', input: { role: 'chat', user: 'again' } } });
const failed = await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'live.chat.insight', input: { role: 'chat', user: 'again' } } });
@@ -138,11 +138,11 @@ t.test('runs still in flight at shutdown end as run.interrupted, retryable after
@@ -138,11 +138,11 @@ t.test('runs still in flight at shutdown end as run.interrupted, retryable after
const d = await getRun(r.body.run.id);
const d = await getRun(r.body.run.id);
assert.strictEqual(d.run.status, 'failed');
assert.strictEqual(d.run.status, 'failed');
assert.strictEqual(d.run.error.code, 'run.interrupted');
assert.strictEqual(d.run.error.code, 'run.interrupted');
- h.db.prepare("UPDATE runs SET status = 'running', error_code = NULL, finished_at = NULL WHERE id = ?").run(r.body.run.id);
+ await h.db.prepare("UPDATE runs SET status = 'running', error_code = NULL, finished_at = NULL WHERE id = ?").run(r.body.run.id);
await h.stop();
await h.stop();
h = await boot({ dir, env: { AI_MAX_CONCURRENT_RUNS: '1', AI_STUB_FALLBACK: 'false' } });
h = await boot({ dir, env: { AI_MAX_CONCURRENT_RUNS: '1', AI_STUB_FALLBACK: 'false' } });
assert.strictEqual((await getRun(r.body.run.id)).run.error.code, 'run.interrupted', 'boot recovery marks orphans too');
assert.strictEqual((await getRun(r.body.run.id)).run.error.code, 'run.interrupted', 'boot recovery marks orphans too');
- assert.strictEqual(h.registry.getWorkflow('ai.generate').version, 2, 'a reboot does not clobber admin versions with the seed');
+ assert.strictEqual((await h.registry.getWorkflow('ai.generate')).version, 2, 'a reboot does not clobber admin versions with the seed');
});
});
t.test('shutdown', async () => { await h.stop(); });
t.test('shutdown', async () => { await h.stop(); });
@@ -28,7 +28,7 @@ t.test('address policy: internal, loopback, link-local, mapped IPv6 are not publ
@@ -28,7 +28,7 @@ t.test('address policy: internal, loopback, link-local, mapped IPv6 are not publ
for (const ip of ['1.1.1.1', '93.184.216.34', '2606:4700:4700::1111']) assert.strictEqual(isPublicAddress(ip), true, ip);
for (const ip of ['1.1.1.1', '93.184.216.34', '2606:4700:4700::1111']) assert.strictEqual(isPublicAddress(ip), true, ip);
});
});
-t.test('object storage is followed only as a redirect from Media', () => {
+t.test('object storage is followed only as a redirect from Media', async () => {
const f = createFetcher(load({ NODE_ENV: 'test', OV_MEDIA_INTERNAL_URL: 'http://127.0.0.1:4100' }));
const f = createFetcher(load({ NODE_ENV: 'test', OV_MEDIA_INTERNAL_URL: 'http://127.0.0.1:4100' }));
const media = { url: new URL('https://openvibe.media/v/1'), internal: false };
const media = { url: new URL('https://openvibe.media/v/1'), internal: false };
const internal = { url: new URL('http://127.0.0.1:4100/v/1'), internal: true };
const internal = { url: new URL('http://127.0.0.1:4100/v/1'), internal: true };
@@ -37,11 +37,11 @@ t.test('object storage is followed only as a redirect from Media', () => {
@@ -37,11 +37,11 @@ t.test('object storage is followed only as a redirect from Media', () => {
assert.strictEqual(f.follow(media, b2).url.hostname, 's3.us-west-004.backblazeb2.com', 'Media -> B2');
assert.strictEqual(f.follow(media, b2).url.hostname, 's3.us-west-004.backblazeb2.com', 'Media -> B2');
assert.strictEqual(f.follow(internal, 'https://acct.r2.cloudflarestorage.com/b/x').internal, false, 'internal Media -> R2, as a public target');
assert.strictEqual(f.follow(internal, 'https://acct.r2.cloudflarestorage.com/b/x').internal, false, 'internal Media -> R2, as a public target');
const refused = (fn, why) => { try { fn(); assert.fail(why); } catch (e) { assert.strictEqual(e.code, 'fetch.refused', why); } };
const refused = (fn, why) => { try { fn(); assert.fail(why); } catch (e) { assert.strictEqual(e.code, 'fetch.refused', why); } };
- refused(() => f.follow(other, b2), 'another host may not send it to storage');
- refused(() => f.judge(b2), 'storage is never a URL a caller names');
- refused(() => f.follow(media, 'http://s3.us-west-004.backblazeb2.com/x'), 'https only');
- refused(() => f.follow(media, 'https://evil.example/x'), 'not storage');
- refused(() => f.follow(media, 'https://user:[email protected]/x'), 'no credentials in the URL');
+ await refused(() => f.follow(other, b2), 'another host may not send it to storage');
+ await refused(() => f.judge(b2), 'storage is never a URL a caller names');
+ await refused(() => f.follow(media, 'http://s3.us-west-004.backblazeb2.com/x'), 'https only');
+ await refused(() => f.follow(media, 'https://evil.example/x'), 'not storage');
+ await refused(() => f.follow(media, 'https://user:[email protected]/x'), 'no credentials in the URL');
});
});
t.test('URL rules: https + allow-listed host only', async () => {
t.test('URL rules: https + allow-listed host only', async () => {
@@ -20,7 +20,7 @@ t.test('boot with an internal Media stand-in', async () => {
@@ -20,7 +20,7 @@ t.test('boot with an internal Media stand-in', async () => {
});
});
t.test('a window starts at start_sec and its timestamps follow', async () => {
t.test('a window starts at start_sec and its timestamps follow', async () => {
- const run = (input) => request(h.base, 'POST', '/api/v1/runs?wait=20000', { tok, body: { workflow: 'live.media.transcribe', input: { media_url: `${base}/v/7`, language: 'en', ...input } } });
+ const run = async (input) => await request(h.base, 'POST', '/api/v1/runs?wait=20000', { tok, body: { workflow: 'live.media.transcribe', input: { media_url: `${base}/v/7`, language: 'en', ...input } } });
let r = await run({ start_sec: 300, seconds: 60, low_power: true });
let r = await run({ start_sec: 300, seconds: 60, low_power: true });
assert.strictEqual(r.body.run.status, 'succeeded', JSON.stringify(r.body.run.error || r.body));
assert.strictEqual(r.body.run.status, 'succeeded', JSON.stringify(r.body.run.error || r.body));
const segs = r.body.run.output.segments;
const segs = r.body.run.output.segments;
@@ -18,7 +18,7 @@ let h, seam, network;
@@ -18,7 +18,7 @@ let h, seam, network;
const prefs = { [A.id]: { style: 'casual', length: 'short', perspective: 'a night-shift worker', history: false } };
const prefs = { [A.id]: { style: 'casual', length: 'short', perspective: 'a night-shift worker', history: false } };
const puts = [];
const puts = [];
const systemOf = () => (seam.last.system || []).map((s) => s.text || s).join('\n');
const systemOf = () => (seam.last.system || []).map((s) => s.text || s).join('\n');
-const summarize = (extra = {}) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'ai.summarize', input: { text: 'A long day at the plant, told briefly.' }, ...extra } });
+const summarize = async (extra = {}) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok: live, body: { workflow: 'ai.summarize', input: { text: 'A long day at the plant, told briefly.' }, ...extra } });
t.test('boot with a stub Network and a seam provider', async () => {
t.test('boot with a stub Network and a seam provider', async () => {
network = http.createServer((req, res) => {
network = http.createServer((req, res) => {
@@ -56,7 +56,7 @@ t.test("a run on A's behalf follows A's preferences; history off keeps no input
@@ -56,7 +56,7 @@ t.test("a run on A's behalf follows A's preferences; history off keeps no input
assert.strictEqual(r.body.run.status, 'succeeded', r.text);
assert.strictEqual(r.body.run.status, 'succeeded', r.text);
const sys = systemOf();
const sys = systemOf();
assert.ok(sys.includes('casual style') && sys.includes('Keep it short.') && sys.includes('a night-shift worker'), sys);
assert.ok(sys.includes('casual style') && sys.includes('Keep it short.') && sys.includes('a night-shift worker'), sys);
- const row = h.db.prepare('SELECT input, cache_key, options FROM runs WHERE id = ?').get(r.body.run.id);
+ const row = await h.db.prepare('SELECT input, cache_key, options FROM runs WHERE id = ?').get(r.body.run.id);
assert.strictEqual(row.input, 'null'); assert.strictEqual(row.cache_key, null);
assert.strictEqual(row.input, 'null'); assert.strictEqual(row.cache_key, null);
assert.strictEqual(JSON.parse(row.options).history, false);
assert.strictEqual(JSON.parse(row.options).history, false);
const calls = seam.calls;
const calls = seam.calls;
@@ -70,13 +70,13 @@ t.test('runs for someone without preferences, or for nobody, are untouched', asy
@@ -70,13 +70,13 @@ t.test('runs for someone without preferences, or for nobody, are untouched', asy
assert.ok(!systemOf().includes('The person this is for asked'));
assert.ok(!systemOf().includes('The person this is for asked'));
const r = await summarize();
const r = await summarize();
assert.ok(!systemOf().includes('The person this is for asked'));
assert.ok(!systemOf().includes('The person this is for asked'));
- assert.notStrictEqual(h.db.prepare('SELECT input FROM runs WHERE id = ?').get(r.body.run.id).input, 'null');
+ assert.notStrictEqual((await h.db.prepare('SELECT input FROM runs WHERE id = ?').get(r.body.run.id)).input, 'null');
});
});
t.test('ai.usage_summary: 30-day runs and tokens per person, written only when changed', async () => {
t.test('ai.usage_summary: 30-day runs and tokens per person, written only when changed', async () => {
const mods = require('../server/user-modules').createUserModules({ db: h.db, config: h.config || { networkInternalUrl: h.env.OV_NETWORK_INTERNAL_URL }, env: h.env, log: {} });
const mods = require('../server/user-modules').createUserModules({ db: h.db, config: h.config || { networkInternalUrl: h.env.OV_NETWORK_INTERNAL_URL }, env: h.env, log: {} });
mods.ensureSchema();
mods.ensureSchema();
- const s = mods.summarize(A.id);
+ const s = await mods.summarize(A.id);
assert.strictEqual(s.runs_30d, 2); assert.strictEqual(s.tokens_30d, 80);
assert.strictEqual(s.runs_30d, 2); assert.strictEqual(s.tokens_30d, 80);
assert.deepStrictEqual(s.by_service, { live: 2 });
assert.deepStrictEqual(s.by_service, { live: 2 });
assert.ok(await mods.push(A.id));
assert.ok(await mods.push(A.id));
@@ -87,7 +87,7 @@ t.test('ai.usage_summary: 30-day runs and tokens per person, written only when c
@@ -87,7 +87,7 @@ t.test('ai.usage_summary: 30-day runs and tokens per person, written only when c
assert.strictEqual(await mods.push(A.id), false, 'unchanged: not written again');
assert.strictEqual(await mods.push(A.id), false, 'unchanged: not written again');
assert.strictEqual(await mods.scan(Date.now() + 1000), 2, 'the scan sees A and B (their runs were just made)');
assert.strictEqual(await mods.scan(Date.now() + 1000), 2, 'the scan sees A and B (their runs were just made)');
assert.deepStrictEqual(puts.map((p) => p.subject).sort(), [A.id, B.id].sort());
assert.deepStrictEqual(puts.map((p) => p.subject).sort(), [A.id, B.id].sort());
- assert.strictEqual(mods.summarize('usr_01JAB2C3D4E5F6G7H8J9K0MNPC'), null);
+ assert.strictEqual(await mods.summarize('usr_01JAB2C3D4E5F6G7H8J9K0MNPC'), null);
});
});
t.test('shutdown', async () => { await h.stop(); await seam.close(); network.close(); });
t.test('shutdown', async () => { await h.stop(); await seam.close(); network.close(); });
@@ -10,7 +10,7 @@ const { boot, request, token, suite, ALL } = require('./helpers');
@@ -10,7 +10,7 @@ const { boot, request, token, suite, ALL } = require('./helpers');
const t = suite('versioning');
const t = suite('versioning');
const tok = token('live', ALL);
const tok = token('live', ALL);
let h;
let h;
-const run = (body) => request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body });
+const run = async (body) => await request(h.base, 'POST', '/api/v1/runs?wait=5000', { tok, body });
t.test('boot', async () => { h = await boot(); });
t.test('boot', async () => { h = await boot(); });
@@ -34,7 +34,7 @@ t.test('an invalid edit is refused and creates no version', async () => {
@@ -34,7 +34,7 @@ t.test('an invalid edit is refused and creates no version', async () => {
assert.strictEqual(bad.status, 422);
assert.strictEqual(bad.status, 422);
const bad2 = await request(h.base, 'POST', '/api/v1/workflows/ai.summarize/versions', { tok, body: { steps: [{ kind: 'llm', template: 'no.such.template' }] } });
const bad2 = await request(h.base, 'POST', '/api/v1/workflows/ai.summarize/versions', { tok, body: { steps: [{ kind: 'llm', template: 'no.such.template' }] } });
assert.strictEqual(bad2.status, 422);
assert.strictEqual(bad2.status, 422);
- assert.strictEqual(h.registry.getWorkflow('ai.summarize').version, 1);
+ assert.strictEqual((await h.registry.getWorkflow('ai.summarize')).version, 1);
});
});
t.test('deprecating the newest workflow version sends new runs to the newest active one', async () => {
t.test('deprecating the newest workflow version sends new runs to the newest active one', async () => {
@@ -72,11 +72,11 @@ t.test('historical route keys are explicit aliases of default.json', async () =>
@@ -72,11 +72,11 @@ t.test('historical route keys are explicit aliases of default.json', async () =>
});
});
t.test('seed: a code change becomes a new version, unless an admin versioned the key', async () => {
t.test('seed: a code change becomes a new version, unless an admin versioned the key', async () => {
- const before = h.registry.getTemplate('ai.generate').version;
- h.registry.seedVersioned('template', 'ai.generate', { name: 'Generate text', system_prompt: '{{system}}', user_prompt: 'CHANGED {{prompt}}', default_route: 'default.chat', input_schema: { type: 'object' }, output_schema: { type: 'object' } });
- assert.strictEqual(h.registry.getTemplate('ai.generate').version, before + 1);
- assert.strictEqual(h.registry.getTemplate('ai.generate').created_by, 'seed');
- const again = h.registry.seedVersioned('template', 'ai.summarize', { name: 'x', user_prompt: 'seed wants this' });
+ const before = (await h.registry.getTemplate('ai.generate')).version;
+ await h.registry.seedVersioned('template', 'ai.generate', { name: 'Generate text', system_prompt: '{{system}}', user_prompt: 'CHANGED {{prompt}}', default_route: 'default.chat', input_schema: { type: 'object' }, output_schema: { type: 'object' } });
+ assert.strictEqual((await h.registry.getTemplate('ai.generate')).version, before + 1);
+ assert.strictEqual((await h.registry.getTemplate('ai.generate')).created_by, 'seed');
+ const again = await h.registry.seedVersioned('template', 'ai.summarize', { name: 'x', user_prompt: 'seed wants this' });
assert.strictEqual(again.created_by, 'svc:live', 'admin version kept');
assert.strictEqual(again.created_by, 'svc:live', 'admin version kept');
});
});
@@ -82,19 +82,19 @@ t.test('boot seeds every required workflow key', async () => {
@@ -82,19 +82,19 @@ t.test('boot seeds every required workflow key', async () => {
EXAMPLES['live.media.transcribe'].media_url = `${mediaBase}/v/1`;
EXAMPLES['live.media.transcribe'].media_url = `${mediaBase}/v/1`;
EXAMPLES['media.analyze'].media_url = `${mediaBase}/fixture.mp4`;
EXAMPLES['media.analyze'].media_url = `${mediaBase}/fixture.mp4`;
h = await boot({ env: { OV_MEDIA_INTERNAL_URL: mediaBase } });
h = await boot({ env: { OV_MEDIA_INTERNAL_URL: mediaBase } });
- const keys = h.registry.listWorkflows().map(w => w.key);
+ const keys = (await h.registry.listWorkflows()).map(w => w.key);
for (const k of ['wiki.generate_space', 'wiki.generate_page', 'blog.draft_post', 'news.summarize_story', 'news.compare_perspectives', 'reviews.summarize_entity', 'deals.enrich_deal',
for (const k of ['wiki.generate_space', 'wiki.generate_page', 'blog.draft_post', 'news.summarize_story', 'news.compare_perspectives', 'reviews.summarize_entity', 'deals.enrich_deal',
'coupons.extract_coupon', 'trade.summarize_market_context', 'codes.generate_docs', 'games.generate_lore', 'moderation.classify', 'tools.describe',
'coupons.extract_coupon', 'trade.summarize_market_context', 'codes.generate_docs', 'games.generate_lore', 'moderation.classify', 'tools.describe',
'live.translate', 'live.stream.describe_frame', 'live.stream.summarize', 'live.viewers.reply', 'live.chat.insight', 'live.hero.slogans', 'live.stream.recap', 'network.site_copy']) {
'live.translate', 'live.stream.describe_frame', 'live.stream.summarize', 'live.viewers.reply', 'live.chat.insight', 'live.hero.slogans', 'live.stream.recap', 'network.site_copy']) {
assert.ok(keys.includes(k), `missing ${k}`);
assert.ok(keys.includes(k), `missing ${k}`);
}
}
for (const k of ['default.chat', 'default.json', 'default.embedding', 'wiki.generate', 'blog.draft', 'news.summarize', 'reviews.summarize', 'deals.enrich', 'coupons.extract', 'trade.summarize', 'codes.generate_docs', 'games.generate_lore', 'tools.describe', 'moderation.classify']) {
for (const k of ['default.chat', 'default.json', 'default.embedding', 'wiki.generate', 'blog.draft', 'news.summarize', 'reviews.summarize', 'deals.enrich', 'coupons.extract', 'trade.summarize', 'codes.generate_docs', 'games.generate_lore', 'tools.describe', 'moderation.classify']) {
- assert.ok(h.registry.getRoute(k), `missing historical route ${k}`);
+ assert.ok(await h.registry.getRoute(k), `missing historical route ${k}`);
}
}
});
});
t.test('every workflow has compiling, versioned input and output schemas and an example', async () => {
t.test('every workflow has compiling, versioned input and output schemas and an example', async () => {
- for (const w of h.registry.listWorkflows()) {
+ for (const w of await h.registry.listWorkflows()) {
assert.strictEqual(w.version, 1);
assert.strictEqual(w.version, 1);
schemas.assertSchema(w.input_schema, `${w.key} input`);
schemas.assertSchema(w.input_schema, `${w.key} input`);
schemas.assertSchema(w.output_schema, `${w.key} output`);
schemas.assertSchema(w.output_schema, `${w.key} output`);
@@ -105,7 +105,7 @@ t.test('every workflow has compiling, versioned input and output schemas and an
@@ -105,7 +105,7 @@ t.test('every workflow has compiling, versioned input and output schemas and an
});
});
t.test('every workflow runs end to end on the stub and returns schema-valid output', async () => {
t.test('every workflow runs end to end on the stub and returns schema-valid output', async () => {
- for (const w of h.registry.listWorkflows()) {
+ for (const w of await h.registry.listWorkflows()) {
if (w.key === 'media.analyze' && !HAS_FFMPEG) { console.log('workflow media.analyze end to end: skipped (no ffprobe)'); continue; }
if (w.key === 'media.analyze' && !HAS_FFMPEG) { console.log('workflow media.analyze end to end: skipped (no ffprobe)'); continue; }
const r = await request(h.base, 'POST', '/api/v1/runs?wait=10000', { tok, body: { workflow: w.key, input: EXAMPLES[w.key] } });
const r = await request(h.base, 'POST', '/api/v1/runs?wait=10000', { tok, body: { workflow: w.key, input: EXAMPLES[w.key] } });
assert.strictEqual(r.status, 201, `${w.key}: ${r.status} ${r.text.slice(0, 300)}`);
assert.strictEqual(r.status, 201, `${w.key}: ${r.status} ${r.text.slice(0, 300)}`);