openvibers/openvibe.network Commits
Project usage (roadmap WS-N task 4; Contracts v0.63.0 common.usage-recorded@1, network.project-usage-result@1): the Events consumer (POST /internal/events) now also takes tools.usage.recorded and events.usage.recorded, the owning services' hourly rollups of each developer project's use, and records them inside its inbox transaction (server/developer/usage.js): dev_usage_windows keeps one row per rollup key (service, project, env, capability, dimension, unit, window start; a newer or equal revision replaces it, so a re-sent hour never counts twice; 35 days), dev_usage_daily adds each day up again from its windows (400 days), dev_usage_errors keeps the sampled failures (30 days, 200 per project). Rollups for an unknown project, from the wrong source, with a subject other than their project, a window that is not a whole hour or day, from the future, older than 35 days or failing their contract are ignored, and none notifies anyone. GET /api/v1/projects/:project/usage?days=1-90&env=all|sandbox|production answers the owner, admins and staff (403 developers and viewers, 404 strangers, 422 usage.invalid) with network.project-usage-result@1: daily rows, range totals, each recorded quota (dev_quotas) with used and remaining for its current window (today, this month, the kept total; null with a note for minute and hour windows and for units no service reports) and errors by code with the 50 most recent sampled failures and their trace ids. scripts/subscribe-events.js subscribes the two topics (--topic tools.usage.recorded --topic events.usage.recorded). Pins: openvibe-contracts 0.62.0 -> 0.63.0. test/project-usage.test.js; docs/developer-projects.md#usage
Changes
@@@ -18,7 +18,7 @@ Identity and account service for the OpenVibe network. Manages user accounts, OA- **Theme Catalog** — Shared theme system with ~30 built-in themes and community submissions. Theme preferences sync across all services.- **Admin Panel** — email configuration, user management (role changes, bans), broadcast notifications, system health dashboard, and audit log.- **Internal API** — Server-to-server endpoints for token verification, user lookup, notification push, account linking, and audit logging. Protected by `X-Internal-Key`; the routes listed in [docs/retirement.md](docs/retirement.md#routes-that-take-a-token-network-side-of-step-2) also take a capability-scoped service token, and the key is retired route by route.--- **Developer projects (foundation)** — projects, members, apps (OAuth clients with `app_` subjects), hashed client secrets with rotation and revocation, capability grants within a staff-set allowance (sandbox apps also get a default sandbox allowance of public Media, Events-app and Tools-job capabilities, so a new project works without staff), recorded quotas and an append-only audit at `/api/v1/projects`. Its events can be relayed to OpenVibe.Events through an outbox when `OV_EVENTS_INTERNAL_URL` is set. Bearer user tokens only; no portal UI yet (that is OpenVibe.Codes). See [docs/developer-projects.md](docs/developer-projects.md).++- **Developer projects (foundation)** — projects, members, apps (OAuth clients with `app_` subjects), hashed client secrets with rotation and revocation, capability grants within a staff-set allowance (sandbox apps also get a default sandbox allowance of public Media, Events-app and Tools-job capabilities, so a new project works without staff), recorded quotas, per-project usage from the services' hourly rollups (`GET /api/v1/projects/:project/usage`) and an append-only audit at `/api/v1/projects`. Its events can be relayed to OpenVibe.Events through an outbox when `OV_EVENTS_INTERNAL_URL` is set. Bearer user tokens only; no portal UI yet (that is OpenVibe.Codes). See [docs/developer-projects.md](docs/developer-projects.md).- **Public discovery** — `/.well-known/openvibe`, `/api/v1/registry/*` and `/contracts/*.json` (services, capabilities, contract schemas) answer any origin with `Access-Control-Allow-Origin: *`, preflight included, so a browser app can discover the platform directly. Every other route keeps the first-party CORS allow-list (`server/public-cors.js`).- **OpenCoins Wallet** — Network-wide currency. User balance/history at `/api/coins/*`; atomic credit/debit/transfer for services at `/internal/coins/*` with idempotency-key dedupe.- **User modules** — versioned per-person preferences and summaries in namespaces owned by services (openvibe-contracts `manifests/namespaces`); see [User modules](#user-modules).@@@ -219,6 +219,12 @@ announcement an hour, eight a day; `stream_live_cooldown_min`, `stream_live_dailLive's direct `POST /internal/events/stream-live` (`server/notifications/stream-live.js`), so while Livestill makes that call, whichever arrives first announces and the other is skipped.++**Project usage.** `tools.usage.recorded` and `events.usage.recorded` (hourly rollups of a developer++project's use, `common.usage-recorded@1`) are kept per project and day (`server/developer/usage.js`,++tables `dev_usage_windows`, `dev_usage_daily`, `dev_usage_errors`) for `GET /api/v1/projects/:project/usage`++and the project's usage page on openvibe.codes; they notify nobody. See++[docs/developer-projects.md](docs/developer-projects.md#usage).++Coupons publishes no watch event yet: its merchant watches never leave Coupons, and its `coupons.*`events name no person. Every other type is acknowledged and ignored. Deliveries are v2-signed only(`openvibe-sdk` `parseDelivery` with `requireV2`, ±300 s), and each `event_id` is handled once@@@ -237,7 +243,8 @@ The route is inert until the operator does both of these:`deals.watch.matched`, `trade.alert.triggered` and `live.stream.started`; secret = the first`NETWORK_EVENTS_SECRET`). Subscribe `live.stream.started` only once Live serves`GET /internal/followers`; on a host that already has the other two, add it alone with-- `--topic live.stream.started`:++ `--topic live.stream.started` (and the project usage rollups with++ `--topic tools.usage.recorded --topic events.usage.recorded`):```bashcd /opt/openvibe.network@@@ -599,6 +606,7 @@ Accessible to users with `role = 'admin'`. All endpoints under `/api/admin/`.| `dev_apps`, `dev_credentials`, `dev_auth_codes` | Developer apps, hashed client secrets, app authorization codes || `dev_grants`, `dev_quotas` | App capability grants; per-project quotas (enforced by the owning service) || `dev_audit` | Append-only developer audit; rows that are platform events carry an event envelope |++| `dev_usage_windows`, `dev_usage_daily`, `dev_usage_errors` | Developer projects' usage from the services' hourly rollups (35 days), per day (400 days), sampled failures (30 days) || `analytics_events`, `analytics_hourly`, `analytics_daily` | Request analytics (raw ≤ 30 days) and rollups, within ADR-021; see [Analytics](#analytics-adr-021) || `user_modules`, `user_module_revisions`, `user_module_retirements` | User-module records, the last revision issued per (subject, namespace), retired namespace owners || `network_event_outbox` | Network's events on their way to OpenVibe.Events (developer projects, user modules, blocks) |@@@ -133,7 +133,23 @@ This is ADR-014's acceptance rule: "a revoked credential fails everywhere within## Quotas--A quota says how much of one capability a project may use: for example, `media.object.upload` with limit 1073741824, window `total`, unit `bytes`. Network records quotas and shows them to members. **Network does not enforce them.** The service that owns the capability does (`enforced_by` in the response is that audience). Until an owning service reads project quotas (proposed capability `network.project.read`), a quota is a recorded limit and not a guarantee. No usage numbers are collected here.++A quota says how much of one capability a project may use: for example, `media.object.upload` with limit 1073741824, window `total`, unit `bytes`. Network records quotas and shows them to members. **Network does not enforce them.** The service that owns the capability does (`enforced_by` in the response is that audience). Until an owning service reads project quotas (proposed capability `network.project.read`), a quota is a recorded limit and not a guarantee. What a quota's current window has used is measured from the services' usage rollups ([Usage](#usage)).++++## Usage++++The services that own developer capabilities count each project's use per environment and UTC hour, and send one rollup per closed hour through OpenVibe.Events (roadmap WS-N task 4): `tools.usage.recorded` from each Tools satellite (jobs, per `tools.job.create` or `tools.tool.run` and job type or tool) and `events.usage.recorded` from Events (`events.app.publish` in `events`, `events.app.subscribe` in `deliveries`). The payload is `common.usage-recorded@1` (openvibe-contracts 0.63.0): project, environment, capability, dimension, unit, window, quantity, errors, errors by code and up to ten sampled failures (time, code, status, trace id, job or event id). A rollup never names who did the work.++++The Events consumer (`POST /internal/events`, [server/developer/usage.js](../server/developer/usage.js)) records each one inside its inbox transaction:++++- `dev_usage_windows`: one row per rollup key (service, project, environment, capability, dimension, unit, window start). A later rollup for the same key replaces it (the higher `revision` wins), so a re-sent hour is never counted twice. Kept 35 days.++- `dev_usage_daily`: the same key per UTC day, added up again from its windows on every change. Kept 400 days.++- `dev_usage_errors`: the sampled failures. Kept 30 days, at most 200 per project.++++A rollup for a project Network does not know, from the wrong source, whose subject is not its project, whose window is not a whole hour (or day), from the future or older than 35 days, or that fails its contract is ignored (the consumer's `outcome` says which).++++`GET /api/v1/projects/:project/usage?days=30&env=all` (owner and admins, and staff) answers `network.project-usage-result@1`: `daily` rows (newest first), `totals` for the range, `quotas` (each recorded quota with `used` and `remaining` for its current window: today or this month in UTC, or the 400 days kept for `total`, in every environment together; `null` with a `note` for a minute or hour window, which rollups cannot show, and for a capability or unit no service reports), and `errors` (`total`, `by_code`, and the 50 most recent sampled failures). `days` is 1 to 90; `env` is `all`, `sandbox` or `production`; anything else is `422 usage.invalid`. The numbers lag by one hour: an hour appears a few minutes after it closes. OpenVibe.Codes shows it as the project's usage page (`/projects/:project/usage`).++++Operator steps: subscribe Network to the two topics once (`node --env-file=/etc/openvibe/network.env scripts/subscribe-events.js --topic tools.usage.recorded --topic events.usage.recorded`), with Events' usage on (default) and Tools' job events on (`EVENTS_URL` set for the satellites).## API@@@ -175,6 +191,7 @@ Base: `/api/v1/projects`. Every call needs `Authorization: Bearer <Network user| `DELETE /:project/apps/:app/grants/:capability` | admin+, staff | revoke || `GET /:project/quotas` | viewer+ | || `PUT /:project/quotas/:capability` / `DELETE` | staff | `{ limit, window, unit }` |++| `GET /:project/usage[?days=&env=]` | admin+, staff | usage per day, totals, quotas with their use, errors ([Usage](#usage)) || `GET /:project/audit[?before=&limit=]` | admin+, staff | newest first, paged by `next_before` || `POST /:project/export-tokens` | owner, admin (not staff as such) | `{ audience, env }` → a 5-minute read-only export token ([Export tokens](#export-tokens)) |@@@ -20,7 +20,7 @@"helmet": "^8.1.0","jsonwebtoken": "^9.0.0","multer": "^1.4.5-lts.1",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.62.0",++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.63.0","openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0","openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.22.0","uuid": "^11.1.0",@@@ -1465,9 +1465,9 @@}},"node_modules/openvibe-contracts": {-- "version": "0.62.0",-- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.62.0",-- "integrity": "sha512-mmVQToHBY77NrZEHqOBoPuRdZ/bnAzp11E7VkBrN9maPdGCoG8YIJqIq/Hi/dFiDrLJtAcdXRKl/tahdWrkkdQ==",++ "version": "0.63.0",++ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.63.0",++ "integrity": "sha512-vmc1b1vWg9M5fm57X50MlZEZHkP2IshHwBXt68+kHDV85nyg6FxaYyUWa4ibi1eeFqzMcECPlILSRTOgg2Xbgw==","license": "AGPL-3.0-only","dependencies": {"ajv": "^8.17.1",@@@ -6,7 +6,7 @@"scripts": {"start": "node server/index.js","dev": "NODE_ENV=development node server/index.js",-- "test": "node test/registry-bootstrap.test.js && node test/nginx-generator-live.test.js && node test/admin-refresh.test.js && node test/history-sso.test.js && node test/sso-check.test.js && node test/fedcm.test.js && node test/domains.test.js && node test/frame.test.js && node test/github-integration.test.js && node test/revocation.test.js && node test/usernames.test.js && node test/avatar.test.js && node test/roadmap-baseline.test.js && node test/identity-subjects.test.js && node test/principals.test.js && node test/operator-alerts.test.js && node test/modules.test.js && node test/module-events.test.js && node test/registry-ecosystem.test.js && node test/registry-exposure.test.js && node test/registry-manifest-fixture.test.js && node test/staff-claims.test.js && node test/profile-events.test.js && node test/admin-role-events.test.js && node test/grants-admin.test.js && node test/events-ops.test.js && node test/deploy-drift.test.js && node test/registry-topics.test.js && node test/service-principal.test.js && node test/oauth-pkce.test.js && node test/developer-projects.test.js && node test/developer-defaults.test.js && node test/developer-export-tokens.test.js && node test/public-cors.test.js && node test/observability.test.js && node test/status.test.js && node test/events-consumer.test.js && node test/notification-events.test.js && node test/security-session.test.js && node test/security-redirects.test.js && node test/analytics-privacy.test.js && node test/data-ownership.test.js && node test/unknown-paths.test.js && node test/oidc-discovery.test.js && node test/refresh-tokens.test.js && node test/provider-secrets.test.js && node test/first-party-origins.test.js && node test/updates.test.js && node test/static-headers.test.js && node test/library-tags.test.js && node test/moderation-audit.test.js && node test/blocks.test.js && node test/site-config.test.js && node test/perf-budget.test.js && node test/shutdown.test.js && node test/loyalty-policy.test.js && node test/themes-review.test.js && node test/page-checks.test.js && node test/operator-checklist.test.js && node test/release-health.test.js",++ "test": "node test/registry-bootstrap.test.js && node test/nginx-generator-live.test.js && node test/admin-refresh.test.js && node test/history-sso.test.js && node test/sso-check.test.js && node test/fedcm.test.js && node test/domains.test.js && node test/frame.test.js && node test/github-integration.test.js && node test/revocation.test.js && node test/usernames.test.js && node test/avatar.test.js && node test/roadmap-baseline.test.js && node test/identity-subjects.test.js && node test/principals.test.js && node test/operator-alerts.test.js && node test/modules.test.js && node test/module-events.test.js && node test/registry-ecosystem.test.js && node test/registry-exposure.test.js && node test/registry-manifest-fixture.test.js && node test/staff-claims.test.js && node test/profile-events.test.js && node test/admin-role-events.test.js && node test/grants-admin.test.js && node test/events-ops.test.js && node test/deploy-drift.test.js && node test/registry-topics.test.js && node test/service-principal.test.js && node test/oauth-pkce.test.js && node test/developer-projects.test.js && node test/developer-defaults.test.js && node test/developer-export-tokens.test.js && node test/project-usage.test.js && node test/public-cors.test.js && node test/observability.test.js && node test/status.test.js && node test/events-consumer.test.js && node test/notification-events.test.js && node test/security-session.test.js && node test/security-redirects.test.js && node test/analytics-privacy.test.js && node test/data-ownership.test.js && node test/unknown-paths.test.js && node test/oidc-discovery.test.js && node test/refresh-tokens.test.js && node test/provider-secrets.test.js && node test/first-party-origins.test.js && node test/updates.test.js && node test/static-headers.test.js && node test/library-tags.test.js && node test/moderation-audit.test.js && node test/blocks.test.js && node test/site-config.test.js && node test/perf-budget.test.js && node test/shutdown.test.js && node test/loyalty-policy.test.js && node test/themes-review.test.js && node test/page-checks.test.js && node test/operator-checklist.test.js && node test/release-health.test.js","reset-db": "node server/reset-db.js","grant-admin": "node server/grant-admin.js"},@@@ -23,7 +23,7 @@"helmet": "^8.1.0","jsonwebtoken": "^9.0.0","multer": "^1.4.5-lts.1",-- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.62.0",++ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.63.0","openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0","openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.22.0","uuid": "^11.1.0",@@@ -3,14 +3,16 @@/*** Create Network's OpenVibe.Events subscriptions for the notification consumer* (server/notifications/events-consumer.js): one per topic in TOPICS (deals.watch.matched,-- * trade.alert.triggered, live.stream.started), all delivering to POST /internal/events. Events names-- * the consumer after the calling service: `network`.++ * trade.alert.triggered, live.stream.started, the moderation audit topics and the project usage++ * rollups tools.usage.recorded and events.usage.recorded), all delivering to POST /internal/events.++ * Events names the consumer after the calling service: `network`.** sudo node --env-file=/etc/openvibe/network.env scripts/subscribe-events.js \* [--endpoint http://127.0.0.1:4000/internal/events] [--topic live.stream.started] [--dry-run]** --topic limits the run to one of TOPICS (repeatable), e.g. to add live.stream.started to a host that-- * already has the other two. live.stream.started needs Live's GET /internal/followers first (the++ * already has the other two, or `--topic tools.usage.recorded --topic events.usage.recorded` for the++ * project dashboards (WS-N task 4). live.stream.started needs Live's GET /internal/followers first (the* consumer answers 503 until Live serves it, and Events retries), and the token Network signs for it* carries live.follower.read for audience openvibe.live.*@@@ -744,6 +744,8 @@ function initDb(dbPath) {require('../identity/modules').ensureSchema(db);// Developer projects, apps, credentials, grants, quotas and audit (server/developer/store.js).require('../developer/store').ensureSchema(db);++ // Their usage, from the services' rollups (server/developer/usage.js).++ require('../developer/usage').ensure(db);// Platform blocks, keyed by subjects (server/identity/blocks.js).require('../identity/blocks').ensureSchema(db);@@@ -35,6 +35,8 @@* GET /:project/quotas viewer+* PUT /:project/quotas/:capability { limit, window, unit } staff* DELETE /:project/quotas/:capability staff++ * GET /:project/usage[?days=30&env=all] owner/admin or staff: usage per day, quotas and++ * errors from the services' rollups (usage.js)* GET /:project/audit[?before=&limit=] admin+ or staff* POST /:project/export-tokens { audience, env } owner or admin member (not staff as such):* a 5-minute read-only export token (tokens.js)@@@ -45,6 +47,7 @@ const { verifySession } = require('../auth/session');const store = require('./store');const policy = require('./policy');const tokens = require('./tokens');++const usage = require('./usage');function router() {const r = express.Router();@@@ -118,6 +121,12 @@ function router() {r.put('/:project/quotas/:capability', handle((db, a, req, o) => store.setQuota(db, a, req.params.project, req.params.capability, req.body || {}, o)));r.delete('/:project/quotas/:capability', handle((db, a, req, o) => store.deleteQuota(db, a, req.params.project, req.params.capability, o)));++ // Usage (WS-N task 4): network.project-usage-result@1, for the owner and admins (and staff).++ r.get('/:project/usage', handle((db, a, req) => {++ const { project } = store.access(db, a, req.params.project, { need: 'admin', allowArchived: true });++ return usage.summary(db, project.id, usage.parseQuery(req.query));++ }));++r.get('/:project/audit', handle((db, a, req) => {const { project } = store.access(db, a, req.params.project, { need: 'admin', allowArchived: true });return store.listAudit(db, project.id, { before: req.query.before, limit: req.query.limit });@@@ -134,7 +143,7 @@ function router() {}function send(req, res, err) {-- if (err instanceof store.DevError) return http.sendProblem(res, err.status, err.code, { detail: err.detail, ctx: req.ov });++ if (err instanceof store.DevError || err instanceof usage.UsageQueryError) return http.sendProblem(res, err.status, err.code, { detail: err.detail, ctx: req.ov });if (err && err.type === 'entity.parse.failed') return http.sendProblem(res, 400, 'request.malformed_json', { detail: 'body is not valid JSON', ctx: req.ov });if (err && err.type === 'entity.too.large') return http.sendProblem(res, 413, 'request.too_large', { ctx: req.ov });// Never echo internals (and so never a secret) to the client.nnew file mode 100644@@@ -0,0 +1,278 @@++'use strict';++/**++ * Project usage (roadmap WS-N task 4, ADR-014): the rollups the owning services send of each developer++ * project's use (<service>.usage.recorded, openvibe-contracts common.usage-recorded@1), kept per++ * project and day for the project's dashboard on OpenVibe.Codes.++ *++ * tools.usage.recorded a Tools satellite's hour of a project's jobs (unit jobs)++ * events.usage.recorded OpenVibe.Events: published events (events) and webhook deliveries (deliveries)++ *++ * The Events consumer (server/notifications/events-consumer.js) calls record() inside its inbox++ * transaction, so a redelivery changes nothing. A rollup is the totals of its key (service, project,++ * env, capability, dimension, unit, window_start): a newer or equal revision replaces the stored one++ * and its day is added up again from its windows, so a producer re-sending an hour never counts it++ * twice. The failures a rollup samples (time, code, status, trace id, job or event id) are kept for++ * the dashboard's recent errors. Nothing here names who did the work.++ *++ * dev_usage_windows one row per rollup key (WINDOW_DAYS, 35 days)++ * dev_usage_daily per project, env, service, capability, dimension, unit and UTC day (DAILY_DAYS, 400 days)++ * dev_usage_errors sampled failures (ERROR_DAYS, 30 days; the newest ERRORS_PER_PROJECT per project)++ *++ * summary() is what GET /api/v1/projects/:project/usage (./routes.js) answers the project's owner, its++ * admins and staff: network.project-usage-result@1, with the recorded quotas (dev_quotas) and what++ * their current window used where the rollups can tell (a day, a month, the kept total).++ */++const { validate, capabilities } = require('openvibe-contracts');++++const SERVICES = Object.freeze(['tools', 'events']);++const TOPICS = Object.freeze(SERVICES.map((s) => `${s}.usage.recorded`));++const HOUR_MS = 60 * 60 * 1000;++const DAY_MS = 24 * HOUR_MS;++const WINDOW_DAYS = 35;++const DAILY_DAYS = 400;++const ERROR_DAYS = 30;++const ERRORS_PER_PROJECT = 200;++const MAX_DAYS = 90;++const RECENT_ERRORS = 50;++// What each service reports, so a quota on one of these is measured even before its first rollup.++const REPORTED = Object.freeze({++ 'tools.job.create': ['jobs'], 'tools.tool.run': ['jobs'],++ 'events.app.publish': ['events'], 'events.app.subscribe': ['deliveries'],++});++++const iso = (ms) => new Date(ms).toISOString();++const dayOf = (ms) => iso(ms).slice(0, 10);++const parse = (s, d) => { try { return JSON.parse(s); } catch { return d; } };++const obj = (v) => (v && typeof v === 'object' && !Array.isArray(v) ? v : {});++++function ensure(db) {++ db.exec(`++ CREATE TABLE IF NOT EXISTS dev_usage_windows (++ project_id TEXT NOT NULL,++ env TEXT NOT NULL CHECK (env IN ('sandbox', 'production')),++ service TEXT NOT NULL,++ capability TEXT NOT NULL,++ dimension TEXT NOT NULL DEFAULT '',++ unit TEXT NOT NULL,++ window_start TEXT NOT NULL,++ window_end TEXT NOT NULL,++ quantity INTEGER NOT NULL,++ errors INTEGER NOT NULL,++ error_codes TEXT NOT NULL DEFAULT '{}',++ revision INTEGER NOT NULL DEFAULT 1,++ event_id TEXT NOT NULL,++ recorded_at TEXT NOT NULL,++ PRIMARY KEY (project_id, env, service, capability, dimension, unit, window_start)++ );++ CREATE INDEX IF NOT EXISTS idx_dev_usage_windows_when ON dev_usage_windows(window_start);++ CREATE INDEX IF NOT EXISTS idx_dev_usage_windows_project ON dev_usage_windows(project_id, recorded_at);++ CREATE TABLE IF NOT EXISTS dev_usage_daily (++ project_id TEXT NOT NULL,++ env TEXT NOT NULL,++ service TEXT NOT NULL,++ capability TEXT NOT NULL,++ dimension TEXT NOT NULL DEFAULT '',++ unit TEXT NOT NULL,++ day TEXT NOT NULL,++ quantity INTEGER NOT NULL DEFAULT 0,++ errors INTEGER NOT NULL DEFAULT 0,++ error_codes TEXT NOT NULL DEFAULT '{}',++ updated_at TEXT NOT NULL,++ PRIMARY KEY (project_id, env, service, capability, dimension, unit, day)++ );++ CREATE INDEX IF NOT EXISTS idx_dev_usage_daily_day ON dev_usage_daily(project_id, day);++ CREATE TABLE IF NOT EXISTS dev_usage_errors (++ id INTEGER PRIMARY KEY AUTOINCREMENT,++ project_id TEXT NOT NULL,++ env TEXT NOT NULL,++ service TEXT NOT NULL,++ capability TEXT NOT NULL,++ at TEXT NOT NULL,++ code TEXT NOT NULL,++ status INTEGER,++ trace_id TEXT,++ ref TEXT NOT NULL DEFAULT '',++ event_id TEXT NOT NULL++ );++ CREATE UNIQUE INDEX IF NOT EXISTS idx_dev_usage_errors_once ON dev_usage_errors(project_id, env, service, capability, at, code, ref);++ CREATE INDEX IF NOT EXISTS idx_dev_usage_errors_recent ON dev_usage_errors(project_id, at);++ `);++}++++function mergeCodes(into, codes) {++ for (const [k, n] of Object.entries(obj(codes))) if (Number.isSafeInteger(n) && n > 0) into[k] = (into[k] || 0) + n;++ return into;++}++++/**++ * @param {import('better-sqlite3').Database} db++ * @param {{ now?: () => number, log?: object }} [o]++ */++function createProjectUsage(db, { now = () => Date.now(), log = console } = {}) {++ ensure(db);++ const q = {++ project: db.prepare('SELECT id FROM dev_projects WHERE id = ?'),++ window: db.prepare(`SELECT revision FROM dev_usage_windows WHERE project_id = ? AND env = ? AND service = ? AND capability = ?++ AND dimension = ? AND unit = ? AND window_start = ?`),++ upsertWindow: db.prepare(`INSERT INTO dev_usage_windows (project_id, env, service, capability, dimension, unit, window_start, window_end,++ quantity, errors, error_codes, revision, event_id, recorded_at)++ VALUES (@project_id, @env, @service, @capability, @dimension, @unit, @window_start, @window_end, @quantity, @errors, @error_codes, @revision, @event_id, @recorded_at)++ ON CONFLICT(project_id, env, service, capability, dimension, unit, window_start) DO UPDATE SET++ window_end = excluded.window_end, quantity = excluded.quantity, errors = excluded.errors, error_codes = excluded.error_codes,++ revision = excluded.revision, event_id = excluded.event_id, recorded_at = excluded.recorded_at`),++ dayWindows: db.prepare(`SELECT quantity, errors, error_codes FROM dev_usage_windows WHERE project_id = ? AND env = ? AND service = ?++ AND capability = ? AND dimension = ? AND unit = ? AND window_start >= ? AND window_start < ?`),++ upsertDaily: db.prepare(`INSERT INTO dev_usage_daily (project_id, env, service, capability, dimension, unit, day, quantity, errors, error_codes, updated_at)++ VALUES (@project_id, @env, @service, @capability, @dimension, @unit, @day, @quantity, @errors, @error_codes, @updated_at)++ ON CONFLICT(project_id, env, service, capability, dimension, unit, day) DO UPDATE SET++ quantity = excluded.quantity, errors = excluded.errors, error_codes = excluded.error_codes, updated_at = excluded.updated_at`),++ addError: db.prepare(`INSERT OR IGNORE INTO dev_usage_errors (project_id, env, service, capability, at, code, status, trace_id, ref, event_id)++ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`),++ capErrors: db.prepare(`DELETE FROM dev_usage_errors WHERE project_id = ? AND id NOT IN++ (SELECT id FROM dev_usage_errors WHERE project_id = ? ORDER BY at DESC, id DESC LIMIT ${ERRORS_PER_PROJECT})`),++ };++ let lastPrune = 0;++++ /** Inside the consumer's inbox transaction: 'recorded' or an 'ignored:*' outcome. */++ function record(event) {++ const type = String(event && event.event_type || '');++ if (!TOPICS.includes(type)) return 'ignored:type';++ const service = type.split('.')[0];++ if (event.source !== service) return 'ignored:source';++ const p = obj(event.payload);++ if (p.redacted === true) return 'ignored:redacted';++ const v = validate(`${type}@${event.version || 1}`, p);++ if (!v.valid) {++ log.warn(`[Usage] ${event.event_id} (${type}) does not match its contract: ${v.errors.slice(0, 3).map((e) => `${e.path} ${e.message}`).join('; ')}`);++ return 'ignored:payload';++ }++ const subject = obj(event.subject);++ if (subject.type !== 'project' || subject.id !== p.project_id) return 'ignored:subject';++ const start = Date.parse(p.window_start);++ const end = Date.parse(p.window_end);++ const len = p.window === 'day' ? DAY_MS : HOUR_MS;++ if (end - start !== len || start % len !== 0) return 'ignored:window';++ const t = now();++ if (start > t + HOUR_MS) return 'ignored:future';++ if (start < t - WINDOW_DAYS * DAY_MS) return 'ignored:too_old';++ if (!q.project.get(p.project_id)) return 'ignored:project';++ const key = { project_id: p.project_id, env: p.env, service, capability: p.capability, dimension: p.dimension || '', unit: p.unit };++ const windowStart = iso(start);++ const revision = p.revision || 1;++ const prev = q.window.get(key.project_id, key.env, key.service, key.capability, key.dimension, key.unit, windowStart);++ if (prev && prev.revision > revision) return 'ignored:stale';++ const recordedAt = iso(t);++ q.upsertWindow.run({++ ...key, window_start: windowStart, window_end: iso(end), quantity: p.quantity, errors: p.errors,++ error_codes: JSON.stringify(obj(p.error_codes)), revision, event_id: event.event_id, recorded_at: recordedAt,++ });++ // The day this window falls in, added up again from its windows (a day window is its own day).++ const day = dayOf(start);++ const from = `${day}T00:00:00.000Z`;++ const to = iso(Date.parse(from) + DAY_MS);++ const sum = { quantity: 0, errors: 0, codes: {} };++ for (const w of q.dayWindows.all(key.project_id, key.env, key.service, key.capability, key.dimension, key.unit, from, to)) {++ sum.quantity += w.quantity; sum.errors += w.errors; mergeCodes(sum.codes, parse(w.error_codes, {}));++ }++ q.upsertDaily.run({ ...key, day, quantity: sum.quantity, errors: sum.errors, error_codes: JSON.stringify(sum.codes), updated_at: recordedAt });++ for (const s of Array.isArray(p.samples) ? p.samples : []) {++ q.addError.run(key.project_id, key.env, service, key.capability, iso(Date.parse(s.at)), s.code, s.status || null, s.trace_id || null, s.ref || '', event.event_id);++ }++ if (p.samples && p.samples.length) q.capErrors.run(key.project_id, key.project_id);++ if (t - lastPrune > HOUR_MS) { lastPrune = t; prune(t); }++ return 'recorded';++ }++++ function prune(t = now()) {++ db.prepare('DELETE FROM dev_usage_windows WHERE window_start < ?').run(iso(t - WINDOW_DAYS * DAY_MS));++ db.prepare('DELETE FROM dev_usage_daily WHERE day < ?').run(dayOf(t - DAILY_DAYS * DAY_MS));++ db.prepare('DELETE FROM dev_usage_errors WHERE at < ?').run(iso(t - ERROR_DAYS * DAY_MS));++ }++++ return { record, prune, summary: (projectId, o) => summary(db, projectId, { now, ...o }) };++}++++class UsageQueryError extends Error {++ constructor(detail) { super(detail); this.status = 422; this.code = 'usage.invalid'; this.detail = detail; }++}++++/** ?days=&env= → { days, env }; throws UsageQueryError for anything else. */++function parseQuery(query = {}) {++ const rawDays = query.days == null || query.days === '' ? '30' : String(query.days);++ if (!/^\d{1,3}$/.test(rawDays) || Number(rawDays) < 1 || Number(rawDays) > MAX_DAYS) throw new UsageQueryError(`days is a whole number from 1 to ${MAX_DAYS}`);++ const env = query.env == null || query.env === '' ? 'all' : String(query.env);++ if (!['all', 'sandbox', 'production'].includes(env)) throw new UsageQueryError('env is all, sandbox or production');++ return { days: Number(rawDays), env };++}++++/** network.project-usage-result@1 for one project (the caller checked access). */++function summary(db, projectId, { days = 30, env = 'all', now = () => Date.now() } = {}) {++ const t = now();++ const to = dayOf(t);++ const from = dayOf(t - (days - 1) * DAY_MS);++ const envSql = env === 'all' ? '' : ' AND env = ?';++ const envArgs = env === 'all' ? [] : [env];++ const rows = db.prepare(`SELECT * FROM dev_usage_daily WHERE project_id = ? AND day >= ?${envSql}++ ORDER BY day DESC, service, capability, dimension, unit, env`).all(projectId, from, ...envArgs);++++ const daily = rows.map((r) => ({ day: r.day, service: r.service, capability: r.capability, dimension: r.dimension || null, unit: r.unit, env: r.env, quantity: r.quantity, errors: r.errors }));++ const totals = new Map();++ const byCode = new Map();++ let errorTotal = 0;++ for (const r of rows) {++ const k = `${r.service}|${r.capability}|${r.unit}|${r.env}`;++ const tot = totals.get(k) || { service: r.service, capability: r.capability, unit: r.unit, env: r.env, quantity: 0, errors: 0 };++ tot.quantity += r.quantity; tot.errors += r.errors;++ totals.set(k, tot);++ errorTotal += r.errors;++ for (const [code, n] of Object.entries(parse(r.error_codes, {}))) {++ const ck = `${r.service}|${r.capability}|${code}`;++ const c = byCode.get(ck) || { service: r.service, capability: r.capability, code, count: 0 };++ c.count += n;++ byCode.set(ck, c);++ }++ }++++ // Quotas (dev_quotas, staff-set): what their current window used, every environment together.++ const seen = new Set(db.prepare('SELECT DISTINCT capability, unit FROM dev_usage_daily WHERE project_id = ?').all(projectId).map((r) => `${r.capability}|${r.unit}`));++ const usedSince = db.prepare('SELECT COALESCE(SUM(quantity), 0) AS n FROM dev_usage_daily WHERE project_id = ? AND capability = ? AND unit = ? AND day >= ?');++ const quotas = db.prepare('SELECT * FROM dev_quotas WHERE project_id = ? ORDER BY capability').all(projectId).map((qr) => {++ const cap = capabilities.get(qr.capability);++ const out = { capability: qr.capability, limit: qr.limit_value, window: qr.quota_window, unit: qr.unit, enforced_by: cap ? `openvibe.${cap.owner}` : null, used: null, remaining: null, window_start: null, note: null };++ const measured = seen.has(`${qr.capability}|${qr.unit}`) || (REPORTED[qr.capability] || []).includes(qr.unit);++ if (!measured) {++ const units = REPORTED[qr.capability];++ out.note = units ? `the service reports ${units.join(', ')} for this capability, not ${qr.unit}` : 'no service reports usage of this capability yet';++ return out;++ }++ if (qr.quota_window === 'minute' || qr.quota_window === 'hour') {++ out.note = `a ${qr.quota_window} window is enforced by the service as it happens; hourly rollups cannot show it`;++ return out;++ }++ let since;++ if (qr.quota_window === 'day') { since = to; out.window_start = `${to}T00:00:00.000Z`; }++ else if (qr.quota_window === 'month') { since = `${to.slice(0, 7)}-01`; out.window_start = `${since}T00:00:00.000Z`; }++ else { since = '0000-00-00'; out.note = `total of the last ${DAILY_DAYS} days kept`; }++ out.used = usedSince.get(projectId, qr.capability, qr.unit, since).n;++ out.remaining = Math.max(0, qr.limit_value - out.used);++ return out;++ });++++ const recent = db.prepare(`SELECT * FROM dev_usage_errors WHERE project_id = ? AND at >= ?${envSql} ORDER BY at DESC, id DESC LIMIT ${RECENT_ERRORS}`)++ .all(projectId, `${from}T00:00:00.000Z`, ...envArgs)++ .map((e) => ({ at: e.at, env: e.env, service: e.service, capability: e.capability, code: e.code, status: e.status == null ? null : e.status, trace_id: e.trace_id || null, ref: e.ref || null }));++ const last = db.prepare('SELECT MAX(recorded_at) AS at FROM dev_usage_windows WHERE project_id = ?').get(projectId).at || null;++++ return {++ project_id: projectId, env, range: { days, from, to }, generated_at: iso(t), last_recorded_at: last,++ freshness: last++ ? 'Usage arrives as hourly rollups a few minutes after each hour closes (UTC); the current hour is not counted yet.'++ : 'No usage has been recorded for this project yet. Rollups arrive a few minutes after each hour (UTC) closes.',++ totals: [...totals.values()].sort((a, b) => a.service.localeCompare(b.service) || a.capability.localeCompare(b.capability) || a.env.localeCompare(b.env)),++ daily,++ quotas,++ errors: { total: errorTotal, by_code: [...byCode.values()].sort((a, b) => b.count - a.count || a.code.localeCompare(b.code)), recent },++ };++}++++module.exports = { createProjectUsage, summary, parseQuery, ensure, UsageQueryError, TOPICS, SERVICES, MAX_DAYS };@@@ -371,9 +371,11 @@ const liveFollowers = privateKey.includes('BEGIN'): null;// The moderation audit log (ADR-022): staff actions every service reports, readable by staff.const moderationAudit = require('./admin/moderation-audit').createModerationAudit(db);++// Developer projects' usage (WS-N task 4): the services' hourly rollups, per project and day.++const projectUsage = require('./developer/usage').createProjectUsage(db);eventsConsumer = require('./notifications/events-consumer').createEventsConsumer({db, notifications: notificationService, secrets: config.eventsWebhookSecrets,-- liveFollowers, discord: () => app.locals.discordService || null, moderationAudit,++ liveFollowers, discord: () => app.locals.discordService || null, moderationAudit, projectUsage,});console.log(`[Events consumer] ${eventsConsumer.enabled ? 'on' : 'off (NETWORK_EVENTS_SECRET unset)'}: POST /internal/events`);@@@ -20,6 +20,10 @@* A started event older than LIVE_STARTED_MAX_AGE (30 min; a replay, or Events catching up after an* outage) announces nothing: the stream may be long over.*++ * <service>.usage.recorded a developer project's hourly usage rollup from Tools or Events++ * (common.usage-recorded@1): kept per project and day for its dashboard++ * (server/developer/usage.js), never anyone's notification.++ ** Coupons publishes no watch event: its merchant watches never leave Coupons (server/domain/watches.js* there: "Delivery is not built yet"), and its coupons.* events name no person, so there is nothing* of Coupons' to consume yet. README "Notifications from Events" has the operator steps.@@@ -50,7 +54,8 @@ const { LiveFollowersError } = require('./live-followers');const CONSUMER = 'network-notifications';const INBOX_TABLE = 'network_event_inbox';const AUDIT_TOPICS = require('../admin/moderation-audit').TOPICS;--const TOPICS = Object.freeze(['deals.watch.matched', 'trade.alert.triggered', 'live.stream.started', ...AUDIT_TOPICS]);++const USAGE_TOPICS = require('../developer/usage').TOPICS;++const TOPICS = Object.freeze(['deals.watch.matched', 'trade.alert.triggered', 'live.stream.started', ...AUDIT_TOPICS, ...USAGE_TOPICS]);const LIVE_STARTED_MAX_AGE_MS = 30 * 60 * 1000;const EVENT_ID_RE = /^evt_[0-9A-HJKMNP-TV-Z]{26}$/;@@@ -155,8 +160,9 @@ function liveStarted(event, { now, maxAgeMs }) {* @param {string|string[]} o.secrets NETWORK_EVENTS_SECRET (comma list) or an array* @param {{ forStream(id: number): Promise<object> }} [o.liveFollowers] ./live-followers.js* @param {() => ({ sendLiveAlert(streamer: object, stream: object): Promise<object> }|null)} [o.discord]++ * @param {{ record(event: object): string }} [o.projectUsage] ../developer/usage.js createProjectUsage()*/--function createEventsConsumer({ db, notifications, secrets, liveFollowers = null, discord = () => null, moderationAudit = null, liveStartedMaxAgeMs = LIVE_STARTED_MAX_AGE_MS, now = () => Date.now(), log = console }) {++function createEventsConsumer({ db, notifications, secrets, liveFollowers = null, discord = () => null, moderationAudit = null, projectUsage = null, liveStartedMaxAgeMs = LIVE_STARTED_MAX_AGE_MS, now = () => Date.now(), log = console }) {const keys = Array.isArray(secrets) ? secrets.filter((s) => typeof s === 'string' && s.length >= 32) : secretsFrom(secrets);const inbox = createInbox(db, { table: INBOX_TABLE, now });inbox.ensureSchema();@@@ -222,6 +228,8 @@ function createEventsConsumer({ db, notifications, secrets, liveFollowers = nullconst r = inbox.once(CONSUMER, event.event_id, () => {// Staff actions go to the moderation audit log (ADR-022), never to anyone's inbox.if (AUDIT_TOPICS.includes(event.event_type)) return moderationAudit ? moderationAudit.record(event) : 'ignored:audit_off';++ // Developer projects' usage rollups go to their dashboards (WS-N task 4), never to an inbox.++ if (USAGE_TOPICS.includes(event.event_type)) return projectUsage ? projectUsage.record(event) : 'ignored:usage_off';if (event.event_type === 'live.stream.started') {const out = liveStreamStarted(event, prep);if (typeof out === 'string') return out;@@@ -59,7 +59,8 @@ const rows = (userId) => db.prepare('SELECT * FROM notifications WHERE user_id =(async () => {// The subscriptions this consumer is meant for, and nothing else.assert.deepStrictEqual(TOPICS, ['deals.watch.matched', 'trade.alert.triggered', 'live.stream.started', 'chat.moderation.action', 'live.moderation.action', 'community.moderation.action', 'tips.interaction.moderated', 'billing.staff.action',-- ...['media', 'tools', 'games', 'wiki', 'blog', 'news', 'reviews', 'deals', 'coupons', 'trade', 'codes'].map((svc) => `${svc}.moderation.action`)]);++ ...['media', 'tools', 'games', 'wiki', 'blog', 'news', 'reviews', 'deals', 'coupons', 'trade', 'codes'].map((svc) => `${svc}.moderation.action`),++ 'tools.usage.recorded', 'events.usage.recorded']);assert.deepStrictEqual(topicsFrom([]), TOPICS, 'scripts/subscribe-events.js subscribes every topic');assert.deepStrictEqual(topicsFrom(['--topic', 'live.stream.started']), ['live.stream.started']);assert.throws(() => topicsFrom(['--topic', 'live.*']));nnew file mode 100644@@@ -0,0 +1,185 @@++'use strict';++// Project usage (WS-N task 4, server/developer/usage.js): tools.usage.recorded and events.usage.recorded++// arrive at POST /internal/events, are kept per project and day inside the consumer's inbox transaction++// (a redelivery or a re-sent hour never counts twice; an older revision never replaces a newer one), and++// GET /api/v1/projects/:project/usage answers the owner, admins and staff (not developers, viewers or++// strangers) with network.project-usage-result@1: daily rows, totals, the recorded quotas with what++// their window used, errors by code and the sampled failures with their trace ids.++// node test/project-usage.test.js++const assert = require('assert');++const crypto = require('crypto');++const fs = require('fs');++const os = require('os');++const path = require('path');++const http = require('http');++const express = require('express');++const jwt = require('jsonwebtoken');++const { ids, validate } = require('openvibe-contracts');++const { signDeliveryHeaders } = require('openvibe-sdk/events');++const { initDb } = require('../server/db/database');++const subjects = require('../server/identity/subjects');++const { NotificationService } = require('../server/notifications/notification-service');++const { createEventsConsumer, TOPICS } = require('../server/notifications/events-consumer');++const { createProjectUsage, parseQuery } = require('../server/developer/usage');++const { topicsFrom } = require('../scripts/subscribe-events');++++const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-project-usage-'));++const quiet = { log: console.log, warn: console.warn };++console.log = () => {}; console.warn = () => {};++const db = initDb(path.join(dir, 'network.db'));++db.prepare(`INSERT INTO users (id, username, password_hash, role) VALUES++ (10, 'owner', 'x', 'user'), (11, 'admin', 'x', 'user'), (12, 'dev', 'x', 'user'), (13, 'viewer', 'x', 'user'), (14, 'stranger', 'x', 'user'), (15, 'staff', 'x', 'admin')`).run();++const sid = (id) => subjects.ensureUserSubject(db, db.prepare('SELECT * FROM users WHERE id = ?').get(id));++++const keys = crypto.generateKeyPairSync('rsa', { modulusLength: 2048, publicKeyEncoding: { type: 'spki', format: 'pem' }, privateKeyEncoding: { type: 'pkcs8', format: 'pem' } });++const ISSUER = 'https://openvibe.network';++const SECRET = 'u'.repeat(40);++const app = express();++app.locals.db = db;++app.locals.config = { baseUrl: ISSUER, jwt: { issuer: ISSUER, accessTokenExpiry: '1h' }, developer: { sandboxAllowance: '' } };++app.locals.privateKey = keys.privateKey;++app.locals.publicKey = keys.publicKey;++const projectUsage = createProjectUsage(db, { log: { warn() {} } });++const consumer = createEventsConsumer({ db, notifications: new NotificationService(db), secrets: SECRET, projectUsage });++app.use('/internal/events', consumer.router);++app.use('/api/v1/projects', require('../server/developer/routes').router());++const server = http.createServer(app);++const token = (id) => jwt.sign({ sub: id, id }, keys.privateKey, { algorithm: 'RS256', issuer: ISSUER, expiresIn: '1h' });++const T = { owner: token(10), admin: token(11), dev: token(12), viewer: token(13), stranger: token(14), staff: token(15) };++++const HOUR = 60 * 60 * 1000;++const thisHour = Math.floor(Date.now() / HOUR) * HOUR;++const yesterday = thisHour - 24 * HOUR;++const TRACE = '4bf92f3577b34da6a3ce929d0e0e4736';++const JOB = 'job_01JAB2C3D4E5F6G7H8J9K0MNPR';++++(async () => {++ await new Promise((r) => server.listen(0, '127.0.0.1', r));++ const base = `http://127.0.0.1:${server.address().port}`;++ const api = async (who, method, p, body) => {++ const r = await fetch(`${base}/api/v1/projects${p}`, { method, headers: { authorization: `Bearer ${T[who]}`, ...(body ? { 'content-type': 'application/json' } : {}) }, body: body ? JSON.stringify(body) : undefined });++ const text = await r.text();++ return { status: r.status, body: text ? JSON.parse(text) : null, text };++ };++ try {++ // ── A project with an admin, a developer and a viewer; staff record two quotas ──++ let r = await api('owner', 'POST', '', { name: 'Usage demo' });++ assert.strictEqual(r.status, 201);++ const P = r.body.id;++ for (const [who, role] of [['admin', 'admin'], ['dev', 'developer'], ['viewer', 'viewer']]) {++ assert.strictEqual((await api('owner', 'POST', `/${P}/members`, { username: who, role })).status, 201);++ }++ assert.strictEqual((await api('staff', 'PUT', `/${P}/quotas/tools.job.create`, { limit: 100, window: 'day', unit: 'jobs' })).status, 200);++ assert.strictEqual((await api('staff', 'PUT', `/${P}/quotas/events.app.publish`, { limit: 120, window: 'minute', unit: 'events' })).status, 200);++ assert.strictEqual((await api('staff', 'PUT', `/${P}/quotas/media.object.upload`, { limit: 1000000, window: 'month', unit: 'bytes' })).status, 200);++++ // ── Nothing yet: an empty, valid answer ──++ r = await api('owner', 'GET', `/${P}/usage`);++ assert.strictEqual(r.status, 200);++ assert.ok(validate('network.project-usage-result@1', r.body).valid, JSON.stringify(validate('network.project-usage-result@1', r.body).errors));++ assert.deepStrictEqual([r.body.last_recorded_at, r.body.daily.length, r.body.errors.total], [null, 0, 0]);++ assert.match(r.body.freshness, /No usage/);++++ // ── The consumer subscribes to both rollups, and records them ──++ assert.ok(TOPICS.includes('tools.usage.recorded') && TOPICS.includes('events.usage.recorded'));++ assert.deepStrictEqual(topicsFrom(['--topic', 'tools.usage.recorded', '--topic', 'events.usage.recorded']), ['tools.usage.recorded', 'events.usage.recorded']);++ const rollup = (svc, payload, over = {}) => ({++ event_id: ids.newId('event'), event_type: `${svc}.usage.recorded`, version: 1, source: svc, actor: { type: 'service', id: svc },++ timestamp: new Date().toISOString(), subject: { type: 'project', id: payload.project_id }, visibility: 'internal', priority: 'low', payload, ...over,++ });++ const hour = (start, extra) => ({ project_id: P, env: 'production', unit: 'jobs', window: 'hour', window_start: new Date(start).toISOString(), window_end: new Date(start + HOUR).toISOString(), ...extra });++ const jobs = rollup('tools', hour(thisHour, {++ capability: 'tools.job.create', dimension: 'img.process', quantity: 40, errors: 2, error_codes: { 'tools.job.failed': 1, 'tools.job.timeout': 1 },++ samples: [{ at: new Date(thisHour + 60000).toISOString(), code: 'tools.job.timeout', status: 504, trace_id: TRACE, ref: JOB }],++ }));++ // Over HTTP, signed: the route the Events subscription delivers to.++ const raw = JSON.stringify({ event: jobs, seq: 1 });++ const res = await fetch(`${base}/internal/events`, { method: 'POST', headers: { 'content-type': 'application/json', ...signDeliveryHeaders(raw, SECRET) }, body: raw });++ assert.deepStrictEqual([res.status, (await res.json()).outcome], [200, 'recorded']);++ assert.strictEqual(consumer.apply(jobs).duplicate, true, 'a redelivery records nothing twice');++ // The same hour re-sent with new totals replaces it; an older revision never does.++ const resent = rollup('tools', { ...jobs.payload, quantity: 42, errors: 3, error_codes: { 'tools.job.failed': 2, 'tools.job.timeout': 1 }, revision: 2 });++ assert.strictEqual(consumer.apply(resent).outcome, 'recorded');++ assert.strictEqual(consumer.apply(rollup('tools', { ...jobs.payload, quantity: 1, revision: 1 })).outcome, 'ignored:stale');++ assert.strictEqual(consumer.apply(rollup('tools', hour(yesterday, { capability: 'tools.tool.run', dimension: 'image-resize', quantity: 5, errors: 0 }))).outcome, 'recorded');++ assert.strictEqual(consumer.apply(rollup('tools', hour(thisHour - HOUR, { capability: 'tools.job.create', dimension: 'img.process', env: 'sandbox', quantity: 7, errors: 0 }))).outcome, 'recorded');++ const published = rollup('events', hour(thisHour, {++ capability: 'events.app.publish', unit: 'events', env: 'sandbox', quantity: 1800, errors: 61, error_codes: { 'events.quota_exceeded': 60, 'events.type_not_allowed': 1 },++ samples: [{ at: new Date(thisHour + 120000).toISOString(), code: 'events.quota_exceeded', status: 429, trace_id: TRACE }],++ }));++ assert.strictEqual(consumer.apply(published).outcome, 'recorded');++++ // What is not a project's rollup is ignored.++ const other = await api('stranger', 'POST', '', { name: 'Someone else' });++ const ignored = [++ [rollup('tools', { ...jobs.payload, project_id: `prj_${ids.ulid()}` }), 'ignored:project'],++ [rollup('events', jobs.payload, { source: 'tools' }), 'ignored:source'],++ [rollup('tools', { ...jobs.payload, owner: { type: 'user', id: sid(10) } }), 'ignored:payload'],++ [rollup('tools', { ...jobs.payload, window_end: new Date(thisHour + 2 * HOUR).toISOString() }), 'ignored:window'],++ [rollup('tools', jobs.payload, { subject: { type: 'project', id: other.body.id } }), 'ignored:subject'],++ [rollup('tools', { ...jobs.payload, window_start: new Date(thisHour + 3 * HOUR).toISOString(), window_end: new Date(thisHour + 4 * HOUR).toISOString() }), 'ignored:future'],++ ];++ for (const [e, outcome] of ignored) assert.strictEqual(consumer.apply(e).outcome, outcome, outcome);++ assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM notifications').get().n, 0, 'usage notifies nobody');++++ // ── The dashboard's answer ──++ r = await api('owner', 'GET', `/${P}/usage?days=7`);++ assert.strictEqual(r.status, 200);++ const v = validate('network.project-usage-result@1', r.body);++ assert.ok(v.valid, JSON.stringify(v.errors));++ const u = r.body;++ assert.deepStrictEqual([u.env, u.range.days, u.range.to], ['all', 7, new Date().toISOString().slice(0, 10)]);++ assert.ok(u.last_recorded_at);++ const today = new Date(thisHour).toISOString().slice(0, 10);++ const row = (cap, env) => u.daily.find((d) => d.capability === cap && d.env === env && d.day === today);++ assert.deepStrictEqual([row('tools.job.create', 'production').quantity, row('tools.job.create', 'production').errors, row('tools.job.create', 'production').dimension], [42, 3, 'img.process'], 'the re-sent hour replaced the first');++ assert.deepStrictEqual([row('events.app.publish', 'sandbox').quantity, row('events.app.publish', 'sandbox').dimension], [1800, null]);++ assert.ok(u.daily.some((d) => d.capability === 'tools.tool.run' && d.day === new Date(yesterday).toISOString().slice(0, 10)));++ const total = (cap, env) => u.totals.find((x) => x.capability === cap && x.env === env);++ assert.deepStrictEqual([total('tools.job.create', 'production').quantity, total('tools.tool.run', 'production').quantity, total('events.app.publish', 'sandbox').errors], [42, 5, 61]);++ const quota = (cap) => u.quotas.find((x) => x.capability === cap);++ const sandboxToday = new Date(thisHour - HOUR).toISOString().slice(0, 10) === today ? 7 : 0;++ assert.deepStrictEqual([quota('tools.job.create').used, quota('tools.job.create').remaining, quota('tools.job.create').enforced_by], [42 + sandboxToday, 100 - 42 - sandboxToday, 'openvibe.tools'], 'a day quota counts today, every environment');++ assert.strictEqual(quota('tools.job.create').window_start, `${today}T00:00:00.000Z`);++ assert.deepStrictEqual([quota('events.app.publish').used, quota('events.app.publish').remaining], [null, null]);++ assert.match(quota('events.app.publish').note, /minute window/);++ assert.strictEqual(quota('media.object.upload').used, null);++ assert.match(quota('media.object.upload').note, /no service reports/);++ assert.strictEqual(u.errors.total, 64);++ assert.deepStrictEqual(u.errors.by_code[0], { service: 'events', capability: 'events.app.publish', code: 'events.quota_exceeded', count: 60 });++ assert.deepStrictEqual(u.errors.recent.map((e) => [e.service, e.code, e.status, e.trace_id, e.ref]), [++ ['events', 'events.quota_exceeded', 429, TRACE, null],++ ['tools', 'tools.job.timeout', 504, TRACE, JOB],++ ], 'newest first; the re-sent hour did not duplicate its sample');++ for (const who of [10, 11, 12]) assert.ok(!r.text.includes(sid(who)), 'no subject id in the answer');++++ // Filters.++ r = await api('owner', 'GET', `/${P}/usage?env=sandbox`);++ assert.ok(r.body.daily.every((d) => d.env === 'sandbox') && r.body.errors.recent.every((e) => e.env === 'sandbox'));++ for (const q of ['days=0', 'days=91', 'days=abc', 'env=staging']) {++ r = await api('owner', 'GET', `/${P}/usage?${q}`);++ assert.deepStrictEqual([r.status, r.body.code], [422, 'usage.invalid'], q);++ }++ assert.deepStrictEqual(parseQuery({}), { days: 30, env: 'all' });++++ // ── Who may read it: owner, admins and staff; not developers, viewers or strangers ──++ for (const [who, status] of [['admin', 200], ['staff', 200], ['dev', 403], ['viewer', 403], ['stranger', 404]]) {++ r = await api(who, 'GET', `/${P}/usage`);++ assert.strictEqual(r.status, status, who);++ }++ r = await fetch(`${base}/api/v1/projects/${P}/usage`);++ assert.strictEqual(r.status, 401);++++ // ── Retention: windows and samples of the past go; the daily numbers stay ──++ const later = createProjectUsage(db, { now: () => Date.now() + 40 * 24 * HOUR });++ later.prune();++ assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM dev_usage_windows').get().n, 0);++ assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM dev_usage_errors').get().n, 0);++ assert.ok(db.prepare('SELECT COUNT(*) AS n FROM dev_usage_daily').get().n > 0);++ } finally {++ server.close();++ fs.rmSync(dir, { recursive: true, force: true });++ console.log = quiet.log; console.warn = quiet.warn;++ }++ console.log('project usage: all checks passed');++})().catch((e) => { console.log = quiet.log; console.error(e); process.exit(1); });
@@ -18,7 +18,7 @@ Identity and account service for the OpenVibe network. Manages user accounts, OA
@@ -18,7 +18,7 @@ Identity and account service for the OpenVibe network. Manages user accounts, OA
- **Theme Catalog** — Shared theme system with ~30 built-in themes and community submissions. Theme preferences sync across all services.
- **Theme Catalog** — Shared theme system with ~30 built-in themes and community submissions. Theme preferences sync across all services.
- **Admin Panel** — email configuration, user management (role changes, bans), broadcast notifications, system health dashboard, and audit log.
- **Admin Panel** — email configuration, user management (role changes, bans), broadcast notifications, system health dashboard, and audit log.
- **Internal API** — Server-to-server endpoints for token verification, user lookup, notification push, account linking, and audit logging. Protected by `X-Internal-Key`; the routes listed in [docs/retirement.md](docs/retirement.md#routes-that-take-a-token-network-side-of-step-2) also take a capability-scoped service token, and the key is retired route by route.
- **Internal API** — Server-to-server endpoints for token verification, user lookup, notification push, account linking, and audit logging. Protected by `X-Internal-Key`; the routes listed in [docs/retirement.md](docs/retirement.md#routes-that-take-a-token-network-side-of-step-2) also take a capability-scoped service token, and the key is retired route by route.
-- **Developer projects (foundation)** — projects, members, apps (OAuth clients with `app_` subjects), hashed client secrets with rotation and revocation, capability grants within a staff-set allowance (sandbox apps also get a default sandbox allowance of public Media, Events-app and Tools-job capabilities, so a new project works without staff), recorded quotas and an append-only audit at `/api/v1/projects`. Its events can be relayed to OpenVibe.Events through an outbox when `OV_EVENTS_INTERNAL_URL` is set. Bearer user tokens only; no portal UI yet (that is OpenVibe.Codes). See [docs/developer-projects.md](docs/developer-projects.md).
+- **Developer projects (foundation)** — projects, members, apps (OAuth clients with `app_` subjects), hashed client secrets with rotation and revocation, capability grants within a staff-set allowance (sandbox apps also get a default sandbox allowance of public Media, Events-app and Tools-job capabilities, so a new project works without staff), recorded quotas, per-project usage from the services' hourly rollups (`GET /api/v1/projects/:project/usage`) and an append-only audit at `/api/v1/projects`. Its events can be relayed to OpenVibe.Events through an outbox when `OV_EVENTS_INTERNAL_URL` is set. Bearer user tokens only; no portal UI yet (that is OpenVibe.Codes). See [docs/developer-projects.md](docs/developer-projects.md).
- **Public discovery** — `/.well-known/openvibe`, `/api/v1/registry/*` and `/contracts/*.json` (services, capabilities, contract schemas) answer any origin with `Access-Control-Allow-Origin: *`, preflight included, so a browser app can discover the platform directly. Every other route keeps the first-party CORS allow-list (`server/public-cors.js`).
- **Public discovery** — `/.well-known/openvibe`, `/api/v1/registry/*` and `/contracts/*.json` (services, capabilities, contract schemas) answer any origin with `Access-Control-Allow-Origin: *`, preflight included, so a browser app can discover the platform directly. Every other route keeps the first-party CORS allow-list (`server/public-cors.js`).
- **OpenCoins Wallet** — Network-wide currency. User balance/history at `/api/coins/*`; atomic credit/debit/transfer for services at `/internal/coins/*` with idempotency-key dedupe.
- **OpenCoins Wallet** — Network-wide currency. User balance/history at `/api/coins/*`; atomic credit/debit/transfer for services at `/internal/coins/*` with idempotency-key dedupe.
- **User modules** — versioned per-person preferences and summaries in namespaces owned by services (openvibe-contracts `manifests/namespaces`); see [User modules](#user-modules).
- **User modules** — versioned per-person preferences and summaries in namespaces owned by services (openvibe-contracts `manifests/namespaces`); see [User modules](#user-modules).
@@ -219,6 +219,12 @@ announcement an hour, eight a day; `stream_live_cooldown_min`, `stream_live_dail
@@ -219,6 +219,12 @@ announcement an hour, eight a day; `stream_live_cooldown_min`, `stream_live_dail
Live's direct `POST /internal/events/stream-live` (`server/notifications/stream-live.js`), so while Live
Live's direct `POST /internal/events/stream-live` (`server/notifications/stream-live.js`), so while Live
still makes that call, whichever arrives first announces and the other is skipped.
still makes that call, whichever arrives first announces and the other is skipped.
+**Project usage.** `tools.usage.recorded` and `events.usage.recorded` (hourly rollups of a developer
+project's use, `common.usage-recorded@1`) are kept per project and day (`server/developer/usage.js`,
+tables `dev_usage_windows`, `dev_usage_daily`, `dev_usage_errors`) for `GET /api/v1/projects/:project/usage`
+and the project's usage page on openvibe.codes; they notify nobody. See
+[docs/developer-projects.md](docs/developer-projects.md#usage).
+
Coupons publishes no watch event yet: its merchant watches never leave Coupons, and its `coupons.*`
Coupons publishes no watch event yet: its merchant watches never leave Coupons, and its `coupons.*`
events name no person. Every other type is acknowledged and ignored. Deliveries are v2-signed only
events name no person. Every other type is acknowledged and ignored. Deliveries are v2-signed only
(`openvibe-sdk` `parseDelivery` with `requireV2`, ±300 s), and each `event_id` is handled once
(`openvibe-sdk` `parseDelivery` with `requireV2`, ±300 s), and each `event_id` is handled once
@@ -237,7 +243,8 @@ The route is inert until the operator does both of these:
@@ -237,7 +243,8 @@ The route is inert until the operator does both of these:
`deals.watch.matched`, `trade.alert.triggered` and `live.stream.started`; secret = the first
`deals.watch.matched`, `trade.alert.triggered` and `live.stream.started`; secret = the first
`NETWORK_EVENTS_SECRET`). Subscribe `live.stream.started` only once Live serves
`NETWORK_EVENTS_SECRET`). Subscribe `live.stream.started` only once Live serves
`GET /internal/followers`; on a host that already has the other two, add it alone with
`GET /internal/followers`; on a host that already has the other two, add it alone with
- `--topic live.stream.started`:
+ `--topic live.stream.started` (and the project usage rollups with
+ `--topic tools.usage.recorded --topic events.usage.recorded`):
```bash
```bash
cd /opt/openvibe.network
cd /opt/openvibe.network
@@ -599,6 +606,7 @@ Accessible to users with `role = 'admin'`. All endpoints under `/api/admin/`.
@@ -599,6 +606,7 @@ Accessible to users with `role = 'admin'`. All endpoints under `/api/admin/`.
| `dev_apps`, `dev_credentials`, `dev_auth_codes` | Developer apps, hashed client secrets, app authorization codes |
| `dev_apps`, `dev_credentials`, `dev_auth_codes` | Developer apps, hashed client secrets, app authorization codes |
| `dev_grants`, `dev_quotas` | App capability grants; per-project quotas (enforced by the owning service) |
| `dev_grants`, `dev_quotas` | App capability grants; per-project quotas (enforced by the owning service) |
| `dev_audit` | Append-only developer audit; rows that are platform events carry an event envelope |
| `dev_audit` | Append-only developer audit; rows that are platform events carry an event envelope |
+| `dev_usage_windows`, `dev_usage_daily`, `dev_usage_errors` | Developer projects' usage from the services' hourly rollups (35 days), per day (400 days), sampled failures (30 days) |
| `analytics_events`, `analytics_hourly`, `analytics_daily` | Request analytics (raw ≤ 30 days) and rollups, within ADR-021; see [Analytics](#analytics-adr-021) |
| `analytics_events`, `analytics_hourly`, `analytics_daily` | Request analytics (raw ≤ 30 days) and rollups, within ADR-021; see [Analytics](#analytics-adr-021) |
| `user_modules`, `user_module_revisions`, `user_module_retirements` | User-module records, the last revision issued per (subject, namespace), retired namespace owners |
| `user_modules`, `user_module_revisions`, `user_module_retirements` | User-module records, the last revision issued per (subject, namespace), retired namespace owners |
| `network_event_outbox` | Network's events on their way to OpenVibe.Events (developer projects, user modules, blocks) |
| `network_event_outbox` | Network's events on their way to OpenVibe.Events (developer projects, user modules, blocks) |
@@ -133,7 +133,23 @@ This is ADR-014's acceptance rule: "a revoked credential fails everywhere within
@@ -133,7 +133,23 @@ This is ADR-014's acceptance rule: "a revoked credential fails everywhere within
## Quotas
## Quotas
-A quota says how much of one capability a project may use: for example, `media.object.upload` with limit 1073741824, window `total`, unit `bytes`. Network records quotas and shows them to members. **Network does not enforce them.** The service that owns the capability does (`enforced_by` in the response is that audience). Until an owning service reads project quotas (proposed capability `network.project.read`), a quota is a recorded limit and not a guarantee. No usage numbers are collected here.
+A quota says how much of one capability a project may use: for example, `media.object.upload` with limit 1073741824, window `total`, unit `bytes`. Network records quotas and shows them to members. **Network does not enforce them.** The service that owns the capability does (`enforced_by` in the response is that audience). Until an owning service reads project quotas (proposed capability `network.project.read`), a quota is a recorded limit and not a guarantee. What a quota's current window has used is measured from the services' usage rollups ([Usage](#usage)).
+
+## Usage
+
+The services that own developer capabilities count each project's use per environment and UTC hour, and send one rollup per closed hour through OpenVibe.Events (roadmap WS-N task 4): `tools.usage.recorded` from each Tools satellite (jobs, per `tools.job.create` or `tools.tool.run` and job type or tool) and `events.usage.recorded` from Events (`events.app.publish` in `events`, `events.app.subscribe` in `deliveries`). The payload is `common.usage-recorded@1` (openvibe-contracts 0.63.0): project, environment, capability, dimension, unit, window, quantity, errors, errors by code and up to ten sampled failures (time, code, status, trace id, job or event id). A rollup never names who did the work.
+
+The Events consumer (`POST /internal/events`, [server/developer/usage.js](../server/developer/usage.js)) records each one inside its inbox transaction:
+
+- `dev_usage_windows`: one row per rollup key (service, project, environment, capability, dimension, unit, window start). A later rollup for the same key replaces it (the higher `revision` wins), so a re-sent hour is never counted twice. Kept 35 days.
+- `dev_usage_daily`: the same key per UTC day, added up again from its windows on every change. Kept 400 days.
+- `dev_usage_errors`: the sampled failures. Kept 30 days, at most 200 per project.
+
+A rollup for a project Network does not know, from the wrong source, whose subject is not its project, whose window is not a whole hour (or day), from the future or older than 35 days, or that fails its contract is ignored (the consumer's `outcome` says which).
+
+`GET /api/v1/projects/:project/usage?days=30&env=all` (owner and admins, and staff) answers `network.project-usage-result@1`: `daily` rows (newest first), `totals` for the range, `quotas` (each recorded quota with `used` and `remaining` for its current window: today or this month in UTC, or the 400 days kept for `total`, in every environment together; `null` with a `note` for a minute or hour window, which rollups cannot show, and for a capability or unit no service reports), and `errors` (`total`, `by_code`, and the 50 most recent sampled failures). `days` is 1 to 90; `env` is `all`, `sandbox` or `production`; anything else is `422 usage.invalid`. The numbers lag by one hour: an hour appears a few minutes after it closes. OpenVibe.Codes shows it as the project's usage page (`/projects/:project/usage`).
+
+Operator steps: subscribe Network to the two topics once (`node --env-file=/etc/openvibe/network.env scripts/subscribe-events.js --topic tools.usage.recorded --topic events.usage.recorded`), with Events' usage on (default) and Tools' job events on (`EVENTS_URL` set for the satellites).
## API
## API
@@ -175,6 +191,7 @@ Base: `/api/v1/projects`. Every call needs `Authorization: Bearer <Network user
@@ -175,6 +191,7 @@ Base: `/api/v1/projects`. Every call needs `Authorization: Bearer <Network user
| `DELETE /:project/apps/:app/grants/:capability` | admin+, staff | revoke |
| `DELETE /:project/apps/:app/grants/:capability` | admin+, staff | revoke |
| `GET /:project/quotas` | viewer+ | |
| `GET /:project/quotas` | viewer+ | |
| `PUT /:project/quotas/:capability` / `DELETE` | staff | `{ limit, window, unit }` |
| `PUT /:project/quotas/:capability` / `DELETE` | staff | `{ limit, window, unit }` |
+| `GET /:project/usage[?days=&env=]` | admin+, staff | usage per day, totals, quotas with their use, errors ([Usage](#usage)) |
| `GET /:project/audit[?before=&limit=]` | admin+, staff | newest first, paged by `next_before` |
| `GET /:project/audit[?before=&limit=]` | admin+, staff | newest first, paged by `next_before` |
| `POST /:project/export-tokens` | owner, admin (not staff as such) | `{ audience, env }` → a 5-minute read-only export token ([Export tokens](#export-tokens)) |
| `POST /:project/export-tokens` | owner, admin (not staff as such) | `{ audience, env }` → a 5-minute read-only export token ([Export tokens](#export-tokens)) |
@@ -20,7 +20,7 @@
@@ -20,7 +20,7 @@
"helmet": "^8.1.0",
"helmet": "^8.1.0",
"jsonwebtoken": "^9.0.0",
"jsonwebtoken": "^9.0.0",
"multer": "^1.4.5-lts.1",
"multer": "^1.4.5-lts.1",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.62.0",
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.63.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.22.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.22.0",
"uuid": "^11.1.0",
"uuid": "^11.1.0",
@@ -1465,9 +1465,9 @@
@@ -1465,9 +1465,9 @@
}
}
},
},
"node_modules/openvibe-contracts": {
"node_modules/openvibe-contracts": {
- "version": "0.62.0",
- "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.62.0",
- "integrity": "sha512-mmVQToHBY77NrZEHqOBoPuRdZ/bnAzp11E7VkBrN9maPdGCoG8YIJqIq/Hi/dFiDrLJtAcdXRKl/tahdWrkkdQ==",
+ "version": "0.63.0",
+ "resolved": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.63.0",
+ "integrity": "sha512-vmc1b1vWg9M5fm57X50MlZEZHkP2IshHwBXt68+kHDV85nyg6FxaYyUWa4ibi1eeFqzMcECPlILSRTOgg2Xbgw==",
"license": "AGPL-3.0-only",
"license": "AGPL-3.0-only",
"dependencies": {
"dependencies": {
"ajv": "^8.17.1",
"ajv": "^8.17.1",
@@ -6,7 +6,7 @@
@@ -6,7 +6,7 @@
"scripts": {
"scripts": {
"start": "node server/index.js",
"start": "node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
"dev": "NODE_ENV=development node server/index.js",
- "test": "node test/registry-bootstrap.test.js && node test/nginx-generator-live.test.js && node test/admin-refresh.test.js && node test/history-sso.test.js && node test/sso-check.test.js && node test/fedcm.test.js && node test/domains.test.js && node test/frame.test.js && node test/github-integration.test.js && node test/revocation.test.js && node test/usernames.test.js && node test/avatar.test.js && node test/roadmap-baseline.test.js && node test/identity-subjects.test.js && node test/principals.test.js && node test/operator-alerts.test.js && node test/modules.test.js && node test/module-events.test.js && node test/registry-ecosystem.test.js && node test/registry-exposure.test.js && node test/registry-manifest-fixture.test.js && node test/staff-claims.test.js && node test/profile-events.test.js && node test/admin-role-events.test.js && node test/grants-admin.test.js && node test/events-ops.test.js && node test/deploy-drift.test.js && node test/registry-topics.test.js && node test/service-principal.test.js && node test/oauth-pkce.test.js && node test/developer-projects.test.js && node test/developer-defaults.test.js && node test/developer-export-tokens.test.js && node test/public-cors.test.js && node test/observability.test.js && node test/status.test.js && node test/events-consumer.test.js && node test/notification-events.test.js && node test/security-session.test.js && node test/security-redirects.test.js && node test/analytics-privacy.test.js && node test/data-ownership.test.js && node test/unknown-paths.test.js && node test/oidc-discovery.test.js && node test/refresh-tokens.test.js && node test/provider-secrets.test.js && node test/first-party-origins.test.js && node test/updates.test.js && node test/static-headers.test.js && node test/library-tags.test.js && node test/moderation-audit.test.js && node test/blocks.test.js && node test/site-config.test.js && node test/perf-budget.test.js && node test/shutdown.test.js && node test/loyalty-policy.test.js && node test/themes-review.test.js && node test/page-checks.test.js && node test/operator-checklist.test.js && node test/release-health.test.js",
+ "test": "node test/registry-bootstrap.test.js && node test/nginx-generator-live.test.js && node test/admin-refresh.test.js && node test/history-sso.test.js && node test/sso-check.test.js && node test/fedcm.test.js && node test/domains.test.js && node test/frame.test.js && node test/github-integration.test.js && node test/revocation.test.js && node test/usernames.test.js && node test/avatar.test.js && node test/roadmap-baseline.test.js && node test/identity-subjects.test.js && node test/principals.test.js && node test/operator-alerts.test.js && node test/modules.test.js && node test/module-events.test.js && node test/registry-ecosystem.test.js && node test/registry-exposure.test.js && node test/registry-manifest-fixture.test.js && node test/staff-claims.test.js && node test/profile-events.test.js && node test/admin-role-events.test.js && node test/grants-admin.test.js && node test/events-ops.test.js && node test/deploy-drift.test.js && node test/registry-topics.test.js && node test/service-principal.test.js && node test/oauth-pkce.test.js && node test/developer-projects.test.js && node test/developer-defaults.test.js && node test/developer-export-tokens.test.js && node test/project-usage.test.js && node test/public-cors.test.js && node test/observability.test.js && node test/status.test.js && node test/events-consumer.test.js && node test/notification-events.test.js && node test/security-session.test.js && node test/security-redirects.test.js && node test/analytics-privacy.test.js && node test/data-ownership.test.js && node test/unknown-paths.test.js && node test/oidc-discovery.test.js && node test/refresh-tokens.test.js && node test/provider-secrets.test.js && node test/first-party-origins.test.js && node test/updates.test.js && node test/static-headers.test.js && node test/library-tags.test.js && node test/moderation-audit.test.js && node test/blocks.test.js && node test/site-config.test.js && node test/perf-budget.test.js && node test/shutdown.test.js && node test/loyalty-policy.test.js && node test/themes-review.test.js && node test/page-checks.test.js && node test/operator-checklist.test.js && node test/release-health.test.js",
"reset-db": "node server/reset-db.js",
"reset-db": "node server/reset-db.js",
"grant-admin": "node server/grant-admin.js"
"grant-admin": "node server/grant-admin.js"
},
},
@@ -23,7 +23,7 @@
@@ -23,7 +23,7 @@
"helmet": "^8.1.0",
"helmet": "^8.1.0",
"jsonwebtoken": "^9.0.0",
"jsonwebtoken": "^9.0.0",
"multer": "^1.4.5-lts.1",
"multer": "^1.4.5-lts.1",
- "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.62.0",
+ "openvibe-contracts": "https://codeload.github.com/OpenVibers/OpenVibe.Contracts/tar.gz/refs/tags/v0.63.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0",
"openvibe-sdk": "https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.11.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.22.0",
"openvibe-shared": "https://codeload.github.com/OpenVibers/OpenVibe.Shared/tar.gz/refs/tags/v1.22.0",
"uuid": "^11.1.0",
"uuid": "^11.1.0",
@@ -3,14 +3,16 @@
@@ -3,14 +3,16 @@
/**
/**
* Create Network's OpenVibe.Events subscriptions for the notification consumer
* Create Network's OpenVibe.Events subscriptions for the notification consumer
* (server/notifications/events-consumer.js): one per topic in TOPICS (deals.watch.matched,
* (server/notifications/events-consumer.js): one per topic in TOPICS (deals.watch.matched,
- * trade.alert.triggered, live.stream.started), all delivering to POST /internal/events. Events names
- * the consumer after the calling service: `network`.
+ * trade.alert.triggered, live.stream.started, the moderation audit topics and the project usage
+ * rollups tools.usage.recorded and events.usage.recorded), all delivering to POST /internal/events.
+ * Events names the consumer after the calling service: `network`.
*
*
* sudo node --env-file=/etc/openvibe/network.env scripts/subscribe-events.js \
* sudo node --env-file=/etc/openvibe/network.env scripts/subscribe-events.js \
* [--endpoint http://127.0.0.1:4000/internal/events] [--topic live.stream.started] [--dry-run]
* [--endpoint http://127.0.0.1:4000/internal/events] [--topic live.stream.started] [--dry-run]
*
*
* --topic limits the run to one of TOPICS (repeatable), e.g. to add live.stream.started to a host that
* --topic limits the run to one of TOPICS (repeatable), e.g. to add live.stream.started to a host that
- * already has the other two. live.stream.started needs Live's GET /internal/followers first (the
+ * already has the other two, or `--topic tools.usage.recorded --topic events.usage.recorded` for the
+ * project dashboards (WS-N task 4). live.stream.started needs Live's GET /internal/followers first (the
* consumer answers 503 until Live serves it, and Events retries), and the token Network signs for it
* consumer answers 503 until Live serves it, and Events retries), and the token Network signs for it
* carries live.follower.read for audience openvibe.live.
* carries live.follower.read for audience openvibe.live.
*
*
@@ -744,6 +744,8 @@ function initDb(dbPath) {
@@ -744,6 +744,8 @@ function initDb(dbPath) {
require('../identity/modules').ensureSchema(db);
require('../identity/modules').ensureSchema(db);
// Developer projects, apps, credentials, grants, quotas and audit (server/developer/store.js).
// Developer projects, apps, credentials, grants, quotas and audit (server/developer/store.js).
require('../developer/store').ensureSchema(db);
require('../developer/store').ensureSchema(db);
+ // Their usage, from the services' rollups (server/developer/usage.js).
+ require('../developer/usage').ensure(db);
// Platform blocks, keyed by subjects (server/identity/blocks.js).
// Platform blocks, keyed by subjects (server/identity/blocks.js).
require('../identity/blocks').ensureSchema(db);
require('../identity/blocks').ensureSchema(db);
@@ -35,6 +35,8 @@
@@ -35,6 +35,8 @@
* GET /:project/quotas viewer+
* GET /:project/quotas viewer+
* PUT /:project/quotas/:capability { limit, window, unit } staff
* PUT /:project/quotas/:capability { limit, window, unit } staff
* DELETE /:project/quotas/:capability staff
* DELETE /:project/quotas/:capability staff
+ * GET /:project/usage[?days=30&env=all] owner/admin or staff: usage per day, quotas and
+ * errors from the services' rollups (usage.js)
* GET /:project/audit[?before=&limit=] admin+ or staff
* GET /:project/audit[?before=&limit=] admin+ or staff
* POST /:project/export-tokens { audience, env } owner or admin member (not staff as such):
* POST /:project/export-tokens { audience, env } owner or admin member (not staff as such):
* a 5-minute read-only export token (tokens.js)
* a 5-minute read-only export token (tokens.js)
@@ -45,6 +47,7 @@ const { verifySession } = require('../auth/session');
@@ -45,6 +47,7 @@ const { verifySession } = require('../auth/session');
const store = require('./store');
const store = require('./store');
const policy = require('./policy');
const policy = require('./policy');
const tokens = require('./tokens');
const tokens = require('./tokens');
+const usage = require('./usage');
function router() {
function router() {
const r = express.Router();
const r = express.Router();
@@ -118,6 +121,12 @@ function router() {
@@ -118,6 +121,12 @@ function router() {
r.put('/:project/quotas/:capability', handle((db, a, req, o) => store.setQuota(db, a, req.params.project, req.params.capability, req.body || {}, o)));
r.put('/:project/quotas/:capability', handle((db, a, req, o) => store.setQuota(db, a, req.params.project, req.params.capability, req.body || {}, o)));
r.delete('/:project/quotas/:capability', handle((db, a, req, o) => store.deleteQuota(db, a, req.params.project, req.params.capability, o)));
r.delete('/:project/quotas/:capability', handle((db, a, req, o) => store.deleteQuota(db, a, req.params.project, req.params.capability, o)));
+ // Usage (WS-N task 4): network.project-usage-result@1, for the owner and admins (and staff).
+ r.get('/:project/usage', handle((db, a, req) => {
+ const { project } = store.access(db, a, req.params.project, { need: 'admin', allowArchived: true });
+ return usage.summary(db, project.id, usage.parseQuery(req.query));
+ }));
+
r.get('/:project/audit', handle((db, a, req) => {
r.get('/:project/audit', handle((db, a, req) => {
const { project } = store.access(db, a, req.params.project, { need: 'admin', allowArchived: true });
const { project } = store.access(db, a, req.params.project, { need: 'admin', allowArchived: true });
return store.listAudit(db, project.id, { before: req.query.before, limit: req.query.limit });
return store.listAudit(db, project.id, { before: req.query.before, limit: req.query.limit });
@@ -134,7 +143,7 @@ function router() {
@@ -134,7 +143,7 @@ function router() {
}
}
function send(req, res, err) {
function send(req, res, err) {
- if (err instanceof store.DevError) return http.sendProblem(res, err.status, err.code, { detail: err.detail, ctx: req.ov });
+ if (err instanceof store.DevError || err instanceof usage.UsageQueryError) return http.sendProblem(res, err.status, err.code, { detail: err.detail, ctx: req.ov });
if (err && err.type === 'entity.parse.failed') return http.sendProblem(res, 400, 'request.malformed_json', { detail: 'body is not valid JSON', ctx: req.ov });
if (err && err.type === 'entity.parse.failed') return http.sendProblem(res, 400, 'request.malformed_json', { detail: 'body is not valid JSON', ctx: req.ov });
if (err && err.type === 'entity.too.large') return http.sendProblem(res, 413, 'request.too_large', { ctx: req.ov });
if (err && err.type === 'entity.too.large') return http.sendProblem(res, 413, 'request.too_large', { ctx: req.ov });
// Never echo internals (and so never a secret) to the client.
// Never echo internals (and so never a secret) to the client.
new file mode 100644
new file mode 100644
@@ -0,0 +1,278 @@
@@ -0,0 +1,278 @@
+'use strict';
+/**
+ * Project usage (roadmap WS-N task 4, ADR-014): the rollups the owning services send of each developer
+ * project's use (<service>.usage.recorded, openvibe-contracts common.usage-recorded@1), kept per
+ * project and day for the project's dashboard on OpenVibe.Codes.
+ *
+ * tools.usage.recorded a Tools satellite's hour of a project's jobs (unit jobs)
+ * events.usage.recorded OpenVibe.Events: published events (events) and webhook deliveries (deliveries)
+ *
+ * The Events consumer (server/notifications/events-consumer.js) calls record() inside its inbox
+ * transaction, so a redelivery changes nothing. A rollup is the totals of its key (service, project,
+ * env, capability, dimension, unit, window_start): a newer or equal revision replaces the stored one
+ * and its day is added up again from its windows, so a producer re-sending an hour never counts it
+ * twice. The failures a rollup samples (time, code, status, trace id, job or event id) are kept for
+ * the dashboard's recent errors. Nothing here names who did the work.
+ *
+ * dev_usage_windows one row per rollup key (WINDOW_DAYS, 35 days)
+ * dev_usage_daily per project, env, service, capability, dimension, unit and UTC day (DAILY_DAYS, 400 days)
+ * dev_usage_errors sampled failures (ERROR_DAYS, 30 days; the newest ERRORS_PER_PROJECT per project)
+ *
+ * summary() is what GET /api/v1/projects/:project/usage (./routes.js) answers the project's owner, its
+ * admins and staff: network.project-usage-result@1, with the recorded quotas (dev_quotas) and what
+ * their current window used where the rollups can tell (a day, a month, the kept total).
+ */
+const { validate, capabilities } = require('openvibe-contracts');
+
+const SERVICES = Object.freeze(['tools', 'events']);
+const TOPICS = Object.freeze(SERVICES.map((s) => `${s}.usage.recorded`));
+const HOUR_MS = 60 * 60 * 1000;
+const DAY_MS = 24 * HOUR_MS;
+const WINDOW_DAYS = 35;
+const DAILY_DAYS = 400;
+const ERROR_DAYS = 30;
+const ERRORS_PER_PROJECT = 200;
+const MAX_DAYS = 90;
+const RECENT_ERRORS = 50;
+// What each service reports, so a quota on one of these is measured even before its first rollup.
+const REPORTED = Object.freeze({
+ 'tools.job.create': ['jobs'], 'tools.tool.run': ['jobs'],
+ 'events.app.publish': ['events'], 'events.app.subscribe': ['deliveries'],
+});
+
+const iso = (ms) => new Date(ms).toISOString();
+const dayOf = (ms) => iso(ms).slice(0, 10);
+const parse = (s, d) => { try { return JSON.parse(s); } catch { return d; } };
+const obj = (v) => (v && typeof v === 'object' && !Array.isArray(v) ? v : {});
+
+function ensure(db) {
+ db.exec(`
+ CREATE TABLE IF NOT EXISTS dev_usage_windows (
+ project_id TEXT NOT NULL,
+ env TEXT NOT NULL CHECK (env IN ('sandbox', 'production')),
+ service TEXT NOT NULL,
+ capability TEXT NOT NULL,
+ dimension TEXT NOT NULL DEFAULT '',
+ unit TEXT NOT NULL,
+ window_start TEXT NOT NULL,
+ window_end TEXT NOT NULL,
+ quantity INTEGER NOT NULL,
+ errors INTEGER NOT NULL,
+ error_codes TEXT NOT NULL DEFAULT '{}',
+ revision INTEGER NOT NULL DEFAULT 1,
+ event_id TEXT NOT NULL,
+ recorded_at TEXT NOT NULL,
+ PRIMARY KEY (project_id, env, service, capability, dimension, unit, window_start)
+ );
+ CREATE INDEX IF NOT EXISTS idx_dev_usage_windows_when ON dev_usage_windows(window_start);
+ CREATE INDEX IF NOT EXISTS idx_dev_usage_windows_project ON dev_usage_windows(project_id, recorded_at);
+ CREATE TABLE IF NOT EXISTS dev_usage_daily (
+ project_id TEXT NOT NULL,
+ env TEXT NOT NULL,
+ service TEXT NOT NULL,
+ capability TEXT NOT NULL,
+ dimension TEXT NOT NULL DEFAULT '',
+ unit TEXT NOT NULL,
+ day TEXT NOT NULL,
+ quantity INTEGER NOT NULL DEFAULT 0,
+ errors INTEGER NOT NULL DEFAULT 0,
+ error_codes TEXT NOT NULL DEFAULT '{}',
+ updated_at TEXT NOT NULL,
+ PRIMARY KEY (project_id, env, service, capability, dimension, unit, day)
+ );
+ CREATE INDEX IF NOT EXISTS idx_dev_usage_daily_day ON dev_usage_daily(project_id, day);
+ CREATE TABLE IF NOT EXISTS dev_usage_errors (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ project_id TEXT NOT NULL,
+ env TEXT NOT NULL,
+ service TEXT NOT NULL,
+ capability TEXT NOT NULL,
+ at TEXT NOT NULL,
+ code TEXT NOT NULL,
+ status INTEGER,
+ trace_id TEXT,
+ ref TEXT NOT NULL DEFAULT '',
+ event_id TEXT NOT NULL
+ );
+ CREATE UNIQUE INDEX IF NOT EXISTS idx_dev_usage_errors_once ON dev_usage_errors(project_id, env, service, capability, at, code, ref);
+ CREATE INDEX IF NOT EXISTS idx_dev_usage_errors_recent ON dev_usage_errors(project_id, at);
+ `);
+}
+
+function mergeCodes(into, codes) {
+ for (const [k, n] of Object.entries(obj(codes))) if (Number.isSafeInteger(n) && n > 0) into[k] = (into[k] || 0) + n;
+ return into;
+}
+
+/**
+ * @param {import('better-sqlite3').Database} db
+ * @param {{ now?: () => number, log?: object }} [o]
+ */
+function createProjectUsage(db, { now = () => Date.now(), log = console } = {}) {
+ ensure(db);
+ const q = {
+ project: db.prepare('SELECT id FROM dev_projects WHERE id = ?'),
+ window: db.prepare(`SELECT revision FROM dev_usage_windows WHERE project_id = ? AND env = ? AND service = ? AND capability = ?
+ AND dimension = ? AND unit = ? AND window_start = ?`),
+ upsertWindow: db.prepare(`INSERT INTO dev_usage_windows (project_id, env, service, capability, dimension, unit, window_start, window_end,
+ quantity, errors, error_codes, revision, event_id, recorded_at)
+ VALUES (@project_id, @env, @service, @capability, @dimension, @unit, @window_start, @window_end, @quantity, @errors, @error_codes, @revision, @event_id, @recorded_at)
+ ON CONFLICT(project_id, env, service, capability, dimension, unit, window_start) DO UPDATE SET
+ window_end = excluded.window_end, quantity = excluded.quantity, errors = excluded.errors, error_codes = excluded.error_codes,
+ revision = excluded.revision, event_id = excluded.event_id, recorded_at = excluded.recorded_at`),
+ dayWindows: db.prepare(`SELECT quantity, errors, error_codes FROM dev_usage_windows WHERE project_id = ? AND env = ? AND service = ?
+ AND capability = ? AND dimension = ? AND unit = ? AND window_start >= ? AND window_start < ?`),
+ upsertDaily: db.prepare(`INSERT INTO dev_usage_daily (project_id, env, service, capability, dimension, unit, day, quantity, errors, error_codes, updated_at)
+ VALUES (@project_id, @env, @service, @capability, @dimension, @unit, @day, @quantity, @errors, @error_codes, @updated_at)
+ ON CONFLICT(project_id, env, service, capability, dimension, unit, day) DO UPDATE SET
+ quantity = excluded.quantity, errors = excluded.errors, error_codes = excluded.error_codes, updated_at = excluded.updated_at`),
+ addError: db.prepare(`INSERT OR IGNORE INTO dev_usage_errors (project_id, env, service, capability, at, code, status, trace_id, ref, event_id)
+ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`),
+ capErrors: db.prepare(`DELETE FROM dev_usage_errors WHERE project_id = ? AND id NOT IN
+ (SELECT id FROM dev_usage_errors WHERE project_id = ? ORDER BY at DESC, id DESC LIMIT ${ERRORS_PER_PROJECT})`),
+ };
+ let lastPrune = 0;
+
+ /** Inside the consumer's inbox transaction: 'recorded' or an 'ignored:*' outcome. */
+ function record(event) {
+ const type = String(event && event.event_type || '');
+ if (!TOPICS.includes(type)) return 'ignored:type';
+ const service = type.split('.')[0];
+ if (event.source !== service) return 'ignored:source';
+ const p = obj(event.payload);
+ if (p.redacted === true) return 'ignored:redacted';
+ const v = validate(`${type}@${event.version || 1}`, p);
+ if (!v.valid) {
+ log.warn(`[Usage] ${event.event_id} (${type}) does not match its contract: ${v.errors.slice(0, 3).map((e) => `${e.path} ${e.message}`).join('; ')}`);
+ return 'ignored:payload';
+ }
+ const subject = obj(event.subject);
+ if (subject.type !== 'project' || subject.id !== p.project_id) return 'ignored:subject';
+ const start = Date.parse(p.window_start);
+ const end = Date.parse(p.window_end);
+ const len = p.window === 'day' ? DAY_MS : HOUR_MS;
+ if (end - start !== len || start % len !== 0) return 'ignored:window';
+ const t = now();
+ if (start > t + HOUR_MS) return 'ignored:future';
+ if (start < t - WINDOW_DAYS * DAY_MS) return 'ignored:too_old';
+ if (!q.project.get(p.project_id)) return 'ignored:project';
+ const key = { project_id: p.project_id, env: p.env, service, capability: p.capability, dimension: p.dimension || '', unit: p.unit };
+ const windowStart = iso(start);
+ const revision = p.revision || 1;
+ const prev = q.window.get(key.project_id, key.env, key.service, key.capability, key.dimension, key.unit, windowStart);
+ if (prev && prev.revision > revision) return 'ignored:stale';
+ const recordedAt = iso(t);
+ q.upsertWindow.run({
+ ...key, window_start: windowStart, window_end: iso(end), quantity: p.quantity, errors: p.errors,
+ error_codes: JSON.stringify(obj(p.error_codes)), revision, event_id: event.event_id, recorded_at: recordedAt,
+ });
+ // The day this window falls in, added up again from its windows (a day window is its own day).
+ const day = dayOf(start);
+ const from = `${day}T00:00:00.000Z`;
+ const to = iso(Date.parse(from) + DAY_MS);
+ const sum = { quantity: 0, errors: 0, codes: {} };
+ for (const w of q.dayWindows.all(key.project_id, key.env, key.service, key.capability, key.dimension, key.unit, from, to)) {
+ sum.quantity += w.quantity; sum.errors += w.errors; mergeCodes(sum.codes, parse(w.error_codes, {}));
+ }
+ q.upsertDaily.run({ ...key, day, quantity: sum.quantity, errors: sum.errors, error_codes: JSON.stringify(sum.codes), updated_at: recordedAt });
+ for (const s of Array.isArray(p.samples) ? p.samples : []) {
+ q.addError.run(key.project_id, key.env, service, key.capability, iso(Date.parse(s.at)), s.code, s.status || null, s.trace_id || null, s.ref || '', event.event_id);
+ }
+ if (p.samples && p.samples.length) q.capErrors.run(key.project_id, key.project_id);
+ if (t - lastPrune > HOUR_MS) { lastPrune = t; prune(t); }
+ return 'recorded';
+ }
+
+ function prune(t = now()) {
+ db.prepare('DELETE FROM dev_usage_windows WHERE window_start < ?').run(iso(t - WINDOW_DAYS * DAY_MS));
+ db.prepare('DELETE FROM dev_usage_daily WHERE day < ?').run(dayOf(t - DAILY_DAYS * DAY_MS));
+ db.prepare('DELETE FROM dev_usage_errors WHERE at < ?').run(iso(t - ERROR_DAYS * DAY_MS));
+ }
+
+ return { record, prune, summary: (projectId, o) => summary(db, projectId, { now, ...o }) };
+}
+
+class UsageQueryError extends Error {
+ constructor(detail) { super(detail); this.status = 422; this.code = 'usage.invalid'; this.detail = detail; }
+}
+
+/** ?days=&env= → { days, env }; throws UsageQueryError for anything else. */
+function parseQuery(query = {}) {
+ const rawDays = query.days == null || query.days === '' ? '30' : String(query.days);
+ if (!/^\d{1,3}$/.test(rawDays) || Number(rawDays) < 1 || Number(rawDays) > MAX_DAYS) throw new UsageQueryError(`days is a whole number from 1 to ${MAX_DAYS}`);
+ const env = query.env == null || query.env === '' ? 'all' : String(query.env);
+ if (!['all', 'sandbox', 'production'].includes(env)) throw new UsageQueryError('env is all, sandbox or production');
+ return { days: Number(rawDays), env };
+}
+
+/** network.project-usage-result@1 for one project (the caller checked access). */
+function summary(db, projectId, { days = 30, env = 'all', now = () => Date.now() } = {}) {
+ const t = now();
+ const to = dayOf(t);
+ const from = dayOf(t - (days - 1) * DAY_MS);
+ const envSql = env === 'all' ? '' : ' AND env = ?';
+ const envArgs = env === 'all' ? [] : [env];
+ const rows = db.prepare(`SELECT * FROM dev_usage_daily WHERE project_id = ? AND day >= ?${envSql}
+ ORDER BY day DESC, service, capability, dimension, unit, env`).all(projectId, from, ...envArgs);
+
+ const daily = rows.map((r) => ({ day: r.day, service: r.service, capability: r.capability, dimension: r.dimension || null, unit: r.unit, env: r.env, quantity: r.quantity, errors: r.errors }));
+ const totals = new Map();
+ const byCode = new Map();
+ let errorTotal = 0;
+ for (const r of rows) {
+ const k = `${r.service}|${r.capability}|${r.unit}|${r.env}`;
+ const tot = totals.get(k) || { service: r.service, capability: r.capability, unit: r.unit, env: r.env, quantity: 0, errors: 0 };
+ tot.quantity += r.quantity; tot.errors += r.errors;
+ totals.set(k, tot);
+ errorTotal += r.errors;
+ for (const [code, n] of Object.entries(parse(r.error_codes, {}))) {
+ const ck = `${r.service}|${r.capability}|${code}`;
+ const c = byCode.get(ck) || { service: r.service, capability: r.capability, code, count: 0 };
+ c.count += n;
+ byCode.set(ck, c);
+ }
+ }
+
+ // Quotas (dev_quotas, staff-set): what their current window used, every environment together.
+ const seen = new Set(db.prepare('SELECT DISTINCT capability, unit FROM dev_usage_daily WHERE project_id = ?').all(projectId).map((r) => `${r.capability}|${r.unit}`));
+ const usedSince = db.prepare('SELECT COALESCE(SUM(quantity), 0) AS n FROM dev_usage_daily WHERE project_id = ? AND capability = ? AND unit = ? AND day >= ?');
+ const quotas = db.prepare('SELECT * FROM dev_quotas WHERE project_id = ? ORDER BY capability').all(projectId).map((qr) => {
+ const cap = capabilities.get(qr.capability);
+ const out = { capability: qr.capability, limit: qr.limit_value, window: qr.quota_window, unit: qr.unit, enforced_by: cap ? `openvibe.${cap.owner}` : null, used: null, remaining: null, window_start: null, note: null };
+ const measured = seen.has(`${qr.capability}|${qr.unit}`) || (REPORTED[qr.capability] || []).includes(qr.unit);
+ if (!measured) {
+ const units = REPORTED[qr.capability];
+ out.note = units ? `the service reports ${units.join(', ')} for this capability, not ${qr.unit}` : 'no service reports usage of this capability yet';
+ return out;
+ }
+ if (qr.quota_window === 'minute' || qr.quota_window === 'hour') {
+ out.note = `a ${qr.quota_window} window is enforced by the service as it happens; hourly rollups cannot show it`;
+ return out;
+ }
+ let since;
+ if (qr.quota_window === 'day') { since = to; out.window_start = `${to}T00:00:00.000Z`; }
+ else if (qr.quota_window === 'month') { since = `${to.slice(0, 7)}-01`; out.window_start = `${since}T00:00:00.000Z`; }
+ else { since = '0000-00-00'; out.note = `total of the last ${DAILY_DAYS} days kept`; }
+ out.used = usedSince.get(projectId, qr.capability, qr.unit, since).n;
+ out.remaining = Math.max(0, qr.limit_value - out.used);
+ return out;
+ });
+
+ const recent = db.prepare(`SELECT * FROM dev_usage_errors WHERE project_id = ? AND at >= ?${envSql} ORDER BY at DESC, id DESC LIMIT ${RECENT_ERRORS}`)
+ .all(projectId, `${from}T00:00:00.000Z`, ...envArgs)
+ .map((e) => ({ at: e.at, env: e.env, service: e.service, capability: e.capability, code: e.code, status: e.status == null ? null : e.status, trace_id: e.trace_id || null, ref: e.ref || null }));
+ const last = db.prepare('SELECT MAX(recorded_at) AS at FROM dev_usage_windows WHERE project_id = ?').get(projectId).at || null;
+
+ return {
+ project_id: projectId, env, range: { days, from, to }, generated_at: iso(t), last_recorded_at: last,
+ freshness: last
+ ? 'Usage arrives as hourly rollups a few minutes after each hour closes (UTC); the current hour is not counted yet.'
+ : 'No usage has been recorded for this project yet. Rollups arrive a few minutes after each hour (UTC) closes.',
+ totals: [...totals.values()].sort((a, b) => a.service.localeCompare(b.service) || a.capability.localeCompare(b.capability) || a.env.localeCompare(b.env)),
+ daily,
+ quotas,
+ errors: { total: errorTotal, by_code: [...byCode.values()].sort((a, b) => b.count - a.count || a.code.localeCompare(b.code)), recent },
+ };
+}
+
+module.exports = { createProjectUsage, summary, parseQuery, ensure, UsageQueryError, TOPICS, SERVICES, MAX_DAYS };
@@ -371,9 +371,11 @@ const liveFollowers = privateKey.includes('BEGIN')
@@ -371,9 +371,11 @@ const liveFollowers = privateKey.includes('BEGIN')
: null;
: null;
// The moderation audit log (ADR-022): staff actions every service reports, readable by staff.
// The moderation audit log (ADR-022): staff actions every service reports, readable by staff.
const moderationAudit = require('./admin/moderation-audit').createModerationAudit(db);
const moderationAudit = require('./admin/moderation-audit').createModerationAudit(db);
+// Developer projects' usage (WS-N task 4): the services' hourly rollups, per project and day.
+const projectUsage = require('./developer/usage').createProjectUsage(db);
eventsConsumer = require('./notifications/events-consumer').createEventsConsumer({
eventsConsumer = require('./notifications/events-consumer').createEventsConsumer({
db, notifications: notificationService, secrets: config.eventsWebhookSecrets,
db, notifications: notificationService, secrets: config.eventsWebhookSecrets,
- liveFollowers, discord: () => app.locals.discordService || null, moderationAudit,
+ liveFollowers, discord: () => app.locals.discordService || null, moderationAudit, projectUsage,
});
});
console.log(`[Events consumer] ${eventsConsumer.enabled ? 'on' : 'off (NETWORK_EVENTS_SECRET unset)'}: POST /internal/events`);
console.log(`[Events consumer] ${eventsConsumer.enabled ? 'on' : 'off (NETWORK_EVENTS_SECRET unset)'}: POST /internal/events`);
@@ -20,6 +20,10 @@
@@ -20,6 +20,10 @@
* A started event older than LIVE_STARTED_MAX_AGE (30 min; a replay, or Events catching up after an
* A started event older than LIVE_STARTED_MAX_AGE (30 min; a replay, or Events catching up after an
* outage) announces nothing: the stream may be long over.
* outage) announces nothing: the stream may be long over.
*
*
+ * <service>.usage.recorded a developer project's hourly usage rollup from Tools or Events
+ * (common.usage-recorded@1): kept per project and day for its dashboard
+ * (server/developer/usage.js), never anyone's notification.
+ *
* Coupons publishes no watch event: its merchant watches never leave Coupons (server/domain/watches.js
* Coupons publishes no watch event: its merchant watches never leave Coupons (server/domain/watches.js
* there: "Delivery is not built yet"), and its coupons.* events name no person, so there is nothing
* there: "Delivery is not built yet"), and its coupons.* events name no person, so there is nothing
* of Coupons' to consume yet. README "Notifications from Events" has the operator steps.
* of Coupons' to consume yet. README "Notifications from Events" has the operator steps.
@@ -50,7 +54,8 @@ const { LiveFollowersError } = require('./live-followers');
@@ -50,7 +54,8 @@ const { LiveFollowersError } = require('./live-followers');
const CONSUMER = 'network-notifications';
const CONSUMER = 'network-notifications';
const INBOX_TABLE = 'network_event_inbox';
const INBOX_TABLE = 'network_event_inbox';
const AUDIT_TOPICS = require('../admin/moderation-audit').TOPICS;
const AUDIT_TOPICS = require('../admin/moderation-audit').TOPICS;
-const TOPICS = Object.freeze(['deals.watch.matched', 'trade.alert.triggered', 'live.stream.started', ...AUDIT_TOPICS]);
+const USAGE_TOPICS = require('../developer/usage').TOPICS;
+const TOPICS = Object.freeze(['deals.watch.matched', 'trade.alert.triggered', 'live.stream.started', ...AUDIT_TOPICS, ...USAGE_TOPICS]);
const LIVE_STARTED_MAX_AGE_MS = 30 * 60 * 1000;
const LIVE_STARTED_MAX_AGE_MS = 30 * 60 * 1000;
const EVENT_ID_RE = /^evt_[0-9A-HJKMNP-TV-Z]{26}$/;
const EVENT_ID_RE = /^evt_[0-9A-HJKMNP-TV-Z]{26}$/;
@@ -155,8 +160,9 @@ function liveStarted(event, { now, maxAgeMs }) {
@@ -155,8 +160,9 @@ function liveStarted(event, { now, maxAgeMs }) {
* @param {string|string[]} o.secrets NETWORK_EVENTS_SECRET (comma list) or an array
* @param {string|string[]} o.secrets NETWORK_EVENTS_SECRET (comma list) or an array
* @param {{ forStream(id: number): Promise<object> }} [o.liveFollowers] ./live-followers.js
* @param {{ forStream(id: number): Promise<object> }} [o.liveFollowers] ./live-followers.js
* @param {() => ({ sendLiveAlert(streamer: object, stream: object): Promise<object> }|null)} [o.discord]
* @param {() => ({ sendLiveAlert(streamer: object, stream: object): Promise<object> }|null)} [o.discord]
+ * @param {{ record(event: object): string }} [o.projectUsage] ../developer/usage.js createProjectUsage()
*/
*/
-function createEventsConsumer({ db, notifications, secrets, liveFollowers = null, discord = () => null, moderationAudit = null, liveStartedMaxAgeMs = LIVE_STARTED_MAX_AGE_MS, now = () => Date.now(), log = console }) {
+function createEventsConsumer({ db, notifications, secrets, liveFollowers = null, discord = () => null, moderationAudit = null, projectUsage = null, liveStartedMaxAgeMs = LIVE_STARTED_MAX_AGE_MS, now = () => Date.now(), log = console }) {
const keys = Array.isArray(secrets) ? secrets.filter((s) => typeof s === 'string' && s.length >= 32) : secretsFrom(secrets);
const keys = Array.isArray(secrets) ? secrets.filter((s) => typeof s === 'string' && s.length >= 32) : secretsFrom(secrets);
const inbox = createInbox(db, { table: INBOX_TABLE, now });
const inbox = createInbox(db, { table: INBOX_TABLE, now });
inbox.ensureSchema();
inbox.ensureSchema();
@@ -222,6 +228,8 @@ function createEventsConsumer({ db, notifications, secrets, liveFollowers = null
@@ -222,6 +228,8 @@ function createEventsConsumer({ db, notifications, secrets, liveFollowers = null
const r = inbox.once(CONSUMER, event.event_id, () => {
const r = inbox.once(CONSUMER, event.event_id, () => {
// Staff actions go to the moderation audit log (ADR-022), never to anyone's inbox.
// Staff actions go to the moderation audit log (ADR-022), never to anyone's inbox.
if (AUDIT_TOPICS.includes(event.event_type)) return moderationAudit ? moderationAudit.record(event) : 'ignored:audit_off';
if (AUDIT_TOPICS.includes(event.event_type)) return moderationAudit ? moderationAudit.record(event) : 'ignored:audit_off';
+ // Developer projects' usage rollups go to their dashboards (WS-N task 4), never to an inbox.
+ if (USAGE_TOPICS.includes(event.event_type)) return projectUsage ? projectUsage.record(event) : 'ignored:usage_off';
if (event.event_type === 'live.stream.started') {
if (event.event_type === 'live.stream.started') {
const out = liveStreamStarted(event, prep);
const out = liveStreamStarted(event, prep);
if (typeof out === 'string') return out;
if (typeof out === 'string') return out;
@@ -59,7 +59,8 @@ const rows = (userId) => db.prepare('SELECT * FROM notifications WHERE user_id =
@@ -59,7 +59,8 @@ const rows = (userId) => db.prepare('SELECT * FROM notifications WHERE user_id =
(async () => {
(async () => {
// The subscriptions this consumer is meant for, and nothing else.
// The subscriptions this consumer is meant for, and nothing else.
assert.deepStrictEqual(TOPICS, ['deals.watch.matched', 'trade.alert.triggered', 'live.stream.started', 'chat.moderation.action', 'live.moderation.action', 'community.moderation.action', 'tips.interaction.moderated', 'billing.staff.action',
assert.deepStrictEqual(TOPICS, ['deals.watch.matched', 'trade.alert.triggered', 'live.stream.started', 'chat.moderation.action', 'live.moderation.action', 'community.moderation.action', 'tips.interaction.moderated', 'billing.staff.action',
- ...['media', 'tools', 'games', 'wiki', 'blog', 'news', 'reviews', 'deals', 'coupons', 'trade', 'codes'].map((svc) => `${svc}.moderation.action`)]);
+ ...['media', 'tools', 'games', 'wiki', 'blog', 'news', 'reviews', 'deals', 'coupons', 'trade', 'codes'].map((svc) => `${svc}.moderation.action`),
+ 'tools.usage.recorded', 'events.usage.recorded']);
assert.deepStrictEqual(topicsFrom([]), TOPICS, 'scripts/subscribe-events.js subscribes every topic');
assert.deepStrictEqual(topicsFrom([]), TOPICS, 'scripts/subscribe-events.js subscribes every topic');
assert.deepStrictEqual(topicsFrom(['--topic', 'live.stream.started']), ['live.stream.started']);
assert.deepStrictEqual(topicsFrom(['--topic', 'live.stream.started']), ['live.stream.started']);
assert.throws(() => topicsFrom(['--topic', 'live.*']));
assert.throws(() => topicsFrom(['--topic', 'live.*']));
new file mode 100644
new file mode 100644
@@ -0,0 +1,185 @@
@@ -0,0 +1,185 @@
+'use strict';
+// Project usage (WS-N task 4, server/developer/usage.js): tools.usage.recorded and events.usage.recorded
+// arrive at POST /internal/events, are kept per project and day inside the consumer's inbox transaction
+// (a redelivery or a re-sent hour never counts twice; an older revision never replaces a newer one), and
+// GET /api/v1/projects/:project/usage answers the owner, admins and staff (not developers, viewers or
+// strangers) with network.project-usage-result@1: daily rows, totals, the recorded quotas with what
+// their window used, errors by code and the sampled failures with their trace ids.
+// node test/project-usage.test.js
+const assert = require('assert');
+const crypto = require('crypto');
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const http = require('http');
+const express = require('express');
+const jwt = require('jsonwebtoken');
+const { ids, validate } = require('openvibe-contracts');
+const { signDeliveryHeaders } = require('openvibe-sdk/events');
+const { initDb } = require('../server/db/database');
+const subjects = require('../server/identity/subjects');
+const { NotificationService } = require('../server/notifications/notification-service');
+const { createEventsConsumer, TOPICS } = require('../server/notifications/events-consumer');
+const { createProjectUsage, parseQuery } = require('../server/developer/usage');
+const { topicsFrom } = require('../scripts/subscribe-events');
+
+const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'ov-project-usage-'));
+const quiet = { log: console.log, warn: console.warn };
+console.log = () => {}; console.warn = () => {};
+const db = initDb(path.join(dir, 'network.db'));
+db.prepare(`INSERT INTO users (id, username, password_hash, role) VALUES
+ (10, 'owner', 'x', 'user'), (11, 'admin', 'x', 'user'), (12, 'dev', 'x', 'user'), (13, 'viewer', 'x', 'user'), (14, 'stranger', 'x', 'user'), (15, 'staff', 'x', 'admin')`).run();
+const sid = (id) => subjects.ensureUserSubject(db, db.prepare('SELECT * FROM users WHERE id = ?').get(id));
+
+const keys = crypto.generateKeyPairSync('rsa', { modulusLength: 2048, publicKeyEncoding: { type: 'spki', format: 'pem' }, privateKeyEncoding: { type: 'pkcs8', format: 'pem' } });
+const ISSUER = 'https://openvibe.network';
+const SECRET = 'u'.repeat(40);
+const app = express();
+app.locals.db = db;
+app.locals.config = { baseUrl: ISSUER, jwt: { issuer: ISSUER, accessTokenExpiry: '1h' }, developer: { sandboxAllowance: '' } };
+app.locals.privateKey = keys.privateKey;
+app.locals.publicKey = keys.publicKey;
+const projectUsage = createProjectUsage(db, { log: { warn() {} } });
+const consumer = createEventsConsumer({ db, notifications: new NotificationService(db), secrets: SECRET, projectUsage });
+app.use('/internal/events', consumer.router);
+app.use('/api/v1/projects', require('../server/developer/routes').router());
+const server = http.createServer(app);
+const token = (id) => jwt.sign({ sub: id, id }, keys.privateKey, { algorithm: 'RS256', issuer: ISSUER, expiresIn: '1h' });
+const T = { owner: token(10), admin: token(11), dev: token(12), viewer: token(13), stranger: token(14), staff: token(15) };
+
+const HOUR = 60 * 60 * 1000;
+const thisHour = Math.floor(Date.now() / HOUR) * HOUR;
+const yesterday = thisHour - 24 * HOUR;
+const TRACE = '4bf92f3577b34da6a3ce929d0e0e4736';
+const JOB = 'job_01JAB2C3D4E5F6G7H8J9K0MNPR';
+
+(async () => {
+ await new Promise((r) => server.listen(0, '127.0.0.1', r));
+ const base = `http://127.0.0.1:${server.address().port}`;
+ const api = async (who, method, p, body) => {
+ const r = await fetch(`${base}/api/v1/projects${p}`, { method, headers: { authorization: `Bearer ${T[who]}`, ...(body ? { 'content-type': 'application/json' } : {}) }, body: body ? JSON.stringify(body) : undefined });
+ const text = await r.text();
+ return { status: r.status, body: text ? JSON.parse(text) : null, text };
+ };
+ try {
+ // ── A project with an admin, a developer and a viewer; staff record two quotas ──
+ let r = await api('owner', 'POST', '', { name: 'Usage demo' });
+ assert.strictEqual(r.status, 201);
+ const P = r.body.id;
+ for (const [who, role] of [['admin', 'admin'], ['dev', 'developer'], ['viewer', 'viewer']]) {
+ assert.strictEqual((await api('owner', 'POST', `/${P}/members`, { username: who, role })).status, 201);
+ }
+ assert.strictEqual((await api('staff', 'PUT', `/${P}/quotas/tools.job.create`, { limit: 100, window: 'day', unit: 'jobs' })).status, 200);
+ assert.strictEqual((await api('staff', 'PUT', `/${P}/quotas/events.app.publish`, { limit: 120, window: 'minute', unit: 'events' })).status, 200);
+ assert.strictEqual((await api('staff', 'PUT', `/${P}/quotas/media.object.upload`, { limit: 1000000, window: 'month', unit: 'bytes' })).status, 200);
+
+ // ── Nothing yet: an empty, valid answer ──
+ r = await api('owner', 'GET', `/${P}/usage`);
+ assert.strictEqual(r.status, 200);
+ assert.ok(validate('network.project-usage-result@1', r.body).valid, JSON.stringify(validate('network.project-usage-result@1', r.body).errors));
+ assert.deepStrictEqual([r.body.last_recorded_at, r.body.daily.length, r.body.errors.total], [null, 0, 0]);
+ assert.match(r.body.freshness, /No usage/);
+
+ // ── The consumer subscribes to both rollups, and records them ──
+ assert.ok(TOPICS.includes('tools.usage.recorded') && TOPICS.includes('events.usage.recorded'));
+ assert.deepStrictEqual(topicsFrom(['--topic', 'tools.usage.recorded', '--topic', 'events.usage.recorded']), ['tools.usage.recorded', 'events.usage.recorded']);
+ const rollup = (svc, payload, over = {}) => ({
+ event_id: ids.newId('event'), event_type: `${svc}.usage.recorded`, version: 1, source: svc, actor: { type: 'service', id: svc },
+ timestamp: new Date().toISOString(), subject: { type: 'project', id: payload.project_id }, visibility: 'internal', priority: 'low', payload, ...over,
+ });
+ const hour = (start, extra) => ({ project_id: P, env: 'production', unit: 'jobs', window: 'hour', window_start: new Date(start).toISOString(), window_end: new Date(start + HOUR).toISOString(), ...extra });
+ const jobs = rollup('tools', hour(thisHour, {
+ capability: 'tools.job.create', dimension: 'img.process', quantity: 40, errors: 2, error_codes: { 'tools.job.failed': 1, 'tools.job.timeout': 1 },
+ samples: [{ at: new Date(thisHour + 60000).toISOString(), code: 'tools.job.timeout', status: 504, trace_id: TRACE, ref: JOB }],
+ }));
+ // Over HTTP, signed: the route the Events subscription delivers to.
+ const raw = JSON.stringify({ event: jobs, seq: 1 });
+ const res = await fetch(`${base}/internal/events`, { method: 'POST', headers: { 'content-type': 'application/json', ...signDeliveryHeaders(raw, SECRET) }, body: raw });
+ assert.deepStrictEqual([res.status, (await res.json()).outcome], [200, 'recorded']);
+ assert.strictEqual(consumer.apply(jobs).duplicate, true, 'a redelivery records nothing twice');
+ // The same hour re-sent with new totals replaces it; an older revision never does.
+ const resent = rollup('tools', { ...jobs.payload, quantity: 42, errors: 3, error_codes: { 'tools.job.failed': 2, 'tools.job.timeout': 1 }, revision: 2 });
+ assert.strictEqual(consumer.apply(resent).outcome, 'recorded');
+ assert.strictEqual(consumer.apply(rollup('tools', { ...jobs.payload, quantity: 1, revision: 1 })).outcome, 'ignored:stale');
+ assert.strictEqual(consumer.apply(rollup('tools', hour(yesterday, { capability: 'tools.tool.run', dimension: 'image-resize', quantity: 5, errors: 0 }))).outcome, 'recorded');
+ assert.strictEqual(consumer.apply(rollup('tools', hour(thisHour - HOUR, { capability: 'tools.job.create', dimension: 'img.process', env: 'sandbox', quantity: 7, errors: 0 }))).outcome, 'recorded');
+ const published = rollup('events', hour(thisHour, {
+ capability: 'events.app.publish', unit: 'events', env: 'sandbox', quantity: 1800, errors: 61, error_codes: { 'events.quota_exceeded': 60, 'events.type_not_allowed': 1 },
+ samples: [{ at: new Date(thisHour + 120000).toISOString(), code: 'events.quota_exceeded', status: 429, trace_id: TRACE }],
+ }));
+ assert.strictEqual(consumer.apply(published).outcome, 'recorded');
+
+ // What is not a project's rollup is ignored.
+ const other = await api('stranger', 'POST', '', { name: 'Someone else' });
+ const ignored = [
+ [rollup('tools', { ...jobs.payload, project_id: `prj_${ids.ulid()}` }), 'ignored:project'],
+ [rollup('events', jobs.payload, { source: 'tools' }), 'ignored:source'],
+ [rollup('tools', { ...jobs.payload, owner: { type: 'user', id: sid(10) } }), 'ignored:payload'],
+ [rollup('tools', { ...jobs.payload, window_end: new Date(thisHour + 2 * HOUR).toISOString() }), 'ignored:window'],
+ [rollup('tools', jobs.payload, { subject: { type: 'project', id: other.body.id } }), 'ignored:subject'],
+ [rollup('tools', { ...jobs.payload, window_start: new Date(thisHour + 3 * HOUR).toISOString(), window_end: new Date(thisHour + 4 * HOUR).toISOString() }), 'ignored:future'],
+ ];
+ for (const [e, outcome] of ignored) assert.strictEqual(consumer.apply(e).outcome, outcome, outcome);
+ assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM notifications').get().n, 0, 'usage notifies nobody');
+
+ // ── The dashboard's answer ──
+ r = await api('owner', 'GET', `/${P}/usage?days=7`);
+ assert.strictEqual(r.status, 200);
+ const v = validate('network.project-usage-result@1', r.body);
+ assert.ok(v.valid, JSON.stringify(v.errors));
+ const u = r.body;
+ assert.deepStrictEqual([u.env, u.range.days, u.range.to], ['all', 7, new Date().toISOString().slice(0, 10)]);
+ assert.ok(u.last_recorded_at);
+ const today = new Date(thisHour).toISOString().slice(0, 10);
+ const row = (cap, env) => u.daily.find((d) => d.capability === cap && d.env === env && d.day === today);
+ assert.deepStrictEqual([row('tools.job.create', 'production').quantity, row('tools.job.create', 'production').errors, row('tools.job.create', 'production').dimension], [42, 3, 'img.process'], 'the re-sent hour replaced the first');
+ assert.deepStrictEqual([row('events.app.publish', 'sandbox').quantity, row('events.app.publish', 'sandbox').dimension], [1800, null]);
+ assert.ok(u.daily.some((d) => d.capability === 'tools.tool.run' && d.day === new Date(yesterday).toISOString().slice(0, 10)));
+ const total = (cap, env) => u.totals.find((x) => x.capability === cap && x.env === env);
+ assert.deepStrictEqual([total('tools.job.create', 'production').quantity, total('tools.tool.run', 'production').quantity, total('events.app.publish', 'sandbox').errors], [42, 5, 61]);
+ const quota = (cap) => u.quotas.find((x) => x.capability === cap);
+ const sandboxToday = new Date(thisHour - HOUR).toISOString().slice(0, 10) === today ? 7 : 0;
+ assert.deepStrictEqual([quota('tools.job.create').used, quota('tools.job.create').remaining, quota('tools.job.create').enforced_by], [42 + sandboxToday, 100 - 42 - sandboxToday, 'openvibe.tools'], 'a day quota counts today, every environment');
+ assert.strictEqual(quota('tools.job.create').window_start, `${today}T00:00:00.000Z`);
+ assert.deepStrictEqual([quota('events.app.publish').used, quota('events.app.publish').remaining], [null, null]);
+ assert.match(quota('events.app.publish').note, /minute window/);
+ assert.strictEqual(quota('media.object.upload').used, null);
+ assert.match(quota('media.object.upload').note, /no service reports/);
+ assert.strictEqual(u.errors.total, 64);
+ assert.deepStrictEqual(u.errors.by_code[0], { service: 'events', capability: 'events.app.publish', code: 'events.quota_exceeded', count: 60 });
+ assert.deepStrictEqual(u.errors.recent.map((e) => [e.service, e.code, e.status, e.trace_id, e.ref]), [
+ ['events', 'events.quota_exceeded', 429, TRACE, null],
+ ['tools', 'tools.job.timeout', 504, TRACE, JOB],
+ ], 'newest first; the re-sent hour did not duplicate its sample');
+ for (const who of [10, 11, 12]) assert.ok(!r.text.includes(sid(who)), 'no subject id in the answer');
+
+ // Filters.
+ r = await api('owner', 'GET', `/${P}/usage?env=sandbox`);
+ assert.ok(r.body.daily.every((d) => d.env === 'sandbox') && r.body.errors.recent.every((e) => e.env === 'sandbox'));
+ for (const q of ['days=0', 'days=91', 'days=abc', 'env=staging']) {
+ r = await api('owner', 'GET', `/${P}/usage?${q}`);
+ assert.deepStrictEqual([r.status, r.body.code], [422, 'usage.invalid'], q);
+ }
+ assert.deepStrictEqual(parseQuery({}), { days: 30, env: 'all' });
+
+ // ── Who may read it: owner, admins and staff; not developers, viewers or strangers ──
+ for (const [who, status] of [['admin', 200], ['staff', 200], ['dev', 403], ['viewer', 403], ['stranger', 404]]) {
+ r = await api(who, 'GET', `/${P}/usage`);
+ assert.strictEqual(r.status, status, who);
+ }
+ r = await fetch(`${base}/api/v1/projects/${P}/usage`);
+ assert.strictEqual(r.status, 401);
+
+ // ── Retention: windows and samples of the past go; the daily numbers stay ──
+ const later = createProjectUsage(db, { now: () => Date.now() + 40 * 24 * HOUR });
+ later.prune();
+ assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM dev_usage_windows').get().n, 0);
+ assert.strictEqual(db.prepare('SELECT COUNT(*) AS n FROM dev_usage_errors').get().n, 0);
+ assert.ok(db.prepare('SELECT COUNT(*) AS n FROM dev_usage_daily').get().n > 0);
+ } finally {
+ server.close();
+ fs.rmSync(dir, { recursive: true, force: true });
+ console.log = quiet.log; console.warn = quiet.warn;
+ }
+ console.log('project usage: all checks passed');
+})().catch((e) => { console.log = quiet.log; console.error(e); process.exit(1); });