openvibers/openvibe.ai Commits
Your own endpoint: a private GPU rent, billed through Balance (#338)
Changes
@@@ -159,6 +159,33 @@ server {proxy_pass http://127.0.0.1:4700; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header CF-Connecting-IP $remote_addr; proxy_set_header X-Forwarded-Proto $scheme;}++ # ── Your own endpoint (the Exchange remake, build order 7): its public status and the person API that++ # starts, lists and stops a private GPU rent, plus the OpenAI-compatible proxy to it. The person's own++ # Network user token guards it in the app (server/api/own-endpoints.js); a service token is refused.++ location = /api/v1/exchange/endpoints {++ limit_except GET { deny all; }++ proxy_pass http://127.0.0.1:4700;++ proxy_set_header Host $host;++ proxy_set_header X-Real-IP $remote_addr;++ proxy_set_header X-Forwarded-For $remote_addr;++ proxy_set_header CF-Connecting-IP $remote_addr;++ }++ location ^~ /api/v1/own-endpoints/ {++ limit_req zone=ovai_apps burst=10 nodelay;++ limit_req_status 429;++ limit_except GET POST { deny all; }++ client_max_body_size 12m;++ proxy_pass http://127.0.0.1:4700;++ proxy_http_version 1.1;++ proxy_set_header Host $host;++ proxy_set_header X-Real-IP $remote_addr;++ proxy_set_header X-Forwarded-For $remote_addr;++ proxy_set_header CF-Connecting-IP $remote_addr;++ proxy_set_header X-Forwarded-Proto $scheme;++ proxy_buffering off;++ proxy_read_timeout 300s;++ }++# ── Unknown API paths: a problem document, never a page; the API itself stays host-local ──location /api/ {default_type application/problem+json;@@@ -97,6 +97,23 @@ server {proxy_set_header CF-Connecting-IP $remote_addr;proxy_set_header X-Forwarded-Proto $scheme;}++ # Your own endpoint (the Exchange remake, build order 7): the person API and the OpenAI-compatible++ # proxy to their private GPU. Only the person's own Network user token reaches it (a service token is++ # refused in the app); answers may stream, so buffering is off and the read timeout covers one answer.++ location ^~ /api/v1/own-endpoints/ {++ limit_req zone=ovai_apps burst=10 nodelay;++ limit_req_status 429;++ limit_except GET POST { deny all; }++ proxy_pass http://127.0.0.1:4700;++ proxy_http_version 1.1;++ proxy_set_header Host $host;++ proxy_set_header X-Real-IP $remote_addr;++ proxy_set_header X-Forwarded-For $remote_addr;++ proxy_set_header CF-Connecting-IP $remote_addr;++ proxy_set_header X-Forwarded-Proto $scheme;++ proxy_buffering off;++ proxy_read_timeout 300s;++ }location = /openapi.json {limit_except GET { deny all; }proxy_pass http://127.0.0.1:4700;@@@ -88,6 +88,38 @@ state, the measured cold start (from the launch to the first answered request),with its price (the hour and the second), endpoint, cold start and money. "Start" and "Stop" write a request row;the reconciler (one implementation, the same one the service runs) does the work.++## Your own endpoint (the private rent)++++The Exchange's product line 6, "your own endpoint", is the same provisioner with a private rent on top++(`server/gpu/endpoints.js`, migration `0039_own_endpoints.sql`):++++- a person picks one of the models above, or brings their own Hugging Face `.gguf` (and an optional++ LoRA), from OpenVibe.Exchange's `/own` page or the person API (`/api/v1/own-endpoints`);++- the reconciler starts it on the cheapest offer that fits, exactly like the shared models, and **never++ registers it as an offer**: the router and the catalogue never see it, and the GPU is reached only++ through AI's proxy at `/api/v1/own-endpoints/:id/v1` with the owner's own Network user token;++- the person's Balance pays for it. Before the marketplace is called, a Billing hold is taken for the++ worst case the rent may cost (the reservation); when it stops, the exact seconds are settled:++ `usage.input_tokens` carries the milliseconds, `retail.version` is `gpu-second@1`, and the per-request++ remainder carries the last nano so the Billing charge equals the GPU ledger's `spent_nano` to the++ nano. Billing is the only place the person is charged; the `gpu_spend` rows are the owner's cost++ record, never the person's charge;++- the idle timer is the endpoint's own (`idle_minutes`, 1-120, default the owner's `idle_minutes`),++ measured from the last proxied request (`gpu_instances.last_used_at`). The maximum run is 120 minutes++ (or the owner's `max_runtime_minutes` when lower), so a start's hold is bounded;++- the owner's monthly cap is checked before the hold: a new start that would pass it is refused with a++ plain reason and nothing is rented.++++The feature is off while **any** of the three is missing, and the public++`GET /api/v1/exchange/endpoints` answers the plain reason the Exchange page shows:++++1. `ai.gpus` `vast.key` or `runpod.key` (the owner's GPU provider key),++2. `ai.gpus` `monthly_cap` (the hard monthly cap),++3. Balance billing (`AI_CREDITS` and Billing), because an endpoint is paid for per second.++++The first two are `x-owner-input` fields (group `GPUs`, `x-required-for: "own endpoints"`) so the admin++panel's Setup tab lists them with what they turn on and where to get them.++## The tests`test/gpu.test.js` runs the provisioner against the recorded marketplace answers@@@ -102,6 +134,17 @@ the reconciler (one implementation, the same one the service runs) does the work- an instance with our label and no row of ours is found and given back,- a destroy that keeps failing is tried again every pass and the operator is alerted.++`test/own-endpoints.test.js` covers the private rent with the same recorded marketplace, a fake clock, a++local fake Billing and a canned GPU answer:++++- off without a GPU key or a cap, and off without Balance billing, each with its plain reason;++- the Billing hold is taken before the marketplace is called;++- a private rent is never registered as an offer, and the owner's OpenAI-compatible call is proxied;++- a service token cannot manage or call a person's endpoint;++- the endpoint's own idle timer scales it to zero, and the settle charge equals the GPU `spent_nano`;++- the monthly cap refuses a new start with a plain reason and moves no money;++- own weights and a LoRA produce the right container command, and a non-Hugging-Face address is refused.++## What the owner has to check before the first launchThe weights URL, the image and the container command are data in `server/gpu/catalog.js` - they arennew file mode 100644@@@ -0,0 +1,48 @@++-- phase: expand++-- Your own endpoint (the Exchange remake, product line 6 / build order 7): a private GPU endpoint a person++-- rents for a chosen catalog model or their own Hugging Face weights. The instance is the existing++-- gpu_instances row (the GPU provisioner rents, health-checks, scales it to zero and enforces the owner's++-- monthly cap); this table is the person's endpoint on top of it: who owns it, what it serves, the Billing++-- hold that pays for the seconds, and the public proxy its owner calls. Additive only: nothing existing is++-- changed, gpu_instances gains two nullable columns.++CREATE TABLE IF NOT EXISTS own_endpoints (++ id text COLLATE "C" PRIMARY KEY, -- oep_<ULID>++ subject text COLLATE "C" NOT NULL, -- usr_<ULID>: the owner and the payer++ status text COLLATE "C" NOT NULL CHECK (status IN ('requested','starting','running','stopping','stopped','refused','error')),++ source text COLLATE "C" NOT NULL CHECK (source IN ('catalog','weights')),++ catalog_key text COLLATE "C", -- a model the provisioner knows, when source=catalog++ display_name text COLLATE "C" NOT NULL,++ weights_repo text COLLATE "C",++ weights_file text COLLATE "C",++ lora_repo text COLLATE "C",++ lora_file text COLLATE "C",++ context_window integer,++ idle_minutes integer NOT NULL,++ max_runtime_minutes integer NOT NULL,++ marketplace text COLLATE "C",++ price_per_hour text COLLATE "C", -- the marketplace's own decimal price++ hourly_nano bigint, -- that price as exact nano-USD per hour++ reserved_nano bigint NOT NULL DEFAULT 0, -- the budget held from Billing for this start++ instance_id text COLLATE "C", -- gpu_instances.id++ endpoint text COLLATE "C", -- the rented instance's own URL (never public)++ url text COLLATE "C", -- this endpoint's public URL (the AI proxy)++ billing_hold_id text COLLATE "C",++ billing_settled_at timestamptz,++ charged_nano bigint NOT NULL DEFAULT 0, -- what the person's Balance paid for this start++ month text COLLATE "C" NOT NULL,++ error text COLLATE "C", -- a plain reason when refused or stopped++ created_at timestamptz NOT NULL,++ updated_at timestamptz NOT NULL,++ started_at timestamptz,++ ready_at timestamptz,++ stopped_at timestamptz++);++CREATE INDEX IF NOT EXISTS own_endpoints_subject ON own_endpoints (subject, created_at DESC);++CREATE INDEX IF NOT EXISTS own_endpoints_status ON own_endpoints (status);++CREATE INDEX IF NOT EXISTS own_endpoints_month ON own_endpoints (month);++++-- The instance knows the endpoint it serves, so the reconciler can tell a private rent from a shared model++-- offer (a private rent is never registered for the router) and give it the endpoint's own idle timer.++ALTER TABLE gpu_instances ADD COLUMN IF NOT EXISTS own_endpoint_id text COLLATE "C";++ALTER TABLE gpu_instances ADD COLUMN IF NOT EXISTS idle_minutes integer;++CREATE INDEX IF NOT EXISTS gpu_instances_own ON gpu_instances (own_endpoint_id) WHERE own_endpoint_id IS NOT NULL;@@@ -61,6 +61,14 @@ const TABLES = [// removed when the account goes, so it stops receiving traffic too.{ table: 'community_endpoint_offers', subject: 'owner_subject', file: 'community-endpoint-offers.json', order: 'created_at',columns: ['id', 'name', 'base_url', 'models', 'hours', 'concurrency', 'price_in', 'price_out', 'audience', 'group_id', 'created_at', 'updated_at'] },++ // Your own endpoint (the Exchange remake, build order 7): a private GPU rent. It is the person's own++ // data while the account lives, so it goes in the export. On deletion the row is anonymized at once and++ // marked stopping, and the next reconciler pass gives the GPU back and lets Billing settle the seconds;++ // the money rows themselves are kept like the ledger.++ { table: 'own_endpoints', subject: 'subject', file: 'own-endpoints.json', order: 'created_at',++ columns: ['id', 'status', 'source', 'catalog_key', 'display_name', 'weights_repo', 'weights_file', 'lora_repo', 'lora_file',++ 'idle_minutes', 'max_runtime_minutes', 'marketplace', 'price_per_hour', 'charged_nano', 'created_at', 'ready_at', 'stopped_at'],++ erase: { keep: 'the GPU rent and its Billing hold are money records; extraErase anonymizes them and stops the rent so Billing can settle' } },// The person's own quota (server/quota.js): a limit that only means something while the account lives.{ table: 'quotas', subject: 'scope_id', value: (usr) => `user:${usr}`, file: null },// What the person used, counted against their allowance: the spend ledger, kept.@@@ -79,6 +87,13 @@ async function extraExport(db, subject) {/** The person's runs go, with the rows that hang under them (the requests, the citations, the cached answers). */async function extraErase(t, subjects, counts) {++ // A private GPU rent is stopped and anonymized now; its Billing hold and money stay until the reconciler++ // gives the instance back and settles the seconds (the table's erase is a keep for that reason).++ counts.add(counts.erased, 'own_endpoints', await t.exec(`UPDATE own_endpoints++ SET status = CASE WHEN status IN ('requested','starting','running','stopping') THEN 'stopping' ELSE status END,++ subject = NULL, display_name = 'deleted endpoint', weights_repo = NULL, weights_file = NULL,++ lora_repo = NULL, lora_file = NULL, url = NULL, updated_at = now()++ WHERE subject = ANY($1::text[])`, [subjects]));// Plan E5: the free-tier counters. A shared device counter is kept while another account still uses the device;// when this person was the last one, the device counter and its velocity bucket go too.const hashes = (await t.many('SELECT DISTINCT device_hash FROM abuse_devices WHERE subject = ANY($1::text[])', [subjects])).map((r) => r.device_hash);@@@ -35,7 +35,7 @@ function memo(fn, ttlMs, clock) {};}--function exchangeRouter({ db, registry, pool, quotas = null, env = process.env, clock = { now: () => Date.now() }, config = null, presets, receipts, auth, log = console }) {++function exchangeRouter({ db, registry, pool, quotas = null, env = process.env, clock = { now: () => Date.now() }, config = null, presets, receipts, auth, log = console, ownEndpoints = null }) {const catalog = createCatalog({ db, registry, pool, env, clock, config });const usage = createUsage({ db, clock });// The price book's log: what a price was, what it became and when it was read.@@@ -158,6 +158,15 @@ function exchangeRouter({ db, registry, pool, quotas = null, env = process.env,} catch (err) { next(err); }});++ // Your own endpoint: whether it is on and, when it is off, the plain reason. Public: the Exchange page++ // reads it before anyone signs in, and nothing here is per person.++ r.get('/api/v1/exchange/endpoints', publicCache(30), (req, res) => {++ res.json(ownEndpoints ? ownEndpoints.status() : {++ enabled: false, reason: 'The GPU provider is not set up yet, so no endpoint can start.', reasons: ['The GPU provider is not set up yet, so no endpoint can start.'],++ billing: false, models: [],++ });++ });++r.get('/api/v1/exchange/models/:slug', publicCache(30), async (req, res, next) => {try {const out = await all();nnew file mode 100644@@@ -0,0 +1,96 @@++'use strict';++/**++ * Your own endpoint's person API (the Exchange remake, build order 7).++ *++ * GET /api/v1/own-endpoints the caller's endpoints++ * POST /api/v1/own-endpoints { model } or { weights: { repo, file, lora_repo?, lora_file? }, idle_minutes? }++ * GET /api/v1/own-endpoints/:id one endpoint++ * POST /api/v1/own-endpoints/:id/stop give it back++ * * /api/v1/own-endpoints/:id/v1/* the OpenAI-compatible proxy to the rented GPU++ *++ * Only the person themselves: a Network user token (audience openvibe.network or openvibe.ai) whose++ * subject is a usr_…; a service, app, module or node token is refused whatever capabilities it holds.++ * The endpoint is private: the rented GPU is never registered as an offer, so the router and the++ * catalogue never see it, and the proxy is the only way in.++ */++const express = require('express');++const { Readable } = require('node:stream');++const { http } = require('openvibe-contracts');++const { verifyUserToken } = require('openvibe-sdk/auth');++const { AiError, sendError } = require('../util');++const { bearer } = require('../auth');++++const SUBJECT_RE = /^usr_[0-9A-HJKMNP-TV-Z]{26}$/;++++function createOwnEndpointsRouter({ endpoints, auth, config, clock = { now: () => Date.now() }, log = console }) {++ const r = express.Router();++++ /** A person, never a service. Sets req.ownPerson = { subject, token }. */++ async function guard(req, res, next) {++ try {++ const token = bearer(req);++ if (!token) return http.sendProblem(res, 401, 'token.missing', { detail: 'sign in to manage your endpoint', ctx: req.ov });++ let service = null;++ try { service = await auth.verify(token); } catch { service = { ok: false }; }++ if (service && service.ok) return http.sendProblem(res, 403, 'capability.denied', { detail: "a service token cannot manage a person's endpoint", ctx: req.ov });++ let claims;++ try {++ claims = await verifyUserToken(token, {++ ...(config.networkPublicKey ? { publicKey: config.networkPublicKey } : { jwks: config.networkJwksUrl }),++ issuer: config.issuer, audience: ['openvibe.network', 'openvibe.ai'], now: clock.now(), log,++ });++ } catch (err) {++ log.warn && log.warn('[own-endpoint] person token refused: ' + String((err && (err.code || err.message)) || err));++ return http.sendProblem(res, 401, 'token.invalid', { detail: 'your sign-in could not be verified', ctx: req.ov });++ }++ const subject = claims && (claims.subject_id || claims.sub);++ if (!SUBJECT_RE.test(String(subject || ''))) return http.sendProblem(res, 403, 'capability.denied', { detail: 'only a person may manage an endpoint', ctx: req.ov });++ req.ownPerson = { subject: String(subject), token };++ return next();++ } catch (err) {++ return sendError(res, err, req.ov, log);++ }++ }++++ const wrap = (fn) => async (req, res) => {++ try { const out = await fn(req, res); if (out !== undefined) res.json(out); } catch (err) { sendError(res, err, req.ov, log); }++ };++++ r.use('/api/v1/own-endpoints', guard);++++ r.get('/api/v1/own-endpoints', wrap(async (req) => ({ endpoints: await endpoints.list(req.ownPerson.subject), status: endpoints.status() })));++++ r.post('/api/v1/own-endpoints', wrap(async (req, res) => {++ const body = req.body && typeof req.body === 'object' ? req.body : {};++ const out = await endpoints.create({ subject: req.ownPerson.subject, body });++ res.status(201).json({ endpoint: out });++ }));++++ r.get('/api/v1/own-endpoints/:id', wrap(async (req) => ({ endpoint: await endpoints.get(req.ownPerson.subject, req.params.id) })));++++ r.post('/api/v1/own-endpoints/:id/stop', wrap(async (req) => ({ endpoint: await endpoints.stop(req.ownPerson.subject, req.params.id) })));++++ // The OpenAI-compatible proxy: /v1/models and /v1/chat/completions (streaming included). The person's++ // token decides who may come in; the rented GPU only ever sees the body.++ r.all('/api/v1/own-endpoints/:id/v1/*', async (req, res) => {++ try {++ const id = req.params.id;++ const rest = req.params[0] || '';++ if (!id) throw new AiError(404, 'own_endpoint.not_found', 'no such endpoint');++ const path = '/v1/' + rest;++ const out = await endpoints.forward({ subject: req.ownPerson.subject, id, method: req.method, path, body: req.body });++ res.status(out.status);++ res.set('Content-Type', out.contentType);++ res.set('Cache-Control', 'no-store');++ if (out.body && typeof out.body.pipe === 'function') return out.body.pipe(res);++ if (out.body && typeof out.body.getReader === 'function') return Readable.fromWeb(out.body).pipe(res);++ return res.end();++ } catch (err) {++ return sendError(res, err, req.ov, log);++ }++ });++++ return r;++}++++module.exports = { createOwnEndpointsRouter };@@@ -24,7 +24,7 @@ const pkg = require('../package.json');const { adminRoutes } = require('openvibe-shared/config');--function createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env = process.env, clock = { now: () => Date.now() }, fetchImpl = globalThis.fetch, log = console, credentials = null, preferences = null, grants = null, vault = null, settings = null, benchEvents = null, runBudgets = null, memory = null, memoryImport = null, memoryServices = null, memorySharing = null, community = null, endpoints = null, webhooks = null, presets = null, receipts = null, balance = null, gpus = null, abuse = null, priceBook = null }) {++function createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env = process.env, clock = { now: () => Date.now() }, fetchImpl = globalThis.fetch, log = console, credentials = null, preferences = null, grants = null, vault = null, settings = null, benchEvents = null, runBudgets = null, memory = null, memoryImport = null, memoryServices = null, memorySharing = null, community = null, endpoints = null, webhooks = null, presets = null, receipts = null, balance = null, gpus = null, abuse = null, priceBook = null, ownEndpoints = null }) {const app = express();// /.well-known/agent.json (openvibe-shared/agent-layer, SEO-2): what this site serves, for agents.require('openvibe-shared/agent-layer').mount(app, { descriptor: require('./agent-descriptor') });@@@ -116,7 +116,10 @@ function createApp({ config, db, registry, pool, quotas, cache, runs, auth, keysopenapi.mount(app);// OpenVibe.Exchange's read API (server/api/exchange.js): the public catalog and aggregate usage. Public on// purpose — the site and anyone comparing prices reads the same numbers the router ranks on.-- app.use(require('./api/exchange').exchangeRouter({ config, db, registry, pool, quotas, env, clock, presets, receipts, auth, log }));++ app.use(require('./api/exchange').exchangeRouter({ config, db, registry, pool, quotas, env, clock, presets, receipts, auth, log, ownEndpoints }));++ // A person's own endpoint (the Exchange remake, product line 6): the private rent's person API and its++ // OpenAI-compatible proxy. Only the owner's own Network user token reaches it; a service token never does.++ if (ownEndpoints) app.use(require('./api/own-endpoints').createOwnEndpointsRouter({ endpoints: ownEndpoints, auth, config, clock, log }));// The public seller API: a standalone OpenAI-compatible server registered by its own https address with the// seller's developer key (the design's SDK gap). Its offers join the community router's trust and ledger.if (endpoints) app.use(require('./api/seller').sellerRouter({ endpoints, community, webhooks, auth, log }));nnew file mode 100644@@@ -0,0 +1,497 @@++'use strict';++/**++ * Your own endpoint (the Exchange remake, product line 6 / build order 7).++ *++ * A person picks one of the GPU provisioner's models or brings their own Hugging Face weights (and an++ * optional LoRA), and gets a private OpenAI-compatible endpoint on a rented GPU. The rent is the++ * existing gpu_instances row: the same cheapest-fit search, the same owner monthly cap, the same++ * marketplace and the same scale-to-zero sweep (server/gpu/provisioner.js). What this module adds:++ *++ * - the endpoint row (migration 0039) and its owner;++ * - a Billing hold before the marketplace is called, of the worst case the rent may cost, and a++ * settle of the exact seconds when it stops. Billing is the only ledger: no money is written here;++ * - privacy: a private rent is never registered as an offer, so the router and the catalogue never++ * see it. Its owner reaches it through AI's proxy, and only with their own Network token;++ * - the public status the Exchange page reads, with the plain reason the feature is off.++ *++ * The feature is off while ANY of these is true, and every one of them is an owner setting:++ *++ * 1. no GPU marketplace key (ai.gpus vast.key / runpod.key),++ * 2. no monthly GPU cap (ai.gpus monthly_cap),++ * 3. Balance billing is not on (AI_CREDITS + Billing), because an endpoint must be paid for.++ *++ * Nothing here reaches a GPU provider in a test: the marketplace calls go through the provisioner's++ * injected fetch (the recorded answers in test/fixtures/gpu), and Billing through the Credits manager's.++ */++const { ids } = require('openvibe-contracts');++const { AiError, sha256 } = require('../util');++const money = require('./money');++const catalog = require('./catalog');++++const ACTIVE = ['requested', 'starting', 'running', 'stopping'];++const LIVE = ['starting', 'running', 'stopping'];++const MAX_ACTIVE = 2;++const MAX_IDLE_MINUTES = 120;++const MAX_RUNTIME_MINUTES = 120;++const WEIGHTS_REPO = /^[A-Za-z0-9][A-Za-z0-9._-]{0,99}\/[A-Za-z0-9][A-Za-z0-9._-]{0,99}$/;++const WEIGHTS_FILE = /^[A-Za-z0-9][A-Za-z0-9._ -]{0,180}\.gguf$/;++const BILLING_OFF = 'Balance billing is not on yet, so an endpoint cannot be paid for. The owner must turn on Balance first.';++++function plainGpuReason(reason) {++ if (reason === 'no Vast.ai or RunPod API key') return 'The owner has not added a GPU provider key (Vast.ai or RunPod) yet, so no endpoint can start.';++ if (reason === 'no monthly cap') return 'The owner has not set a monthly GPU cap yet, so no endpoint can start.';++ return 'The GPU provider is not set up yet, so no endpoint can start.';++}++++/** A short, plain sentence for why a rent stopped. The status line is a sentence, never a code. */++function reasonText(reason) {++ const r = String(reason || '');++ if (r === 'idle') return 'Stopped after it sat idle; it scales to zero when nobody uses it.';++ if (r === 'the owner stopped it') return 'Stopped.';++ if (r === 'the reservation ran out') return 'Stopped: the time it was paid for ran out.';++ if (r === 'the endpoint stopped answering' || r === 'the endpoint never answered') return 'Stopped because the GPU stopped answering.';++ if (/^the instance /.test(r)) return 'Stopped because the GPU instance ' + r.slice('the instance '.length) + '.';++ return 'Stopped.';++}++++function clamp(value, min, max, fallback) {++ if (value == null || value === '') return fallback;++ const n = Number(value);++ if (!Number.isFinite(n)) return fallback;++ return Math.max(min, Math.min(max, Math.trunc(n)));++}++++/**++ * The module. `credits` is AI's Credits facade (server/index.js): `credits.enabled` is the live++ * AI_CREDITS flag and `credits.manager` the client to Billing's hold/settle API.++ */++function createOwnEndpoints({ db, gpus, credits, config = null, clock = { now: () => Date.now() }, fetchImpl = globalThis.fetch, log = console, models = catalog.list() } = {}) {++ const now = () => clock.now();++ const at = (ms) => new Date(ms).toISOString();++ const byKey = new Map((models || []).map((m) => [m.key, m]));++++ function manager() {++ return credits && credits.enabled && credits.manager ? credits.manager : null;++ }++++ function gpuConfig() {++ return gpus && typeof gpus.config === 'function' ? gpus.config() : { enabled: false, reason: 'no Vast.ai or RunPod API key', idle_minutes: 10, max_runtime_minutes: 720, vast_key: '', runpod_key: '' };++ }++++ /** The public status the Exchange page reads: on, or off with a plain sentence. */++ function status() {++ const cfg = gpuConfig();++ const billing = Boolean(manager());++ const reasons = [];++ if (!cfg.enabled) reasons.push(plainGpuReason(cfg.reason));++ if (!billing) reasons.push(BILLING_OFF);++ const marketplace = cfg.vast_key ? 'vast' : (cfg.runpod_key ? 'runpod' : null);++ return {++ enabled: Boolean(cfg.enabled && billing),++ reason: reasons.join(' '),++ reasons,++ marketplace,++ billing,++ models: (models || []).map((m) => ({++ key: m.key, display_name: m.display_name, context_window: m.context_window,++ vram_mb: m.requirements && m.requirements.vram_mb,++ gpu_names: (m.requirements && m.requirements.gpu_names) || [],++ })),++ idle_minutes: cfg.idle_minutes,++ max_idle_minutes: MAX_IDLE_MINUTES,++ max_runtime_minutes: Math.min(cfg.max_runtime_minutes || MAX_RUNTIME_MINUTES, MAX_RUNTIME_MINUTES),++ public_base: publicBase(),++ };++ }++++ function publicBase() {++ const base = String((config && config.baseUrl) || '').replace(/\/+$/, '');++ return (base || 'https://ai.openvibe.services') + '/api/v1/own-endpoints';++ }++++ async function rowById(id) {++ return (await db.prepare('SELECT * FROM own_endpoints WHERE id = ?').get(String(id))) || null;++ }++++ async function ownedRow(subject, id) {++ const row = await rowById(id);++ if (!row || row.subject !== String(subject)) throw new AiError(404, 'own_endpoint.not_found', 'no such endpoint');++ return row;++ }++++ async function liveFor(id, statuses = LIVE) {++ return (await db.prepare(`SELECT * FROM gpu_instances WHERE own_endpoint_id = ? AND status IN (${statuses.map(() => '?').join(', ')}) ORDER BY created_at DESC LIMIT 1`)++ .get(String(id), ...statuses)) || null;++ }++++ /** What the Exchange page and its owner see. */++ function describe(row, instance = null) {++ if (!row) return null;++ return {++ id: row.id,++ status: row.status,++ source: row.source,++ display_name: row.display_name,++ catalog_key: row.catalog_key,++ weights: row.source === 'weights' ? { repo: row.weights_repo, file: row.weights_file, lora_repo: row.lora_repo, lora_file: row.lora_file } : null,++ idle_minutes: Number(row.idle_minutes),++ max_runtime_minutes: Number(row.max_runtime_minutes),++ marketplace: row.marketplace,++ price_per_hour: row.price_per_hour,++ hourly_nano: row.hourly_nano == null ? null : String(row.hourly_nano),++ reserved_nano: String(row.reserved_nano == null ? '0' : row.reserved_nano),++ charged_nano: String(row.charged_nano == null ? '0' : row.charged_nano),++ url: row.url || (publicBase() + '/' + row.id + '/v1'),++ model: row.id,++ instance: instance ? {++ status: instance.status, gpu_name: instance.gpu_name, region: instance.region,++ cold_start_ms: instance.cold_start_ms == null ? null : String(instance.cold_start_ms),++ started_at: instance.started_at, ready_at: instance.ready_at,++ last_used_at: instance.last_used_at,++ } : null,++ month: row.month,++ error: row.error,++ reason: row.error || null,++ created_at: row.created_at,++ ready_at: row.ready_at,++ stopped_at: row.stopped_at,++ };++ }++++ async function describeWithInstance(row) {++ return describe(row, row.instance_id ? await db.prepare('SELECT * FROM gpu_instances WHERE id = ?').get(row.instance_id) : null);++ }++++ async function list(subject) {++ const rows = await db.prepare('SELECT * FROM own_endpoints WHERE subject = ? ORDER BY created_at DESC').all(String(subject));++ const out = [];++ for (const row of rows || []) out.push(await describeWithInstance(row));++ return out;++ }++++ async function get(subject, id) {++ return describeWithInstance(await ownedRow(subject, id));++ }++++ /** The spec the provisioner serves for this endpoint. A private rent is never registered. */++ function specFor(row) {++ if (!row) return null;++ const cfg = gpuConfig();++ const marketplace = cfg.vast_key ? 'vast' : (cfg.runpod_key ? 'runpod' : 'vast');++ const key = 'own-' + row.id;++ if (row.source === 'catalog') {++ const base = byKey.get(row.catalog_key) || catalog.find(row.catalog_key);++ if (!base) return null;++ return {++ ...base,++ key,++ provider_key: key,++ model_key: key,++ marketplace,++ display_name: 'Your ' + (base.display_name || base.key),++ };++ }++ const weightsUrl = 'https://huggingface.co/' + row.weights_repo + '/resolve/main/' + encodeURIComponent(row.weights_file).replace(/%2F/g, '/');++ const loraUrl = row.lora_repo ? 'https://huggingface.co/' + row.lora_repo + '/resolve/main/' + encodeURIComponent(row.lora_file).replace(/%2F/g, '/') : null;++ const ctx = Number(row.context_window) || 8192;++ const command = 'mkdir -p /models'++ + ' && curl -fL --retry 3 -o /models/' + row.weights_file + ' ' + weightsUrl++ + (loraUrl ? ' && curl -fL --retry 3 -o /models/' + row.lora_file + ' ' + loraUrl : '')++ + ' && exec /app/llama-server --model /models/' + row.weights_file++ + ' --host 0.0.0.0 --port 8000 --alias ' + key++ + ' --ctx-size ' + ctx + ' --n-gpu-layers 999'++ + (loraUrl ? ' --lora /models/' + row.lora_file : '');++ return {++ key,++ provider_key: key,++ model_key: key,++ marketplace,++ display_name: row.display_name || 'Your model',++ context_window: ctx,++ throughput_tokens_per_hour: 180000,++ requirements: { vram_mb: 22000, disk_gb: 60, gpu_names: ['RTX 3090', 'RTX 4090'], num_gpus: 1, min_reliability: 0.98 },++ runtime: { kind: 'llama.cpp', image: 'ghcr.io/ggml-org/llama.cpp:server-cuda', port: 8000, weights: { repo: row.weights_repo, file: row.weights_file }, command },++ };++ }++++ /** Validate what a person asked for and write the request row. The reconciler starts it. */++ async function create({ subject, body = {} }) {++ const st = status();++ if (!st.enabled) throw new AiError(409, 'own_endpoint.off', st.reason || 'Your own endpoint is off.');++ const cfg = gpuConfig();++ const active = await db.prepare(`SELECT COUNT(*) AS n FROM own_endpoints WHERE subject = ? AND status IN (${ACTIVE.map(() => '?').join(', ')})`)++ .get(String(subject), ...ACTIVE);++ if (Number(active && active.n) >= MAX_ACTIVE) throw new AiError(409, 'own_endpoint.limit', 'You already have ' + MAX_ACTIVE + ' endpoints running or starting. Stop one first.');++++ const wantsCatalog = body.model != null && body.model !== '';++ const wantsWeights = body.weights && typeof body.weights === 'object';++ if (wantsCatalog === wantsWeights) throw new AiError(422, 'input.invalid', 'name one model, or one set of weights');++ let source, spec = null, display = '', weights = null;++ if (wantsCatalog) {++ spec = byKey.get(String(body.model)) || catalog.find(String(body.model));++ if (!spec) throw new AiError(422, 'input.invalid', 'no GPU model is called ' + String(body.model).slice(0, 80));++ source = 'catalog';++ display = spec.display_name;++ } else {++ const repo = String(body.weights.repo || '').trim();++ const file = String(body.weights.file || '').trim();++ const loraRepo = body.weights.lora_repo == null ? '' : String(body.weights.lora_repo).trim();++ const loraFile = body.weights.lora_file == null ? '' : String(body.weights.lora_file).trim();++ if (!WEIGHTS_REPO.test(repo)) throw new AiError(422, 'input.invalid', 'weights.repo must be a Hugging Face repository, like "user/model-GGUF"');++ if (!WEIGHTS_FILE.test(file)) throw new AiError(422, 'input.invalid', 'weights.file must be a .gguf file name from that repository');++ if (Boolean(loraRepo) !== Boolean(loraFile)) throw new AiError(422, 'input.invalid', 'name both the LoRA repository and its .gguf file, or neither');++ if (loraRepo && !WEIGHTS_REPO.test(loraRepo)) throw new AiError(422, 'input.invalid', 'weights.lora_repo must be a Hugging Face repository');++ if (loraFile && !WEIGHTS_FILE.test(loraFile)) throw new AiError(422, 'input.invalid', 'weights.lora_file must be a .gguf file name');++ source = 'weights';++ display = String(body.display_name || repo.split('/').pop() || 'Your model').slice(0, 120);++ weights = { repo, file, loraRepo, loraFile };++ }++ const idleMinutes = clamp(body.idle_minutes, 1, MAX_IDLE_MINUTES, cfg.idle_minutes || 10);++ const maxRuntime = Math.min(cfg.max_runtime_minutes || MAX_RUNTIME_MINUTES, MAX_RUNTIME_MINUTES);++ const t = now();++ const id = 'oep_' + ids.ulid();++ await db.prepare(`INSERT INTO own_endpoints (id, subject, status, source, catalog_key, display_name, weights_repo, weights_file,++ lora_repo, lora_file, context_window, idle_minutes, max_runtime_minutes, month, created_at, updated_at)++ VALUES (?, ?, 'requested', ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(++ id, String(subject), source, spec ? spec.key : null, display,++ weights ? weights.repo : null, weights ? weights.file : null, weights ? weights.loraRepo || null : null, weights ? weights.loraFile || null : null,++ spec ? spec.context_window : 8192, idleMinutes, maxRuntime, at(t).slice(0, 7), at(t), at(t));++ return describe(await rowById(id));++ }++++ /** The Billing hold of a start: the worst case the rent may cost, taken before the marketplace. */++ async function takeHold(row, { hourly, reserved, maxMinutes, marketplace, offerPrice }) {++ const mgr = manager();++ if (!mgr) throw new AiError(409, 'own_endpoint.billing_off', BILLING_OFF);++ const key = 'gpu-hold:' + row.id;++ const fingerprint = sha256([row.id, row.subject, String(hourly), String(reserved), String(maxMinutes), marketplace].join(':'));++ const hold = await mgr.hold({++ idempotency_key: key,++ subject: { type: 'user', id: row.subject },++ max_nano_usd: String(reserved),++ rate_version: 'gpu-second@1',++ request_fingerprint: fingerprint,++ service: 'ai',++ }, 'own-' + row.id);++ const holdId = hold && (hold.hold_id || hold.id);++ if (!holdId) throw new AiError(503, 'billing.unavailable', 'Billing did not answer with a hold');++ await db.prepare(`UPDATE own_endpoints SET billing_hold_id = ?, marketplace = ?, price_per_hour = ?, hourly_nano = ?,++ reserved_nano = ?, max_runtime_minutes = ?, updated_at = ? WHERE id = ?`)++ .run(String(holdId), marketplace, String(offerPrice || ''), String(hourly), String(reserved), Number(maxMinutes), at(now()), row.id);++ return holdId;++ }++++ /** Start a requested endpoint: the provisioner searches, checks the cap, takes the hold, rents. */++ async function start(row) {++ const cfg = gpuConfig();++ if (!cfg.enabled) {++ await refuse(row.id, plainGpuReason(cfg.reason));++ return { ok: false, reason: cfg.reason };++ }++ if (!manager()) {++ await refuse(row.id, BILLING_OFF);++ return { ok: false, reason: BILLING_OFF };++ }++ const spec = specFor(row);++ if (!spec) {++ await refuse(row.id, 'The model this endpoint asked for is no longer available.');++ return { ok: false, reason: 'no such model' };++ }++ const res = await gpus.launchSpec(spec, {++ actor: 'user:' + row.subject,++ ownEndpointId: row.id,++ idleMinutes: row.idle_minutes,++ maxRuntimeMinutes: row.max_runtime_minutes,++ beforeCreate: async ({ offer, hourly, reserved, maxMinutes, marketplace }) => {++ await takeHold(row, { hourly, reserved, maxMinutes, marketplace, offerPrice: String(offer.price_per_hour || '') });++ },++ });++ if (!res.ok) {++ await db.prepare('UPDATE own_endpoints SET status = ?, error = ?, updated_at = ? WHERE id = ?')++ .run('refused', String(res.reason || 'it could not start').slice(0, 400), at(now()), row.id);++ await releaseHold(row, 'the endpoint did not start');++ return res;++ }++ const inst = res.instance;++ await db.prepare(`UPDATE own_endpoints SET status = 'starting', instance_id = ?, error = NULL, started_at = ?, updated_at = ? WHERE id = ?`)++ .run(String(inst.id), at(now()), at(now()), row.id);++ return res;++ }++++ async function refuse(id, reason) {++ await db.prepare("UPDATE own_endpoints SET status = 'refused', error = ?, updated_at = ? WHERE id = ?").run(String(reason).slice(0, 400), at(now()), id);++ }++++ async function releaseHold(row, reason) {++ if (!row || !row.billing_hold_id || row.billing_settled_at) return;++ const mgr = manager();++ if (!mgr) return;++ try {++ await mgr.release(row.billing_hold_id, 'own-' + row.id);++ await db.prepare('UPDATE own_endpoints SET billing_settled_at = ?, updated_at = ? WHERE id = ?').run(at(now()), at(now()), row.id);++ } catch (err) {++ log.warn && log.warn('[own-endpoint] releasing the hold of ' + row.id + ' failed: ' + String(err && err.message));++ }++ }++++ /**++ * Settle the exact seconds of a finished rent against the person's Balance. Billing's charge is++ * ceil(units x rate / 1e6) + per_request; the seconds go in as milliseconds and the per-request++ * remainder carries the last nano, so the charge equals what the GPU ledger says it cost.++ */++ async function settle(row) {++ if (!row || row.billing_settled_at) return null;++ const inst = row.instance_id ? await db.prepare('SELECT * FROM gpu_instances WHERE id = ?').get(row.instance_id) : null;++ const spent = BigInt((inst && inst.spent_nano) || 0);++ const mgr = manager();++ if (!mgr || !row.billing_hold_id) {++ await db.prepare('UPDATE own_endpoints SET billing_settled_at = ?, updated_at = ? WHERE id = ?').run(at(now()), at(now()), row.id);++ return null;++ }++ if (spent <= 0n) {++ await releaseHold(row, 'it never ran');++ return null;++ }++ const startMs = Date.parse((inst && inst.started_at) || row.started_at || at(now()));++ const endMs = Date.parse((inst && inst.destroyed_at) || at(now()));++ const ms = Number.isFinite(startMs) && Number.isFinite(endMs) ? Math.max(0, Math.min(Number.MAX_SAFE_INTEGER, endMs - startMs)) : 0;++ let rate = 0n;++ let perRequest = spent;++ if (ms > 0) {++ rate = (spent * 1000000n) / BigInt(ms);++ const part = (BigInt(ms) * rate + 999999n) / 1000000n;++ perRequest = spent > part ? spent - part : 0n;++ }++ const body = {++ idempotency_key: 'gpu-settle:' + row.id,++ usage: { input_tokens: ms, cached_read_tokens: 0, cache_write_tokens: 0, output_tokens: 0 },++ retail: {++ version: 'gpu-second@1',++ nano_usd_per_m: { input: rate.toString(), cached_read: '0', cache_write: '0', output: '0' },++ per_request_nano_usd: perRequest.toString(),++ },++ supply_cost_nano_usd: spent.toString(),++ byok: false,++ attempts: [],++ };++ try {++ await mgr.settle(row.billing_hold_id, body, 'own-' + row.id);++ } catch (err) {++ if (typeof mgr.enqueueSettle === 'function') {++ await mgr.enqueueSettle({ holdId: row.billing_hold_id, requestId: 'own-' + row.id, body, error: err });++ } else {++ throw err;++ }++ }++ await db.prepare('UPDATE own_endpoints SET billing_settled_at = ?, charged_nano = ?, updated_at = ? WHERE id = ?')++ .run(at(now()), spent.toString(), at(now()), row.id);++ return spent.toString();++ }++++ /** The provisioner calls this when a rent becomes ready: the public URL is the AI proxy. */++ async function ready(instanceRow, _endpoint) {++ if (!instanceRow || !instanceRow.own_endpoint_id) return;++ const url = publicBase() + '/' + instanceRow.own_endpoint_id + '/v1';++ await db.prepare("UPDATE own_endpoints SET status = 'running', endpoint = ?, url = ?, ready_at = ?, error = NULL, updated_at = ? WHERE id = ? AND status IN ('starting','requested')")++ .run(String(_endpoint || ''), url, at(now()), at(now()), instanceRow.own_endpoint_id);++ }++++ /** The provisioner calls this when a private rent is given back (idle, stop, death, time). */++ async function afterDestroy(instanceRow, reason) {++ if (!instanceRow || !instanceRow.own_endpoint_id) return;++ const row = await rowById(instanceRow.own_endpoint_id);++ if (!row) return;++ const fresh = await db.prepare('SELECT * FROM gpu_instances WHERE id = ?').get(instanceRow.id);++ const failed = /answer|died|state|never|dead|error|refused/i.test(String(reason || ''));++ await db.prepare('UPDATE own_endpoints SET status = ?, stopped_at = ?, error = ?, updated_at = ? WHERE id = ?')++ .run(failed ? 'error' : 'stopped', at(now()), reasonText(reason), at(now()), row.id);++ await settle(row).catch((err) => log.warn && log.warn('[own-endpoint] the settle of ' + row.id + ' failed: ' + String(err && err.message)));++ return fresh;++ }++++ /** Stop an endpoint: the reconciler gives the rent back, then the hold settles. */++ async function stop(subject, id) {++ const row = await ownedRow(subject, id);++ if (!ACTIVE.includes(row.status)) return describeWithInstance(row);++ if (row.status === 'requested' && !row.instance_id) {++ // it was never started: nothing to give back, and any hold goes with it++ await releaseHold(row, 'it never started');++ await db.prepare("UPDATE own_endpoints SET status = 'stopped', stopped_at = ?, error = NULL, updated_at = ? WHERE id = ?").run(at(now()), at(now()), row.id);++ return describeWithInstance(await rowById(row.id));++ }++ await db.prepare("UPDATE own_endpoints SET status = 'stopping', updated_at = ? WHERE id = ?").run(at(now()), row.id);++ return describeWithInstance(await rowById(row.id));++ }++++ /** One pass: start the requested, stop the stopping, and settle a rent whose death the provisioner saw. */++ async function tick() {++ const rows = await db.prepare('SELECT * FROM own_endpoints ORDER BY created_at').all();++ let started = 0;++ let stopped = 0;++ let settled = 0;++ for (const row of rows || []) {++ if (row.status === 'requested') {++ const res = await start(row).catch(async (err) => {++ await refuse(row.id, String((err && err.message) || err));++ return { ok: false, reason: String((err && err.message) || err) };++ });++ if (res && res.ok) started += 1;++ continue;++ }++ if (row.status === 'stopping') {++ const inst = await liveFor(row.id);++ if (inst) {++ await gpus.destroyEndpoint(row.id, 'the owner stopped it', { actor: 'user:' + row.subject }).catch(() => null);++ stopped += 1;++ }++ const still = await liveFor(row.id);++ if (!still) {++ await db.prepare("UPDATE own_endpoints SET status = 'stopped', stopped_at = ?, updated_at = ? WHERE id = ?").run(at(now()), at(now()), row.id);++ await settle(await rowById(row.id)).catch(() => null);++ }++ continue;++ }++ if (row.instance_id && !row.billing_settled_at) {++ const inst = await db.prepare('SELECT * FROM gpu_instances WHERE id = ?').get(row.instance_id);++ if (inst && (inst.status === 'destroyed' || inst.status === 'dead' || inst.status === 'refused')) {++ await afterDestroy(inst, inst.error || 'the rent ended').catch(() => null);++ settled += 1;++ }++ }++ }++ return { started, stopped, settled };++ }++++ /**++ * Forward one request from an endpoint's owner to the rented GPU. The caller's token, headers and++ * cookies never reach the instance: it gets the body and nothing else. Marking the use is what++ * keeps the scale-to-zero timer honest.++ */++ async function forward({ subject, id, method, path, body, signal = null }) {++ const row = await ownedRow(subject, id);++ if (row.status !== 'running') throw new AiError(409, 'own_endpoint.not_running', row.error || 'This endpoint is not running.');++ if (!['GET', 'POST'].includes(String(method).toUpperCase())) throw new AiError(405, 'own_endpoint.method', 'only GET and POST reach an endpoint');++ if (!/^\/v1\/[A-Za-z0-9/_.-]+$/.test(String(path || '')) || String(path).includes('..')) throw new AiError(422, 'own_endpoint.path', 'that is not an OpenAI-compatible path');++ const inst = await db.prepare('SELECT * FROM gpu_instances WHERE id = ?').get(row.instance_id);++ if (!inst || inst.status !== 'running' || !inst.endpoint) throw new AiError(409, 'own_endpoint.not_running', 'The GPU is not answering yet.');++ await gpus.noteUse(inst.provider_key);++ const url = String(inst.endpoint).replace(/\/+$/, '') + path;++ const headers = { Accept: 'application/json' };++ let payload;++ if (body !== undefined && String(method).toUpperCase() !== 'GET') {++ headers['Content-Type'] = 'application/json';++ payload = typeof body === 'string' ? body : JSON.stringify(body);++ }++ let res;++ try {++ res = await fetchImpl(url, { method: String(method).toUpperCase(), headers, ...(payload !== undefined ? { body: payload } : {}), signal: signal || AbortSignal.timeout(300_000) });++ } catch (err) {++ throw new AiError(502, 'own_endpoint.unreachable', 'The GPU did not answer: ' + String((err && err.message) || err).slice(0, 200));++ }++ return { status: res.status, contentType: res.headers.get('content-type') || 'application/json', body: res.body };++ }++++ return { status, create, list, get, stop, tick, specFor, ready, afterDestroy, forward, describe, settle };++}++++module.exports = { createOwnEndpoints, MAX_ACTIVE, MAX_IDLE_MINUTES, MAX_RUNTIME_MINUTES, BILLING_OFF, plainGpuReason, reasonText };@@@ -85,6 +85,7 @@ function createGpuProvisioner({env = process.env,log = console,models = catalog.list(),++ ownEndpoints = null,} = {}) {const now = () => clock.now();const iso = (ms) => new Date(ms).toISOString();@@@ -229,16 +230,35 @@ function createGpuProvisioner({return false;}++ /** The live private rent of an endpoint, if one is still up. */++ async function liveInstanceForEndpoint(ownEndpointId) {++ if (!ownEndpointId) return null;++ const row = await db.prepare('SELECT * FROM gpu_instances WHERE own_endpoint_id = ? AND status IN (?, ?, ?) ORDER BY created_at DESC LIMIT 1')++ .get(String(ownEndpointId), 'starting', 'running', 'stopping');++ return row || null;++ }++/*** Start an instance for a model: the cheapest offer that fits, inside the cap.* Returns { ok, reason } - it never throws for the owner's reasons (off, no offer, cap).*/-- async function launch(modelKey, { actor = 'service:ai' } = {}) {-- const cfg = config();-- if (!cfg.enabled) return { ok: false, code: 'gpu.off', reason: cfg.reason };++ async function launch(modelKey, opts = {}) {const spec = models.find((m) => m.key === modelKey) || catalog.find(modelKey);if (!spec) throw new AiError(404, 'ai.not_found', 'no such GPU model');-- const existing = await liveInstance(spec.key);++ return launchSpec(spec, opts);++ }++++ /**++ * Start an instance for a spec. A shared model offer registers itself with the router; a person's own++ * endpoint (ownEndpointId set) never does: it stays private, and its owner reaches it through AI.++ *++ * beforeCreate({ offer, hourly, reserved, maxMinutes, marketplace }) the endpoint's Billing hold, if++ * any, is taken here - before the marketplace is called, so nothing runs that the person cannot pay.++ */++ async function launchSpec(spec, { actor = 'service:ai', ownEndpointId = null, idleMinutes = null, maxRuntimeMinutes = null, beforeCreate = null } = {}) {++ const cfg = config();++ if (!cfg.enabled) return { ok: false, code: 'gpu.off', reason: cfg.reason };++ const existing = ownEndpointId ? await liveInstanceForEndpoint(ownEndpointId) : await liveInstance(spec.key);if (existing) return { ok: true, instance: describe(existing), note: 'already running' };const adapter = adapterFor(spec.marketplace, cfg);if (!adapter) return { ok: false, code: 'gpu.no_key', reason: 'no ' + spec.marketplace + ' API key' };@@@ -278,12 +298,24 @@ function createGpuProvisioner({if (affordableHours < 1n) {return { ok: false, code: 'gpu.cap', reason: 'the monthly cap leaves less than an hour of GPU time (' + money.usdText(cap - sum.committed_nano) + ' left)' };}-- const maxMinutes = Math.min(cfg.max_runtime_minutes, Number(affordableHours) * 60);++ const affordableMinutes = Number(affordableHours) * 60;++ const requestedMax = maxRuntimeMinutes == null ? affordableMinutes : Math.max(1, Math.min(affordableMinutes, Math.trunc(Number(maxRuntimeMinutes) || 0)));++ const maxMinutes = Math.min(cfg.max_runtime_minutes, requestedMax);const reserved = hourly * BigInt(Math.ceil(maxMinutes / 60));if (sum.committed_nano + reserved > cap) {return { ok: false, code: 'gpu.cap', reason: 'the monthly cap leaves no room for ' + spec.display_name };}++ // 2b. the payer's hold, for a person's own endpoint: taken before the marketplace is called, so an++ // instance never runs that the person cannot pay for.++ if (beforeCreate) {++ try {++ await beforeCreate({ offer, hourly, reserved, maxMinutes, marketplace: spec.marketplace });++ } catch (err) {++ return { ok: false, code: (err && err.code) || 'gpu.billing', reason: String((err && err.message) || err).slice(0, 300) };++ }++ }++// 3. the row comes first. The marketplace is told the label of a row that already exists and// holds the reservation, so an instance we paid for is never an orphan: the sweep finds it by// the label and gives it back when no live row of ours owns it.@@@ -291,10 +323,12 @@ function createGpuProvisioner({const t = now();await db.prepare(`INSERT INTO gpu_instances (id, model_key, provider_key, marketplace, offer_id, remote_id, status, endpoint, month,gpu_name, gpu_ram_mb, num_gpus, disk_gb, reliability, region, price_per_hour, hourly_nano, reserved_nano, spent_nano,-- max_runtime_minutes, started_at, charged_at, destroy_failures, label, cold_start_ms, health_failures, created_at, updated_at)++ max_runtime_minutes, started_at, charged_at, destroy_failures, label, cold_start_ms, health_failures, created_at, updated_at,++ own_endpoint_id, idle_minutes)VALUES (@id, @model_key, @provider_key, @marketplace, @offer_id, NULL, 'starting', NULL, @month,@gpu_name, @gpu_ram_mb, @num_gpus, @disk_gb, @reliability, @region, @price_per_hour, @hourly_nano, @reserved_nano, 0,-- @max_runtime_minutes, @started_at, @charged_at, 0, @label, NULL, 0, @created_at, @updated_at)`)++ @max_runtime_minutes, @started_at, @charged_at, 0, @label, NULL, 0, @created_at, @updated_at,++ @own_endpoint_id, @idle_minutes)`).run({id, model_key: spec.key, provider_key: spec.provider_key, marketplace: spec.marketplace,offer_id: String(offer.id), month: monthOf(t),@@@ -304,10 +338,12 @@ function createGpuProvisioner({price_per_hour: String(offer.price_per_hour), hourly_nano: hourly.toString(), reserved_nano: reserved.toString(),max_runtime_minutes: maxMinutes, started_at: iso(t), charged_at: iso(t), label: id,created_at: iso(t), updated_at: iso(t),++ own_endpoint_id: ownEndpointId, idle_minutes: idleMinutes == null ? null : clampInt(idleMinutes, 1, 1440, cfg.idle_minutes),});await audit(actor, 'gpu.instance.requested', 'gpu_instance', id, {model: spec.key, marketplace: spec.marketplace, offer_id: String(offer.id), hourly_nano: hourly.toString(),reserved_nano: reserved.toString(), max_runtime_minutes: maxMinutes, label: id,++ ...(ownEndpointId ? { own_endpoint_id: ownEndpointId } : {}),});// 4. rent it. The marketplace is given the label of the row above, so the instance can always be@@@ -435,7 +471,12 @@ function createGpuProvisioner({await charge(fresh || row, t, reason);await db.prepare("UPDATE gpu_instances SET status = 'destroyed', destroyed_at = ?, endpoint = NULL, destroy_failures = 0, updated_at = ? WHERE id = ?").run(iso(t), iso(t), row.id);-- await disableProvider(row.provider_key, reason);++ if (row.own_endpoint_id && ownEndpoints && typeof ownEndpoints.afterDestroy === 'function') {++ // the person's endpoint is over: settle what it cost and say why it stopped++ await ownEndpoints.afterDestroy(fresh || row, reason).catch((err) => log.warn && log.warn('[gpu] the endpoint settle failed: ' + String(err && err.message)));++ } else {++ await disableProvider(row.provider_key, reason);++ }await audit(actor, 'gpu.instance.destroy', 'gpu_instance', row.id, { reason, model: row.model_key });return { ok: true, reason };}@@@ -546,8 +587,12 @@ function createGpuProvisioner({/** One pass over the live instances: the reservation, the idle timer, the health. */async function reconcileInstance(row, cfg) {-- const spec = models.find((m) => m.key === row.model_key) || catalog.find(row.model_key);++ const priv = Boolean(row.own_endpoint_id);++ const spec = priv++ ? (ownEndpoints && typeof ownEndpoints.specFor === 'function' ? ownEndpoints.specFor(row) : null)++ : (models.find((m) => m.key === row.model_key) || catalog.find(row.model_key));if (!spec) return { changed: false };++ const idleMinutes = Number(row.idle_minutes) > 0 ? Number(row.idle_minutes) : cfg.idle_minutes;const t = now();const started = toMs(row.started_at) || t;@@@ -562,12 +607,12 @@ function createGpuProvisioner({// 2. the idle timer: the offer scales to zero (unless it is inside the warm window). The clock runs// from the moment the instance was ready, not from the moment it was rented: an instance that is still-- // starting is never idle.++ // starting is never idle. A person's own endpoint has its own idle timer.if (row.status === 'running') {const lastUsed = toMs(row.last_used_at) || toMs(row.ready_at) || started;-- if (t - lastUsed >= cfg.idle_minutes * 60_000 && !inWarmWindow(cfg.warm_window, t)) {++ if (t - lastUsed >= idleMinutes * 60_000 && !inWarmWindow(cfg.warm_window, t)) {await destroy({ ...row, charged_at: row.charged_at }, 'idle');-- return { changed: true, action: 'scaled_to_zero', reason: 'idle for ' + cfg.idle_minutes + ' minutes' };++ return { changed: true, action: 'scaled_to_zero', reason: 'idle for ' + idleMinutes + ' minutes' };}}@@@ -587,7 +632,7 @@ function createGpuProvisioner({const state = String((info && info.state) || 'unknown').toLowerCase();if (DEAD_STATES.includes(state)) {await destroy({ ...row, charged_at: row.charged_at }, 'the instance ' + state);-- await request(spec.key, 'launch', 'service:ai');++ if (!priv) await request(spec.key, 'launch', 'service:ai');return { changed: true, action: 'died', reason: 'the instance ' + state };}const endpoint = info && info.endpoint ? String(info.endpoint) : '';@@@ -598,7 +643,7 @@ function createGpuProvisioner({.run(failures, iso(t), iso(t), row.id);if (failures >= HEALTH_FAILURES) {await destroy({ ...row, charged_at: row.charged_at }, 'the endpoint never answered');-- await request(spec.key, 'launch', 'service:ai');++ if (!priv) await request(spec.key, 'launch', 'service:ai');return { changed: true, action: 'died', reason: 'the endpoint never answered' };}return { changed: false, health_failures: failures };@@@ -607,7 +652,11 @@ function createGpuProvisioner({// the cold start: from the launch to the first answered requestawait db.prepare("UPDATE gpu_instances SET status = 'running', endpoint = ?, ready_at = ?, last_health_at = ?, health_failures = 0, cold_start_ms = ?, updated_at = ? WHERE id = ?").run(endpoint, iso(t), iso(t), String(Math.max(0, t - started)), iso(t), row.id);-- await register(spec, { ...row, endpoint });++ if (priv) {++ if (ownEndpoints && typeof ownEndpoints.ready === 'function') await ownEndpoints.ready({ ...row, endpoint }, endpoint);++ } else {++ await register(spec, { ...row, endpoint });++ }return { changed: true, action: 'ready' };}return { changed: false };@@@ -626,9 +675,10 @@ function createGpuProvisioner({await db.prepare('UPDATE gpu_instances SET health_failures = ?, last_health_at = ?, updated_at = ? WHERE id = ?').run(failures, iso(t), iso(t), row.id);if (failures < HEALTH_FAILURES) return { changed: false, health_failures: failures };-- // the instance died: the offer leaves the pool and a new one is rented++ // the instance died: a shared offer leaves the pool and a new one is rented; a person's own++ // endpoint stops (they can start it again when they want it).await destroy({ ...row, charged_at: row.charged_at }, 'the endpoint stopped answering');-- await request(spec.key, 'launch', 'service:ai');++ if (!priv) await request(spec.key, 'launch', 'service:ai');return { changed: true, action: 'died', reason: 'the endpoint stopped answering' };}return { changed: false };@@@ -670,6 +720,8 @@ function createGpuProvisioner({reserved_nano: String(toNano(row.reserved_nano)),spent_nano: String(toNano(row.spent_nano)),max_runtime_minutes: row.max_runtime_minutes,++ idle_minutes: row.idle_minutes == null ? null : Number(row.idle_minutes),++ own_endpoint_id: row.own_endpoint_id || null,label: row.label,cold_start_ms: row.cold_start_ms == null ? null : String(row.cold_start_ms),destroy_failures: Number(row.destroy_failures || 0),@@@ -778,16 +830,47 @@ function createGpuProvisioner({};}++ /** The private-endpoint hooks, attached once the endpoint module exists (server/index.js). */++ function attachEndpoints(hooks) {++ ownEndpoints = hooks || null;++ return ownEndpoints;++ }++++ /** The live rent of an endpoint (for its owner's proxy and its stop). */++ async function instanceForEndpoint(ownEndpointId) {++ return liveInstanceForEndpoint(ownEndpointId);++ }++++ /** A request reached an offer: the idle timer runs from now, not from the moment it became ready. */++ async function noteUse(providerKey) {++ if (!providerKey) return 0;++ const res = await db.prepare("UPDATE gpu_instances SET last_used_at = ?, updated_at = ? WHERE provider_key = ? AND status = 'running'")++ .run(iso(now()), iso(now()), String(providerKey));++ return Number(res && res.changes) || 0;++ }++++ /** Give an endpoint's rent back (its owner pressed stop); the afterDestroy hook settles the money. */++ async function destroyEndpoint(ownEndpointId, reason = 'the owner stopped it', { actor = 'service:ai' } = {}) {++ const row = await liveInstanceForEndpoint(ownEndpointId);++ if (!row) return { ok: true, note: 'not running' };++ return destroy(row, reason, { actor });++ }++return {state,tick,launch,++ attachEndpoints,++ launchSpec,request,config,moneySummary,sweepOrphans,destroyFailures,orphanCount,++ instanceForEndpoint,++ noteUse,++ destroyEndpoint,/** For the tests: the same thing the reconciler does with one instance. */reconcileInstance,inWarmWindow,@@@ -131,6 +131,11 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.nowstartSettlements() { return credits ? credits.startSettlements() : null; },stopSettlements() { return credits ? credits.stopSettlements() : undefined; },};++ // A person's own endpoint (the design's product line 6): a private rent on the same GPU provisioner, with a++ // Billing hold before it starts and the exact seconds settled when it stops. It is off until the owner has++ // a GPU key and a monthly cap AND Balance billing is on; the public status says which one is missing.++ const ownEndpoints = require('./gpu/endpoints').createOwnEndpoints({ db, gpus, credits: creditsRef, config, clock, fetchImpl, log });++ gpus.attachEndpoints(ownEndpoints);const runs = createRuns({ db, registry, engine, cache, quotas, config, clock, log, userModules, credentials, preferences, tiers, projectSecrets, pool, runBudgets, credits: creditsRef, community, presets, receipts, abuse, webhooks });await seed({ registry, quotas, config, env, db });// The seed creates or moves providers from the environment: publish their origins (the schema and every key's@@@ -168,7 +173,7 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.nowclientSecret: env.OV_OAUTH_CLIENT_SECRET, fetch: fetchImpl,}) : null;const benchEvents = createBenchEvents({ bench, accountData, accountSend, config, log, fetchImpl, clientSecret: env.OV_OAUTH_CLIENT_SECRET });-- const app = createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env, clock, fetchImpl, log, credentials, preferences, grants, vault, settings, benchEvents, runBudgets, memory, memoryImport, memoryServices, memorySharing, community, endpoints, webhooks, presets, receipts, balance, gpus, abuse, priceBook });++ const app = createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env, clock, fetchImpl, log, credentials, preferences, grants, vault, settings, benchEvents, runBudgets, memory, memoryImport, memoryServices, memorySharing, community, endpoints, webhooks, presets, receipts, balance, gpus, abuse, priceBook, ownEndpoints });let jwks = null;let keyLoaded = Promise.resolve(null);if (!keys.pinned && config.nodeEnv !== 'test') {@@@ -196,7 +201,9 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.nowlet gpuReconciler = null;if (listen && config.nodeEnv !== 'test') {gpuReconciler = setInterval(() => {-- gpus.tick().catch((err) => log.warn('[gpu] the reconciler failed: ' + err.message));++ ownEndpoints.tick()++ .then(() => gpus.tick())++ .catch((err) => log.warn('[gpu] the reconciler failed: ' + err.message));}, 60 * 1000);gpuReconciler.unref?.();}@@@ -266,7 +273,7 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.nowif (!givenDb) await db.close();}-- return { config, db, registry, pool, quotas, tiers, cache, fetcher, engine, runs, keys, keyLoaded, auth, app, server, bench, benchEvents, memory, memoryJobs, memoryServices, memorySharing, priceBook, credits: creditsRef, settings, vault, community, endpoints, webhooks, gpus, abuse, close };++ return { config, db, registry, pool, quotas, tiers, cache, fetcher, engine, runs, keys, keyLoaded, auth, app, server, bench, benchEvents, memory, memoryJobs, memoryServices, memorySharing, priceBook, credits: creditsRef, settings, vault, community, endpoints, webhooks, gpus, abuse, ownEndpoints, close };}if (require.main === module) {@@@ -398,6 +398,47 @@ route('get', '/exchange/catalog', {models: arrayOf(exchangeModel),} }) },});++route('get', '/exchange/endpoints', {++ operationId: 'exchangeOwnEndpointStatus', summary: 'Your own endpoint: whether it is on, and the plain reason when it is not', tags: ['exchange'], security: [],++ description: 'Public. Your own endpoint (the Exchange remake, product line 6) is a private GPU rent: a chosen model or your own Hugging Face weights, reached only by its owner, scaled to zero when idle, and paid for by the owner\'s Balance. It is off while the owner has no GPU provider key (ai.gpus vast.key or runpod.key), no monthly GPU cap (ai.gpus monthly_cap), or Balance billing is not on. `reason` is a plain sentence the Exchange page shows; `reasons` lists each missing piece. `models` is what a private endpoint may serve today.',++ responses: { 200: json('The status', { type: 'object', required: ['enabled', 'reason'], properties: {++ enabled: { type: 'boolean' },++ reason: str(),++ reasons: arrayOf(str()),++ billing: { type: 'boolean' },++ marketplace: nullable('string'),++ idle_minutes: integer(), max_idle_minutes: integer(), max_runtime_minutes: integer(),++ public_base: str(),++ models: arrayOf({ type: 'object', properties: { key: str(), display_name: str(), context_window: integer(), vram_mb: integer(), gpu_names: arrayOf(str()) } }),++ } }) },++});++route('get', '/own-endpoints', {++ operationId: 'listOwnEndpoints', summary: 'The caller\'s private endpoints', tags: ['exchange'], security: [{ bearer: [] }],++ description: 'Only the person themselves: a Network user token (audience openvibe.network or openvibe.ai) whose subject is usr_…. A service, app, module or node token is refused whatever capabilities it holds. A private endpoint is never registered as an offer, so no router and no catalogue sees it.',++ responses: { 200: json('The endpoints and the public status', { type: 'object', required: ['endpoints'], properties: { endpoints: arrayOf({ type: 'object' }), status: { type: 'object' } } }), ...errors([401, 403]) },++});++route('post', '/own-endpoints', {++ operationId: 'createOwnEndpoint', summary: 'Start a private endpoint on a chosen model or your own weights', tags: ['exchange'], security: [{ bearer: [] }],++ description: 'Only a person. Name `model` (a key from GET /exchange/endpoints) or `weights` ({ repo, file, lora_repo?, lora_file? }: a Hugging Face .gguf and an optional LoRA, fetched by the rented container). The endpoint is started by the reconciler on the cheapest fit, after a Billing hold of its worst case; a start that would pass the owner\'s monthly cap is refused with a plain reason, and nothing is rented. 409 own_endpoint.off carries the same reason the public status does.',++ requestBody: { required: true, content: { 'application/json': { schema: { type: 'object', properties: {++ model: str(), idle_minutes: integer({ minimum: 1, maximum: 120 }),++ weights: { type: 'object', required: ['repo', 'file'], properties: { repo: str(), file: str(), lora_repo: str(), lora_file: str() } },++ display_name: str(),++ } } } } },++ responses: { 201: json('The endpoint', { type: 'object', required: ['endpoint'], properties: { endpoint: { type: 'object' } } }), 409: problem, 422: problem, ...errors([401, 403]) },++});++route('get', '/own-endpoints/{id}', {++ operationId: 'getOwnEndpoint', summary: 'One of the caller\'s endpoints', tags: ['exchange'], security: [{ bearer: [] }],++ description: 'Only its owner. The status, the price an hour, what it has charged, the public URL while it runs, and the plain reason it stopped.',++ parameters: [path('id', str({ pattern: '^oep_[0-9A-HJKMNP-TV-Z]{26}$' }))],++ responses: { 200: json('The endpoint', { type: 'object', required: ['endpoint'], properties: { endpoint: { type: 'object' } } }), 404: problem, ...errors([401, 403]) },++});++route('post', '/own-endpoints/{id}/stop', {++ operationId: 'stopOwnEndpoint', summary: 'Give the private endpoint back', tags: ['exchange'], security: [{ bearer: [] }],++ description: 'Only its owner. The reconciler gives the rent back; Billing settles the seconds it ran, and the price stops.',++ parameters: [path('id', str({ pattern: '^oep_[0-9A-HJKMNP-TV-Z]{26}$' }))],++ responses: { 200: json('The endpoint', { type: 'object', required: ['endpoint'], properties: { endpoint: { type: 'object' } } }), 404: problem, ...errors([401, 403]) },++});route('get', '/exchange/models/{slug}', {operationId: 'exchangeModel', summary: 'One model and its offers', tags: ['exchange'], security: [],description: 'Public. The slug is "<org>/<model>", lowercased, e.g. deepseek/deepseek-flash.',@@@ -140,6 +140,14 @@ const GPU_SCHEMA = Object.freeze({'vast.key': {type: 'string', title: 'Vast.ai API key', 'x-group': GPU_GROUP, 'x-bind': 'vast.origin',description: "The owner's write-only Vast.ai API key, sealed to the origin above.",++ 'x-owner-input': {++ group: 'GPUs',++ unlocks: 'Lets OpenVibe rent a GPU on Vast.ai for OpenVibe GPUs and for a person’s own endpoint. Without it Vast.ai is never searched and nothing is rented there.',++ get_it_at: 'https://console.vast.ai/account/',++ get_it_how: 'Open Account → API Keys, create a key, and paste it here. It is stored locked and never shown again.',++ test: false,++ },++ 'x-required-for': 'own endpoints',},'runpod.origin': {type: 'string', title: 'RunPod origin', 'x-group': GPU_GROUP,@@@ -148,11 +156,27 @@ const GPU_SCHEMA = Object.freeze({'runpod.key': {type: 'string', title: 'RunPod API key', 'x-group': GPU_GROUP, 'x-bind': 'runpod.origin',description: "The owner's write-only RunPod API key, sealed to the origin above.",++ 'x-owner-input': {++ group: 'GPUs',++ unlocks: 'Lets OpenVibe rent a GPU on RunPod when RunPod is cheaper or Vast.ai has no fit. Without it RunPod is never searched and nothing is rented there.',++ get_it_at: 'https://www.runpod.io/console/user/settings',++ get_it_how: 'Open Settings → API Keys, create a key, and paste it here. It is stored locked and never shown again.',++ test: false,++ },++ 'x-required-for': 'own endpoints',},monthly_cap: {type: 'integer', title: 'Monthly GPU cap', 'x-group': GPU_GROUP, 'x-unit': 'nano_usd', 'x-danger': true,minimum: 0,description: 'The hard monthly cap for the rented GPUs, in nano-USD. The provisioner never spends past it: it refuses to start an instance and says why. Without a cap the provisioner stays off.',++ 'x-owner-input': {++ group: 'GPUs',++ unlocks: 'Sets the most the rented GPUs may cost you in a month. A new instance (including a person’s own endpoint) is refused before it starts when its time would pass this cap. Without it nothing is rented.',++ get_it_at: 'https://openvibe.network/admin#ai',++ get_it_how: 'Decide the most you are willing to spend on rented GPUs in a month, and enter it here. Lower is safer; higher lets more models and endpoints run.',++ test: false,++ },++ 'x-required-for': 'own endpoints',},idle_minutes: {type: 'integer', title: 'Idle minutes', 'x-group': GPU_GROUP,nnew file mode 100644@@@ -0,0 +1,279 @@++'use strict';++/**++ * Your own endpoint (the Exchange remake, product line 6): off without the owner's GPU key, cap and Balance++ * billing; started through the same GPU provisioner (never registered as an offer); reached only by its++ * owner's token; scaled to zero after its idle timer; and settled against Billing for the exact seconds,++ * never past the owner's monthly cap. Every GPU and Billing call is a local fake: no test reaches the++ * internet.++ */++const assert = require('assert');++const http = require('http');++const { boot, request, token, suite } = require('./helpers');++const { createGpuFetch } = require('./fixtures/gpu-fetch');++++const t = suite('own-endpoints');++const T0 = Date.parse('2026-11-06T12:00:00Z');++const OWNER = { type: 'user', id: 'usr_' + 'A'.repeat(26) };++const PERSON = 'usr_01JAB2C3D4E5F6G7H8J9K0MNP7';++const b64 = (o) => Buffer.from(JSON.stringify(o)).toString('base64url');++++/** A person's Network token, as Exchange holds it in the site session (audience openvibe.network). */++function userToken(subject = PERSON) {++ return token('a-person', [], { sub: subject, actorType: 'user', aud: 'openvibe.network', ns: null, exp: Math.floor(T0 / 1000) + 3600, extra: { iat: Math.floor(T0 / 1000) } });++}++++/** A local fake Billing: the OAuth token and the four hold routes, with a balance and a real settle charge. */++async function fakeBilling({ balance = 10_000_000_000n } = {}) {++ const state = { balance, held: 0n, holds: new Map(), settlements: new Map(), releases: [], calls: [] };++ const server = http.createServer((req, res) => {++ let raw = '';++ req.on('data', (c) => { raw += c; });++ req.on('end', () => {++ const path = new URL(req.url, 'http://x').pathname;++ let body = {};++ try { body = raw ? JSON.parse(raw) : {}; } catch { body = {}; }++ if (path !== '/oauth/token') state.calls.push({ path, body });++ const send = (status, out) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(out)); };++ if (path === '/oauth/token') return send(200, { access_token: 'test-token', token_type: 'Bearer', expires_in: 300 });++ if (path === '/internal/credits/holds') {++ const prev = state.holds.get(body.idempotency_key);++ if (prev) return send(201, { hold_id: prev.hold_id, held_nano_usd: prev.held, available_nano_usd: String(state.balance - state.held) });++ const amount = BigInt(String(body.max_nano_usd));++ if (state.balance - state.held < amount) return send(402, { code: 'billing.insufficient_funds', detail: 'not enough Balance' });++ const hold = { hold_id: `hld_${String(state.holds.size + 1).padStart(26, '0')}`, held: String(amount), status: 'active', max: String(amount) };++ state.holds.set(body.idempotency_key, hold);++ state.held += amount;++ return send(201, { hold_id: hold.hold_id, held_nano_usd: hold.held, available_nano_usd: String(state.balance - state.held) });++ }++ let m = /^\/internal\/credits\/holds\/([^/]+)\/settle$/.exec(path);++ if (m) {++ const prev = state.settlements.get(body.idempotency_key);++ if (prev) return send(201, prev);++ const hold = [...state.holds.values()].find((x) => x.hold_id === decodeURIComponent(m[1]));++ if (!hold) return send(404, { code: 'billing.hold_not_found' });++ const perM = body.retail.nano_usd_per_m || {};++ const part = (n, r) => { const x = BigInt(n || 0) * BigInt(r || 0); return x === 0n ? 0n : (x % 1000000n === 0n ? x / 1000000n : x / 1000000n + 1n); };++ const charge = part(body.usage.input_tokens, perM.input) + part(body.usage.output_tokens, perM.output) + BigInt(body.retail.per_request_nano_usd || 0);++ const out = { charge_nano_usd: String(charge), released_nano_usd: String(BigInt(hold.held) - charge), journal_id: 'txn_1', shortfall_nano_usd: '0' };++ state.settlements.set(body.idempotency_key, out);++ hold.status = 'settled';++ state.held -= BigInt(hold.held);++ state.balance -= charge;++ return send(201, out);++ }++ m = /^\/internal\/credits\/holds\/([^/]+)\/release$/.exec(path);++ if (m) {++ const id = decodeURIComponent(m[1]);++ const hold = [...state.holds.values()].find((x) => x.hold_id === id);++ if (hold && hold.status !== 'settled') { hold.status = 'released'; state.held -= BigInt(hold.held); state.releases.push(id); }++ return send(200, { hold_id: id, released_nano_usd: hold ? hold.held : '0' });++ }++ return send(404, { code: 'not_found', detail: path });++ });++ });++ await new Promise((r) => server.listen(0, '127.0.0.1', r));++ state.url = `http://127.0.0.1:${server.address().port}`;++ state.close = () => new Promise((r) => { server.closeAllConnections?.(); server.close(() => r()); });++ return state;++}++++/** The GPU fixture plus a canned OpenAI answer and the fake Billing's own HTTP, so one fetch serves all. */++function marketplace(billing) {++ const api = createGpuFetch();++ const chat = [];++ const fetchImpl = async (url, opts = {}) => {++ const u = String(url);++ if (billing && u.startsWith(billing.url)) return globalThis.fetch(u, opts);++ if (u.includes('/v1/chat/completions')) {++ chat.push({ url: u, body: JSON.parse(opts.body || '{}') });++ return new Response(JSON.stringify({ id: 'chatcmpl-own', object: 'chat.completion', model: 'own-model', choices: [{ index: 0, message: { role: 'assistant', content: 'from your GPU' }, finish_reason: 'stop' }], usage: { prompt_tokens: 3, completion_tokens: 4, total_tokens: 7 } }), { status: 200, headers: { 'Content-Type': 'application/json' } });++ }++ return api.fetchImpl(u, opts);++ };++ return { api, chat, fetchImpl };++}++++async function setGpus(h, { key = 'vast-test-key', cap = '50000000000', idle = null } = {}) {++ const store = h.settings.gpus.store;++ const values = { monthly_cap: Number(cap) };++ if (idle != null) values.idle_minutes = idle;++ await store.apply(values, { merge: true, actor: OWNER, reason: 'the test' });++ if (key) await store.setSecret('vast.key', key, { actor: OWNER, reason: 'the test' });++}++++const billingEnv = (billing) => ({ AI_CREDITS: '1', OV_BILLING_INTERNAL_URL: billing.url, OV_NETWORK_INTERNAL_URL: billing.url, OV_OAUTH_CLIENT_SECRET: 'test-secret' });++++t.test('the two owner inputs are marked for Setup: key and cap, group GPUs, needed for own endpoints', async () => {++ const h = await boot({ clock: { now: () => T0 } });++ try {++ const sum = await h.settings.gpus.store.summary();++ for (const key of ['vast.key', 'runpod.key', 'monthly_cap']) {++ const field = sum.schema.properties[key];++ assert.strictEqual(field['x-owner-input'].group, 'GPUs', key + ' is grouped under GPUs');++ assert.strictEqual(field['x-required-for'], 'own endpoints', key + ' is needed for own endpoints');++ assert.ok(String(field.title || '').length > 2, key + ' has a plain title');++ assert.ok(String(field.description || '').length >= 30, key + ' has a description');++ assert.match(String(field['x-owner-input'].get_it_at), /^https:\/\//, key + ' says where to get it');++ }++ assert.strictEqual(sum.schema.properties['vast.key']['x-owner-input'].unlocks.length > 30, true);++ } finally { await h.stop(); }++});++++t.test('off without a GPU key or a cap, with a plain reason; nothing is rented', async () => {++ const now = T0;++ const h = await boot({ clock: { now: () => now } });++ try {++ const st = h.ownEndpoints.status();++ assert.strictEqual(st.enabled, false);++ assert.match(st.reason, /GPU provider key/);++ const pub = await request(h.base, 'GET', '/api/v1/exchange/endpoints');++ assert.strictEqual(pub.status, 200);++ assert.strictEqual(pub.body.enabled, false);++ assert.match(pub.body.reason, /GPU provider key/, 'the public status carries the plain reason');++ const anonymous = await request(h.base, 'GET', '/api/v1/own-endpoints');++ assert.strictEqual(anonymous.status, 401, 'the person API needs a person');++ const r = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { model: 'deepseek-r1-32b' } });++ assert.strictEqual(r.status, 409);++ assert.match(r.text, /GPU provider key/);++ // a key but no cap: still off, and it says which one is missing++ await setGpus(h, { cap: '0' });++ const noCap = h.ownEndpoints.status();++ assert.strictEqual(noCap.enabled, false);++ assert.match(noCap.reason, /monthly GPU cap/);++ } finally { await h.stop(); }++});++++t.test('off when Balance billing is not on, even with a key and a cap', async () => {++ const now = T0;++ const h = await boot({ clock: { now: () => now } });++ try {++ await setGpus(h);++ const st = h.ownEndpoints.status();++ assert.strictEqual(st.enabled, false);++ assert.match(st.reason, /Balance billing is not on/);++ const r = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { model: 'deepseek-r1-32b' } });++ assert.strictEqual(r.status, 409);++ assert.match(r.text, /Balance billing is not on/);++ } finally { await h.stop(); }++});++++t.test('a person starts a private endpoint: the hold is taken first, the rent is never an offer, and the proxy answers', async () => {++ const billing = await fakeBilling();++ const m = marketplace(billing);++ let now = T0;++ const h = await boot({ clock: { now: () => now }, fetchImpl: m.fetchImpl, env: billingEnv(billing) });++ try {++ await setGpus(h, { cap: '50000000000', idle: 1 });++ const created = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { model: 'deepseek-r1-32b', idle_minutes: 1 } });++ assert.strictEqual(created.status, 201, created.text);++ const ep = created.body.endpoint;++ assert.strictEqual(ep.status, 'requested');++ assert.match(ep.url, /\/api\/v1\/own-endpoints\/oep_[^/]+\/v1$/);++ assert.strictEqual(m.api.calls.create.length, 0, 'nothing is rented before the reconciler takes the hold');++ assert.strictEqual(billing.calls.length, 0);++++ // the reconciler starts it: search, hold, then the marketplace++ await h.ownEndpoints.tick();++ assert.strictEqual(m.api.calls.search, 1);++ assert.strictEqual(billing.calls.filter((c) => c.path === '/internal/credits/holds').length, 1, 'the hold was taken');++ assert.strictEqual(m.api.calls.create.length, 1, 'the instance was rented');++ let cur = await h.ownEndpoints.get(PERSON, ep.id);++ assert.strictEqual(cur.status, 'starting');++ assert.ok(BigInt(cur.reserved_nano) > 0n);++++ // it comes up: the provider is NOT registered, so the router and the catalogue never see it++ now += 60_000;++ await h.gpus.tick();++ cur = await h.ownEndpoints.get(PERSON, ep.id);++ assert.strictEqual(cur.status, 'running');++ assert.strictEqual(await h.registry.getProvider('own-' + ep.id), null, 'a private endpoint is never an offer');++++ // the owner's OpenAI-compatible call reaches the rented GPU and marks the use++ const proxied = await request(h.base, 'POST', `/api/v1/own-endpoints/${ep.id}/v1/chat/completions`, { tok: userToken(), body: { model: ep.id, messages: [{ role: 'user', content: 'hello' }] } });++ assert.strictEqual(proxied.status, 200, proxied.text);++ assert.strictEqual(proxied.body.choices[0].message.content, 'from your GPU');++ assert.strictEqual(m.chat.length, 1);++ const fresh = await h.db.prepare('SELECT last_used_at FROM gpu_instances WHERE own_endpoint_id = ?').get(ep.id);++ assert.ok(fresh.last_used_at, 'the use was marked (the idle timer is honest)');++++ // a service token can never manage or call it++ const service = token('live', ['ai.run.create']);++ const refused = await request(h.base, 'GET', '/api/v1/own-endpoints', { tok: service });++ assert.strictEqual(refused.status, 403);++++ // idle: it scales to zero after its own minute, and Billing settles the exact spent seconds++ now += 61_000;++ await h.gpus.tick();++ const stopped = await h.ownEndpoints.get(PERSON, ep.id);++ assert.strictEqual(stopped.status, 'stopped');++ assert.match(String(stopped.error || ''), /idle|scale/i);++ assert.strictEqual(m.api.calls.destroy.length, 1, 'the instance was given back');++ const settled = [...billing.settlements.values()][0];++ assert.ok(settled, 'the hold was settled');++ const row = await h.db.prepare('SELECT charged_nano FROM own_endpoints WHERE id = ?').get(ep.id);++ assert.strictEqual(settled.charge_nano_usd, String(row.charged_nano), 'the person pays exactly what the GPU ledger says');++ assert.ok(BigInt(settled.charge_nano_usd) > 0n);++ const spend = await h.gpus.moneySummary();++ assert.ok(spend.committed_nano <= spend.cap_nano, 'inside the owner cap');++ } finally { await h.stop(); await billing.close(); }++});++++t.test('the owner monthly cap stops a new start, with a plain reason', async () => {++ const billing = await fakeBilling();++ const m = marketplace(billing);++ const now = T0;++ const h = await boot({ clock: { now: () => now }, fetchImpl: m.fetchImpl, env: billingEnv(billing) });++ try {++ await setGpus(h, { cap: '1000000' }); // $0.001: less than an hour of a $0.1348/h GPU++ const created = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { model: 'deepseek-r1-32b' } });++ assert.strictEqual(created.status, 201);++ await h.ownEndpoints.tick();++ const cur = await h.ownEndpoints.get(PERSON, created.body.endpoint.id);++ assert.strictEqual(cur.status, 'refused');++ assert.match(String(cur.error), /cap/i);++ assert.strictEqual(m.api.calls.create.length, 0, 'nothing was rented');++ assert.strictEqual(billing.calls.length, 0, 'no hold was taken, no money moved');++ } finally { await h.stop(); await billing.close(); }++});++++t.test('own weights and a LoRA run on the same provisioner, never on a catalog key', async () => {++ const billing = await fakeBilling();++ const m = marketplace(billing);++ let now = T0;++ const h = await boot({ clock: { now: () => now }, fetchImpl: m.fetchImpl, env: billingEnv(billing) });++ try {++ await setGpus(h);++ const made = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: {++ weights: { repo: 'someone/My-Model-GGUF', file: 'My-Model-Q4_K_M.gguf', lora_repo: 'someone/my-lora', lora_file: 'adapter.gguf' },++ display_name: 'My own weights',++ } });++ assert.strictEqual(made.status, 201, made.text);++ assert.strictEqual(made.body.endpoint.source, 'weights');++ await h.ownEndpoints.tick();++ assert.strictEqual(m.api.calls.create.length, 1);++ const body = m.api.calls.create[0].body;++ assert.match(String(body.args), /huggingface\.co\/someone\/My-Model-GGUF\/resolve\/main\/My-Model-Q4_K_M\.gguf/);++ assert.match(String(body.args), /--lora \/models\/adapter\.gguf/);++ // an invalid repository never reaches the marketplace++ const bad = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { weights: { repo: 'file:///etc/passwd', file: 'x.gguf' } } });++ assert.strictEqual(bad.status, 422);++ } finally { await h.stop(); await billing.close(); }++});++++t.test('a person sees only their own endpoints', async () => {++ const billing = await fakeBilling();++ const m = marketplace(billing);++ const now = T0;++ const h = await boot({ clock: { now: () => now }, fetchImpl: m.fetchImpl, env: billingEnv(billing) });++ try {++ await setGpus(h);++ const made = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { model: 'deepseek-r1-32b' } });++ assert.strictEqual(made.status, 201);++ const other = 'usr_01JAB2C3D4E5F6G7H8J9K0MNP8';++ const peek = await request(h.base, 'GET', `/api/v1/own-endpoints/${made.body.endpoint.id}`, { tok: userToken(other) });++ assert.strictEqual(peek.status, 404);++ } finally { await h.stop(); await billing.close(); }++});++++if (require.main === module) t.run();++module.exports = { suite: t };
@@ -159,6 +159,33 @@ server {
@@ -159,6 +159,33 @@ server {
proxy_pass http://127.0.0.1:4700; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header CF-Connecting-IP $remote_addr; proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:4700; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header CF-Connecting-IP $remote_addr; proxy_set_header X-Forwarded-Proto $scheme;
}
}
+ # ── Your own endpoint (the Exchange remake, build order 7): its public status and the person API that
+ # starts, lists and stops a private GPU rent, plus the OpenAI-compatible proxy to it. The person's own
+ # Network user token guards it in the app (server/api/own-endpoints.js); a service token is refused.
+ location = /api/v1/exchange/endpoints {
+ limit_except GET { deny all; }
+ proxy_pass http://127.0.0.1:4700;
+ proxy_set_header Host $host;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header X-Forwarded-For $remote_addr;
+ proxy_set_header CF-Connecting-IP $remote_addr;
+ }
+ location ^~ /api/v1/own-endpoints/ {
+ limit_req zone=ovai_apps burst=10 nodelay;
+ limit_req_status 429;
+ limit_except GET POST { deny all; }
+ client_max_body_size 12m;
+ proxy_pass http://127.0.0.1:4700;
+ proxy_http_version 1.1;
+ proxy_set_header Host $host;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header X-Forwarded-For $remote_addr;
+ proxy_set_header CF-Connecting-IP $remote_addr;
+ proxy_set_header X-Forwarded-Proto $scheme;
+ proxy_buffering off;
+ proxy_read_timeout 300s;
+ }
+
# ── Unknown API paths: a problem document, never a page; the API itself stays host-local ──
# ── Unknown API paths: a problem document, never a page; the API itself stays host-local ──
location /api/ {
location /api/ {
default_type application/problem+json;
default_type application/problem+json;
@@ -97,6 +97,23 @@ server {
@@ -97,6 +97,23 @@ server {
proxy_set_header CF-Connecting-IP $remote_addr;
proxy_set_header CF-Connecting-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
+ # Your own endpoint (the Exchange remake, build order 7): the person API and the OpenAI-compatible
+ # proxy to their private GPU. Only the person's own Network user token reaches it (a service token is
+ # refused in the app); answers may stream, so buffering is off and the read timeout covers one answer.
+ location ^~ /api/v1/own-endpoints/ {
+ limit_req zone=ovai_apps burst=10 nodelay;
+ limit_req_status 429;
+ limit_except GET POST { deny all; }
+ proxy_pass http://127.0.0.1:4700;
+ proxy_http_version 1.1;
+ proxy_set_header Host $host;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header X-Forwarded-For $remote_addr;
+ proxy_set_header CF-Connecting-IP $remote_addr;
+ proxy_set_header X-Forwarded-Proto $scheme;
+ proxy_buffering off;
+ proxy_read_timeout 300s;
+ }
location = /openapi.json {
location = /openapi.json {
limit_except GET { deny all; }
limit_except GET { deny all; }
proxy_pass http://127.0.0.1:4700;
proxy_pass http://127.0.0.1:4700;
@@ -88,6 +88,38 @@ state, the measured cold start (from the launch to the first answered request),
@@ -88,6 +88,38 @@ state, the measured cold start (from the launch to the first answered request),
with its price (the hour and the second), endpoint, cold start and money. "Start" and "Stop" write a request row;
with its price (the hour and the second), endpoint, cold start and money. "Start" and "Stop" write a request row;
the reconciler (one implementation, the same one the service runs) does the work.
the reconciler (one implementation, the same one the service runs) does the work.
+## Your own endpoint (the private rent)
+
+The Exchange's product line 6, "your own endpoint", is the same provisioner with a private rent on top
+(`server/gpu/endpoints.js`, migration `0039_own_endpoints.sql`):
+
+- a person picks one of the models above, or brings their own Hugging Face `.gguf` (and an optional
+ LoRA), from OpenVibe.Exchange's `/own` page or the person API (`/api/v1/own-endpoints`);
+- the reconciler starts it on the cheapest offer that fits, exactly like the shared models, and **never
+ registers it as an offer**: the router and the catalogue never see it, and the GPU is reached only
+ through AI's proxy at `/api/v1/own-endpoints/:id/v1` with the owner's own Network user token;
+- the person's Balance pays for it. Before the marketplace is called, a Billing hold is taken for the
+ worst case the rent may cost (the reservation); when it stops, the exact seconds are settled:
+ `usage.input_tokens` carries the milliseconds, `retail.version` is `gpu-second@1`, and the per-request
+ remainder carries the last nano so the Billing charge equals the GPU ledger's `spent_nano` to the
+ nano. Billing is the only place the person is charged; the `gpu_spend` rows are the owner's cost
+ record, never the person's charge;
+- the idle timer is the endpoint's own (`idle_minutes`, 1-120, default the owner's `idle_minutes`),
+ measured from the last proxied request (`gpu_instances.last_used_at`). The maximum run is 120 minutes
+ (or the owner's `max_runtime_minutes` when lower), so a start's hold is bounded;
+- the owner's monthly cap is checked before the hold: a new start that would pass it is refused with a
+ plain reason and nothing is rented.
+
+The feature is off while **any** of the three is missing, and the public
+`GET /api/v1/exchange/endpoints` answers the plain reason the Exchange page shows:
+
+1. `ai.gpus` `vast.key` or `runpod.key` (the owner's GPU provider key),
+2. `ai.gpus` `monthly_cap` (the hard monthly cap),
+3. Balance billing (`AI_CREDITS` and Billing), because an endpoint is paid for per second.
+
+The first two are `x-owner-input` fields (group `GPUs`, `x-required-for: "own endpoints"`) so the admin
+panel's Setup tab lists them with what they turn on and where to get them.
+
## The tests
## The tests
`test/gpu.test.js` runs the provisioner against the recorded marketplace answers
`test/gpu.test.js` runs the provisioner against the recorded marketplace answers
@@ -102,6 +134,17 @@ the reconciler (one implementation, the same one the service runs) does the work
@@ -102,6 +134,17 @@ the reconciler (one implementation, the same one the service runs) does the work
- an instance with our label and no row of ours is found and given back,
- an instance with our label and no row of ours is found and given back,
- a destroy that keeps failing is tried again every pass and the operator is alerted.
- a destroy that keeps failing is tried again every pass and the operator is alerted.
+`test/own-endpoints.test.js` covers the private rent with the same recorded marketplace, a fake clock, a
+local fake Billing and a canned GPU answer:
+
+- off without a GPU key or a cap, and off without Balance billing, each with its plain reason;
+- the Billing hold is taken before the marketplace is called;
+- a private rent is never registered as an offer, and the owner's OpenAI-compatible call is proxied;
+- a service token cannot manage or call a person's endpoint;
+- the endpoint's own idle timer scales it to zero, and the settle charge equals the GPU `spent_nano`;
+- the monthly cap refuses a new start with a plain reason and moves no money;
+- own weights and a LoRA produce the right container command, and a non-Hugging-Face address is refused.
+
## What the owner has to check before the first launch
## What the owner has to check before the first launch
The weights URL, the image and the container command are data in `server/gpu/catalog.js` - they are
The weights URL, the image and the container command are data in `server/gpu/catalog.js` - they are
new file mode 100644
new file mode 100644
@@ -0,0 +1,48 @@
@@ -0,0 +1,48 @@
+-- phase: expand
+-- Your own endpoint (the Exchange remake, product line 6 / build order 7): a private GPU endpoint a person
+-- rents for a chosen catalog model or their own Hugging Face weights. The instance is the existing
+-- gpu_instances row (the GPU provisioner rents, health-checks, scales it to zero and enforces the owner's
+-- monthly cap); this table is the person's endpoint on top of it: who owns it, what it serves, the Billing
+-- hold that pays for the seconds, and the public proxy its owner calls. Additive only: nothing existing is
+-- changed, gpu_instances gains two nullable columns.
+CREATE TABLE IF NOT EXISTS own_endpoints (
+ id text COLLATE "C" PRIMARY KEY, -- oep_<ULID>
+ subject text COLLATE "C" NOT NULL, -- usr_<ULID>: the owner and the payer
+ status text COLLATE "C" NOT NULL CHECK (status IN ('requested','starting','running','stopping','stopped','refused','error')),
+ source text COLLATE "C" NOT NULL CHECK (source IN ('catalog','weights')),
+ catalog_key text COLLATE "C", -- a model the provisioner knows, when source=catalog
+ display_name text COLLATE "C" NOT NULL,
+ weights_repo text COLLATE "C",
+ weights_file text COLLATE "C",
+ lora_repo text COLLATE "C",
+ lora_file text COLLATE "C",
+ context_window integer,
+ idle_minutes integer NOT NULL,
+ max_runtime_minutes integer NOT NULL,
+ marketplace text COLLATE "C",
+ price_per_hour text COLLATE "C", -- the marketplace's own decimal price
+ hourly_nano bigint, -- that price as exact nano-USD per hour
+ reserved_nano bigint NOT NULL DEFAULT 0, -- the budget held from Billing for this start
+ instance_id text COLLATE "C", -- gpu_instances.id
+ endpoint text COLLATE "C", -- the rented instance's own URL (never public)
+ url text COLLATE "C", -- this endpoint's public URL (the AI proxy)
+ billing_hold_id text COLLATE "C",
+ billing_settled_at timestamptz,
+ charged_nano bigint NOT NULL DEFAULT 0, -- what the person's Balance paid for this start
+ month text COLLATE "C" NOT NULL,
+ error text COLLATE "C", -- a plain reason when refused or stopped
+ created_at timestamptz NOT NULL,
+ updated_at timestamptz NOT NULL,
+ started_at timestamptz,
+ ready_at timestamptz,
+ stopped_at timestamptz
+);
+CREATE INDEX IF NOT EXISTS own_endpoints_subject ON own_endpoints (subject, created_at DESC);
+CREATE INDEX IF NOT EXISTS own_endpoints_status ON own_endpoints (status);
+CREATE INDEX IF NOT EXISTS own_endpoints_month ON own_endpoints (month);
+
+-- The instance knows the endpoint it serves, so the reconciler can tell a private rent from a shared model
+-- offer (a private rent is never registered for the router) and give it the endpoint's own idle timer.
+ALTER TABLE gpu_instances ADD COLUMN IF NOT EXISTS own_endpoint_id text COLLATE "C";
+ALTER TABLE gpu_instances ADD COLUMN IF NOT EXISTS idle_minutes integer;
+CREATE INDEX IF NOT EXISTS gpu_instances_own ON gpu_instances (own_endpoint_id) WHERE own_endpoint_id IS NOT NULL;
@@ -61,6 +61,14 @@ const TABLES = [
@@ -61,6 +61,14 @@ const TABLES = [
// removed when the account goes, so it stops receiving traffic too.
// removed when the account goes, so it stops receiving traffic too.
{ table: 'community_endpoint_offers', subject: 'owner_subject', file: 'community-endpoint-offers.json', order: 'created_at',
{ table: 'community_endpoint_offers', subject: 'owner_subject', file: 'community-endpoint-offers.json', order: 'created_at',
columns: ['id', 'name', 'base_url', 'models', 'hours', 'concurrency', 'price_in', 'price_out', 'audience', 'group_id', 'created_at', 'updated_at'] },
columns: ['id', 'name', 'base_url', 'models', 'hours', 'concurrency', 'price_in', 'price_out', 'audience', 'group_id', 'created_at', 'updated_at'] },
+ // Your own endpoint (the Exchange remake, build order 7): a private GPU rent. It is the person's own
+ // data while the account lives, so it goes in the export. On deletion the row is anonymized at once and
+ // marked stopping, and the next reconciler pass gives the GPU back and lets Billing settle the seconds;
+ // the money rows themselves are kept like the ledger.
+ { table: 'own_endpoints', subject: 'subject', file: 'own-endpoints.json', order: 'created_at',
+ columns: ['id', 'status', 'source', 'catalog_key', 'display_name', 'weights_repo', 'weights_file', 'lora_repo', 'lora_file',
+ 'idle_minutes', 'max_runtime_minutes', 'marketplace', 'price_per_hour', 'charged_nano', 'created_at', 'ready_at', 'stopped_at'],
+ erase: { keep: 'the GPU rent and its Billing hold are money records; extraErase anonymizes them and stops the rent so Billing can settle' } },
// The person's own quota (server/quota.js): a limit that only means something while the account lives.
// The person's own quota (server/quota.js): a limit that only means something while the account lives.
{ table: 'quotas', subject: 'scope_id', value: (usr) => `user:${usr}`, file: null },
{ table: 'quotas', subject: 'scope_id', value: (usr) => `user:${usr}`, file: null },
// What the person used, counted against their allowance: the spend ledger, kept.
// What the person used, counted against their allowance: the spend ledger, kept.
@@ -79,6 +87,13 @@ async function extraExport(db, subject) {
@@ -79,6 +87,13 @@ async function extraExport(db, subject) {
/** The person's runs go, with the rows that hang under them (the requests, the citations, the cached answers). */
/** The person's runs go, with the rows that hang under them (the requests, the citations, the cached answers). */
async function extraErase(t, subjects, counts) {
async function extraErase(t, subjects, counts) {
+ // A private GPU rent is stopped and anonymized now; its Billing hold and money stay until the reconciler
+ // gives the instance back and settles the seconds (the table's erase is a keep for that reason).
+ counts.add(counts.erased, 'own_endpoints', await t.exec(`UPDATE own_endpoints
+ SET status = CASE WHEN status IN ('requested','starting','running','stopping') THEN 'stopping' ELSE status END,
+ subject = NULL, display_name = 'deleted endpoint', weights_repo = NULL, weights_file = NULL,
+ lora_repo = NULL, lora_file = NULL, url = NULL, updated_at = now()
+ WHERE subject = ANY($1::text[])`, [subjects]));
// Plan E5: the free-tier counters. A shared device counter is kept while another account still uses the device;
// Plan E5: the free-tier counters. A shared device counter is kept while another account still uses the device;
// when this person was the last one, the device counter and its velocity bucket go too.
// when this person was the last one, the device counter and its velocity bucket go too.
const hashes = (await t.many('SELECT DISTINCT device_hash FROM abuse_devices WHERE subject = ANY($1::text[])', [subjects])).map((r) => r.device_hash);
const hashes = (await t.many('SELECT DISTINCT device_hash FROM abuse_devices WHERE subject = ANY($1::text[])', [subjects])).map((r) => r.device_hash);
@@ -35,7 +35,7 @@ function memo(fn, ttlMs, clock) {
@@ -35,7 +35,7 @@ function memo(fn, ttlMs, clock) {
};
};
}
}
-function exchangeRouter({ db, registry, pool, quotas = null, env = process.env, clock = { now: () => Date.now() }, config = null, presets, receipts, auth, log = console }) {
+function exchangeRouter({ db, registry, pool, quotas = null, env = process.env, clock = { now: () => Date.now() }, config = null, presets, receipts, auth, log = console, ownEndpoints = null }) {
const catalog = createCatalog({ db, registry, pool, env, clock, config });
const catalog = createCatalog({ db, registry, pool, env, clock, config });
const usage = createUsage({ db, clock });
const usage = createUsage({ db, clock });
// The price book's log: what a price was, what it became and when it was read.
// The price book's log: what a price was, what it became and when it was read.
@@ -158,6 +158,15 @@ function exchangeRouter({ db, registry, pool, quotas = null, env = process.env,
@@ -158,6 +158,15 @@ function exchangeRouter({ db, registry, pool, quotas = null, env = process.env,
} catch (err) { next(err); }
} catch (err) { next(err); }
});
});
+ // Your own endpoint: whether it is on and, when it is off, the plain reason. Public: the Exchange page
+ // reads it before anyone signs in, and nothing here is per person.
+ r.get('/api/v1/exchange/endpoints', publicCache(30), (req, res) => {
+ res.json(ownEndpoints ? ownEndpoints.status() : {
+ enabled: false, reason: 'The GPU provider is not set up yet, so no endpoint can start.', reasons: ['The GPU provider is not set up yet, so no endpoint can start.'],
+ billing: false, models: [],
+ });
+ });
+
r.get('/api/v1/exchange/models/:slug', publicCache(30), async (req, res, next) => {
r.get('/api/v1/exchange/models/:slug', publicCache(30), async (req, res, next) => {
try {
try {
const out = await all();
const out = await all();
new file mode 100644
new file mode 100644
@@ -0,0 +1,96 @@
@@ -0,0 +1,96 @@
+'use strict';
+/**
+ * Your own endpoint's person API (the Exchange remake, build order 7).
+ *
+ * GET /api/v1/own-endpoints the caller's endpoints
+ * POST /api/v1/own-endpoints { model } or { weights: { repo, file, lora_repo?, lora_file? }, idle_minutes? }
+ * GET /api/v1/own-endpoints/:id one endpoint
+ * POST /api/v1/own-endpoints/:id/stop give it back
+ * * /api/v1/own-endpoints/:id/v1/* the OpenAI-compatible proxy to the rented GPU
+ *
+ * Only the person themselves: a Network user token (audience openvibe.network or openvibe.ai) whose
+ * subject is a usr_…; a service, app, module or node token is refused whatever capabilities it holds.
+ * The endpoint is private: the rented GPU is never registered as an offer, so the router and the
+ * catalogue never see it, and the proxy is the only way in.
+ */
+const express = require('express');
+const { Readable } = require('node:stream');
+const { http } = require('openvibe-contracts');
+const { verifyUserToken } = require('openvibe-sdk/auth');
+const { AiError, sendError } = require('../util');
+const { bearer } = require('../auth');
+
+const SUBJECT_RE = /^usr_[0-9A-HJKMNP-TV-Z]{26}$/;
+
+function createOwnEndpointsRouter({ endpoints, auth, config, clock = { now: () => Date.now() }, log = console }) {
+ const r = express.Router();
+
+ /** A person, never a service. Sets req.ownPerson = { subject, token }. */
+ async function guard(req, res, next) {
+ try {
+ const token = bearer(req);
+ if (!token) return http.sendProblem(res, 401, 'token.missing', { detail: 'sign in to manage your endpoint', ctx: req.ov });
+ let service = null;
+ try { service = await auth.verify(token); } catch { service = { ok: false }; }
+ if (service && service.ok) return http.sendProblem(res, 403, 'capability.denied', { detail: "a service token cannot manage a person's endpoint", ctx: req.ov });
+ let claims;
+ try {
+ claims = await verifyUserToken(token, {
+ ...(config.networkPublicKey ? { publicKey: config.networkPublicKey } : { jwks: config.networkJwksUrl }),
+ issuer: config.issuer, audience: ['openvibe.network', 'openvibe.ai'], now: clock.now(), log,
+ });
+ } catch (err) {
+ log.warn && log.warn('[own-endpoint] person token refused: ' + String((err && (err.code || err.message)) || err));
+ return http.sendProblem(res, 401, 'token.invalid', { detail: 'your sign-in could not be verified', ctx: req.ov });
+ }
+ const subject = claims && (claims.subject_id || claims.sub);
+ if (!SUBJECT_RE.test(String(subject || ''))) return http.sendProblem(res, 403, 'capability.denied', { detail: 'only a person may manage an endpoint', ctx: req.ov });
+ req.ownPerson = { subject: String(subject), token };
+ return next();
+ } catch (err) {
+ return sendError(res, err, req.ov, log);
+ }
+ }
+
+ const wrap = (fn) => async (req, res) => {
+ try { const out = await fn(req, res); if (out !== undefined) res.json(out); } catch (err) { sendError(res, err, req.ov, log); }
+ };
+
+ r.use('/api/v1/own-endpoints', guard);
+
+ r.get('/api/v1/own-endpoints', wrap(async (req) => ({ endpoints: await endpoints.list(req.ownPerson.subject), status: endpoints.status() })));
+
+ r.post('/api/v1/own-endpoints', wrap(async (req, res) => {
+ const body = req.body && typeof req.body === 'object' ? req.body : {};
+ const out = await endpoints.create({ subject: req.ownPerson.subject, body });
+ res.status(201).json({ endpoint: out });
+ }));
+
+ r.get('/api/v1/own-endpoints/:id', wrap(async (req) => ({ endpoint: await endpoints.get(req.ownPerson.subject, req.params.id) })));
+
+ r.post('/api/v1/own-endpoints/:id/stop', wrap(async (req) => ({ endpoint: await endpoints.stop(req.ownPerson.subject, req.params.id) })));
+
+ // The OpenAI-compatible proxy: /v1/models and /v1/chat/completions (streaming included). The person's
+ // token decides who may come in; the rented GPU only ever sees the body.
+ r.all('/api/v1/own-endpoints/:id/v1/*', async (req, res) => {
+ try {
+ const id = req.params.id;
+ const rest = req.params[0] || '';
+ if (!id) throw new AiError(404, 'own_endpoint.not_found', 'no such endpoint');
+ const path = '/v1/' + rest;
+ const out = await endpoints.forward({ subject: req.ownPerson.subject, id, method: req.method, path, body: req.body });
+ res.status(out.status);
+ res.set('Content-Type', out.contentType);
+ res.set('Cache-Control', 'no-store');
+ if (out.body && typeof out.body.pipe === 'function') return out.body.pipe(res);
+ if (out.body && typeof out.body.getReader === 'function') return Readable.fromWeb(out.body).pipe(res);
+ return res.end();
+ } catch (err) {
+ return sendError(res, err, req.ov, log);
+ }
+ });
+
+ return r;
+}
+
+module.exports = { createOwnEndpointsRouter };
@@ -24,7 +24,7 @@ const pkg = require('../package.json');
@@ -24,7 +24,7 @@ const pkg = require('../package.json');
const { adminRoutes } = require('openvibe-shared/config');
const { adminRoutes } = require('openvibe-shared/config');
-function createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env = process.env, clock = { now: () => Date.now() }, fetchImpl = globalThis.fetch, log = console, credentials = null, preferences = null, grants = null, vault = null, settings = null, benchEvents = null, runBudgets = null, memory = null, memoryImport = null, memoryServices = null, memorySharing = null, community = null, endpoints = null, webhooks = null, presets = null, receipts = null, balance = null, gpus = null, abuse = null, priceBook = null }) {
+function createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env = process.env, clock = { now: () => Date.now() }, fetchImpl = globalThis.fetch, log = console, credentials = null, preferences = null, grants = null, vault = null, settings = null, benchEvents = null, runBudgets = null, memory = null, memoryImport = null, memoryServices = null, memorySharing = null, community = null, endpoints = null, webhooks = null, presets = null, receipts = null, balance = null, gpus = null, abuse = null, priceBook = null, ownEndpoints = null }) {
const app = express();
const app = express();
// /.well-known/agent.json (openvibe-shared/agent-layer, SEO-2): what this site serves, for agents.
// /.well-known/agent.json (openvibe-shared/agent-layer, SEO-2): what this site serves, for agents.
require('openvibe-shared/agent-layer').mount(app, { descriptor: require('./agent-descriptor') });
require('openvibe-shared/agent-layer').mount(app, { descriptor: require('./agent-descriptor') });
@@ -116,7 +116,10 @@ function createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys
@@ -116,7 +116,10 @@ function createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys
openapi.mount(app);
openapi.mount(app);
// OpenVibe.Exchange's read API (server/api/exchange.js): the public catalog and aggregate usage. Public on
// OpenVibe.Exchange's read API (server/api/exchange.js): the public catalog and aggregate usage. Public on
// purpose — the site and anyone comparing prices reads the same numbers the router ranks on.
// purpose — the site and anyone comparing prices reads the same numbers the router ranks on.
- app.use(require('./api/exchange').exchangeRouter({ config, db, registry, pool, quotas, env, clock, presets, receipts, auth, log }));
+ app.use(require('./api/exchange').exchangeRouter({ config, db, registry, pool, quotas, env, clock, presets, receipts, auth, log, ownEndpoints }));
+ // A person's own endpoint (the Exchange remake, product line 6): the private rent's person API and its
+ // OpenAI-compatible proxy. Only the owner's own Network user token reaches it; a service token never does.
+ if (ownEndpoints) app.use(require('./api/own-endpoints').createOwnEndpointsRouter({ endpoints: ownEndpoints, auth, config, clock, log }));
// The public seller API: a standalone OpenAI-compatible server registered by its own https address with the
// The public seller API: a standalone OpenAI-compatible server registered by its own https address with the
// seller's developer key (the design's SDK gap). Its offers join the community router's trust and ledger.
// seller's developer key (the design's SDK gap). Its offers join the community router's trust and ledger.
if (endpoints) app.use(require('./api/seller').sellerRouter({ endpoints, community, webhooks, auth, log }));
if (endpoints) app.use(require('./api/seller').sellerRouter({ endpoints, community, webhooks, auth, log }));
new file mode 100644
new file mode 100644
@@ -0,0 +1,497 @@
@@ -0,0 +1,497 @@
+'use strict';
+/**
+ * Your own endpoint (the Exchange remake, product line 6 / build order 7).
+ *
+ * A person picks one of the GPU provisioner's models or brings their own Hugging Face weights (and an
+ * optional LoRA), and gets a private OpenAI-compatible endpoint on a rented GPU. The rent is the
+ * existing gpu_instances row: the same cheapest-fit search, the same owner monthly cap, the same
+ * marketplace and the same scale-to-zero sweep (server/gpu/provisioner.js). What this module adds:
+ *
+ * - the endpoint row (migration 0039) and its owner;
+ * - a Billing hold before the marketplace is called, of the worst case the rent may cost, and a
+ * settle of the exact seconds when it stops. Billing is the only ledger: no money is written here;
+ * - privacy: a private rent is never registered as an offer, so the router and the catalogue never
+ * see it. Its owner reaches it through AI's proxy, and only with their own Network token;
+ * - the public status the Exchange page reads, with the plain reason the feature is off.
+ *
+ * The feature is off while ANY of these is true, and every one of them is an owner setting:
+ *
+ * 1. no GPU marketplace key (ai.gpus vast.key / runpod.key),
+ * 2. no monthly GPU cap (ai.gpus monthly_cap),
+ * 3. Balance billing is not on (AI_CREDITS + Billing), because an endpoint must be paid for.
+ *
+ * Nothing here reaches a GPU provider in a test: the marketplace calls go through the provisioner's
+ * injected fetch (the recorded answers in test/fixtures/gpu), and Billing through the Credits manager's.
+ */
+const { ids } = require('openvibe-contracts');
+const { AiError, sha256 } = require('../util');
+const money = require('./money');
+const catalog = require('./catalog');
+
+const ACTIVE = ['requested', 'starting', 'running', 'stopping'];
+const LIVE = ['starting', 'running', 'stopping'];
+const MAX_ACTIVE = 2;
+const MAX_IDLE_MINUTES = 120;
+const MAX_RUNTIME_MINUTES = 120;
+const WEIGHTS_REPO = /^[A-Za-z0-9][A-Za-z0-9._-]{0,99}\/[A-Za-z0-9][A-Za-z0-9._-]{0,99}$/;
+const WEIGHTS_FILE = /^[A-Za-z0-9][A-Za-z0-9._ -]{0,180}\.gguf$/;
+const BILLING_OFF = 'Balance billing is not on yet, so an endpoint cannot be paid for. The owner must turn on Balance first.';
+
+function plainGpuReason(reason) {
+ if (reason === 'no Vast.ai or RunPod API key') return 'The owner has not added a GPU provider key (Vast.ai or RunPod) yet, so no endpoint can start.';
+ if (reason === 'no monthly cap') return 'The owner has not set a monthly GPU cap yet, so no endpoint can start.';
+ return 'The GPU provider is not set up yet, so no endpoint can start.';
+}
+
+/** A short, plain sentence for why a rent stopped. The status line is a sentence, never a code. */
+function reasonText(reason) {
+ const r = String(reason || '');
+ if (r === 'idle') return 'Stopped after it sat idle; it scales to zero when nobody uses it.';
+ if (r === 'the owner stopped it') return 'Stopped.';
+ if (r === 'the reservation ran out') return 'Stopped: the time it was paid for ran out.';
+ if (r === 'the endpoint stopped answering' || r === 'the endpoint never answered') return 'Stopped because the GPU stopped answering.';
+ if (/^the instance /.test(r)) return 'Stopped because the GPU instance ' + r.slice('the instance '.length) + '.';
+ return 'Stopped.';
+}
+
+function clamp(value, min, max, fallback) {
+ if (value == null || value === '') return fallback;
+ const n = Number(value);
+ if (!Number.isFinite(n)) return fallback;
+ return Math.max(min, Math.min(max, Math.trunc(n)));
+}
+
+/**
+ * The module. `credits` is AI's Credits facade (server/index.js): `credits.enabled` is the live
+ * AI_CREDITS flag and `credits.manager` the client to Billing's hold/settle API.
+ */
+function createOwnEndpoints({ db, gpus, credits, config = null, clock = { now: () => Date.now() }, fetchImpl = globalThis.fetch, log = console, models = catalog.list() } = {}) {
+ const now = () => clock.now();
+ const at = (ms) => new Date(ms).toISOString();
+ const byKey = new Map((models || []).map((m) => [m.key, m]));
+
+ function manager() {
+ return credits && credits.enabled && credits.manager ? credits.manager : null;
+ }
+
+ function gpuConfig() {
+ return gpus && typeof gpus.config === 'function' ? gpus.config() : { enabled: false, reason: 'no Vast.ai or RunPod API key', idle_minutes: 10, max_runtime_minutes: 720, vast_key: '', runpod_key: '' };
+ }
+
+ /** The public status the Exchange page reads: on, or off with a plain sentence. */
+ function status() {
+ const cfg = gpuConfig();
+ const billing = Boolean(manager());
+ const reasons = [];
+ if (!cfg.enabled) reasons.push(plainGpuReason(cfg.reason));
+ if (!billing) reasons.push(BILLING_OFF);
+ const marketplace = cfg.vast_key ? 'vast' : (cfg.runpod_key ? 'runpod' : null);
+ return {
+ enabled: Boolean(cfg.enabled && billing),
+ reason: reasons.join(' '),
+ reasons,
+ marketplace,
+ billing,
+ models: (models || []).map((m) => ({
+ key: m.key, display_name: m.display_name, context_window: m.context_window,
+ vram_mb: m.requirements && m.requirements.vram_mb,
+ gpu_names: (m.requirements && m.requirements.gpu_names) || [],
+ })),
+ idle_minutes: cfg.idle_minutes,
+ max_idle_minutes: MAX_IDLE_MINUTES,
+ max_runtime_minutes: Math.min(cfg.max_runtime_minutes || MAX_RUNTIME_MINUTES, MAX_RUNTIME_MINUTES),
+ public_base: publicBase(),
+ };
+ }
+
+ function publicBase() {
+ const base = String((config && config.baseUrl) || '').replace(/\/+$/, '');
+ return (base || 'https://ai.openvibe.services') + '/api/v1/own-endpoints';
+ }
+
+ async function rowById(id) {
+ return (await db.prepare('SELECT * FROM own_endpoints WHERE id = ?').get(String(id))) || null;
+ }
+
+ async function ownedRow(subject, id) {
+ const row = await rowById(id);
+ if (!row || row.subject !== String(subject)) throw new AiError(404, 'own_endpoint.not_found', 'no such endpoint');
+ return row;
+ }
+
+ async function liveFor(id, statuses = LIVE) {
+ return (await db.prepare(`SELECT * FROM gpu_instances WHERE own_endpoint_id = ? AND status IN (${statuses.map(() => '?').join(', ')}) ORDER BY created_at DESC LIMIT 1`)
+ .get(String(id), ...statuses)) || null;
+ }
+
+ /** What the Exchange page and its owner see. */
+ function describe(row, instance = null) {
+ if (!row) return null;
+ return {
+ id: row.id,
+ status: row.status,
+ source: row.source,
+ display_name: row.display_name,
+ catalog_key: row.catalog_key,
+ weights: row.source === 'weights' ? { repo: row.weights_repo, file: row.weights_file, lora_repo: row.lora_repo, lora_file: row.lora_file } : null,
+ idle_minutes: Number(row.idle_minutes),
+ max_runtime_minutes: Number(row.max_runtime_minutes),
+ marketplace: row.marketplace,
+ price_per_hour: row.price_per_hour,
+ hourly_nano: row.hourly_nano == null ? null : String(row.hourly_nano),
+ reserved_nano: String(row.reserved_nano == null ? '0' : row.reserved_nano),
+ charged_nano: String(row.charged_nano == null ? '0' : row.charged_nano),
+ url: row.url || (publicBase() + '/' + row.id + '/v1'),
+ model: row.id,
+ instance: instance ? {
+ status: instance.status, gpu_name: instance.gpu_name, region: instance.region,
+ cold_start_ms: instance.cold_start_ms == null ? null : String(instance.cold_start_ms),
+ started_at: instance.started_at, ready_at: instance.ready_at,
+ last_used_at: instance.last_used_at,
+ } : null,
+ month: row.month,
+ error: row.error,
+ reason: row.error || null,
+ created_at: row.created_at,
+ ready_at: row.ready_at,
+ stopped_at: row.stopped_at,
+ };
+ }
+
+ async function describeWithInstance(row) {
+ return describe(row, row.instance_id ? await db.prepare('SELECT * FROM gpu_instances WHERE id = ?').get(row.instance_id) : null);
+ }
+
+ async function list(subject) {
+ const rows = await db.prepare('SELECT * FROM own_endpoints WHERE subject = ? ORDER BY created_at DESC').all(String(subject));
+ const out = [];
+ for (const row of rows || []) out.push(await describeWithInstance(row));
+ return out;
+ }
+
+ async function get(subject, id) {
+ return describeWithInstance(await ownedRow(subject, id));
+ }
+
+ /** The spec the provisioner serves for this endpoint. A private rent is never registered. */
+ function specFor(row) {
+ if (!row) return null;
+ const cfg = gpuConfig();
+ const marketplace = cfg.vast_key ? 'vast' : (cfg.runpod_key ? 'runpod' : 'vast');
+ const key = 'own-' + row.id;
+ if (row.source === 'catalog') {
+ const base = byKey.get(row.catalog_key) || catalog.find(row.catalog_key);
+ if (!base) return null;
+ return {
+ ...base,
+ key,
+ provider_key: key,
+ model_key: key,
+ marketplace,
+ display_name: 'Your ' + (base.display_name || base.key),
+ };
+ }
+ const weightsUrl = 'https://huggingface.co/' + row.weights_repo + '/resolve/main/' + encodeURIComponent(row.weights_file).replace(/%2F/g, '/');
+ const loraUrl = row.lora_repo ? 'https://huggingface.co/' + row.lora_repo + '/resolve/main/' + encodeURIComponent(row.lora_file).replace(/%2F/g, '/') : null;
+ const ctx = Number(row.context_window) || 8192;
+ const command = 'mkdir -p /models'
+ + ' && curl -fL --retry 3 -o /models/' + row.weights_file + ' ' + weightsUrl
+ + (loraUrl ? ' && curl -fL --retry 3 -o /models/' + row.lora_file + ' ' + loraUrl : '')
+ + ' && exec /app/llama-server --model /models/' + row.weights_file
+ + ' --host 0.0.0.0 --port 8000 --alias ' + key
+ + ' --ctx-size ' + ctx + ' --n-gpu-layers 999'
+ + (loraUrl ? ' --lora /models/' + row.lora_file : '');
+ return {
+ key,
+ provider_key: key,
+ model_key: key,
+ marketplace,
+ display_name: row.display_name || 'Your model',
+ context_window: ctx,
+ throughput_tokens_per_hour: 180000,
+ requirements: { vram_mb: 22000, disk_gb: 60, gpu_names: ['RTX 3090', 'RTX 4090'], num_gpus: 1, min_reliability: 0.98 },
+ runtime: { kind: 'llama.cpp', image: 'ghcr.io/ggml-org/llama.cpp:server-cuda', port: 8000, weights: { repo: row.weights_repo, file: row.weights_file }, command },
+ };
+ }
+
+ /** Validate what a person asked for and write the request row. The reconciler starts it. */
+ async function create({ subject, body = {} }) {
+ const st = status();
+ if (!st.enabled) throw new AiError(409, 'own_endpoint.off', st.reason || 'Your own endpoint is off.');
+ const cfg = gpuConfig();
+ const active = await db.prepare(`SELECT COUNT(*) AS n FROM own_endpoints WHERE subject = ? AND status IN (${ACTIVE.map(() => '?').join(', ')})`)
+ .get(String(subject), ...ACTIVE);
+ if (Number(active && active.n) >= MAX_ACTIVE) throw new AiError(409, 'own_endpoint.limit', 'You already have ' + MAX_ACTIVE + ' endpoints running or starting. Stop one first.');
+
+ const wantsCatalog = body.model != null && body.model !== '';
+ const wantsWeights = body.weights && typeof body.weights === 'object';
+ if (wantsCatalog === wantsWeights) throw new AiError(422, 'input.invalid', 'name one model, or one set of weights');
+ let source, spec = null, display = '', weights = null;
+ if (wantsCatalog) {
+ spec = byKey.get(String(body.model)) || catalog.find(String(body.model));
+ if (!spec) throw new AiError(422, 'input.invalid', 'no GPU model is called ' + String(body.model).slice(0, 80));
+ source = 'catalog';
+ display = spec.display_name;
+ } else {
+ const repo = String(body.weights.repo || '').trim();
+ const file = String(body.weights.file || '').trim();
+ const loraRepo = body.weights.lora_repo == null ? '' : String(body.weights.lora_repo).trim();
+ const loraFile = body.weights.lora_file == null ? '' : String(body.weights.lora_file).trim();
+ if (!WEIGHTS_REPO.test(repo)) throw new AiError(422, 'input.invalid', 'weights.repo must be a Hugging Face repository, like "user/model-GGUF"');
+ if (!WEIGHTS_FILE.test(file)) throw new AiError(422, 'input.invalid', 'weights.file must be a .gguf file name from that repository');
+ if (Boolean(loraRepo) !== Boolean(loraFile)) throw new AiError(422, 'input.invalid', 'name both the LoRA repository and its .gguf file, or neither');
+ if (loraRepo && !WEIGHTS_REPO.test(loraRepo)) throw new AiError(422, 'input.invalid', 'weights.lora_repo must be a Hugging Face repository');
+ if (loraFile && !WEIGHTS_FILE.test(loraFile)) throw new AiError(422, 'input.invalid', 'weights.lora_file must be a .gguf file name');
+ source = 'weights';
+ display = String(body.display_name || repo.split('/').pop() || 'Your model').slice(0, 120);
+ weights = { repo, file, loraRepo, loraFile };
+ }
+ const idleMinutes = clamp(body.idle_minutes, 1, MAX_IDLE_MINUTES, cfg.idle_minutes || 10);
+ const maxRuntime = Math.min(cfg.max_runtime_minutes || MAX_RUNTIME_MINUTES, MAX_RUNTIME_MINUTES);
+ const t = now();
+ const id = 'oep_' + ids.ulid();
+ await db.prepare(`INSERT INTO own_endpoints (id, subject, status, source, catalog_key, display_name, weights_repo, weights_file,
+ lora_repo, lora_file, context_window, idle_minutes, max_runtime_minutes, month, created_at, updated_at)
+ VALUES (?, ?, 'requested', ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`).run(
+ id, String(subject), source, spec ? spec.key : null, display,
+ weights ? weights.repo : null, weights ? weights.file : null, weights ? weights.loraRepo || null : null, weights ? weights.loraFile || null : null,
+ spec ? spec.context_window : 8192, idleMinutes, maxRuntime, at(t).slice(0, 7), at(t), at(t));
+ return describe(await rowById(id));
+ }
+
+ /** The Billing hold of a start: the worst case the rent may cost, taken before the marketplace. */
+ async function takeHold(row, { hourly, reserved, maxMinutes, marketplace, offerPrice }) {
+ const mgr = manager();
+ if (!mgr) throw new AiError(409, 'own_endpoint.billing_off', BILLING_OFF);
+ const key = 'gpu-hold:' + row.id;
+ const fingerprint = sha256([row.id, row.subject, String(hourly), String(reserved), String(maxMinutes), marketplace].join(':'));
+ const hold = await mgr.hold({
+ idempotency_key: key,
+ subject: { type: 'user', id: row.subject },
+ max_nano_usd: String(reserved),
+ rate_version: 'gpu-second@1',
+ request_fingerprint: fingerprint,
+ service: 'ai',
+ }, 'own-' + row.id);
+ const holdId = hold && (hold.hold_id || hold.id);
+ if (!holdId) throw new AiError(503, 'billing.unavailable', 'Billing did not answer with a hold');
+ await db.prepare(`UPDATE own_endpoints SET billing_hold_id = ?, marketplace = ?, price_per_hour = ?, hourly_nano = ?,
+ reserved_nano = ?, max_runtime_minutes = ?, updated_at = ? WHERE id = ?`)
+ .run(String(holdId), marketplace, String(offerPrice || ''), String(hourly), String(reserved), Number(maxMinutes), at(now()), row.id);
+ return holdId;
+ }
+
+ /** Start a requested endpoint: the provisioner searches, checks the cap, takes the hold, rents. */
+ async function start(row) {
+ const cfg = gpuConfig();
+ if (!cfg.enabled) {
+ await refuse(row.id, plainGpuReason(cfg.reason));
+ return { ok: false, reason: cfg.reason };
+ }
+ if (!manager()) {
+ await refuse(row.id, BILLING_OFF);
+ return { ok: false, reason: BILLING_OFF };
+ }
+ const spec = specFor(row);
+ if (!spec) {
+ await refuse(row.id, 'The model this endpoint asked for is no longer available.');
+ return { ok: false, reason: 'no such model' };
+ }
+ const res = await gpus.launchSpec(spec, {
+ actor: 'user:' + row.subject,
+ ownEndpointId: row.id,
+ idleMinutes: row.idle_minutes,
+ maxRuntimeMinutes: row.max_runtime_minutes,
+ beforeCreate: async ({ offer, hourly, reserved, maxMinutes, marketplace }) => {
+ await takeHold(row, { hourly, reserved, maxMinutes, marketplace, offerPrice: String(offer.price_per_hour || '') });
+ },
+ });
+ if (!res.ok) {
+ await db.prepare('UPDATE own_endpoints SET status = ?, error = ?, updated_at = ? WHERE id = ?')
+ .run('refused', String(res.reason || 'it could not start').slice(0, 400), at(now()), row.id);
+ await releaseHold(row, 'the endpoint did not start');
+ return res;
+ }
+ const inst = res.instance;
+ await db.prepare(`UPDATE own_endpoints SET status = 'starting', instance_id = ?, error = NULL, started_at = ?, updated_at = ? WHERE id = ?`)
+ .run(String(inst.id), at(now()), at(now()), row.id);
+ return res;
+ }
+
+ async function refuse(id, reason) {
+ await db.prepare("UPDATE own_endpoints SET status = 'refused', error = ?, updated_at = ? WHERE id = ?").run(String(reason).slice(0, 400), at(now()), id);
+ }
+
+ async function releaseHold(row, reason) {
+ if (!row || !row.billing_hold_id || row.billing_settled_at) return;
+ const mgr = manager();
+ if (!mgr) return;
+ try {
+ await mgr.release(row.billing_hold_id, 'own-' + row.id);
+ await db.prepare('UPDATE own_endpoints SET billing_settled_at = ?, updated_at = ? WHERE id = ?').run(at(now()), at(now()), row.id);
+ } catch (err) {
+ log.warn && log.warn('[own-endpoint] releasing the hold of ' + row.id + ' failed: ' + String(err && err.message));
+ }
+ }
+
+ /**
+ * Settle the exact seconds of a finished rent against the person's Balance. Billing's charge is
+ * ceil(units x rate / 1e6) + per_request; the seconds go in as milliseconds and the per-request
+ * remainder carries the last nano, so the charge equals what the GPU ledger says it cost.
+ */
+ async function settle(row) {
+ if (!row || row.billing_settled_at) return null;
+ const inst = row.instance_id ? await db.prepare('SELECT * FROM gpu_instances WHERE id = ?').get(row.instance_id) : null;
+ const spent = BigInt((inst && inst.spent_nano) || 0);
+ const mgr = manager();
+ if (!mgr || !row.billing_hold_id) {
+ await db.prepare('UPDATE own_endpoints SET billing_settled_at = ?, updated_at = ? WHERE id = ?').run(at(now()), at(now()), row.id);
+ return null;
+ }
+ if (spent <= 0n) {
+ await releaseHold(row, 'it never ran');
+ return null;
+ }
+ const startMs = Date.parse((inst && inst.started_at) || row.started_at || at(now()));
+ const endMs = Date.parse((inst && inst.destroyed_at) || at(now()));
+ const ms = Number.isFinite(startMs) && Number.isFinite(endMs) ? Math.max(0, Math.min(Number.MAX_SAFE_INTEGER, endMs - startMs)) : 0;
+ let rate = 0n;
+ let perRequest = spent;
+ if (ms > 0) {
+ rate = (spent * 1000000n) / BigInt(ms);
+ const part = (BigInt(ms) * rate + 999999n) / 1000000n;
+ perRequest = spent > part ? spent - part : 0n;
+ }
+ const body = {
+ idempotency_key: 'gpu-settle:' + row.id,
+ usage: { input_tokens: ms, cached_read_tokens: 0, cache_write_tokens: 0, output_tokens: 0 },
+ retail: {
+ version: 'gpu-second@1',
+ nano_usd_per_m: { input: rate.toString(), cached_read: '0', cache_write: '0', output: '0' },
+ per_request_nano_usd: perRequest.toString(),
+ },
+ supply_cost_nano_usd: spent.toString(),
+ byok: false,
+ attempts: [],
+ };
+ try {
+ await mgr.settle(row.billing_hold_id, body, 'own-' + row.id);
+ } catch (err) {
+ if (typeof mgr.enqueueSettle === 'function') {
+ await mgr.enqueueSettle({ holdId: row.billing_hold_id, requestId: 'own-' + row.id, body, error: err });
+ } else {
+ throw err;
+ }
+ }
+ await db.prepare('UPDATE own_endpoints SET billing_settled_at = ?, charged_nano = ?, updated_at = ? WHERE id = ?')
+ .run(at(now()), spent.toString(), at(now()), row.id);
+ return spent.toString();
+ }
+
+ /** The provisioner calls this when a rent becomes ready: the public URL is the AI proxy. */
+ async function ready(instanceRow, _endpoint) {
+ if (!instanceRow || !instanceRow.own_endpoint_id) return;
+ const url = publicBase() + '/' + instanceRow.own_endpoint_id + '/v1';
+ await db.prepare("UPDATE own_endpoints SET status = 'running', endpoint = ?, url = ?, ready_at = ?, error = NULL, updated_at = ? WHERE id = ? AND status IN ('starting','requested')")
+ .run(String(_endpoint || ''), url, at(now()), at(now()), instanceRow.own_endpoint_id);
+ }
+
+ /** The provisioner calls this when a private rent is given back (idle, stop, death, time). */
+ async function afterDestroy(instanceRow, reason) {
+ if (!instanceRow || !instanceRow.own_endpoint_id) return;
+ const row = await rowById(instanceRow.own_endpoint_id);
+ if (!row) return;
+ const fresh = await db.prepare('SELECT * FROM gpu_instances WHERE id = ?').get(instanceRow.id);
+ const failed = /answer|died|state|never|dead|error|refused/i.test(String(reason || ''));
+ await db.prepare('UPDATE own_endpoints SET status = ?, stopped_at = ?, error = ?, updated_at = ? WHERE id = ?')
+ .run(failed ? 'error' : 'stopped', at(now()), reasonText(reason), at(now()), row.id);
+ await settle(row).catch((err) => log.warn && log.warn('[own-endpoint] the settle of ' + row.id + ' failed: ' + String(err && err.message)));
+ return fresh;
+ }
+
+ /** Stop an endpoint: the reconciler gives the rent back, then the hold settles. */
+ async function stop(subject, id) {
+ const row = await ownedRow(subject, id);
+ if (!ACTIVE.includes(row.status)) return describeWithInstance(row);
+ if (row.status === 'requested' && !row.instance_id) {
+ // it was never started: nothing to give back, and any hold goes with it
+ await releaseHold(row, 'it never started');
+ await db.prepare("UPDATE own_endpoints SET status = 'stopped', stopped_at = ?, error = NULL, updated_at = ? WHERE id = ?").run(at(now()), at(now()), row.id);
+ return describeWithInstance(await rowById(row.id));
+ }
+ await db.prepare("UPDATE own_endpoints SET status = 'stopping', updated_at = ? WHERE id = ?").run(at(now()), row.id);
+ return describeWithInstance(await rowById(row.id));
+ }
+
+ /** One pass: start the requested, stop the stopping, and settle a rent whose death the provisioner saw. */
+ async function tick() {
+ const rows = await db.prepare('SELECT * FROM own_endpoints ORDER BY created_at').all();
+ let started = 0;
+ let stopped = 0;
+ let settled = 0;
+ for (const row of rows || []) {
+ if (row.status === 'requested') {
+ const res = await start(row).catch(async (err) => {
+ await refuse(row.id, String((err && err.message) || err));
+ return { ok: false, reason: String((err && err.message) || err) };
+ });
+ if (res && res.ok) started += 1;
+ continue;
+ }
+ if (row.status === 'stopping') {
+ const inst = await liveFor(row.id);
+ if (inst) {
+ await gpus.destroyEndpoint(row.id, 'the owner stopped it', { actor: 'user:' + row.subject }).catch(() => null);
+ stopped += 1;
+ }
+ const still = await liveFor(row.id);
+ if (!still) {
+ await db.prepare("UPDATE own_endpoints SET status = 'stopped', stopped_at = ?, updated_at = ? WHERE id = ?").run(at(now()), at(now()), row.id);
+ await settle(await rowById(row.id)).catch(() => null);
+ }
+ continue;
+ }
+ if (row.instance_id && !row.billing_settled_at) {
+ const inst = await db.prepare('SELECT * FROM gpu_instances WHERE id = ?').get(row.instance_id);
+ if (inst && (inst.status === 'destroyed' || inst.status === 'dead' || inst.status === 'refused')) {
+ await afterDestroy(inst, inst.error || 'the rent ended').catch(() => null);
+ settled += 1;
+ }
+ }
+ }
+ return { started, stopped, settled };
+ }
+
+ /**
+ * Forward one request from an endpoint's owner to the rented GPU. The caller's token, headers and
+ * cookies never reach the instance: it gets the body and nothing else. Marking the use is what
+ * keeps the scale-to-zero timer honest.
+ */
+ async function forward({ subject, id, method, path, body, signal = null }) {
+ const row = await ownedRow(subject, id);
+ if (row.status !== 'running') throw new AiError(409, 'own_endpoint.not_running', row.error || 'This endpoint is not running.');
+ if (!['GET', 'POST'].includes(String(method).toUpperCase())) throw new AiError(405, 'own_endpoint.method', 'only GET and POST reach an endpoint');
+ if (!/^\/v1\/[A-Za-z0-9/_.-]+$/.test(String(path || '')) || String(path).includes('..')) throw new AiError(422, 'own_endpoint.path', 'that is not an OpenAI-compatible path');
+ const inst = await db.prepare('SELECT * FROM gpu_instances WHERE id = ?').get(row.instance_id);
+ if (!inst || inst.status !== 'running' || !inst.endpoint) throw new AiError(409, 'own_endpoint.not_running', 'The GPU is not answering yet.');
+ await gpus.noteUse(inst.provider_key);
+ const url = String(inst.endpoint).replace(/\/+$/, '') + path;
+ const headers = { Accept: 'application/json' };
+ let payload;
+ if (body !== undefined && String(method).toUpperCase() !== 'GET') {
+ headers['Content-Type'] = 'application/json';
+ payload = typeof body === 'string' ? body : JSON.stringify(body);
+ }
+ let res;
+ try {
+ res = await fetchImpl(url, { method: String(method).toUpperCase(), headers, ...(payload !== undefined ? { body: payload } : {}), signal: signal || AbortSignal.timeout(300_000) });
+ } catch (err) {
+ throw new AiError(502, 'own_endpoint.unreachable', 'The GPU did not answer: ' + String((err && err.message) || err).slice(0, 200));
+ }
+ return { status: res.status, contentType: res.headers.get('content-type') || 'application/json', body: res.body };
+ }
+
+ return { status, create, list, get, stop, tick, specFor, ready, afterDestroy, forward, describe, settle };
+}
+
+module.exports = { createOwnEndpoints, MAX_ACTIVE, MAX_IDLE_MINUTES, MAX_RUNTIME_MINUTES, BILLING_OFF, plainGpuReason, reasonText };
@@ -85,6 +85,7 @@ function createGpuProvisioner({
@@ -85,6 +85,7 @@ function createGpuProvisioner({
env = process.env,
env = process.env,
log = console,
log = console,
models = catalog.list(),
models = catalog.list(),
+ ownEndpoints = null,
} = {}) {
} = {}) {
const now = () => clock.now();
const now = () => clock.now();
const iso = (ms) => new Date(ms).toISOString();
const iso = (ms) => new Date(ms).toISOString();
@@ -229,16 +230,35 @@ function createGpuProvisioner({
@@ -229,16 +230,35 @@ function createGpuProvisioner({
return false;
return false;
}
}
+ /** The live private rent of an endpoint, if one is still up. */
+ async function liveInstanceForEndpoint(ownEndpointId) {
+ if (!ownEndpointId) return null;
+ const row = await db.prepare('SELECT * FROM gpu_instances WHERE own_endpoint_id = ? AND status IN (?, ?, ?) ORDER BY created_at DESC LIMIT 1')
+ .get(String(ownEndpointId), 'starting', 'running', 'stopping');
+ return row || null;
+ }
+
/**
/**
* Start an instance for a model: the cheapest offer that fits, inside the cap.
* Start an instance for a model: the cheapest offer that fits, inside the cap.
* Returns { ok, reason } - it never throws for the owner's reasons (off, no offer, cap).
* Returns { ok, reason } - it never throws for the owner's reasons (off, no offer, cap).
*/
*/
- async function launch(modelKey, { actor = 'service:ai' } = {}) {
- const cfg = config();
- if (!cfg.enabled) return { ok: false, code: 'gpu.off', reason: cfg.reason };
+ async function launch(modelKey, opts = {}) {
const spec = models.find((m) => m.key === modelKey) || catalog.find(modelKey);
const spec = models.find((m) => m.key === modelKey) || catalog.find(modelKey);
if (!spec) throw new AiError(404, 'ai.not_found', 'no such GPU model');
if (!spec) throw new AiError(404, 'ai.not_found', 'no such GPU model');
- const existing = await liveInstance(spec.key);
+ return launchSpec(spec, opts);
+ }
+
+ /**
+ * Start an instance for a spec. A shared model offer registers itself with the router; a person's own
+ * endpoint (ownEndpointId set) never does: it stays private, and its owner reaches it through AI.
+ *
+ * beforeCreate({ offer, hourly, reserved, maxMinutes, marketplace }) the endpoint's Billing hold, if
+ * any, is taken here - before the marketplace is called, so nothing runs that the person cannot pay.
+ */
+ async function launchSpec(spec, { actor = 'service:ai', ownEndpointId = null, idleMinutes = null, maxRuntimeMinutes = null, beforeCreate = null } = {}) {
+ const cfg = config();
+ if (!cfg.enabled) return { ok: false, code: 'gpu.off', reason: cfg.reason };
+ const existing = ownEndpointId ? await liveInstanceForEndpoint(ownEndpointId) : await liveInstance(spec.key);
if (existing) return { ok: true, instance: describe(existing), note: 'already running' };
if (existing) return { ok: true, instance: describe(existing), note: 'already running' };
const adapter = adapterFor(spec.marketplace, cfg);
const adapter = adapterFor(spec.marketplace, cfg);
if (!adapter) return { ok: false, code: 'gpu.no_key', reason: 'no ' + spec.marketplace + ' API key' };
if (!adapter) return { ok: false, code: 'gpu.no_key', reason: 'no ' + spec.marketplace + ' API key' };
@@ -278,12 +298,24 @@ function createGpuProvisioner({
@@ -278,12 +298,24 @@ function createGpuProvisioner({
if (affordableHours < 1n) {
if (affordableHours < 1n) {
return { ok: false, code: 'gpu.cap', reason: 'the monthly cap leaves less than an hour of GPU time (' + money.usdText(cap - sum.committed_nano) + ' left)' };
return { ok: false, code: 'gpu.cap', reason: 'the monthly cap leaves less than an hour of GPU time (' + money.usdText(cap - sum.committed_nano) + ' left)' };
}
}
- const maxMinutes = Math.min(cfg.max_runtime_minutes, Number(affordableHours) * 60);
+ const affordableMinutes = Number(affordableHours) * 60;
+ const requestedMax = maxRuntimeMinutes == null ? affordableMinutes : Math.max(1, Math.min(affordableMinutes, Math.trunc(Number(maxRuntimeMinutes) || 0)));
+ const maxMinutes = Math.min(cfg.max_runtime_minutes, requestedMax);
const reserved = hourly * BigInt(Math.ceil(maxMinutes / 60));
const reserved = hourly * BigInt(Math.ceil(maxMinutes / 60));
if (sum.committed_nano + reserved > cap) {
if (sum.committed_nano + reserved > cap) {
return { ok: false, code: 'gpu.cap', reason: 'the monthly cap leaves no room for ' + spec.display_name };
return { ok: false, code: 'gpu.cap', reason: 'the monthly cap leaves no room for ' + spec.display_name };
}
}
+ // 2b. the payer's hold, for a person's own endpoint: taken before the marketplace is called, so an
+ // instance never runs that the person cannot pay for.
+ if (beforeCreate) {
+ try {
+ await beforeCreate({ offer, hourly, reserved, maxMinutes, marketplace: spec.marketplace });
+ } catch (err) {
+ return { ok: false, code: (err && err.code) || 'gpu.billing', reason: String((err && err.message) || err).slice(0, 300) };
+ }
+ }
+
// 3. the row comes first. The marketplace is told the label of a row that already exists and
// 3. the row comes first. The marketplace is told the label of a row that already exists and
// holds the reservation, so an instance we paid for is never an orphan: the sweep finds it by
// holds the reservation, so an instance we paid for is never an orphan: the sweep finds it by
// the label and gives it back when no live row of ours owns it.
// the label and gives it back when no live row of ours owns it.
@@ -291,10 +323,12 @@ function createGpuProvisioner({
@@ -291,10 +323,12 @@ function createGpuProvisioner({
const t = now();
const t = now();
await db.prepare(`INSERT INTO gpu_instances (id, model_key, provider_key, marketplace, offer_id, remote_id, status, endpoint, month,
await db.prepare(`INSERT INTO gpu_instances (id, model_key, provider_key, marketplace, offer_id, remote_id, status, endpoint, month,
gpu_name, gpu_ram_mb, num_gpus, disk_gb, reliability, region, price_per_hour, hourly_nano, reserved_nano, spent_nano,
gpu_name, gpu_ram_mb, num_gpus, disk_gb, reliability, region, price_per_hour, hourly_nano, reserved_nano, spent_nano,
- max_runtime_minutes, started_at, charged_at, destroy_failures, label, cold_start_ms, health_failures, created_at, updated_at)
+ max_runtime_minutes, started_at, charged_at, destroy_failures, label, cold_start_ms, health_failures, created_at, updated_at,
+ own_endpoint_id, idle_minutes)
VALUES (@id, @model_key, @provider_key, @marketplace, @offer_id, NULL, 'starting', NULL, @month,
VALUES (@id, @model_key, @provider_key, @marketplace, @offer_id, NULL, 'starting', NULL, @month,
@gpu_name, @gpu_ram_mb, @num_gpus, @disk_gb, @reliability, @region, @price_per_hour, @hourly_nano, @reserved_nano, 0,
@gpu_name, @gpu_ram_mb, @num_gpus, @disk_gb, @reliability, @region, @price_per_hour, @hourly_nano, @reserved_nano, 0,
- @max_runtime_minutes, @started_at, @charged_at, 0, @label, NULL, 0, @created_at, @updated_at)`)
+ @max_runtime_minutes, @started_at, @charged_at, 0, @label, NULL, 0, @created_at, @updated_at,
+ @own_endpoint_id, @idle_minutes)`)
.run({
.run({
id, model_key: spec.key, provider_key: spec.provider_key, marketplace: spec.marketplace,
id, model_key: spec.key, provider_key: spec.provider_key, marketplace: spec.marketplace,
offer_id: String(offer.id), month: monthOf(t),
offer_id: String(offer.id), month: monthOf(t),
@@ -304,10 +338,12 @@ function createGpuProvisioner({
@@ -304,10 +338,12 @@ function createGpuProvisioner({
price_per_hour: String(offer.price_per_hour), hourly_nano: hourly.toString(), reserved_nano: reserved.toString(),
price_per_hour: String(offer.price_per_hour), hourly_nano: hourly.toString(), reserved_nano: reserved.toString(),
max_runtime_minutes: maxMinutes, started_at: iso(t), charged_at: iso(t), label: id,
max_runtime_minutes: maxMinutes, started_at: iso(t), charged_at: iso(t), label: id,
created_at: iso(t), updated_at: iso(t),
created_at: iso(t), updated_at: iso(t),
+ own_endpoint_id: ownEndpointId, idle_minutes: idleMinutes == null ? null : clampInt(idleMinutes, 1, 1440, cfg.idle_minutes),
});
});
await audit(actor, 'gpu.instance.requested', 'gpu_instance', id, {
await audit(actor, 'gpu.instance.requested', 'gpu_instance', id, {
model: spec.key, marketplace: spec.marketplace, offer_id: String(offer.id), hourly_nano: hourly.toString(),
model: spec.key, marketplace: spec.marketplace, offer_id: String(offer.id), hourly_nano: hourly.toString(),
reserved_nano: reserved.toString(), max_runtime_minutes: maxMinutes, label: id,
reserved_nano: reserved.toString(), max_runtime_minutes: maxMinutes, label: id,
+ ...(ownEndpointId ? { own_endpoint_id: ownEndpointId } : {}),
});
});
// 4. rent it. The marketplace is given the label of the row above, so the instance can always be
// 4. rent it. The marketplace is given the label of the row above, so the instance can always be
@@ -435,7 +471,12 @@ function createGpuProvisioner({
@@ -435,7 +471,12 @@ function createGpuProvisioner({
await charge(fresh || row, t, reason);
await charge(fresh || row, t, reason);
await db.prepare("UPDATE gpu_instances SET status = 'destroyed', destroyed_at = ?, endpoint = NULL, destroy_failures = 0, updated_at = ? WHERE id = ?")
await db.prepare("UPDATE gpu_instances SET status = 'destroyed', destroyed_at = ?, endpoint = NULL, destroy_failures = 0, updated_at = ? WHERE id = ?")
.run(iso(t), iso(t), row.id);
.run(iso(t), iso(t), row.id);
- await disableProvider(row.provider_key, reason);
+ if (row.own_endpoint_id && ownEndpoints && typeof ownEndpoints.afterDestroy === 'function') {
+ // the person's endpoint is over: settle what it cost and say why it stopped
+ await ownEndpoints.afterDestroy(fresh || row, reason).catch((err) => log.warn && log.warn('[gpu] the endpoint settle failed: ' + String(err && err.message)));
+ } else {
+ await disableProvider(row.provider_key, reason);
+ }
await audit(actor, 'gpu.instance.destroy', 'gpu_instance', row.id, { reason, model: row.model_key });
await audit(actor, 'gpu.instance.destroy', 'gpu_instance', row.id, { reason, model: row.model_key });
return { ok: true, reason };
return { ok: true, reason };
}
}
@@ -546,8 +587,12 @@ function createGpuProvisioner({
@@ -546,8 +587,12 @@ function createGpuProvisioner({
/** One pass over the live instances: the reservation, the idle timer, the health. */
/** One pass over the live instances: the reservation, the idle timer, the health. */
async function reconcileInstance(row, cfg) {
async function reconcileInstance(row, cfg) {
- const spec = models.find((m) => m.key === row.model_key) || catalog.find(row.model_key);
+ const priv = Boolean(row.own_endpoint_id);
+ const spec = priv
+ ? (ownEndpoints && typeof ownEndpoints.specFor === 'function' ? ownEndpoints.specFor(row) : null)
+ : (models.find((m) => m.key === row.model_key) || catalog.find(row.model_key));
if (!spec) return { changed: false };
if (!spec) return { changed: false };
+ const idleMinutes = Number(row.idle_minutes) > 0 ? Number(row.idle_minutes) : cfg.idle_minutes;
const t = now();
const t = now();
const started = toMs(row.started_at) || t;
const started = toMs(row.started_at) || t;
@@ -562,12 +607,12 @@ function createGpuProvisioner({
@@ -562,12 +607,12 @@ function createGpuProvisioner({
// 2. the idle timer: the offer scales to zero (unless it is inside the warm window). The clock runs
// 2. the idle timer: the offer scales to zero (unless it is inside the warm window). The clock runs
// from the moment the instance was ready, not from the moment it was rented: an instance that is still
// from the moment the instance was ready, not from the moment it was rented: an instance that is still
- // starting is never idle.
+ // starting is never idle. A person's own endpoint has its own idle timer.
if (row.status === 'running') {
if (row.status === 'running') {
const lastUsed = toMs(row.last_used_at) || toMs(row.ready_at) || started;
const lastUsed = toMs(row.last_used_at) || toMs(row.ready_at) || started;
- if (t - lastUsed >= cfg.idle_minutes * 60_000 && !inWarmWindow(cfg.warm_window, t)) {
+ if (t - lastUsed >= idleMinutes * 60_000 && !inWarmWindow(cfg.warm_window, t)) {
await destroy({ ...row, charged_at: row.charged_at }, 'idle');
await destroy({ ...row, charged_at: row.charged_at }, 'idle');
- return { changed: true, action: 'scaled_to_zero', reason: 'idle for ' + cfg.idle_minutes + ' minutes' };
+ return { changed: true, action: 'scaled_to_zero', reason: 'idle for ' + idleMinutes + ' minutes' };
}
}
}
}
@@ -587,7 +632,7 @@ function createGpuProvisioner({
@@ -587,7 +632,7 @@ function createGpuProvisioner({
const state = String((info && info.state) || 'unknown').toLowerCase();
const state = String((info && info.state) || 'unknown').toLowerCase();
if (DEAD_STATES.includes(state)) {
if (DEAD_STATES.includes(state)) {
await destroy({ ...row, charged_at: row.charged_at }, 'the instance ' + state);
await destroy({ ...row, charged_at: row.charged_at }, 'the instance ' + state);
- await request(spec.key, 'launch', 'service:ai');
+ if (!priv) await request(spec.key, 'launch', 'service:ai');
return { changed: true, action: 'died', reason: 'the instance ' + state };
return { changed: true, action: 'died', reason: 'the instance ' + state };
}
}
const endpoint = info && info.endpoint ? String(info.endpoint) : '';
const endpoint = info && info.endpoint ? String(info.endpoint) : '';
@@ -598,7 +643,7 @@ function createGpuProvisioner({
@@ -598,7 +643,7 @@ function createGpuProvisioner({
.run(failures, iso(t), iso(t), row.id);
.run(failures, iso(t), iso(t), row.id);
if (failures >= HEALTH_FAILURES) {
if (failures >= HEALTH_FAILURES) {
await destroy({ ...row, charged_at: row.charged_at }, 'the endpoint never answered');
await destroy({ ...row, charged_at: row.charged_at }, 'the endpoint never answered');
- await request(spec.key, 'launch', 'service:ai');
+ if (!priv) await request(spec.key, 'launch', 'service:ai');
return { changed: true, action: 'died', reason: 'the endpoint never answered' };
return { changed: true, action: 'died', reason: 'the endpoint never answered' };
}
}
return { changed: false, health_failures: failures };
return { changed: false, health_failures: failures };
@@ -607,7 +652,11 @@ function createGpuProvisioner({
@@ -607,7 +652,11 @@ function createGpuProvisioner({
// the cold start: from the launch to the first answered request
// the cold start: from the launch to the first answered request
await db.prepare("UPDATE gpu_instances SET status = 'running', endpoint = ?, ready_at = ?, last_health_at = ?, health_failures = 0, cold_start_ms = ?, updated_at = ? WHERE id = ?")
await db.prepare("UPDATE gpu_instances SET status = 'running', endpoint = ?, ready_at = ?, last_health_at = ?, health_failures = 0, cold_start_ms = ?, updated_at = ? WHERE id = ?")
.run(endpoint, iso(t), iso(t), String(Math.max(0, t - started)), iso(t), row.id);
.run(endpoint, iso(t), iso(t), String(Math.max(0, t - started)), iso(t), row.id);
- await register(spec, { ...row, endpoint });
+ if (priv) {
+ if (ownEndpoints && typeof ownEndpoints.ready === 'function') await ownEndpoints.ready({ ...row, endpoint }, endpoint);
+ } else {
+ await register(spec, { ...row, endpoint });
+ }
return { changed: true, action: 'ready' };
return { changed: true, action: 'ready' };
}
}
return { changed: false };
return { changed: false };
@@ -626,9 +675,10 @@ function createGpuProvisioner({
@@ -626,9 +675,10 @@ function createGpuProvisioner({
await db.prepare('UPDATE gpu_instances SET health_failures = ?, last_health_at = ?, updated_at = ? WHERE id = ?')
await db.prepare('UPDATE gpu_instances SET health_failures = ?, last_health_at = ?, updated_at = ? WHERE id = ?')
.run(failures, iso(t), iso(t), row.id);
.run(failures, iso(t), iso(t), row.id);
if (failures < HEALTH_FAILURES) return { changed: false, health_failures: failures };
if (failures < HEALTH_FAILURES) return { changed: false, health_failures: failures };
- // the instance died: the offer leaves the pool and a new one is rented
+ // the instance died: a shared offer leaves the pool and a new one is rented; a person's own
+ // endpoint stops (they can start it again when they want it).
await destroy({ ...row, charged_at: row.charged_at }, 'the endpoint stopped answering');
await destroy({ ...row, charged_at: row.charged_at }, 'the endpoint stopped answering');
- await request(spec.key, 'launch', 'service:ai');
+ if (!priv) await request(spec.key, 'launch', 'service:ai');
return { changed: true, action: 'died', reason: 'the endpoint stopped answering' };
return { changed: true, action: 'died', reason: 'the endpoint stopped answering' };
}
}
return { changed: false };
return { changed: false };
@@ -670,6 +720,8 @@ function createGpuProvisioner({
@@ -670,6 +720,8 @@ function createGpuProvisioner({
reserved_nano: String(toNano(row.reserved_nano)),
reserved_nano: String(toNano(row.reserved_nano)),
spent_nano: String(toNano(row.spent_nano)),
spent_nano: String(toNano(row.spent_nano)),
max_runtime_minutes: row.max_runtime_minutes,
max_runtime_minutes: row.max_runtime_minutes,
+ idle_minutes: row.idle_minutes == null ? null : Number(row.idle_minutes),
+ own_endpoint_id: row.own_endpoint_id || null,
label: row.label,
label: row.label,
cold_start_ms: row.cold_start_ms == null ? null : String(row.cold_start_ms),
cold_start_ms: row.cold_start_ms == null ? null : String(row.cold_start_ms),
destroy_failures: Number(row.destroy_failures || 0),
destroy_failures: Number(row.destroy_failures || 0),
@@ -778,16 +830,47 @@ function createGpuProvisioner({
@@ -778,16 +830,47 @@ function createGpuProvisioner({
};
};
}
}
+ /** The private-endpoint hooks, attached once the endpoint module exists (server/index.js). */
+ function attachEndpoints(hooks) {
+ ownEndpoints = hooks || null;
+ return ownEndpoints;
+ }
+
+ /** The live rent of an endpoint (for its owner's proxy and its stop). */
+ async function instanceForEndpoint(ownEndpointId) {
+ return liveInstanceForEndpoint(ownEndpointId);
+ }
+
+ /** A request reached an offer: the idle timer runs from now, not from the moment it became ready. */
+ async function noteUse(providerKey) {
+ if (!providerKey) return 0;
+ const res = await db.prepare("UPDATE gpu_instances SET last_used_at = ?, updated_at = ? WHERE provider_key = ? AND status = 'running'")
+ .run(iso(now()), iso(now()), String(providerKey));
+ return Number(res && res.changes) || 0;
+ }
+
+ /** Give an endpoint's rent back (its owner pressed stop); the afterDestroy hook settles the money. */
+ async function destroyEndpoint(ownEndpointId, reason = 'the owner stopped it', { actor = 'service:ai' } = {}) {
+ const row = await liveInstanceForEndpoint(ownEndpointId);
+ if (!row) return { ok: true, note: 'not running' };
+ return destroy(row, reason, { actor });
+ }
+
return {
return {
state,
state,
tick,
tick,
launch,
launch,
+ attachEndpoints,
+ launchSpec,
request,
request,
config,
config,
moneySummary,
moneySummary,
sweepOrphans,
sweepOrphans,
destroyFailures,
destroyFailures,
orphanCount,
orphanCount,
+ instanceForEndpoint,
+ noteUse,
+ destroyEndpoint,
/** For the tests: the same thing the reconciler does with one instance. */
/** For the tests: the same thing the reconciler does with one instance. */
reconcileInstance,
reconcileInstance,
inWarmWindow,
inWarmWindow,
@@ -131,6 +131,11 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.now
@@ -131,6 +131,11 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.now
startSettlements() { return credits ? credits.startSettlements() : null; },
startSettlements() { return credits ? credits.startSettlements() : null; },
stopSettlements() { return credits ? credits.stopSettlements() : undefined; },
stopSettlements() { return credits ? credits.stopSettlements() : undefined; },
};
};
+ // A person's own endpoint (the design's product line 6): a private rent on the same GPU provisioner, with a
+ // Billing hold before it starts and the exact seconds settled when it stops. It is off until the owner has
+ // a GPU key and a monthly cap AND Balance billing is on; the public status says which one is missing.
+ const ownEndpoints = require('./gpu/endpoints').createOwnEndpoints({ db, gpus, credits: creditsRef, config, clock, fetchImpl, log });
+ gpus.attachEndpoints(ownEndpoints);
const runs = createRuns({ db, registry, engine, cache, quotas, config, clock, log, userModules, credentials, preferences, tiers, projectSecrets, pool, runBudgets, credits: creditsRef, community, presets, receipts, abuse, webhooks });
const runs = createRuns({ db, registry, engine, cache, quotas, config, clock, log, userModules, credentials, preferences, tiers, projectSecrets, pool, runBudgets, credits: creditsRef, community, presets, receipts, abuse, webhooks });
await seed({ registry, quotas, config, env, db });
await seed({ registry, quotas, config, env, db });
// The seed creates or moves providers from the environment: publish their origins (the schema and every key's
// The seed creates or moves providers from the environment: publish their origins (the schema and every key's
@@ -168,7 +173,7 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.now
@@ -168,7 +173,7 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.now
clientSecret: env.OV_OAUTH_CLIENT_SECRET, fetch: fetchImpl,
clientSecret: env.OV_OAUTH_CLIENT_SECRET, fetch: fetchImpl,
}) : null;
}) : null;
const benchEvents = createBenchEvents({ bench, accountData, accountSend, config, log, fetchImpl, clientSecret: env.OV_OAUTH_CLIENT_SECRET });
const benchEvents = createBenchEvents({ bench, accountData, accountSend, config, log, fetchImpl, clientSecret: env.OV_OAUTH_CLIENT_SECRET });
- const app = createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env, clock, fetchImpl, log, credentials, preferences, grants, vault, settings, benchEvents, runBudgets, memory, memoryImport, memoryServices, memorySharing, community, endpoints, webhooks, presets, receipts, balance, gpus, abuse, priceBook });
+ const app = createApp({ config, db, registry, pool, quotas, cache, runs, auth, keys, env, clock, fetchImpl, log, credentials, preferences, grants, vault, settings, benchEvents, runBudgets, memory, memoryImport, memoryServices, memorySharing, community, endpoints, webhooks, presets, receipts, balance, gpus, abuse, priceBook, ownEndpoints });
let jwks = null;
let jwks = null;
let keyLoaded = Promise.resolve(null);
let keyLoaded = Promise.resolve(null);
if (!keys.pinned && config.nodeEnv !== 'test') {
if (!keys.pinned && config.nodeEnv !== 'test') {
@@ -196,7 +201,9 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.now
@@ -196,7 +201,9 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.now
let gpuReconciler = null;
let gpuReconciler = null;
if (listen && config.nodeEnv !== 'test') {
if (listen && config.nodeEnv !== 'test') {
gpuReconciler = setInterval(() => {
gpuReconciler = setInterval(() => {
- gpus.tick().catch((err) => log.warn('[gpu] the reconciler failed: ' + err.message));
+ ownEndpoints.tick()
+ .then(() => gpus.tick())
+ .catch((err) => log.warn('[gpu] the reconciler failed: ' + err.message));
}, 60 * 1000);
}, 60 * 1000);
gpuReconciler.unref?.();
gpuReconciler.unref?.();
}
}
@@ -266,7 +273,7 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.now
@@ -266,7 +273,7 @@ async function start({ config, db: givenDb = null, clock = { now: () => Date.now
if (!givenDb) await db.close();
if (!givenDb) await db.close();
}
}
- return { config, db, registry, pool, quotas, tiers, cache, fetcher, engine, runs, keys, keyLoaded, auth, app, server, bench, benchEvents, memory, memoryJobs, memoryServices, memorySharing, priceBook, credits: creditsRef, settings, vault, community, endpoints, webhooks, gpus, abuse, close };
+ return { config, db, registry, pool, quotas, tiers, cache, fetcher, engine, runs, keys, keyLoaded, auth, app, server, bench, benchEvents, memory, memoryJobs, memoryServices, memorySharing, priceBook, credits: creditsRef, settings, vault, community, endpoints, webhooks, gpus, abuse, ownEndpoints, close };
}
}
if (require.main === module) {
if (require.main === module) {
@@ -398,6 +398,47 @@ route('get', '/exchange/catalog', {
@@ -398,6 +398,47 @@ route('get', '/exchange/catalog', {
models: arrayOf(exchangeModel),
models: arrayOf(exchangeModel),
} }) },
} }) },
});
});
+route('get', '/exchange/endpoints', {
+ operationId: 'exchangeOwnEndpointStatus', summary: 'Your own endpoint: whether it is on, and the plain reason when it is not', tags: ['exchange'], security: [],
+ description: 'Public. Your own endpoint (the Exchange remake, product line 6) is a private GPU rent: a chosen model or your own Hugging Face weights, reached only by its owner, scaled to zero when idle, and paid for by the owner\'s Balance. It is off while the owner has no GPU provider key (ai.gpus vast.key or runpod.key), no monthly GPU cap (ai.gpus monthly_cap), or Balance billing is not on. `reason` is a plain sentence the Exchange page shows; `reasons` lists each missing piece. `models` is what a private endpoint may serve today.',
+ responses: { 200: json('The status', { type: 'object', required: ['enabled', 'reason'], properties: {
+ enabled: { type: 'boolean' },
+ reason: str(),
+ reasons: arrayOf(str()),
+ billing: { type: 'boolean' },
+ marketplace: nullable('string'),
+ idle_minutes: integer(), max_idle_minutes: integer(), max_runtime_minutes: integer(),
+ public_base: str(),
+ models: arrayOf({ type: 'object', properties: { key: str(), display_name: str(), context_window: integer(), vram_mb: integer(), gpu_names: arrayOf(str()) } }),
+ } }) },
+});
+route('get', '/own-endpoints', {
+ operationId: 'listOwnEndpoints', summary: 'The caller\'s private endpoints', tags: ['exchange'], security: [{ bearer: [] }],
+ description: 'Only the person themselves: a Network user token (audience openvibe.network or openvibe.ai) whose subject is usr_…. A service, app, module or node token is refused whatever capabilities it holds. A private endpoint is never registered as an offer, so no router and no catalogue sees it.',
+ responses: { 200: json('The endpoints and the public status', { type: 'object', required: ['endpoints'], properties: { endpoints: arrayOf({ type: 'object' }), status: { type: 'object' } } }), ...errors([401, 403]) },
+});
+route('post', '/own-endpoints', {
+ operationId: 'createOwnEndpoint', summary: 'Start a private endpoint on a chosen model or your own weights', tags: ['exchange'], security: [{ bearer: [] }],
+ description: 'Only a person. Name `model` (a key from GET /exchange/endpoints) or `weights` ({ repo, file, lora_repo?, lora_file? }: a Hugging Face .gguf and an optional LoRA, fetched by the rented container). The endpoint is started by the reconciler on the cheapest fit, after a Billing hold of its worst case; a start that would pass the owner\'s monthly cap is refused with a plain reason, and nothing is rented. 409 own_endpoint.off carries the same reason the public status does.',
+ requestBody: { required: true, content: { 'application/json': { schema: { type: 'object', properties: {
+ model: str(), idle_minutes: integer({ minimum: 1, maximum: 120 }),
+ weights: { type: 'object', required: ['repo', 'file'], properties: { repo: str(), file: str(), lora_repo: str(), lora_file: str() } },
+ display_name: str(),
+ } } } } },
+ responses: { 201: json('The endpoint', { type: 'object', required: ['endpoint'], properties: { endpoint: { type: 'object' } } }), 409: problem, 422: problem, ...errors([401, 403]) },
+});
+route('get', '/own-endpoints/{id}', {
+ operationId: 'getOwnEndpoint', summary: 'One of the caller\'s endpoints', tags: ['exchange'], security: [{ bearer: [] }],
+ description: 'Only its owner. The status, the price an hour, what it has charged, the public URL while it runs, and the plain reason it stopped.',
+ parameters: [path('id', str({ pattern: '^oep_[0-9A-HJKMNP-TV-Z]{26}$' }))],
+ responses: { 200: json('The endpoint', { type: 'object', required: ['endpoint'], properties: { endpoint: { type: 'object' } } }), 404: problem, ...errors([401, 403]) },
+});
+route('post', '/own-endpoints/{id}/stop', {
+ operationId: 'stopOwnEndpoint', summary: 'Give the private endpoint back', tags: ['exchange'], security: [{ bearer: [] }],
+ description: 'Only its owner. The reconciler gives the rent back; Billing settles the seconds it ran, and the price stops.',
+ parameters: [path('id', str({ pattern: '^oep_[0-9A-HJKMNP-TV-Z]{26}$' }))],
+ responses: { 200: json('The endpoint', { type: 'object', required: ['endpoint'], properties: { endpoint: { type: 'object' } } }), 404: problem, ...errors([401, 403]) },
+});
route('get', '/exchange/models/{slug}', {
route('get', '/exchange/models/{slug}', {
operationId: 'exchangeModel', summary: 'One model and its offers', tags: ['exchange'], security: [],
operationId: 'exchangeModel', summary: 'One model and its offers', tags: ['exchange'], security: [],
description: 'Public. The slug is "<org>/<model>", lowercased, e.g. deepseek/deepseek-flash.',
description: 'Public. The slug is "<org>/<model>", lowercased, e.g. deepseek/deepseek-flash.',
@@ -140,6 +140,14 @@ const GPU_SCHEMA = Object.freeze({
@@ -140,6 +140,14 @@ const GPU_SCHEMA = Object.freeze({
'vast.key': {
'vast.key': {
type: 'string', title: 'Vast.ai API key', 'x-group': GPU_GROUP, 'x-bind': 'vast.origin',
type: 'string', title: 'Vast.ai API key', 'x-group': GPU_GROUP, 'x-bind': 'vast.origin',
description: "The owner's write-only Vast.ai API key, sealed to the origin above.",
description: "The owner's write-only Vast.ai API key, sealed to the origin above.",
+ 'x-owner-input': {
+ group: 'GPUs',
+ unlocks: 'Lets OpenVibe rent a GPU on Vast.ai for OpenVibe GPUs and for a person’s own endpoint. Without it Vast.ai is never searched and nothing is rented there.',
+ get_it_at: 'https://console.vast.ai/account/',
+ get_it_how: 'Open Account → API Keys, create a key, and paste it here. It is stored locked and never shown again.',
+ test: false,
+ },
+ 'x-required-for': 'own endpoints',
},
},
'runpod.origin': {
'runpod.origin': {
type: 'string', title: 'RunPod origin', 'x-group': GPU_GROUP,
type: 'string', title: 'RunPod origin', 'x-group': GPU_GROUP,
@@ -148,11 +156,27 @@ const GPU_SCHEMA = Object.freeze({
@@ -148,11 +156,27 @@ const GPU_SCHEMA = Object.freeze({
'runpod.key': {
'runpod.key': {
type: 'string', title: 'RunPod API key', 'x-group': GPU_GROUP, 'x-bind': 'runpod.origin',
type: 'string', title: 'RunPod API key', 'x-group': GPU_GROUP, 'x-bind': 'runpod.origin',
description: "The owner's write-only RunPod API key, sealed to the origin above.",
description: "The owner's write-only RunPod API key, sealed to the origin above.",
+ 'x-owner-input': {
+ group: 'GPUs',
+ unlocks: 'Lets OpenVibe rent a GPU on RunPod when RunPod is cheaper or Vast.ai has no fit. Without it RunPod is never searched and nothing is rented there.',
+ get_it_at: 'https://www.runpod.io/console/user/settings',
+ get_it_how: 'Open Settings → API Keys, create a key, and paste it here. It is stored locked and never shown again.',
+ test: false,
+ },
+ 'x-required-for': 'own endpoints',
},
},
monthly_cap: {
monthly_cap: {
type: 'integer', title: 'Monthly GPU cap', 'x-group': GPU_GROUP, 'x-unit': 'nano_usd', 'x-danger': true,
type: 'integer', title: 'Monthly GPU cap', 'x-group': GPU_GROUP, 'x-unit': 'nano_usd', 'x-danger': true,
minimum: 0,
minimum: 0,
description: 'The hard monthly cap for the rented GPUs, in nano-USD. The provisioner never spends past it: it refuses to start an instance and says why. Without a cap the provisioner stays off.',
description: 'The hard monthly cap for the rented GPUs, in nano-USD. The provisioner never spends past it: it refuses to start an instance and says why. Without a cap the provisioner stays off.',
+ 'x-owner-input': {
+ group: 'GPUs',
+ unlocks: 'Sets the most the rented GPUs may cost you in a month. A new instance (including a person’s own endpoint) is refused before it starts when its time would pass this cap. Without it nothing is rented.',
+ get_it_at: 'https://openvibe.network/admin#ai',
+ get_it_how: 'Decide the most you are willing to spend on rented GPUs in a month, and enter it here. Lower is safer; higher lets more models and endpoints run.',
+ test: false,
+ },
+ 'x-required-for': 'own endpoints',
},
},
idle_minutes: {
idle_minutes: {
type: 'integer', title: 'Idle minutes', 'x-group': GPU_GROUP,
type: 'integer', title: 'Idle minutes', 'x-group': GPU_GROUP,
new file mode 100644
new file mode 100644
@@ -0,0 +1,279 @@
@@ -0,0 +1,279 @@
+'use strict';
+/**
+ * Your own endpoint (the Exchange remake, product line 6): off without the owner's GPU key, cap and Balance
+ * billing; started through the same GPU provisioner (never registered as an offer); reached only by its
+ * owner's token; scaled to zero after its idle timer; and settled against Billing for the exact seconds,
+ * never past the owner's monthly cap. Every GPU and Billing call is a local fake: no test reaches the
+ * internet.
+ */
+const assert = require('assert');
+const http = require('http');
+const { boot, request, token, suite } = require('./helpers');
+const { createGpuFetch } = require('./fixtures/gpu-fetch');
+
+const t = suite('own-endpoints');
+const T0 = Date.parse('2026-11-06T12:00:00Z');
+const OWNER = { type: 'user', id: 'usr_' + 'A'.repeat(26) };
+const PERSON = 'usr_01JAB2C3D4E5F6G7H8J9K0MNP7';
+const b64 = (o) => Buffer.from(JSON.stringify(o)).toString('base64url');
+
+/** A person's Network token, as Exchange holds it in the site session (audience openvibe.network). */
+function userToken(subject = PERSON) {
+ return token('a-person', [], { sub: subject, actorType: 'user', aud: 'openvibe.network', ns: null, exp: Math.floor(T0 / 1000) + 3600, extra: { iat: Math.floor(T0 / 1000) } });
+}
+
+/** A local fake Billing: the OAuth token and the four hold routes, with a balance and a real settle charge. */
+async function fakeBilling({ balance = 10_000_000_000n } = {}) {
+ const state = { balance, held: 0n, holds: new Map(), settlements: new Map(), releases: [], calls: [] };
+ const server = http.createServer((req, res) => {
+ let raw = '';
+ req.on('data', (c) => { raw += c; });
+ req.on('end', () => {
+ const path = new URL(req.url, 'http://x').pathname;
+ let body = {};
+ try { body = raw ? JSON.parse(raw) : {}; } catch { body = {}; }
+ if (path !== '/oauth/token') state.calls.push({ path, body });
+ const send = (status, out) => { res.writeHead(status, { 'Content-Type': 'application/json' }); res.end(JSON.stringify(out)); };
+ if (path === '/oauth/token') return send(200, { access_token: 'test-token', token_type: 'Bearer', expires_in: 300 });
+ if (path === '/internal/credits/holds') {
+ const prev = state.holds.get(body.idempotency_key);
+ if (prev) return send(201, { hold_id: prev.hold_id, held_nano_usd: prev.held, available_nano_usd: String(state.balance - state.held) });
+ const amount = BigInt(String(body.max_nano_usd));
+ if (state.balance - state.held < amount) return send(402, { code: 'billing.insufficient_funds', detail: 'not enough Balance' });
+ const hold = { hold_id: `hld_${String(state.holds.size + 1).padStart(26, '0')}`, held: String(amount), status: 'active', max: String(amount) };
+ state.holds.set(body.idempotency_key, hold);
+ state.held += amount;
+ return send(201, { hold_id: hold.hold_id, held_nano_usd: hold.held, available_nano_usd: String(state.balance - state.held) });
+ }
+ let m = /^\/internal\/credits\/holds\/([^/]+)\/settle$/.exec(path);
+ if (m) {
+ const prev = state.settlements.get(body.idempotency_key);
+ if (prev) return send(201, prev);
+ const hold = [...state.holds.values()].find((x) => x.hold_id === decodeURIComponent(m[1]));
+ if (!hold) return send(404, { code: 'billing.hold_not_found' });
+ const perM = body.retail.nano_usd_per_m || {};
+ const part = (n, r) => { const x = BigInt(n || 0) * BigInt(r || 0); return x === 0n ? 0n : (x % 1000000n === 0n ? x / 1000000n : x / 1000000n + 1n); };
+ const charge = part(body.usage.input_tokens, perM.input) + part(body.usage.output_tokens, perM.output) + BigInt(body.retail.per_request_nano_usd || 0);
+ const out = { charge_nano_usd: String(charge), released_nano_usd: String(BigInt(hold.held) - charge), journal_id: 'txn_1', shortfall_nano_usd: '0' };
+ state.settlements.set(body.idempotency_key, out);
+ hold.status = 'settled';
+ state.held -= BigInt(hold.held);
+ state.balance -= charge;
+ return send(201, out);
+ }
+ m = /^\/internal\/credits\/holds\/([^/]+)\/release$/.exec(path);
+ if (m) {
+ const id = decodeURIComponent(m[1]);
+ const hold = [...state.holds.values()].find((x) => x.hold_id === id);
+ if (hold && hold.status !== 'settled') { hold.status = 'released'; state.held -= BigInt(hold.held); state.releases.push(id); }
+ return send(200, { hold_id: id, released_nano_usd: hold ? hold.held : '0' });
+ }
+ return send(404, { code: 'not_found', detail: path });
+ });
+ });
+ await new Promise((r) => server.listen(0, '127.0.0.1', r));
+ state.url = `http://127.0.0.1:${server.address().port}`;
+ state.close = () => new Promise((r) => { server.closeAllConnections?.(); server.close(() => r()); });
+ return state;
+}
+
+/** The GPU fixture plus a canned OpenAI answer and the fake Billing's own HTTP, so one fetch serves all. */
+function marketplace(billing) {
+ const api = createGpuFetch();
+ const chat = [];
+ const fetchImpl = async (url, opts = {}) => {
+ const u = String(url);
+ if (billing && u.startsWith(billing.url)) return globalThis.fetch(u, opts);
+ if (u.includes('/v1/chat/completions')) {
+ chat.push({ url: u, body: JSON.parse(opts.body || '{}') });
+ return new Response(JSON.stringify({ id: 'chatcmpl-own', object: 'chat.completion', model: 'own-model', choices: [{ index: 0, message: { role: 'assistant', content: 'from your GPU' }, finish_reason: 'stop' }], usage: { prompt_tokens: 3, completion_tokens: 4, total_tokens: 7 } }), { status: 200, headers: { 'Content-Type': 'application/json' } });
+ }
+ return api.fetchImpl(u, opts);
+ };
+ return { api, chat, fetchImpl };
+}
+
+async function setGpus(h, { key = 'vast-test-key', cap = '50000000000', idle = null } = {}) {
+ const store = h.settings.gpus.store;
+ const values = { monthly_cap: Number(cap) };
+ if (idle != null) values.idle_minutes = idle;
+ await store.apply(values, { merge: true, actor: OWNER, reason: 'the test' });
+ if (key) await store.setSecret('vast.key', key, { actor: OWNER, reason: 'the test' });
+}
+
+const billingEnv = (billing) => ({ AI_CREDITS: '1', OV_BILLING_INTERNAL_URL: billing.url, OV_NETWORK_INTERNAL_URL: billing.url, OV_OAUTH_CLIENT_SECRET: 'test-secret' });
+
+t.test('the two owner inputs are marked for Setup: key and cap, group GPUs, needed for own endpoints', async () => {
+ const h = await boot({ clock: { now: () => T0 } });
+ try {
+ const sum = await h.settings.gpus.store.summary();
+ for (const key of ['vast.key', 'runpod.key', 'monthly_cap']) {
+ const field = sum.schema.properties[key];
+ assert.strictEqual(field['x-owner-input'].group, 'GPUs', key + ' is grouped under GPUs');
+ assert.strictEqual(field['x-required-for'], 'own endpoints', key + ' is needed for own endpoints');
+ assert.ok(String(field.title || '').length > 2, key + ' has a plain title');
+ assert.ok(String(field.description || '').length >= 30, key + ' has a description');
+ assert.match(String(field['x-owner-input'].get_it_at), /^https:\/\//, key + ' says where to get it');
+ }
+ assert.strictEqual(sum.schema.properties['vast.key']['x-owner-input'].unlocks.length > 30, true);
+ } finally { await h.stop(); }
+});
+
+t.test('off without a GPU key or a cap, with a plain reason; nothing is rented', async () => {
+ const now = T0;
+ const h = await boot({ clock: { now: () => now } });
+ try {
+ const st = h.ownEndpoints.status();
+ assert.strictEqual(st.enabled, false);
+ assert.match(st.reason, /GPU provider key/);
+ const pub = await request(h.base, 'GET', '/api/v1/exchange/endpoints');
+ assert.strictEqual(pub.status, 200);
+ assert.strictEqual(pub.body.enabled, false);
+ assert.match(pub.body.reason, /GPU provider key/, 'the public status carries the plain reason');
+ const anonymous = await request(h.base, 'GET', '/api/v1/own-endpoints');
+ assert.strictEqual(anonymous.status, 401, 'the person API needs a person');
+ const r = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { model: 'deepseek-r1-32b' } });
+ assert.strictEqual(r.status, 409);
+ assert.match(r.text, /GPU provider key/);
+ // a key but no cap: still off, and it says which one is missing
+ await setGpus(h, { cap: '0' });
+ const noCap = h.ownEndpoints.status();
+ assert.strictEqual(noCap.enabled, false);
+ assert.match(noCap.reason, /monthly GPU cap/);
+ } finally { await h.stop(); }
+});
+
+t.test('off when Balance billing is not on, even with a key and a cap', async () => {
+ const now = T0;
+ const h = await boot({ clock: { now: () => now } });
+ try {
+ await setGpus(h);
+ const st = h.ownEndpoints.status();
+ assert.strictEqual(st.enabled, false);
+ assert.match(st.reason, /Balance billing is not on/);
+ const r = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { model: 'deepseek-r1-32b' } });
+ assert.strictEqual(r.status, 409);
+ assert.match(r.text, /Balance billing is not on/);
+ } finally { await h.stop(); }
+});
+
+t.test('a person starts a private endpoint: the hold is taken first, the rent is never an offer, and the proxy answers', async () => {
+ const billing = await fakeBilling();
+ const m = marketplace(billing);
+ let now = T0;
+ const h = await boot({ clock: { now: () => now }, fetchImpl: m.fetchImpl, env: billingEnv(billing) });
+ try {
+ await setGpus(h, { cap: '50000000000', idle: 1 });
+ const created = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { model: 'deepseek-r1-32b', idle_minutes: 1 } });
+ assert.strictEqual(created.status, 201, created.text);
+ const ep = created.body.endpoint;
+ assert.strictEqual(ep.status, 'requested');
+ assert.match(ep.url, /\/api\/v1\/own-endpoints\/oep_[^/]+\/v1$/);
+ assert.strictEqual(m.api.calls.create.length, 0, 'nothing is rented before the reconciler takes the hold');
+ assert.strictEqual(billing.calls.length, 0);
+
+ // the reconciler starts it: search, hold, then the marketplace
+ await h.ownEndpoints.tick();
+ assert.strictEqual(m.api.calls.search, 1);
+ assert.strictEqual(billing.calls.filter((c) => c.path === '/internal/credits/holds').length, 1, 'the hold was taken');
+ assert.strictEqual(m.api.calls.create.length, 1, 'the instance was rented');
+ let cur = await h.ownEndpoints.get(PERSON, ep.id);
+ assert.strictEqual(cur.status, 'starting');
+ assert.ok(BigInt(cur.reserved_nano) > 0n);
+
+ // it comes up: the provider is NOT registered, so the router and the catalogue never see it
+ now += 60_000;
+ await h.gpus.tick();
+ cur = await h.ownEndpoints.get(PERSON, ep.id);
+ assert.strictEqual(cur.status, 'running');
+ assert.strictEqual(await h.registry.getProvider('own-' + ep.id), null, 'a private endpoint is never an offer');
+
+ // the owner's OpenAI-compatible call reaches the rented GPU and marks the use
+ const proxied = await request(h.base, 'POST', `/api/v1/own-endpoints/${ep.id}/v1/chat/completions`, { tok: userToken(), body: { model: ep.id, messages: [{ role: 'user', content: 'hello' }] } });
+ assert.strictEqual(proxied.status, 200, proxied.text);
+ assert.strictEqual(proxied.body.choices[0].message.content, 'from your GPU');
+ assert.strictEqual(m.chat.length, 1);
+ const fresh = await h.db.prepare('SELECT last_used_at FROM gpu_instances WHERE own_endpoint_id = ?').get(ep.id);
+ assert.ok(fresh.last_used_at, 'the use was marked (the idle timer is honest)');
+
+ // a service token can never manage or call it
+ const service = token('live', ['ai.run.create']);
+ const refused = await request(h.base, 'GET', '/api/v1/own-endpoints', { tok: service });
+ assert.strictEqual(refused.status, 403);
+
+ // idle: it scales to zero after its own minute, and Billing settles the exact spent seconds
+ now += 61_000;
+ await h.gpus.tick();
+ const stopped = await h.ownEndpoints.get(PERSON, ep.id);
+ assert.strictEqual(stopped.status, 'stopped');
+ assert.match(String(stopped.error || ''), /idle|scale/i);
+ assert.strictEqual(m.api.calls.destroy.length, 1, 'the instance was given back');
+ const settled = [...billing.settlements.values()][0];
+ assert.ok(settled, 'the hold was settled');
+ const row = await h.db.prepare('SELECT charged_nano FROM own_endpoints WHERE id = ?').get(ep.id);
+ assert.strictEqual(settled.charge_nano_usd, String(row.charged_nano), 'the person pays exactly what the GPU ledger says');
+ assert.ok(BigInt(settled.charge_nano_usd) > 0n);
+ const spend = await h.gpus.moneySummary();
+ assert.ok(spend.committed_nano <= spend.cap_nano, 'inside the owner cap');
+ } finally { await h.stop(); await billing.close(); }
+});
+
+t.test('the owner monthly cap stops a new start, with a plain reason', async () => {
+ const billing = await fakeBilling();
+ const m = marketplace(billing);
+ const now = T0;
+ const h = await boot({ clock: { now: () => now }, fetchImpl: m.fetchImpl, env: billingEnv(billing) });
+ try {
+ await setGpus(h, { cap: '1000000' }); // $0.001: less than an hour of a $0.1348/h GPU
+ const created = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { model: 'deepseek-r1-32b' } });
+ assert.strictEqual(created.status, 201);
+ await h.ownEndpoints.tick();
+ const cur = await h.ownEndpoints.get(PERSON, created.body.endpoint.id);
+ assert.strictEqual(cur.status, 'refused');
+ assert.match(String(cur.error), /cap/i);
+ assert.strictEqual(m.api.calls.create.length, 0, 'nothing was rented');
+ assert.strictEqual(billing.calls.length, 0, 'no hold was taken, no money moved');
+ } finally { await h.stop(); await billing.close(); }
+});
+
+t.test('own weights and a LoRA run on the same provisioner, never on a catalog key', async () => {
+ const billing = await fakeBilling();
+ const m = marketplace(billing);
+ let now = T0;
+ const h = await boot({ clock: { now: () => now }, fetchImpl: m.fetchImpl, env: billingEnv(billing) });
+ try {
+ await setGpus(h);
+ const made = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: {
+ weights: { repo: 'someone/My-Model-GGUF', file: 'My-Model-Q4_K_M.gguf', lora_repo: 'someone/my-lora', lora_file: 'adapter.gguf' },
+ display_name: 'My own weights',
+ } });
+ assert.strictEqual(made.status, 201, made.text);
+ assert.strictEqual(made.body.endpoint.source, 'weights');
+ await h.ownEndpoints.tick();
+ assert.strictEqual(m.api.calls.create.length, 1);
+ const body = m.api.calls.create[0].body;
+ assert.match(String(body.args), /huggingface\.co\/someone\/My-Model-GGUF\/resolve\/main\/My-Model-Q4_K_M\.gguf/);
+ assert.match(String(body.args), /--lora \/models\/adapter\.gguf/);
+ // an invalid repository never reaches the marketplace
+ const bad = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { weights: { repo: 'file:///etc/passwd', file: 'x.gguf' } } });
+ assert.strictEqual(bad.status, 422);
+ } finally { await h.stop(); await billing.close(); }
+});
+
+t.test('a person sees only their own endpoints', async () => {
+ const billing = await fakeBilling();
+ const m = marketplace(billing);
+ const now = T0;
+ const h = await boot({ clock: { now: () => now }, fetchImpl: m.fetchImpl, env: billingEnv(billing) });
+ try {
+ await setGpus(h);
+ const made = await request(h.base, 'POST', '/api/v1/own-endpoints', { tok: userToken(), body: { model: 'deepseek-r1-32b' } });
+ assert.strictEqual(made.status, 201);
+ const other = 'usr_01JAB2C3D4E5F6G7H8J9K0MNP8';
+ const peek = await request(h.base, 'GET', `/api/v1/own-endpoints/${made.body.endpoint.id}`, { tok: userToken(other) });
+ assert.strictEqual(peek.status, 404);
+ } finally { await h.stop(); await billing.close(); }
+});
+
+if (require.main === module) t.run();
+module.exports = { suite: t };