openvibers/openvibe.ai Commits
Generate images on route default.image, metered per image
Changes
@@@ -82,6 +82,9 @@ AI_API_KEY=# Blank = claude-sonnet-5 for anthropic, gpt-4o-mini otherwise (Live's defaults)AI_MODEL=#AI_EMBEDDING_MODEL=text-embedding-3-small++# Image generation (POST /api/v1/image, route default.image): off unless set. The shared provider's image model.++#AI_IMAGE_MODEL=gpt-image-1++#AI_IMAGE_SIZE=1024x1024#AI_PROVIDER_TIMEOUT_MS=30000# USD per million tokens, longest model-prefix match: {"gpt-5-nano":{"in":0.05,"out":0.4,"cached":0.005}}# Each entry may also carry its platform.rate-card@1 terms: source (URI), verified_at, effective_from (YYYY-MM-DD),@@@ -99,8 +99,8 @@ idempotent replay):| Route | Capability ||---|---|| `POST /api/v1/runs` (`?wait=ms`, `Idempotency-Key`), `POST /runs/:id/cancel`, `POST /runs/:id/retry`, `POST /runs/:id/citations` | `ai.run.create` |--| `POST /api/v1/{chat,generate,summarize,classify,extract,enrich,embed}` — each a run of workflow `ai.<op>` | `ai.run.create` |--| `POST /api/v1/{chat,generate,summarize,classify,extract,embed}` (no `enrich`), `GET /runs/:id` (the app's own runs) | `ai.app.run` for a developer-app token ([Developer apps](#developer-apps)) |++| `POST /api/v1/{chat,generate,summarize,classify,extract,enrich,embed,image}` — each a run of workflow `ai.<op>` | `ai.run.create` |++| `POST /api/v1/{chat,generate,summarize,classify,extract,embed,image}` (no `enrich`), `GET /runs/:id` (the app's own runs) | `ai.app.run` for a developer-app token ([Developer apps](#developer-apps)) || `GET /api/v1/runs`, `GET /runs/:id` (+ citations, request-log metadata), `GET /runs/:id/citations` | `ai.run.read` (own runs) || `GET /api/v1/workflows\|templates\|routes[/:key]` | `ai.run.create`, `ai.workflow.manage` or `ai.usage.read` || `POST /api/v1/{templates,workflows,routes}/:key/versions`, `…/versions/:v/status` | `ai.workflow.manage` |@@@ -151,7 +151,7 @@ once OpenVibe staff add it to the project's allowance. Create the project and it| Route | What ||---|---|--| `POST /api/v1/{chat,generate,summarize,classify,extract,embed}` | the direct operations; `enrich` is first-party only |++| `POST /api/v1/{chat,generate,summarize,classify,extract,embed,image}` | the direct operations; `enrich` is first-party only || `GET /api/v1/runs/:id` | the app's own runs (404 for another project's) |Every other route refuses an app token (403 `capability.denied`), whatever its grant carries. An@@@ -257,7 +257,7 @@ definition, but never over an admin's version.| Kind | What | Configured by ||---|---|---|| `stub` | Deterministic, no key, realistically shaped output (schema-shaped JSON, sentences, unit-vector embeddings, timed transcript segments), clearly marked synthetic (`(synthetic)` text, `synthetic: true` on the run). Never cached. | always present |--| `openai` | Any OpenAI-compatible API (OpenAI, OpenRouter, Groq, Together, Ollama, LM Studio, llama.cpp): chat with structured-output step-down, embeddings, Whisper-style transcription. Ported from Live's `llm.js` / `ai-provider.js`. | `AI_PROVIDER`, `AI_BASE_URL`, `AI_API_KEY`, `AI_MODEL` — Live's setting names |++| `openai` | Any OpenAI-compatible API (OpenAI, OpenRouter, Groq, Together, Ollama, LM Studio, llama.cpp): chat with structured-output step-down, embeddings, Whisper-style transcription, image generation (`POST {base}/images/generations`). Ported from Live's `llm.js` / `ai-provider.js`. | `AI_PROVIDER`, `AI_BASE_URL`, `AI_API_KEY`, `AI_MODEL` — Live's setting names || `responses` | The OpenAI Responses API (`POST {base}/responses`) for GPT-family models that only speak it, at any OpenAI-compatible `base_url` (default `https://api.openai.com/v1`): chat and every text operation, structured output through `text.format` with the same step-down, vision; no embeddings or transcription. | the admin API (`kind: "responses"`, `base_url`, `secret_ref`) || `anthropic` | Messages API with cached system blocks and forced-tool JSON. Ported from Live's `llm.js`. | same, with `AI_PROVIDER=anthropic` || `http` | The local HTTP seam: `POST {operation, …}` → `{text, json, usage}` | `AI_HTTP_SEAM_URL` or the admin API |@@@ -355,7 +355,27 @@ the service boots without Valkey; with it on, production refuses to boot without`overview.source` says which (`local_model`, `paid`, `extractive`), and `gaps` lists what is missing (no localmodel, no speech-to-text, no budget). The output is not cached: callers keep it. Live's grant names`media.analyze`.--- **Direct operations** — `ai.chat|generate|summarize|classify|extract|enrich|embed`.++- **Direct operations** — `ai.chat|generate|summarize|classify|extract|enrich|embed|image`.++++## Image generation++++Off unless `AI_IMAGE_MODEL` names the shared provider's image model (`dall-e-3`, `gpt-image-1`; OpenAI-shaped++providers only). Then boot gives `shared` the `image` capability, registers the model as type `image` and seeds++route `default.image` (a capability pool of registered `image` models, objective `correctness`,++`provider_scope: paid`, timeout 120 s). Without it no route is seeded and `POST /api/v1/image` answers++`503 route.unavailable`, like any operation whose route is not configured.++++`POST /api/v1/image` (`ai.run.create`, or `ai.app.run` for an app) takes `{ prompt, size?, n? }` (`n` 1–4;++`size` defaults to `AI_IMAGE_SIZE`, `1024x1024`) and answers `output: { images: [{ b64, url }], count }`++(`dall-e-*` is asked for `b64_json`; `gpt-image-1` answers base64 already). A missing prompt is++`422 input.invalid`, a provider that returns no image `502 output.invalid`, no provider `503 provider.unavailable`.++Outside production the stub answers a 1×1 PNG as the last resort (`AI_STUB_FALLBACK`).++++Images are metered per image, never per token: the request log's `operation` is `image` and `tokens_out` the++image count, and each attempt is one `platform.usage-sample@1` reading (`resource` `output-images:<model>`,++`unit` `image`, `quantity` the images returned), queued like every other reading. No rate card prices an image++yet (an owner/T5 item, never AI-written), so an image costs 0 in AI's usage and cost quotas; Billing stores the++reading unrated until a per-image card exists and its `unitFits()` accepts `image`.## Local model@@@ -45,7 +45,14 @@"server-rendered operator console for Network staff","Live import","platform.usage-sample@1 readings to Billing with a per-subject provider free allowance; each run's reading state in explain and the free allowance on the staff console",-- "tier budgets (free, paid, staff) per subject and per project on openvibe-sdk/govern in Valkey, behind AI_GOVERN_TIERS (off by default); readings from govern's onUsage"++ "tier budgets (free, paid, staff) per subject and per project on openvibe-sdk/govern in Valkey, behind AI_GOVERN_TIERS (off by default); readings from govern's onUsage",++ "image generation (POST /api/v1/image, workflow ai.image, app op ai.image, route default.image over the OpenAI Images API), off unless AI_IMAGE_MODEL is set; metered per image (one platform.usage-sample@1 per attempt, resource output-images:<model>, unit image)"],-- "updated": "2026-10-12"++ "todo": [++ "owner: a provider key that serves the OpenAI Images API, then AI_IMAGE_MODEL (and AI_IMAGE_SIZE) in /etc/openvibe/ai.env — image generation stays off until then",++ "owner/T5: a per-image rate card (metric output-images:<model>, per provider) — none is written by AI; until it exists an image run costs 0 in AI's usage and quotas (max_requests still bounds it)",++ "owner/T5: Billing unitFits() to accept unit image (OpenVibe.Billing server/ops/rating.js) — until then Billing stores image readings unrated (billing.no_card / billing.unit_mismatch)",++ "Contracts: list POST /api/v1/image under ai.run.create and ai.app.run implementedBy (manifests/capabilities)"++ ],++ "updated": "2026-10-13"}@@@ -129,9 +129,9 @@ function runsRouter({ runs, registry, auth, config, log = console }) {});// Direct operations — each is a run of workflow ai.<op>, waited for (default: the max wait).-- // The six ai.app.run names (chat, generate, summarize, classify, extract, embed) take an app token;++ // The ai.app.run names (chat, generate, summarize, classify, extract, embed, image) take an app token;// enrich is first-party only, like every other run route.-- for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed']) {++ for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed', 'image']) {r.post(`/api/v1/${op}`, DIRECT_OPS.includes(`ai.${op}`) ? createApp : create, async (req, res) => {try {const body = req.body || {};@@@ -14,7 +14,7 @@* workflows are `app.<project_key>.*` beside its event types `app.<project_key>.<name>`.** The direct operations ai.app.run names (ai.chat | ai.generate | ai.summarize | ai.classify |-- * ai.extract | ai.embed) are fixed workflows, not namespace lookups: they are the operations the++ * ai.extract | ai.embed | ai.image) are fixed workflows, not namespace lookups: they are the operations the* capability grants by route. Everything else an app runs must be inside its own namespace.*/const capabilities = require('openvibe-contracts').capabilities;@@@ -24,7 +24,7 @@ const APP_SUB_RE = new RegExp(`^app:app_(${ULID})$`);const PROJECT_RE = new RegExp(`^prj_(${ULID})$`);const ENVS = ['sandbox', 'production'];/** The operations ai.app.run grants by route, as workflow keys (POST /api/v1/<op> is a run of ai.<op>). */--const DIRECT_OPS = Object.freeze(['chat', 'generate', 'summarize', 'classify', 'extract', 'embed'].map((op) => `ai.${op}`));++const DIRECT_OPS = Object.freeze(['chat', 'generate', 'summarize', 'classify', 'extract', 'embed', 'image'].map((op) => `ai.${op}`));/** 'prj_01JAB…' -> 'p01jab…' (null for anything that is not a project id). */function projectKey(projectId) {@@@ -105,6 +105,9 @@ function load(env = process.env) {// route media.local (roadmap WS-O task 5). No key; never a paid provider. embedModel (an embedding GGUF the// server serves at /embeddings) makes `local` a member of the default.embedding pool.localLlm: { url: trimUrl(env.AI_LOCAL_LLM_URL), model: String(env.AI_LOCAL_LLM_MODEL || '').trim(), embedModel: String(env.AI_LOCAL_EMBED_MODEL || '').trim(), timeoutMs: int(env.AI_LOCAL_LLM_TIMEOUT_MS, 180000) },++ // Image generation (POST /api/v1/image, workflow ai.image): off until AI_IMAGE_MODEL names the shared++ // provider's image model (dall-e-3, gpt-image-1); then boot seeds that model and route default.image.++ image: { model: String(env.AI_IMAGE_MODEL || '').trim(), size: String(env.AI_IMAGE_SIZE || '').trim() || '1024x1024' },// media.analyze (server/workflows/media-analysis.js): the file cap (also bounded by free disk minus a reserve),// the download timeout, the scene-change threshold (scdet, 0-100) and the tools.mediaAnalysis: {@@@ -171,6 +171,8 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.noif (ctx.freeOnly && !isFreeProvider(p)) return false;return true;}++ /** Operations served only by registered models of one type (never a provider's default chat model). */++ const MODEL_TYPE = { embed: 'embedding', transcribe: 'stt', image: 'image' };const offerId = (provider, model) => (model ? `${provider}:${model}` : provider);const splitOfferId = (id) => {const i = String(id).indexOf(':');@@@ -232,9 +234,9 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.noif (!appOffered(ctx, p)) continue;if (scope === 'local' && !isLocalProvider(p)) continue;if (scope === 'paid' && isFreeProvider(p)) continue;-- if (operation === 'embed' || operation === 'transcribe') {-- // A chat model advertised by an OpenAI-shaped provider is not an embedding or STT model.-- const type = operation === 'embed' ? 'embedding' : 'stt';++ if (MODEL_TYPE[operation]) {++ // A chat model advertised by an OpenAI-shaped provider is not an embedding, STT or image model.++ const type = MODEL_TYPE[operation];for (const m of await registry.listModels({ provider: p.key })) {if (m.type === type && m.status === 'active') await addOffer(p, m.model_key, p.capabilities);}@@@ -325,7 +327,9 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.noconst { order, explain, placement: result = null, ...rest } = route.capability && operation? await poolCandidates(route, features, operation, ctx): await pinnedCandidates(route, features, ctx);-- if (config.stubFallback && !(route.constraints && route.constraints.provider_scope)++ // A scoped route never ends on synthetic text (media.paid falls back to its extractive overview); an image++ // has no such fallback, so the dev/test stub PNG (AI_STUB_FALLBACK, off in production) still closes it.++ if (config.stubFallback && (!(route.constraints && route.constraints.provider_scope) || operation === 'image')&& !order.some(c => c.provider === 'stub') && await registry.getProvider('stub')) {order.push({ provider: 'stub', model: null });explain.candidates.push({ provider: 'stub', model: null, eligible: true, excluded_reason: null, cost: 0, latency: null });@@@ -428,12 +432,14 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.nothrow new ProviderError('embedding response has invalid vectors', { code: 'provider.output' });const latency = Date.now() - t0;const usage = result.usage || { input: 0, output: 0, cached: 0 };-- const cost = await costOf(c.provider, result.model || model, usage);++ // An image is metered per image (usage.output = images), and no rate card prices one yet (an owner/T5++ // item): its cost is 0 rather than the image count read as output tokens on the token cards.++ const cost = operation === 'image' ? 0 : await costOf(c.provider, result.model || model, usage);statFor(c.provider).ok++;await recordSuccess(c.provider);if (ctx.logRequest) {ctx.logRequest({-- ...base, model_key: result.model || model, status: 'ok', output_hash: sha256(result.json || result.text || result.vectors || result.segments || ''),++ ...base, model_key: result.model || model, status: 'ok', output_hash: sha256(result.json || result.text || result.vectors || result.segments || result.images || ''),tokens_in: usage.input || 0, tokens_out: usage.output || 0, tokens_cached: usage.cached || 0, tokens_estimated: usage.estimated ? 1 : 0,cost_usd: cost, latency_ms: latency,debug_prompt: ctx.debugRaw ? JSON.stringify({ system: req.system, messages: req.messages }).slice(0, 200000) : null,@@@ -7,6 +7,7 @@* chat and every text operation POST {base}/chat/completions* embed POST {base}/embeddings* transcribe POST {base}/audio/transcriptions (Whisper-style)++ * image POST {base}/images/generations (dall-e-*, gpt-image-1)** Structured output steps down json_schema -> json_object -> prose for gateways that lack it* (Live's behaviour); prose JSON is repaired by the engine.@@@ -48,7 +49,7 @@ function buildBody({ model, system, messages, image, json, maxTokens, temperaturfunction createOpenAiProvider(record, { apiKey = '', fetchImpl = globalThis.fetch } = {}) {const baseUrl = String(record.base_url || DEFAULT_BASE_URL).replace(/\/+$/, '');const headers = () => (apiKey ? { Authorization: `Bearer ${apiKey}` } : {});-- const caps = new Set(record.capabilities && record.capabilities.length ? record.capabilities : [...TEXT_OPS, 'json', 'vision', 'embed', 'transcribe']);++ const caps = new Set(record.capabilities && record.capabilities.length ? record.capabilities : [...TEXT_OPS, 'json', 'vision', 'embed', 'transcribe', 'image']);async function chat(req) {let jsonMode = req.json ? 'schema' : null;@@@ -108,6 +109,21 @@ function createOpenAiProvider(record, { apiKey = '', fetchImpl = globalThis.fetc}}++ // OpenAI Images API: metered per image (usage.output = images returned), never per token.++ // dall-e-* answers a URL unless asked for b64_json; gpt-image-1 answers base64 already.++ async function images(req) {++ const model = req.model;++ if (!model) throw new ProviderError('image model is not configured', { code: 'provider.model' });++ const body = { model, prompt: String(req.prompt || ''), n: Math.max(1, Math.min(Number(req.n) || 1, 4)) };++ if (req.size) body.size = String(req.size);++ if (!/^gpt-image-1/i.test(model)) body.response_format = 'b64_json';++ const j = await postJson(`${baseUrl}/images/generations`, headers(), body, { signal: req.signal, timeoutMs: req.timeoutMs, fetchImpl });++ const list = (Array.isArray(j.data) ? j.data : []).filter(d => d && (d.b64_json || d.url))++ .map(d => ({ b64: d.b64_json || null, url: d.url || null, revised_prompt: d.revised_prompt || null }));++ if (!list.length) throw new ProviderError('image response has no data', { code: 'provider.output' });++ return { images: list, model: j.model || model, usage: { input: 0, output: list.length, cached: 0 } };++ }++const adapter = {key: record.key,kind: 'openai',@@@ -115,6 +131,7 @@ function createOpenAiProvider(record, { apiKey = '', fetchImpl = globalThis.fetcchat,embed,transcribe,++ image: images,};for (const op of TEXT_OPS) if (op !== 'chat') adapter[op] = chat; // the engine shapes the prompt per operationreturn adapter;@@@ -101,6 +101,9 @@ function textFor(op, req, rnd) {}}++/** A transparent 1x1 PNG, base64: the stub's image for every prompt. */++const STUB_PNG = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==';++function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = null } = {}) {const fail = record.metadata && record.metadata.fail;@@@ -123,6 +126,12 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nu});return { vectors, model, synthetic: true, usage: { input: inputs.reduce((n, t) => n + approxTokens(t), 0), output: 0, cached: 0, estimated: true } };}++ if (op === 'image') {++ // One fixed 1x1 PNG per requested image: deterministic, and metered per image like a real provider.++ const n = Math.max(1, Math.min(Number(req.n) || 1, 4));++ const images = Array.from({ length: n }, () => ({ b64: STUB_PNG, url: null, revised_prompt: null }));++ return { images, model, synthetic: true, usage: { input: 0, output: n, cached: 0, estimated: true } };++ }if (op === 'transcribe') {const n = 2 + Math.floor(rnd() * 3);const at = Number(req.offsetSec) || 0; // a window of a longer recording keeps the recording's times@@@ -148,9 +157,10 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nusupports: () => true,transcribe: async (req) => await run('transcribe', req),embed: async (req) => await run('embed', req),++ image: async (req) => await run('image', req),};for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich']) adapter[op] = async (req) => await run(op, req);return adapter;}--module.exports = { createStubProvider, synthesize, seeded };++module.exports = { createStubProvider, synthesize, seeded, STUB_PNG };@@@ -197,6 +197,11 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry, freeAllconst a = list[attempt];if (!a || !a.provider_key || a.status === 'skipped') continue; // a skipped attempt never touched a providerconst attemptModel = a.model_key || null;++ if (a.operation === 'image') {++ // Metered per image (tokens_out is the image count); no token card or free allowance applies.++ readings.push({ attempt, kind: usageSamples.IMAGE_KIND, provider: a.provider_key, model: attemptModel, quantity: Math.max(0, Number(a.tokens_out) || 0), cost: Math.max(0, Number(a.cost_usd) || 0), freeAllowanceUsed: 0 });++ continue;++ }const qty = { in: Math.max(0, (Number(a.tokens_in) || 0) - (Number(a.tokens_cached) || 0)), cached: Math.max(0, Number(a.tokens_cached) || 0), out: Math.max(0, Number(a.tokens_out) || 0) };const free = freeAllowance ? await freeAllowance.claim(subjectOf(ctx), a.provider_key, attemptModel, qty, at) : null;const byKind = (free && free.byKind) || {};@@@ -15,7 +15,7 @@ const { provenance } = require('./providers/rate-cards');const PROVIDER_KINDS = ['stub', 'openai', 'responses', 'anthropic', 'http', 'whisper'];const BILLING_PROFILES = ['metered', 'subscription', 'payg', 'free', 'byok'];--const FEATURES = ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed', 'vision', 'json', 'transcribe'];++const FEATURES = ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed', 'vision', 'json', 'transcribe', 'image'];const LIFECYCLE = ['draft', 'active', 'deprecated', 'archived'];const KEY_RE = /^[a-z][a-z0-9_-]*(\.[a-z0-9_-]+)*$/;@@@ -137,7 +137,7 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.eif (!input.model_key || String(input.model_key).length > 200) throw new AiError(422, 'ai.invalid', 'model_key required');const prev = await getModel(input.provider_key, input.model_key);const type = input.type || (prev && prev.type) || 'chat';-- if (!['chat', 'vision', 'embedding', 'stt'].includes(type)) throw new AiError(422, 'ai.invalid', 'model type must be chat, vision, embedding or stt');++ if (!['chat', 'vision', 'embedding', 'stt', 'image'].includes(type)) throw new AiError(422, 'ai.invalid', 'model type must be chat, vision, embedding, stt or image');const status = input.status || (prev && prev.status) || 'active';if (!['active', 'disabled'].includes(status)) throw new AiError(422, 'ai.invalid', 'model status must be active or disabled');// A new cost replaces the old one whole: new prices never inherit the old ones' source or verification date.@@@ -340,7 +340,7 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.eschemas.assertSchema(w.output_schema, 'output_schema');if (!Array.isArray(w.steps) || !w.steps.length) throw new AiError(422, 'ai.invalid', 'workflow needs at least one step');for (const s of w.steps) {-- if (!s || !['llm', 'passthrough', 'transcribe', 'embed', 'media_analysis'].includes(s.kind)) throw new AiError(422, 'ai.invalid', 'step kind must be llm, passthrough, transcribe, embed or media_analysis');++ if (!s || !['llm', 'passthrough', 'transcribe', 'embed', 'image', 'media_analysis'].includes(s.kind)) throw new AiError(422, 'ai.invalid', 'step kind must be llm, passthrough, transcribe, embed, image or media_analysis');if ((s.kind === 'llm' || (s.kind === 'media_analysis' && s.template)) && !(s.template && await latestAny('templates', s.template))) throw new AiError(422, 'ai.invalid', `step references unknown template ${s.template}`);}if (!['none', 'private', 'service'].includes(w.cache_mode)) throw new AiError(422, 'ai.invalid', 'cache_mode must be none, private or service');@@@ -32,7 +32,11 @@ const TABLE = 'usage_sample_outbox'; const SCHEMA = 'platform.usage-sample@1';// Billing's unitFits() accepts the card metric's leading word ("GiB" fits gib-delivered; a trailing plural s// is dropped from the unit, so the full `<kind>-tokens` would not). The metric is `<KINDS[kind]>-tokens[:model]`.const KINDS = ['in', 'cached', 'out'];--const UNIT = { in: 'input', cached: 'cached-input', out: 'output' };++// An image attempt is one reading of kind `images`: quantity = images returned, resource `output-images[:model]`,++// unit `image`. Billing stores it; until a per-image card exists and unitFits() takes `image` (owner/T5) it waits unrated.++const IMAGE_KIND = 'images';++const UNIT = { in: 'input', cached: 'cached-input', out: 'output', [IMAGE_KIND]: 'image' };++const resourceOf = (kind, model) => (kind === IMAGE_KIND ? (model ? `output-images:${model}` : 'output-images') : rates.metricFor(kind, model));const REFUSED = new Set([400, 409, 413, 422]); const PRUNE_EVERY_MS = 6 * 60 * 60 * 1000;let relay = null; let on = false; let pruneTimer = null; let lastError = null; let queued = 0;const keyOf = (runId, attempt, kind) => `ai:${runId}:${attempt}:${kind}`;@@@ -40,7 +44,7 @@ const meter = createMeter();/** One reading: an attempt's tokens of one kind, under its rate-card metric and the metric's leading-word unit. */function sampleOf({ runId, attempt, kind, workflowKey, requester, project = null, provider = null, model = null, quantity = 0, cost = 0, freeAllowanceUsed = 0, at, traceId = null }) {const key = keyOf(runId, attempt, kind);-- const s = { id: key, idempotency_key: key, service: 'ai', subject: requester, resource: rates.metricFor(kind, model || null), operation: workflowKey,++ const s = { id: key, idempotency_key: key, service: 'ai', subject: requester, resource: resourceOf(kind, model || null), operation: workflowKey,quantity: Math.max(0, Number(quantity) || 0), unit: UNIT[kind], at: new Date(at).toISOString(),cost_estimate: Math.max(0, Number(cost) || 0), source: 'ai.runs' };// A developer app's run (ADR-014): the reading names the project Billing bills it to.@@@ -147,4 +151,4 @@ async function stop({ drainMs = 3000 } = {}) {if (r) { let timer; await Promise.race([r.flush().catch(() => {}), new Promise((res) => { timer = setTimeout(res, drainMs); if (timer.unref) timer.unref(); })]); clearTimeout(timer); await r.stop(); }}function _reset() { if (pruneTimer) clearInterval(pruneTimer); pruneTimer = null; if (relay) relay.stop(); relay = null; on = false; queued = 0; lastError = null; }--module.exports = { init, record, sampleOf, sampleFromUsage, keyOf, runIdOf, readingsFor, deliveryReport, status, stop, _reset, TABLE, KINDS, UNIT };++module.exports = { IMAGE_KIND, init, record, sampleOf, sampleFromUsage, keyOf, runIdOf, readingsFor, deliveryReport, status, stop, _reset, TABLE, KINDS, UNIT };@@@ -1,6 +1,6 @@'use strict';/**-- * The direct operations (POST /api/v1/{chat,generate,summarize,classify,extract,enrich,embed}).++ * The direct operations (POST /api/v1/{chat,generate,summarize,classify,extract,enrich,embed,image}).* Each one IS a workflow (ai.<op>) so a direct call gets the same run record, quota, cache, audit* and attribution as any product workflow — the convenience routes are thin, not a second engine.*/@@@ -84,6 +84,13 @@ const workflows = [output_schema: OBJ({ vectors: { type: 'array', items: { type: 'array', items: { type: 'number' } } }, dimensions: { type: 'integer' } }, ['vectors', 'dimensions']),steps: [{ kind: 'embed', route: 'default.embedding' }], cache_mode: 'private',},++ {++ // Off until AI_IMAGE_MODEL seeds route default.image: without it a run answers 503 route.unavailable.++ key: 'ai.image', name: 'Generate an image', namespace: 'ai',++ input_schema: OBJ({ prompt: STR(4000), size: { type: 'string', pattern: '^[0-9]{3,4}x[0-9]{3,4}$' }, n: { type: 'integer', minimum: 1, maximum: 4 } }, ['prompt']),++ output_schema: OBJ({ images: { type: 'array', minItems: 1, items: OBJ({ b64: { type: ['string', 'null'] }, url: { type: ['string', 'null'] } }) }, count: { type: 'integer', minimum: 1 } }, ['images', 'count']),++ steps: [{ kind: 'image', route: 'default.image' }], cache_mode: 'private',++ },];module.exports = { templates, workflows };@@@ -215,7 +215,18 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spareturn { output: { vectors, dimensions: vectors[0] ? vectors[0].length : 0 }, exec, route };}-- const STEP = { llm: llmStep, passthrough: passthroughStep, transcribe: transcribeStep, embed: embedStep, media_analysis: mediaAnalysis.step };++ async function imageStep(step, input, wf, run, ctx) {++ const route = await routeFor(step.route || 'default.image');++ const prompt = String(input.prompt || '').trim();++ if (!prompt) throw new AiError(422, 'input.invalid', 'image needs a prompt');++ const req = { prompt, size: input.size || (step.params && step.params.size) || (config.image && config.image.size) || null, n: Math.max(1, Math.min(Number(input.n) || 1, 4)), timeoutMs: route.timeout_ms || 120000 };++ const exec = await pool.execute(route, 'image', req, { ...ctx, routeKey: route.key, routeVersion: route.version, promptHash: sha256({ p: prompt }) });++ const images = (exec.result.images || []).filter(i => i && (i.b64 || i.url));++ if (!images.length) throw new AiError(502, 'output.invalid', 'no image was returned');++ return { output: { images: images.map(i => ({ b64: i.b64 || null, url: i.url || null })), count: images.length }, exec, route };++ }++++ const STEP = { llm: llmStep, passthrough: passthroughStep, transcribe: transcribeStep, embed: embedStep, image: imageStep, media_analysis: mediaAnalysis.step };/*** Run a workflow. ctx: { signal, logRequest, debugRaw, inputHash }@@@ -40,7 +40,8 @@ function sharedProviderRecord(config, env) {default_model: s.model || (kind === 'anthropic' ? 'claude-sonnet-5' : 'gpt-4o-mini'),capabilities: kind === 'anthropic'? ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'json', 'vision']-- : ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'json', 'vision', 'embed', 'transcribe'],++ // image only with AI_IMAGE_MODEL: the Images API is OpenAI-shaped, and image generation is off without it.++ : ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'json', 'vision', 'embed', 'transcribe', ...(config.image && config.image.model ? ['image'] : [])],timeout_ms: s.timeoutMs, priority: 10, metadata: { configured_by: 'AI_PROVIDER/AI_BASE_URL/AI_API_KEY/AI_MODEL', provider_name: s.provider || 'openai-compatible', billing_profile: 'metered' },};}@@@ -82,6 +83,9 @@ async function seed({ registry, quotas, config, env = process.env, db }) {if (!await registry.getModel('whisper', 'whisper.cpp')) await registry.upsertModel({ provider_key: 'whisper', model_key: 'whisper.cpp', type: 'stt', supports: { json: false } }, { actor: 'seed' });const embeddingModel = env.AI_EMBEDDING_MODEL || 'text-embedding-3-small';if (!await registry.getModel('shared', embeddingModel)) await registry.upsertModel({ provider_key: 'shared', model_key: embeddingModel, type: 'embedding' }, { actor: 'seed' });++ // The image model (AI_IMAGE_MODEL) on the shared provider: the only member of default.image until another is added.++ const imageModel = shared && shared.kind === 'openai' && config.image && config.image.model;++ if (imageModel && !await registry.getModel('shared', imageModel)) await registry.upsertModel({ provider_key: 'shared', model_key: imageModel, type: 'image' }, { actor: 'seed' });// The local embedding model joins default.embedding as its free member (vectors differ per upstream).const localEmbed = config.localLlm && config.localLlm.url && config.localLlm.embedModel;if (localEmbed && !await registry.getModel('local', localEmbed)) await registry.upsertModel({ provider_key: 'local', model_key: localEmbed, type: 'embedding', metadata: { seeded: true } }, { actor: 'seed' });@@@ -114,6 +118,8 @@ async function seed({ registry, quotas, config, env = process.env, db }) {// media.analyze (WS-O task 5): the local model when there is one; the paid route is used only under a media.paid budget.if (config.localLlm && config.localLlm.url) await registry.seedVersioned('route', 'media.local', pool(config.localLlm.timeoutMs, { 'capability': 'summarize', constraints: { objective: 'balanced', provider_scope: 'local' }, options: { temperature: 0.3 }, max_output_tokens: 350 }));await registry.seedVersioned('route', 'media.paid', pool(60000, { 'capability': 'summarize', constraints: { objective: 'correctness', provider_scope: 'paid', ...(authority ? { authority } : {}) }, options: { temperature: 0.3 }, max_output_tokens: 350 }));++ // Images (metered per image) only with AI_IMAGE_MODEL: paid providers' registered image models, the shared one first.++ if (imageModel) await registry.seedVersioned('route', 'default.image', pool(120000, { 'capability': 'image', constraints: { objective: 'correctness', provider_scope: 'paid', authority: `shared:${imageModel}` } }));for (const key of HISTORICAL_ROUTES) await registry.seedVersioned('route', key, { primary: { provider: 'shared', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'json', timeout_ms: null, alias_of: 'default.json' });// Templates, then workflows (workflows reference templates)nnew file mode 100644@@@ -0,0 +1,144 @@++'use strict';++// Image generation (T6 step 1): route default.image over the OpenAI Images API, POST /api/v1/image (workflow++// ai.image). Off unless AI_IMAGE_MODEL is set; metered per image (request log operation 'image', tokens_out =++// images; one platform.usage-sample@1 reading per attempt, resource output-images:<model>, unit image).++const assert = require('assert');++const contracts = require('openvibe-contracts');++const { boot, request, token, suite, seamServer, ALL } = require('./helpers');++const { STUB_PNG } = require('../server/providers/stub');++++const t = suite('provider-image');++const tok = token('live', ALL);++const PNG = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk';++const ENV = { AI_STUB_FALLBACK: 'false', AI_PROVIDER_RETRIES: '0', AI_BREAKER_FAILURES: '50', AI_PROVIDER: 'openai', AI_IMAGE_MODEL: 'gpt-image-1' };++++/** An OpenAI-shaped upstream: /images/generations answers `images(body)`, anything else 404. */++const imagesServer = (images) => seamServer((body, req) => (req.url.endsWith('/images/generations') ? images(body) : { status: 404, body: { error: { message: 'not here' } } }));++const image = async (h, body) => {++ const r = await request(h.base, 'POST', '/api/v1/image?wait=10000', { tok, body });++ assert.strictEqual(r.status, 201, r.text);++ return r.body.run;++};++const requestsOf = async (h, id) => (await request(h.base, 'GET', `/api/v1/runs/${id}`, { tok })).body.requests;++++t.test('(a) the Images API answers: output.images, a request log with operation image and usage.output = 1, one per-image reading', async () => {++ const up = await imagesServer((body) => ({ created: 1, data: [{ b64_json: PNG }] }));++ const h = await boot({ env: { ...ENV, AI_BASE_URL: `${up.url}/v1` } });++ try {++ const shared = await h.registry.getProvider('shared');++ assert.ok(shared.capabilities.includes('image'), 'shared advertises image with AI_IMAGE_MODEL');++ assert.strictEqual((await h.registry.getModel('shared', 'gpt-image-1')).type, 'image');++ const route = await h.registry.resolveRoute('default.image');++ assert.strictEqual(route.capability, 'image');++ assert.strictEqual(route.constraints.objective, 'correctness');++ assert.strictEqual(route.constraints.provider_scope, 'paid');++++ const run = await image(h, { prompt: 'a goose eating croutons' });++ assert.strictEqual(run.status, 'succeeded', JSON.stringify(run.error));++ assert.deepStrictEqual(run.output, { images: [{ b64: PNG, url: null }], count: 1 });++ assert.strictEqual(run.synthetic, false);++ assert.deepStrictEqual(up.last, { model: 'gpt-image-1', prompt: 'a goose eating croutons', n: 1, size: '1024x1024' }, 'gpt-image-1 gets no response_format; size from AI_IMAGE_SIZE');++ assert.strictEqual(run.usage.tokens_out, 1, 'usage.output is the image count');++ assert.strictEqual(run.usage.cost_usd, 0, 'no rate card prices an image: never the image count read as tokens');++++ const log = await requestsOf(h, run.id);++ assert.strictEqual(log.length, 1);++ assert.strictEqual(log[0].operation, 'image');++ assert.strictEqual(log[0].status, 'ok');++ assert.strictEqual(log[0].tokens_out, 1);++ assert.strictEqual(log[0].tokens_in, 0);++ assert.ok(log[0].prompt_hash && !JSON.stringify(log).includes('croutons'), 'the log keeps a hash, never the prompt');++++ // Billing unset: the per-image reading is queued (written, never sent), and the run is unaffected.++ const rows = await h.db.prepare('SELECT event_id, envelope, sent_at FROM usage_sample_outbox ORDER BY id').all();++ assert.strictEqual(rows.length, 1, 'one reading per image attempt, no token readings');++ const s = typeof rows[0].envelope === 'string' ? JSON.parse(rows[0].envelope) : rows[0].envelope;++ assert.strictEqual(rows[0].event_id, `ai:${run.id}:0:images`);++ assert.strictEqual(rows[0].sent_at, null, 'queued');++ assert.ok(contracts.validate('platform.usage-sample@1', s).valid);++ assert.deepStrictEqual({ resource: s.resource, unit: s.unit, quantity: s.quantity, operation: s.operation, provider: s.provider },++ { resource: 'output-images:gpt-image-1', unit: 'image', quantity: 1, operation: 'ai.image', provider: 'shared' });++++ // dall-e-* is asked for b64_json; n is clamped to 1..4.++ await h.registry.upsertModel({ provider_key: 'shared', model_key: 'dall-e-3', type: 'image' });++ const a = (await h.pool.adapter('shared')).adapter;++ const r = await a.image({ model: 'dall-e-3', prompt: 'x', n: 9, timeoutMs: 3000 });++ assert.strictEqual(up.last.response_format, 'b64_json');++ assert.strictEqual(up.last.n, 4);++ assert.strictEqual(r.usage.output, 1);++ // A missing prompt is the caller's: 422 input.invalid.++ const bad = await request(h.base, 'POST', '/api/v1/image?wait=10000', { tok, body: {} });++ assert.strictEqual(bad.status, 422, bad.text);++ assert.strictEqual(bad.body.code, 'input.invalid');++ } finally { await h.stop(); await up.close(); }++});++++t.test('(b) the first image provider answers 500: the pool falls back, with explain.candidates', async () => {++ const dead = await imagesServer(() => ({ status: 500, body: { error: { message: 'upstream down' } } }));++ const ok = await imagesServer((body) => ({ data: [{ b64_json: PNG }, { b64_json: PNG }].slice(0, body.n) }));++ const h = await boot({ env: { ...ENV, AI_BASE_URL: `${dead.url}/v1` } });++ try {++ await h.registry.upsertProvider({ key: 'img2', kind: 'openai', status: 'active', auth_mode: 'none', base_url: `${ok.url}/v1`, default_model: null, capabilities: ['image'], timeout_ms: 3000, metadata: { paid: true } });++ await h.registry.upsertModel({ provider_key: 'img2', model_key: 'gpt-image-1', type: 'image' });++ const run = await image(h, { prompt: 'a lighthouse', n: 2 });++ assert.strictEqual(run.status, 'succeeded', JSON.stringify(run.error));++ assert.strictEqual(run.output.count, 2);++ assert.strictEqual(run.provenance.provider, 'img2');++ const providers = run.explain.candidates.map(c => c.provider).sort();++ assert.deepStrictEqual(providers, ['img2', 'shared'], 'both paid image providers are candidates; never the stub or a chat model');++ assert.strictEqual(run.explain.selected, 'img2');++ const log = await requestsOf(h, run.id);++ assert.deepStrictEqual(log.map(e => [e.provider_key, e.status, e.fallback, e.operation]), [['shared', 'error', 0, 'image'], ['img2', 'ok', 1, 'image']]);++ assert.strictEqual(dead.calls, 1);++ assert.strictEqual(run.usage.tokens_out, 2, 'metered per image returned');++ } finally { await h.stop(); await dead.close(); await ok.close(); }++});++++t.test('(c) no image provider answers: 503 provider.unavailable, never synthetic', async () => {++ const h = await boot({ env: { ...ENV, AI_BASE_URL: 'http://127.0.0.1:1/v1' } });++ try {++ const run = await image(h, { prompt: 'a lighthouse' });++ assert.strictEqual(run.status, 'failed');++ assert.strictEqual(run.error.code, 'provider.unavailable');++ const route = await h.registry.resolveRoute('default.image');++ let err = null;++ try { await h.pool.execute(route, 'image', { prompt: 'x', n: 1, timeoutMs: 2000 }, { routeKey: route.key }); } catch (e) { err = e; }++ assert.ok(err, 'no answer');++ assert.strictEqual(err.status, 503);++ assert.strictEqual(err.code, 'provider.unavailable');++ assert.ok(err.extra.explain.candidates.every(c => c.provider !== 'stub'), 'the stub is not appended without AI_STUB_FALLBACK');++ } finally { await h.stop(); }++});++++t.test('(d) AI_STUB_FALLBACK: the stub answers a deterministic 1x1 PNG, marked synthetic', async () => {++ const h = await boot({ env: { ...ENV, AI_STUB_FALLBACK: 'true', AI_BASE_URL: 'http://127.0.0.1:1/v1' } });++ try {++ const run = await image(h, { prompt: 'a lighthouse', n: 2 });++ assert.strictEqual(run.status, 'succeeded', JSON.stringify(run.error));++ assert.strictEqual(run.synthetic, true);++ assert.deepStrictEqual(run.output.images, [{ b64: STUB_PNG, url: null }, { b64: STUB_PNG, url: null }]);++ const png = Buffer.from(STUB_PNG, 'base64');++ assert.strictEqual(png.subarray(1, 4).toString(), 'PNG');++ assert.strictEqual(png.readUInt32BE(16), 1, 'width 1');++ assert.strictEqual(png.readUInt32BE(20), 1, 'height 1');++ const log = await requestsOf(h, run.id);++ assert.deepStrictEqual(log.map(e => [e.provider_key, e.status]), [['shared', 'error'], ['stub', 'ok']]);++ } finally { await h.stop(); }++});++++t.test('(e) AI_IMAGE_MODEL unset: no default.image route, no image capability, and POST /api/v1/image is not configured', async () => {++ const h = await boot({ env: { AI_STUB_FALLBACK: 'true', AI_PROVIDER: 'openai' } });++ try {++ assert.ok(!await h.registry.getRoute('default.image'), 'no default.image route');++ assert.ok(!(await h.registry.getProvider('shared')).capabilities.includes('image'));++ assert.ok(!(await h.registry.listModels()).some(m => m.type === 'image'), 'no image model is seeded');++ const run = await image(h, { prompt: 'a lighthouse' });++ assert.strictEqual(run.status, 'failed');++ // The answer every operation gives when its route is not configured (engine routeFor).++ assert.deepStrictEqual(run.error, { code: 'route.unavailable', detail: 'no route default.image' });++ assert.strictEqual((await requestsOf(h, run.id)).length, 0, 'no provider was called');++ assert.strictEqual((await h.db.prepare('SELECT count(*) AS n FROM usage_sample_outbox').get()).n * 1, 0, 'nothing metered');++ } finally { await h.stop(); }++});++++t.run();@@@ -46,6 +46,7 @@ t.test('forced failure on every route: a logged fallback, or an explicit 503 (neif (rr && !rr.disabled) routes.set(rr.key, rr);}assert.ok(routes.size >= 8, `routes discovered: ${[...routes.keys()].join(', ')}`);++ assert.ok(!routes.has('default.image'), 'no image route without AI_IMAGE_MODEL: the walk is unchanged');const failed = [];for (const rr of routes.values()) {@@@ -127,4 +127,33 @@ t.test('local provider: never json, embed only with AI_LOCAL_EMBED_MODEL', () =>assert.strictEqual(load({ AI_LOCAL_EMBED_MODEL: ' nomic-embed-text ' }).localLlm.embedModel, 'nomic-embed-text');});++t.test('image: off without AI_IMAGE_MODEL; the Images API body, per-image usage and the stub PNG', async () => {++ const shared = (env) => providerRecords(load(env), env).find(x => x.key === 'shared');++ assert.ok(!shared({}).capabilities.includes('image'), 'no image capability without AI_IMAGE_MODEL');++ assert.ok(shared({ AI_IMAGE_MODEL: 'gpt-image-1' }).capabilities.includes('image'));++ assert.ok(!shared({ AI_PROVIDER: 'anthropic', AI_IMAGE_MODEL: 'gpt-image-1' }).capabilities.includes('image'), 'never on Anthropic');++ assert.deepStrictEqual(load({}).image, { model: '', size: '1024x1024' });++ assert.deepStrictEqual(load({ AI_IMAGE_MODEL: ' dall-e-3 ', AI_IMAGE_SIZE: '1792x1024' }).image, { model: 'dall-e-3', size: '1792x1024' });++++ const seen = [];++ const s = await seamServer((body, r) => { seen.push([r.url, body]); return { data: [{ url: 'https://img.example/1.png', revised_prompt: 'a goose' }], model: body.model }; });++ const a = createOpenAiProvider({ key: 'o', base_url: s.url }, { apiKey: 'k' });++ assert.ok(a.supports('image'));++ const r = await a.image({ model: 'dall-e-3', prompt: 'goose', n: 0, size: '1024x1024', timeoutMs: 3000 });++ assert.deepStrictEqual(seen[0], ['/images/generations', { model: 'dall-e-3', prompt: 'goose', n: 1, size: '1024x1024', response_format: 'b64_json' }]);++ assert.deepStrictEqual(r.images, [{ b64: null, url: 'https://img.example/1.png', revised_prompt: 'a goose' }]);++ assert.deepStrictEqual(r.usage, { input: 0, output: 1, cached: 0 }, 'metered per image, never per token');++ s.handler = () => ({ data: [] });++ await assert.rejects(a.image({ model: 'gpt-image-1', prompt: 'goose', timeoutMs: 3000 }), (e) => e.code === 'provider.output');++ await s.close();++++ const stub = createStubProvider();++ const one = await stub.image({ prompt: 'goose', n: 3 });++ assert.strictEqual(one.images.length, 3);++ assert.deepStrictEqual(one, await stub.image({ prompt: 'another prompt', n: 3 }), 'the same PNG for every prompt');++ assert.strictEqual(Buffer.from(one.images[0].b64, 'base64').subarray(1, 4).toString(), 'PNG');++ assert.strictEqual(one.usage.output, 3);++ assert.strictEqual((await stub.image({ prompt: 'goose', n: 99 })).images.length, 4, 'n is clamped to 4');++});++t.run();@@@ -32,6 +32,7 @@ const EXAMPLES = {'ai.extract': { text: 'Order 42 ships to Berlin.', schema: { type: 'object', additionalProperties: false, required: ['order', 'city'], properties: { order: { type: 'string' }, city: { type: 'string' } } } },'ai.enrich': { record: { name: 'Widget' }, instructions: 'normalise the name' },'ai.embed': { input: ['one', 'two'] },++ 'ai.image': { prompt: 'A goose eating croutons, watercolour.', size: '1024x1024', n: 1 },'wiki.generate_space': { topic: 'SQLite', sources: SRC(2) },'wiki.generate_page': { title: 'WAL mode', sources: SRC(3) },'blog.draft_post': { topic: 'Why we self-host', sources: SRC(1) },@@@ -85,14 +86,15 @@ t.test('boot seeds every required workflow key', async () => {const mediaBase = `http://127.0.0.1:${media.address().port}`;EXAMPLES['live.media.transcribe'].media_url = `${mediaBase}/v/1`;EXAMPLES['media.analyze'].media_url = `${mediaBase}/fixture.mp4`;-- h = await boot({ env: { OV_MEDIA_INTERNAL_URL: mediaBase } });++ // AI_IMAGE_MODEL seeds default.image, so ai.image runs end to end on the stub's PNG like every other workflow.++ h = await boot({ env: { OV_MEDIA_INTERNAL_URL: mediaBase, AI_IMAGE_MODEL: 'gpt-image-1' } });const keys = (await h.registry.listWorkflows()).map(w => w.key);for (const k of ['wiki.generate_space', 'wiki.generate_page', 'blog.draft_post', 'news.summarize_story', 'news.compare_perspectives', 'reviews.summarize_entity', 'deals.enrich_deal','coupons.extract_coupon', 'trade.summarize_market_context', 'codes.generate_docs', 'games.generate_lore', 'moderation.classify', 'tools.describe', 'food.suggest_recipes','live.translate', 'live.stream.describe_frame', 'live.stream.summarize', 'live.viewers.reply', 'live.chat.insight', 'live.hero.slogans', 'live.stream.recap', 'network.site_copy']) {assert.ok(keys.includes(k), `missing ${k}`);}-- for (const k of ['default.chat', 'default.json', 'default.embedding', 'wiki.generate', 'blog.draft', 'news.summarize', 'reviews.summarize', 'deals.enrich', 'coupons.extract', 'trade.summarize', 'codes.generate_docs', 'games.generate_lore', 'tools.describe', 'moderation.classify', 'food.suggest']) {++ for (const k of ['default.chat', 'default.json', 'default.embedding', 'default.image', 'wiki.generate', 'blog.draft', 'news.summarize', 'reviews.summarize', 'deals.enrich', 'coupons.extract', 'trade.summarize', 'codes.generate_docs', 'games.generate_lore', 'tools.describe', 'moderation.classify', 'food.suggest']) {assert.ok(await h.registry.getRoute(k), `missing historical route ${k}`);}});
@@ -82,6 +82,9 @@ AI_API_KEY=
@@ -82,6 +82,9 @@ AI_API_KEY=
# Blank = claude-sonnet-5 for anthropic, gpt-4o-mini otherwise (Live's defaults)
# Blank = claude-sonnet-5 for anthropic, gpt-4o-mini otherwise (Live's defaults)
AI_MODEL=
AI_MODEL=
#AI_EMBEDDING_MODEL=text-embedding-3-small
#AI_EMBEDDING_MODEL=text-embedding-3-small
+# Image generation (POST /api/v1/image, route default.image): off unless set. The shared provider's image model.
+#AI_IMAGE_MODEL=gpt-image-1
+#AI_IMAGE_SIZE=1024x1024
#AI_PROVIDER_TIMEOUT_MS=30000
#AI_PROVIDER_TIMEOUT_MS=30000
# USD per million tokens, longest model-prefix match: {"gpt-5-nano":{"in":0.05,"out":0.4,"cached":0.005}}
# USD per million tokens, longest model-prefix match: {"gpt-5-nano":{"in":0.05,"out":0.4,"cached":0.005}}
# Each entry may also carry its platform.rate-card@1 terms: source (URI), verified_at, effective_from (YYYY-MM-DD),
# Each entry may also carry its platform.rate-card@1 terms: source (URI), verified_at, effective_from (YYYY-MM-DD),
@@ -99,8 +99,8 @@ idempotent replay):
@@ -99,8 +99,8 @@ idempotent replay):
| Route | Capability |
| Route | Capability |
|---|---|
|---|---|
| `POST /api/v1/runs` (`?wait=ms`, `Idempotency-Key`), `POST /runs/:id/cancel`, `POST /runs/:id/retry`, `POST /runs/:id/citations` | `ai.run.create` |
| `POST /api/v1/runs` (`?wait=ms`, `Idempotency-Key`), `POST /runs/:id/cancel`, `POST /runs/:id/retry`, `POST /runs/:id/citations` | `ai.run.create` |
-| `POST /api/v1/{chat,generate,summarize,classify,extract,enrich,embed}` — each a run of workflow `ai.<op>` | `ai.run.create` |
-| `POST /api/v1/{chat,generate,summarize,classify,extract,embed}` (no `enrich`), `GET /runs/:id` (the app's own runs) | `ai.app.run` for a developer-app token ([Developer apps](#developer-apps)) |
+| `POST /api/v1/{chat,generate,summarize,classify,extract,enrich,embed,image}` — each a run of workflow `ai.<op>` | `ai.run.create` |
+| `POST /api/v1/{chat,generate,summarize,classify,extract,embed,image}` (no `enrich`), `GET /runs/:id` (the app's own runs) | `ai.app.run` for a developer-app token ([Developer apps](#developer-apps)) |
| `GET /api/v1/runs`, `GET /runs/:id` (+ citations, request-log metadata), `GET /runs/:id/citations` | `ai.run.read` (own runs) |
| `GET /api/v1/runs`, `GET /runs/:id` (+ citations, request-log metadata), `GET /runs/:id/citations` | `ai.run.read` (own runs) |
| `GET /api/v1/workflows\|templates\|routes[/:key]` | `ai.run.create`, `ai.workflow.manage` or `ai.usage.read` |
| `GET /api/v1/workflows\|templates\|routes[/:key]` | `ai.run.create`, `ai.workflow.manage` or `ai.usage.read` |
| `POST /api/v1/{templates,workflows,routes}/:key/versions`, `…/versions/:v/status` | `ai.workflow.manage` |
| `POST /api/v1/{templates,workflows,routes}/:key/versions`, `…/versions/:v/status` | `ai.workflow.manage` |
@@ -151,7 +151,7 @@ once OpenVibe staff add it to the project's allowance. Create the project and it
@@ -151,7 +151,7 @@ once OpenVibe staff add it to the project's allowance. Create the project and it
| Route | What |
| Route | What |
|---|---|
|---|---|
-| `POST /api/v1/{chat,generate,summarize,classify,extract,embed}` | the direct operations; `enrich` is first-party only |
+| `POST /api/v1/{chat,generate,summarize,classify,extract,embed,image}` | the direct operations; `enrich` is first-party only |
| `GET /api/v1/runs/:id` | the app's own runs (404 for another project's) |
| `GET /api/v1/runs/:id` | the app's own runs (404 for another project's) |
Every other route refuses an app token (403 `capability.denied`), whatever its grant carries. An
Every other route refuses an app token (403 `capability.denied`), whatever its grant carries. An
@@ -257,7 +257,7 @@ definition, but never over an admin's version.
@@ -257,7 +257,7 @@ definition, but never over an admin's version.
| Kind | What | Configured by |
| Kind | What | Configured by |
|---|---|---|
|---|---|---|
| `stub` | Deterministic, no key, realistically shaped output (schema-shaped JSON, sentences, unit-vector embeddings, timed transcript segments), clearly marked synthetic (`(synthetic)` text, `synthetic: true` on the run). Never cached. | always present |
| `stub` | Deterministic, no key, realistically shaped output (schema-shaped JSON, sentences, unit-vector embeddings, timed transcript segments), clearly marked synthetic (`(synthetic)` text, `synthetic: true` on the run). Never cached. | always present |
-| `openai` | Any OpenAI-compatible API (OpenAI, OpenRouter, Groq, Together, Ollama, LM Studio, llama.cpp): chat with structured-output step-down, embeddings, Whisper-style transcription. Ported from Live's `llm.js` / `ai-provider.js`. | `AI_PROVIDER`, `AI_BASE_URL`, `AI_API_KEY`, `AI_MODEL` — Live's setting names |
+| `openai` | Any OpenAI-compatible API (OpenAI, OpenRouter, Groq, Together, Ollama, LM Studio, llama.cpp): chat with structured-output step-down, embeddings, Whisper-style transcription, image generation (`POST {base}/images/generations`). Ported from Live's `llm.js` / `ai-provider.js`. | `AI_PROVIDER`, `AI_BASE_URL`, `AI_API_KEY`, `AI_MODEL` — Live's setting names |
| `responses` | The OpenAI Responses API (`POST {base}/responses`) for GPT-family models that only speak it, at any OpenAI-compatible `base_url` (default `https://api.openai.com/v1`): chat and every text operation, structured output through `text.format` with the same step-down, vision; no embeddings or transcription. | the admin API (`kind: "responses"`, `base_url`, `secret_ref`) |
| `responses` | The OpenAI Responses API (`POST {base}/responses`) for GPT-family models that only speak it, at any OpenAI-compatible `base_url` (default `https://api.openai.com/v1`): chat and every text operation, structured output through `text.format` with the same step-down, vision; no embeddings or transcription. | the admin API (`kind: "responses"`, `base_url`, `secret_ref`) |
| `anthropic` | Messages API with cached system blocks and forced-tool JSON. Ported from Live's `llm.js`. | same, with `AI_PROVIDER=anthropic` |
| `anthropic` | Messages API with cached system blocks and forced-tool JSON. Ported from Live's `llm.js`. | same, with `AI_PROVIDER=anthropic` |
| `http` | The local HTTP seam: `POST {operation, …}` → `{text, json, usage}` | `AI_HTTP_SEAM_URL` or the admin API |
| `http` | The local HTTP seam: `POST {operation, …}` → `{text, json, usage}` | `AI_HTTP_SEAM_URL` or the admin API |
@@ -355,7 +355,27 @@ the service boots without Valkey; with it on, production refuses to boot without
@@ -355,7 +355,27 @@ the service boots without Valkey; with it on, production refuses to boot without
`overview.source` says which (`local_model`, `paid`, `extractive`), and `gaps` lists what is missing (no local
`overview.source` says which (`local_model`, `paid`, `extractive`), and `gaps` lists what is missing (no local
model, no speech-to-text, no budget). The output is not cached: callers keep it. Live's grant names
model, no speech-to-text, no budget). The output is not cached: callers keep it. Live's grant names
`media.analyze`.
`media.analyze`.
-- **Direct operations** — `ai.chat|generate|summarize|classify|extract|enrich|embed`.
+- **Direct operations** — `ai.chat|generate|summarize|classify|extract|enrich|embed|image`.
+
+## Image generation
+
+Off unless `AI_IMAGE_MODEL` names the shared provider's image model (`dall-e-3`, `gpt-image-1`; OpenAI-shaped
+providers only). Then boot gives `shared` the `image` capability, registers the model as type `image` and seeds
+route `default.image` (a capability pool of registered `image` models, objective `correctness`,
+`provider_scope: paid`, timeout 120 s). Without it no route is seeded and `POST /api/v1/image` answers
+`503 route.unavailable`, like any operation whose route is not configured.
+
+`POST /api/v1/image` (`ai.run.create`, or `ai.app.run` for an app) takes `{ prompt, size?, n? }` (`n` 1–4;
+`size` defaults to `AI_IMAGE_SIZE`, `1024x1024`) and answers `output: { images: [{ b64, url }], count }`
+(`dall-e-*` is asked for `b64_json`; `gpt-image-1` answers base64 already). A missing prompt is
+`422 input.invalid`, a provider that returns no image `502 output.invalid`, no provider `503 provider.unavailable`.
+Outside production the stub answers a 1×1 PNG as the last resort (`AI_STUB_FALLBACK`).
+
+Images are metered per image, never per token: the request log's `operation` is `image` and `tokens_out` the
+image count, and each attempt is one `platform.usage-sample@1` reading (`resource` `output-images:<model>`,
+`unit` `image`, `quantity` the images returned), queued like every other reading. No rate card prices an image
+yet (an owner/T5 item, never AI-written), so an image costs 0 in AI's usage and cost quotas; Billing stores the
+reading unrated until a per-image card exists and its `unitFits()` accepts `image`.
## Local model
## Local model
@@ -45,7 +45,14 @@
@@ -45,7 +45,14 @@
"server-rendered operator console for Network staff",
"server-rendered operator console for Network staff",
"Live import",
"Live import",
"platform.usage-sample@1 readings to Billing with a per-subject provider free allowance; each run's reading state in explain and the free allowance on the staff console",
"platform.usage-sample@1 readings to Billing with a per-subject provider free allowance; each run's reading state in explain and the free allowance on the staff console",
- "tier budgets (free, paid, staff) per subject and per project on openvibe-sdk/govern in Valkey, behind AI_GOVERN_TIERS (off by default); readings from govern's onUsage"
+ "tier budgets (free, paid, staff) per subject and per project on openvibe-sdk/govern in Valkey, behind AI_GOVERN_TIERS (off by default); readings from govern's onUsage",
+ "image generation (POST /api/v1/image, workflow ai.image, app op ai.image, route default.image over the OpenAI Images API), off unless AI_IMAGE_MODEL is set; metered per image (one platform.usage-sample@1 per attempt, resource output-images:<model>, unit image)"
],
],
- "updated": "2026-10-12"
+ "todo": [
+ "owner: a provider key that serves the OpenAI Images API, then AI_IMAGE_MODEL (and AI_IMAGE_SIZE) in /etc/openvibe/ai.env — image generation stays off until then",
+ "owner/T5: a per-image rate card (metric output-images:<model>, per provider) — none is written by AI; until it exists an image run costs 0 in AI's usage and quotas (max_requests still bounds it)",
+ "owner/T5: Billing unitFits() to accept unit image (OpenVibe.Billing server/ops/rating.js) — until then Billing stores image readings unrated (billing.no_card / billing.unit_mismatch)",
+ "Contracts: list POST /api/v1/image under ai.run.create and ai.app.run implementedBy (manifests/capabilities)"
+ ],
+ "updated": "2026-10-13"
}
}
@@ -129,9 +129,9 @@ function runsRouter({ runs, registry, auth, config, log = console }) {
@@ -129,9 +129,9 @@ function runsRouter({ runs, registry, auth, config, log = console }) {
});
});
// Direct operations — each is a run of workflow ai.<op>, waited for (default: the max wait).
// Direct operations — each is a run of workflow ai.<op>, waited for (default: the max wait).
- // The six ai.app.run names (chat, generate, summarize, classify, extract, embed) take an app token;
+ // The ai.app.run names (chat, generate, summarize, classify, extract, embed, image) take an app token;
// enrich is first-party only, like every other run route.
// enrich is first-party only, like every other run route.
- for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed']) {
+ for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed', 'image']) {
r.post(`/api/v1/${op}`, DIRECT_OPS.includes(`ai.${op}`) ? createApp : create, async (req, res) => {
r.post(`/api/v1/${op}`, DIRECT_OPS.includes(`ai.${op}`) ? createApp : create, async (req, res) => {
try {
try {
const body = req.body || {};
const body = req.body || {};
@@ -14,7 +14,7 @@
@@ -14,7 +14,7 @@
* workflows are `app.<project_key>.*` beside its event types `app.<project_key>.<name>`.
* workflows are `app.<project_key>.*` beside its event types `app.<project_key>.<name>`.
*
*
* The direct operations ai.app.run names (ai.chat | ai.generate | ai.summarize | ai.classify |
* The direct operations ai.app.run names (ai.chat | ai.generate | ai.summarize | ai.classify |
- * ai.extract | ai.embed) are fixed workflows, not namespace lookups: they are the operations the
+ * ai.extract | ai.embed | ai.image) are fixed workflows, not namespace lookups: they are the operations the
* capability grants by route. Everything else an app runs must be inside its own namespace.
* capability grants by route. Everything else an app runs must be inside its own namespace.
*/
*/
const capabilities = require('openvibe-contracts').capabilities;
const capabilities = require('openvibe-contracts').capabilities;
@@ -24,7 +24,7 @@ const APP_SUB_RE = new RegExp(`^app:app_(${ULID})$`);
@@ -24,7 +24,7 @@ const APP_SUB_RE = new RegExp(`^app:app_(${ULID})$`);
const PROJECT_RE = new RegExp(`^prj_(${ULID})$`);
const PROJECT_RE = new RegExp(`^prj_(${ULID})$`);
const ENVS = ['sandbox', 'production'];
const ENVS = ['sandbox', 'production'];
/** The operations ai.app.run grants by route, as workflow keys (POST /api/v1/<op> is a run of ai.<op>). */
/** The operations ai.app.run grants by route, as workflow keys (POST /api/v1/<op> is a run of ai.<op>). */
-const DIRECT_OPS = Object.freeze(['chat', 'generate', 'summarize', 'classify', 'extract', 'embed'].map((op) => `ai.${op}`));
+const DIRECT_OPS = Object.freeze(['chat', 'generate', 'summarize', 'classify', 'extract', 'embed', 'image'].map((op) => `ai.${op}`));
/** 'prj_01JAB…' -> 'p01jab…' (null for anything that is not a project id). */
/** 'prj_01JAB…' -> 'p01jab…' (null for anything that is not a project id). */
function projectKey(projectId) {
function projectKey(projectId) {
@@ -105,6 +105,9 @@ function load(env = process.env) {
@@ -105,6 +105,9 @@ function load(env = process.env) {
// route media.local (roadmap WS-O task 5). No key; never a paid provider. embedModel (an embedding GGUF the
// route media.local (roadmap WS-O task 5). No key; never a paid provider. embedModel (an embedding GGUF the
// server serves at /embeddings) makes `local` a member of the default.embedding pool.
// server serves at /embeddings) makes `local` a member of the default.embedding pool.
localLlm: { url: trimUrl(env.AI_LOCAL_LLM_URL), model: String(env.AI_LOCAL_LLM_MODEL || '').trim(), embedModel: String(env.AI_LOCAL_EMBED_MODEL || '').trim(), timeoutMs: int(env.AI_LOCAL_LLM_TIMEOUT_MS, 180000) },
localLlm: { url: trimUrl(env.AI_LOCAL_LLM_URL), model: String(env.AI_LOCAL_LLM_MODEL || '').trim(), embedModel: String(env.AI_LOCAL_EMBED_MODEL || '').trim(), timeoutMs: int(env.AI_LOCAL_LLM_TIMEOUT_MS, 180000) },
+ // Image generation (POST /api/v1/image, workflow ai.image): off until AI_IMAGE_MODEL names the shared
+ // provider's image model (dall-e-3, gpt-image-1); then boot seeds that model and route default.image.
+ image: { model: String(env.AI_IMAGE_MODEL || '').trim(), size: String(env.AI_IMAGE_SIZE || '').trim() || '1024x1024' },
// media.analyze (server/workflows/media-analysis.js): the file cap (also bounded by free disk minus a reserve),
// media.analyze (server/workflows/media-analysis.js): the file cap (also bounded by free disk minus a reserve),
// the download timeout, the scene-change threshold (scdet, 0-100) and the tools.
// the download timeout, the scene-change threshold (scdet, 0-100) and the tools.
mediaAnalysis: {
mediaAnalysis: {
@@ -171,6 +171,8 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
@@ -171,6 +171,8 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
if (ctx.freeOnly && !isFreeProvider(p)) return false;
if (ctx.freeOnly && !isFreeProvider(p)) return false;
return true;
return true;
}
}
+ /** Operations served only by registered models of one type (never a provider's default chat model). */
+ const MODEL_TYPE = { embed: 'embedding', transcribe: 'stt', image: 'image' };
const offerId = (provider, model) => (model ? `${provider}:${model}` : provider);
const offerId = (provider, model) => (model ? `${provider}:${model}` : provider);
const splitOfferId = (id) => {
const splitOfferId = (id) => {
const i = String(id).indexOf(':');
const i = String(id).indexOf(':');
@@ -232,9 +234,9 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
@@ -232,9 +234,9 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
if (!appOffered(ctx, p)) continue;
if (!appOffered(ctx, p)) continue;
if (scope === 'local' && !isLocalProvider(p)) continue;
if (scope === 'local' && !isLocalProvider(p)) continue;
if (scope === 'paid' && isFreeProvider(p)) continue;
if (scope === 'paid' && isFreeProvider(p)) continue;
- if (operation === 'embed' || operation === 'transcribe') {
- // A chat model advertised by an OpenAI-shaped provider is not an embedding or STT model.
- const type = operation === 'embed' ? 'embedding' : 'stt';
+ if (MODEL_TYPE[operation]) {
+ // A chat model advertised by an OpenAI-shaped provider is not an embedding, STT or image model.
+ const type = MODEL_TYPE[operation];
for (const m of await registry.listModels({ provider: p.key })) {
for (const m of await registry.listModels({ provider: p.key })) {
if (m.type === type && m.status === 'active') await addOffer(p, m.model_key, p.capabilities);
if (m.type === type && m.status === 'active') await addOffer(p, m.model_key, p.capabilities);
}
}
@@ -325,7 +327,9 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
@@ -325,7 +327,9 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
const { order, explain, placement: result = null, ...rest } = route.capability && operation
const { order, explain, placement: result = null, ...rest } = route.capability && operation
? await poolCandidates(route, features, operation, ctx)
? await poolCandidates(route, features, operation, ctx)
: await pinnedCandidates(route, features, ctx);
: await pinnedCandidates(route, features, ctx);
- if (config.stubFallback && !(route.constraints && route.constraints.provider_scope)
+ // A scoped route never ends on synthetic text (media.paid falls back to its extractive overview); an image
+ // has no such fallback, so the dev/test stub PNG (AI_STUB_FALLBACK, off in production) still closes it.
+ if (config.stubFallback && (!(route.constraints && route.constraints.provider_scope) || operation === 'image')
&& !order.some(c => c.provider === 'stub') && await registry.getProvider('stub')) {
&& !order.some(c => c.provider === 'stub') && await registry.getProvider('stub')) {
order.push({ provider: 'stub', model: null });
order.push({ provider: 'stub', model: null });
explain.candidates.push({ provider: 'stub', model: null, eligible: true, excluded_reason: null, cost: 0, latency: null });
explain.candidates.push({ provider: 'stub', model: null, eligible: true, excluded_reason: null, cost: 0, latency: null });
@@ -428,12 +432,14 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
@@ -428,12 +432,14 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
throw new ProviderError('embedding response has invalid vectors', { code: 'provider.output' });
throw new ProviderError('embedding response has invalid vectors', { code: 'provider.output' });
const latency = Date.now() - t0;
const latency = Date.now() - t0;
const usage = result.usage || { input: 0, output: 0, cached: 0 };
const usage = result.usage || { input: 0, output: 0, cached: 0 };
- const cost = await costOf(c.provider, result.model || model, usage);
+ // An image is metered per image (usage.output = images), and no rate card prices one yet (an owner/T5
+ // item): its cost is 0 rather than the image count read as output tokens on the token cards.
+ const cost = operation === 'image' ? 0 : await costOf(c.provider, result.model || model, usage);
statFor(c.provider).ok++;
statFor(c.provider).ok++;
await recordSuccess(c.provider);
await recordSuccess(c.provider);
if (ctx.logRequest) {
if (ctx.logRequest) {
ctx.logRequest({
ctx.logRequest({
- ...base, model_key: result.model || model, status: 'ok', output_hash: sha256(result.json || result.text || result.vectors || result.segments || ''),
+ ...base, model_key: result.model || model, status: 'ok', output_hash: sha256(result.json || result.text || result.vectors || result.segments || result.images || ''),
tokens_in: usage.input || 0, tokens_out: usage.output || 0, tokens_cached: usage.cached || 0, tokens_estimated: usage.estimated ? 1 : 0,
tokens_in: usage.input || 0, tokens_out: usage.output || 0, tokens_cached: usage.cached || 0, tokens_estimated: usage.estimated ? 1 : 0,
cost_usd: cost, latency_ms: latency,
cost_usd: cost, latency_ms: latency,
debug_prompt: ctx.debugRaw ? JSON.stringify({ system: req.system, messages: req.messages }).slice(0, 200000) : null,
debug_prompt: ctx.debugRaw ? JSON.stringify({ system: req.system, messages: req.messages }).slice(0, 200000) : null,
@@ -7,6 +7,7 @@
@@ -7,6 +7,7 @@
* chat and every text operation POST {base}/chat/completions
* chat and every text operation POST {base}/chat/completions
* embed POST {base}/embeddings
* embed POST {base}/embeddings
* transcribe POST {base}/audio/transcriptions (Whisper-style)
* transcribe POST {base}/audio/transcriptions (Whisper-style)
+ * image POST {base}/images/generations (dall-e-*, gpt-image-1)
*
*
* Structured output steps down json_schema -> json_object -> prose for gateways that lack it
* Structured output steps down json_schema -> json_object -> prose for gateways that lack it
* (Live's behaviour); prose JSON is repaired by the engine.
* (Live's behaviour); prose JSON is repaired by the engine.
@@ -48,7 +49,7 @@ function buildBody({ model, system, messages, image, json, maxTokens, temperatur
@@ -48,7 +49,7 @@ function buildBody({ model, system, messages, image, json, maxTokens, temperatur
function createOpenAiProvider(record, { apiKey = '', fetchImpl = globalThis.fetch } = {}) {
function createOpenAiProvider(record, { apiKey = '', fetchImpl = globalThis.fetch } = {}) {
const baseUrl = String(record.base_url || DEFAULT_BASE_URL).replace(/\/+$/, '');
const baseUrl = String(record.base_url || DEFAULT_BASE_URL).replace(/\/+$/, '');
const headers = () => (apiKey ? { Authorization: `Bearer ${apiKey}` } : {});
const headers = () => (apiKey ? { Authorization: `Bearer ${apiKey}` } : {});
- const caps = new Set(record.capabilities && record.capabilities.length ? record.capabilities : [...TEXT_OPS, 'json', 'vision', 'embed', 'transcribe']);
+ const caps = new Set(record.capabilities && record.capabilities.length ? record.capabilities : [...TEXT_OPS, 'json', 'vision', 'embed', 'transcribe', 'image']);
async function chat(req) {
async function chat(req) {
let jsonMode = req.json ? 'schema' : null;
let jsonMode = req.json ? 'schema' : null;
@@ -108,6 +109,21 @@ function createOpenAiProvider(record, { apiKey = '', fetchImpl = globalThis.fetc
@@ -108,6 +109,21 @@ function createOpenAiProvider(record, { apiKey = '', fetchImpl = globalThis.fetc
}
}
}
}
+ // OpenAI Images API: metered per image (usage.output = images returned), never per token.
+ // dall-e-* answers a URL unless asked for b64_json; gpt-image-1 answers base64 already.
+ async function images(req) {
+ const model = req.model;
+ if (!model) throw new ProviderError('image model is not configured', { code: 'provider.model' });
+ const body = { model, prompt: String(req.prompt || ''), n: Math.max(1, Math.min(Number(req.n) || 1, 4)) };
+ if (req.size) body.size = String(req.size);
+ if (!/^gpt-image-1/i.test(model)) body.response_format = 'b64_json';
+ const j = await postJson(`${baseUrl}/images/generations`, headers(), body, { signal: req.signal, timeoutMs: req.timeoutMs, fetchImpl });
+ const list = (Array.isArray(j.data) ? j.data : []).filter(d => d && (d.b64_json || d.url))
+ .map(d => ({ b64: d.b64_json || null, url: d.url || null, revised_prompt: d.revised_prompt || null }));
+ if (!list.length) throw new ProviderError('image response has no data', { code: 'provider.output' });
+ return { images: list, model: j.model || model, usage: { input: 0, output: list.length, cached: 0 } };
+ }
+
const adapter = {
const adapter = {
key: record.key,
key: record.key,
kind: 'openai',
kind: 'openai',
@@ -115,6 +131,7 @@ function createOpenAiProvider(record, { apiKey = '', fetchImpl = globalThis.fetc
@@ -115,6 +131,7 @@ function createOpenAiProvider(record, { apiKey = '', fetchImpl = globalThis.fetc
chat,
chat,
embed,
embed,
transcribe,
transcribe,
+ image: images,
};
};
for (const op of TEXT_OPS) if (op !== 'chat') adapter[op] = chat; // the engine shapes the prompt per operation
for (const op of TEXT_OPS) if (op !== 'chat') adapter[op] = chat; // the engine shapes the prompt per operation
return adapter;
return adapter;
@@ -101,6 +101,9 @@ function textFor(op, req, rnd) {
@@ -101,6 +101,9 @@ function textFor(op, req, rnd) {
}
}
}
}
+/** A transparent 1x1 PNG, base64: the stub's image for every prompt. */
+const STUB_PNG = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==';
+
function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = null } = {}) {
function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = null } = {}) {
const fail = record.metadata && record.metadata.fail;
const fail = record.metadata && record.metadata.fail;
@@ -123,6 +126,12 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nu
@@ -123,6 +126,12 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nu
});
});
return { vectors, model, synthetic: true, usage: { input: inputs.reduce((n, t) => n + approxTokens(t), 0), output: 0, cached: 0, estimated: true } };
return { vectors, model, synthetic: true, usage: { input: inputs.reduce((n, t) => n + approxTokens(t), 0), output: 0, cached: 0, estimated: true } };
}
}
+ if (op === 'image') {
+ // One fixed 1x1 PNG per requested image: deterministic, and metered per image like a real provider.
+ const n = Math.max(1, Math.min(Number(req.n) || 1, 4));
+ const images = Array.from({ length: n }, () => ({ b64: STUB_PNG, url: null, revised_prompt: null }));
+ return { images, model, synthetic: true, usage: { input: 0, output: n, cached: 0, estimated: true } };
+ }
if (op === 'transcribe') {
if (op === 'transcribe') {
const n = 2 + Math.floor(rnd() * 3);
const n = 2 + Math.floor(rnd() * 3);
const at = Number(req.offsetSec) || 0; // a window of a longer recording keeps the recording's times
const at = Number(req.offsetSec) || 0; // a window of a longer recording keeps the recording's times
@@ -148,9 +157,10 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nu
@@ -148,9 +157,10 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nu
supports: () => true,
supports: () => true,
transcribe: async (req) => await run('transcribe', req),
transcribe: async (req) => await run('transcribe', req),
embed: async (req) => await run('embed', req),
embed: async (req) => await run('embed', req),
+ image: async (req) => await run('image', req),
};
};
for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich']) adapter[op] = async (req) => await run(op, req);
for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich']) adapter[op] = async (req) => await run(op, req);
return adapter;
return adapter;
}
}
-module.exports = { createStubProvider, synthesize, seeded };
+module.exports = { createStubProvider, synthesize, seeded, STUB_PNG };
@@ -197,6 +197,11 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry, freeAll
@@ -197,6 +197,11 @@ function createQuotas(db, { clock = { now: () => Date.now() }, registry, freeAll
const a = list[attempt];
const a = list[attempt];
if (!a || !a.provider_key || a.status === 'skipped') continue; // a skipped attempt never touched a provider
if (!a || !a.provider_key || a.status === 'skipped') continue; // a skipped attempt never touched a provider
const attemptModel = a.model_key || null;
const attemptModel = a.model_key || null;
+ if (a.operation === 'image') {
+ // Metered per image (tokens_out is the image count); no token card or free allowance applies.
+ readings.push({ attempt, kind: usageSamples.IMAGE_KIND, provider: a.provider_key, model: attemptModel, quantity: Math.max(0, Number(a.tokens_out) || 0), cost: Math.max(0, Number(a.cost_usd) || 0), freeAllowanceUsed: 0 });
+ continue;
+ }
const qty = { in: Math.max(0, (Number(a.tokens_in) || 0) - (Number(a.tokens_cached) || 0)), cached: Math.max(0, Number(a.tokens_cached) || 0), out: Math.max(0, Number(a.tokens_out) || 0) };
const qty = { in: Math.max(0, (Number(a.tokens_in) || 0) - (Number(a.tokens_cached) || 0)), cached: Math.max(0, Number(a.tokens_cached) || 0), out: Math.max(0, Number(a.tokens_out) || 0) };
const free = freeAllowance ? await freeAllowance.claim(subjectOf(ctx), a.provider_key, attemptModel, qty, at) : null;
const free = freeAllowance ? await freeAllowance.claim(subjectOf(ctx), a.provider_key, attemptModel, qty, at) : null;
const byKind = (free && free.byKind) || {};
const byKind = (free && free.byKind) || {};
@@ -15,7 +15,7 @@ const { provenance } = require('./providers/rate-cards');
@@ -15,7 +15,7 @@ const { provenance } = require('./providers/rate-cards');
const PROVIDER_KINDS = ['stub', 'openai', 'responses', 'anthropic', 'http', 'whisper'];
const PROVIDER_KINDS = ['stub', 'openai', 'responses', 'anthropic', 'http', 'whisper'];
const BILLING_PROFILES = ['metered', 'subscription', 'payg', 'free', 'byok'];
const BILLING_PROFILES = ['metered', 'subscription', 'payg', 'free', 'byok'];
-const FEATURES = ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed', 'vision', 'json', 'transcribe'];
+const FEATURES = ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed', 'vision', 'json', 'transcribe', 'image'];
const LIFECYCLE = ['draft', 'active', 'deprecated', 'archived'];
const LIFECYCLE = ['draft', 'active', 'deprecated', 'archived'];
const KEY_RE = /^[a-z][a-z0-9_-]*(\.[a-z0-9_-]+)*$/;
const KEY_RE = /^[a-z][a-z0-9_-]*(\.[a-z0-9_-]+)*$/;
@@ -137,7 +137,7 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -137,7 +137,7 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
if (!input.model_key || String(input.model_key).length > 200) throw new AiError(422, 'ai.invalid', 'model_key required');
if (!input.model_key || String(input.model_key).length > 200) throw new AiError(422, 'ai.invalid', 'model_key required');
const prev = await getModel(input.provider_key, input.model_key);
const prev = await getModel(input.provider_key, input.model_key);
const type = input.type || (prev && prev.type) || 'chat';
const type = input.type || (prev && prev.type) || 'chat';
- if (!['chat', 'vision', 'embedding', 'stt'].includes(type)) throw new AiError(422, 'ai.invalid', 'model type must be chat, vision, embedding or stt');
+ if (!['chat', 'vision', 'embedding', 'stt', 'image'].includes(type)) throw new AiError(422, 'ai.invalid', 'model type must be chat, vision, embedding, stt or image');
const status = input.status || (prev && prev.status) || 'active';
const status = input.status || (prev && prev.status) || 'active';
if (!['active', 'disabled'].includes(status)) throw new AiError(422, 'ai.invalid', 'model status must be active or disabled');
if (!['active', 'disabled'].includes(status)) throw new AiError(422, 'ai.invalid', 'model status must be active or disabled');
// A new cost replaces the old one whole: new prices never inherit the old ones' source or verification date.
// A new cost replaces the old one whole: new prices never inherit the old ones' source or verification date.
@@ -340,7 +340,7 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
@@ -340,7 +340,7 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
schemas.assertSchema(w.output_schema, 'output_schema');
schemas.assertSchema(w.output_schema, 'output_schema');
if (!Array.isArray(w.steps) || !w.steps.length) throw new AiError(422, 'ai.invalid', 'workflow needs at least one step');
if (!Array.isArray(w.steps) || !w.steps.length) throw new AiError(422, 'ai.invalid', 'workflow needs at least one step');
for (const s of w.steps) {
for (const s of w.steps) {
- if (!s || !['llm', 'passthrough', 'transcribe', 'embed', 'media_analysis'].includes(s.kind)) throw new AiError(422, 'ai.invalid', 'step kind must be llm, passthrough, transcribe, embed or media_analysis');
+ if (!s || !['llm', 'passthrough', 'transcribe', 'embed', 'image', 'media_analysis'].includes(s.kind)) throw new AiError(422, 'ai.invalid', 'step kind must be llm, passthrough, transcribe, embed, image or media_analysis');
if ((s.kind === 'llm' || (s.kind === 'media_analysis' && s.template)) && !(s.template && await latestAny('templates', s.template))) throw new AiError(422, 'ai.invalid', `step references unknown template ${s.template}`);
if ((s.kind === 'llm' || (s.kind === 'media_analysis' && s.template)) && !(s.template && await latestAny('templates', s.template))) throw new AiError(422, 'ai.invalid', `step references unknown template ${s.template}`);
}
}
if (!['none', 'private', 'service'].includes(w.cache_mode)) throw new AiError(422, 'ai.invalid', 'cache_mode must be none, private or service');
if (!['none', 'private', 'service'].includes(w.cache_mode)) throw new AiError(422, 'ai.invalid', 'cache_mode must be none, private or service');
@@ -32,7 +32,11 @@ const TABLE = 'usage_sample_outbox'; const SCHEMA = 'platform.usage-sample@1';
@@ -32,7 +32,11 @@ const TABLE = 'usage_sample_outbox'; const SCHEMA = 'platform.usage-sample@1';
// Billing's unitFits() accepts the card metric's leading word ("GiB" fits gib-delivered; a trailing plural s
// Billing's unitFits() accepts the card metric's leading word ("GiB" fits gib-delivered; a trailing plural s
// is dropped from the unit, so the full `<kind>-tokens` would not). The metric is `<KINDS[kind]>-tokens[:model]`.
// is dropped from the unit, so the full `<kind>-tokens` would not). The metric is `<KINDS[kind]>-tokens[:model]`.
const KINDS = ['in', 'cached', 'out'];
const KINDS = ['in', 'cached', 'out'];
-const UNIT = { in: 'input', cached: 'cached-input', out: 'output' };
+// An image attempt is one reading of kind `images`: quantity = images returned, resource `output-images[:model]`,
+// unit `image`. Billing stores it; until a per-image card exists and unitFits() takes `image` (owner/T5) it waits unrated.
+const IMAGE_KIND = 'images';
+const UNIT = { in: 'input', cached: 'cached-input', out: 'output', [IMAGE_KIND]: 'image' };
+const resourceOf = (kind, model) => (kind === IMAGE_KIND ? (model ? `output-images:${model}` : 'output-images') : rates.metricFor(kind, model));
const REFUSED = new Set([400, 409, 413, 422]); const PRUNE_EVERY_MS = 6 * 60 * 60 * 1000;
const REFUSED = new Set([400, 409, 413, 422]); const PRUNE_EVERY_MS = 6 * 60 * 60 * 1000;
let relay = null; let on = false; let pruneTimer = null; let lastError = null; let queued = 0;
let relay = null; let on = false; let pruneTimer = null; let lastError = null; let queued = 0;
const keyOf = (runId, attempt, kind) => `ai:${runId}:${attempt}:${kind}`;
const keyOf = (runId, attempt, kind) => `ai:${runId}:${attempt}:${kind}`;
@@ -40,7 +44,7 @@ const meter = createMeter();
@@ -40,7 +44,7 @@ const meter = createMeter();
/** One reading: an attempt's tokens of one kind, under its rate-card metric and the metric's leading-word unit. */
/** One reading: an attempt's tokens of one kind, under its rate-card metric and the metric's leading-word unit. */
function sampleOf({ runId, attempt, kind, workflowKey, requester, project = null, provider = null, model = null, quantity = 0, cost = 0, freeAllowanceUsed = 0, at, traceId = null }) {
function sampleOf({ runId, attempt, kind, workflowKey, requester, project = null, provider = null, model = null, quantity = 0, cost = 0, freeAllowanceUsed = 0, at, traceId = null }) {
const key = keyOf(runId, attempt, kind);
const key = keyOf(runId, attempt, kind);
- const s = { id: key, idempotency_key: key, service: 'ai', subject: requester, resource: rates.metricFor(kind, model || null), operation: workflowKey,
+ const s = { id: key, idempotency_key: key, service: 'ai', subject: requester, resource: resourceOf(kind, model || null), operation: workflowKey,
quantity: Math.max(0, Number(quantity) || 0), unit: UNIT[kind], at: new Date(at).toISOString(),
quantity: Math.max(0, Number(quantity) || 0), unit: UNIT[kind], at: new Date(at).toISOString(),
cost_estimate: Math.max(0, Number(cost) || 0), source: 'ai.runs' };
cost_estimate: Math.max(0, Number(cost) || 0), source: 'ai.runs' };
// A developer app's run (ADR-014): the reading names the project Billing bills it to.
// A developer app's run (ADR-014): the reading names the project Billing bills it to.
@@ -147,4 +151,4 @@ async function stop({ drainMs = 3000 } = {}) {
@@ -147,4 +151,4 @@ async function stop({ drainMs = 3000 } = {}) {
if (r) { let timer; await Promise.race([r.flush().catch(() => {}), new Promise((res) => { timer = setTimeout(res, drainMs); if (timer.unref) timer.unref(); })]); clearTimeout(timer); await r.stop(); }
if (r) { let timer; await Promise.race([r.flush().catch(() => {}), new Promise((res) => { timer = setTimeout(res, drainMs); if (timer.unref) timer.unref(); })]); clearTimeout(timer); await r.stop(); }
}
}
function _reset() { if (pruneTimer) clearInterval(pruneTimer); pruneTimer = null; if (relay) relay.stop(); relay = null; on = false; queued = 0; lastError = null; }
function _reset() { if (pruneTimer) clearInterval(pruneTimer); pruneTimer = null; if (relay) relay.stop(); relay = null; on = false; queued = 0; lastError = null; }
-module.exports = { init, record, sampleOf, sampleFromUsage, keyOf, runIdOf, readingsFor, deliveryReport, status, stop, _reset, TABLE, KINDS, UNIT };
+module.exports = { IMAGE_KIND, init, record, sampleOf, sampleFromUsage, keyOf, runIdOf, readingsFor, deliveryReport, status, stop, _reset, TABLE, KINDS, UNIT };
@@ -1,6 +1,6 @@
@@ -1,6 +1,6 @@
'use strict';
'use strict';
/**
/**
- * The direct operations (POST /api/v1/{chat,generate,summarize,classify,extract,enrich,embed}).
+ * The direct operations (POST /api/v1/{chat,generate,summarize,classify,extract,enrich,embed,image}).
* Each one IS a workflow (ai.<op>) so a direct call gets the same run record, quota, cache, audit
* Each one IS a workflow (ai.<op>) so a direct call gets the same run record, quota, cache, audit
* and attribution as any product workflow — the convenience routes are thin, not a second engine.
* and attribution as any product workflow — the convenience routes are thin, not a second engine.
*/
*/
@@ -84,6 +84,13 @@ const workflows = [
@@ -84,6 +84,13 @@ const workflows = [
output_schema: OBJ({ vectors: { type: 'array', items: { type: 'array', items: { type: 'number' } } }, dimensions: { type: 'integer' } }, ['vectors', 'dimensions']),
output_schema: OBJ({ vectors: { type: 'array', items: { type: 'array', items: { type: 'number' } } }, dimensions: { type: 'integer' } }, ['vectors', 'dimensions']),
steps: [{ kind: 'embed', route: 'default.embedding' }], cache_mode: 'private',
steps: [{ kind: 'embed', route: 'default.embedding' }], cache_mode: 'private',
},
},
+ {
+ // Off until AI_IMAGE_MODEL seeds route default.image: without it a run answers 503 route.unavailable.
+ key: 'ai.image', name: 'Generate an image', namespace: 'ai',
+ input_schema: OBJ({ prompt: STR(4000), size: { type: 'string', pattern: '^[0-9]{3,4}x[0-9]{3,4}$' }, n: { type: 'integer', minimum: 1, maximum: 4 } }, ['prompt']),
+ output_schema: OBJ({ images: { type: 'array', minItems: 1, items: OBJ({ b64: { type: ['string', 'null'] }, url: { type: ['string', 'null'] } }) }, count: { type: 'integer', minimum: 1 } }, ['images', 'count']),
+ steps: [{ kind: 'image', route: 'default.image' }], cache_mode: 'private',
+ },
];
];
module.exports = { templates, workflows };
module.exports = { templates, workflows };
@@ -215,7 +215,18 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
@@ -215,7 +215,18 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
return { output: { vectors, dimensions: vectors[0] ? vectors[0].length : 0 }, exec, route };
return { output: { vectors, dimensions: vectors[0] ? vectors[0].length : 0 }, exec, route };
}
}
- const STEP = { llm: llmStep, passthrough: passthroughStep, transcribe: transcribeStep, embed: embedStep, media_analysis: mediaAnalysis.step };
+ async function imageStep(step, input, wf, run, ctx) {
+ const route = await routeFor(step.route || 'default.image');
+ const prompt = String(input.prompt || '').trim();
+ if (!prompt) throw new AiError(422, 'input.invalid', 'image needs a prompt');
+ const req = { prompt, size: input.size || (step.params && step.params.size) || (config.image && config.image.size) || null, n: Math.max(1, Math.min(Number(input.n) || 1, 4)), timeoutMs: route.timeout_ms || 120000 };
+ const exec = await pool.execute(route, 'image', req, { ...ctx, routeKey: route.key, routeVersion: route.version, promptHash: sha256({ p: prompt }) });
+ const images = (exec.result.images || []).filter(i => i && (i.b64 || i.url));
+ if (!images.length) throw new AiError(502, 'output.invalid', 'no image was returned');
+ return { output: { images: images.map(i => ({ b64: i.b64 || null, url: i.url || null })), count: images.length }, exec, route };
+ }
+
+ const STEP = { llm: llmStep, passthrough: passthroughStep, transcribe: transcribeStep, embed: embedStep, image: imageStep, media_analysis: mediaAnalysis.step };
/**
/**
* Run a workflow. ctx: { signal, logRequest, debugRaw, inputHash }
* Run a workflow. ctx: { signal, logRequest, debugRaw, inputHash }
@@ -40,7 +40,8 @@ function sharedProviderRecord(config, env) {
@@ -40,7 +40,8 @@ function sharedProviderRecord(config, env) {
default_model: s.model || (kind === 'anthropic' ? 'claude-sonnet-5' : 'gpt-4o-mini'),
default_model: s.model || (kind === 'anthropic' ? 'claude-sonnet-5' : 'gpt-4o-mini'),
capabilities: kind === 'anthropic'
capabilities: kind === 'anthropic'
? ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'json', 'vision']
? ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'json', 'vision']
- : ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'json', 'vision', 'embed', 'transcribe'],
+ // image only with AI_IMAGE_MODEL: the Images API is OpenAI-shaped, and image generation is off without it.
+ : ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'json', 'vision', 'embed', 'transcribe', ...(config.image && config.image.model ? ['image'] : [])],
timeout_ms: s.timeoutMs, priority: 10, metadata: { configured_by: 'AI_PROVIDER/AI_BASE_URL/AI_API_KEY/AI_MODEL', provider_name: s.provider || 'openai-compatible', billing_profile: 'metered' },
timeout_ms: s.timeoutMs, priority: 10, metadata: { configured_by: 'AI_PROVIDER/AI_BASE_URL/AI_API_KEY/AI_MODEL', provider_name: s.provider || 'openai-compatible', billing_profile: 'metered' },
};
};
}
}
@@ -82,6 +83,9 @@ async function seed({ registry, quotas, config, env = process.env, db }) {
@@ -82,6 +83,9 @@ async function seed({ registry, quotas, config, env = process.env, db }) {
if (!await registry.getModel('whisper', 'whisper.cpp')) await registry.upsertModel({ provider_key: 'whisper', model_key: 'whisper.cpp', type: 'stt', supports: { json: false } }, { actor: 'seed' });
if (!await registry.getModel('whisper', 'whisper.cpp')) await registry.upsertModel({ provider_key: 'whisper', model_key: 'whisper.cpp', type: 'stt', supports: { json: false } }, { actor: 'seed' });
const embeddingModel = env.AI_EMBEDDING_MODEL || 'text-embedding-3-small';
const embeddingModel = env.AI_EMBEDDING_MODEL || 'text-embedding-3-small';
if (!await registry.getModel('shared', embeddingModel)) await registry.upsertModel({ provider_key: 'shared', model_key: embeddingModel, type: 'embedding' }, { actor: 'seed' });
if (!await registry.getModel('shared', embeddingModel)) await registry.upsertModel({ provider_key: 'shared', model_key: embeddingModel, type: 'embedding' }, { actor: 'seed' });
+ // The image model (AI_IMAGE_MODEL) on the shared provider: the only member of default.image until another is added.
+ const imageModel = shared && shared.kind === 'openai' && config.image && config.image.model;
+ if (imageModel && !await registry.getModel('shared', imageModel)) await registry.upsertModel({ provider_key: 'shared', model_key: imageModel, type: 'image' }, { actor: 'seed' });
// The local embedding model joins default.embedding as its free member (vectors differ per upstream).
// The local embedding model joins default.embedding as its free member (vectors differ per upstream).
const localEmbed = config.localLlm && config.localLlm.url && config.localLlm.embedModel;
const localEmbed = config.localLlm && config.localLlm.url && config.localLlm.embedModel;
if (localEmbed && !await registry.getModel('local', localEmbed)) await registry.upsertModel({ provider_key: 'local', model_key: localEmbed, type: 'embedding', metadata: { seeded: true } }, { actor: 'seed' });
if (localEmbed && !await registry.getModel('local', localEmbed)) await registry.upsertModel({ provider_key: 'local', model_key: localEmbed, type: 'embedding', metadata: { seeded: true } }, { actor: 'seed' });
@@ -114,6 +118,8 @@ async function seed({ registry, quotas, config, env = process.env, db }) {
@@ -114,6 +118,8 @@ async function seed({ registry, quotas, config, env = process.env, db }) {
// media.analyze (WS-O task 5): the local model when there is one; the paid route is used only under a media.paid budget.
// media.analyze (WS-O task 5): the local model when there is one; the paid route is used only under a media.paid budget.
if (config.localLlm && config.localLlm.url) await registry.seedVersioned('route', 'media.local', pool(config.localLlm.timeoutMs, { 'capability': 'summarize', constraints: { objective: 'balanced', provider_scope: 'local' }, options: { temperature: 0.3 }, max_output_tokens: 350 }));
if (config.localLlm && config.localLlm.url) await registry.seedVersioned('route', 'media.local', pool(config.localLlm.timeoutMs, { 'capability': 'summarize', constraints: { objective: 'balanced', provider_scope: 'local' }, options: { temperature: 0.3 }, max_output_tokens: 350 }));
await registry.seedVersioned('route', 'media.paid', pool(60000, { 'capability': 'summarize', constraints: { objective: 'correctness', provider_scope: 'paid', ...(authority ? { authority } : {}) }, options: { temperature: 0.3 }, max_output_tokens: 350 }));
await registry.seedVersioned('route', 'media.paid', pool(60000, { 'capability': 'summarize', constraints: { objective: 'correctness', provider_scope: 'paid', ...(authority ? { authority } : {}) }, options: { temperature: 0.3 }, max_output_tokens: 350 }));
+ // Images (metered per image) only with AI_IMAGE_MODEL: paid providers' registered image models, the shared one first.
+ if (imageModel) await registry.seedVersioned('route', 'default.image', pool(120000, { 'capability': 'image', constraints: { objective: 'correctness', provider_scope: 'paid', authority: `shared:${imageModel}` } }));
for (const key of HISTORICAL_ROUTES) await registry.seedVersioned('route', key, { primary: { provider: 'shared', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'json', timeout_ms: null, alias_of: 'default.json' });
for (const key of HISTORICAL_ROUTES) await registry.seedVersioned('route', key, { primary: { provider: 'shared', model: null }, fallbacks: [], options: {}, max_output_tokens: null, response_format: 'json', timeout_ms: null, alias_of: 'default.json' });
// Templates, then workflows (workflows reference templates)
// Templates, then workflows (workflows reference templates)
new file mode 100644
new file mode 100644
@@ -0,0 +1,144 @@
@@ -0,0 +1,144 @@
+'use strict';
+// Image generation (T6 step 1): route default.image over the OpenAI Images API, POST /api/v1/image (workflow
+// ai.image). Off unless AI_IMAGE_MODEL is set; metered per image (request log operation 'image', tokens_out =
+// images; one platform.usage-sample@1 reading per attempt, resource output-images:<model>, unit image).
+const assert = require('assert');
+const contracts = require('openvibe-contracts');
+const { boot, request, token, suite, seamServer, ALL } = require('./helpers');
+const { STUB_PNG } = require('../server/providers/stub');
+
+const t = suite('provider-image');
+const tok = token('live', ALL);
+const PNG = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk';
+const ENV = { AI_STUB_FALLBACK: 'false', AI_PROVIDER_RETRIES: '0', AI_BREAKER_FAILURES: '50', AI_PROVIDER: 'openai', AI_IMAGE_MODEL: 'gpt-image-1' };
+
+/** An OpenAI-shaped upstream: /images/generations answers `images(body)`, anything else 404. */
+const imagesServer = (images) => seamServer((body, req) => (req.url.endsWith('/images/generations') ? images(body) : { status: 404, body: { error: { message: 'not here' } } }));
+const image = async (h, body) => {
+ const r = await request(h.base, 'POST', '/api/v1/image?wait=10000', { tok, body });
+ assert.strictEqual(r.status, 201, r.text);
+ return r.body.run;
+};
+const requestsOf = async (h, id) => (await request(h.base, 'GET', `/api/v1/runs/${id}`, { tok })).body.requests;
+
+t.test('(a) the Images API answers: output.images, a request log with operation image and usage.output = 1, one per-image reading', async () => {
+ const up = await imagesServer((body) => ({ created: 1, data: [{ b64_json: PNG }] }));
+ const h = await boot({ env: { ...ENV, AI_BASE_URL: `${up.url}/v1` } });
+ try {
+ const shared = await h.registry.getProvider('shared');
+ assert.ok(shared.capabilities.includes('image'), 'shared advertises image with AI_IMAGE_MODEL');
+ assert.strictEqual((await h.registry.getModel('shared', 'gpt-image-1')).type, 'image');
+ const route = await h.registry.resolveRoute('default.image');
+ assert.strictEqual(route.capability, 'image');
+ assert.strictEqual(route.constraints.objective, 'correctness');
+ assert.strictEqual(route.constraints.provider_scope, 'paid');
+
+ const run = await image(h, { prompt: 'a goose eating croutons' });
+ assert.strictEqual(run.status, 'succeeded', JSON.stringify(run.error));
+ assert.deepStrictEqual(run.output, { images: [{ b64: PNG, url: null }], count: 1 });
+ assert.strictEqual(run.synthetic, false);
+ assert.deepStrictEqual(up.last, { model: 'gpt-image-1', prompt: 'a goose eating croutons', n: 1, size: '1024x1024' }, 'gpt-image-1 gets no response_format; size from AI_IMAGE_SIZE');
+ assert.strictEqual(run.usage.tokens_out, 1, 'usage.output is the image count');
+ assert.strictEqual(run.usage.cost_usd, 0, 'no rate card prices an image: never the image count read as tokens');
+
+ const log = await requestsOf(h, run.id);
+ assert.strictEqual(log.length, 1);
+ assert.strictEqual(log[0].operation, 'image');
+ assert.strictEqual(log[0].status, 'ok');
+ assert.strictEqual(log[0].tokens_out, 1);
+ assert.strictEqual(log[0].tokens_in, 0);
+ assert.ok(log[0].prompt_hash && !JSON.stringify(log).includes('croutons'), 'the log keeps a hash, never the prompt');
+
+ // Billing unset: the per-image reading is queued (written, never sent), and the run is unaffected.
+ const rows = await h.db.prepare('SELECT event_id, envelope, sent_at FROM usage_sample_outbox ORDER BY id').all();
+ assert.strictEqual(rows.length, 1, 'one reading per image attempt, no token readings');
+ const s = typeof rows[0].envelope === 'string' ? JSON.parse(rows[0].envelope) : rows[0].envelope;
+ assert.strictEqual(rows[0].event_id, `ai:${run.id}:0:images`);
+ assert.strictEqual(rows[0].sent_at, null, 'queued');
+ assert.ok(contracts.validate('platform.usage-sample@1', s).valid);
+ assert.deepStrictEqual({ resource: s.resource, unit: s.unit, quantity: s.quantity, operation: s.operation, provider: s.provider },
+ { resource: 'output-images:gpt-image-1', unit: 'image', quantity: 1, operation: 'ai.image', provider: 'shared' });
+
+ // dall-e-* is asked for b64_json; n is clamped to 1..4.
+ await h.registry.upsertModel({ provider_key: 'shared', model_key: 'dall-e-3', type: 'image' });
+ const a = (await h.pool.adapter('shared')).adapter;
+ const r = await a.image({ model: 'dall-e-3', prompt: 'x', n: 9, timeoutMs: 3000 });
+ assert.strictEqual(up.last.response_format, 'b64_json');
+ assert.strictEqual(up.last.n, 4);
+ assert.strictEqual(r.usage.output, 1);
+ // A missing prompt is the caller's: 422 input.invalid.
+ const bad = await request(h.base, 'POST', '/api/v1/image?wait=10000', { tok, body: {} });
+ assert.strictEqual(bad.status, 422, bad.text);
+ assert.strictEqual(bad.body.code, 'input.invalid');
+ } finally { await h.stop(); await up.close(); }
+});
+
+t.test('(b) the first image provider answers 500: the pool falls back, with explain.candidates', async () => {
+ const dead = await imagesServer(() => ({ status: 500, body: { error: { message: 'upstream down' } } }));
+ const ok = await imagesServer((body) => ({ data: [{ b64_json: PNG }, { b64_json: PNG }].slice(0, body.n) }));
+ const h = await boot({ env: { ...ENV, AI_BASE_URL: `${dead.url}/v1` } });
+ try {
+ await h.registry.upsertProvider({ key: 'img2', kind: 'openai', status: 'active', auth_mode: 'none', base_url: `${ok.url}/v1`, default_model: null, capabilities: ['image'], timeout_ms: 3000, metadata: { paid: true } });
+ await h.registry.upsertModel({ provider_key: 'img2', model_key: 'gpt-image-1', type: 'image' });
+ const run = await image(h, { prompt: 'a lighthouse', n: 2 });
+ assert.strictEqual(run.status, 'succeeded', JSON.stringify(run.error));
+ assert.strictEqual(run.output.count, 2);
+ assert.strictEqual(run.provenance.provider, 'img2');
+ const providers = run.explain.candidates.map(c => c.provider).sort();
+ assert.deepStrictEqual(providers, ['img2', 'shared'], 'both paid image providers are candidates; never the stub or a chat model');
+ assert.strictEqual(run.explain.selected, 'img2');
+ const log = await requestsOf(h, run.id);
+ assert.deepStrictEqual(log.map(e => [e.provider_key, e.status, e.fallback, e.operation]), [['shared', 'error', 0, 'image'], ['img2', 'ok', 1, 'image']]);
+ assert.strictEqual(dead.calls, 1);
+ assert.strictEqual(run.usage.tokens_out, 2, 'metered per image returned');
+ } finally { await h.stop(); await dead.close(); await ok.close(); }
+});
+
+t.test('(c) no image provider answers: 503 provider.unavailable, never synthetic', async () => {
+ const h = await boot({ env: { ...ENV, AI_BASE_URL: 'http://127.0.0.1:1/v1' } });
+ try {
+ const run = await image(h, { prompt: 'a lighthouse' });
+ assert.strictEqual(run.status, 'failed');
+ assert.strictEqual(run.error.code, 'provider.unavailable');
+ const route = await h.registry.resolveRoute('default.image');
+ let err = null;
+ try { await h.pool.execute(route, 'image', { prompt: 'x', n: 1, timeoutMs: 2000 }, { routeKey: route.key }); } catch (e) { err = e; }
+ assert.ok(err, 'no answer');
+ assert.strictEqual(err.status, 503);
+ assert.strictEqual(err.code, 'provider.unavailable');
+ assert.ok(err.extra.explain.candidates.every(c => c.provider !== 'stub'), 'the stub is not appended without AI_STUB_FALLBACK');
+ } finally { await h.stop(); }
+});
+
+t.test('(d) AI_STUB_FALLBACK: the stub answers a deterministic 1x1 PNG, marked synthetic', async () => {
+ const h = await boot({ env: { ...ENV, AI_STUB_FALLBACK: 'true', AI_BASE_URL: 'http://127.0.0.1:1/v1' } });
+ try {
+ const run = await image(h, { prompt: 'a lighthouse', n: 2 });
+ assert.strictEqual(run.status, 'succeeded', JSON.stringify(run.error));
+ assert.strictEqual(run.synthetic, true);
+ assert.deepStrictEqual(run.output.images, [{ b64: STUB_PNG, url: null }, { b64: STUB_PNG, url: null }]);
+ const png = Buffer.from(STUB_PNG, 'base64');
+ assert.strictEqual(png.subarray(1, 4).toString(), 'PNG');
+ assert.strictEqual(png.readUInt32BE(16), 1, 'width 1');
+ assert.strictEqual(png.readUInt32BE(20), 1, 'height 1');
+ const log = await requestsOf(h, run.id);
+ assert.deepStrictEqual(log.map(e => [e.provider_key, e.status]), [['shared', 'error'], ['stub', 'ok']]);
+ } finally { await h.stop(); }
+});
+
+t.test('(e) AI_IMAGE_MODEL unset: no default.image route, no image capability, and POST /api/v1/image is not configured', async () => {
+ const h = await boot({ env: { AI_STUB_FALLBACK: 'true', AI_PROVIDER: 'openai' } });
+ try {
+ assert.ok(!await h.registry.getRoute('default.image'), 'no default.image route');
+ assert.ok(!(await h.registry.getProvider('shared')).capabilities.includes('image'));
+ assert.ok(!(await h.registry.listModels()).some(m => m.type === 'image'), 'no image model is seeded');
+ const run = await image(h, { prompt: 'a lighthouse' });
+ assert.strictEqual(run.status, 'failed');
+ // The answer every operation gives when its route is not configured (engine routeFor).
+ assert.deepStrictEqual(run.error, { code: 'route.unavailable', detail: 'no route default.image' });
+ assert.strictEqual((await requestsOf(h, run.id)).length, 0, 'no provider was called');
+ assert.strictEqual((await h.db.prepare('SELECT count(*) AS n FROM usage_sample_outbox').get()).n * 1, 0, 'nothing metered');
+ } finally { await h.stop(); }
+});
+
+t.run();
@@ -46,6 +46,7 @@ t.test('forced failure on every route: a logged fallback, or an explicit 503 (ne
@@ -46,6 +46,7 @@ t.test('forced failure on every route: a logged fallback, or an explicit 503 (ne
if (rr && !rr.disabled) routes.set(rr.key, rr);
if (rr && !rr.disabled) routes.set(rr.key, rr);
}
}
assert.ok(routes.size >= 8, `routes discovered: ${[...routes.keys()].join(', ')}`);
assert.ok(routes.size >= 8, `routes discovered: ${[...routes.keys()].join(', ')}`);
+ assert.ok(!routes.has('default.image'), 'no image route without AI_IMAGE_MODEL: the walk is unchanged');
const failed = [];
const failed = [];
for (const rr of routes.values()) {
for (const rr of routes.values()) {
@@ -127,4 +127,33 @@ t.test('local provider: never json, embed only with AI_LOCAL_EMBED_MODEL', () =>
@@ -127,4 +127,33 @@ t.test('local provider: never json, embed only with AI_LOCAL_EMBED_MODEL', () =>
assert.strictEqual(load({ AI_LOCAL_EMBED_MODEL: ' nomic-embed-text ' }).localLlm.embedModel, 'nomic-embed-text');
assert.strictEqual(load({ AI_LOCAL_EMBED_MODEL: ' nomic-embed-text ' }).localLlm.embedModel, 'nomic-embed-text');
});
});
+t.test('image: off without AI_IMAGE_MODEL; the Images API body, per-image usage and the stub PNG', async () => {
+ const shared = (env) => providerRecords(load(env), env).find(x => x.key === 'shared');
+ assert.ok(!shared({}).capabilities.includes('image'), 'no image capability without AI_IMAGE_MODEL');
+ assert.ok(shared({ AI_IMAGE_MODEL: 'gpt-image-1' }).capabilities.includes('image'));
+ assert.ok(!shared({ AI_PROVIDER: 'anthropic', AI_IMAGE_MODEL: 'gpt-image-1' }).capabilities.includes('image'), 'never on Anthropic');
+ assert.deepStrictEqual(load({}).image, { model: '', size: '1024x1024' });
+ assert.deepStrictEqual(load({ AI_IMAGE_MODEL: ' dall-e-3 ', AI_IMAGE_SIZE: '1792x1024' }).image, { model: 'dall-e-3', size: '1792x1024' });
+
+ const seen = [];
+ const s = await seamServer((body, r) => { seen.push([r.url, body]); return { data: [{ url: 'https://img.example/1.png', revised_prompt: 'a goose' }], model: body.model }; });
+ const a = createOpenAiProvider({ key: 'o', base_url: s.url }, { apiKey: 'k' });
+ assert.ok(a.supports('image'));
+ const r = await a.image({ model: 'dall-e-3', prompt: 'goose', n: 0, size: '1024x1024', timeoutMs: 3000 });
+ assert.deepStrictEqual(seen[0], ['/images/generations', { model: 'dall-e-3', prompt: 'goose', n: 1, size: '1024x1024', response_format: 'b64_json' }]);
+ assert.deepStrictEqual(r.images, [{ b64: null, url: 'https://img.example/1.png', revised_prompt: 'a goose' }]);
+ assert.deepStrictEqual(r.usage, { input: 0, output: 1, cached: 0 }, 'metered per image, never per token');
+ s.handler = () => ({ data: [] });
+ await assert.rejects(a.image({ model: 'gpt-image-1', prompt: 'goose', timeoutMs: 3000 }), (e) => e.code === 'provider.output');
+ await s.close();
+
+ const stub = createStubProvider();
+ const one = await stub.image({ prompt: 'goose', n: 3 });
+ assert.strictEqual(one.images.length, 3);
+ assert.deepStrictEqual(one, await stub.image({ prompt: 'another prompt', n: 3 }), 'the same PNG for every prompt');
+ assert.strictEqual(Buffer.from(one.images[0].b64, 'base64').subarray(1, 4).toString(), 'PNG');
+ assert.strictEqual(one.usage.output, 3);
+ assert.strictEqual((await stub.image({ prompt: 'goose', n: 99 })).images.length, 4, 'n is clamped to 4');
+});
+
t.run();
t.run();
@@ -32,6 +32,7 @@ const EXAMPLES = {
@@ -32,6 +32,7 @@ const EXAMPLES = {
'ai.extract': { text: 'Order 42 ships to Berlin.', schema: { type: 'object', additionalProperties: false, required: ['order', 'city'], properties: { order: { type: 'string' }, city: { type: 'string' } } } },
'ai.extract': { text: 'Order 42 ships to Berlin.', schema: { type: 'object', additionalProperties: false, required: ['order', 'city'], properties: { order: { type: 'string' }, city: { type: 'string' } } } },
'ai.enrich': { record: { name: 'Widget' }, instructions: 'normalise the name' },
'ai.enrich': { record: { name: 'Widget' }, instructions: 'normalise the name' },
'ai.embed': { input: ['one', 'two'] },
'ai.embed': { input: ['one', 'two'] },
+ 'ai.image': { prompt: 'A goose eating croutons, watercolour.', size: '1024x1024', n: 1 },
'wiki.generate_space': { topic: 'SQLite', sources: SRC(2) },
'wiki.generate_space': { topic: 'SQLite', sources: SRC(2) },
'wiki.generate_page': { title: 'WAL mode', sources: SRC(3) },
'wiki.generate_page': { title: 'WAL mode', sources: SRC(3) },
'blog.draft_post': { topic: 'Why we self-host', sources: SRC(1) },
'blog.draft_post': { topic: 'Why we self-host', sources: SRC(1) },
@@ -85,14 +86,15 @@ t.test('boot seeds every required workflow key', async () => {
@@ -85,14 +86,15 @@ t.test('boot seeds every required workflow key', async () => {
const mediaBase = `http://127.0.0.1:${media.address().port}`;
const mediaBase = `http://127.0.0.1:${media.address().port}`;
EXAMPLES['live.media.transcribe'].media_url = `${mediaBase}/v/1`;
EXAMPLES['live.media.transcribe'].media_url = `${mediaBase}/v/1`;
EXAMPLES['media.analyze'].media_url = `${mediaBase}/fixture.mp4`;
EXAMPLES['media.analyze'].media_url = `${mediaBase}/fixture.mp4`;
- h = await boot({ env: { OV_MEDIA_INTERNAL_URL: mediaBase } });
+ // AI_IMAGE_MODEL seeds default.image, so ai.image runs end to end on the stub's PNG like every other workflow.
+ h = await boot({ env: { OV_MEDIA_INTERNAL_URL: mediaBase, AI_IMAGE_MODEL: 'gpt-image-1' } });
const keys = (await h.registry.listWorkflows()).map(w => w.key);
const keys = (await h.registry.listWorkflows()).map(w => w.key);
for (const k of ['wiki.generate_space', 'wiki.generate_page', 'blog.draft_post', 'news.summarize_story', 'news.compare_perspectives', 'reviews.summarize_entity', 'deals.enrich_deal',
for (const k of ['wiki.generate_space', 'wiki.generate_page', 'blog.draft_post', 'news.summarize_story', 'news.compare_perspectives', 'reviews.summarize_entity', 'deals.enrich_deal',
'coupons.extract_coupon', 'trade.summarize_market_context', 'codes.generate_docs', 'games.generate_lore', 'moderation.classify', 'tools.describe', 'food.suggest_recipes',
'coupons.extract_coupon', 'trade.summarize_market_context', 'codes.generate_docs', 'games.generate_lore', 'moderation.classify', 'tools.describe', 'food.suggest_recipes',
'live.translate', 'live.stream.describe_frame', 'live.stream.summarize', 'live.viewers.reply', 'live.chat.insight', 'live.hero.slogans', 'live.stream.recap', 'network.site_copy']) {
'live.translate', 'live.stream.describe_frame', 'live.stream.summarize', 'live.viewers.reply', 'live.chat.insight', 'live.hero.slogans', 'live.stream.recap', 'network.site_copy']) {
assert.ok(keys.includes(k), `missing ${k}`);
assert.ok(keys.includes(k), `missing ${k}`);
}
}
- for (const k of ['default.chat', 'default.json', 'default.embedding', 'wiki.generate', 'blog.draft', 'news.summarize', 'reviews.summarize', 'deals.enrich', 'coupons.extract', 'trade.summarize', 'codes.generate_docs', 'games.generate_lore', 'tools.describe', 'moderation.classify', 'food.suggest']) {
+ for (const k of ['default.chat', 'default.json', 'default.embedding', 'default.image', 'wiki.generate', 'blog.draft', 'news.summarize', 'reviews.summarize', 'deals.enrich', 'coupons.extract', 'trade.summarize', 'codes.generate_docs', 'games.generate_lore', 'tools.describe', 'moderation.classify', 'food.suggest']) {
assert.ok(await h.registry.getRoute(k), `missing historical route ${k}`);
assert.ok(await h.registry.getRoute(k), `missing historical route ${k}`);
}
}
});
});