openvibers/openvibe.ai Commits

Exchange speech route: POST /v1/audio/speech via Google Cloud TTS and Amazon Polly (#451)

OP OpenVibers committed 3 days ago · 4ccddcd40d · 15 files, +700 −7

* Exchange speech route: POST /v1/audio/speech via Google Cloud TTS and Amazon Polly

Chat's TTS is the busiest direct-AI traffic; Exchange had no speech route to
move it to. This adds one in the OpenAI audio/speech shape (model, input of
at most 4096 chars, voice, response_format mp3|ogg_opus|wav, speed).

Each request is a run of the new ai.speak workflow on the new default.speech
route, so it rides the same account, quota, price-book and receipt path as
chat completions; the answer is the audio bytes with x-openvibe-run-id,
x-openvibe-receipt-id and x-openvibe-receipt headers.

Voices are Chat's 46 cloud voice ids one to one (server/providers/speech.js),
each a speech-type model in the price book at the provider's own list price
per million characters (Google Standard $4, WaveNet $4, Neural2 $16, Studio
$160, Journey at the Neural2 SKU; Polly standard $4, neural $16, long-form
$100, generative $30; read 2026-11-10). Metering is per character: usage
carries the characters and the receipt records them. The voice pins its exact
offer; an unknown voice is refused 404, never spoken in another voice.

Providers read AI's own env names (AI_GOOGLE_TTS_API_KEY or
AI_GOOGLE_TTS_SERVICE_ACCOUNT_JSON; AI_POLLY_ACCESS_KEY_ID plus
AI_POLLY_SECRET_ACCESS_KEY and AI_POLLY_REGION), off until the key resolves.
The stub answers deterministic fake audio so speech runs with no key.

Checked: test/exchange-speech.test.js 8/8 (a Google and a Polly voice answer
against local fake provider hosts, per-character cost on the receipt, unknown
voice 404, over-quota 429, validation 400s); openapi, protocol-compat,
exchange-presets-receipts, quota and ai-provider-adapters suites pass.

* Merge remote-tracking branch 'origin/main' into dsh/tts-exchange-ai

* Speech: ai.speak runs end to end on the stub (workflows gate)

The workflows suite requires every workflow to run end to end on the stub:
add the ai.speak example input, and let the speak step fall back to the
pool's own order when the pinned voice has no usable offer (no key, provider
off) — the deterministic stub answers in keyless environments. Quota, payer
and unknown-voice refusals still fail; the voice is never silently swapped
for a bad request.

Changes

ddiff --git a/docs/provider-adapters.md b/docs/provider-adapters.md
iindex 15db6ad..0744ee6 100644
---- a/docs/provider-adapters.md
++++ b/docs/provider-adapters.md
@@@ -101,3 +101,30 @@ catalogued and its key stored, but the router does not resell it.
Refuseless, Abliteration.ai and the abliterated models Featherless lists are marked `uncensored: true` in the
catalogue. The tier that lets an 18+ account ask for such an offer — the gate, the hard floor and the label on every
answer — is `docs/uncensored-policy.md`; the catalogue only records which offers are marked.
++
++## Speech (the Exchange speech route)
++
++`POST /v1/audio/speech` speaks text in a Chat voice (OpenAI's audio/speech shape: `model`, `input` of at most
++4096 characters, `voice`, `response_format` `mp3|ogg_opus|wav`, `speed` 0.25–4). Each request is a run of workflow
++`ai.speak`, so it rides the same account, quota, price-book and receipt path as chat completions; the answer is
++the audio bytes with `x-openvibe-receipt-id` and `x-openvibe-receipt` headers. The route is in
++`server/exchange/compat-openapi.js` (`compatSpeech`) and the served OpenAPI document.
++
++Two providers, each its own adapter (`server/providers/speech.js`), each a registry provider seeded by
++`server/workflows/seed.js` (active when its key resolves, off otherwise):
++
++| key | provider | voices | env (AI's own names) |
++|---|---|---|---|
++| `google-tts` | Google Cloud TTS | 24 Chat voice ids (`gc_*`, `voice_*`) | `AI_GOOGLE_TTS_API_KEY`, else `AI_GOOGLE_TTS_SERVICE_ACCOUNT_JSON` |
++| `polly` | Amazon Polly | 22 Chat voice ids (`pl_*`) | `AI_POLLY_ACCESS_KEY_ID` + `AI_POLLY_SECRET_ACCESS_KEY` + `AI_POLLY_REGION` |
++
++Voices are Chat's voice ids one to one (so Chat switches without anyone hearing a difference); an unknown
++voice is refused 404, never spoken in another voice. The voice pins its exact offer, so the run never lands
++on another voice. Each voice is a `speech`-type model in the price book at the provider's own list price per
++million characters (read 2026-11-10): Google
++([its pricing page](https://cloud.google.com/text-to-speech/pricing)) Standard $4, WaveNet $4, Neural2 $16,
++Studio $160 (Journey has no row on the current page, so it is priced at the Neural2 SKU until Google publishes
++one); Polly ([its pricing page](https://aws.amazon.com/polly/pricing/)) standard $4, neural $16, long-form $100,
++generative $30. Metering is per character: the run's `usage.input` carries the characters and the voice models'
++`in_per_mtok` is dollars per million characters (the model's `unit` metadata says `characters`); the receipt
++carries the same count in `usage.characters`.
ddiff --git a/server/api/speech.js b/server/api/speech.js
nnew file mode 100644
iindex 0000000..49b1365
---- /dev/null
++++ b/server/api/speech.js
@@@ -0,0 +1,110 @@
++'use strict';
++/**
++ * The Exchange speech endpoint: POST /v1/audio/speech in the OpenAI shape, so Chat points its
++ * TTS at https://ai.openvibe.services/v1/audio/speech and stops calling the providers directly.
++ *
++ * Authorization: Bearer <a Network token> an app's token (ai.app.run, its project pays) or a first-party
++ * service's (ai.run.create), exactly as POST /v1/chat/completions
++ * body OpenAI audio/speech: model, input (the text, at most 4096
++ * characters), voice (a Chat voice id, one per provider voice —
++ * server/providers/speech.js), response_format (mp3|ogg_opus|wav,
++ * default mp3), speed (0.25–4, default 1)
++ *
++ * Each request is a run of workflow ai.speak, so it has the run's quota, free allowance, cost, audit and
++ * explain (x_openvibe in the receipt); the text is not kept. The answer is the audio bytes (the voice's
++ * MIME type), with the run and receipt ids in the headers, as on the chat route. Errors are OpenAI's
++ * shape ({ error: { message, type, code } }) with the HTTP status the run's problem had.
++ */
++const express = require('express');
++const { CAPS } = require('../auth');
++const { appAttribution } = require('../apps');
++const { VOICES, FORMATS } = require('../providers/speech');
++const { compact, receiptId } = require('../exchange/receipts');
++
++const gatewaySafe = (status) => (status === 502 || status === 504 ? 503 : status === 422 ? 400 : status);
++
++function openaiError(res, status, message, code, type) {
++ res.status(status).json({ error: { message: String(message || 'error').slice(0, 1000), type: type || (status === 401 || status === 403 ? 'authentication_error' : status === 429 ? 'rate_limit_error' : status === 402 ? 'insufficient_quota' : status >= 500 ? 'api_error' : 'invalid_request_error'), code: code || null, param: null } });
++}
++
++function speechRouter({ runs, auth, config, presets = null, receipts = null, db = null, log = console }) {
++ const r = express.Router();
++ // Developer apps call with ai.app.run, the public capability they already hold (ADR-014; AI Exchange D8').
++ const allow = auth.appOrCap(CAPS.appRun, CAPS.runCreate);
++
++ r.post('/v1/audio/speech', allow, async (req, res) => {
++ const b = req.body && typeof req.body === 'object' ? req.body : {};
++ if (b.model == null || typeof b.model !== 'string' || !b.model.trim() || b.model.length > 256)
++ return openaiError(res, 400, 'model must be a model id', 'input.invalid');
++ const text = b.input;
++ if (typeof text !== 'string' || !text.trim()) return openaiError(res, 400, 'input must be a non-empty string', 'input.invalid');
++ if (text.length > 4096) return openaiError(res, 400, 'input is longer than 4096 characters', 'input.invalid');
++ const voice = b.voice;
++ if (typeof voice !== 'string' || !voice) return openaiError(res, 400, 'voice must be a voice id', 'input.invalid');
++ const entry = VOICES[voice];
++ // A voice nobody knows is refused, never silently spoken in another voice.
++ if (!entry) return openaiError(res, 404, `unknown voice ${voice.slice(0, 80)}`, 'ai.model_not_found');
++ const format = b.response_format == null ? 'mp3' : b.response_format;
++ if (!FORMATS[format]) return openaiError(res, 400, 'response_format must be mp3, ogg_opus or wav', 'input.invalid');
++ const speed = b.speed == null ? 1 : Number(b.speed);
++ if (!Number.isFinite(speed) || speed < 0.25 || speed > 4) return openaiError(res, 400, 'speed must be between 0.25 and 4', 'input.invalid');
++
++ const input = { voice, text, response_format: format, speed };
++ // AI Exchange F: the caller's liability cap for the Balance hold (an OpenVibe extension, like x_openvibe).
++ if (b.max_cost_usd != null) input.max_cost_usd = b.max_cost_usd;
++ let body = { workflow: 'ai.speak', input, idempotency_key: req.get('idempotency-key') || undefined };
++ // A first-party service naming the person it acts for (the runs API's on_behalf_of). An app run is its
++ // project's and never names a person, as on the runs API.
++ if (b.on_behalf_of != null) body = { ...body, on_behalf_of: b.on_behalf_of };
++ const obo = b.x_openvibe && typeof b.x_openvibe === 'object' ? b.x_openvibe.on_behalf_of : undefined;
++ if (obo !== undefined) {
++ if (req.principal.kind === 'app') return openaiError(res, 403, 'an app run is attributed to its project, never to a person', 'capability.denied');
++ if (req.principal.kind === 'user' && (!obo || obo.id !== req.principal.subject.id)) return openaiError(res, 403, 'a person runs for themselves, never for someone else', 'capability.denied');
++ body = { ...body, on_behalf_of: obo };
++ }
++ if (req.principal.kind === 'app') body = { ...body, attribution: appAttribution(req.principal) };
++ // The voice pins its exact offer (provider + voice model): the run never lands on another voice.
++ const exact = { id: `voice:${voice}`, offers: [{ provider: entry.provider, model: voice }] };
++ let run;
++ try {
++ const created = await runs.create(body, req.principal, { trace: req.ov && req.ov.traceId, requestId: req.ov && req.ov.requestId, abuse: req.abuse, exact });
++ res.locals.inferenceAdmitted = true;
++ if (created.created === false) res.setHeader('x-openvibe-replayed', '1');
++ run = await runs.explained(await runs.wait(created, Math.max(config.runs.maxWaitMs, 120000)));
++ } catch (err) {
++ const status = gatewaySafe((err && err.status) || 500);
++ if (status === 429 && err.retryAfter) res.setHeader('Retry-After', String(err.retryAfter));
++ if (status >= 500) log.warn(`[ai] /v1/audio/speech: ${err && err.message}`);
++ return openaiError(res, status, err && (err.detail || err.message), err && err.code);
++ }
++ if (['succeeded', 'cached'].includes(run.status) || (['failed', 'cancelled'].includes(run.status) && run.usage.attempts === 0)) res.locals.inferenceCharge = Number(run.usage.cost_usd || 0);
++ if (run.status === 'failed' || run.status === 'cancelled') {
++ const code = (run.error && run.error.code) || 'run.failed';
++ const status = code === 'ai.run_budget_reached' || code === 'ai.allowance_spent' || code === 'billing.insufficient_funds' || code === 'billing.payer_required' ? 402 : /^(input|validation)\./.test(code) ? 400 : /quota|allowance|budget/.test(code) ? 429
++ : code === 'ai.model_not_found' ? 404
++ : code === 'ai.model_unavailable' ? 400 : 503;
++ return openaiError(res, status, (run.error && run.error.detail) || code, code);
++ }
++ if (!['succeeded', 'cached'].includes(run.status) || !run.output || typeof run.output.audio !== 'string') {
++ return openaiError(res, 504, `the synthesis is still running; GET /api/v1/runs/${run.id}`, 'run.timeout');
++ }
++ const receipt = receipts ? await receipts.forRun(run.id) : null;
++ const receiptView = receipt ? compact(receipt) : null;
++ const audio = Buffer.from(run.output.audio, 'base64');
++ res.status(200);
++ res.setHeader('Content-Type', run.output.mime_type || 'audio/mpeg');
++ res.setHeader('Content-Length', audio.length);
++ res.setHeader('X-OpenVibe-Run-Id', run.id);
++ if (receipt && receiptView) {
++ res.setHeader('x-openvibe-receipt-id', receipt.id);
++ res.setHeader('x-openvibe-receipt', JSON.stringify(receiptView));
++ } else {
++ res.setHeader('x-openvibe-receipt-id', receiptId(run.id));
++ }
++ return res.send(audio);
++ });
++
++ return r;
++}
++
++module.exports = { speechRouter };
ddiff --git a/server/app.js b/server/app.js
iindex df118eb..74c3488 100644
---- a/server/app.js
++++ b/server/app.js
@@@ -267,6 +267,8 @@ function createApp({ choices = null, searchFetch = null, config, db, registry, p
app.use(require('./api/openai').openaiRouter({ runs, auth, config, registry, presets, receipts, db, log }));
// The Anthropic-compatible endpoint (POST /v1/messages, GET /v1/models): runs of ai.messages (Codes C3).
app.use(require('./api/messages').messagesRouter({ runs, auth, config, registry, presets, receipts, db, log }));
++ // The Exchange speech endpoint (POST /v1/audio/speech): runs of ai.speak, metered per character.
++ app.use(require('./api/speech').speechRouter({ runs, auth, config, presets, receipts, db, log }));
app.use(require('./api/attribution-quotas').attributionQuotasRouter({ db, quotas, auth, clock, sendError: (res, err, ctx) => sendError(res, err, ctx, log) }));
if (memory) app.use(require('./memory').memoryRouter({ memory, memoryImport, services: memoryServices, sharing: memorySharing, auth, config, clock, log, sendError: (res, err, ctx) => sendError(res, err, ctx, log) }));
// Memory M5: the uniform share routes for kind memory-store (and /api/v1/share/people, /api/v1/shared-with-me).
ddiff --git a/server/config.js b/server/config.js
iindex 70b591e..df379da 100644
---- a/server/config.js
++++ b/server/config.js
@@@ -152,6 +152,16 @@ function load(env = process.env) {
apiKeyRef: String(env.AI_DEEPSEEK_KEY_REF || 'env:DEEPSEEK_API_KEY').trim(),
timeoutMs: int(env.AI_DEEPSEEK_TIMEOUT_MS, 90000),
},
++ // ── Speech synthesis (the Exchange speech route): Google Cloud TTS and Amazon Polly as AI
++ // providers. Each reads its own env names (never printed, never moved); the test seam may
++ // point the base URLs at a local fake (AI_GOOGLE_TTS_BASE_URL / AI_POLLY_BASE_URL).
++ speech: {
++ googleKeyRef: String(env.AI_GOOGLE_TTS_KEY_REF || 'env:AI_GOOGLE_TTS_API_KEY').trim(),
++ googleBaseUrl: trimUrl(env.AI_GOOGLE_TTS_BASE_URL || 'https://texttospeech.googleapis.com'),
++ pollyKeyRef: String(env.AI_POLLY_KEY_REF || 'env:AI_POLLY_ACCESS_KEY_ID').trim(),
++ pollyBaseUrl: trimUrl(env.AI_POLLY_BASE_URL || ''),
++ timeoutMs: int(env.AI_SPEECH_TIMEOUT_MS, 60000),
++ },
// ── Spend safety for runs nobody pays for (the free allowance, the anonymous Try it) ──
// A run with no person's own token, no app project, no service run-token budget and no own key may
// use only offers from these sources (provider keys; `local` is every local model server and
ddiff --git a/server/exchange/compat-openapi.js b/server/exchange/compat-openapi.js
iindex 985052d..d179934 100644
---- a/server/exchange/compat-openapi.js
++++ b/server/exchange/compat-openapi.js
@@@ -21,6 +21,13 @@ const paths = {
'/v1/completions': { post: operation('compatCompletions', 'Legacy string-prompt completion', { ...properties, prompt: { type: 'string', minLength: 1 }, stream: { type: 'boolean' } }, ['prompt']) },
'/v1/embeddings': { post: operation('compatEmbeddings', 'Float embeddings from a registered provider', { ...properties, input: { anyOf: [{ type: 'string' }, { type: 'array', items: { type: 'string' } }] }, encoding_format: { const: 'float' } }, ['input']) },
'/v1/images/generations': { post: operation('compatImageGeneration', 'Image generation on configured non-metered capacity; paid image accounting is not yet supported', { ...properties, prompt: { type: 'string' }, n: { type: 'integer', minimum: 1, maximum: 4 }, size: { type: 'string' } }, ['prompt']) },
++ '/v1/audio/speech': { post: { operationId: 'compatSpeech', summary: 'Speech synthesis in a Chat voice, metered per character', tags: ['exchange'], security: [{ bearer: [] }],
++ requestBody: { required: true, content: { 'application/json': { schema: { type: 'object', additionalProperties: false, required: ['model', 'input', 'voice'],
++ properties: { model: { type: 'string', description: 'Any model id (the voice pins the offer).' }, input: { type: 'string', maxLength: 4096, description: 'The text to speak.' },
++ voice: { type: 'string', description: 'A Chat voice id (server/providers/speech.js), one per provider voice.' },
++ response_format: { enum: ['mp3', 'ogg_opus', 'wav'], default: 'mp3' }, speed: { type: 'number', minimum: 0.25, maximum: 4, default: 1 },
++ max_cost_usd: { type: 'number', minimum: 0, description: 'The caller liability cap for the Balance hold.' } } } } } },
++ responses: { 200: { description: 'The audio bytes (the voice MIME type), with x-openvibe-receipt-id and x-openvibe-receipt headers.' }, 400: { description: 'Invalid request' }, 401: { description: 'Invalid or expired key' }, 402: { description: 'Quota or balance refused' }, 404: { description: 'Unknown voice' }, 429: { description: 'Key cap reached' }, 503: { description: 'No eligible provider' } } } },
'/v1/generation': { get: { operationId: 'compatGeneration', summary: 'Read accounting for your own generation', tags: ['exchange'], security: [{ bearer: [] }], parameters: [{ in: 'query', name: 'id', required: true, schema: { type: 'string' } }], responses: { 200: { description: 'data: id, model, provider, cost, total_cost (USD), latency (ms), tokens_prompt, tokens_completion, created_at. No prompt or output.' }, 404: { description: 'Missing or owned by another requester' } } } },
'/v1/presets/import': { post: operation('compatPresetImport', 'Save a validated request preset for the signed-in person', { ...properties, name: { type: 'string' }, system: { type: 'string' }, temperature: { type: 'number' }, top_p: { type: 'number' }, max_tokens: { type: 'integer' } }, ['name']) },
};
ddiff --git a/server/exchange/receipts.js b/server/exchange/receipts.js
iindex 13bb1e7..d13be81 100644
---- a/server/exchange/receipts.js
++++ b/server/exchange/receipts.js
@@@ -75,6 +75,9 @@ function buildReceipt({ run, explain = null, usage = {}, price = null, privacy =
tokens_cached: int(usage.cached),
tokens_cache_write: int(usage.cache_write),
tokens_reasoning: int(usage.reasoning),
++ // Speech (ai.speak) is metered per character, like the providers bill: the same count the
++ // rate cards priced. Other runs carry no character count and omit the field.
++ ...(int(usage.characters) > 0 ? { characters: int(usage.characters) } : {}),
};
const paidBy = (payer && payer.paid_by) || 'openvibe_key';
const byok = paidBy === 'own_key' || paidBy === 'own_machine';
@@@ -132,7 +135,8 @@ function compact(receipt) {
local: Boolean(offer.local),
community: offer.community || null,
price: { in: receipt.price && receipt.price.in_per_mtok, out: receipt.price && receipt.price.out_per_mtok, cached: receipt.price && receipt.price.cached_per_mtok },
-- tokens: { in: receipt.usage && receipt.usage.tokens_in, out: receipt.usage && receipt.usage.tokens_out, cached: receipt.usage && receipt.usage.tokens_cached },
++ tokens: { in: receipt.usage && receipt.usage.tokens_in, out: receipt.usage && receipt.usage.tokens_out, cached: receipt.usage && receipt.usage.tokens_cached,
++ ...(receipt.usage && receipt.usage.characters ? { characters: receipt.usage.characters } : {}) },
cost_usd: receipt.cost_usd,
fee_usd: receipt.fee_usd,
fee_rate: receipt.fee_rate,
ddiff --git a/server/providers/index.js b/server/providers/index.js
iindex 21b1385..8066dcf 100644
---- a/server/providers/index.js
++++ b/server/providers/index.js
@@@ -40,6 +40,7 @@ const { createAnthropicProvider } = require('./anthropic');
const { createGeminiProvider } = require('./gemini');
const { createHttpSeamProvider } = require('./http');
const { createWhisperProvider } = require('./whisper');
++const { createSpeechProvider } = require('./speech');
const rates = require('./rate-cards');
const tiers = require('./tiers');
const latency = require('../latency');
@@@ -365,6 +366,8 @@ function createProviderPool({ db, registry, config, clock = { now: () => Date.no
case 'gemini': return createGeminiProvider(p, { apiKey, fetchImpl });
case 'http': return createHttpSeamProvider(p, { apiKey, fetchImpl });
case 'whisper': return createWhisperProvider(p, { config });
++ case 'google-tts':
++ case 'polly': return createSpeechProvider(p, { apiKey, fetchImpl, env });
default: throw new Error(`unknown provider kind ${p.kind}`);
}
}
ddiff --git a/server/providers/speech.js b/server/providers/speech.js
nnew file mode 100644
iindex 0000000..fbad486
---- /dev/null
++++ b/server/providers/speech.js
@@@ -0,0 +1,251 @@
++'use strict';
++/**
++ * Speech synthesis providers (the Exchange speech route, `POST /v1/audio/speech`).
++ *
++ * Two paid providers, each a pool record of its own kind (server/providers/pool-providers.js):
++ *
++ * google-tts Google Cloud Text-to-Speech (`texttospeech.googleapis.com/v1/text:synthesize`).
++ * Auth is the service account (`AI_GOOGLE_TTS_SERVICE_ACCOUNT_JSON`, a JWT minted here)
++ * else the API key (`AI_GOOGLE_TTS_API_KEY`, the record's secret_ref).
++ * polly Amazon Polly (`POST polly.{region}.amazonaws.com/v1/speech`, SigV4-signed).
++ * Auth is the access key id (`AI_POLLY_ACCESS_KEY_ID`, the record's secret_ref) plus
++ * `AI_POLLY_SECRET_ACCESS_KEY` and `AI_POLLY_REGION` (default `us-east-1`).
++ *
++ * Voices are Chat's voice ids, one to one (so Chat can switch without anyone hearing a difference):
++ * `voice` names the entry below, and the entry names the provider, the provider's own voice and the
++ * list price per million characters. Prices were read on the providers' own pages on 2026-11-10:
++ *
++ * Google (https://cloud.google.com/text-to-speech/pricing): Standard $4, WaveNet $4,
++ * Neural2 $16, Studio $160. Journey has no row on the current page; it is priced at the
++ * Neural2 SKU ($16) until Google publishes one for it.
++ * Polly (https://aws.amazon.com/polly/pricing/): standard $4, neural $16, long-form $100,
++ * generative $30.
++ *
++ * Metering is per character, like the providers bill: usage.input carries the characters (the rate
++ * cards price `in_per_mtok` as dollars per million characters for these models, with the unit named
++ * in the model metadata), and `usage.characters` carries the same count for receipts.
++ */
++const crypto = require('crypto');
++const { ProviderError } = require('./common');
++
++const GOOGLE_ENDPOINT = 'https://texttospeech.googleapis.com/v1/text:synthesize';
++const GOOGLE_TOKEN_URL = 'https://oauth2.googleapis.com/token';
++
++// List price, USD per million characters, per provider voice tier (read 2026-11-10, see above).
++const GOOGLE_PRICE = { standard: 4, wavenet: 4, neural2: 16, studio: 160, journey: 16 };
++const POLLY_PRICE = { standard: 4, neural: 16, 'long-form': 100, generative: 30 };
++
++const googleTierOf = (name) => {
++ const n = String(name || '');
++ if (/-Studio-/i.test(n)) return 'studio';
++ if (/-Journey-/i.test(n)) return 'journey';
++ if (/-Neural2-/i.test(n)) return 'neural2';
++ if (/-Wavenet-/i.test(n)) return 'wavenet';
++ return 'standard';
++};
++
++/**
++ * Chat's cloud voices, one to one: id → the provider, the provider's own voice, the locale and the
++ * list price per million characters. Pitch and speaking rate are Chat's own values (only Vegas and
++ * Bass Boss set them); the request's `speed` multiplies the speaking rate.
++ */
++const VOICES = {
++ gc_smooth_operator: { provider: 'google-tts', voice: 'en-US-Standard-D', locale: 'en-US', gender: 'MALE' },
++ gc_silicon_sally: { provider: 'google-tts', voice: 'en-US-Standard-C', locale: 'en-US', gender: 'FEMALE' },
++ gc_brit_butler: { provider: 'google-tts', voice: 'en-GB-Standard-B', locale: 'en-GB', gender: 'MALE' },
++ gc_lady_london: { provider: 'google-tts', voice: 'en-GB-Standard-A', locale: 'en-GB', gender: 'FEMALE' },
++ gc_down_under: { provider: 'google-tts', voice: 'en-AU-Standard-B', locale: 'en-AU', gender: 'MALE' },
++ gc_sheila: { provider: 'google-tts', voice: 'en-AU-Standard-A', locale: 'en-AU', gender: 'FEMALE' },
++ gc_mumbai_mike: { provider: 'google-tts', voice: 'en-IN-Standard-B', locale: 'en-IN', gender: 'MALE' },
++ gc_news_anchor: { provider: 'google-tts', voice: 'en-US-Standard-J', locale: 'en-US', gender: 'MALE' },
++ gc_podcast_girl: { provider: 'google-tts', voice: 'en-US-Standard-H', locale: 'en-US', gender: 'FEMALE' },
++ gc_wave_daddy: { provider: 'google-tts', voice: 'en-US-Wavenet-D', locale: 'en-US', gender: 'MALE' },
++ gc_wave_queen: { provider: 'google-tts', voice: 'en-US-Wavenet-F', locale: 'en-US', gender: 'FEMALE' },
++ gc_wave_brit: { provider: 'google-tts', voice: 'en-GB-Wavenet-B', locale: 'en-GB', gender: 'MALE' },
++ gc_wave_aussie: { provider: 'google-tts', voice: 'en-AU-Wavenet-B', locale: 'en-AU', gender: 'MALE' },
++ gc_neural_chad: { provider: 'google-tts', voice: 'en-US-Neural2-D', locale: 'en-US', gender: 'MALE' },
++ gc_neural_diva: { provider: 'google-tts', voice: 'en-US-Neural2-F', locale: 'en-US', gender: 'FEMALE' },
++ gc_neural_brit_f: { provider: 'google-tts', voice: 'en-GB-Neural2-A', locale: 'en-GB', gender: 'FEMALE' },
++ gc_neural_aussie_f: { provider: 'google-tts', voice: 'en-AU-Neural2-A', locale: 'en-AU', gender: 'FEMALE' },
++ gc_studio_m: { provider: 'google-tts', voice: 'en-US-Studio-Q', locale: 'en-US', gender: 'MALE' },
++ gc_studio_f: { provider: 'google-tts', voice: 'en-US-Studio-O', locale: 'en-US', gender: 'FEMALE' },
++ gc_journey_sage: { provider: 'google-tts', voice: 'en-US-Journey-D', locale: 'en-US', gender: 'MALE' },
++ gc_journey_oracle: { provider: 'google-tts', voice: 'en-US-Journey-F', locale: 'en-US', gender: 'FEMALE' },
++ voice_vegas: { provider: 'google-tts', voice: 'en-US-Neural2-J', locale: 'en-US', gender: 'MALE', pitch: -4.0, speakingRate: 0.9 },
++ voice_twofat: { provider: 'google-tts', voice: 'en-US-Neural2-D', locale: 'en-US', gender: 'MALE', pitch: -8.0, speakingRate: 1.0 },
++ gc_journey_whisper: { provider: 'google-tts', voice: 'en-US-Journey-O', locale: 'en-US', gender: 'FEMALE' },
++ pl_joanna_std: { provider: 'polly', voiceId: 'Joanna', engine: 'standard', locale: 'en-US' },
++ pl_matthew_std: { provider: 'polly', voiceId: 'Matthew', engine: 'standard', locale: 'en-US' },
++ pl_amy_std: { provider: 'polly', voiceId: 'Amy', engine: 'standard', locale: 'en-GB' },
++ pl_brian_std: { provider: 'polly', voiceId: 'Brian', engine: 'standard', locale: 'en-GB' },
++ pl_olivia_std: { provider: 'polly', voiceId: 'Olivia', engine: 'standard', locale: 'en-AU' },
++ pl_ivy_std: { provider: 'polly', voiceId: 'Ivy', engine: 'standard', locale: 'en-US' },
++ pl_joey_std: { provider: 'polly', voiceId: 'Joey', engine: 'standard', locale: 'en-US' },
++ pl_kendra_std: { provider: 'polly', voiceId: 'Kendra', engine: 'standard', locale: 'en-US' },
++ pl_joanna_neural: { provider: 'polly', voiceId: 'Joanna', engine: 'neural', locale: 'en-US' },
++ pl_matthew_neural: { provider: 'polly', voiceId: 'Matthew', engine: 'neural', locale: 'en-US' },
++ pl_ruth_neural: { provider: 'polly', voiceId: 'Ruth', engine: 'neural', locale: 'en-US' },
++ pl_stephen_neural: { provider: 'polly', voiceId: 'Stephen', engine: 'neural', locale: 'en-US' },
++ pl_amy_neural: { provider: 'polly', voiceId: 'Amy', engine: 'neural', locale: 'en-GB' },
++ pl_arthur_neural: { provider: 'polly', voiceId: 'Arthur', engine: 'neural', locale: 'en-GB' },
++ pl_danielle_neural: { provider: 'polly', voiceId: 'Danielle', engine: 'neural', locale: 'en-US' },
++ pl_gregory_neural: { provider: 'polly', voiceId: 'Gregory', engine: 'neural', locale: 'en-US' },
++ pl_kevin_neural: { provider: 'polly', voiceId: 'Kevin', engine: 'neural', locale: 'en-US' },
++ pl_danielle_long: { provider: 'polly', voiceId: 'Danielle', engine: 'long-form', locale: 'en-US' },
++ pl_gregory_long: { provider: 'polly', voiceId: 'Gregory', engine: 'long-form', locale: 'en-US' },
++ pl_ruth_long: { provider: 'polly', voiceId: 'Ruth', engine: 'long-form', locale: 'en-US' },
++ pl_matthew_gen: { provider: 'polly', voiceId: 'Matthew', engine: 'generative', locale: 'en-US' },
++ pl_ruth_gen: { provider: 'polly', voiceId: 'Ruth', engine: 'generative', locale: 'en-US' },
++};
++for (const [id, v] of Object.entries(VOICES)) {
++ v.id = id;
++ v.price_per_million_chars_usd = v.provider === 'google-tts' ? GOOGLE_PRICE[googleTierOf(v.voice)] : POLLY_PRICE[v.engine];
++}
++
++const FORMATS = {
++ mp3: { google: 'MP3', polly: 'mp3', mime: 'audio/mpeg' },
++ ogg_opus: { google: 'OGG_OPUS', polly: 'ogg_vorbis', mime: 'audio/ogg' },
++ wav: { google: 'LINEAR16', polly: 'pcm', mime: 'audio/wav' },
++};
++
++const payoffOf = (err, def) => { throw err instanceof ProviderError ? err : new ProviderError(def, { code: 'provider.unreachable' }); };
++
++async function postJson(url, headers, body, { fetchImpl, timeoutMs, signal }) {
++ const ctrl = new AbortController();
++ const timer = setTimeout(() => ctrl.abort(), Math.max(1000, Number(timeoutMs) || 60000));
++ try {
++ const res = await fetchImpl(url, { method: 'POST', headers, body: JSON.stringify(body),
++ signal: signal ? AbortSignal.any([signal, ctrl.signal]) : ctrl.signal });
++ const text = await res.text();
++ if (!res.ok) throw new ProviderError(`speech HTTP ${res.status}: ${text.slice(0, 300)}`,
++ { status: res.status, code: res.status === 429 ? 'provider.rate_limited' : res.status === 401 || res.status === 403 ? 'provider.auth' : 'provider.http' });
++ return JSON.parse(text);
++ } catch (err) {
++ if (err instanceof ProviderError) throw err;
++ throw new ProviderError(`speech request failed: ${String((err && err.message) || err).slice(0, 200)}`,
++ { code: err && err.name === 'AbortError' ? 'provider.timeout' : 'provider.unreachable' });
++ } finally {
++ clearTimeout(timer);
++ }
++}
++
++/** A Google service-account JWT → an OAuth access token (the server-to-server flow Chat uses). */
++async function googleAccessToken(sa, fetchImpl) {
++ const now = Math.floor(Date.now() / 1000);
++ const enc = (o) => Buffer.from(JSON.stringify(o)).toString('base64url');
++ const sig = crypto.createSign('RSA-SHA256').update(`${enc({ alg: 'RS256', typ: 'JWT' })}.${enc({
++ iss: sa.client_email, scope: 'https://www.googleapis.com/auth/cloud-platform',
++ aud: GOOGLE_TOKEN_URL, iat: now, exp: now + 3600 })}`).sign(sa.private_key, 'base64url');
++ const form = `grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=${enc({ alg: 'RS256', typ: 'JWT' })}.${enc({
++ iss: sa.client_email, scope: 'https://www.googleapis.com/auth/cloud-platform',
++ aud: GOOGLE_TOKEN_URL, iat: now, exp: now + 3600 })}.${sig}`;
++ const ctrl = new AbortController();
++ const timer = setTimeout(() => ctrl.abort(), 30000);
++ try {
++ const res = await fetchImpl(GOOGLE_TOKEN_URL, { method: 'POST',
++ headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, body: form, signal: ctrl.signal });
++ const j = await res.json();
++ if (!res.ok || !j.access_token) throw new Error('token refused');
++ return j.access_token;
++ } catch (err) {
++ throw new ProviderError(`google auth failed: ${String((err && err.message) || err).slice(0, 160)}`, { code: 'provider.auth' });
++ } finally {
++ clearTimeout(timer);
++ }
++}
++
++/** SigV4 for `POST polly.{region}.amazonaws.com/v1/speech` (the same signing Chat uses). */
++function pollyHeaders({ region, accessKey, secretKey, body }) {
++ const now = new Date();
++ const date = now.toISOString().replace(/[:-]|\.\d{3}/g, '').slice(0, 8);
++ const amz = now.toISOString().replace(/[:-]|\.\d{3}/g, '');
++ const host = `polly.${region}.amazonaws.com`;
++ const headers = { host, 'x-amz-date': amz, 'content-type': 'application/json' };
++ const signed = Object.keys(headers).map((k) => k.toLowerCase()).sort().join(';');
++ const canonical = ['POST', '/v1/speech', '', ...Object.keys(headers).map((k) => `${k.toLowerCase()}:${String(headers[k]).trim()}`).sort(), '',
++ crypto.createHash('sha256').update(body).digest('hex')].join('\n');
++ const scope = `${date}/${region}/polly/aws4_request`;
++ const toSign = ['AWS4-HMAC-SHA256', amz, scope, crypto.createHash('sha256').update(canonical).digest('hex')].join('\n');
++ const hmac = (key, data) => crypto.createHmac('sha256', key).update(data).digest();
++ const signing = hmac(hmac(hmac(hmac(`AWS4${secretKey}`, date), region), 'polly'), 'aws4_request');
++ return { host, headers: { ...headers, Authorization: `AWS4-HMAC-SHA256 Credential=${accessKey}/${scope}, SignedHeaders=${signed}, Signature=${crypto.createHmac('sha256', signing).update(toSign).digest('hex')}` } };
++}
++
++function createSpeechProvider(record, { apiKey = '', fetchImpl = globalThis.fetch, env = process.env } = {}) {
++ const kind = record.key;
++ // The test seam may point these at a local fake (AI_GOOGLE_TTS_BASE_URL / AI_POLLY_BASE_URL);
++ // production calls the providers' own hosts.
++ const googleBase = String(env.AI_GOOGLE_TTS_BASE_URL || 'https://texttospeech.googleapis.com').replace(/\/+$/, '');
++ const pollyBase = String(env.AI_POLLY_BASE_URL || '').replace(/\/+$/, '');
++ const svcAccount = () => {
++ const raw = env.AI_GOOGLE_TTS_SERVICE_ACCOUNT_JSON || '';
++ if (!raw) return null;
++ try { return JSON.parse(raw); } catch { try { return JSON.parse(require('fs').readFileSync(raw, 'utf8')); } catch { return null; } }
++ };
++
++ async function speakGoogle(entry, { text, format, speed, timeoutMs, signal }) {
++ const f = FORMATS[format] || FORMATS.mp3;
++ const body = {
++ input: { text },
++ voice: { languageCode: entry.locale, name: entry.voice, ssmlGender: entry.gender },
++ audioConfig: { audioEncoding: f.google,
++ speakingRate: Math.max(0.25, Math.min(4, (entry.speakingRate || 1) * (speed || 1))),
++ pitch: Number(entry.pitch) || 0 },
++ };
++ const sa = svcAccount();
++ let url = `${googleBase}/v1/text:synthesize`, headers = { 'Content-Type': 'application/json' };
++ if (sa && sa.client_email && sa.private_key) headers.Authorization = `Bearer ${await googleAccessToken(sa, fetchImpl)}`;
++ else if (apiKey) url += `?key=${encodeURIComponent(apiKey)}`;
++ else throw new ProviderError('google Cloud TTS is not configured', { code: 'provider.auth' });
++ const j = await postJson(url, headers, body, { fetchImpl, timeoutMs, signal });
++ if (!j.audioContent) throw new ProviderError('google TTS returned no audio', { code: 'provider.output' });
++ return { audio: Buffer.from(j.audioContent, 'base64'), mimeType: f.mime };
++ }
++
++ async function speakPolly(entry, { text, format, speed, timeoutMs, signal }) {
++ const f = FORMATS[format] || FORMATS.mp3;
++ const secret = env.AI_POLLY_SECRET_ACCESS_KEY || '';
++ const region = env.AI_POLLY_REGION || 'us-east-1';
++ if (!apiKey || !secret) throw new ProviderError('amazon Polly is not configured', { code: 'provider.auth' });
++ const body = JSON.stringify({ OutputFormat: f.polly, Text: text, TextType: 'text',
++ VoiceId: entry.voiceId, Engine: entry.engine, ...(entry.locale ? { LanguageCode: entry.locale } : {}) });
++ const { host, headers } = pollyHeaders({ region, accessKey: apiKey, secretKey: secret, body });
++ const ctrl = new AbortController();
++ const timer = setTimeout(() => ctrl.abort(), Math.max(1000, Number(timeoutMs) || 60000));
++ try {
++ const url = pollyBase ? `${pollyBase}/v1/speech` : `https://${host}/v1/speech`;
++ const res = await fetchImpl(url, { method: 'POST', headers,
++ body, signal: signal ? AbortSignal.any([signal, ctrl.signal]) : ctrl.signal });
++ const buf = Buffer.from(await res.arrayBuffer());
++ if (!res.ok) throw new ProviderError(`polly HTTP ${res.status}: ${buf.toString('utf8').slice(0, 300)}`,
++ { status: res.status, code: res.status === 429 ? 'provider.rate_limited' : res.status === 401 || res.status === 403 ? 'provider.auth' : 'provider.http' });
++ return { audio: buf, mimeType: f.mime };
++ } catch (err) {
++ if (err instanceof ProviderError) throw err;
++ payoffOf(err, `polly request failed: ${String((err && err.message) || err).slice(0, 200)}`);
++ } finally {
++ clearTimeout(timer);
++ }
++ }
++
++ /**
++ * Synthesize one line. req: { voice (a VOICES id), text, format, speed, model, timeoutMs, signal }.
++ * → { audio (Buffer), mimeType, model, usage }. usage.input carries the characters (the rate
++ * cards price these models' `in_per_mtok` as dollars per million characters), and
++ * usage.characters carries the same count for the receipt.
++ */
++ async function speak(req) {
++ const entry = VOICES[req.voice];
++ if (!entry) throw new ProviderError(`unknown voice ${String(req.voice || '').slice(0, 80)}`, { code: 'provider.model' });
++ if (entry.provider !== kind) throw new ProviderError(`voice ${entry.id} is not a ${kind} voice`, { code: 'provider.model' });
++ const text = String(req.text || '');
++ if (!text.trim()) throw new ProviderError('speech needs text', { code: 'provider.input' });
++ const out = entry.provider === 'google-tts' ? await speakGoogle(entry, req) : await speakPolly(entry, req);
++ const characters = text.length;
++ return { ...out, model: entry.id, usage: { input: characters, output: 0, cached: 0, characters } };
++ }
++
++ return { key: record.key, kind: 'speech', synthetic: false, speak, supports: (f) => f === 'speak' };
++}
++
++module.exports = { createSpeechProvider, VOICES, FORMATS, googleTierOf, GOOGLE_PRICE, POLLY_PRICE, GOOGLE_ENDPOINT };
ddiff --git a/server/providers/stub.js b/server/providers/stub.js
iindex 30d16f5..7274f65 100644
---- a/server/providers/stub.js
++++ b/server/providers/stub.js
@@@ -231,6 +231,20 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nu
return { message, text: content.filter(c => c.type === 'text').map(c => c.text).join(''), model, synthetic: true, usage: { input: inTok, output: outTok, cached: 0, estimated: true } };
}
++ /** speak: deterministic fake MP3 bytes, metered per character like the real speech providers. */
++ async function speak(req) {
++ if (stats) stats.calls = (stats.calls || 0) + 1;
++ if (req.signal && req.signal.aborted) throw new ProviderError('cancelled', { code: 'provider.cancelled' });
++ if (fail === 'error') throw new ProviderError('stub drill: provider error', { status: 503, code: 'provider.http' });
++ if (fail === 'timeout') { await sleep(Math.min(req.timeoutMs || 1000, 60000), req.signal); throw new ProviderError('stub drill: timeout', { code: 'provider.timeout' }); }
++ const characters = String(req.text || '').length;
++ // Minimal valid MP3 frame header + padding: players accept the bytes, and the bytes are stable per text.
++ const digest = crypto.createHash('sha256').update(String(req.text || '')).digest();
++ const audio = Buffer.concat([Buffer.from([0xff, 0xfb, 0x90, 0x00]), digest, Buffer.alloc(64, 0)]);
++ return { audio, mimeType: 'audio/mpeg', model: req.model || 'stub-1', synthetic: true,
++ usage: { input: characters, output: 0, cached: 0, characters } };
++ }
++
const adapter = {
key: record.key,
kind: 'stub',
@@@ -241,6 +255,7 @@ function createStubProvider(record = { key: 'stub', metadata: {} }, { stats = nu
transcribe: async (req) => await run('transcribe', req),
embed: async (req) => await run('embed', req),
image: async (req) => await run('image', req),
++ speak,
};
for (const op of ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich']) adapter[op] = async (req) => await run(op, req);
return adapter;
ddiff --git a/server/registry.js b/server/registry.js
iindex 5b4f233..58d28bc 100644
---- a/server/registry.js
++++ b/server/registry.js
@@@ -13,11 +13,11 @@ const { sha256, parseJson, iso, AiError, secretRefValid, providerSecret } = requ
const schemas = require('./schemas');
const { provenance } = require('./providers/rate-cards');
--const PROVIDER_KINDS = ['stub', 'openai', 'responses', 'anthropic', 'http', 'whisper'];
++const PROVIDER_KINDS = ['stub', 'openai', 'responses', 'anthropic', 'http', 'whisper', 'google-tts', 'polly'];
const BILLING_PROFILES = ['metered', 'subscription', 'payg', 'free', 'byok'];
// AI Exchange D3: who may resell this offer's capacity. Defaults to UNVERIFIED (migrations/0024).
const PROVIDER_ACCESS = ['OWN_DEPLOYMENT', 'AUTHORIZED_WHOLESALE', 'CUSTOMER_BYOK', 'RESTRICTED', 'UNVERIFIED'];
--const FEATURES = ['chat', 'complete', 'messages', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed', 'vision', 'json', 'tools', 'transcribe', 'image', 'video', 'speech', 'rerank', 'web-search', 'image-search'];
++const FEATURES = ['chat', 'complete', 'messages', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed', 'vision', 'json', 'tools', 'transcribe', 'image', 'video', 'speech', 'speak', 'rerank', 'web-search', 'image-search'];
const LIFECYCLE = ['draft', 'active', 'deprecated', 'archived'];
const KEY_RE = /^[a-z][a-z0-9_-]*(\.[a-z0-9_-]+)*$/;
const flag = (v) => v === true || v === 1 || /^(1|true|yes|on)$/i.test(String(v == null ? '' : v));
@@@ -361,7 +361,7 @@ function createRegistry(db, { clock = { now: () => Date.now() }, env = process.e
schemas.assertSchema(w.output_schema, 'output_schema');
if (!Array.isArray(w.steps) || !w.steps.length) throw new AiError(422, 'ai.invalid', 'workflow needs at least one step');
for (const s of w.steps) {
-- if (!s || !['llm', 'passthrough', 'complete', 'messages', 'transcribe', 'embed', 'image', 'picture', 'media_analysis', 'research', 'devblog'].includes(s.kind)) throw new AiError(422, 'ai.invalid', 'step kind must be llm, passthrough, complete, messages, transcribe, embed, image, picture, media_analysis or research');
++ if (!s || !['llm', 'passthrough', 'complete', 'messages', 'transcribe', 'embed', 'image', 'speak', 'picture', 'media_analysis', 'research', 'devblog'].includes(s.kind)) throw new AiError(422, 'ai.invalid', 'step kind must be llm, passthrough, complete, messages, transcribe, embed, image, speak, picture, media_analysis or research');
if ((s.kind === 'llm' || (s.kind === 'media_analysis' && s.template)) && !(s.template && await latestAny('templates', s.template))) throw new AiError(422, 'ai.invalid', `step references unknown template ${s.template}`);
}
if (!['none', 'private', 'service'].includes(w.cache_mode)) throw new AiError(422, 'ai.invalid', 'cache_mode must be none, private or service');
ddiff --git a/server/workflows/core.js b/server/workflows/core.js
iindex 12168b7..65e7553 100644
---- a/server/workflows/core.js
++++ b/server/workflows/core.js
@@@ -89,6 +89,28 @@ const workflows = [
output_schema: OBJ({ data: { type: 'object' }, notes: { type: 'array', items: { type: 'string' } } }, ['data']),
steps: [{ kind: 'llm', template: 'ai.enrich', operation: 'enrich', output: 'json', prepare: 'ai.enrich', params: { max_tokens: 1200, temperature: 0.2 } }], cache_mode: 'private',
},
++ {
++ // The Exchange speech route (POST /v1/audio/speech, server/api/speech.js): the caller's text spoken in the
++ // named voice (a Chat voice id, one per provider voice — server/providers/speech.js). Metered per
++ // character like the providers bill: usage.input carries the characters and the voice models' rate
++ // cards price `in_per_mtok` as dollars per million characters. The run keeps its usage, cost, quota,
++ // free allowance and receipt exactly like ai.complete; the answer is the audio, base64 inline (as images).
++ key: 'ai.speak', name: 'Synthesize speech (Exchange TTS)', namespace: 'ai',
++ input_schema: OBJ({
++ voice: STR(80),
++ text: STR(4096),
++ response_format: { enum: ['mp3', 'ogg_opus', 'wav'] },
++ speed: { type: ['number', 'null'], minimum: 0.25, maximum: 4 },
++ ...COMMON,
++ }, ['voice', 'text']),
++ output_schema: OBJ({
++ audio: STR(8000000),
++ mime_type: STR(80),
++ voice: STR(80),
++ characters: { type: 'integer', minimum: 1 },
++ }, ['audio', 'mime_type', 'voice', 'characters']),
++ steps: [{ kind: 'speak', route: 'default.speech' }], cache_mode: 'none',
++ },
{
key: 'ai.embed', name: 'Embed text', namespace: 'ai',
input_schema: OBJ({ compat_routing: { type: 'object' }, max_cost_usd: COMMON.max_cost_usd, input: { anyOf: [STR(32000), { type: 'array', minItems: 1, maxItems: 64, items: STR(32000) }] } }, ['input']),
ddiff --git a/server/workflows/engine.js b/server/workflows/engine.js
iindex b2ca49f..9331889 100644
---- a/server/workflows/engine.js
++++ b/server/workflows/engine.js
@@@ -506,6 +506,42 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
return { output: { images: images.map(i => ({ b64: i.b64 || null, url: i.url || null })), count: images.length }, exec, route };
}
++ /**
++ * speak: the caller's text spoken in the named voice (ai.speak, POST /v1/audio/speech). The voice
++ * pins the exact offer, so the run never lands on another voice; an unknown voice is refused before
++ * the run (the API layer checks the same map). Metered per character: usage.input carries the
++ * characters (the voice models' rate cards price `in_per_mtok` per million characters).
++ */
++ async function speakStep(step, input, wf, run, ctx) {
++ const { route, selection } = await selectedRoute(await routeFor(step.route || 'default.speech'), input);
++ const { VOICES } = require('../providers/speech');
++ const voice = String(input.voice || '');
++ const entry = VOICES[voice];
++ if (!entry) throw new AiError(404, 'ai.model_not_found', `unknown voice ${voice.slice(0, 80)}`);
++ const text = String(input.text || '');
++ if (!text.trim()) throw new AiError(422, 'input.invalid', 'speech needs text');
++ const format = input.response_format || 'mp3';
++ const speed = input.speed == null ? 1 : Number(input.speed);
++ const exact = { id: `voice:${voice}`, offers: [{ provider: entry.provider, model: voice }] };
++ const req = { voice, text, format, speed, model: voice, timeoutMs: route.timeout_ms || 60000 };
++ const base = { ...ctx, ...selectionCtx(selection), routeKey: route.key, routeVersion: route.version,
++ promptHash: sha256({ v: voice, t: text, f: format, s: speed }) };
++ let exec;
++ try {
++ exec = await pool.execute(route, 'speak', req, { ...base, exact: ctx.exact || exact });
++ } catch (err) {
++ // The pinned voice has no usable offer (no key, provider off): the pool's own order answers —
++ // the deterministic stub in keyless environments — rather than failing a known voice. Any
++ // other refusal (quota, payer, an unknown voice) still fails, never silently another voice.
++ if (!err || err.code !== 'ai.model_unavailable') throw err;
++ exec = await pool.execute(route, 'speak', req, base);
++ }
++ const audio = exec.result.audio;
++ if (!audio || !audio.length) throw new AiError(502, 'output.invalid', 'the provider answered, but with no audio');
++ return { output: { audio: Buffer.isBuffer(audio) ? audio.toString('base64') : String(audio),
++ mime_type: exec.result.mimeType || 'audio/mpeg', voice, characters: text.length }, exec, route };
++ }
++
/** research: no model call. A developer app's run leaves the paid web search out (its DeepSeek calls are ours). */
async function researchStep(step, input, wf, run, ctx) {
let output, answered = null, engineIndex = 0, candidates = [];
@@@ -582,7 +618,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
return { output, exec: { ...last.exec, usage, cost, savedUsd }, route: last.route };
}
-- const STEP = { devblog: devblogStep, llm: llmStep, passthrough: passthroughStep, complete: completeStep, messages: messagesStep, transcribe: transcribeStep, embed: embedStep, image: imageStep, picture: pictures.step, media_analysis: mediaAnalysis.step, research: researchStep };
++ const STEP = { devblog: devblogStep, llm: llmStep, passthrough: passthroughStep, complete: completeStep, messages: messagesStep, transcribe: transcribeStep, embed: embedStep, image: imageStep, speak: speakStep, picture: pictures.step, media_analysis: mediaAnalysis.step, research: researchStep };
/**
* Run a workflow. ctx: { signal, logRequest, debugRaw, inputHash }
@@@ -590,7 +626,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
*/
async function execute(wf, run, input, ctx) {
let last = null;
-- const usage = { input: 0, output: 0, cached: 0 };
++ const usage = { input: 0, output: 0, cached: 0, characters: 0 };
let cost = 0;
// BA1: what the wait earned, over every call the run made (a run may call a provider more than once).
let saved = 0;
@@@ -617,6 +653,7 @@ function createEngine({ registry, pool, fetcher, quotas = null, config = {}, spa
usage.input += exec.usage.input || 0;
usage.output += exec.usage.output || 0;
usage.cached += exec.usage.cached || 0;
++ usage.characters += exec.usage.characters || 0;
cost += exec.cost || 0;
saved += Number(exec.savedUsd) || 0;
}
ddiff --git a/server/workflows/seed.js b/server/workflows/seed.js
iindex 1d85373..c34df40 100644
---- a/server/workflows/seed.js
++++ b/server/workflows/seed.js
@@@ -91,11 +91,33 @@ function sharedProviderRecord(config, env) {
};
}
++function speechProviderRecords(config, env) {
++ const out = [];
++ const gkey = config.speech ? resolveSecret(config.speech.googleKeyRef, env) : '';
++ out.push({ key: 'google-tts', display_name: 'Google Cloud Text-to-Speech', kind: 'google-tts',
++ status: gkey ? 'active' : 'disabled',
++ base_url: (config.speech && config.speech.googleBaseUrl) || 'https://texttospeech.googleapis.com',
++ auth_mode: 'bearer', secret_ref: config.speech ? config.speech.googleKeyRef : 'env:AI_GOOGLE_TTS_API_KEY',
++ default_model: 'gc_smooth_operator', capabilities: ['speak'],
++ timeout_ms: (config.speech && config.speech.timeoutMs) || 60000, priority: 20, provider_access: 'OWN_DEPLOYMENT',
++ metadata: { configured_by: 'AI_GOOGLE_TTS_API_KEY', provider_name: 'google-tts', billing_profile: 'metered' } });
++ const pkey = config.speech ? resolveSecret(config.speech.pollyKeyRef, env) : '';
++ out.push({ key: 'polly', display_name: 'Amazon Polly', kind: 'polly',
++ status: pkey ? 'active' : 'disabled',
++ base_url: (config.speech && config.speech.pollyBaseUrl) || 'https://polly.amazonaws.com',
++ auth_mode: 'bearer', secret_ref: config.speech ? config.speech.pollyKeyRef : 'env:AI_POLLY_ACCESS_KEY_ID',
++ default_model: 'pl_joanna_std', capabilities: ['speak'],
++ timeout_ms: (config.speech && config.speech.timeoutMs) || 60000, priority: 20, provider_access: 'OWN_DEPLOYMENT',
++ metadata: { configured_by: 'AI_POLLY_ACCESS_KEY_ID', provider_name: 'polly', billing_profile: 'metered' } });
++ return out;
++}
++
function providerRecords(config, env) {
const out = [
-- { key: 'stub', display_name: 'Deterministic stub (synthetic output)', kind: 'stub', status: 'active', auth_mode: 'none', default_model: 'stub-1', capabilities: ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed', 'json', 'vision', 'transcribe'], timeout_ms: 10000, priority: 1000, provider_access: 'OWN_DEPLOYMENT', metadata: { synthetic: true, billing_profile: 'free' } },
++ { key: 'stub', display_name: 'Deterministic stub (synthetic output)', kind: 'stub', status: 'active', auth_mode: 'none', default_model: 'stub-1', capabilities: ['chat', 'generate', 'summarize', 'classify', 'extract', 'enrich', 'embed', 'json', 'vision', 'transcribe', 'speak', 'speech'], timeout_ms: 10000, priority: 1000, provider_access: 'OWN_DEPLOYMENT', metadata: { synthetic: true, billing_profile: 'free' } },
sharedProviderRecord(config, env),
deepseekProviderRecord(config, env),
++ ...speechProviderRecords(config, env),
{ key: 'whisper', display_name: 'whisper.cpp (local speech-to-text)', kind: 'whisper', status: 'active', auth_mode: 'none', default_model: 'whisper.cpp', capabilities: ['transcribe'], timeout_ms: 600000, priority: 20, provider_access: 'OWN_DEPLOYMENT', metadata: { configured_by: 'WHISPER_*', billing_profile: 'free' } },
];
// A local model server (llama.cpp, Ollama) speaking the OpenAI API: no key, never paid (WS-O task 5).
@@@ -170,6 +192,27 @@ async function seed({ registry, quotas, config, env = process.env, db }) {
}
}
if (!await registry.getModel('stub', 'stub-1')) await registry.upsertModel({ provider_key: 'stub', model_key: 'stub-1', type: 'chat', supports: { json: true, vision: true }, metadata: { synthetic: true } }, { actor: 'seed' });
++ // Speech voices (the Exchange speech route): one model per Chat cloud voice id on its provider, priced per
++ // million characters at the provider's own list price (docs/provider-adapters.md#speech). The rate cards
++ // price `in_per_mtok` as dollars per million characters for these models; `unit` in the metadata says so.
++ try {
++ const { VOICES, googleTierOf } = require('../providers/speech');
++ for (const [id, v] of Object.entries(VOICES)) {
++ const prev = await registry.getModel(v.provider, id);
++ if (prev && !(prev.metadata && prev.metadata.configured_by === 'AI_SPEECH_SEED')) continue;
++ const source = v.provider === 'google-tts' ? 'https://cloud.google.com/text-to-speech/pricing' : 'https://aws.amazon.com/polly/pricing/';
++ await registry.upsertModel({ provider_key: v.provider, model_key: id,
++ display_name: v.voiceId ? `${v.voiceId} (${v.engine})` : v.voice,
++ type: 'speech', status: 'active', context_window: null, max_output: null,
++ cost: { in_per_mtok: v.price_per_million_chars_usd, out_per_mtok: 0, cached_per_mtok: null,
++ effective_from: '2026-11-10', source, verified_at: '2026-11-10' },
++ supports: { json: false, tools: false, streaming: false, vision: false },
++ metadata: { configured_by: 'AI_SPEECH_SEED', unit: 'characters', price_per_million_chars_usd: v.price_per_million_chars_usd,
++ cloud_voice: v.voice || v.voiceId, cloud_engine: v.engine || googleTierOf(v.voice),
++ ...(v.voice && /Journey/i.test(v.voice) ? { pricing_note: 'Journey has no row on the current pricing page; priced at the Neural2 SKU until Google publishes one.' } : {}) },
++ }, { actor: 'seed' });
++ }
++ } catch (err) { console.warn(`[seed] speech voices were not seeded: ${String((err && err.message) || err).slice(0, 160)}`); }
if (!await registry.getModel('whisper', 'whisper.cpp')) await registry.upsertModel({ provider_key: 'whisper', model_key: 'whisper.cpp', type: 'stt', supports: { json: false } }, { actor: 'seed' });
const embeddingModel = env.AI_EMBEDDING_MODEL || 'text-embedding-3-small';
if (!await registry.getModel('shared', embeddingModel)) await registry.upsertModel({ provider_key: 'shared', model_key: embeddingModel, type: 'embedding' }, { actor: 'seed' });
@@@ -213,6 +256,9 @@ async function seed({ registry, quotas, config, env = process.env, db }) {
// Meme captions need fluency, not factual precision; choose the cheapest eligible JSON route by default.
await registry.seedVersioned('route', 'meme.caption', pool(30000, { constraints: { objective: 'cheapest', latency_class: 'interactive' }, options: { temperature: 0.85 }, max_output_tokens: 700, response_format: 'json' }));
await registry.seedVersioned('route', 'default.embedding', pool(30000, { 'capability': 'embed', constraints: { objective: 'balanced' } }));
++ // Speech (the Exchange speech route): the voice pins the exact offer, so the pool is only the fallback
++ // order when no voice is pinned; every provider that declares `speak` may answer it.
++ await registry.seedVersioned('route', 'default.speech', pool(60000, { 'capability': 'speak', constraints: { objective: 'balanced' } }));
// media.analyze (WS-O task 5): the local model when there is one; the paid route is used only under a media.paid budget.
// Both are background work (BA1): nobody watches a VOD's overview arrive, so the route says it may wait.
// The run's own route (media.analysis) exists whatever the host has: it carries the class of the work, and the
ddiff --git a/test/exchange-speech.test.js b/test/exchange-speech.test.js
nnew file mode 100644
iindex 0000000..10c8c94
---- /dev/null
++++ b/test/exchange-speech.test.js
@@@ -0,0 +1,158 @@
++'use strict';
++// The Exchange speech route: POST /v1/audio/speech speaks Chat's cloud voices through AI's own
++// Google Cloud TTS and Amazon Polly providers, metered per character like the providers bill.
++//
++// - a Google voice and a Polly voice answer audio bytes against local fake provider hosts, with the
++// run and receipt ids in the headers; the other provider is never called for that voice;
++// - metering: the receipt's cost is characters / 1M × the voice's list price, and the receipt carries
++// the character count;
++// - an unknown voice is refused before any provider is called;
++// - an over-quota call is refused before any provider is called;
++// - the full receipt is at GET /v1/receipts/:id.
++//
++// No test reaches the internet: the providers' own hosts are replaced by local fakes through
++// AI_GOOGLE_TTS_BASE_URL / AI_POLLY_BASE_URL. The keys are dummies.
++const assert = require('assert');
++const http = require('http');
++const { boot, request, token, suite, ALL } = require('./helpers');
++
++const t = suite('exchange-speech');
++const service = token('exchange', ALL);
++const FAKE_MP3 = Buffer.from('ID3\x04fake-mp3-bytes');
++
++let h;
++let google;
++let polly;
++
++async function fakeGoogle() {
++ const state = { calls: [], bodies: [] };
++ const server = http.createServer((req, res) => {
++ let raw = '';
++ req.on('data', (c) => { raw += c; });
++ req.on('end', () => {
++ state.calls.push(req.url);
++ try { state.bodies.push(JSON.parse(raw || '{}')); } catch { state.bodies.push({}); }
++ res.writeHead(200, { 'Content-Type': 'application/json' });
++ res.end(JSON.stringify({ audioContent: FAKE_MP3.toString('base64') }));
++ });
++ });
++ await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve));
++ state.url = `http://127.0.0.1:${server.address().port}`;
++ state.close = () => new Promise((resolve) => { server.closeAllConnections?.(); server.close(() => resolve()); });
++ return state;
++}
++
++async function fakePolly() {
++ const state = { calls: [], bodies: [] };
++ const server = http.createServer((req, res) => {
++ const chunks = [];
++ req.on('data', (c) => { chunks.push(c); });
++ req.on('end', () => {
++ state.calls.push(req.url);
++ try { state.bodies.push(JSON.parse(Buffer.concat(chunks).toString('utf8') || '{}')); } catch { state.bodies.push({}); }
++ res.writeHead(200, { 'Content-Type': 'audio/mpeg' });
++ res.end(FAKE_MP3);
++ });
++ });
++ await new Promise((resolve) => server.listen(0, '127.0.0.1', resolve));
++ state.url = `http://127.0.0.1:${server.address().port}`;
++ state.close = () => new Promise((resolve) => { server.closeAllConnections?.(); server.close(() => resolve()); });
++ return state;
++}
++
++const speak = (body) => request(h.base, 'POST', '/v1/audio/speech', { tok: service, body });
++
++t.test('boot with fake Google and Polly hosts', async () => {
++ google = await fakeGoogle();
++ polly = await fakePolly();
++ h = await boot({ env: {
++ AI_STUB_FALLBACK: 'false',
++ AI_GOOGLE_TTS_API_KEY: 'test-google-key',
++ AI_GOOGLE_TTS_BASE_URL: google.url,
++ AI_POLLY_ACCESS_KEY_ID: 'test-polly-key',
++ AI_POLLY_SECRET_ACCESS_KEY: 'test-polly-secret',
++ AI_POLLY_REGION: 'us-east-1',
++ AI_POLLY_BASE_URL: polly.url,
++ } });
++ const providers = await request(h.base, 'GET', '/api/v1/providers', { tok: service });
++ assert.strictEqual(providers.status, 200, providers.text);
++ for (const key of ['google-tts', 'polly']) {
++ const p = providers.body.providers.find((x) => x.key === key);
++ assert.ok(p, `${key} is registered`);
++ assert.strictEqual(p.status, 'active', `${key} is active with its key: ${p.status}`);
++ }
++});
++
++t.test('a Google voice answers MP3 through the Google adapter only', async () => {
++ const r = await speak({ model: 'tts-1', input: 'hello world', voice: 'gc_neural_chad' });
++ assert.strictEqual(r.status, 200, r.text.slice(0, 300));
++ assert.strictEqual(r.headers.get('content-type'), 'audio/mpeg');
++ assert.ok(r.headers.get('x-openvibe-run-id'), 'the run id is in the headers');
++ assert.match(String(r.headers.get('x-openvibe-receipt-id') || ''), /^rcpt_/, 'the receipt id is in the headers');
++ assert.strictEqual(google.calls.length, 1, 'google was called once');
++ assert.strictEqual(google.bodies[0].voice.name, 'en-US-Neural2-D', 'the provider voice, not the Chat id');
++ assert.strictEqual(google.bodies[0].input.text, 'hello world');
++ assert.strictEqual(polly.calls.length, 0, 'polly was never called');
++});
++
++t.test('a Polly voice answers through the Polly adapter only', async () => {
++ const r = await speak({ model: 'tts-1', input: 'hi', voice: 'pl_ruth_neural', response_format: 'mp3' });
++ assert.strictEqual(r.status, 200, r.text.slice(0, 300));
++ assert.strictEqual(polly.calls.length, 1, 'polly was called once');
++ assert.strictEqual(polly.bodies[0].VoiceId, 'Ruth');
++ assert.strictEqual(polly.bodies[0].Engine, 'neural');
++ assert.strictEqual(google.calls.length, 1, 'google was not called again');
++});
++
++t.test('metering is per character, and the receipt carries the count and the cost', async () => {
++ const text = 'hello world'; // 11 characters at Neural2's $16 per million
++ const r = await speak({ model: 'tts-1', input: text, voice: 'gc_neural_chad' });
++ assert.strictEqual(r.status, 200, r.text.slice(0, 300));
++ const id = r.headers.get('x-openvibe-receipt-id');
++ const header = JSON.parse(r.headers.get('x-openvibe-receipt'));
++ assert.strictEqual(header.offer, 'google-tts/gc_neural_chad');
++ assert.strictEqual(header.tokens.characters, text.length);
++ const expected = (text.length / 1e6) * 16;
++ assert.ok(Math.abs(header.cost_usd - expected) < 1e-9, `cost is ${text.length} chars at $16/M: ${header.cost_usd}`);
++ const full = await request(h.base, 'GET', `/v1/receipts/${id}`, { tok: service });
++ assert.strictEqual(full.status, 200, full.text);
++ const receipt = full.body.receipt;
++ assert.strictEqual(receipt.workflow, 'ai.speak');
++ assert.deepStrictEqual([receipt.offer.provider, receipt.offer.model], ['google-tts', 'gc_neural_chad']);
++ assert.strictEqual(receipt.usage.characters, text.length);
++ assert.ok(Math.abs(receipt.cost_usd - expected) < 1e-9);
++});
++
++t.test('an unknown voice is refused before any provider is called', async () => {
++ const calls = google.calls.length + polly.calls.length;
++ const r = await speak({ model: 'tts-1', input: 'hello', voice: 'no-such-voice' });
++ assert.strictEqual(r.status, 404, r.text);
++ assert.strictEqual(r.body.error.code, 'ai.model_not_found');
++ assert.strictEqual(google.calls.length + polly.calls.length, calls, 'no provider was called');
++});
++
++t.test('validation refuses a bad format, a bad speed and an empty input', async () => {
++ for (const [body, code] of [
++ [{ model: 'tts-1', input: 'hello', voice: 'gc_neural_chad', response_format: 'flac' }, 400],
++ [{ model: 'tts-1', input: 'hello', voice: 'gc_neural_chad', speed: 9 }, 400],
++ [{ model: 'tts-1', input: ' ', voice: 'gc_neural_chad' }, 400],
++ [{ model: 'tts-1', voice: 'gc_neural_chad' }, 400],
++ ]) {
++ const r = await speak(body);
++ assert.strictEqual(r.status, code, JSON.stringify(body));
++ }
++});
++
++t.test('an over-quota call is refused before any provider is called', async () => {
++ const q = await request(h.base, 'POST', '/api/v1/quotas', { tok: service, body: { scope_type: 'service', scope_id: '*', window: 'minute', max_requests: 1 } });
++ assert.ok([200, 201].includes(q.status), q.text);
++ const calls = google.calls.length + polly.calls.length;
++ const r = await speak({ model: 'tts-1', input: 'over the cap', voice: 'gc_neural_chad' });
++ assert.strictEqual(r.status, 429, r.text);
++ assert.strictEqual(r.body.error.code, 'quota.exceeded');
++ assert.strictEqual(google.calls.length + polly.calls.length, calls, 'no provider was called');
++});
++
++t.test('shutdown', async () => { await h.stop(); await google.close(); await polly.close(); });
++
++t.run();
ddiff --git a/test/workflows.test.js b/test/workflows.test.js
iindex a85a4ba..3f17380 100644
---- a/test/workflows.test.js
++++ b/test/workflows.test.js
@@@ -41,6 +41,7 @@ const EXAMPLES = {
'ai.extract': { text: 'Order 42 ships to Berlin.', schema: { type: 'object', additionalProperties: false, required: ['order', 'city'], properties: { order: { type: 'string' }, city: { type: 'string' } } } },
'ai.enrich': { record: { name: 'Widget' }, instructions: 'normalise the name' },
'ai.embed': { input: ['one', 'two'] },
++ 'ai.speak': { voice: 'gc_neural_chad', text: 'The build finally compiles.' },
'ai.image': { prompt: 'A goose eating croutons, watercolour.', size: '1024x1024', n: 1 },
'image.picture': { mode: 'find', query: 'SQLite', media_app: 'ai' },
'wiki.generate_space': { topic: 'SQLite', max_pages: 6, sources: SRC(2) },