openvibers/openvibe.sdk Commits

0.75.0: openvibe-sdk/redact, the shared secret scrubber for memory (#103)

OP OpenVibers committed 14 days ago · 092a6a1bea · 22 files, +1252 −40

Changes

ddiff --git a/.gitleaksignore b/.gitleaksignore
nnew file mode 100644
iindex 0000000..cace637
---- /dev/null
++++ b/.gitleaksignore
@@@ -0,0 +1,13 @@
++# The shared redaction corpus holds fake secrets on purpose: every case proves that a shape is removed before an item
++# is stored. The values are obviously fake (FAKE, EXAMPLE, repeated letters) and exist only to be redacted. Each line
++# is the finding's fingerprint (file:rule:line), and only this fixture path is listed.
++test/redact-corpus.json:jwt:406
++test/redact-corpus.json:jwt:408
++test/redact-corpus.json:stripe-access-token:439
++test/redact-corpus.json:stripe-access-token:441
++test/redact-corpus.json:stripe-access-token:496
++test/redact-corpus.json:stripe-access-token:499
++test/redact-corpus.json:private-key:5
++test/redact-corpus.json:private-key:29
++test/redact-corpus.json:private-key:41
++test/redact-corpus.json:private-key:473
ddiff --git a/CHANGELOG.md b/CHANGELOG.md
iindex c8e2e00..bd437f8 100644
---- a/CHANGELOG.md
++++ b/CHANGELOG.md
@@@ -1,5 +1,17 @@
# Changelog
++## 0.75.0 (2026-10-30)
++
++- **`openvibe-sdk/redact`: one secret scrubber for the Node and AI (Memory M2).**
++ - `redact(text)` → `{ text, count }` ports OpenVibe.Node's `internal/memory/redact.go` (Node #87): the same shapes,
++ the whole-text pass with a sentinel before the per-item pass, the unterminated private key block, Luhn card
++ numbers, and the deliberately wide environment-assignment rule. `onlySecret(text)` says an item was nothing but a
++ secret, so AI refuses it.
++ - `test/redact-corpus.json` holds every case of Node's `TestRedactEveryShape`, the split and unterminated key
++ blocks, the `.env` block and the harmless texts that must not change. Node copies the file and runs the same
++ cases against its Go scrubber, so the two ends share one list and one corpus.
++ - Pure functions, no dependencies: exported for Node and the browser, with ESM and TypeScript entries.
++
All notable changes to `openvibe-sdk`. The package follows semver; while it is `0.x`, a minor
release may change an API and says so here.
ddiff --git a/README.md b/README.md
iindex 403b9d6..9ae8db6 100644
---- a/README.md
++++ b/README.md
@@@ -2,16 +2,16 @@
> Supported browser and server clients for the OpenVibe platform.
--**Status:** alpha, v0.74.0 (roadmap Wave 2; developer apps from Wave 20; Media objects v2 from Wave 4; the Tools platform API; Search, Chat, AI and Vision clients; sign-in with recovery and a client health check). Every release is tagged (see [CHANGELOG.md](CHANGELOG.md)). Tested against local stub servers and the built-in mock platform only, never against the live platform. Services pin release tags; each repository's `STATUS.json` names the one it uses.
++**Status:** alpha, v0.75.0 (roadmap Wave 2; developer apps from Wave 20; Media objects v2 from Wave 4; the Tools platform API; Search, Chat, AI and Vision clients; sign-in with recovery and a client health check). Every release is tagged (see [CHANGELOG.md](CHANGELOG.md)). Tested against local stub servers and the built-in mock platform only, never against the live platform. Services pin release tags; each repository's `STATUS.json` names the one it uses.
**Plan:** OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §3.2; roadmap §30.
**License:** MIT ([LICENSE](LICENSE)). This package is a library that apps outside the network embed, so it uses MIT. The OpenVibe services themselves stay AGPL-3.0.
**If a capability is not in the SDK, it is not public.** Apps call services through `openvibe-sdk` and never build internal routes themselves. A route with no SDK wrapper is internal, even when you can reach it, and it can change without notice. To make a capability public, first define it in OpenVibe.Contracts, then wrap it here.
--After v0.74.0 is tagged:
++After v0.75.0 is tagged:
```bash
--npm install https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.74.0
++npm install https://codeload.github.com/OpenVibers/OpenVibe.SDK/tar.gz/refs/tags/v0.75.0
```
It has no runtime dependencies. It needs Node ≥ 20, or any browser with `fetch`, Web Crypto and `TextDecoder`. There is no build step. The package is CommonJS with ESM entry points (`import` works). Each subpath has its own `.d.ts`. For a page with no bundler, `browser/openvibe-sdk.mjs` is one self-contained ES module of the browser-safe subpaths (see [Browser without a bundler](#browser-without-a-bundler)). It does not depend on `openvibe-contracts` at runtime: it copies the contract types it uses (from Contracts v0.86.0, a devDependency the tests check them against).
@@@ -203,6 +203,7 @@ Until OpenVibe.Tools serves the gateway facade (Tools S6), point the jobs client
| `openvibe-sdk/trade` | server | `createTradeClient(client)`: OpenVibe.Trade. `instruments.resolve(q)/list/get/observations(symbol, {metric, before, limit})/documents`, `sources()`; the person's `watchlists.list/get/create/rename/add/removeItem/remove` and `alerts.list/deliveries/create/remove`. Public capabilities since openvibe-contracts 0.208.0. Audience `openvibe.trade` |
| `openvibe-sdk/space` | server | `createSpaceClient(client)`: OpenVibe.Space, static sites for apps and CI. `deploys.uploadArchive(site, tarGz, {activate, preview, root})`, `uploadFiles(site, [{path, data}], …)`, `uploadSourceBuild(site, archive, {ref, commit_sha})`, `list/get/log/activate/rollback(site, {deploy_id})/remove`; `projects.list/create/get/remove/quota/sites/members.set(p, principal, role)/remove`; `sites.create/get/remove`, `sites.config.get/set/reset` (`{ headers, redirects, spa }`), `sites.source.get/set/remove`; `domains.list/add/verify/remove`. An app reaches only the projects a person added it to (deployer, maintainer or owner). Public capabilities since openvibe-contracts 0.209.0. Audience `openvibe.space` |
| `openvibe-sdk/host` | server | `createHostClient(client)`: OpenVibe.Host's public hosting catalog, sent without a token. `providers.list/get(slug)`, `plans.list({kind, min_vcpu, min_ram_mb, min_disk_gb, max_price, currency, location, provider, sort, cursor})/get(id)`, `promos.list`, `offers.list({sort, currency, max_monthly, source, priced, cursor})`, `domains.list/tld('dev')/check('example.dev')`, `benchmarks.list/get`. Any other filter is refused (422). Editing the catalog is first-party |
++| `openvibe-sdk/redact` | any | The shared secret scrubber (Memory M2): `redact(text)` → `{ text, count }` removes the tool and service keys, PEM private key blocks (including one with no END line), authorization headers, bearer tokens, chat webhooks, connection-string passwords, credential-named assignments and Luhn card numbers, keeping the words around them (`password=[redacted]`); `onlySecret(text)` says an item was nothing but a secret. The same shapes and corpus as OpenVibe.Node's `internal/memory/redact.go`, so the Node and AI cannot drift |
| `openvibe-sdk/no-looks` | any | Public AI text never describes a person: `noLooks(text)` removes looks, body, clothing, hair, face, pose, age and guessed gender ("a shirtless man sits" → "the streamer sits", "two men" → "two people"; quoted speech and names stay), `noLooksTags(tags)` drops tags that name them, `hasLooks(title)` says a title should be replaced, and `RULES_VERSION` gates a one-time repair of stored rows. Every service that stores or serves public AI text runs it |
| `openvibe-sdk/front-page` | any | What may lead a front page (a latest or most viewed list, a quote strip, a feed): `frontPageOk(text)` is false for a built-in slur category (OpenVibe.Chat's, spaced out or leet-spelled included), strong profanity in English, Portuguese, Spanish, French or German, or a hate topic out of context; `containsCoreSlur(text)` is the slur check alone. Leave such an item off the front page and show the next one; it stays on its own page |
| `openvibe-sdk/news` | server | `createNewsClient(client)`: OpenVibe.News' published stories, sent without a token. `topics()`, `stories.list({topic, limit, offset})` (published and retracted, newest first) and `stories.get(id)` (sources, timeline, perspectives). Writing and the editorial reads stay first-party |
ddiff --git a/STATUS.json b/STATUS.json
iindex 7624682..17ca59d 100644
---- a/STATUS.json
++++ b/STATUS.json
@@@ -3,12 +3,13 @@
"stage": "alpha",
"code": true,
"wave": "W2",
-- "version": "0.74.0",
-- "plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), §3.2; roadmap §30",
++ "version": "0.75.0",
++ "plan": "OpenVibe End-to-End Realignment & Implementation Plan, revision 3 (20 Sep 2026), \u00a73.2; roadmap \u00a730",
"deployed": false,
"deployment": "none: a package released as git tags; services and apps pin a tag",
"contracts": "openvibe-contracts v0.222.0 (devDependency; runtime contracts range >=0.5.0 <1.0.0)",
"features": [
++ "redact (openvibe-sdk/redact): the shared secret scrubber for memory, the same shapes and corpus as OpenVibe.Node's internal/memory/redact.go (redact, onlySecret)",
"core client: retries, timeouts, problem+json errors, trace propagation",
"auth: service and app tokens, PKCE, offline verification, the Network signing keys (createNetworkKeys), revocation cutoffs",
"registry discovery and contracts version checks",
@@@ -27,7 +28,7 @@
"valkey, cache, queue, pubsub and a shared limits store on Valkey 9, each with an in-process twin for tests",
"commerce (openvibe-sdk/commerce): the shared Billing client for Tips and VIP, with intentKey and receipt",
"events: the PostgreSQL outbox (lease-claimed, several relays per table) and inbox (handler in its transaction)",
-- "usage: createUsageReporter (openvibe-sdk/usage), the shared platform.usage-sample@1 reporter — validated readings queued idempotency-keyed in an outbox and relayed to Billing with retry, never dropped, refused readings kept",
++ "usage: createUsageReporter (openvibe-sdk/usage), the shared platform.usage-sample@1 reporter \u2014 validated readings queued idempotency-keyed in an outbox and relayed to Billing with retry, never dropped, refused readings kept",
"the HTTP telemetry kit (openvibe-sdk/service): the platform.telemetry-sample@1 middleware aggregated per route|method|status_class per flush, with active-requests, p95 and event-loop-lag gauges, lifted from Network's server/telemetry.js and server/observability.js",
"the telemetry sink (openvibe-sdk/telemetry): telemetrySchema() DDL, createPgTelemetrySink() and pruneTelemetry(), a per-service PostgreSQL table every service stores platform.telemetry-sample@1 into",
"account-data (openvibe-sdk/account-data, ADR-033): telemetry_samples covered once by the SDK (exported as telemetry.json, anonymized on deletion), and uncovered() listing every table with a person column that no table map, task-keyed list or reasoned exemption covers",
ddiff --git a/browser.js b/browser.js
iindex 02f4a50..bab0a8e 100644
---- a/browser.js
++++ b/browser.js
@@@ -18,6 +18,7 @@ module.exports = {
bench: require('./src/bench'),
vision: require('./src/vision'),
themes: require('./src/themes'),
++ redact: require('./src/redact'),
chat: require('./src/chat'),
ai: require('./src/ai'),
jobs: require('./src/jobs'),
ddiff --git a/browser/openvibe-sdk.mjs b/browser/openvibe-sdk.mjs
iindex 801b368..d6a341f 100644
---- a/browser/openvibe-sdk.mjs
++++ b/browser/openvibe-sdk.mjs
@@@ -1,6 +1,6 @@
--// Generated by scripts/browser-bundle.js from browser.js (openvibe-sdk 0.74.0, MIT). Do not edit.
++// Generated by scripts/browser-bundle.js from browser.js (openvibe-sdk 0.75.0, MIT). Do not edit.
// One self-contained ES module: core at the top level plus the browser-safe subpaths as namespaces
--// (auth, registry, modules, realtime, media, community, bot, search, bench, vision, themes, chat, ai, jobs, tools, projects, geo). No server-only code: no service tokens, JWT verification, events, identity.
++// (auth, registry, modules, realtime, media, community, bot, search, bench, vision, themes, redact, chat, ai, jobs, tools, projects, geo). No server-only code: no service tokens, JWT verification, events, identity.
/* eslint-disable */
const __defs = [
// ── 0: browser.js ──────────────────────────────────────────────────
@@@ -25,12 +25,13 @@ module.exports = {
bench: __load(17),
vision: __load(18),
themes: __load(19),
-- chat: __load(20),
-- ai: __load(21),
-- jobs: __load(22),
-- tools: __load(24),
-- projects: __load(25),
-- geo: __load(26),
++ redact: __load(20),
++ chat: __load(21),
++ ai: __load(22),
++ jobs: __load(23),
++ tools: __load(25),
++ projects: __load(26),
++ geo: __load(27),
};
},
// ── 1: src/core/index.js ───────────────────────────────────────────
@@@ -45,7 +46,7 @@ const { paginate, offsetPager } = __load(7);
const semver = __load(6);
/** The package version (test/version.test.js keeps it equal to package.json). */
--const SDK_VERSION = '0.74.0';
++const SDK_VERSION = '0.75.0';
module.exports = {
createClient,
@@@ -2481,7 +2482,254 @@ function followTheme(fn, opts = {}) {
module.exports = { createThemesClient, followTheme, readTheme, applyTheme, seasonOn, onColor, DISPLAY_OPTIONS, TOKENS };
},
--// ── 20: src/chat.js ─────────────────────────────────────────────────
++// ── 20: src/redact.js ───────────────────────────────────────────────
++function (module, exports) {
++'use strict';
++/**
++ * openvibe-sdk/redact: the shared secret scrubber (Memory M2). It is the JavaScript port of OpenVibe.Node's
++ * internal/memory/redact.go (Node #87), so the Node and OpenVibe.AI run one list of shapes against one corpus
++ * (test/redact-corpus.json). The Node scrubs a memory file before it uploads it; AI scrubs every item again before it
++ * seals it, and refuses an item that was nothing but a secret.
++ *
++ * - redact(text) -> { text, count }: every secret shape is replaced with [redacted], and the count says how many were
++ * removed. The whole-text pass uses an internal sentinel first, the way the Node scrubs a file before it splits it
++ * into items; the sentinel is resolved to [redacted] before the result is returned. A text that is already redacted
++ * is left alone.
++ * - onlySecret(text): true when nothing but redacted secrets is left ("the item was only a secret").
++ *
++ * The shapes are the ones Node took from OpenVibe.Codes' harness and from gitleaks, plus the card numbers that pass
++ * the Luhn check and the deliberately wide environment-assignment rule (max_tokens: 100000 is a value too, because a
++ * credential name may be plural and a value may be a token count).
++ *
++ * Pure functions, no dependencies and no Node built-ins: safe on a server and in a browser.
++ */
++
++// A credential name ends in one of these words. A name is a run of [A-Za-z0-9_.-] ending in one, so
++// DATABASE_PASSWORD, MY_SECRET and GITHUB_TOKEN are names, and so are password and api_key. The name may end in an
++// "s" too, so max_tokens and api_keys are names.
++const SECRET_NAME_SUFFIXES = 'password|passwd|pwd|passphrase|secret|token|api[_-]?key|apikey|access[_-]?key|private[_-]?key|client[_-]?secret|auth|credential|credentials|cookie|session';
++const SECRET_NAME = `(?:${SECRET_NAME_SUFFIXES})s?`;
++
++// The bytes of the two fence markers of markdown.
++const FENCE_TICK = 0x60; // `
++const FENCE_TILDE = 0x7e; // ~
++
++/** What a secret is replaced with. */
++const REDACTED = '[redacted]';
++
++/** The sentinel the whole-text pass writes before the text is split into items (Node's secretMark). */
++const SECRET_MARK = '\x00redacted\x00';
++
++/**
++ * One shape and how to replace it. `keep` is the number of capture groups kept in front of the marker, so an
++ * assignment still reads as "password=[redacted]". `template` rebuilds the match from its groups ("$1" …) with "%s"
++ * where the marker goes.
++ */
++const SECRET_PATTERNS = [
++ // A webhook URL carries its token in the path: the whole URL goes.
++ { re: /https:\/\/hooks\.slack\.com\/services\/[^\s"']+/g },
++ { re: /https:\/\/(?:discord|discordapp)\.com\/api\/webhooks\/[^\s"']+/g },
++ // A connection string with a password: the scheme, the user and the host stay, the password goes.
++ { re: /([a-zA-Z][a-zA-Z0-9+.-]*:\/\/[^\s:/@]+:)([^\s@/]+)(@[^\s"']*)/g, keep: 1, template: '$1%s$3' },
++ // A private key block, in every form: RSA, EC, DSA, OPENSSH, PGP and ENCRYPTED.
++ { re: /-----BEGIN [A-Z0-9 ]*PRIVATE KEY[A-Z ]*-----[\s\S]*?-----END [A-Z0-9 ]*PRIVATE KEY[A-Z ]*-----/g },
++ // The keys of the tools and the services.
++ { re: /\b(?:sk|pk|rk)-[A-Za-z0-9_-]{16,}/g },
++ { re: /\bgh[pousr]_[A-Za-z0-9]{20,}/g },
++ { re: /\bgithub_pat_[A-Za-z0-9_]{20,}/g },
++ { re: /\bglpat-[0-9A-Za-z_-]{20,}/g },
++ { re: /\bnpm_[A-Za-z0-9]{36}/g },
++ { re: /\bhf_[A-Za-z0-9]{30,}/g },
++ { re: /\bAIza[0-9A-Za-z_-]{35}/g },
++ { re: /\bSK[0-9a-fA-F]{32}\b/g },
++ { re: /\bxox[baprs]-[A-Za-z0-9-]{10,}/g },
++ { re: /\bxapp-[0-9A-Za-z-]{10,}/g },
++ { re: /\b(?:sk|rk|pk)_(?:live|test)_[A-Za-z0-9]{10,}/g },
++ { re: /\bSG\.[A-Za-z0-9_-]{16,}\.[A-Za-z0-9_-]{16,}/g },
++ { re: /\b\d{8,10}:[A-Za-z0-9_-]{35}\b/g },
++ { re: /[MN][A-Za-z\d]{23,25}\.[\w-]{6}\.[\w-]{27,}/g },
++ { re: /\bhbt_[A-Za-z0-9_-]{16,}/g },
++ { re: /\bpgsk_[A-Za-z0-9_]{16,}/g },
++ { re: /\bAKIA[0-9A-Z]{16}/g },
++ { re: /\bASIA[0-9A-Z]{16}/g },
++ { re: /\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}/g },
++ // A header that carries a credential.
++ { re: /(authorization[ \t]*:[ \t]*(?:bearer|basic)[ \t]+)(\S+)/gi, keep: 1 },
++ { re: /(x-api-key[ \t]*:[ \t]*)(\S+)/gi, keep: 1 },
++ { re: /\b(bearer[ \t]+)([A-Za-z0-9._~+/=-]{20,})/gi, keep: 1 },
++ // An assignment: the name and the separator stay, the value goes. The name ends in a word that means "this is a
++ // credential". A quoted value keeps its quotes, so the text still reads as the config it was.
++ { re: new RegExp(`([A-Za-z0-9_.-]*${SECRET_NAME})(["']?[ \\t]*(?:=>|[:=])[ \\t]*)"([^"\\n]{4,})"`, 'gi'), template: '$1$2"%s"' },
++ { re: new RegExp(`([A-Za-z0-9_.-]*${SECRET_NAME})(["']?[ \\t]*(?:=>|[:=])[ \\t]*)'([^'\\n]{4,})'`, 'gi'), template: "$1$2'%s'" },
++ // A value whose closing quote was lost: it still goes, to the end of the line.
++ { re: new RegExp(`([A-Za-z0-9_.-]*${SECRET_NAME})(["']?[ \\t]*(?:=>|[:=])[ \\t]*)"([^"\\n]{4,})`, 'gi'), template: '$1$2"%s' },
++ // An unquoted value.
++ { re: new RegExp(`([A-Za-z0-9_.-]*${SECRET_NAME})(["']?[ \\t]*(?:=>|[:=])[ \\t]*)([^\\s"',;]{6,})`, 'gi'), keep: 2 },
++ // Prose: "the password is hunter2".
++ { re: /\b(password|passphrase|passcode)([ \t]+(?:is|was|=)[ \t]+)(\S{4,})/gi, keep: 2 },
++];
++
++// A run of 13 to 19 digits with optional spaces or dashes between them. Whether it is a card is decided by Luhn.
++const CARD_CANDIDATE = /[0-9](?:[ -]?[0-9]){12,18}/g;
++
++// The lines of a private key block.
++const PEM_BEGIN = /-----BEGIN [A-Z0-9 ]*PRIVATE KEY[A-Z ]*-----/;
++const PEM_END = /-----END [A-Z0-9 ]*PRIVATE KEY[A-Z ]*-----/;
++
++const isDigit = (b) => b >= '0' && b <= '9';
++
++/** A copy of a global pattern without "g": one replacement on one match (Go's ReplaceAllString on m). */
++function nonGlobal(re) {
++ return new RegExp(re.source, re.flags.replace('g', ''));
++}
++
++/** The replacement string for one match: the kept groups then the marker, or the pattern's own template. */
++function replacementFor(pattern, match, marker) {
++ if (pattern.template) {
++ // A marker with a "$" in it must not be read as a replacement reference.
++ return match.replace(nonGlobal(pattern.re), pattern.template.replace('%s', marker.replace(/\$/g, '$$$$')));
++ }
++ if (!pattern.keep) return marker;
++ const groups = nonGlobal(pattern.re).exec(match);
++ if (!groups || groups.length <= pattern.keep) return marker;
++ return groups.slice(1, pattern.keep + 1).join('') + marker;
++}
++
++/** Replaces every match of the pattern, never touching a marker that is already there. */
++function replacePattern(pattern, text, marker, state) {
++ return text.replace(pattern.re, (match) => {
++ if (match.includes(marker) || match.includes(REDACTED)) return match;
++ state.count++;
++ return replacementFor(pattern, match, marker);
++ });
++}
++
++/** The length of the run of fence characters at s[i], or 0 when the run is shorter than three. */
++function fenceRun(s, i) {
++ if (i >= s.length || (s.charCodeAt(i) !== FENCE_TICK && s.charCodeAt(i) !== FENCE_TILDE)) return 0;
++ let n = 0;
++ while (i + n < s.length && s[i + n] === s[i]) n++;
++ return n < 3 ? 0 : n;
++}
++
++/** The character and width of the fence open at pos, or [0, 0] when pos is not inside one. */
++function openFence(text, pos) {
++ let char = null;
++ let width = 0;
++ for (let i = 0; i < pos;) {
++ const n = fenceRun(text, i);
++ if (!n) { i++; continue; }
++ if (!char) { char = text[i]; width = n; } else if (text[i] === char && n >= width) { char = null; width = 0; }
++ i += n;
++ }
++ return [char, width];
++}
++
++/** The index of the fence that closes the block at pos, or the end of the text when pos is not inside a fence. */
++function fenceEnd(text, pos) {
++ const [char, width] = openFence(text, pos);
++ if (!char) return text.length;
++ for (let i = pos; i < text.length;) {
++ const n = fenceRun(text, i);
++ if (!n) { i++; continue; }
++ if (text[i] === char && n >= width) return i;
++ i += n;
++ }
++ return text.length;
++}
++
++/**
++ * A private key block that has no END line: a truncated file, or a key pasted in halves. It is redacted from its BEGIN
++ * line to the end of the fence it sits in, or to the end of the text when it is not in a fence. A block with its END
++ * line is left to the pattern list, which redacts it whole.
++ */
++function redactUnterminatedKey(text, marker, state) {
++ const blocks = [];
++ for (const m of text.matchAll(/-----BEGIN [A-Z0-9 ]*PRIVATE KEY[A-Z ]*-----/g)) blocks.push({ start: m.index, end: m.index + m[0].length });
++ // Work from the last block backwards, so the indexes of the earlier ones stay valid.
++ for (let i = blocks.length - 1; i >= 0; i--) {
++ const { start, end: after } = blocks[i];
++ if (PEM_END.test(text.slice(after))) continue; // complete: the pattern list redacts it whole
++ let end = text.length;
++ const fence = fenceEnd(text, start);
++ if (fence > start) end = fence;
++ text = text.slice(0, start) + marker + '\n' + text.slice(end);
++ state.count++;
++ }
++ return text;
++}
++
++/** The Luhn check: a run of digits that passes it is a card number, and a long order number is not. */
++function luhn(digits) {
++ let sum = 0;
++ let doubled = false;
++ for (let i = digits.length - 1; i >= 0; i--) {
++ let d = digits.charCodeAt(i) - 48;
++ if (doubled) { d *= 2; if (d > 9) d -= 9; }
++ sum += d;
++ doubled = !doubled;
++ }
++ return sum % 10 === 0;
++}
++
++/** Redacts the card numbers that pass the Luhn check. */
++function redactCards(text, marker, state) {
++ let out = '';
++ let last = 0;
++ for (const m of text.matchAll(CARD_CANDIDATE)) {
++ const start = m.index;
++ const end = start + m[0].length;
++ if (start > 0 && isDigit(text[start - 1])) continue; // part of a longer number
++ let j = end;
++ while (j < text.length && text[j] === ' ') j++;
++ if (j < text.length && isDigit(text[j])) continue; // the number continues past the match
++ let digits = '';
++ for (let i = start; i < end; i++) if (isDigit(text[i])) digits += text[i];
++ if (digits.length < 13 || digits.length > 19 || !luhn(digits)) continue;
++ out += text.slice(last, start) + marker;
++ state.count++;
++ last = end;
++ }
++ return out + text.slice(last);
++}
++
++/** The per-item pass: the shapes, then cards, with one marker. */
++function redactWith(text, marker) {
++ const state = { count: 0 };
++ let out = redactUnterminatedKey(text, marker, state);
++ for (const pattern of SECRET_PATTERNS) out = replacePattern(pattern, out, marker, state);
++ out = redactCards(out, marker, state);
++ return { text: out, count: state.count };
++}
++
++/**
++ * The whole-text pass with the sentinel, then the per-item pass. A secret that spans two items (a key block in a
++ * fence, an .env block) is only seen whole, which is why the Node scrubs a file before it splits it. The sentinel is
++ * resolved to [redacted] before the per-item pass, which catches anything only visible once the text is on its own.
++ */
++function redactDocument(text) {
++ const whole = redactWith(text, SECRET_MARK);
++ const resolved = whole.text.split(SECRET_MARK).join(REDACTED);
++ const perItem = redactWith(resolved, REDACTED);
++ return { text: perItem.text, count: whole.count + perItem.count };
++}
++
++/**
++ * Removes the secrets from a text. Returns the text and how many secrets were removed; the secrets themselves are
++ * never returned. A text that was already redacted is left alone.
++ */
++function redact(text) {
++ return redactDocument(String(text == null ? '' : text));
++}
++
++/** True when a text holds nothing but redacted secrets ("the item was only a secret"). */
++function onlySecret(text) {
++ return String(text == null ? '' : text).split(REDACTED).join('').trim() === '';
++}
++
++module.exports = { redact, onlySecret };
++},
++// ── 21: src/chat.js ─────────────────────────────────────────────────
function (module, exports) {
'use strict';
/**
@@@ -2723,7 +2971,7 @@ function createFeedPublisher(client, { feed, key, baseUrl } = {}) {
module.exports = { createChatClient, createInboxVisitor, createFeedPublisher };
},
--// ── 21: src/ai.js ───────────────────────────────────────────────────
++// ── 22: src/ai.js ───────────────────────────────────────────────────
function (module, exports) {
'use strict';
/**
@@@ -2798,7 +3046,7 @@ function createAiClient(client, defaults = {}) {
module.exports = { createAiClient, TERMINAL, OPS };
},
--// ── 22: src/jobs.js ─────────────────────────────────────────────────
++// ── 23: src/jobs.js ─────────────────────────────────────────────────
function (module, exports) {
'use strict';
/**
@@@ -2836,7 +3084,7 @@ function (module, exports) {
*/
const { OpenVibeError, isOpenVibeError } = __load(3);
const { newIdempotencyKey } = __load(5);
--const { appendFiles } = __load(23);
++const { appendFiles } = __load(24);
const { parseSSE } = __load(11);
const TERMINAL = new Set(['succeeded', 'failed', 'cancelled']);
@@@ -3000,7 +3248,7 @@ function createJobsClient(client, { baseUrl, service = 'tools', audience = 'open
module.exports = { createJobsClient, isTerminal, TERMINAL_STATES: [...TERMINAL] };
},
--// ── 23: src/core/form.js ────────────────────────────────────────────
++// ── 24: src/core/form.js ────────────────────────────────────────────
function (module, exports) {
'use strict';
/**
@@@ -3034,7 +3282,7 @@ function appendFiles(form, list, where) {
module.exports = { isBlob, isUpload, appendFiles };
},
--// ── 24: src/tools.js ────────────────────────────────────────────────
++// ── 25: src/tools.js ────────────────────────────────────────────────
function (module, exports) {
'use strict';
/**
@@@ -3068,8 +3316,8 @@ function (module, exports) {
*/
const { OpenVibeError, isOpenVibeError } = __load(3);
const { newIdempotencyKey } = __load(5);
--const { isUpload, appendFiles } = __load(23);
--const { createJobsClient } = __load(22);
++const { isUpload, appendFiles } = __load(24);
++const { createJobsClient } = __load(23);
const enc = encodeURIComponent;
const ID_RE = /^[a-z][a-z0-9-]{0,39}$/;
@@@ -3319,7 +3567,7 @@ function createToolsClient(client, { baseUrl, service = 'tools', audience = 'ope
module.exports = { createToolsClient, ToolRunError, isToolRunError };
},
--// ── 25: src/projects.js ─────────────────────────────────────────────
++// ── 26: src/projects.js ─────────────────────────────────────────────
function (module, exports) {
'use strict';
/**
@@@ -3435,7 +3683,7 @@ function createProjectsClient(client, { baseUrl } = {}) {
module.exports = { createProjectsClient };
},
--// ── 26: src/geo.js ──────────────────────────────────────────────────
++// ── 27: src/geo.js ──────────────────────────────────────────────────
function (module, exports) {
'use strict';
/**
@@@ -3515,7 +3763,7 @@ export const {
startSpan, contextFromHeaders, ulid, newEventId, newIdempotencyKey, isActingSubjectId,
satisfiesRange, compareVersions, CONTRACTS_RANGE, DEFAULT_NETWORK, SDK_VERSION, auth,
registry, modules, realtime, media, community, bot,
-- search, bench, vision, themes, chat, ai,
-- jobs, tools, projects, geo,
++ search, bench, vision, themes, redact, chat,
++ ai, jobs, tools, projects, geo,
} = sdk;
export default sdk;
ddiff --git a/docs/api-surface.json b/docs/api-surface.json
iindex c7ee270..e439bc2 100644
---- a/docs/api-surface.json
++++ b/docs/api-surface.json
@@@ -210,6 +210,12 @@
"subscribe"
]
},
++ "redact": {
++ "object": [
++ "onlySecret",
++ "redact"
++ ]
++ },
"registry": {
"object": [
"createRegistryClient"
@@@ -623,6 +629,10 @@
"parseSSE": "function",
"subscribe": "function"
},
++ "openvibe-sdk/redact": {
++ "onlySecret": "function",
++ "redact": "function"
++ },
"openvibe-sdk/registry": {
"createRegistryClient": "function"
},
ddiff --git a/docs/redact.md b/docs/redact.md
nnew file mode 100644
iindex 0000000..807fa33
---- /dev/null
++++ b/docs/redact.md
@@@ -0,0 +1,44 @@
++# openvibe-sdk/redact
++
++The shared secret scrubber for memory (Memory M2). It is the JavaScript port of OpenVibe.Node's
++`internal/memory/redact.go` (Node #87), so the Node and OpenVibe.AI run one shape list against one corpus.
++
++```js
++const { redact, onlySecret } = require('openvibe-sdk/redact');
++
++const { text, count } = redact('the token is ghp_… and the deploy works');
++// text: "the token is [redacted] and the deploy works", count: 1
++
++onlySecret('[redacted]'); // true: an item that was only a secret is refused
++onlySecret('a note about [redacted]'); // false
++```
++
++## What it removes
++
++- the keys and tokens of the tools and services: AWS, GitHub, GitLab, npm, Hugging Face, Google, Slack, Discord,
++ Telegram, Stripe, Twilio, SendGrid, OpenAI, Anthropic, JWTs, OpenVibe `hbt_` and Phoenix Grove `pgsk_` keys;
++- private key blocks in every PEM form, including a block with no END line (from its BEGIN line to the end of the
++ markdown fence it sits in, or to the end of the text);
++- authorization and `x-api-key` headers, bearer tokens, and chat webhook URLs (the whole URL, because the token is in
++ its path);
++- connection strings: the scheme, user and host stay, the password goes;
++- assignments whose name is a credential name (`DATABASE_PASSWORD=`, `api_key:`, `access_key =>`, `max_tokens:`): the
++ name and separator stay, the value goes. The rule is deliberately wide, because a value may be a token count or a
++ passphrase;
++- prose such as "the password is …";
++- card numbers that pass the Luhn check, with spaces or dashes.
++
++A text that was already redacted is left alone. The count is the number of secrets removed, one per match.
++
++## Why the whole-text pass exists
++
++The Node scrubs a whole file with a sentinel before it splits the file into items, then scrubs each item. A key block
++or an `.env` block that spans two items is only seen whole. `redact()` runs the same two passes on one string: the
++sentinel keeps the place of each secret until the text is resolved to `[redacted]`. This is also why AI runs the
++scrubber again before sealing an item: the Node is not the only writer.
++
++## Corpus
++
++`test/redact-corpus.json` is the shared list: every shape, the split and unterminated key blocks, the `.env` block and
++the harmless texts that must not change. `test/redact.test.js` runs it here; OpenVibe.Node copies the file into
++`internal/memory/testdata/` and runs the same cases against its Go `Redact`.
ddiff --git a/esm/index.mjs b/esm/index.mjs
iindex 3cfb5bd..7343a53 100644
---- a/esm/index.mjs
++++ b/esm/index.mjs
@@@ -7,11 +7,11 @@ export const {
satisfiesRange, compareVersions, CONTRACTS_RANGE, DEFAULT_NETWORK, SDK_VERSION, auth,
registry, identity, modules, events, realtime, media,
community, bot, search, bench, vision, themes,
-- chat, ai, jobs, tools, projects, resources,
-- vip, quest, live, inventory, tips, blog,
-- reviews, wiki, coupons, deals, trade, space,
-- host, news, food, actor, run, video,
-- work, zone, play, download, openre, geo,
-- usage, commerce, telemetry,
++ redact, chat, ai, jobs, tools, projects,
++ resources, vip, quest, live, inventory, tips,
++ blog, reviews, wiki, coupons, deals, trade,
++ space, host, news, food, actor, run,
++ video, work, zone, play, download, openre,
++ geo, usage, commerce, telemetry,
} = cjs;
export default cjs;
ddiff --git a/esm/redact.mjs b/esm/redact.mjs
nnew file mode 100644
iindex 0000000..96f2419
---- /dev/null
++++ b/esm/redact.mjs
@@@ -0,0 +1,7 @@
++// Generated by scripts/esm.js from src/redact.js. Do not edit.
++import cjs from '../src/redact.js';
++
++export const {
++ redact, onlySecret,
++} = cjs;
++export default cjs;
ddiff --git a/index.js b/index.js
iindex 8102d3f..52ab784 100644
---- a/index.js
++++ b/index.js
@@@ -21,6 +21,7 @@ module.exports = {
bench: require('./src/bench'),
vision: require('./src/vision'),
themes: require('./src/themes'),
++ redact: require('./src/redact'),
chat: require('./src/chat'),
ai: require('./src/ai'),
jobs: require('./src/jobs'),
ddiff --git a/package-lock.json b/package-lock.json
iindex e9e3742..f1ee088 100644
---- a/package-lock.json
++++ b/package-lock.json
@@@ -1,12 +1,12 @@
{
"name": "openvibe-sdk",
-- "version": "0.74.0",
++ "version": "0.75.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "openvibe-sdk",
-- "version": "0.74.0",
++ "version": "0.75.0",
"license": "MIT",
"bin": {
"openvibe-download": "bin/openvibe-download.js"
ddiff --git a/package.json b/package.json
iindex c714398..e71ad69 100644
---- a/package.json
++++ b/package.json
@@@ -1,6 +1,6 @@
{
"name": "openvibe-sdk",
-- "version": "0.74.0",
++ "version": "0.75.0",
"description": "Supported browser and server clients for the OpenVibe platform: auth (incl. developer apps), registry discovery, identity, user modules, events, realtime, media, community, Bot robots, search, chat, AI runs, Tools (registry, run API and jobs), VIP memberships and developer projects, the async PostgreSQL data layer (db) and the Valkey cache, limits store, queue and pub/sub, with deadlines, safe retries, problem errors and trace propagation.",
"main": "index.js",
"types": "types/index.d.ts",
@@@ -377,6 +377,11 @@
"import": "./esm/no-looks.mjs",
"default": "./src/no-looks.js"
},
++ "./redact": {
++ "types": "./types/redact.d.ts",
++ "import": "./esm/redact.mjs",
++ "default": "./src/redact.js"
++ },
"./vision": {
"types": "./types/vision.d.ts",
"import": "./esm/vision.mjs",
ddiff --git a/scripts/esm.js b/scripts/esm.js
iindex 7ea28b1..9ffe2d4 100644
---- a/scripts/esm.js
++++ b/scripts/esm.js
@@@ -70,6 +70,7 @@ const ENTRIES = {
'host.mjs': '../src/host.js',
'news.mjs': '../src/news.js',
'no-looks.mjs': '../src/no-looks.js',
++ 'redact.mjs': '../src/redact.js',
'vision.mjs': '../src/vision.js',
'front-page.mjs': '../src/front-page.js',
'food.mjs': '../src/food.js',
ddiff --git a/src/core/index.js b/src/core/index.js
iindex 33a4bd8..bc74330 100644
---- a/src/core/index.js
++++ b/src/core/index.js
@@@ -8,7 +8,7 @@ const { paginate, offsetPager } = require('./paginate');
const semver = require('./semver');
/** The package version (test/version.test.js keeps it equal to package.json). */
--const SDK_VERSION = '0.74.0';
++const SDK_VERSION = '0.75.0';
module.exports = {
createClient,
ddiff --git a/src/redact.js b/src/redact.js
nnew file mode 100644
iindex 0000000..6fa4440
---- /dev/null
++++ b/src/redact.js
@@@ -0,0 +1,244 @@
++'use strict';
++/**
++ * openvibe-sdk/redact: the shared secret scrubber (Memory M2). It is the JavaScript port of OpenVibe.Node's
++ * internal/memory/redact.go (Node #87), so the Node and OpenVibe.AI run one list of shapes against one corpus
++ * (test/redact-corpus.json). The Node scrubs a memory file before it uploads it; AI scrubs every item again before it
++ * seals it, and refuses an item that was nothing but a secret.
++ *
++ * - redact(text) -> { text, count }: every secret shape is replaced with [redacted], and the count says how many were
++ * removed. The whole-text pass uses an internal sentinel first, the way the Node scrubs a file before it splits it
++ * into items; the sentinel is resolved to [redacted] before the result is returned. A text that is already redacted
++ * is left alone.
++ * - onlySecret(text): true when nothing but redacted secrets is left ("the item was only a secret").
++ *
++ * The shapes are the ones Node took from OpenVibe.Codes' harness and from gitleaks, plus the card numbers that pass
++ * the Luhn check and the deliberately wide environment-assignment rule (max_tokens: 100000 is a value too, because a
++ * credential name may be plural and a value may be a token count).
++ *
++ * Pure functions, no dependencies and no Node built-ins: safe on a server and in a browser.
++ */
++
++// A credential name ends in one of these words. A name is a run of [A-Za-z0-9_.-] ending in one, so
++// DATABASE_PASSWORD, MY_SECRET and GITHUB_TOKEN are names, and so are password and api_key. The name may end in an
++// "s" too, so max_tokens and api_keys are names.
++const SECRET_NAME_SUFFIXES = 'password|passwd|pwd|passphrase|secret|token|api[_-]?key|apikey|access[_-]?key|private[_-]?key|client[_-]?secret|auth|credential|credentials|cookie|session';
++const SECRET_NAME = `(?:${SECRET_NAME_SUFFIXES})s?`;
++
++// The bytes of the two fence markers of markdown.
++const FENCE_TICK = 0x60; // `
++const FENCE_TILDE = 0x7e; // ~
++
++/** What a secret is replaced with. */
++const REDACTED = '[redacted]';
++
++/** The sentinel the whole-text pass writes before the text is split into items (Node's secretMark). */
++const SECRET_MARK = '\x00redacted\x00';
++
++/**
++ * One shape and how to replace it. `keep` is the number of capture groups kept in front of the marker, so an
++ * assignment still reads as "password=[redacted]". `template` rebuilds the match from its groups ("$1" …) with "%s"
++ * where the marker goes.
++ */
++const SECRET_PATTERNS = [
++ // A webhook URL carries its token in the path: the whole URL goes.
++ { re: /https:\/\/hooks\.slack\.com\/services\/[^\s"']+/g },
++ { re: /https:\/\/(?:discord|discordapp)\.com\/api\/webhooks\/[^\s"']+/g },
++ // A connection string with a password: the scheme, the user and the host stay, the password goes.
++ { re: /([a-zA-Z][a-zA-Z0-9+.-]*:\/\/[^\s:/@]+:)([^\s@/]+)(@[^\s"']*)/g, keep: 1, template: '$1%s$3' },
++ // A private key block, in every form: RSA, EC, DSA, OPENSSH, PGP and ENCRYPTED.
++ { re: /-----BEGIN [A-Z0-9 ]*PRIVATE KEY[A-Z ]*-----[\s\S]*?-----END [A-Z0-9 ]*PRIVATE KEY[A-Z ]*-----/g },
++ // The keys of the tools and the services.
++ { re: /\b(?:sk|pk|rk)-[A-Za-z0-9_-]{16,}/g },
++ { re: /\bgh[pousr]_[A-Za-z0-9]{20,}/g },
++ { re: /\bgithub_pat_[A-Za-z0-9_]{20,}/g },
++ { re: /\bglpat-[0-9A-Za-z_-]{20,}/g },
++ { re: /\bnpm_[A-Za-z0-9]{36}/g },
++ { re: /\bhf_[A-Za-z0-9]{30,}/g },
++ { re: /\bAIza[0-9A-Za-z_-]{35}/g },
++ { re: /\bSK[0-9a-fA-F]{32}\b/g },
++ { re: /\bxox[baprs]-[A-Za-z0-9-]{10,}/g },
++ { re: /\bxapp-[0-9A-Za-z-]{10,}/g },
++ { re: /\b(?:sk|rk|pk)_(?:live|test)_[A-Za-z0-9]{10,}/g },
++ { re: /\bSG\.[A-Za-z0-9_-]{16,}\.[A-Za-z0-9_-]{16,}/g },
++ { re: /\b\d{8,10}:[A-Za-z0-9_-]{35}\b/g },
++ { re: /[MN][A-Za-z\d]{23,25}\.[\w-]{6}\.[\w-]{27,}/g },
++ { re: /\bhbt_[A-Za-z0-9_-]{16,}/g },
++ { re: /\bpgsk_[A-Za-z0-9_]{16,}/g },
++ { re: /\bAKIA[0-9A-Z]{16}/g },
++ { re: /\bASIA[0-9A-Z]{16}/g },
++ { re: /\beyJ[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}/g },
++ // A header that carries a credential.
++ { re: /(authorization[ \t]*:[ \t]*(?:bearer|basic)[ \t]+)(\S+)/gi, keep: 1 },
++ { re: /(x-api-key[ \t]*:[ \t]*)(\S+)/gi, keep: 1 },
++ { re: /\b(bearer[ \t]+)([A-Za-z0-9._~+/=-]{20,})/gi, keep: 1 },
++ // An assignment: the name and the separator stay, the value goes. The name ends in a word that means "this is a
++ // credential". A quoted value keeps its quotes, so the text still reads as the config it was.
++ { re: new RegExp(`([A-Za-z0-9_.-]*${SECRET_NAME})(["']?[ \\t]*(?:=>|[:=])[ \\t]*)"([^"\\n]{4,})"`, 'gi'), template: '$1$2"%s"' },
++ { re: new RegExp(`([A-Za-z0-9_.-]*${SECRET_NAME})(["']?[ \\t]*(?:=>|[:=])[ \\t]*)'([^'\\n]{4,})'`, 'gi'), template: "$1$2'%s'" },
++ // A value whose closing quote was lost: it still goes, to the end of the line.
++ { re: new RegExp(`([A-Za-z0-9_.-]*${SECRET_NAME})(["']?[ \\t]*(?:=>|[:=])[ \\t]*)"([^"\\n]{4,})`, 'gi'), template: '$1$2"%s' },
++ // An unquoted value.
++ { re: new RegExp(`([A-Za-z0-9_.-]*${SECRET_NAME})(["']?[ \\t]*(?:=>|[:=])[ \\t]*)([^\\s"',;]{6,})`, 'gi'), keep: 2 },
++ // Prose: "the password is hunter2".
++ { re: /\b(password|passphrase|passcode)([ \t]+(?:is|was|=)[ \t]+)(\S{4,})/gi, keep: 2 },
++];
++
++// A run of 13 to 19 digits with optional spaces or dashes between them. Whether it is a card is decided by Luhn.
++const CARD_CANDIDATE = /[0-9](?:[ -]?[0-9]){12,18}/g;
++
++// The lines of a private key block.
++const PEM_BEGIN = /-----BEGIN [A-Z0-9 ]*PRIVATE KEY[A-Z ]*-----/;
++const PEM_END = /-----END [A-Z0-9 ]*PRIVATE KEY[A-Z ]*-----/;
++
++const isDigit = (b) => b >= '0' && b <= '9';
++
++/** A copy of a global pattern without "g": one replacement on one match (Go's ReplaceAllString on m). */
++function nonGlobal(re) {
++ return new RegExp(re.source, re.flags.replace('g', ''));
++}
++
++/** The replacement string for one match: the kept groups then the marker, or the pattern's own template. */
++function replacementFor(pattern, match, marker) {
++ if (pattern.template) {
++ // A marker with a "$" in it must not be read as a replacement reference.
++ return match.replace(nonGlobal(pattern.re), pattern.template.replace('%s', marker.replace(/\$/g, '$$$$')));
++ }
++ if (!pattern.keep) return marker;
++ const groups = nonGlobal(pattern.re).exec(match);
++ if (!groups || groups.length <= pattern.keep) return marker;
++ return groups.slice(1, pattern.keep + 1).join('') + marker;
++}
++
++/** Replaces every match of the pattern, never touching a marker that is already there. */
++function replacePattern(pattern, text, marker, state) {
++ return text.replace(pattern.re, (match) => {
++ if (match.includes(marker) || match.includes(REDACTED)) return match;
++ state.count++;
++ return replacementFor(pattern, match, marker);
++ });
++}
++
++/** The length of the run of fence characters at s[i], or 0 when the run is shorter than three. */
++function fenceRun(s, i) {
++ if (i >= s.length || (s.charCodeAt(i) !== FENCE_TICK && s.charCodeAt(i) !== FENCE_TILDE)) return 0;
++ let n = 0;
++ while (i + n < s.length && s[i + n] === s[i]) n++;
++ return n < 3 ? 0 : n;
++}
++
++/** The character and width of the fence open at pos, or [0, 0] when pos is not inside one. */
++function openFence(text, pos) {
++ let char = null;
++ let width = 0;
++ for (let i = 0; i < pos;) {
++ const n = fenceRun(text, i);
++ if (!n) { i++; continue; }
++ if (!char) { char = text[i]; width = n; } else if (text[i] === char && n >= width) { char = null; width = 0; }
++ i += n;
++ }
++ return [char, width];
++}
++
++/** The index of the fence that closes the block at pos, or the end of the text when pos is not inside a fence. */
++function fenceEnd(text, pos) {
++ const [char, width] = openFence(text, pos);
++ if (!char) return text.length;
++ for (let i = pos; i < text.length;) {
++ const n = fenceRun(text, i);
++ if (!n) { i++; continue; }
++ if (text[i] === char && n >= width) return i;
++ i += n;
++ }
++ return text.length;
++}
++
++/**
++ * A private key block that has no END line: a truncated file, or a key pasted in halves. It is redacted from its BEGIN
++ * line to the end of the fence it sits in, or to the end of the text when it is not in a fence. A block with its END
++ * line is left to the pattern list, which redacts it whole.
++ */
++function redactUnterminatedKey(text, marker, state) {
++ const blocks = [];
++ for (const m of text.matchAll(/-----BEGIN [A-Z0-9 ]*PRIVATE KEY[A-Z ]*-----/g)) blocks.push({ start: m.index, end: m.index + m[0].length });
++ // Work from the last block backwards, so the indexes of the earlier ones stay valid.
++ for (let i = blocks.length - 1; i >= 0; i--) {
++ const { start, end: after } = blocks[i];
++ if (PEM_END.test(text.slice(after))) continue; // complete: the pattern list redacts it whole
++ let end = text.length;
++ const fence = fenceEnd(text, start);
++ if (fence > start) end = fence;
++ text = text.slice(0, start) + marker + '\n' + text.slice(end);
++ state.count++;
++ }
++ return text;
++}
++
++/** The Luhn check: a run of digits that passes it is a card number, and a long order number is not. */
++function luhn(digits) {
++ let sum = 0;
++ let doubled = false;
++ for (let i = digits.length - 1; i >= 0; i--) {
++ let d = digits.charCodeAt(i) - 48;
++ if (doubled) { d *= 2; if (d > 9) d -= 9; }
++ sum += d;
++ doubled = !doubled;
++ }
++ return sum % 10 === 0;
++}
++
++/** Redacts the card numbers that pass the Luhn check. */
++function redactCards(text, marker, state) {
++ let out = '';
++ let last = 0;
++ for (const m of text.matchAll(CARD_CANDIDATE)) {
++ const start = m.index;
++ const end = start + m[0].length;
++ if (start > 0 && isDigit(text[start - 1])) continue; // part of a longer number
++ let j = end;
++ while (j < text.length && text[j] === ' ') j++;
++ if (j < text.length && isDigit(text[j])) continue; // the number continues past the match
++ let digits = '';
++ for (let i = start; i < end; i++) if (isDigit(text[i])) digits += text[i];
++ if (digits.length < 13 || digits.length > 19 || !luhn(digits)) continue;
++ out += text.slice(last, start) + marker;
++ state.count++;
++ last = end;
++ }
++ return out + text.slice(last);
++}
++
++/** The per-item pass: the shapes, then cards, with one marker. */
++function redactWith(text, marker) {
++ const state = { count: 0 };
++ let out = redactUnterminatedKey(text, marker, state);
++ for (const pattern of SECRET_PATTERNS) out = replacePattern(pattern, out, marker, state);
++ out = redactCards(out, marker, state);
++ return { text: out, count: state.count };
++}
++
++/**
++ * The whole-text pass with the sentinel, then the per-item pass. A secret that spans two items (a key block in a
++ * fence, an .env block) is only seen whole, which is why the Node scrubs a file before it splits it. The sentinel is
++ * resolved to [redacted] before the per-item pass, which catches anything only visible once the text is on its own.
++ */
++function redactDocument(text) {
++ const whole = redactWith(text, SECRET_MARK);
++ const resolved = whole.text.split(SECRET_MARK).join(REDACTED);
++ const perItem = redactWith(resolved, REDACTED);
++ return { text: perItem.text, count: whole.count + perItem.count };
++}
++
++/**
++ * Removes the secrets from a text. Returns the text and how many secrets were removed; the secrets themselves are
++ * never returned. A text that was already redacted is left alone.
++ */
++function redact(text) {
++ return redactDocument(String(text == null ? '' : text));
++}
++
++/** True when a text holds nothing but redacted secrets ("the item was only a secret"). */
++function onlySecret(text) {
++ return String(text == null ? '' : text).split(REDACTED).join('').trim() === '';
++}
++
++module.exports = { redact, onlySecret };
ddiff --git a/test/pack.test.js b/test/pack.test.js
iindex 0c768c3..1d8ed7c 100644
---- a/test/pack.test.js
++++ b/test/pack.test.js
@@@ -24,7 +24,7 @@ run([
const out = execFileSync(npm, ['pack', '--json', '--pack-destination', tmp], { cwd: ROOT, env, encoding: 'utf8', shell });
const info = JSON.parse(out)[0];
const files = info.files.map((f) => f.path);
-- for (const must of ['package.json', 'index.js', 'browser.js', 'LICENSE', 'README.md', 'CHANGELOG.md', 'src/core/client.js', 'src/jobs.js', 'src/tools.js', 'src/core/form.js', 'src/projects.js', 'src/vip.js', 'esm/vip.mjs', 'types/vip.d.ts', 'src/bot.js', 'esm/bot.mjs', 'types/bot.d.ts', 'src/frame.js', 'esm/frame.mjs', 'types/frame.d.ts', 'src/service/index.js', 'esm/service.mjs', 'types/service.d.ts', 'docs/service.md', 'esm/core.mjs', 'esm/jobs.mjs', 'esm/tools.mjs', 'types/tools.d.ts', 'types/core.d.ts', 'types/contracts.d.ts', 'types/bundle.d.ts', 'src/resources.js', 'esm/resources.mjs', 'types/resources.d.ts', 'src/vision.js', 'esm/vision.mjs', 'types/vision.d.ts', 'browser/openvibe-sdk.mjs']) {
++ for (const must of ['package.json', 'index.js', 'browser.js', 'LICENSE', 'README.md', 'CHANGELOG.md', 'src/core/client.js', 'src/jobs.js', 'src/tools.js', 'src/core/form.js', 'src/projects.js', 'src/vip.js', 'esm/vip.mjs', 'types/vip.d.ts', 'src/bot.js', 'esm/bot.mjs', 'types/bot.d.ts', 'src/frame.js', 'esm/frame.mjs', 'types/frame.d.ts', 'src/service/index.js', 'esm/service.mjs', 'types/service.d.ts', 'docs/service.md', 'esm/core.mjs', 'esm/jobs.mjs', 'esm/tools.mjs', 'types/tools.d.ts', 'types/core.d.ts', 'types/contracts.d.ts', 'types/bundle.d.ts', 'src/resources.js', 'esm/resources.mjs', 'types/resources.d.ts', 'src/vision.js', 'esm/vision.mjs', 'types/vision.d.ts', 'src/redact.js', 'esm/redact.mjs', 'types/redact.d.ts', 'docs/redact.md', 'browser/openvibe-sdk.mjs']) {
assert.ok(files.includes(must), `tarball contains ${must}`);
}
assert.ok(!files.some((f) => f.startsWith('test/') || f.startsWith('scripts/') || f.startsWith('.github/')), 'no tests or tooling in the tarball');
@@@ -68,7 +68,7 @@ run([
})().catch((err) => { console.error(err); process.exit(1); });`;
const res = execFileSync(process.execPath, ['-e', script], { cwd: app, encoding: 'utf8' });
assert.match(res, new RegExp(`consumer ok: ${subpaths.length} subpaths \\+ the browser bundle`));
-- assert.equal(subpaths.length, 70);
++ assert.equal(subpaths.length, 71);
assert.ok(!fs.existsSync(path.join(app, 'node_modules/openvibe-contracts')), 'the optional peer is not installed');
} finally {
fs.rmSync(tmp, { recursive: true, force: true });
ddiff --git a/test/redact-corpus.json b/test/redact-corpus.json
nnew file mode 100644
iindex 0000000..a7e5df9
---- /dev/null
++++ b/test/redact-corpus.json
@@@ -0,0 +1,549 @@
++{
++ "cases": [
++ {
++ "name": "pem rsa",
++ "in": "the deploy key is\n-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAtESTKEYONLYFAKEFIXTURE\n-----END RSA PRIVATE KEY-----\nand the deploy works",
++ "mustNotContain": [
++ "PRIVATE KEY",
++ "MIIEowIBAAKCAQEAtESTKEYONLYFAKEFIXTURE"
++ ],
++ "mustContain": [
++ "the deploy key is",
++ "and the deploy works"
++ ]
++ },
++ {
++ "name": "pem ec",
++ "in": "the key is -----BEGIN EC PRIVATE KEY-----\nMHcCAQEEIBBBBBBBBBBBBBBBBBBBBBBBBBBBB\n-----END EC PRIVATE KEY-----\nin the vault",
++ "mustNotContain": [
++ "PRIVATE KEY",
++ "MHcCAQEEIBBBBBBBBBBBBBBBBBBBBBBBBBBBB"
++ ],
++ "mustContain": [
++ "the key is",
++ "in the vault"
++ ]
++ },
++ {
++ "name": "pem openssh",
++ "in": "the key is -----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQ\n-----END OPENSSH PRIVATE KEY-----\nin the vault",
++ "mustNotContain": [
++ "PRIVATE KEY",
++ "b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQ"
++ ],
++ "mustContain": [
++ "the key is",
++ "in the vault"
++ ]
++ },
++ {
++ "name": "pem pgp",
++ "in": "the key is -----BEGIN PGP PRIVATE KEY BLOCK-----\nlQOYBFc0OaJBCFAVvJ0lhbXVsC\n-----END PGP PRIVATE KEY BLOCK-----\nin the vault",
++ "mustNotContain": [
++ "PRIVATE KEY",
++ "lQOYBFc0OaJBCFAVvJ0lhbXVsC"
++ ],
++ "mustContain": [
++ "the key is",
++ "in the vault"
++ ]
++ },
++ {
++ "name": "pem encrypted",
++ "in": "the key is -----BEGIN ENCRYPTED PRIVATE KEY-----\nMIIEowIBAAKCAQEAtESTKEYONLYFAKEFIXTURE\n-----END ENCRYPTED PRIVATE KEY-----\nin the vault",
++ "mustNotContain": [
++ "PRIVATE KEY",
++ "MIIEowIBAAKCAQEAtESTKEYONLYFAKEFIXTURE"
++ ],
++ "mustContain": [
++ "the key is",
++ "in the vault"
++ ]
++ },
++ {
++ "name": "env assignment",
++ "in": "the line is DATABASE_PASSWORD=hunter2xyz in the file",
++ "mustNotContain": [
++ "hunter2xyz"
++ ],
++ "mustContain": [
++ "DATABASE_PASSWORD=",
++ "in the file"
++ ]
++ },
++ {
++ "name": "env assignment with a dot in the name",
++ "in": "the line is MY.SECRET=abcdefghijkl in the file",
++ "mustNotContain": [
++ "abcdefghijkl"
++ ],
++ "mustContain": [
++ "MY.SECRET=",
++ "in the file"
++ ]
++ },
++ {
++ "name": "env assignment with a colon",
++ "in": "the line is api_key: abcdefghijklmnop in the file",
++ "mustNotContain": [
++ "abcdefghijklmnop"
++ ],
++ "mustContain": [
++ "api_key:",
++ "in the file"
++ ]
++ },
++ {
++ "name": "env assignment with an arrow",
++ "in": "the line is access_key => abcdefghijklmnop in the file",
++ "mustNotContain": [
++ "abcdefghijklmnop"
++ ],
++ "mustContain": [
++ "access_key =>",
++ "in the file"
++ ]
++ },
++ {
++ "name": "env assignment quoted",
++ "in": "the line is password: \"correct horse battery staple\" in the file",
++ "mustNotContain": [
++ "correct horse battery staple"
++ ],
++ "mustContain": [
++ "password:",
++ "in the file"
++ ]
++ },
++ {
++ "name": "env assignment single quoted",
++ "in": "the line is client_secret: 'correct horse battery staple' in the file",
++ "mustNotContain": [
++ "correct horse battery staple"
++ ],
++ "mustContain": [
++ "client_secret:",
++ "in the file"
++ ]
++ },
++ {
++ "name": "env assignment with a plural name",
++ "in": "the line is max_tokens: 100000 in the file",
++ "mustNotContain": [
++ "100000"
++ ],
++ "mustContain": [
++ "max_tokens:",
++ "in the file"
++ ]
++ },
++ {
++ "name": "prose",
++ "in": "the password is hunter2xyz and it must not be shown",
++ "mustNotContain": [
++ "hunter2xyz"
++ ],
++ "mustContain": [
++ "the password is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "prose with was",
++ "in": "the passphrase was correcthorsebattery and it must not be shown",
++ "mustNotContain": [
++ "correcthorsebattery"
++ ],
++ "mustContain": [
++ "the passphrase was",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "authorization bearer",
++ "in": "the header is Authorization: Bearer abcdefghijklmnopqrstuvwxyz012345 and it must not be shown",
++ "mustNotContain": [
++ "abcdefghijklmnopqrstuvwxyz012345"
++ ],
++ "mustContain": [
++ "Authorization: Bearer",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "authorization basic",
++ "in": "the header is Authorization: Basic ZGVtbzpwYXNzd29yZA== and it must not be shown",
++ "mustNotContain": [
++ "ZGVtbzpwYXNzd29yZA=="
++ ],
++ "mustContain": [
++ "Authorization: Basic",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "x api key",
++ "in": "the header is x-api-key: abcdefghijklmnopqrst and it must not be shown",
++ "mustNotContain": [
++ "abcdefghijklmnopqrst"
++ ],
++ "mustContain": [
++ "x-api-key:",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "bare bearer",
++ "in": "the header is Bearer abcdefghijklmnopqrstuvwxyz012345 and it must not be shown",
++ "mustNotContain": [
++ "abcdefghijklmnopqrstuvwxyz012345"
++ ],
++ "mustContain": [
++ "Bearer",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "google api key",
++ "in": "the key is AIzaBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB and it must not be shown",
++ "mustNotContain": [
++ "AIzaBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB"
++ ],
++ "mustContain": [
++ "the key is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "gitlab token",
++ "in": "the token is glpat-cccccccccccccccccccc and it must not be shown",
++ "mustNotContain": [
++ "glpat-cccccccccccccccccccc"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "npm token",
++ "in": "the token is npm_dddddddddddddddddddddddddddddddddddd and it must not be shown",
++ "mustNotContain": [
++ "npm_dddddddddddddddddddddddddddddddddddd"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "hugging face token",
++ "in": "the token is hf_eeeeeeeeeeeeeeeeeeeeeeeeeeeeee and it must not be shown",
++ "mustNotContain": [
++ "hf_eeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "sendgrid key",
++ "in": "the key is SG.ffffffffffffffffffff.gggggggggggggggggggg and it must not be shown",
++ "mustNotContain": [
++ "SG.ffffffffffffffffffff.gggggggggggggggggggg"
++ ],
++ "mustContain": [
++ "the key is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "telegram bot token",
++ "in": "the token is 123456789:hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh and it must not be shown",
++ "mustNotContain": [
++ "123456789:hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "discord bot token",
++ "in": "the token is Miiiiiiiiiiiiiiiiiiiiiii.jjjjjj.kkkkkkkkkkkkkkkkkkkkkkkkkkk and it must not be shown",
++ "mustNotContain": [
++ "Miiiiiiiiiiiiiiiiiiiiiii.jjjjjj.kkkkkkkkkkkkkkkkkkkkkkkkkkk"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "twilio key",
++ "in": "the key is SK1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a and it must not be shown",
++ "mustNotContain": [
++ "SK1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a"
++ ],
++ "mustContain": [
++ "the key is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "slack app token",
++ "in": "the token is xapp-llllllllll and it must not be shown",
++ "mustNotContain": [
++ "xapp-llllllllll"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "openvibe token",
++ "in": "the token is hbt_mmmmmmmmmmmmmmmm and it must not be shown",
++ "mustNotContain": [
++ "hbt_mmmmmmmmmmmmmmmm"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "phoenix grove key",
++ "in": "the key is pgsk_nnnnnnnnnnnnnnnn and it must not be shown",
++ "mustNotContain": [
++ "pgsk_nnnnnnnnnnnnnnnn"
++ ],
++ "mustContain": [
++ "the key is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "slack webhook",
++ "in": "the hook is https://hooks.slack.com/services/T0000/B0000/XXXXXXXXXXXXXXXXXXXXXXXX and it must not be shown",
++ "mustNotContain": [
++ "https://hooks.slack.com/services/T0000/B0000/XXXXXXXXXXXXXXXXXXXXXXXX"
++ ],
++ "mustContain": [
++ "the hook is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "discord webhook",
++ "in": "the hook is https://discord.com/api/webhooks/1234/abcdefghijklmnop and it must not be shown",
++ "mustNotContain": [
++ "https://discord.com/api/webhooks/1234/abcdefghijklmnop"
++ ],
++ "mustContain": [
++ "the hook is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "card number",
++ "in": "the card is 4111111111111111 and it must not be shown",
++ "mustNotContain": [
++ "4111111111111111"
++ ],
++ "mustContain": [
++ "the card is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "card number with spaces",
++ "in": "the card is 4111 1111 1111 1111 and it must not be shown",
++ "mustNotContain": [
++ "4111 1111 1111 1111"
++ ],
++ "mustContain": [
++ "the card is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "aws key",
++ "in": "the key is AKIAIOSFODNN7EXAMPLE and it must not be shown",
++ "mustNotContain": [
++ "AKIAIOSFODNN7EXAMPLE"
++ ],
++ "mustContain": [
++ "the key is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "github token",
++ "in": "the token is ghp_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA and it must not be shown",
++ "mustNotContain": [
++ "ghp_AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "github fine grained token",
++ "in": "the token is github_pat_11AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA and it must not be shown",
++ "mustNotContain": [
++ "github_pat_11AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "jwt",
++ "in": "the token is eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dozjgNryP4J3jVmNHl0w5N_XgL0n3I9PlFUP0THsR8U and it must not be shown",
++ "mustNotContain": [
++ "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dozjgNryP4J3jVmNHl0w5N_XgL0n3I9PlFUP0THsR8U"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "openai key",
++ "in": "the key is sk-proj-A1b2C3d4E5f6G7h8I9j0K1l2M3n4O5p6Q7r8 and it must not be shown",
++ "mustNotContain": [
++ "sk-proj-A1b2C3d4E5f6G7h8I9j0K1l2M3n4O5p6Q7r8"
++ ],
++ "mustContain": [
++ "the key is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "anthropic key",
++ "in": "the key is sk-ant-api03-BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB and it must not be shown",
++ "mustNotContain": [
++ "sk-ant-api03-BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB"
++ ],
++ "mustContain": [
++ "the key is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "stripe key",
++ "in": "the key is sk_live_000000000000000FAKE and it must not be shown",
++ "mustNotContain": [
++ "sk_live_000000000000000FAKE"
++ ],
++ "mustContain": [
++ "the key is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "slack token",
++ "in": "the token is xoxb-0000000000000000FAKE and it must not be shown",
++ "mustNotContain": [
++ "xoxb-0000000000000000FAKE"
++ ],
++ "mustContain": [
++ "the token is",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "connection string",
++ "in": "the url is postgres://dbuser:[email protected]:5432/app and it must not be shown",
++ "mustNotContain": [
++ "hunter2fake"
++ ],
++ "mustContain": [
++ "postgres://dbuser:",
++ "@db.internal:5432/app",
++ "and it must not be shown"
++ ]
++ },
++ {
++ "name": "split key block",
++ "in": "the deploy key:\n\n```text\n-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAtESTKEYONLYFAKEFIXTURE\n\nMIIEowIBAAKCAQEAtESTKEYONLYFAKEFIXTURE2\n-----END RSA PRIVATE KEY-----\n```\n",
++ "mustNotContain": [
++ "PRIVATE KEY",
++ "MIIEow"
++ ],
++ "mustContain": [
++ "the deploy key:"
++ ]
++ },
++ {
++ "name": "unterminated key block",
++ "in": "the vault key:\n\n```text\n-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAtESTKEYONLYFAKEFIXTURE\n```\n\nthe notes after the block stay\n",
++ "mustNotContain": [
++ "PRIVATE KEY",
++ "MIIEow"
++ ],
++ "mustContain": [
++ "the vault key:",
++ "the notes after the block stay"
++ ]
++ },
++ {
++ "name": "env block in a fence",
++ "in": "the environment:\n\n```env\nDATABASE_PASSWORD=\"correct horse battery staple\"\nSTRIPE_SECRET=sk_live_000000000000000FAKE\nSESSION_TOKEN=hunter2xyz\n```\n",
++ "mustNotContain": [
++ "hunter2xyz",
++ "sk_live_000000000000000FAKE",
++ "correct horse battery staple"
++ ],
++ "mustContain": [
++ "DATABASE_PASSWORD=",
++ "STRIPE_SECRET=",
++ "SESSION_TOKEN="
++ ]
++ },
++ {
++ "name": "over-redaction: use 4 spaces for indentation",
++ "in": "use 4 spaces for indentation",
++ "mustNotContain": [],
++ "mustContain": [
++ "use 4 spaces for indentation"
++ ]
++ },
++ {
++ "name": "over-redaction: the build takes 12 minutes",
++ "in": "the build takes 12 minutes",
++ "mustNotContain": [],
++ "mustContain": [
++ "the build takes 12 minutes"
++ ]
++ },
++ {
++ "name": "over-redaction: 5551234567",
++ "in": "5551234567",
++ "mustNotContain": [],
++ "mustContain": [
++ "5551234567"
++ ]
++ },
++ {
++ "name": "over-redaction: the port is 4321 and the timeout is 30 seconds",
++ "in": "the port is 4321 and the timeout is 30 seconds",
++ "mustNotContain": [],
++ "mustContain": [
++ "the port is 4321 and the timeout is 30 seconds"
++ ]
++ },
++ {
++ "name": "over-redaction: the release is 1.24.0 and the build number is 1234",
++ "in": "the release is 1.24.0 and the build number is 1234",
++ "mustNotContain": [],
++ "mustContain": [
++ "the release is 1.24.0 and the build number is 1234"
++ ]
++ }
++ ]
++}
ddiff --git a/test/redact.test.js b/test/redact.test.js
nnew file mode 100644
iindex 0000000..0d90938
---- /dev/null
++++ b/test/redact.test.js
@@@ -0,0 +1,61 @@
++'use strict';
++/**
++ * openvibe-sdk/redact: the shared corpus of secret shapes (Memory M2). It is the same file OpenVibe.Node copies into
++ * its internal/memory/testdata, so the Node's Go scrubber and this port cannot drift: every case says what must be
++ * gone and what must stay.
++ */
++const assert = require('node:assert/strict');
++const fs = require('node:fs');
++const path = require('node:path');
++const { run } = require('./helpers');
++const { redact, onlySecret } = require('../src/redact');
++const corpus = JSON.parse(fs.readFileSync(path.join(__dirname, 'redact-corpus.json'), 'utf8'));
++
++run([
++ ['the corpus has every case and a stable shape', async () => {
++ assert.ok(Array.isArray(corpus.cases) && corpus.cases.length >= 50, `the corpus has ${corpus.cases.length} cases`);
++ for (const c of corpus.cases) {
++ assert.equal(typeof c.name, 'string', 'every case is named');
++ assert.equal(typeof c.in, 'string', `${c.name}: in`);
++ assert.ok(Array.isArray(c.mustNotContain), `${c.name}: mustNotContain`);
++ assert.ok(Array.isArray(c.mustContain), `${c.name}: mustContain`);
++ }
++ }],
++
++ ['every case: the secrets go, the words around them stay, and the count is right', async () => {
++ for (const c of corpus.cases) {
++ const out = redact(c.in);
++ for (const secret of c.mustNotContain) {
++ assert.ok(!out.text.includes(secret), `${c.name}: ${secret} is still there: ${out.text}`);
++ }
++ for (const keep of c.mustContain) {
++ assert.ok(out.text.includes(keep), `${c.name}: the text lost ${keep}: ${out.text}`);
++ }
++ if (c.mustNotContain.length === 0) {
++ assert.equal(out.count, 0, `${c.name}: harmless text was counted as a secret`);
++ assert.equal(out.text, c.in, `${c.name}: harmless text was changed`);
++ } else {
++ assert.ok(out.count > 0, `${c.name}: nothing was counted as redacted`);
++ assert.ok(out.text.includes('[redacted]'), `${c.name}: no marker in ${out.text}`);
++ }
++ }
++ }],
++
++ ['a second pass changes nothing and counts nothing', async () => {
++ for (const c of corpus.cases) {
++ const once = redact(c.in);
++ const twice = redact(once.text);
++ assert.equal(twice.text, once.text, `${c.name}: the second pass changed the text`);
++ assert.equal(twice.count, 0, `${c.name}: the second pass counted ${twice.count}`);
++ }
++ }],
++
++ ['onlySecret: an item that was nothing but a secret is refused, a note is not', async () => {
++ assert.equal(onlySecret('[redacted]'), true);
++ assert.equal(onlySecret(' [redacted]\n[redacted] '), true);
++ assert.equal(onlySecret(''), true);
++ assert.equal(onlySecret(redact('sk-proj-A1b2C3d4E5f6G7h8I9j0K1l2M3n4O5p6Q7r8').text), true);
++ assert.equal(onlySecret('a note about [redacted]'), false);
++ assert.equal(onlySecret('a note'), false);
++ }],
++]);
ddiff --git a/types/bundle.d.ts b/types/bundle.d.ts
iindex 5633cac..bbf8cf5 100644
---- a/types/bundle.d.ts
++++ b/types/bundle.d.ts
@@@ -10,6 +10,7 @@ import * as search from './search';
import * as bench from './bench';
import * as vision from './vision';
import * as themes from './themes';
++import * as redact from './redact';
import * as chat from './chat';
import * as ai from './ai';
import * as jobs from './jobs';
@@@ -30,6 +31,7 @@ export * as search from './search';
export * as bench from './bench';
export * as vision from './vision';
export * as themes from './themes';
++export * as redact from './redact';
export * as chat from './chat';
export * as ai from './ai';
export * as jobs from './jobs';
@@@ -39,6 +41,6 @@ export * as geo from './geo';
declare const sdk: typeof core & {
auth: typeof auth; registry: typeof registry; modules: typeof modules; realtime: typeof realtime;
-- media: typeof media; community: typeof community; bot: typeof bot; search: typeof search; bench: typeof bench; vision: typeof vision; themes: typeof themes; chat: typeof chat; ai: typeof ai; jobs: typeof jobs; tools: typeof tools; projects: typeof projects; geo: typeof geo;
++ media: typeof media; community: typeof community; bot: typeof bot; search: typeof search; bench: typeof bench; vision: typeof vision; themes: typeof themes; redact: typeof redact; chat: typeof chat; ai: typeof ai; jobs: typeof jobs; tools: typeof tools; projects: typeof projects; geo: typeof geo;
};
export default sdk;
ddiff --git a/types/index.d.ts b/types/index.d.ts
iindex 6bd50b9..47774dc 100644
---- a/types/index.d.ts
++++ b/types/index.d.ts
@@@ -12,6 +12,7 @@ export * as search from './search';
export * as bench from './bench';
export * as vision from './vision';
export * as themes from './themes';
++export * as redact from './redact';
export * as chat from './chat';
export * as ai from './ai';
export * as jobs from './jobs';
ddiff --git a/types/redact.d.ts b/types/redact.d.ts
nnew file mode 100644
iindex 0000000..110a99a
---- /dev/null
++++ b/types/redact.d.ts
@@@ -0,0 +1,11 @@
++/** A text with its secrets removed, and how many secrets were removed. */
++export interface RedactResult {
++ text: string;
++ count: number;
++}
++
++/** Remove every secret shape from a text. The secrets themselves are never returned; an already-redacted text is unchanged. */
++export function redact(text: unknown): RedactResult;
++
++/** True when a text holds nothing but redacted secrets: an item that was only a secret is refused. */
++export function onlySecret(text: unknown): boolean;