openvibers/openvibe-git Commits
Archives and checks: tarballs per ref, commit statuses, GitHub intake (#25)
Changes
@@@ -33,10 +33,12 @@ GIT_GITHUB_PUSH_TOKEN=GIT_TOKEN_TTL_HOURS=1GIT_SECRET_KEY=--# Limits (bytes). A push larger than this is refused; a repository may grow to the cap.++# Limits (bytes). A push larger than this is refused; a repository may grow to the cap; an archive++# larger than the cap is refused.GIT_MAX_PUSH_BYTES=536870912GIT_MAX_REPO_BYTES=3221225472GIT_MAX_FILE_BYTES=2097152++GIT_MAX_ARCHIVE_BYTES=268435456# Mirrors: the ten-minute poll and how long one fetch may take.GIT_MIRROR_POLL_MS=600000@@@ -52,6 +52,28 @@ is a row in the database, so a deploy does not reset the interval. The first chejob ticks hourly, in the background: `/api/ready` never waits on it, and a failed run waits for the next due time.An empty `GIT_OFFICIAL_ORG` turns the job off, and so does drill mode.++## Tarballs, pins and checks++++Every ref answers a gzip tarball: `GET /<owner>/<name>/archive/<ref>.tar.gz` (a branch may hold slashes) and++the same path under `/api/v1/repos/<owner>/<name>/`. The bytes are git archive of the resolved commit,++streamed, with the commit as the ETag; a tag or a full commit id caches immutably, a branch for a minute.++The top folder is `<name>-<commit>/`, so a tag's bytes never change and a package manager can pin them:++++```json++"openvibe-shared": "https://git.openvibe.codes/openvibers/openvibe.shared/archive/v4.186.0.tar.gz"++```++++instead of the `codeload.github.com` URL it replaces. Public repositories need no token; private ones answer++404 to anyone who may not see them, like the raw file.++++A commit carries statuses: `POST /api/v1/repos/:owner/:name/commits/:sha/statuses` with++`{context, state: pending|success|failure|error, description?, target_url?}`, reported with a typed Network++service token holding `git.checks.write`; `GET` lists the latest per context. A change shows them and is++mergeable only when its required contexts are green — a per-repository list, empty by default, set with++`POST …/required-checks {contexts}`. For a mirrored repository the checks intake copies GitHub's check-runs++for the open pull-request heads and the default branch (at most every two minutes per repository, and only++when a head moved), so the pages show the truth until our own runners report.++## Layout- `server/git/` — repositories on disk: creation, refs, trees, blobs, commits, diffs, bunches, and `http-backend`.@@@ -6,7 +6,10 @@ JSON routes below `/api/v1/repos/:owner/:name`:- `GET /compare/:base...:head`: commits (up to 200), files (up to 300), added/removed counts and unified lines with old/new numbers. Branch slashes can be percent-encoded.- `GET|POST /changes`: list or create `{title, body?, base_ref, head_ref, source_repo?}`; source is `owner/name`, same repo or its fork family.--- `GET /changes/:number`: change, diff, comments, approvals for the current SHA, checks and cached summary.++- `GET /changes/:number`: change, diff, comments, approvals for the current SHA, checks (`checks`, `checks_label`, combined `state`, per-context `required`, `mergeable`) and cached summary.++- `GET|POST /commits/:sha/statuses`: the latest status per context, or report `{context, state: pending|success|failure|error, description?, target_url?}` with a typed Network service token holding `git.checks.write`.++- `GET|POST /required-checks`: the contexts that must be green before a change merges (`{contexts: [...]}`), empty by default; merge is 409 while one is missing or not green.++- `GET /archive/<ref>.tar.gz` (page and API): a streamed gzip tarball of the ref, stable for a tag; the ETag is the commit. See README for the pin form.- `POST /changes/:number/comments`: `{head_sha,path,line,side:"old"|"new",message}`.- `POST /changes/:number/approve`: `{head_sha}`.- `POST /changes/:number/merge`: `{head_sha,method:"merge"|"squash"|"rebase"}`; conflicts are 409 with `files`.@@@ -24,7 +27,7 @@ Community uses the existing `community.comment.write` and `community.comment.modNetwork needs `network.notifications.push` and `identity.subject.resolve`. Notifications address the change author, repository owner and watchers, excluding the actor and checking current visibility before delivery. These scopes must be granted to the Git service principal before deployment; this job does not alter production grants.--AI needs `ai.run.create` and `ai.run.read`, plus the AI-choices API from the concurrent AI-choices job (Contracts #389). G2 checks `GET /api/v1/tasks` for `ai.generate` before posting a run with `{workflow:"ai.generate",choice:{model,source:"openvibe"},on_behalf_of,input}`. Older AI fails closed, without silently using a different model. Local tests use mock runs; no paid calls were made. Run currently has job APIs but no repository/head-commit check lookup: the slot says **No checks**.++AI needs `ai.run.create` and `ai.run.read`, plus the AI-choices API from the concurrent AI-choices job (Contracts #389). G2 checks `GET /api/v1/tasks` for `ai.generate` before posting a run with `{workflow:"ai.generate",choice:{model,source:"openvibe"},on_behalf_of,input}`. Older AI fails closed, without silently using a different model. Local tests use mock runs; no paid calls were made. Run currently has job APIs but no repository/head-commit check lookup; until it posts statuses, the checks intake copies GitHub's check-runs for mirrored repositories (off in drill mode, two minutes per repository, only when a head moved), and the slot says **No checks** only when nothing was reported.## Merge and maintenancennew file mode 100644@@@ -0,0 +1,34 @@++-- phase: expand++-- Commit statuses (checks): what CI says about a commit, who requires it, and what the GitHub intake copied.++--++-- A status is one CI verdict on one commit: {context, state, description, target_url}. History is kept (one row++-- per report); reads take the latest row per context. A repository names the contexts that must be green before++-- a change merges (required_checks, empty by default, so nothing changes until one is set). The intake remembers++-- the heads it already copied per repository, so it only asks GitHub about heads that moved.++++CREATE TABLE git_commit_statuses (++ id text PRIMARY KEY, -- cst_<ulid>++ repo_id text NOT NULL REFERENCES git_repos (id) ON DELETE CASCADE,++ sha text NOT NULL, -- the full commit id, lower-case hex++ context text NOT NULL, -- e.g. "test", "security/secrets"++ state text NOT NULL CHECK (state IN ('pending', 'success', 'failure', 'error')),++ description text NOT NULL DEFAULT '',++ target_url text NOT NULL DEFAULT '',++ creator_subject text, -- svc:<service> or usr_… that reported it++ created_at timestamptz NOT NULL DEFAULT now(),++ CHECK (sha ~ '^[0-9a-f]{40}([0-9a-f]{24})?$'),++ CHECK (char_length(context) BETWEEN 1 AND 255),++ CHECK (char_length(description) <= 1024),++ CHECK (char_length(target_url) <= 2048)++);++CREATE INDEX git_commit_statuses_repo_sha ON git_commit_statuses (repo_id, sha, created_at DESC);++++ALTER TABLE git_repos ADD COLUMN required_checks jsonb NOT NULL DEFAULT '[]'::jsonb;++++CREATE TABLE git_checks_sync (++ repo_id text PRIMARY KEY REFERENCES git_repos (id) ON DELETE CASCADE,++ last_run_at timestamptz,++ heads jsonb NOT NULL DEFAULT '{}'::jsonb, -- {"main": "<sha>", "pr/12": "<sha>"}++ last_result jsonb,++ updated_at timestamptz NOT NULL DEFAULT now()++);@@@ -27,6 +27,7 @@ const { createGithubClient } = require('./github');const { createInsights } = require('./insights');const { createRepos } = require('./repos');const { createMirrors } = require('./mirror');++const { createChecksIntake } = require('./checks-intake');const { createOfficialMirrors } = require('./official');const { createSmartHttp, parseGitPath } = require('./git/http-backend');const { identify } = require('./auth/git-auth');@@@ -34,6 +35,8 @@ const { createApi } = require('./http/api');const { createPageRoutes } = require('./http/pages');const { createReviewRoutes } = require('./http/review');const { createReviews } = require('./reviews');++const { createChecks } = require('./checks');++const { createArchive } = require('./archive');const { createReviewServices } = require('./review-services');const { createLinkRelay } = require('./link-relay');const { createLinkRoutes } = require('./http/link');@@@ -61,6 +64,9 @@ function createApp(opts = {}) {secureCookies: config.cookies.secure, fetch: fetchImpl, log,});const ctx = { config, store: s, s, git, github, insights, repos, mirrors, official, smartHttp, sso, log };++ ctx.checks = createChecks({ store: s, log });++ ctx.archive = createArchive({ config, git });++ ctx.checksIntake = opts.checksIntake || createChecksIntake({ store: s, checks: ctx.checks, config, git, github, log });ctx.relay = createLinkRelay(ctx);ctx.reviews = createReviews(ctx, opts.reviewServices || createReviewServices(config, fetchImpl));nnew file mode 100644@@@ -0,0 +1,110 @@++'use strict';++++/**++ * Tarballs: GET /<owner>/<name>/archive/<ref>.tar.gz (and the API form) answers a gzip tarball of that++ * ref, made by git archive from the bare repository and streamed to the reader.++ *++ * The bytes are git's own archive of the resolved commit, so a tag always answers the same bytes; the top++ * folder is <name>-<commit>/, so a package manager can pin it. The ETag is the commit; a full commit id or++ * a tag caches immutably, a branch for a minute. Reads follow the raw file's auth: public repositories++ * need no token, private ones answer 404 to anyone who may not see them.++ */++const { spawnGit } = require('./git/run');++const { validRef } = require('./git/refs');++++const SUFFIX = '.tar.gz';++const FULL_SHA_RE = /^[0-9a-f]{40}([0-9a-f]{24})?$/;++++/** The ref a /archive/<file> path names, or null. <file> is <ref>.tar.gz; a branch may hold slashes. */++function refOfArchive(file) {++ const name = String(file || '');++ if (!name.endsWith(SUFFIX) || name.length === SUFFIX.length) return null;++ const ref = name.slice(0, -SUFFIX.length);++ return validRef(ref) ? ref : null;++}++++function createArchive({ config, git }) {++ /** A ref to its commit, and whether the answer never changes (a full id or a tag). */++ async function resolve(owner, name, ref) {++ const sha = await git.resolveCommit(owner, name, ref);++ if (!sha) return null;++ const lower = ref.toLowerCase();++ const isFullId = FULL_SHA_RE.test(lower) && lower === sha.slice(0, lower.length) && lower.length >= 40;++ let isTag = false;++ if (!isFullId) {++ const tagged = await git.resolveCommit(owner, name, `refs/tags/${ref}`);++ isTag = Boolean(tagged) && tagged === sha;++ }++ return { sha, immutable: isFullId || isTag };++ }++++ /** Stream git archive to the response. Headers (ETag, caching) are the caller's. */++ function stream(res, owner, name, sha, { prefix, maxBytes, timeoutMs, log } = {}) {++ const dir = git.repoDir(owner, name);++ const cap = Math.max(1024, Number(maxBytes) || config.data.maxArchiveBytes);++ const child = spawnGit(['archive', '--format=tar.gz', `--prefix=${prefix}`, sha], { cwd: dir, timeoutMs: timeoutMs || 120_000 });++ let sent = 0;++ let failed = false;++ const kill = () => { try { child.kill('SIGKILL'); } catch { /* already gone */ } };++ res.on('close', kill);++ child.on('error', (err) => {++ failed = true;++ res.removeListener('close', kill);++ if (!res.headersSent) {++ res.status(500).type('text/plain').send('The archive could not be made.');++ } else {++ try { res.destroy(); } catch { /* gone */ }++ }++ (log || console).warn(`[OpenVibe.Git] archive ${owner}/${name}: ${String(err.message || err).slice(0, 200)}`);++ });++ child.stdout.on('data', (chunk) => {++ sent += chunk.length;++ if (sent > cap) {++ failed = true;++ kill();++ if (!res.headersSent) {++ res.status(413).type('text/plain').send('The archive is larger than this service serves.');++ } else {++ try { res.destroy(); } catch { /* gone */ }++ }++ return;++ }++ if (!res.write(chunk)) child.stdout.pause();++ });++ res.on('drain', () => child.stdout.resume());++ child.on('close', (code) => {++ res.removeListener('close', kill);++ if (failed) return;++ if (code !== 0) {++ if (!res.headersSent) res.status(500).type('text/plain').send('The archive could not be made.');++ else { try { res.destroy(); } catch { /* gone */ } }++ return;++ }++ res.end();++ });++ return child;++ }++++ return { resolve, stream };++}++++/**++ * Answer one archive request on either surface. fail(status, message) renders the surface's own error;++ * visibility was already checked by the caller, like the raw file.++ */++async function serveArchive(archive, req, res, row, file, { log, fail }) {++ const ref = refOfArchive(file);++ if (!ref) { fail(404, 'No such archive.'); return; }++ const resolved = await archive.resolve(row.owner_username, row.name, ref);++ if (!resolved) { fail(404, 'No such ref.'); return; }++ const etag = `"${resolved.sha}-tar.gz"`;++ if (req.get('if-none-match') === etag) { res.status(304).end(); return; }++ const safe = ref.replace(/[^A-Za-z0-9._-]+/g, '-').slice(0, 120);++ res.set('Content-Type', 'application/gzip');++ res.set('Content-Disposition', `attachment; filename="${row.name}-${safe}.tar.gz"`);++ res.set('ETag', etag);++ res.set('Cache-Control', resolved.immutable ? 'public, max-age=31536000, immutable' : 'public, max-age=60');++ archive.stream(res, row.owner_username, row.name, resolved.sha, { prefix: `${row.name}-${resolved.sha}/`, log });++}++++module.exports = { createArchive, refOfArchive, serveArchive, SUFFIX };@@@ -17,7 +17,7 @@ const crypto = require('node:crypto');const { createJwksClient } = require('openvibe-sdk/auth');const TOKEN_PREFIX = 'git_';--const SCOPES = Object.freeze(['git.repo.read', 'git.repo.write']);++const SCOPES = Object.freeze(['git.repo.read', 'git.repo.write', 'git.checks.write']);const USER_RE = /^usr_[0-9A-HJKMNP-TV-Z]{26}$/;const SVC_RE = /^svc:[a-z][a-z0-9-]{1,39}$/;@@@ -33,7 +33,7 @@ const newToken = () => `${TOKEN_PREFIX}${crypto.randomBytes(32).toString('base64/** Make a Git token; the raw value is returned once and never stored. */async function mintToken(store, { subject, actorType = 'user', scopes = ['git.repo.read'], hours = 1, label = null }) {const wanted = [...new Set((Array.isArray(scopes) ? scopes : [scopes]).filter((s) => SCOPES.includes(s)))];-- if (!wanted.length) throw new Error('a token needs git.repo.read or git.repo.write');++ if (!wanted.length) throw new Error('a token needs git.repo.read, git.repo.write or git.checks.write');const raw = newToken();const id = store.newId('tok');const expiresAt = new Date(store.now() + Math.max(1, Math.min(24 * 30, Number(hours) || 1)) * 3600_000).toISOString();nnew file mode 100644@@@ -0,0 +1,131 @@++'use strict';++++/**++ * Checks intake from GitHub, read-only: our pages show the truth today. For a mirrored repository it copies++ * the check-runs of the head commits of the open GitHub pull requests and of the default branch into commit++ * statuses, so a change page shows GitHub's CI until our own runners report.++ *++ * Few API calls: the default branch's head is read from our own mirror (free); the open PRs cost one call;++ * check-runs are asked only for heads that moved since the last run. A repository is visited at most once++ * every two minutes. The GitHub token is the mirror's own (config.github.token); it is sent, never logged.++ * Off in drill mode, like the mirror poll.++ */++const MIN_MS = 2 * 60_000; // at most once every two minutes per repository++const TICK_MS = 60_000; // the tick that looks for due repositories++const CREATOR = 'github-intake';++++/** A GitHub check-run → a status, or null when it names nothing. */++function mapRun(run) {++ const context = String((run && run.name) || '').trim().slice(0, 255);++ if (!context) return null;++ let state = 'pending';++ if (run.status === 'completed') {++ switch (run.conclusion) {++ case 'success': case 'neutral': case 'skipped': state = 'success'; break;++ case 'failure': state = 'failure'; break;++ default: state = 'error'; break; // cancelled, timed_out, action_required, stale, …++ }++ }++ const description = String((run.output && run.output.title) || run.conclusion || run.status || '').slice(0, 1024);++ const target = String(run.html_url || run.details_url || '').slice(0, 2048);++ return { context, state, description, target_url: /^https?:\/\//.test(target) ? target : '' };++}++++function createChecksIntake({ store, checks, config, git, github, log = console }) {++ /** Copy one mirrored repository's check-runs. force skips the two-minute throttle (tests, sync now). */++ async function syncRepo(row, { force = false } = {}) {++ if (!row || !row.mirror_url || !row.github_full_name) return { ok: false, skipped: 'not-mirrored' };++ const full = row.github_full_name;++ const sync = await store.maybe('SELECT * FROM git_checks_sync WHERE repo_id = ?', [row.id]);++ const lastAt = sync && sync.last_run_at ? Date.parse(sync.last_run_at) : 0;++ if (!force && lastAt && store.now() - lastAt < MIN_MS) return { ok: true, skipped: 'recent' };++ const stored = (sync && typeof sync.heads === 'object' && sync.heads) || (sync && typeof sync.heads === 'string' ? JSON.parse(sync.heads || '{}') : {});++ const summary = { repo: `${row.owner_username}/${row.name}`, heads: 0, fetched: 0, copied: 0, ok: true };++++ // The default branch's head from our own mirror: no API call.++ const heads = {};++ try {++ const main = await git.resolveCommit(row.owner_username, row.name, row.default_branch || 'main');++ if (main) heads.main = main;++ } catch (err) {++ log.warn(`[OpenVibe.Git] checks intake ${summary.repo}: ${String(err.message || err).slice(0, 200)}`);++ }++ let pulls = [];++ try {++ pulls = await github.openPulls(full);++ } catch (err) {++ summary.ok = false;++ summary.error = String(err.message || err).slice(0, 200);++ }++ for (const pr of pulls) heads[`pr/${pr.number}`] = pr.head_sha;++ summary.heads = Object.keys(heads).length;++++ for (const [key, sha] of Object.entries(heads)) {++ if (stored[key] === sha) continue; // only when the head moved++ let runs = [];++ try {++ runs = await github.checkRuns(full, sha);++ summary.fetched += 1;++ } catch (err) {++ summary.ok = false;++ summary.error = String(err.message || err).slice(0, 200);++ continue;++ }++ for (const run of runs) {++ const status = mapRun(run);++ if (!status) continue;++ try {++ const { changed } = await checks.recordIfChanged(row.id, sha, status, CREATOR);++ if (changed) summary.copied += 1;++ } catch (err) {++ log.warn(`[OpenVibe.Git] checks intake ${summary.repo}: ${String(err.message || err).slice(0, 200)}`);++ }++ }++ stored[key] = sha;++ }++ // Heads that are gone (a closed PR) leave the map; a head whose fetch failed keeps no entry, so++ // the next run retries it. The throttle still applies after an error.++ const next = {};++ for (const key of Object.keys(heads)) if (stored[key]) next[key] = stored[key];++ await store.execute(++ `INSERT INTO git_checks_sync (repo_id, last_run_at, heads, last_result, updated_at)++ VALUES (?, now(), ?, ?, now())++ ON CONFLICT (repo_id) DO UPDATE SET last_run_at = now(), heads = EXCLUDED.heads,++ last_result = EXCLUDED.last_result, updated_at = now()`,++ [row.id, JSON.stringify(next), JSON.stringify(summary)],++ );++ return summary;++ }++++ /** Every mirrored repository that is due, two at a time. */++ async function tick() {++ const rows = await store.many(++ `SELECT r.* FROM git_repos r LEFT JOIN git_checks_sync s ON s.repo_id = r.id++ WHERE r.mirror_url IS NOT NULL AND r.github_full_name IS NOT NULL++ ORDER BY s.last_run_at ASC NULLS FIRST LIMIT 200`,++ );++ const out = [];++ for (let i = 0; i < rows.length; i += 2) {++ const batch = await Promise.all(rows.slice(i, i + 2).map((r) => syncRepo(r).catch((err) => {++ log.warn(`[OpenVibe.Git] checks intake ${r.owner_username}/${r.name}: ${String(err.message || err).slice(0, 200)}`);++ return { ok: false, error: String(err.message || err).slice(0, 200) };++ })));++ out.push(...batch);++ }++ return out;++ }++++ let timer = null;++ return {++ syncRepo, tick, mapRun, MIN_MS, TICK_MS,++ start() {++ if (timer || config.drill) return false;++ timer = setInterval(() => { tick().catch((err) => log.warn('[OpenVibe.Git] checks intake:', err.message)); }, TICK_MS);++ timer.unref();++ return true;++ },++ stop() { if (timer) clearInterval(timer); timer = null; },++ };++}++++module.exports = { createChecksIntake, mapRun, MIN_MS, TICK_MS };nnew file mode 100644@@@ -0,0 +1,146 @@++'use strict';++++/**++ * Commit statuses: what CI says about a commit. A service with git.checks.write reports++ * {context, state, description, target_url} per commit; reads take the latest row per context.++ *++ * A change is mergeable when every required context is green. Required contexts are a per-repository list,++ * empty by default, so nothing changes until an owner sets one.++ */++const STATES = Object.freeze(['pending', 'success', 'failure', 'error']);++const SHA_RE = /^[0-9a-f]{40}([0-9a-f]{24})?$/;++// A context names a CI verdict ("test", "security/secrets"): visible characters, no control bytes.++const CONTEXT_RE = /^[^\u0000-\u001f\u007f]{1,255}$/;++++const fail = (status, code, message) => Object.assign(new Error(message), { status, code });++++function cleanSha(sha) {++ const value = String(sha || '').toLowerCase();++ if (!SHA_RE.test(value)) throw fail(422, 'status.sha_invalid', 'A status names the full commit id.');++ return value;++}++++function cleanStatus(body) {++ const context = String((body && body.context) || '').trim();++ if (!CONTEXT_RE.test(context)) throw fail(422, 'status.context_invalid', 'A context is 1–255 visible characters.');++ const state = String((body && body.state) || '');++ if (!STATES.includes(state)) throw fail(422, 'status.state_invalid', 'A state is pending, success, failure or error.');++ const description = String((body && body.description) || '');++ if (description.length > 1024) throw fail(422, 'status.description_invalid', 'A description is at most 1024 characters.');++ const targetUrl = String((body && (body.target_url || body.targetUrl)) || '');++ if (targetUrl.length > 2048) throw fail(422, 'status.target_url_invalid', 'A target URL is at most 2048 characters.');++ if (targetUrl && !/^https?:\/\/[^/\s]+\//.test(`${targetUrl}/`)) throw fail(422, 'status.target_url_invalid', 'A target URL is http(s).');++ return { context, state, description, target_url: targetUrl };++}++++function cleanRequired(value) {++ const list = Array.isArray(value) ? value : null;++ if (!list) throw fail(422, 'checks.required_invalid', 'Required checks are a list of contexts.');++ const contexts = [...new Set(list.map((c) => String(c || '').trim()).filter(Boolean))];++ if (contexts.length > 50) throw fail(422, 'checks.required_invalid', 'At most 50 required contexts.');++ for (const context of contexts) {++ if (!CONTEXT_RE.test(context)) throw fail(422, 'checks.required_invalid', 'A context is 1–255 visible characters.');++ }++ return contexts;++}++++const rowToStatus = (row) => ({++ context: row.context, state: row.state, description: row.description || '',++ target_url: row.target_url || '', created_at: row.created_at,++});++++function createChecks({ store, log = console }) {++ /** Record one verdict. Returns the stored status. */++ async function record(repoId, sha, body, creator = null) {++ const clean = cleanStatus(body);++ const id = store.newId('cst');++ await store.execute(++ 'INSERT INTO git_commit_statuses (id, repo_id, sha, context, state, description, target_url, creator_subject) VALUES (?, ?, ?, ?, ?, ?, ?, ?)',++ [id, repoId, cleanSha(sha), clean.context, clean.state, clean.description, clean.target_url, creator],++ );++ const row = await store.one('SELECT * FROM git_commit_statuses WHERE id = ?', [id]);++ return rowToStatus(row);++ }++++ /** The latest verdict per context for a commit, oldest context first. */++ async function latest(repoId, sha) {++ const rows = await store.many(++ `SELECT DISTINCT ON (context) * FROM git_commit_statuses++ WHERE repo_id = ? AND sha = ? ORDER BY context, created_at DESC, id DESC`,++ [repoId, cleanSha(sha)],++ );++ return rows.map(rowToStatus).sort((a, b) => (a.context < b.context ? -1 : 1));++ }++++ /** Insert only when the latest verdict for the context differs; the intake calls this. */++ async function recordIfChanged(repoId, sha, body, creator = null) {++ const clean = cleanStatus(body);++ const current = await store.maybe(++ 'SELECT * FROM git_commit_statuses WHERE repo_id = ? AND sha = ? AND context = ? ORDER BY created_at DESC, id DESC LIMIT 1',++ [repoId, cleanSha(sha), clean.context],++ );++ if (current && current.state === clean.state && (current.description || '') === clean.description && (current.target_url || '') === clean.target_url) {++ return { status: rowToStatus(current), changed: false };++ }++ return { status: await record(repoId, sha, clean, creator), changed: true };++ }++++ /** The combined verdict: a failure beats an error beats a pending beats success; null when there are none. */++ function combined(statuses) {++ if (!statuses.length) return null;++ if (statuses.some((s) => s.state === 'failure')) return 'failure';++ if (statuses.some((s) => s.state === 'error')) return 'error';++ if (statuses.some((s) => s.state === 'pending')) return 'pending';++ return 'success';++ }++++ /** The one line a change page shows next to the checks. */++ function label(statuses) {++ if (!statuses.length) return 'No checks';++ const failed = statuses.filter((s) => s.state === 'failure' || s.state === 'error').length;++ const pending = statuses.filter((s) => s.state === 'pending').length;++ const total = statuses.length;++ const noun = total === 1 ? 'check' : 'checks';++ if (failed) return `${failed} of ${total} ${noun} failed`;++ if (pending) return `${pending} of ${total} ${noun} pending`;++ return `${total} ${noun} passed`;++ }++++ function requiredFor(row) {++ try {++ const list = typeof row.required_checks === 'string' ? JSON.parse(row.required_checks || '[]') : (row.required_checks || []);++ return Array.isArray(list) ? list.filter((c) => typeof c === 'string') : [];++ } catch (err) {++ log.warn('[OpenVibe.Git] required checks not read:', err.message);++ return [];++ }++ }++++ async function setRequired(row, value) {++ const contexts = cleanRequired(value);++ await store.execute('UPDATE git_repos SET required_checks = ?, updated_at = now() WHERE id = ?', [JSON.stringify(contexts), row.id]);++ return contexts;++ }++++ /**++ * Whether a commit may merge: every required context green. Empty requirements merge. A fork's head is++ * read from its own repository (repoId); the required list always comes from the target.++ */++ async function mergeable(row, sha, { repoId } = {}) {++ const required = requiredFor(row);++ const checks = await latest(repoId || row.id, sha);++ const byContext = new Map(checks.map((s) => [s.context, s]));++ const missing = required.filter((c) => !byContext.has(c) || byContext.get(c).state !== 'success');++ return {++ mergeable: missing.length === 0,++ state: combined(checks),++ checks,++ checks_label: label(checks),++ required: required.map((context) => ({ context, state: byContext.has(context) ? byContext.get(context).state : null, ok: byContext.has(context) && byContext.get(context).state === 'success' })),++ };++ }++++ return { STATES, record, recordIfChanged, latest, combined, label, requiredFor, setRequired, mergeable, cleanStatus, cleanSha };++}++++module.exports = { createChecks, STATES };@@@ -40,6 +40,7 @@ function load(env = process.env) {maxPushBytes: int(env.GIT_MAX_PUSH_BYTES, 512 * 1024 * 1024),maxRepoBytes: int(env.GIT_MAX_REPO_BYTES, 3 * 1024 * 1024 * 1024),maxFileBytes: int(env.GIT_MAX_FILE_BYTES, 2 * 1024 * 1024),++ maxArchiveBytes: int(env.GIT_MAX_ARCHIVE_BYTES, 256 * 1024 * 1024),},// OpenVibe.Network: SSO (OAuth2 + PKCE), the JWKS for typed git tokens, and the account events.@@@ -112,4 +112,23 @@ async function runText(args, opts) {return r.stdout.toString('utf8');}--module.exports = { runGit, runText, configEnv, childEnv, GitError, FIXED_ENV };++/**++ * A streaming run (git archive): the caller pipes child.stdout and ends it. Same rules as runGit — an++ * argument array, no shell, the fixed environment — plus a timeout that kills the child. The caller owns++ * the child: kill it when the reader goes away.++ */++function spawnGit(args, { cwd, env, timeoutMs = 120_000 } = {}) {++ if (!Array.isArray(args) || args.some((a) => typeof a !== 'string')) {++ throw new TypeError('spawnGit: args must be an array of strings');++ }++ const child = spawn('git', args, { cwd, env: childEnv(env), shell: false, windowsHide: true });++ const timer = setTimeout(() => {++ try { child.kill('SIGKILL'); } catch { /* already gone */ }++ }, timeoutMs);++ timer.unref();++ child.on('close', () => clearTimeout(timer));++ child.on('error', () => clearTimeout(timer));++ return child;++}++++module.exports = { runGit, runText, spawnGit, configEnv, childEnv, GitError, FIXED_ENV };@@@ -2,7 +2,8 @@/*** GitHub, read only: the repository card (stars, open issues and PRs, the last CI state), the metadata a-- * mirror is created from, and the signature check on GitHub's push webhook.++ * mirror is created from, the open PR heads and check-runs the checks intake copies, and the signature check on++ * GitHub's push webhook.** Every reading is cached for ten minutes (config.github.cacheMs), so a card costs our quota once per ten* minutes, not once per view. The API token (for the OpenVibers repositories) is sent only to@@@ -224,6 +225,52 @@ function createGithubClient({ config, fetchImpl = globalThis.fetch, now = () =>return all;}++ /**++ * The open pull requests' heads: [{ number, head_sha }]. One REST call per page (a few at most); the++ * checks intake calls this, then asks about the check-runs of the heads that moved only.++ */++ async function openPulls(full) {++ if (!FULL_RE.test(String(full || ''))) return [];++ const [owner, name] = String(full).split('/');++ const out = [];++ let url = `${config.github.apiUrl}/repos/${encodeURIComponent(owner)}/${encodeURIComponent(name)}/pulls?state=open&per_page=100`;++ for (let page = 0; page < 3 && url; page += 1) {++ const res = await fetchImpl(url, { headers: headers(), signal: AbortSignal.timeout(15_000) });++ if (!res.ok) {++ const err = new Error(`GitHub ${res.status} for ${full} pulls`);++ err.status = res.status;++ throw err;++ }++ const batch = await res.json();++ if (!Array.isArray(batch)) break;++ for (const pr of batch) {++ if (pr && pr.head && /^[0-9a-f]{40}$/i.test(pr.head.sha || '')) out.push({ number: pr.number, head_sha: String(pr.head.sha).toLowerCase() });++ }++ if (batch.length < 100) break;++ // The next page stays on the API origin (nextLink refuses anything else).++ url = nextLink(res.headers && typeof res.headers.get === 'function' ? res.headers.get('link') : null);++ }++ return out;++ }++++ /** The check-runs GitHub reports for one commit (at most 100): the checks intake copies these. */++ async function checkRuns(full, sha) {++ if (!FULL_RE.test(String(full || ''))) return [];++ if (!/^[0-9a-f]{40}$/i.test(String(sha || ''))) return [];++ const [owner, name] = String(full).split('/');++ const res = await fetchImpl(++ `${config.github.apiUrl}/repos/${encodeURIComponent(owner)}/${encodeURIComponent(name)}/commits/${encodeURIComponent(String(sha).toLowerCase())}/check-runs?per_page=100`,++ { headers: headers(), signal: AbortSignal.timeout(15_000) },++ );++ if (!res.ok) {++ const err = new Error(`GitHub ${res.status} for ${full} check-runs`);++ err.status = res.status;++ throw err;++ }++ const body = await res.json();++ return Array.isArray(body && body.check_runs) ? body.check_runs : [];++ }++/** GitHub's X-Hub-Signature-256 over the raw body, constant time. */function signatureOk(rawBody, signature) {const secret = config.github.webhookSecret;@@@ -234,7 +281,7 @@ function createGithubClient({ config, fetchImpl = globalThis.fetch, now = () =>return a.length === b.length && crypto.timingSafeEqual(a, b);}-- return { parseGithubUrl, card, meta, probe, orgRepos, signatureOk, _cache: cache };++ return { parseGithubUrl, card, meta, probe, orgRepos, openPulls, checkRuns, signatureOk, _cache: cache };}module.exports = { createGithubClient, parseGithubUrl, parseRefs, pktLine, FULL_RE };@@@ -1,8 +1,8 @@'use strict';/**-- * /api/v1: repositories, browsing, stars and forks, contents writes, clone tokens, the home numbers and the-- * GitHub webhook. Everything a page shows is here too; nothing here needs a browser.++ * /api/v1: repositories, browsing, archives, stars and forks, contents writes, commit statuses, clone++ * tokens, the home numbers and the GitHub webhook. Everything a page shows is here too; nothing here needs a browser.** Public reads need no sign-in. A write needs the session or a token with git.repo.write. A private repository* answers 404 to anyone who may not see it, never 403: its existence is not public.@@@ -11,6 +11,7 @@ const express = require('express');const contracts = require('openvibe-contracts');const { callerOf, viewerFrom } = require('./identity');const { mintToken } = require('../auth/git-auth');++const { serveArchive } = require('../archive');const problem = (req, res, status, code, detail) => contracts.http.sendProblem(res, status, code, { detail, ctx: req.ov });const fail = (req, res, err) => problem(req, res, err.status || 500, err.code || 'request.failed', err.status ? err.message : undefined);@@@ -18,7 +19,7 @@ const fail = (req, res, err) => problem(req, res, err.status || 500, err.code ||function boolParam(v) { return v === true || v === '1' || v === 'true' || v === 'yes'; }function createApi(ctx) {-- const { config, repos, mirrors, git, insights, store, sso } = ctx;++ const { config, repos, mirrors, git, insights, store, sso, checks, archive, log } = ctx;const r = express.Router();const h = (fn) => (req, res, next) => Promise.resolve(fn(req, res)).catch(next);// One identity read per request: the session, or a token (a person's or a service's, and for a service the@@@ -186,6 +187,14 @@ function createApi(ctx) {return res.end(blob.data);}));++ r.get('/repos/:owner/:name/archive/*', h(async (req, res) => {++ const found = await visible(req, req.params.owner, req.params.name);++ if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');++ await serveArchive(archive, req, res, found.row, req.params[0], {++ log, fail: (status, message) => problem(req, res, status, 'archive.not_found', message),++ });++ }));++r.get('/repos/:owner/:name/commits', h(async (req, res) => {const found = await visible(req, req.params.owner, req.params.name);if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');@@@ -210,6 +219,51 @@ function createApi(ctx) {});}));++ // ── Commit statuses ───────────────────────────────────++ r.get('/repos/:owner/:name/commits/:sha/statuses', h(async (req, res) => {++ const found = await visible(req, req.params.owner, req.params.name);++ if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');++ try {++ const sha = checks.cleanSha(req.params.sha);++ const statuses = await checks.latest(found.row.id, sha);++ return res.json({ repo: `${found.row.owner_username}/${found.row.name}`, sha, state: checks.combined(statuses), statuses });++ } catch (err) { return fail(req, res, err); }++ }));++++ r.post('/repos/:owner/:name/commits/:sha/statuses', h(async (req, res) => {++ const caller = await callerFor(req);++ const found = await visible(req, req.params.owner, req.params.name);++ if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');++ if (!caller) return problem(req, res, 401, 'auth.required', 'A service token with git.checks.write reports a status.');++ if (!caller.scopes.has('git.checks.write')) return problem(req, res, 403, 'token.scope', 'This token may not report checks (git.checks.write).');++ const who = actingAs(caller);++ if (caller.actor_type !== 'service' && !repos.isOwner(found.row, who)) return problem(req, res, 403, 'repo.forbidden', 'Only the owner or a checks service may report.');++ try {++ const sha = checks.cleanSha(req.params.sha);++ const status = await checks.record(found.row.id, sha, req.body, caller.subject);++ await repos.audit(who, 'git.status', found.row, { sha, context: status.context, state: status.state, via: caller.actor_type === 'service' ? caller.subject : 'api' }, req.ip);++ return res.status(201).json(status);++ } catch (err) { return fail(req, res, err); }++ }));++++ r.get('/repos/:owner/:name/required-checks', h(async (req, res) => {++ const found = await visible(req, req.params.owner, req.params.name);++ if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');++ return res.json({ repo: `${found.row.owner_username}/${found.row.name}`, required_checks: checks.requiredFor(found.row) });++ }));++++ r.post('/repos/:owner/:name/required-checks', h(async (req, res) => {++ const caller = await callerFor(req);++ const found = await visible(req, req.params.owner, req.params.name);++ if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');++ if (!caller || !repos.isOwner(found.row, actingAs(caller))) return problem(req, res, 403, 'repo.forbidden', 'Only the owner can require checks.');++ try {++ const required = await checks.setRequired(found.row, (req.body || {}).contexts ?? (req.body || {}).required_checks);++ await repos.audit(actingAs(caller), 'git.required_checks', found.row, { required_checks: required }, req.ip);++ return res.json({ repo: `${found.row.owner_username}/${found.row.name}`, required_checks: required });++ } catch (err) { return fail(req, res, err); }++ }));++r.get('/repos/:owner/:name/branches', h(async (req, res) => {const found = await visible(req, req.params.owner, req.params.name);if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');@@@ -12,6 +12,7 @@* /:owner/:name/tree a directory (ref and path in the query)* /:owner/:name/blob a file, highlighted on the server, with line links* /:owner/:name/raw its bytes++ * /:owner/:name/archive/<ref>.tar.gz a gzip tarball of a ref, stable for a tag* /:owner/:name/commits history (optionally of one path)* /:owner/:name/commit/:sha one commit and its diff* /:owner/:name/branches · tags · releases · contributors · insights@@@ -26,9 +27,10 @@ const { viewerOf } = require('./identity');const { esc } = require('../render/html');const { renderMarkdown } = require('../render/markdown');const { validRef, validPath } = require('../git/refs');++const { serveArchive } = require('../archive');function createPageRoutes(ctx) {-- const { config, repos, git, github, mirrors, insights, store, relay } = ctx;++ const { config, repos, git, github, mirrors, insights, store, relay, archive, log } = ctx;const r = express.Router();r.use(express.urlencoded({ extended: false, limit: '256kb' }));const h = (fn) => (req, res, next) => Promise.resolve(fn(req, res)).catch(next);@@@ -330,6 +332,14 @@ function createPageRoutes(ctx) {return res.end(blob.data);}));++ r.get('/:owner/:name/archive/*', h(async (req, res) => {++ const row = await rowFor(req, res);++ if (!row) return undefined;++ await serveArchive(archive, req, res, row, req.params[0], {++ log, fail: (status, message) => res.status(status).type('text/plain').send(`${message}\n`),++ });++ }));++r.get('/:owner/:name/commits', h(async (req, res) => {const row = await rowFor(req, res);if (!row) return undefined;@@@ -452,7 +462,7 @@ function createPageRoutes(ctx) {const official = await repos.official();const lines = ['# OpenVibe Git', '', '> Git for OpenVibe: host code with your OpenVibe account, browse it, mirror GitHub, and clone it to your computer with Link.', '',-- 'Every page is server-rendered. A public repository is readable at /<owner>/<name>; its files at /<owner>/<name>/tree and /<owner>/<name>/blob (ref and path are query parameters). The API is /api/v1/repos.', '',++ 'Every page is server-rendered. A public repository is readable at /<owner>/<name>; its files at /<owner>/<name>/tree and /<owner>/<name>/blob (ref and path are query parameters); a ref as a tarball at /<owner>/<name>/archive/<ref>.tar.gz. The API is /api/v1/repos.', '','## Official repositories', '',...official.map((o) => `- [${o.full_name}](${config.baseUrl.replace(/\/+$/, '')}/${o.full_name}): ${o.description || 'The OpenVibe projects.'}`), '',];@@@ -474,6 +484,8 @@ function createPageRoutes(ctx) {'/api/v1/repos/{owner}/{name}/tree': { get: { summary: 'A directory at a ref', responses: { 200: { description: 'Tree' } } } },'/api/v1/repos/{owner}/{name}/blob': { get: { summary: 'A file at a ref', responses: { 200: { description: 'Blob' } } } },'/api/v1/repos/{owner}/{name}/commits': { get: { summary: 'History' }, post: { summary: 'Write a branch and files without a clone', responses: { 201: { description: 'The new commit' } } } },++ '/api/v1/repos/{owner}/{name}/archive/{ref}.tar.gz': { get: { summary: 'A gzip tarball of a ref (a tag answers the same bytes)', responses: { 200: { description: 'The tarball' } } } },++ '/api/v1/repos/{owner}/{name}/commits/{sha}/statuses': { get: { summary: 'The latest status per context for a commit', responses: { 200: { description: 'Statuses' } } }, post: { summary: 'Report a status (git.checks.write)', responses: { 201: { description: 'The status' } } } },'/api/v1/repos/{owner}/{name}/insights': { get: { summary: 'Views, clones, stars and commits per week', responses: { 200: { description: 'Insights' } } } },'/api/v1/tokens': { post: { summary: 'Make a short-lived Git token', responses: { 201: { description: 'The token, shown once' } } } },},@@@ -2,8 +2,8 @@/*** OpenVibe.Git — process entry. `node server/index.js` listens on PORT (5270) behind nginx (deploy/), starts the-- * mirror poll and the due-based official mirrors seed, and stops in order on SIGTERM: the HTTP drain, the timers,-- * the cat-file processes, the database.++ * mirror poll, the checks intake and the due-based official mirrors seed, and stops in order on SIGTERM: the HTTP++ * drain, the timers, the cat-file processes, the database.*/const { createApp } = require('./app');const { gracefulStop } = require('openvibe-sdk/service');@@@ -12,6 +12,7 @@ const { catFiles } = require('./git/store');function createLifecycle({ server, exit, signals, ctx } = {}) {const mirrors = ctx && ctx.mirrors;const official = ctx && ctx.official;++ const checksIntake = ctx && ctx.checksIntake;const store = ctx && ctx.store;return gracefulStop({name: 'OpenVibe.Git',@@@ -20,7 +21,7 @@ function createLifecycle({ server, exit, signals, ctx } = {}) {exit,signals,deadlineMs: 10_000,-- stop: [() => ctx.reviews && ctx.reviews.stop(), () => (mirrors ? mirrors.stop() : undefined), () => (official ? official.stop() : undefined)],++ stop: [() => ctx.reviews && ctx.reviews.stop(), () => (mirrors ? mirrors.stop() : undefined), () => (official ? official.stop() : undefined), () => (checksIntake ? checksIntake.stop() : undefined)],close: [() => catFiles.close(), () => (store ? store.close() : undefined)],});}@@@ -31,6 +32,7 @@ async function start() {if (config.mirrors.pollMs) ctx.mirrors.start();// Due-based and off unless GIT_OFFICIAL_ORG is set: the first check is shortly after boot, never in this path.ctx.official.start();++ ctx.checksIntake.start();ctx.reviews.start();const server = app.listen(config.port, config.host, () => {console.log(`[OpenVibe.Git] ${config.nodeEnv} on http://${config.host}:${config.port} → ${config.baseUrl} (release ${release.release})`);@@@ -5,7 +5,7 @@ const { runGit } = require('./git/run');const { fail } = plumbing;const text = (v, max, required = true) => { if (typeof v !== 'string' || v.length > max || (required && !v.trim())) throw fail(422, 'Invalid text'); return v.trim(); };function createReviews(ctx, services) {-- const { store: s, repos, git } = ctx;++ const { store: s, repos, git, checks } = ctx;let timer, running = false;const actorId = (a) => a && (a.on_behalf_of || a.subject);const writable = (r, a) => !!a && a.scopes.has('git.repo.write') && repos.isOwner(r, { subject: actorId(a) });@@@ -71,7 +71,9 @@ function createReviews(ctx, services) {const comments = await s.many('SELECT * FROM git_change_comments WHERE change_id = ? ORDER BY created_at', [c.id]);const approvals = await s.many('SELECT * FROM git_change_reviews WHERE change_id = ? AND head_sha = ?', [c.id,diff.head]);const summary = await s.maybe('SELECT result FROM git_change_summaries WHERE change_id = ? AND head_sha = ? AND model = ?', [c.id,diff.head,r.review_model || '']);-- return { change: c, diff, comments, approvals, checks: [], checks_label: 'No checks', summary: summary ? summary.result : null };++ // A fork's head is read from the fork (its CI posts there); the required list is the target's.++ const verdict = await checks.mergeable(r, diff.head, { repoId: source.id });++ return { change: c, diff, comments, approvals, ...verdict, summary: summary ? summary.result : null };}async function action(r, kind, item, op, body, a, viewer) {auth(a);@@@ -109,6 +111,9 @@ function createReviews(ctx, services) {write(r,a);const frozen = await plumbing.compare(r,item.base_ref,item.head_ref,source,true);if (frozen.head !== head) throw fail(409,'Head changed while reading diff');++ // Required checks gate the merge; an empty list merges as before.++ const verdict = await checks.mergeable(r, head, { repoId: source.id });++ if (!verdict.mergeable) throw fail(409, 'Required checks are not passing');await retain(r,item,frozen);const sha = await plumbing.merge(r,source,item,body.method || 'merge',head,a,frozen.base);await s.execute("UPDATE git_changes SET state = 'merged', head_sha = ?, merge_sha = ?, closed_at = ? WHERE id = ?", [head,sha,s.iso(),item.id]);@@@ -75,8 +75,14 @@ function change(r, out, canWrite, signed) {const action = (op, inner) => `<form class="git-inline-form" method="post" action="${b}/${op}">${hidden('head_sha', out.diff.head)}${inner}</form>`;const outdated = out.comments.filter((m) => m.head_sha !== out.diff.head);const approvals = out.approvals.length;++ const checkItems = (out.checks || []).map((s) => {++ const name = s.target_url ? `<a href="${esc(s.target_url)}" rel="noopener">${esc(s.context)}</a>` : `<span>${esc(s.context)}</span>`;++ return `<li class="git-check git-check--${esc(s.state)}">${ic(s.state === 'success' ? 'check' : s.state === 'pending' ? 'eye' : 'shield')}${name}<span class="git-muted">${esc(s.state)}</span></li>`;++ }).join('');++ const requiredNote = (out.required || []).length && !out.mergeable ? '<p class="git-muted">Required checks are not passing.</p>' : '';const box = `<section class="git-card git-merge-box" aria-labelledby="git-merge-h"><h2 id="git-merge-h">Review</h2><ul class="git-status"><li class="${approvals ? 'is-ok' : ''}">${ic(approvals ? 'check' : 'eye')}<span>${approvals ? `${plural(approvals, 'approval')} for this commit` : 'No approvals for this commit yet'}</span></li><li>${ic('shield')}<span>${esc(out.checks_label)}</span></li></ul>++${checkItems ? `<ul class="git-checks">${checkItems}</ul>` : ''}${requiredNote}${open && (signed || canWrite) ? `<div class="git-merge-acts">${signed ? action('approve', submit('Approve', 'quiet', 'check')) : ''}${canWrite ? action('merge', `<label class="git-field git-field--inline"><span>How</span><select name="method"><option value="merge">Merge commit</option><option value="squash">Squash</option><option value="rebase">Rebase</option></select></label>${submit('Merge', 'primary', 'branch')}`) : ''}${signed ? action('close', `<button class="git-close" type="submit">Close this change</button>`) : ''}</div>` : `<p class="git-muted">${open ? 'Sign in to review this change.' : `This change is ${esc(STATE[c.state] || c.state).toLowerCase()}.`}</p>`}</section>`;nnew file mode 100644@@@ -0,0 +1,124 @@++'use strict';++/**++ * Tarballs: /<owner>/<name>/archive/<ref>.tar.gz (and the API form) streams git archive of the ref. The++ * ETag is the commit; a tag answers the same bytes twice with immutable caching; a branch caches short.++ * A package manager can pin the address: npm installs a fixture package from the test server's own URL++ * (loopback only, no network).++ */++const assert = require('assert');++const crypto = require('node:crypto');++const fs = require('node:fs');++const os = require('node:os');++const path = require('node:path');++const zlib = require('node:zlib');++const { execFile } = require('node:child_process');++const { boot } = require('./helpers/boot');++++/** The entry names in a tar.gz buffer, parsed without leaving the process. */++function tarNames(tgz) {++ const tar = zlib.gunzipSync(tgz);++ const names = [];++ let off = 0;++ while (off + 512 <= tar.length) {++ const name = tar.subarray(off, off + 100).toString('utf8').split('\0')[0];++ if (!name) break;++ names.push(name);++ const size = parseInt(tar.subarray(off + 124, off + 136).toString('utf8').trim() || '0', 8);++ off += 512 + Math.ceil(size / 512) * 512;++ }++ return names;++}++++const sha256 = (buf) => crypto.createHash('sha256').update(buf).digest('hex');++++(async () => {++ const t = await boot();++ try {++ const sam = t.asUser('sam');++ await t.ctx.repos.create({ actor: sam, name: 'pkg', starter: 'empty', visibility: 'public' });++ const made = await t.ctx.git.commitFiles('sam', 'pkg', {++ branch: 'main', base: 'main', message: 'A tiny package',++ files: [++ { path: 'package.json', content: JSON.stringify({ name: 'fixture-pkg', version: '1.0.0', main: 'index.js' }) },++ { path: 'index.js', content: 'module.exports = 42;\n' },++ ],++ author: { name: 'Sam', email: '[email protected]' },++ });++ // An empty starter's first commit needs its branch: commitFiles made main from nothing.++ const sha = made.sha;++ await t.git(['tag', 'v1.0.0', sha], t.ctx.git.repoDir('sam', 'pkg'));++++ const page = await t.get('/sam/pkg/archive/v1.0.0.tar.gz');++ assert.equal(page.status, 200, page.text.slice(0, 200));++ assert.match(page.headers.get('content-type'), /application\/gzip/);++ assert.match(page.headers.get('content-disposition'), /attachment; filename="pkg-v1\.0\.0\.tar\.gz"/);++ assert.equal(page.headers.get('etag'), `"${sha}-tar.gz"`);++ assert.equal(page.headers.get('cache-control'), 'public, max-age=31536000, immutable');++ const names = tarNames(page.buffer);++ assert.ok(names.includes(`pkg-${sha}/package.json`), names.join(','));++ assert.ok(names.includes(`pkg-${sha}/index.js`));++ // git archive prepends a pax_global_header with the commit id; everything else sits under one folder.++ assert.ok(names.every((n) => n === 'pax_global_header' || n.startsWith(`pkg-${sha}/`)), `one top folder: ${names.join(',')}`);++++ // A tag's bytes are stable: the same address twice, and the API form, hash alike.++ const again = await t.get('/sam/pkg/archive/v1.0.0.tar.gz');++ assert.equal(sha256(again.buffer), sha256(page.buffer));++ const api = await t.get('/api/v1/repos/sam/pkg/archive/v1.0.0.tar.gz');++ assert.equal(api.status, 200, api.text.slice(0, 200));++ assert.equal(sha256(api.buffer), sha256(page.buffer));++++ // A full commit id caches immutably too; a branch caches short.++ const byId = await t.get(`/sam/pkg/archive/${sha}.tar.gz`);++ assert.equal(byId.status, 200);++ assert.equal(byId.headers.get('cache-control'), 'public, max-age=31536000, immutable');++ const branch = await t.get('/sam/pkg/archive/main.tar.gz');++ assert.equal(branch.status, 200);++ assert.equal(branch.headers.get('cache-control'), 'public, max-age=60');++ assert.equal(branch.headers.get('etag'), `"${sha}-tar.gz"`);++++ // A branch with a slash archives under its own name.++ await t.ctx.git.commitFiles('sam', 'pkg', {++ branch: 'agent/run-1', base: 'main', message: 'A slash branch',++ files: [{ path: 'slash.txt', content: 'slashed\n' }],++ author: { name: 'Sam', email: '[email protected]' },++ });++ const slashed = await t.get('/sam/pkg/archive/agent/run-1.tar.gz');++ assert.equal(slashed.status, 200, slashed.text.slice(0, 200));++ assert.match(slashed.headers.get('content-disposition'), /filename="pkg-agent-run-1\.tar\.gz"/);++ assert.ok(tarNames(slashed.buffer).some((n) => n.endsWith('/slash.txt')));++++ // Conditional reads, unknown refs and the suffix rule.++ assert.equal((await t.get('/sam/pkg/archive/v1.0.0.tar.gz', { headers: { 'if-none-match': `"${sha}-tar.gz"` } })).status, 304);++ assert.equal((await t.get('/sam/pkg/archive/nope.tar.gz')).status, 404);++ assert.equal((await t.get('/sam/pkg/archive/main.zip')).status, 404);++ assert.equal((await t.get('/api/v1/repos/sam/pkg/archive/nope.tar.gz')).status, 404);++++ // Private repositories follow the raw file: 404 to strangers, a tarball to the owner.++ await t.ctx.repos.create({ actor: sam, name: 'secret-pkg', starter: 'static', visibility: 'private' });++ assert.equal((await t.get('/sam/secret-pkg/archive/main.tar.gz')).status, 404);++ assert.equal((await t.get('/api/v1/repos/sam/secret-pkg/archive/main.tar.gz')).status, 404);++ assert.equal((await t.get('/sam/secret-pkg/archive/main.tar.gz', { as: sam })).status, 200);++ const token = await t.get('/api/v1/tokens', { as: sam, json: { scopes: ['git.repo.read'] } });++ assert.equal((await t.get('/api/v1/repos/sam/secret-pkg/archive/main.tar.gz', { bearer: token.json().token })).status, 200);++++ // npm installs the fixture from the forge address, like a package.json pin would.++ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'openvibe-git-pin-'));++ try {++ const url = `${t.base}/sam/pkg/archive/v1.0.0.tar.gz`;++ fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ name: 'pin-consumer', private: true, dependencies: { 'fixture-pkg': url } }));++ await new Promise((resolve, reject) => {++ execFile('npm', ['install', '--no-audit', '--no-fund', '--ignore-scripts', '--no-update-notifier'],++ { cwd: dir, timeout: 90000, env: { ...process.env, HOME: dir, npm_config_cache: path.join(dir, 'cache') } },++ (err, stdout, stderr) => (err ? reject(Object.assign(err, { stdout, stderr })) : resolve(stdout)));++ });++ const installed = JSON.parse(fs.readFileSync(path.join(dir, 'node_modules', 'fixture-pkg', 'package.json'), 'utf8'));++ assert.equal(installed.name, 'fixture-pkg');++ assert.equal(installed.version, '1.0.0');++ assert.equal(require(path.join(dir, 'node_modules', 'fixture-pkg', 'index.js')), 42);++ } finally {++ fs.rmSync(dir, { recursive: true, force: true });++ }++++ console.log('archive: page and API tarballs, stable tag bytes, ETag/cache, auth and an npm pin install passed');++ } finally { await t.close(); }++})().catch((e) => { console.error(e); process.exit(1); });nnew file mode 100644@@@ -0,0 +1,98 @@++'use strict';++/**++ * Checks intake from GitHub, read-only: for a mirrored repository it copies the check-runs of the open++ * pull-request heads and the default branch into statuses. At most every two minutes per repository, and++ * check-runs are asked only for heads that moved. GitHub is a canned fetch; nothing leaves the process.++ */++const assert = require('assert');++const { boot } = require('./helpers/boot');++const { mapRun } = require('../server/checks-intake');++++const ok = (body, link = null) => ({++ ok: true, status: 200,++ headers: { get: (name) => (String(name).toLowerCase() === 'link' ? link : null) },++ json: async () => body,++});++++(async () => {++ const calls = [];++ let prHead;++ const runsBySha = {};++ const fetchImpl = async (url) => {++ calls.push(String(url));++ if (String(url).includes('/pulls')) return ok([{ number: 7, head: { sha: prHead } }]);++ const m = /\/commits\/([0-9a-f]{40})\/check-runs/.exec(String(url));++ if (m) return ok({ check_runs: runsBySha[m[1]] || [] });++ throw new Error(`unexpected GitHub call: ${url}`);++ };++ const t = await boot({ fetchImpl, env: { GIT_GITHUB_TOKEN: 'secret-mirror-token' } });++ try {++ assert.equal(t.ctx.checksIntake.start(), false, 'off in drill mode');++++ const sam = t.asUser('sam');++ await t.ctx.repos.create({ actor: sam, name: 'live', starter: 'static', visibility: 'public' });++ const mainSha = await t.ctx.git.resolveCommit('sam', 'live', 'main');++ const pr = await t.ctx.git.commitFiles('sam', 'live', {++ branch: 'pr-branch', base: 'main', message: 'The PR head', files: [{ path: 'pr.txt', content: 'pr\n' }],++ author: { name: 'Sam', email: '[email protected]' },++ });++ prHead = pr.sha;++ runsBySha[mainSha] = [++ { name: 'test', status: 'completed', conclusion: 'success', html_url: 'https://github.com/OpenVibers/OpenVibe.Live/runs/1', output: { title: 'All green' } },++ { name: 'security', status: 'completed', conclusion: 'failure', html_url: 'https://github.com/OpenVibers/OpenVibe.Live/runs/2', output: { title: 'Leak found' } },++ ];++ runsBySha[pr.sha] = [{ name: 'test', status: 'in_progress', conclusion: null, html_url: 'https://github.com/OpenVibers/OpenVibe.Live/runs/3' }];++ await t.store.execute("UPDATE git_repos SET mirror_url = 'https://github.com/OpenVibers/OpenVibe.Live.git', github_full_name = 'OpenVibers/OpenVibe.Live' WHERE owner_username = 'sam' AND name = 'live'");++ const row = await t.ctx.repos.getRow('sam', 'live');++++ const first = await t.ctx.checksIntake.syncRepo(row, { force: true });++ assert.equal(first.ok, true, JSON.stringify(first));++ assert.equal(first.heads, 2);++ assert.equal(first.fetched, 2);++ assert.equal(first.copied, 3);++ assert.equal(calls.filter((u) => u.includes('/pulls')).length, 1, 'one PR listing');++ assert.equal(calls.filter((u) => u.includes('/check-runs')).length, 2);++++ const main = await t.get(`/api/v1/repos/sam/live/commits/${mainSha}/statuses`);++ assert.deepStrictEqual(main.json().statuses.map((s) => [s.context, s.state]), [['security', 'failure'], ['test', 'success']]);++ assert.equal(main.json().state, 'failure');++ assert.equal(main.json().statuses[0].target_url, 'https://github.com/OpenVibers/OpenVibe.Live/runs/2');++ const prStatuses = await t.get(`/api/v1/repos/sam/live/commits/${pr.sha}/statuses`);++ assert.deepStrictEqual(prStatuses.json().statuses.map((s) => [s.context, s.state]), [['test', 'pending']]);++++ // A second run inside two minutes does nothing; a forced run with unmoved heads asks no check-runs.++ assert.equal((await t.ctx.checksIntake.syncRepo(await t.ctx.repos.getRow('sam', 'live'))).skipped, 'recent');++ const before = calls.length;++ const forced = await t.ctx.checksIntake.syncRepo(await t.ctx.repos.getRow('sam', 'live'), { force: true });++ assert.equal(forced.fetched, 0, 'unmoved heads cost no check-runs call');++ assert.equal(forced.copied, 0);++ assert.equal(calls.length - before, 1, 'only the PR listing re-ran');++++ // A moved head is fetched again; an unchanged one is not.++ const pr2 = await t.ctx.git.commitFiles('sam', 'live', {++ branch: 'pr-branch', base: 'pr-branch', message: 'The PR moves', files: [{ path: 'pr.txt', content: 'pr2\n' }],++ author: { name: 'Sam', email: '[email protected]' },++ });++ prHead = pr2.sha;++ runsBySha[pr2.sha] = [{ name: 'test', status: 'completed', conclusion: 'success', html_url: 'https://github.com/x/runs/4' }];++ const moved = await t.ctx.checksIntake.syncRepo(await t.ctx.repos.getRow('sam', 'live'), { force: true });++ assert.equal(moved.fetched, 1);++ assert.equal(moved.copied, 1);++ assert.deepStrictEqual((await t.get(`/api/v1/repos/sam/live/commits/${pr2.sha}/statuses`)).json().statuses.map((s) => s.state), ['success']);++++ // A repository that is no mirror is left alone, and the token never reaches a log.++ await t.ctx.repos.create({ actor: sam, name: 'plain', starter: 'static', visibility: 'public' });++ assert.equal((await t.ctx.checksIntake.syncRepo(await t.ctx.repos.getRow('sam', 'plain'), { force: true })).skipped, 'not-mirrored');++ assert.ok(!t.logs().join('\n').includes('secret-mirror-token'), 'the mirror token is never logged');++++ // The check-run mapping: queued work pends, neutral passes, a cancelled run errors.++ assert.equal(mapRun({ name: 't', status: 'queued' }).state, 'pending');++ assert.equal(mapRun({ name: 't', status: 'completed', conclusion: 'neutral' }).state, 'success');++ assert.equal(mapRun({ name: 't', status: 'completed', conclusion: 'cancelled' }).state, 'error');++ assert.equal(mapRun({ name: 't', status: 'completed', conclusion: 'timed_out' }).state, 'error');++ assert.equal(mapRun({ name: '' }), null);++ assert.equal(mapRun(null), null);++++ console.log('checks-intake: copy, throttle, moved-heads-only, mapping and token silence passed');++ } finally { await t.close(); }++})().catch((e) => { console.error(e); process.exit(1); });nnew file mode 100644@@@ -0,0 +1,99 @@++'use strict';++/**++ * Commit statuses: a checks service reports {context, state, description, target_url} per commit; reads++ * take the latest per context. A change shows them and is mergeable only when its required contexts are++ * green; an empty required list changes nothing.++ */++const assert = require('assert');++const { boot } = require('./helpers/boot');++++(async () => {++ const t = await boot();++ try {++ const sam = t.asUser('sam'), pat = t.asUser('pat');++ await t.ctx.repos.create({ actor: sam, name: 'ci', starter: 'static', visibility: 'public' });++ const head = await t.ctx.git.commitFiles('sam', 'ci', {++ branch: 'agent/feat', base: 'main', message: 'A feature', files: [{ path: 'feat.txt', content: 'yes\n' }],++ author: { name: 'Sam', email: '[email protected]' },++ });++ const root = '/api/v1/repos/sam/ci';++ const runSvc = t.network.gitToken({ subject: 'svc:run', caps: ['git.checks.write'], type: null });++ const readOnly = t.network.gitToken({ subject: 'svc:run', caps: ['git.repo.read'], type: null });++ const post = (body, bearer = runSvc) => t.get(`${root}/commits/${head.sha}/statuses`, { bearer, json: body });++++ // Who may report: a checks service, or the owner with the capability. Nobody else.++ assert.equal((await t.get(`${root}/commits/${head.sha}/statuses`, { json: { context: 'test', state: 'success' } })).status, 401);++ assert.equal((await post({ context: 'test', state: 'success' }, readOnly)).status, 403);++ assert.equal((await post({ context: 'test', state: 'success' }, readOnly)).json().code, 'token.scope');++ assert.equal((await t.get(`${root}/commits/${head.sha}/statuses`, { as: sam, json: { context: 'test', state: 'success' } })).status, 403);++ assert.equal((await post({ context: 'test', state: 'nope' })).status, 422);++ assert.equal((await t.get(`${root}/commits/notasha/statuses`, { bearer: runSvc, json: { context: 'test', state: 'success' } })).status, 422);++ assert.equal((await post({ context: 'test', state: 'success', target_url: 'gopher://x' })).status, 422);++ assert.equal((await post({ context: '', state: 'success' })).status, 422);++++ const first = await post({ context: 'test', state: 'pending', description: 'Running', target_url: 'https://ci.example/runs/1' });++ assert.equal(first.status, 201, first.text);++ assert.equal(first.json().context, 'test');++ assert.equal(first.json().state, 'pending');++ await post({ context: 'security', state: 'success' });++ await post({ context: 'test', state: 'success', description: 'Green', target_url: 'https://ci.example/runs/1' });++++ // Reads take the latest per context, with the combined state. Public commits read anonymously.++ const listed = await t.get(`${root}/commits/${head.sha}/statuses`);++ assert.equal(listed.status, 200);++ assert.deepStrictEqual(listed.json().statuses.map((s) => [s.context, s.state]), [['security', 'success'], ['test', 'success']]);++ assert.equal(listed.json().statuses.find((s) => s.context === 'test').description, 'Green');++ assert.equal(listed.json().state, 'success');++ assert.equal((await t.get(`${root}/commits/${head.sha}/statuses`)).status, 200);++++ // Required contexts: empty by default, the owner sets them, nobody else.++ assert.deepStrictEqual((await t.get(`${root}/required-checks`)).json().required_checks, []);++ assert.equal((await t.get(`${root}/required-checks`, { as: pat, json: { contexts: ['test'] } })).status, 403);++ assert.equal((await t.get(`${root}/required-checks`, { as: sam, json: { contexts: ['x'.repeat(300)] } })).status, 422);++ const required = await t.get(`${root}/required-checks`, { as: sam, json: { contexts: ['test', 'security'] } });++ assert.equal(required.status, 200, required.text);++ assert.deepStrictEqual(required.json().required_checks, ['test', 'security']);++++ // A change shows the checks and is mergeable while its required contexts are green.++ const created = await t.get(`${root}/changes`, { as: sam, json: { title: 'Ship the feature', base_ref: 'main', head_ref: 'agent/feat' } });++ assert.equal(created.status, 201, created.text);++ const detail = await t.get(`${root}/changes/${created.json().number}`);++ assert.equal(detail.json().checks.length, 2);++ assert.equal(detail.json().checks_label, '2 checks passed');++ assert.equal(detail.json().state, 'success');++ assert.equal(detail.json().mergeable, true);++ assert.deepStrictEqual(detail.json().required.map((r) => [r.context, r.ok]), [['test', true], ['security', true]]);++ const page = await t.get(`/sam/ci/changes/${created.json().number}`, { as: sam });++ assert.match(page.text, /2 checks passed/);++ assert.match(page.text, /git-check--success/);++ assert.match(page.text, /ci\.example\/runs\/1/);++++ // A red required context blocks the merge, on the API and on the page.++ await post({ context: 'test', state: 'failure', description: 'Broke it' });++ const red = await t.get(`${root}/changes/${created.json().number}`);++ assert.equal(red.json().checks_label, '1 of 2 checks failed');++ assert.equal(red.json().mergeable, false);++ assert.equal((await t.get(`${root}/changes/${created.json().number}/merge`, { as: sam, json: { head_sha: head.sha, method: 'squash' } })).status, 409);++ const redPage = await t.get(`/sam/ci/changes/${created.json().number}`, { as: sam });++ assert.match(redPage.text, /Required checks are not passing/);++ await post({ context: 'test', state: 'success', description: 'Fixed' });++ assert.equal((await t.get(`${root}/changes/${created.json().number}`)).json().mergeable, true);++ assert.equal((await t.get(`${root}/changes/${created.json().number}/merge`, { as: sam, json: { head_sha: head.sha, method: 'squash' } })).status, 200);++++ // A missing required context blocks too; clearing the list merges as before.++ const head2 = await t.ctx.git.commitFiles('sam', 'ci', {++ branch: 'agent/more', base: 'main', message: 'More', files: [{ path: 'more.txt', content: 'more\n' }],++ author: { name: 'Sam', email: '[email protected]' },++ });++ const created2 = await t.get(`${root}/changes`, { as: sam, json: { title: 'More work', base_ref: 'main', head_ref: 'agent/more' } });++ const missing = await t.get(`${root}/changes/${created2.json().number}`);++ assert.equal(missing.json().checks_label, 'No checks');++ assert.equal(missing.json().mergeable, false, 'required contexts with no report block');++ assert.equal((await t.get(`${root}/changes/${created2.json().number}/merge`, { as: sam, json: { head_sha: head2.sha, method: 'merge' } })).status, 409);++ assert.deepStrictEqual((await t.get(`${root}/required-checks`, { as: sam, json: { contexts: [] } })).json().required_checks, []);++ assert.equal((await t.get(`${root}/changes/${created2.json().number}`)).json().mergeable, true);++ assert.equal((await t.get(`${root}/changes/${created2.json().number}/merge`, { as: sam, json: { head_sha: head2.sha, method: 'merge' } })).status, 200);++++ console.log('checks: report auth, latest-per-context, required contexts, mergeable and the merge gate passed');++ } finally { await t.close(); }++})().catch((e) => { console.error(e); process.exit(1); });
@@ -33,10 +33,12 @@ GIT_GITHUB_PUSH_TOKEN=
@@ -33,10 +33,12 @@ GIT_GITHUB_PUSH_TOKEN=
GIT_TOKEN_TTL_HOURS=1
GIT_TOKEN_TTL_HOURS=1
GIT_SECRET_KEY=
GIT_SECRET_KEY=
-# Limits (bytes). A push larger than this is refused; a repository may grow to the cap.
+# Limits (bytes). A push larger than this is refused; a repository may grow to the cap; an archive
+# larger than the cap is refused.
GIT_MAX_PUSH_BYTES=536870912
GIT_MAX_PUSH_BYTES=536870912
GIT_MAX_REPO_BYTES=3221225472
GIT_MAX_REPO_BYTES=3221225472
GIT_MAX_FILE_BYTES=2097152
GIT_MAX_FILE_BYTES=2097152
+GIT_MAX_ARCHIVE_BYTES=268435456
# Mirrors: the ten-minute poll and how long one fetch may take.
# Mirrors: the ten-minute poll and how long one fetch may take.
GIT_MIRROR_POLL_MS=600000
GIT_MIRROR_POLL_MS=600000
@@ -52,6 +52,28 @@ is a row in the database, so a deploy does not reset the interval. The first che
@@ -52,6 +52,28 @@ is a row in the database, so a deploy does not reset the interval. The first che
job ticks hourly, in the background: `/api/ready` never waits on it, and a failed run waits for the next due time.
job ticks hourly, in the background: `/api/ready` never waits on it, and a failed run waits for the next due time.
An empty `GIT_OFFICIAL_ORG` turns the job off, and so does drill mode.
An empty `GIT_OFFICIAL_ORG` turns the job off, and so does drill mode.
+## Tarballs, pins and checks
+
+Every ref answers a gzip tarball: `GET /<owner>/<name>/archive/<ref>.tar.gz` (a branch may hold slashes) and
+the same path under `/api/v1/repos/<owner>/<name>/`. The bytes are git archive of the resolved commit,
+streamed, with the commit as the ETag; a tag or a full commit id caches immutably, a branch for a minute.
+The top folder is `<name>-<commit>/`, so a tag's bytes never change and a package manager can pin them:
+
+```json
+"openvibe-shared": "https://git.openvibe.codes/openvibers/openvibe.shared/archive/v4.186.0.tar.gz"
+```
+
+instead of the `codeload.github.com` URL it replaces. Public repositories need no token; private ones answer
+404 to anyone who may not see them, like the raw file.
+
+A commit carries statuses: `POST /api/v1/repos/:owner/:name/commits/:sha/statuses` with
+`{context, state: pending|success|failure|error, description?, target_url?}`, reported with a typed Network
+service token holding `git.checks.write`; `GET` lists the latest per context. A change shows them and is
+mergeable only when its required contexts are green — a per-repository list, empty by default, set with
+`POST …/required-checks {contexts}`. For a mirrored repository the checks intake copies GitHub's check-runs
+for the open pull-request heads and the default branch (at most every two minutes per repository, and only
+when a head moved), so the pages show the truth until our own runners report.
+
## Layout
## Layout
- `server/git/` — repositories on disk: creation, refs, trees, blobs, commits, diffs, bunches, and `http-backend`.
- `server/git/` — repositories on disk: creation, refs, trees, blobs, commits, diffs, bunches, and `http-backend`.
@@ -6,7 +6,10 @@ JSON routes below `/api/v1/repos/:owner/:name`:
@@ -6,7 +6,10 @@ JSON routes below `/api/v1/repos/:owner/:name`:
- `GET /compare/:base...:head`: commits (up to 200), files (up to 300), added/removed counts and unified lines with old/new numbers. Branch slashes can be percent-encoded.
- `GET /compare/:base...:head`: commits (up to 200), files (up to 300), added/removed counts and unified lines with old/new numbers. Branch slashes can be percent-encoded.
- `GET|POST /changes`: list or create `{title, body?, base_ref, head_ref, source_repo?}`; source is `owner/name`, same repo or its fork family.
- `GET|POST /changes`: list or create `{title, body?, base_ref, head_ref, source_repo?}`; source is `owner/name`, same repo or its fork family.
-- `GET /changes/:number`: change, diff, comments, approvals for the current SHA, checks and cached summary.
+- `GET /changes/:number`: change, diff, comments, approvals for the current SHA, checks (`checks`, `checks_label`, combined `state`, per-context `required`, `mergeable`) and cached summary.
+- `GET|POST /commits/:sha/statuses`: the latest status per context, or report `{context, state: pending|success|failure|error, description?, target_url?}` with a typed Network service token holding `git.checks.write`.
+- `GET|POST /required-checks`: the contexts that must be green before a change merges (`{contexts: [...]}`), empty by default; merge is 409 while one is missing or not green.
+- `GET /archive/<ref>.tar.gz` (page and API): a streamed gzip tarball of the ref, stable for a tag; the ETag is the commit. See README for the pin form.
- `POST /changes/:number/comments`: `{head_sha,path,line,side:"old"|"new",message}`.
- `POST /changes/:number/comments`: `{head_sha,path,line,side:"old"|"new",message}`.
- `POST /changes/:number/approve`: `{head_sha}`.
- `POST /changes/:number/approve`: `{head_sha}`.
- `POST /changes/:number/merge`: `{head_sha,method:"merge"|"squash"|"rebase"}`; conflicts are 409 with `files`.
- `POST /changes/:number/merge`: `{head_sha,method:"merge"|"squash"|"rebase"}`; conflicts are 409 with `files`.
@@ -24,7 +27,7 @@ Community uses the existing `community.comment.write` and `community.comment.mod
@@ -24,7 +27,7 @@ Community uses the existing `community.comment.write` and `community.comment.mod
Network needs `network.notifications.push` and `identity.subject.resolve`. Notifications address the change author, repository owner and watchers, excluding the actor and checking current visibility before delivery. These scopes must be granted to the Git service principal before deployment; this job does not alter production grants.
Network needs `network.notifications.push` and `identity.subject.resolve`. Notifications address the change author, repository owner and watchers, excluding the actor and checking current visibility before delivery. These scopes must be granted to the Git service principal before deployment; this job does not alter production grants.
-AI needs `ai.run.create` and `ai.run.read`, plus the AI-choices API from the concurrent AI-choices job (Contracts #389). G2 checks `GET /api/v1/tasks` for `ai.generate` before posting a run with `{workflow:"ai.generate",choice:{model,source:"openvibe"},on_behalf_of,input}`. Older AI fails closed, without silently using a different model. Local tests use mock runs; no paid calls were made. Run currently has job APIs but no repository/head-commit check lookup: the slot says **No checks**.
+AI needs `ai.run.create` and `ai.run.read`, plus the AI-choices API from the concurrent AI-choices job (Contracts #389). G2 checks `GET /api/v1/tasks` for `ai.generate` before posting a run with `{workflow:"ai.generate",choice:{model,source:"openvibe"},on_behalf_of,input}`. Older AI fails closed, without silently using a different model. Local tests use mock runs; no paid calls were made. Run currently has job APIs but no repository/head-commit check lookup; until it posts statuses, the checks intake copies GitHub's check-runs for mirrored repositories (off in drill mode, two minutes per repository, only when a head moved), and the slot says **No checks** only when nothing was reported.
## Merge and maintenance
## Merge and maintenance
new file mode 100644
new file mode 100644
@@ -0,0 +1,34 @@
@@ -0,0 +1,34 @@
+-- phase: expand
+-- Commit statuses (checks): what CI says about a commit, who requires it, and what the GitHub intake copied.
+--
+-- A status is one CI verdict on one commit: {context, state, description, target_url}. History is kept (one row
+-- per report); reads take the latest row per context. A repository names the contexts that must be green before
+-- a change merges (required_checks, empty by default, so nothing changes until one is set). The intake remembers
+-- the heads it already copied per repository, so it only asks GitHub about heads that moved.
+
+CREATE TABLE git_commit_statuses (
+ id text PRIMARY KEY, -- cst_<ulid>
+ repo_id text NOT NULL REFERENCES git_repos (id) ON DELETE CASCADE,
+ sha text NOT NULL, -- the full commit id, lower-case hex
+ context text NOT NULL, -- e.g. "test", "security/secrets"
+ state text NOT NULL CHECK (state IN ('pending', 'success', 'failure', 'error')),
+ description text NOT NULL DEFAULT '',
+ target_url text NOT NULL DEFAULT '',
+ creator_subject text, -- svc:<service> or usr_… that reported it
+ created_at timestamptz NOT NULL DEFAULT now(),
+ CHECK (sha ~ '^[0-9a-f]{40}([0-9a-f]{24})?$'),
+ CHECK (char_length(context) BETWEEN 1 AND 255),
+ CHECK (char_length(description) <= 1024),
+ CHECK (char_length(target_url) <= 2048)
+);
+CREATE INDEX git_commit_statuses_repo_sha ON git_commit_statuses (repo_id, sha, created_at DESC);
+
+ALTER TABLE git_repos ADD COLUMN required_checks jsonb NOT NULL DEFAULT '[]'::jsonb;
+
+CREATE TABLE git_checks_sync (
+ repo_id text PRIMARY KEY REFERENCES git_repos (id) ON DELETE CASCADE,
+ last_run_at timestamptz,
+ heads jsonb NOT NULL DEFAULT '{}'::jsonb, -- {"main": "<sha>", "pr/12": "<sha>"}
+ last_result jsonb,
+ updated_at timestamptz NOT NULL DEFAULT now()
+);
@@ -27,6 +27,7 @@ const { createGithubClient } = require('./github');
@@ -27,6 +27,7 @@ const { createGithubClient } = require('./github');
const { createInsights } = require('./insights');
const { createInsights } = require('./insights');
const { createRepos } = require('./repos');
const { createRepos } = require('./repos');
const { createMirrors } = require('./mirror');
const { createMirrors } = require('./mirror');
+const { createChecksIntake } = require('./checks-intake');
const { createOfficialMirrors } = require('./official');
const { createOfficialMirrors } = require('./official');
const { createSmartHttp, parseGitPath } = require('./git/http-backend');
const { createSmartHttp, parseGitPath } = require('./git/http-backend');
const { identify } = require('./auth/git-auth');
const { identify } = require('./auth/git-auth');
@@ -34,6 +35,8 @@ const { createApi } = require('./http/api');
@@ -34,6 +35,8 @@ const { createApi } = require('./http/api');
const { createPageRoutes } = require('./http/pages');
const { createPageRoutes } = require('./http/pages');
const { createReviewRoutes } = require('./http/review');
const { createReviewRoutes } = require('./http/review');
const { createReviews } = require('./reviews');
const { createReviews } = require('./reviews');
+const { createChecks } = require('./checks');
+const { createArchive } = require('./archive');
const { createReviewServices } = require('./review-services');
const { createReviewServices } = require('./review-services');
const { createLinkRelay } = require('./link-relay');
const { createLinkRelay } = require('./link-relay');
const { createLinkRoutes } = require('./http/link');
const { createLinkRoutes } = require('./http/link');
@@ -61,6 +64,9 @@ function createApp(opts = {}) {
@@ -61,6 +64,9 @@ function createApp(opts = {}) {
secureCookies: config.cookies.secure, fetch: fetchImpl, log,
secureCookies: config.cookies.secure, fetch: fetchImpl, log,
});
});
const ctx = { config, store: s, s, git, github, insights, repos, mirrors, official, smartHttp, sso, log };
const ctx = { config, store: s, s, git, github, insights, repos, mirrors, official, smartHttp, sso, log };
+ ctx.checks = createChecks({ store: s, log });
+ ctx.archive = createArchive({ config, git });
+ ctx.checksIntake = opts.checksIntake || createChecksIntake({ store: s, checks: ctx.checks, config, git, github, log });
ctx.relay = createLinkRelay(ctx);
ctx.relay = createLinkRelay(ctx);
ctx.reviews = createReviews(ctx, opts.reviewServices || createReviewServices(config, fetchImpl));
ctx.reviews = createReviews(ctx, opts.reviewServices || createReviewServices(config, fetchImpl));
new file mode 100644
new file mode 100644
@@ -0,0 +1,110 @@
@@ -0,0 +1,110 @@
+'use strict';
+
+/**
+ * Tarballs: GET /<owner>/<name>/archive/<ref>.tar.gz (and the API form) answers a gzip tarball of that
+ * ref, made by git archive from the bare repository and streamed to the reader.
+ *
+ * The bytes are git's own archive of the resolved commit, so a tag always answers the same bytes; the top
+ * folder is <name>-<commit>/, so a package manager can pin it. The ETag is the commit; a full commit id or
+ * a tag caches immutably, a branch for a minute. Reads follow the raw file's auth: public repositories
+ * need no token, private ones answer 404 to anyone who may not see them.
+ */
+const { spawnGit } = require('./git/run');
+const { validRef } = require('./git/refs');
+
+const SUFFIX = '.tar.gz';
+const FULL_SHA_RE = /^[0-9a-f]{40}([0-9a-f]{24})?$/;
+
+/** The ref a /archive/<file> path names, or null. <file> is <ref>.tar.gz; a branch may hold slashes. */
+function refOfArchive(file) {
+ const name = String(file || '');
+ if (!name.endsWith(SUFFIX) || name.length === SUFFIX.length) return null;
+ const ref = name.slice(0, -SUFFIX.length);
+ return validRef(ref) ? ref : null;
+}
+
+function createArchive({ config, git }) {
+ /** A ref to its commit, and whether the answer never changes (a full id or a tag). */
+ async function resolve(owner, name, ref) {
+ const sha = await git.resolveCommit(owner, name, ref);
+ if (!sha) return null;
+ const lower = ref.toLowerCase();
+ const isFullId = FULL_SHA_RE.test(lower) && lower === sha.slice(0, lower.length) && lower.length >= 40;
+ let isTag = false;
+ if (!isFullId) {
+ const tagged = await git.resolveCommit(owner, name, `refs/tags/${ref}`);
+ isTag = Boolean(tagged) && tagged === sha;
+ }
+ return { sha, immutable: isFullId || isTag };
+ }
+
+ /** Stream git archive to the response. Headers (ETag, caching) are the caller's. */
+ function stream(res, owner, name, sha, { prefix, maxBytes, timeoutMs, log } = {}) {
+ const dir = git.repoDir(owner, name);
+ const cap = Math.max(1024, Number(maxBytes) || config.data.maxArchiveBytes);
+ const child = spawnGit(['archive', '--format=tar.gz', `--prefix=${prefix}`, sha], { cwd: dir, timeoutMs: timeoutMs || 120_000 });
+ let sent = 0;
+ let failed = false;
+ const kill = () => { try { child.kill('SIGKILL'); } catch { /* already gone */ } };
+ res.on('close', kill);
+ child.on('error', (err) => {
+ failed = true;
+ res.removeListener('close', kill);
+ if (!res.headersSent) {
+ res.status(500).type('text/plain').send('The archive could not be made.');
+ } else {
+ try { res.destroy(); } catch { /* gone */ }
+ }
+ (log || console).warn(`[OpenVibe.Git] archive ${owner}/${name}: ${String(err.message || err).slice(0, 200)}`);
+ });
+ child.stdout.on('data', (chunk) => {
+ sent += chunk.length;
+ if (sent > cap) {
+ failed = true;
+ kill();
+ if (!res.headersSent) {
+ res.status(413).type('text/plain').send('The archive is larger than this service serves.');
+ } else {
+ try { res.destroy(); } catch { /* gone */ }
+ }
+ return;
+ }
+ if (!res.write(chunk)) child.stdout.pause();
+ });
+ res.on('drain', () => child.stdout.resume());
+ child.on('close', (code) => {
+ res.removeListener('close', kill);
+ if (failed) return;
+ if (code !== 0) {
+ if (!res.headersSent) res.status(500).type('text/plain').send('The archive could not be made.');
+ else { try { res.destroy(); } catch { /* gone */ } }
+ return;
+ }
+ res.end();
+ });
+ return child;
+ }
+
+ return { resolve, stream };
+}
+
+/**
+ * Answer one archive request on either surface. fail(status, message) renders the surface's own error;
+ * visibility was already checked by the caller, like the raw file.
+ */
+async function serveArchive(archive, req, res, row, file, { log, fail }) {
+ const ref = refOfArchive(file);
+ if (!ref) { fail(404, 'No such archive.'); return; }
+ const resolved = await archive.resolve(row.owner_username, row.name, ref);
+ if (!resolved) { fail(404, 'No such ref.'); return; }
+ const etag = `"${resolved.sha}-tar.gz"`;
+ if (req.get('if-none-match') === etag) { res.status(304).end(); return; }
+ const safe = ref.replace(/[^A-Za-z0-9._-]+/g, '-').slice(0, 120);
+ res.set('Content-Type', 'application/gzip');
+ res.set('Content-Disposition', `attachment; filename="${row.name}-${safe}.tar.gz"`);
+ res.set('ETag', etag);
+ res.set('Cache-Control', resolved.immutable ? 'public, max-age=31536000, immutable' : 'public, max-age=60');
+ archive.stream(res, row.owner_username, row.name, resolved.sha, { prefix: `${row.name}-${resolved.sha}/`, log });
+}
+
+module.exports = { createArchive, refOfArchive, serveArchive, SUFFIX };
@@ -17,7 +17,7 @@ const crypto = require('node:crypto');
@@ -17,7 +17,7 @@ const crypto = require('node:crypto');
const { createJwksClient } = require('openvibe-sdk/auth');
const { createJwksClient } = require('openvibe-sdk/auth');
const TOKEN_PREFIX = 'git_';
const TOKEN_PREFIX = 'git_';
-const SCOPES = Object.freeze(['git.repo.read', 'git.repo.write']);
+const SCOPES = Object.freeze(['git.repo.read', 'git.repo.write', 'git.checks.write']);
const USER_RE = /^usr_[0-9A-HJKMNP-TV-Z]{26}$/;
const USER_RE = /^usr_[0-9A-HJKMNP-TV-Z]{26}$/;
const SVC_RE = /^svc:[a-z][a-z0-9-]{1,39}$/;
const SVC_RE = /^svc:[a-z][a-z0-9-]{1,39}$/;
@@ -33,7 +33,7 @@ const newToken = () => `${TOKEN_PREFIX}${crypto.randomBytes(32).toString('base64
@@ -33,7 +33,7 @@ const newToken = () => `${TOKEN_PREFIX}${crypto.randomBytes(32).toString('base64
/** Make a Git token; the raw value is returned once and never stored. */
/** Make a Git token; the raw value is returned once and never stored. */
async function mintToken(store, { subject, actorType = 'user', scopes = ['git.repo.read'], hours = 1, label = null }) {
async function mintToken(store, { subject, actorType = 'user', scopes = ['git.repo.read'], hours = 1, label = null }) {
const wanted = [...new Set((Array.isArray(scopes) ? scopes : [scopes]).filter((s) => SCOPES.includes(s)))];
const wanted = [...new Set((Array.isArray(scopes) ? scopes : [scopes]).filter((s) => SCOPES.includes(s)))];
- if (!wanted.length) throw new Error('a token needs git.repo.read or git.repo.write');
+ if (!wanted.length) throw new Error('a token needs git.repo.read, git.repo.write or git.checks.write');
const raw = newToken();
const raw = newToken();
const id = store.newId('tok');
const id = store.newId('tok');
const expiresAt = new Date(store.now() + Math.max(1, Math.min(24 * 30, Number(hours) || 1)) * 3600_000).toISOString();
const expiresAt = new Date(store.now() + Math.max(1, Math.min(24 * 30, Number(hours) || 1)) * 3600_000).toISOString();
new file mode 100644
new file mode 100644
@@ -0,0 +1,131 @@
@@ -0,0 +1,131 @@
+'use strict';
+
+/**
+ * Checks intake from GitHub, read-only: our pages show the truth today. For a mirrored repository it copies
+ * the check-runs of the head commits of the open GitHub pull requests and of the default branch into commit
+ * statuses, so a change page shows GitHub's CI until our own runners report.
+ *
+ * Few API calls: the default branch's head is read from our own mirror (free); the open PRs cost one call;
+ * check-runs are asked only for heads that moved since the last run. A repository is visited at most once
+ * every two minutes. The GitHub token is the mirror's own (config.github.token); it is sent, never logged.
+ * Off in drill mode, like the mirror poll.
+ */
+const MIN_MS = 2 * 60_000; // at most once every two minutes per repository
+const TICK_MS = 60_000; // the tick that looks for due repositories
+const CREATOR = 'github-intake';
+
+/** A GitHub check-run → a status, or null when it names nothing. */
+function mapRun(run) {
+ const context = String((run && run.name) || '').trim().slice(0, 255);
+ if (!context) return null;
+ let state = 'pending';
+ if (run.status === 'completed') {
+ switch (run.conclusion) {
+ case 'success': case 'neutral': case 'skipped': state = 'success'; break;
+ case 'failure': state = 'failure'; break;
+ default: state = 'error'; break; // cancelled, timed_out, action_required, stale, …
+ }
+ }
+ const description = String((run.output && run.output.title) || run.conclusion || run.status || '').slice(0, 1024);
+ const target = String(run.html_url || run.details_url || '').slice(0, 2048);
+ return { context, state, description, target_url: /^https?:\/\//.test(target) ? target : '' };
+}
+
+function createChecksIntake({ store, checks, config, git, github, log = console }) {
+ /** Copy one mirrored repository's check-runs. force skips the two-minute throttle (tests, sync now). */
+ async function syncRepo(row, { force = false } = {}) {
+ if (!row || !row.mirror_url || !row.github_full_name) return { ok: false, skipped: 'not-mirrored' };
+ const full = row.github_full_name;
+ const sync = await store.maybe('SELECT * FROM git_checks_sync WHERE repo_id = ?', [row.id]);
+ const lastAt = sync && sync.last_run_at ? Date.parse(sync.last_run_at) : 0;
+ if (!force && lastAt && store.now() - lastAt < MIN_MS) return { ok: true, skipped: 'recent' };
+ const stored = (sync && typeof sync.heads === 'object' && sync.heads) || (sync && typeof sync.heads === 'string' ? JSON.parse(sync.heads || '{}') : {});
+ const summary = { repo: `${row.owner_username}/${row.name}`, heads: 0, fetched: 0, copied: 0, ok: true };
+
+ // The default branch's head from our own mirror: no API call.
+ const heads = {};
+ try {
+ const main = await git.resolveCommit(row.owner_username, row.name, row.default_branch || 'main');
+ if (main) heads.main = main;
+ } catch (err) {
+ log.warn(`[OpenVibe.Git] checks intake ${summary.repo}: ${String(err.message || err).slice(0, 200)}`);
+ }
+ let pulls = [];
+ try {
+ pulls = await github.openPulls(full);
+ } catch (err) {
+ summary.ok = false;
+ summary.error = String(err.message || err).slice(0, 200);
+ }
+ for (const pr of pulls) heads[`pr/${pr.number}`] = pr.head_sha;
+ summary.heads = Object.keys(heads).length;
+
+ for (const [key, sha] of Object.entries(heads)) {
+ if (stored[key] === sha) continue; // only when the head moved
+ let runs = [];
+ try {
+ runs = await github.checkRuns(full, sha);
+ summary.fetched += 1;
+ } catch (err) {
+ summary.ok = false;
+ summary.error = String(err.message || err).slice(0, 200);
+ continue;
+ }
+ for (const run of runs) {
+ const status = mapRun(run);
+ if (!status) continue;
+ try {
+ const { changed } = await checks.recordIfChanged(row.id, sha, status, CREATOR);
+ if (changed) summary.copied += 1;
+ } catch (err) {
+ log.warn(`[OpenVibe.Git] checks intake ${summary.repo}: ${String(err.message || err).slice(0, 200)}`);
+ }
+ }
+ stored[key] = sha;
+ }
+ // Heads that are gone (a closed PR) leave the map; a head whose fetch failed keeps no entry, so
+ // the next run retries it. The throttle still applies after an error.
+ const next = {};
+ for (const key of Object.keys(heads)) if (stored[key]) next[key] = stored[key];
+ await store.execute(
+ `INSERT INTO git_checks_sync (repo_id, last_run_at, heads, last_result, updated_at)
+ VALUES (?, now(), ?, ?, now())
+ ON CONFLICT (repo_id) DO UPDATE SET last_run_at = now(), heads = EXCLUDED.heads,
+ last_result = EXCLUDED.last_result, updated_at = now()`,
+ [row.id, JSON.stringify(next), JSON.stringify(summary)],
+ );
+ return summary;
+ }
+
+ /** Every mirrored repository that is due, two at a time. */
+ async function tick() {
+ const rows = await store.many(
+ `SELECT r.* FROM git_repos r LEFT JOIN git_checks_sync s ON s.repo_id = r.id
+ WHERE r.mirror_url IS NOT NULL AND r.github_full_name IS NOT NULL
+ ORDER BY s.last_run_at ASC NULLS FIRST LIMIT 200`,
+ );
+ const out = [];
+ for (let i = 0; i < rows.length; i += 2) {
+ const batch = await Promise.all(rows.slice(i, i + 2).map((r) => syncRepo(r).catch((err) => {
+ log.warn(`[OpenVibe.Git] checks intake ${r.owner_username}/${r.name}: ${String(err.message || err).slice(0, 200)}`);
+ return { ok: false, error: String(err.message || err).slice(0, 200) };
+ })));
+ out.push(...batch);
+ }
+ return out;
+ }
+
+ let timer = null;
+ return {
+ syncRepo, tick, mapRun, MIN_MS, TICK_MS,
+ start() {
+ if (timer || config.drill) return false;
+ timer = setInterval(() => { tick().catch((err) => log.warn('[OpenVibe.Git] checks intake:', err.message)); }, TICK_MS);
+ timer.unref();
+ return true;
+ },
+ stop() { if (timer) clearInterval(timer); timer = null; },
+ };
+}
+
+module.exports = { createChecksIntake, mapRun, MIN_MS, TICK_MS };
new file mode 100644
new file mode 100644
@@ -0,0 +1,146 @@
@@ -0,0 +1,146 @@
+'use strict';
+
+/**
+ * Commit statuses: what CI says about a commit. A service with git.checks.write reports
+ * {context, state, description, target_url} per commit; reads take the latest row per context.
+ *
+ * A change is mergeable when every required context is green. Required contexts are a per-repository list,
+ * empty by default, so nothing changes until an owner sets one.
+ */
+const STATES = Object.freeze(['pending', 'success', 'failure', 'error']);
+const SHA_RE = /^[0-9a-f]{40}([0-9a-f]{24})?$/;
+// A context names a CI verdict ("test", "security/secrets"): visible characters, no control bytes.
+const CONTEXT_RE = /^[^\u0000-\u001f\u007f]{1,255}$/;
+
+const fail = (status, code, message) => Object.assign(new Error(message), { status, code });
+
+function cleanSha(sha) {
+ const value = String(sha || '').toLowerCase();
+ if (!SHA_RE.test(value)) throw fail(422, 'status.sha_invalid', 'A status names the full commit id.');
+ return value;
+}
+
+function cleanStatus(body) {
+ const context = String((body && body.context) || '').trim();
+ if (!CONTEXT_RE.test(context)) throw fail(422, 'status.context_invalid', 'A context is 1–255 visible characters.');
+ const state = String((body && body.state) || '');
+ if (!STATES.includes(state)) throw fail(422, 'status.state_invalid', 'A state is pending, success, failure or error.');
+ const description = String((body && body.description) || '');
+ if (description.length > 1024) throw fail(422, 'status.description_invalid', 'A description is at most 1024 characters.');
+ const targetUrl = String((body && (body.target_url || body.targetUrl)) || '');
+ if (targetUrl.length > 2048) throw fail(422, 'status.target_url_invalid', 'A target URL is at most 2048 characters.');
+ if (targetUrl && !/^https?:\/\/[^/\s]+\//.test(`${targetUrl}/`)) throw fail(422, 'status.target_url_invalid', 'A target URL is http(s).');
+ return { context, state, description, target_url: targetUrl };
+}
+
+function cleanRequired(value) {
+ const list = Array.isArray(value) ? value : null;
+ if (!list) throw fail(422, 'checks.required_invalid', 'Required checks are a list of contexts.');
+ const contexts = [...new Set(list.map((c) => String(c || '').trim()).filter(Boolean))];
+ if (contexts.length > 50) throw fail(422, 'checks.required_invalid', 'At most 50 required contexts.');
+ for (const context of contexts) {
+ if (!CONTEXT_RE.test(context)) throw fail(422, 'checks.required_invalid', 'A context is 1–255 visible characters.');
+ }
+ return contexts;
+}
+
+const rowToStatus = (row) => ({
+ context: row.context, state: row.state, description: row.description || '',
+ target_url: row.target_url || '', created_at: row.created_at,
+});
+
+function createChecks({ store, log = console }) {
+ /** Record one verdict. Returns the stored status. */
+ async function record(repoId, sha, body, creator = null) {
+ const clean = cleanStatus(body);
+ const id = store.newId('cst');
+ await store.execute(
+ 'INSERT INTO git_commit_statuses (id, repo_id, sha, context, state, description, target_url, creator_subject) VALUES (?, ?, ?, ?, ?, ?, ?, ?)',
+ [id, repoId, cleanSha(sha), clean.context, clean.state, clean.description, clean.target_url, creator],
+ );
+ const row = await store.one('SELECT * FROM git_commit_statuses WHERE id = ?', [id]);
+ return rowToStatus(row);
+ }
+
+ /** The latest verdict per context for a commit, oldest context first. */
+ async function latest(repoId, sha) {
+ const rows = await store.many(
+ `SELECT DISTINCT ON (context) * FROM git_commit_statuses
+ WHERE repo_id = ? AND sha = ? ORDER BY context, created_at DESC, id DESC`,
+ [repoId, cleanSha(sha)],
+ );
+ return rows.map(rowToStatus).sort((a, b) => (a.context < b.context ? -1 : 1));
+ }
+
+ /** Insert only when the latest verdict for the context differs; the intake calls this. */
+ async function recordIfChanged(repoId, sha, body, creator = null) {
+ const clean = cleanStatus(body);
+ const current = await store.maybe(
+ 'SELECT * FROM git_commit_statuses WHERE repo_id = ? AND sha = ? AND context = ? ORDER BY created_at DESC, id DESC LIMIT 1',
+ [repoId, cleanSha(sha), clean.context],
+ );
+ if (current && current.state === clean.state && (current.description || '') === clean.description && (current.target_url || '') === clean.target_url) {
+ return { status: rowToStatus(current), changed: false };
+ }
+ return { status: await record(repoId, sha, clean, creator), changed: true };
+ }
+
+ /** The combined verdict: a failure beats an error beats a pending beats success; null when there are none. */
+ function combined(statuses) {
+ if (!statuses.length) return null;
+ if (statuses.some((s) => s.state === 'failure')) return 'failure';
+ if (statuses.some((s) => s.state === 'error')) return 'error';
+ if (statuses.some((s) => s.state === 'pending')) return 'pending';
+ return 'success';
+ }
+
+ /** The one line a change page shows next to the checks. */
+ function label(statuses) {
+ if (!statuses.length) return 'No checks';
+ const failed = statuses.filter((s) => s.state === 'failure' || s.state === 'error').length;
+ const pending = statuses.filter((s) => s.state === 'pending').length;
+ const total = statuses.length;
+ const noun = total === 1 ? 'check' : 'checks';
+ if (failed) return `${failed} of ${total} ${noun} failed`;
+ if (pending) return `${pending} of ${total} ${noun} pending`;
+ return `${total} ${noun} passed`;
+ }
+
+ function requiredFor(row) {
+ try {
+ const list = typeof row.required_checks === 'string' ? JSON.parse(row.required_checks || '[]') : (row.required_checks || []);
+ return Array.isArray(list) ? list.filter((c) => typeof c === 'string') : [];
+ } catch (err) {
+ log.warn('[OpenVibe.Git] required checks not read:', err.message);
+ return [];
+ }
+ }
+
+ async function setRequired(row, value) {
+ const contexts = cleanRequired(value);
+ await store.execute('UPDATE git_repos SET required_checks = ?, updated_at = now() WHERE id = ?', [JSON.stringify(contexts), row.id]);
+ return contexts;
+ }
+
+ /**
+ * Whether a commit may merge: every required context green. Empty requirements merge. A fork's head is
+ * read from its own repository (repoId); the required list always comes from the target.
+ */
+ async function mergeable(row, sha, { repoId } = {}) {
+ const required = requiredFor(row);
+ const checks = await latest(repoId || row.id, sha);
+ const byContext = new Map(checks.map((s) => [s.context, s]));
+ const missing = required.filter((c) => !byContext.has(c) || byContext.get(c).state !== 'success');
+ return {
+ mergeable: missing.length === 0,
+ state: combined(checks),
+ checks,
+ checks_label: label(checks),
+ required: required.map((context) => ({ context, state: byContext.has(context) ? byContext.get(context).state : null, ok: byContext.has(context) && byContext.get(context).state === 'success' })),
+ };
+ }
+
+ return { STATES, record, recordIfChanged, latest, combined, label, requiredFor, setRequired, mergeable, cleanStatus, cleanSha };
+}
+
+module.exports = { createChecks, STATES };
@@ -40,6 +40,7 @@ function load(env = process.env) {
@@ -40,6 +40,7 @@ function load(env = process.env) {
maxPushBytes: int(env.GIT_MAX_PUSH_BYTES, 512 * 1024 * 1024),
maxPushBytes: int(env.GIT_MAX_PUSH_BYTES, 512 * 1024 * 1024),
maxRepoBytes: int(env.GIT_MAX_REPO_BYTES, 3 * 1024 * 1024 * 1024),
maxRepoBytes: int(env.GIT_MAX_REPO_BYTES, 3 * 1024 * 1024 * 1024),
maxFileBytes: int(env.GIT_MAX_FILE_BYTES, 2 * 1024 * 1024),
maxFileBytes: int(env.GIT_MAX_FILE_BYTES, 2 * 1024 * 1024),
+ maxArchiveBytes: int(env.GIT_MAX_ARCHIVE_BYTES, 256 * 1024 * 1024),
},
},
// OpenVibe.Network: SSO (OAuth2 + PKCE), the JWKS for typed git tokens, and the account events.
// OpenVibe.Network: SSO (OAuth2 + PKCE), the JWKS for typed git tokens, and the account events.
@@ -112,4 +112,23 @@ async function runText(args, opts) {
@@ -112,4 +112,23 @@ async function runText(args, opts) {
return r.stdout.toString('utf8');
return r.stdout.toString('utf8');
}
}
-module.exports = { runGit, runText, configEnv, childEnv, GitError, FIXED_ENV };
+/**
+ * A streaming run (git archive): the caller pipes child.stdout and ends it. Same rules as runGit — an
+ * argument array, no shell, the fixed environment — plus a timeout that kills the child. The caller owns
+ * the child: kill it when the reader goes away.
+ */
+function spawnGit(args, { cwd, env, timeoutMs = 120_000 } = {}) {
+ if (!Array.isArray(args) || args.some((a) => typeof a !== 'string')) {
+ throw new TypeError('spawnGit: args must be an array of strings');
+ }
+ const child = spawn('git', args, { cwd, env: childEnv(env), shell: false, windowsHide: true });
+ const timer = setTimeout(() => {
+ try { child.kill('SIGKILL'); } catch { /* already gone */ }
+ }, timeoutMs);
+ timer.unref();
+ child.on('close', () => clearTimeout(timer));
+ child.on('error', () => clearTimeout(timer));
+ return child;
+}
+
+module.exports = { runGit, runText, spawnGit, configEnv, childEnv, GitError, FIXED_ENV };
@@ -2,7 +2,8 @@
@@ -2,7 +2,8 @@
/**
/**
* GitHub, read only: the repository card (stars, open issues and PRs, the last CI state), the metadata a
* GitHub, read only: the repository card (stars, open issues and PRs, the last CI state), the metadata a
- * mirror is created from, and the signature check on GitHub's push webhook.
+ * mirror is created from, the open PR heads and check-runs the checks intake copies, and the signature check on
+ * GitHub's push webhook.
*
*
* Every reading is cached for ten minutes (config.github.cacheMs), so a card costs our quota once per ten
* Every reading is cached for ten minutes (config.github.cacheMs), so a card costs our quota once per ten
* minutes, not once per view. The API token (for the OpenVibers repositories) is sent only to
* minutes, not once per view. The API token (for the OpenVibers repositories) is sent only to
@@ -224,6 +225,52 @@ function createGithubClient({ config, fetchImpl = globalThis.fetch, now = () =>
@@ -224,6 +225,52 @@ function createGithubClient({ config, fetchImpl = globalThis.fetch, now = () =>
return all;
return all;
}
}
+ /**
+ * The open pull requests' heads: [{ number, head_sha }]. One REST call per page (a few at most); the
+ * checks intake calls this, then asks about the check-runs of the heads that moved only.
+ */
+ async function openPulls(full) {
+ if (!FULL_RE.test(String(full || ''))) return [];
+ const [owner, name] = String(full).split('/');
+ const out = [];
+ let url = `${config.github.apiUrl}/repos/${encodeURIComponent(owner)}/${encodeURIComponent(name)}/pulls?state=open&per_page=100`;
+ for (let page = 0; page < 3 && url; page += 1) {
+ const res = await fetchImpl(url, { headers: headers(), signal: AbortSignal.timeout(15_000) });
+ if (!res.ok) {
+ const err = new Error(`GitHub ${res.status} for ${full} pulls`);
+ err.status = res.status;
+ throw err;
+ }
+ const batch = await res.json();
+ if (!Array.isArray(batch)) break;
+ for (const pr of batch) {
+ if (pr && pr.head && /^[0-9a-f]{40}$/i.test(pr.head.sha || '')) out.push({ number: pr.number, head_sha: String(pr.head.sha).toLowerCase() });
+ }
+ if (batch.length < 100) break;
+ // The next page stays on the API origin (nextLink refuses anything else).
+ url = nextLink(res.headers && typeof res.headers.get === 'function' ? res.headers.get('link') : null);
+ }
+ return out;
+ }
+
+ /** The check-runs GitHub reports for one commit (at most 100): the checks intake copies these. */
+ async function checkRuns(full, sha) {
+ if (!FULL_RE.test(String(full || ''))) return [];
+ if (!/^[0-9a-f]{40}$/i.test(String(sha || ''))) return [];
+ const [owner, name] = String(full).split('/');
+ const res = await fetchImpl(
+ `${config.github.apiUrl}/repos/${encodeURIComponent(owner)}/${encodeURIComponent(name)}/commits/${encodeURIComponent(String(sha).toLowerCase())}/check-runs?per_page=100`,
+ { headers: headers(), signal: AbortSignal.timeout(15_000) },
+ );
+ if (!res.ok) {
+ const err = new Error(`GitHub ${res.status} for ${full} check-runs`);
+ err.status = res.status;
+ throw err;
+ }
+ const body = await res.json();
+ return Array.isArray(body && body.check_runs) ? body.check_runs : [];
+ }
+
/** GitHub's X-Hub-Signature-256 over the raw body, constant time. */
/** GitHub's X-Hub-Signature-256 over the raw body, constant time. */
function signatureOk(rawBody, signature) {
function signatureOk(rawBody, signature) {
const secret = config.github.webhookSecret;
const secret = config.github.webhookSecret;
@@ -234,7 +281,7 @@ function createGithubClient({ config, fetchImpl = globalThis.fetch, now = () =>
@@ -234,7 +281,7 @@ function createGithubClient({ config, fetchImpl = globalThis.fetch, now = () =>
return a.length === b.length && crypto.timingSafeEqual(a, b);
return a.length === b.length && crypto.timingSafeEqual(a, b);
}
}
- return { parseGithubUrl, card, meta, probe, orgRepos, signatureOk, _cache: cache };
+ return { parseGithubUrl, card, meta, probe, orgRepos, openPulls, checkRuns, signatureOk, _cache: cache };
}
}
module.exports = { createGithubClient, parseGithubUrl, parseRefs, pktLine, FULL_RE };
module.exports = { createGithubClient, parseGithubUrl, parseRefs, pktLine, FULL_RE };
@@ -1,8 +1,8 @@
@@ -1,8 +1,8 @@
'use strict';
'use strict';
/**
/**
- * /api/v1: repositories, browsing, stars and forks, contents writes, clone tokens, the home numbers and the
- * GitHub webhook. Everything a page shows is here too; nothing here needs a browser.
+ * /api/v1: repositories, browsing, archives, stars and forks, contents writes, commit statuses, clone
+ * tokens, the home numbers and the GitHub webhook. Everything a page shows is here too; nothing here needs a browser.
*
*
* Public reads need no sign-in. A write needs the session or a token with git.repo.write. A private repository
* Public reads need no sign-in. A write needs the session or a token with git.repo.write. A private repository
* answers 404 to anyone who may not see it, never 403: its existence is not public.
* answers 404 to anyone who may not see it, never 403: its existence is not public.
@@ -11,6 +11,7 @@ const express = require('express');
@@ -11,6 +11,7 @@ const express = require('express');
const contracts = require('openvibe-contracts');
const contracts = require('openvibe-contracts');
const { callerOf, viewerFrom } = require('./identity');
const { callerOf, viewerFrom } = require('./identity');
const { mintToken } = require('../auth/git-auth');
const { mintToken } = require('../auth/git-auth');
+const { serveArchive } = require('../archive');
const problem = (req, res, status, code, detail) => contracts.http.sendProblem(res, status, code, { detail, ctx: req.ov });
const problem = (req, res, status, code, detail) => contracts.http.sendProblem(res, status, code, { detail, ctx: req.ov });
const fail = (req, res, err) => problem(req, res, err.status || 500, err.code || 'request.failed', err.status ? err.message : undefined);
const fail = (req, res, err) => problem(req, res, err.status || 500, err.code || 'request.failed', err.status ? err.message : undefined);
@@ -18,7 +19,7 @@ const fail = (req, res, err) => problem(req, res, err.status || 500, err.code ||
@@ -18,7 +19,7 @@ const fail = (req, res, err) => problem(req, res, err.status || 500, err.code ||
function boolParam(v) { return v === true || v === '1' || v === 'true' || v === 'yes'; }
function boolParam(v) { return v === true || v === '1' || v === 'true' || v === 'yes'; }
function createApi(ctx) {
function createApi(ctx) {
- const { config, repos, mirrors, git, insights, store, sso } = ctx;
+ const { config, repos, mirrors, git, insights, store, sso, checks, archive, log } = ctx;
const r = express.Router();
const r = express.Router();
const h = (fn) => (req, res, next) => Promise.resolve(fn(req, res)).catch(next);
const h = (fn) => (req, res, next) => Promise.resolve(fn(req, res)).catch(next);
// One identity read per request: the session, or a token (a person's or a service's, and for a service the
// One identity read per request: the session, or a token (a person's or a service's, and for a service the
@@ -186,6 +187,14 @@ function createApi(ctx) {
@@ -186,6 +187,14 @@ function createApi(ctx) {
return res.end(blob.data);
return res.end(blob.data);
}));
}));
+ r.get('/repos/:owner/:name/archive/*', h(async (req, res) => {
+ const found = await visible(req, req.params.owner, req.params.name);
+ if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');
+ await serveArchive(archive, req, res, found.row, req.params[0], {
+ log, fail: (status, message) => problem(req, res, status, 'archive.not_found', message),
+ });
+ }));
+
r.get('/repos/:owner/:name/commits', h(async (req, res) => {
r.get('/repos/:owner/:name/commits', h(async (req, res) => {
const found = await visible(req, req.params.owner, req.params.name);
const found = await visible(req, req.params.owner, req.params.name);
if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');
if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');
@@ -210,6 +219,51 @@ function createApi(ctx) {
@@ -210,6 +219,51 @@ function createApi(ctx) {
});
});
}));
}));
+ // ── Commit statuses ───────────────────────────────────
+ r.get('/repos/:owner/:name/commits/:sha/statuses', h(async (req, res) => {
+ const found = await visible(req, req.params.owner, req.params.name);
+ if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');
+ try {
+ const sha = checks.cleanSha(req.params.sha);
+ const statuses = await checks.latest(found.row.id, sha);
+ return res.json({ repo: `${found.row.owner_username}/${found.row.name}`, sha, state: checks.combined(statuses), statuses });
+ } catch (err) { return fail(req, res, err); }
+ }));
+
+ r.post('/repos/:owner/:name/commits/:sha/statuses', h(async (req, res) => {
+ const caller = await callerFor(req);
+ const found = await visible(req, req.params.owner, req.params.name);
+ if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');
+ if (!caller) return problem(req, res, 401, 'auth.required', 'A service token with git.checks.write reports a status.');
+ if (!caller.scopes.has('git.checks.write')) return problem(req, res, 403, 'token.scope', 'This token may not report checks (git.checks.write).');
+ const who = actingAs(caller);
+ if (caller.actor_type !== 'service' && !repos.isOwner(found.row, who)) return problem(req, res, 403, 'repo.forbidden', 'Only the owner or a checks service may report.');
+ try {
+ const sha = checks.cleanSha(req.params.sha);
+ const status = await checks.record(found.row.id, sha, req.body, caller.subject);
+ await repos.audit(who, 'git.status', found.row, { sha, context: status.context, state: status.state, via: caller.actor_type === 'service' ? caller.subject : 'api' }, req.ip);
+ return res.status(201).json(status);
+ } catch (err) { return fail(req, res, err); }
+ }));
+
+ r.get('/repos/:owner/:name/required-checks', h(async (req, res) => {
+ const found = await visible(req, req.params.owner, req.params.name);
+ if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');
+ return res.json({ repo: `${found.row.owner_username}/${found.row.name}`, required_checks: checks.requiredFor(found.row) });
+ }));
+
+ r.post('/repos/:owner/:name/required-checks', h(async (req, res) => {
+ const caller = await callerFor(req);
+ const found = await visible(req, req.params.owner, req.params.name);
+ if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');
+ if (!caller || !repos.isOwner(found.row, actingAs(caller))) return problem(req, res, 403, 'repo.forbidden', 'Only the owner can require checks.');
+ try {
+ const required = await checks.setRequired(found.row, (req.body || {}).contexts ?? (req.body || {}).required_checks);
+ await repos.audit(actingAs(caller), 'git.required_checks', found.row, { required_checks: required }, req.ip);
+ return res.json({ repo: `${found.row.owner_username}/${found.row.name}`, required_checks: required });
+ } catch (err) { return fail(req, res, err); }
+ }));
+
r.get('/repos/:owner/:name/branches', h(async (req, res) => {
r.get('/repos/:owner/:name/branches', h(async (req, res) => {
const found = await visible(req, req.params.owner, req.params.name);
const found = await visible(req, req.params.owner, req.params.name);
if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');
if (!found) return problem(req, res, 404, 'repo.not_found', 'No such repository.');
@@ -12,6 +12,7 @@
@@ -12,6 +12,7 @@
* /:owner/:name/tree a directory (ref and path in the query)
* /:owner/:name/tree a directory (ref and path in the query)
* /:owner/:name/blob a file, highlighted on the server, with line links
* /:owner/:name/blob a file, highlighted on the server, with line links
* /:owner/:name/raw its bytes
* /:owner/:name/raw its bytes
+ * /:owner/:name/archive/<ref>.tar.gz a gzip tarball of a ref, stable for a tag
* /:owner/:name/commits history (optionally of one path)
* /:owner/:name/commits history (optionally of one path)
* /:owner/:name/commit/:sha one commit and its diff
* /:owner/:name/commit/:sha one commit and its diff
* /:owner/:name/branches · tags · releases · contributors · insights
* /:owner/:name/branches · tags · releases · contributors · insights
@@ -26,9 +27,10 @@ const { viewerOf } = require('./identity');
@@ -26,9 +27,10 @@ const { viewerOf } = require('./identity');
const { esc } = require('../render/html');
const { esc } = require('../render/html');
const { renderMarkdown } = require('../render/markdown');
const { renderMarkdown } = require('../render/markdown');
const { validRef, validPath } = require('../git/refs');
const { validRef, validPath } = require('../git/refs');
+const { serveArchive } = require('../archive');
function createPageRoutes(ctx) {
function createPageRoutes(ctx) {
- const { config, repos, git, github, mirrors, insights, store, relay } = ctx;
+ const { config, repos, git, github, mirrors, insights, store, relay, archive, log } = ctx;
const r = express.Router();
const r = express.Router();
r.use(express.urlencoded({ extended: false, limit: '256kb' }));
r.use(express.urlencoded({ extended: false, limit: '256kb' }));
const h = (fn) => (req, res, next) => Promise.resolve(fn(req, res)).catch(next);
const h = (fn) => (req, res, next) => Promise.resolve(fn(req, res)).catch(next);
@@ -330,6 +332,14 @@ function createPageRoutes(ctx) {
@@ -330,6 +332,14 @@ function createPageRoutes(ctx) {
return res.end(blob.data);
return res.end(blob.data);
}));
}));
+ r.get('/:owner/:name/archive/*', h(async (req, res) => {
+ const row = await rowFor(req, res);
+ if (!row) return undefined;
+ await serveArchive(archive, req, res, row, req.params[0], {
+ log, fail: (status, message) => res.status(status).type('text/plain').send(`${message}\n`),
+ });
+ }));
+
r.get('/:owner/:name/commits', h(async (req, res) => {
r.get('/:owner/:name/commits', h(async (req, res) => {
const row = await rowFor(req, res);
const row = await rowFor(req, res);
if (!row) return undefined;
if (!row) return undefined;
@@ -452,7 +462,7 @@ function createPageRoutes(ctx) {
@@ -452,7 +462,7 @@ function createPageRoutes(ctx) {
const official = await repos.official();
const official = await repos.official();
const lines = [
const lines = [
'# OpenVibe Git', '', '> Git for OpenVibe: host code with your OpenVibe account, browse it, mirror GitHub, and clone it to your computer with Link.', '',
'# OpenVibe Git', '', '> Git for OpenVibe: host code with your OpenVibe account, browse it, mirror GitHub, and clone it to your computer with Link.', '',
- 'Every page is server-rendered. A public repository is readable at /<owner>/<name>; its files at /<owner>/<name>/tree and /<owner>/<name>/blob (ref and path are query parameters). The API is /api/v1/repos.', '',
+ 'Every page is server-rendered. A public repository is readable at /<owner>/<name>; its files at /<owner>/<name>/tree and /<owner>/<name>/blob (ref and path are query parameters); a ref as a tarball at /<owner>/<name>/archive/<ref>.tar.gz. The API is /api/v1/repos.', '',
'## Official repositories', '',
'## Official repositories', '',
...official.map((o) => `- [${o.full_name}](${config.baseUrl.replace(/\/+$/, '')}/${o.full_name}): ${o.description || 'The OpenVibe projects.'}`), '',
...official.map((o) => `- [${o.full_name}](${config.baseUrl.replace(/\/+$/, '')}/${o.full_name}): ${o.description || 'The OpenVibe projects.'}`), '',
];
];
@@ -474,6 +484,8 @@ function createPageRoutes(ctx) {
@@ -474,6 +484,8 @@ function createPageRoutes(ctx) {
'/api/v1/repos/{owner}/{name}/tree': { get: { summary: 'A directory at a ref', responses: { 200: { description: 'Tree' } } } },
'/api/v1/repos/{owner}/{name}/tree': { get: { summary: 'A directory at a ref', responses: { 200: { description: 'Tree' } } } },
'/api/v1/repos/{owner}/{name}/blob': { get: { summary: 'A file at a ref', responses: { 200: { description: 'Blob' } } } },
'/api/v1/repos/{owner}/{name}/blob': { get: { summary: 'A file at a ref', responses: { 200: { description: 'Blob' } } } },
'/api/v1/repos/{owner}/{name}/commits': { get: { summary: 'History' }, post: { summary: 'Write a branch and files without a clone', responses: { 201: { description: 'The new commit' } } } },
'/api/v1/repos/{owner}/{name}/commits': { get: { summary: 'History' }, post: { summary: 'Write a branch and files without a clone', responses: { 201: { description: 'The new commit' } } } },
+ '/api/v1/repos/{owner}/{name}/archive/{ref}.tar.gz': { get: { summary: 'A gzip tarball of a ref (a tag answers the same bytes)', responses: { 200: { description: 'The tarball' } } } },
+ '/api/v1/repos/{owner}/{name}/commits/{sha}/statuses': { get: { summary: 'The latest status per context for a commit', responses: { 200: { description: 'Statuses' } } }, post: { summary: 'Report a status (git.checks.write)', responses: { 201: { description: 'The status' } } } },
'/api/v1/repos/{owner}/{name}/insights': { get: { summary: 'Views, clones, stars and commits per week', responses: { 200: { description: 'Insights' } } } },
'/api/v1/repos/{owner}/{name}/insights': { get: { summary: 'Views, clones, stars and commits per week', responses: { 200: { description: 'Insights' } } } },
'/api/v1/tokens': { post: { summary: 'Make a short-lived Git token', responses: { 201: { description: 'The token, shown once' } } } },
'/api/v1/tokens': { post: { summary: 'Make a short-lived Git token', responses: { 201: { description: 'The token, shown once' } } } },
},
},
@@ -2,8 +2,8 @@
@@ -2,8 +2,8 @@
/**
/**
* OpenVibe.Git — process entry. `node server/index.js` listens on PORT (5270) behind nginx (deploy/), starts the
* OpenVibe.Git — process entry. `node server/index.js` listens on PORT (5270) behind nginx (deploy/), starts the
- * mirror poll and the due-based official mirrors seed, and stops in order on SIGTERM: the HTTP drain, the timers,
- * the cat-file processes, the database.
+ * mirror poll, the checks intake and the due-based official mirrors seed, and stops in order on SIGTERM: the HTTP
+ * drain, the timers, the cat-file processes, the database.
*/
*/
const { createApp } = require('./app');
const { createApp } = require('./app');
const { gracefulStop } = require('openvibe-sdk/service');
const { gracefulStop } = require('openvibe-sdk/service');
@@ -12,6 +12,7 @@ const { catFiles } = require('./git/store');
@@ -12,6 +12,7 @@ const { catFiles } = require('./git/store');
function createLifecycle({ server, exit, signals, ctx } = {}) {
function createLifecycle({ server, exit, signals, ctx } = {}) {
const mirrors = ctx && ctx.mirrors;
const mirrors = ctx && ctx.mirrors;
const official = ctx && ctx.official;
const official = ctx && ctx.official;
+ const checksIntake = ctx && ctx.checksIntake;
const store = ctx && ctx.store;
const store = ctx && ctx.store;
return gracefulStop({
return gracefulStop({
name: 'OpenVibe.Git',
name: 'OpenVibe.Git',
@@ -20,7 +21,7 @@ function createLifecycle({ server, exit, signals, ctx } = {}) {
@@ -20,7 +21,7 @@ function createLifecycle({ server, exit, signals, ctx } = {}) {
exit,
exit,
signals,
signals,
deadlineMs: 10_000,
deadlineMs: 10_000,
- stop: [() => ctx.reviews && ctx.reviews.stop(), () => (mirrors ? mirrors.stop() : undefined), () => (official ? official.stop() : undefined)],
+ stop: [() => ctx.reviews && ctx.reviews.stop(), () => (mirrors ? mirrors.stop() : undefined), () => (official ? official.stop() : undefined), () => (checksIntake ? checksIntake.stop() : undefined)],
close: [() => catFiles.close(), () => (store ? store.close() : undefined)],
close: [() => catFiles.close(), () => (store ? store.close() : undefined)],
});
});
}
}
@@ -31,6 +32,7 @@ async function start() {
@@ -31,6 +32,7 @@ async function start() {
if (config.mirrors.pollMs) ctx.mirrors.start();
if (config.mirrors.pollMs) ctx.mirrors.start();
// Due-based and off unless GIT_OFFICIAL_ORG is set: the first check is shortly after boot, never in this path.
// Due-based and off unless GIT_OFFICIAL_ORG is set: the first check is shortly after boot, never in this path.
ctx.official.start();
ctx.official.start();
+ ctx.checksIntake.start();
ctx.reviews.start();
ctx.reviews.start();
const server = app.listen(config.port, config.host, () => {
const server = app.listen(config.port, config.host, () => {
console.log(`[OpenVibe.Git] ${config.nodeEnv} on http://${config.host}:${config.port} → ${config.baseUrl} (release ${release.release})`);
console.log(`[OpenVibe.Git] ${config.nodeEnv} on http://${config.host}:${config.port} → ${config.baseUrl} (release ${release.release})`);
@@ -5,7 +5,7 @@ const { runGit } = require('./git/run');
@@ -5,7 +5,7 @@ const { runGit } = require('./git/run');
const { fail } = plumbing;
const { fail } = plumbing;
const text = (v, max, required = true) => { if (typeof v !== 'string' || v.length > max || (required && !v.trim())) throw fail(422, 'Invalid text'); return v.trim(); };
const text = (v, max, required = true) => { if (typeof v !== 'string' || v.length > max || (required && !v.trim())) throw fail(422, 'Invalid text'); return v.trim(); };
function createReviews(ctx, services) {
function createReviews(ctx, services) {
- const { store: s, repos, git } = ctx;
+ const { store: s, repos, git, checks } = ctx;
let timer, running = false;
let timer, running = false;
const actorId = (a) => a && (a.on_behalf_of || a.subject);
const actorId = (a) => a && (a.on_behalf_of || a.subject);
const writable = (r, a) => !!a && a.scopes.has('git.repo.write') && repos.isOwner(r, { subject: actorId(a) });
const writable = (r, a) => !!a && a.scopes.has('git.repo.write') && repos.isOwner(r, { subject: actorId(a) });
@@ -71,7 +71,9 @@ function createReviews(ctx, services) {
@@ -71,7 +71,9 @@ function createReviews(ctx, services) {
const comments = await s.many('SELECT * FROM git_change_comments WHERE change_id = ? ORDER BY created_at', [c.id]);
const comments = await s.many('SELECT * FROM git_change_comments WHERE change_id = ? ORDER BY created_at', [c.id]);
const approvals = await s.many('SELECT * FROM git_change_reviews WHERE change_id = ? AND head_sha = ?', [c.id,diff.head]);
const approvals = await s.many('SELECT * FROM git_change_reviews WHERE change_id = ? AND head_sha = ?', [c.id,diff.head]);
const summary = await s.maybe('SELECT result FROM git_change_summaries WHERE change_id = ? AND head_sha = ? AND model = ?', [c.id,diff.head,r.review_model || '']);
const summary = await s.maybe('SELECT result FROM git_change_summaries WHERE change_id = ? AND head_sha = ? AND model = ?', [c.id,diff.head,r.review_model || '']);
- return { change: c, diff, comments, approvals, checks: [], checks_label: 'No checks', summary: summary ? summary.result : null };
+ // A fork's head is read from the fork (its CI posts there); the required list is the target's.
+ const verdict = await checks.mergeable(r, diff.head, { repoId: source.id });
+ return { change: c, diff, comments, approvals, ...verdict, summary: summary ? summary.result : null };
}
}
async function action(r, kind, item, op, body, a, viewer) {
async function action(r, kind, item, op, body, a, viewer) {
auth(a);
auth(a);
@@ -109,6 +111,9 @@ function createReviews(ctx, services) {
@@ -109,6 +111,9 @@ function createReviews(ctx, services) {
write(r,a);
write(r,a);
const frozen = await plumbing.compare(r,item.base_ref,item.head_ref,source,true);
const frozen = await plumbing.compare(r,item.base_ref,item.head_ref,source,true);
if (frozen.head !== head) throw fail(409,'Head changed while reading diff');
if (frozen.head !== head) throw fail(409,'Head changed while reading diff');
+ // Required checks gate the merge; an empty list merges as before.
+ const verdict = await checks.mergeable(r, head, { repoId: source.id });
+ if (!verdict.mergeable) throw fail(409, 'Required checks are not passing');
await retain(r,item,frozen);
await retain(r,item,frozen);
const sha = await plumbing.merge(r,source,item,body.method || 'merge',head,a,frozen.base);
const sha = await plumbing.merge(r,source,item,body.method || 'merge',head,a,frozen.base);
await s.execute("UPDATE git_changes SET state = 'merged', head_sha = ?, merge_sha = ?, closed_at = ? WHERE id = ?", [head,sha,s.iso(),item.id]);
await s.execute("UPDATE git_changes SET state = 'merged', head_sha = ?, merge_sha = ?, closed_at = ? WHERE id = ?", [head,sha,s.iso(),item.id]);
@@ -75,8 +75,14 @@ function change(r, out, canWrite, signed) {
@@ -75,8 +75,14 @@ function change(r, out, canWrite, signed) {
const action = (op, inner) => `<form class="git-inline-form" method="post" action="${b}/${op}">${hidden('head_sha', out.diff.head)}${inner}</form>`;
const action = (op, inner) => `<form class="git-inline-form" method="post" action="${b}/${op}">${hidden('head_sha', out.diff.head)}${inner}</form>`;
const outdated = out.comments.filter((m) => m.head_sha !== out.diff.head);
const outdated = out.comments.filter((m) => m.head_sha !== out.diff.head);
const approvals = out.approvals.length;
const approvals = out.approvals.length;
+ const checkItems = (out.checks || []).map((s) => {
+ const name = s.target_url ? `<a href="${esc(s.target_url)}" rel="noopener">${esc(s.context)}</a>` : `<span>${esc(s.context)}</span>`;
+ return `<li class="git-check git-check--${esc(s.state)}">${ic(s.state === 'success' ? 'check' : s.state === 'pending' ? 'eye' : 'shield')}${name}<span class="git-muted">${esc(s.state)}</span></li>`;
+ }).join('');
+ const requiredNote = (out.required || []).length && !out.mergeable ? '<p class="git-muted">Required checks are not passing.</p>' : '';
const box = `<section class="git-card git-merge-box" aria-labelledby="git-merge-h"><h2 id="git-merge-h">Review</h2>
const box = `<section class="git-card git-merge-box" aria-labelledby="git-merge-h"><h2 id="git-merge-h">Review</h2>
<ul class="git-status"><li class="${approvals ? 'is-ok' : ''}">${ic(approvals ? 'check' : 'eye')}<span>${approvals ? `${plural(approvals, 'approval')} for this commit` : 'No approvals for this commit yet'}</span></li><li>${ic('shield')}<span>${esc(out.checks_label)}</span></li></ul>
<ul class="git-status"><li class="${approvals ? 'is-ok' : ''}">${ic(approvals ? 'check' : 'eye')}<span>${approvals ? `${plural(approvals, 'approval')} for this commit` : 'No approvals for this commit yet'}</span></li><li>${ic('shield')}<span>${esc(out.checks_label)}</span></li></ul>
+${checkItems ? `<ul class="git-checks">${checkItems}</ul>` : ''}${requiredNote}
${open && (signed || canWrite) ? `<div class="git-merge-acts">${signed ? action('approve', submit('Approve', 'quiet', 'check')) : ''}
${open && (signed || canWrite) ? `<div class="git-merge-acts">${signed ? action('approve', submit('Approve', 'quiet', 'check')) : ''}
${canWrite ? action('merge', `<label class="git-field git-field--inline"><span>How</span><select name="method"><option value="merge">Merge commit</option><option value="squash">Squash</option><option value="rebase">Rebase</option></select></label>${submit('Merge', 'primary', 'branch')}`) : ''}
${canWrite ? action('merge', `<label class="git-field git-field--inline"><span>How</span><select name="method"><option value="merge">Merge commit</option><option value="squash">Squash</option><option value="rebase">Rebase</option></select></label>${submit('Merge', 'primary', 'branch')}`) : ''}
${signed ? action('close', `<button class="git-close" type="submit">Close this change</button>`) : ''}</div>` : `<p class="git-muted">${open ? 'Sign in to review this change.' : `This change is ${esc(STATE[c.state] || c.state).toLowerCase()}.`}</p>`}</section>`;
${signed ? action('close', `<button class="git-close" type="submit">Close this change</button>`) : ''}</div>` : `<p class="git-muted">${open ? 'Sign in to review this change.' : `This change is ${esc(STATE[c.state] || c.state).toLowerCase()}.`}</p>`}</section>`;
new file mode 100644
new file mode 100644
@@ -0,0 +1,124 @@
@@ -0,0 +1,124 @@
+'use strict';
+/**
+ * Tarballs: /<owner>/<name>/archive/<ref>.tar.gz (and the API form) streams git archive of the ref. The
+ * ETag is the commit; a tag answers the same bytes twice with immutable caching; a branch caches short.
+ * A package manager can pin the address: npm installs a fixture package from the test server's own URL
+ * (loopback only, no network).
+ */
+const assert = require('assert');
+const crypto = require('node:crypto');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const zlib = require('node:zlib');
+const { execFile } = require('node:child_process');
+const { boot } = require('./helpers/boot');
+
+/** The entry names in a tar.gz buffer, parsed without leaving the process. */
+function tarNames(tgz) {
+ const tar = zlib.gunzipSync(tgz);
+ const names = [];
+ let off = 0;
+ while (off + 512 <= tar.length) {
+ const name = tar.subarray(off, off + 100).toString('utf8').split('\0')[0];
+ if (!name) break;
+ names.push(name);
+ const size = parseInt(tar.subarray(off + 124, off + 136).toString('utf8').trim() || '0', 8);
+ off += 512 + Math.ceil(size / 512) * 512;
+ }
+ return names;
+}
+
+const sha256 = (buf) => crypto.createHash('sha256').update(buf).digest('hex');
+
+(async () => {
+ const t = await boot();
+ try {
+ const sam = t.asUser('sam');
+ await t.ctx.repos.create({ actor: sam, name: 'pkg', starter: 'empty', visibility: 'public' });
+ const made = await t.ctx.git.commitFiles('sam', 'pkg', {
+ branch: 'main', base: 'main', message: 'A tiny package',
+ files: [
+ { path: 'package.json', content: JSON.stringify({ name: 'fixture-pkg', version: '1.0.0', main: 'index.js' }) },
+ { path: 'index.js', content: 'module.exports = 42;\n' },
+ ],
+ author: { name: 'Sam', email: '[email protected]' },
+ });
+ // An empty starter's first commit needs its branch: commitFiles made main from nothing.
+ const sha = made.sha;
+ await t.git(['tag', 'v1.0.0', sha], t.ctx.git.repoDir('sam', 'pkg'));
+
+ const page = await t.get('/sam/pkg/archive/v1.0.0.tar.gz');
+ assert.equal(page.status, 200, page.text.slice(0, 200));
+ assert.match(page.headers.get('content-type'), /application\/gzip/);
+ assert.match(page.headers.get('content-disposition'), /attachment; filename="pkg-v1\.0\.0\.tar\.gz"/);
+ assert.equal(page.headers.get('etag'), `"${sha}-tar.gz"`);
+ assert.equal(page.headers.get('cache-control'), 'public, max-age=31536000, immutable');
+ const names = tarNames(page.buffer);
+ assert.ok(names.includes(`pkg-${sha}/package.json`), names.join(','));
+ assert.ok(names.includes(`pkg-${sha}/index.js`));
+ // git archive prepends a pax_global_header with the commit id; everything else sits under one folder.
+ assert.ok(names.every((n) => n === 'pax_global_header' || n.startsWith(`pkg-${sha}/`)), `one top folder: ${names.join(',')}`);
+
+ // A tag's bytes are stable: the same address twice, and the API form, hash alike.
+ const again = await t.get('/sam/pkg/archive/v1.0.0.tar.gz');
+ assert.equal(sha256(again.buffer), sha256(page.buffer));
+ const api = await t.get('/api/v1/repos/sam/pkg/archive/v1.0.0.tar.gz');
+ assert.equal(api.status, 200, api.text.slice(0, 200));
+ assert.equal(sha256(api.buffer), sha256(page.buffer));
+
+ // A full commit id caches immutably too; a branch caches short.
+ const byId = await t.get(`/sam/pkg/archive/${sha}.tar.gz`);
+ assert.equal(byId.status, 200);
+ assert.equal(byId.headers.get('cache-control'), 'public, max-age=31536000, immutable');
+ const branch = await t.get('/sam/pkg/archive/main.tar.gz');
+ assert.equal(branch.status, 200);
+ assert.equal(branch.headers.get('cache-control'), 'public, max-age=60');
+ assert.equal(branch.headers.get('etag'), `"${sha}-tar.gz"`);
+
+ // A branch with a slash archives under its own name.
+ await t.ctx.git.commitFiles('sam', 'pkg', {
+ branch: 'agent/run-1', base: 'main', message: 'A slash branch',
+ files: [{ path: 'slash.txt', content: 'slashed\n' }],
+ author: { name: 'Sam', email: '[email protected]' },
+ });
+ const slashed = await t.get('/sam/pkg/archive/agent/run-1.tar.gz');
+ assert.equal(slashed.status, 200, slashed.text.slice(0, 200));
+ assert.match(slashed.headers.get('content-disposition'), /filename="pkg-agent-run-1\.tar\.gz"/);
+ assert.ok(tarNames(slashed.buffer).some((n) => n.endsWith('/slash.txt')));
+
+ // Conditional reads, unknown refs and the suffix rule.
+ assert.equal((await t.get('/sam/pkg/archive/v1.0.0.tar.gz', { headers: { 'if-none-match': `"${sha}-tar.gz"` } })).status, 304);
+ assert.equal((await t.get('/sam/pkg/archive/nope.tar.gz')).status, 404);
+ assert.equal((await t.get('/sam/pkg/archive/main.zip')).status, 404);
+ assert.equal((await t.get('/api/v1/repos/sam/pkg/archive/nope.tar.gz')).status, 404);
+
+ // Private repositories follow the raw file: 404 to strangers, a tarball to the owner.
+ await t.ctx.repos.create({ actor: sam, name: 'secret-pkg', starter: 'static', visibility: 'private' });
+ assert.equal((await t.get('/sam/secret-pkg/archive/main.tar.gz')).status, 404);
+ assert.equal((await t.get('/api/v1/repos/sam/secret-pkg/archive/main.tar.gz')).status, 404);
+ assert.equal((await t.get('/sam/secret-pkg/archive/main.tar.gz', { as: sam })).status, 200);
+ const token = await t.get('/api/v1/tokens', { as: sam, json: { scopes: ['git.repo.read'] } });
+ assert.equal((await t.get('/api/v1/repos/sam/secret-pkg/archive/main.tar.gz', { bearer: token.json().token })).status, 200);
+
+ // npm installs the fixture from the forge address, like a package.json pin would.
+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'openvibe-git-pin-'));
+ try {
+ const url = `${t.base}/sam/pkg/archive/v1.0.0.tar.gz`;
+ fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ name: 'pin-consumer', private: true, dependencies: { 'fixture-pkg': url } }));
+ await new Promise((resolve, reject) => {
+ execFile('npm', ['install', '--no-audit', '--no-fund', '--ignore-scripts', '--no-update-notifier'],
+ { cwd: dir, timeout: 90000, env: { ...process.env, HOME: dir, npm_config_cache: path.join(dir, 'cache') } },
+ (err, stdout, stderr) => (err ? reject(Object.assign(err, { stdout, stderr })) : resolve(stdout)));
+ });
+ const installed = JSON.parse(fs.readFileSync(path.join(dir, 'node_modules', 'fixture-pkg', 'package.json'), 'utf8'));
+ assert.equal(installed.name, 'fixture-pkg');
+ assert.equal(installed.version, '1.0.0');
+ assert.equal(require(path.join(dir, 'node_modules', 'fixture-pkg', 'index.js')), 42);
+ } finally {
+ fs.rmSync(dir, { recursive: true, force: true });
+ }
+
+ console.log('archive: page and API tarballs, stable tag bytes, ETag/cache, auth and an npm pin install passed');
+ } finally { await t.close(); }
+})().catch((e) => { console.error(e); process.exit(1); });
new file mode 100644
new file mode 100644
@@ -0,0 +1,98 @@
@@ -0,0 +1,98 @@
+'use strict';
+/**
+ * Checks intake from GitHub, read-only: for a mirrored repository it copies the check-runs of the open
+ * pull-request heads and the default branch into statuses. At most every two minutes per repository, and
+ * check-runs are asked only for heads that moved. GitHub is a canned fetch; nothing leaves the process.
+ */
+const assert = require('assert');
+const { boot } = require('./helpers/boot');
+const { mapRun } = require('../server/checks-intake');
+
+const ok = (body, link = null) => ({
+ ok: true, status: 200,
+ headers: { get: (name) => (String(name).toLowerCase() === 'link' ? link : null) },
+ json: async () => body,
+});
+
+(async () => {
+ const calls = [];
+ let prHead;
+ const runsBySha = {};
+ const fetchImpl = async (url) => {
+ calls.push(String(url));
+ if (String(url).includes('/pulls')) return ok([{ number: 7, head: { sha: prHead } }]);
+ const m = /\/commits\/([0-9a-f]{40})\/check-runs/.exec(String(url));
+ if (m) return ok({ check_runs: runsBySha[m[1]] || [] });
+ throw new Error(`unexpected GitHub call: ${url}`);
+ };
+ const t = await boot({ fetchImpl, env: { GIT_GITHUB_TOKEN: 'secret-mirror-token' } });
+ try {
+ assert.equal(t.ctx.checksIntake.start(), false, 'off in drill mode');
+
+ const sam = t.asUser('sam');
+ await t.ctx.repos.create({ actor: sam, name: 'live', starter: 'static', visibility: 'public' });
+ const mainSha = await t.ctx.git.resolveCommit('sam', 'live', 'main');
+ const pr = await t.ctx.git.commitFiles('sam', 'live', {
+ branch: 'pr-branch', base: 'main', message: 'The PR head', files: [{ path: 'pr.txt', content: 'pr\n' }],
+ author: { name: 'Sam', email: '[email protected]' },
+ });
+ prHead = pr.sha;
+ runsBySha[mainSha] = [
+ { name: 'test', status: 'completed', conclusion: 'success', html_url: 'https://github.com/OpenVibers/OpenVibe.Live/runs/1', output: { title: 'All green' } },
+ { name: 'security', status: 'completed', conclusion: 'failure', html_url: 'https://github.com/OpenVibers/OpenVibe.Live/runs/2', output: { title: 'Leak found' } },
+ ];
+ runsBySha[pr.sha] = [{ name: 'test', status: 'in_progress', conclusion: null, html_url: 'https://github.com/OpenVibers/OpenVibe.Live/runs/3' }];
+ await t.store.execute("UPDATE git_repos SET mirror_url = 'https://github.com/OpenVibers/OpenVibe.Live.git', github_full_name = 'OpenVibers/OpenVibe.Live' WHERE owner_username = 'sam' AND name = 'live'");
+ const row = await t.ctx.repos.getRow('sam', 'live');
+
+ const first = await t.ctx.checksIntake.syncRepo(row, { force: true });
+ assert.equal(first.ok, true, JSON.stringify(first));
+ assert.equal(first.heads, 2);
+ assert.equal(first.fetched, 2);
+ assert.equal(first.copied, 3);
+ assert.equal(calls.filter((u) => u.includes('/pulls')).length, 1, 'one PR listing');
+ assert.equal(calls.filter((u) => u.includes('/check-runs')).length, 2);
+
+ const main = await t.get(`/api/v1/repos/sam/live/commits/${mainSha}/statuses`);
+ assert.deepStrictEqual(main.json().statuses.map((s) => [s.context, s.state]), [['security', 'failure'], ['test', 'success']]);
+ assert.equal(main.json().state, 'failure');
+ assert.equal(main.json().statuses[0].target_url, 'https://github.com/OpenVibers/OpenVibe.Live/runs/2');
+ const prStatuses = await t.get(`/api/v1/repos/sam/live/commits/${pr.sha}/statuses`);
+ assert.deepStrictEqual(prStatuses.json().statuses.map((s) => [s.context, s.state]), [['test', 'pending']]);
+
+ // A second run inside two minutes does nothing; a forced run with unmoved heads asks no check-runs.
+ assert.equal((await t.ctx.checksIntake.syncRepo(await t.ctx.repos.getRow('sam', 'live'))).skipped, 'recent');
+ const before = calls.length;
+ const forced = await t.ctx.checksIntake.syncRepo(await t.ctx.repos.getRow('sam', 'live'), { force: true });
+ assert.equal(forced.fetched, 0, 'unmoved heads cost no check-runs call');
+ assert.equal(forced.copied, 0);
+ assert.equal(calls.length - before, 1, 'only the PR listing re-ran');
+
+ // A moved head is fetched again; an unchanged one is not.
+ const pr2 = await t.ctx.git.commitFiles('sam', 'live', {
+ branch: 'pr-branch', base: 'pr-branch', message: 'The PR moves', files: [{ path: 'pr.txt', content: 'pr2\n' }],
+ author: { name: 'Sam', email: '[email protected]' },
+ });
+ prHead = pr2.sha;
+ runsBySha[pr2.sha] = [{ name: 'test', status: 'completed', conclusion: 'success', html_url: 'https://github.com/x/runs/4' }];
+ const moved = await t.ctx.checksIntake.syncRepo(await t.ctx.repos.getRow('sam', 'live'), { force: true });
+ assert.equal(moved.fetched, 1);
+ assert.equal(moved.copied, 1);
+ assert.deepStrictEqual((await t.get(`/api/v1/repos/sam/live/commits/${pr2.sha}/statuses`)).json().statuses.map((s) => s.state), ['success']);
+
+ // A repository that is no mirror is left alone, and the token never reaches a log.
+ await t.ctx.repos.create({ actor: sam, name: 'plain', starter: 'static', visibility: 'public' });
+ assert.equal((await t.ctx.checksIntake.syncRepo(await t.ctx.repos.getRow('sam', 'plain'), { force: true })).skipped, 'not-mirrored');
+ assert.ok(!t.logs().join('\n').includes('secret-mirror-token'), 'the mirror token is never logged');
+
+ // The check-run mapping: queued work pends, neutral passes, a cancelled run errors.
+ assert.equal(mapRun({ name: 't', status: 'queued' }).state, 'pending');
+ assert.equal(mapRun({ name: 't', status: 'completed', conclusion: 'neutral' }).state, 'success');
+ assert.equal(mapRun({ name: 't', status: 'completed', conclusion: 'cancelled' }).state, 'error');
+ assert.equal(mapRun({ name: 't', status: 'completed', conclusion: 'timed_out' }).state, 'error');
+ assert.equal(mapRun({ name: '' }), null);
+ assert.equal(mapRun(null), null);
+
+ console.log('checks-intake: copy, throttle, moved-heads-only, mapping and token silence passed');
+ } finally { await t.close(); }
+})().catch((e) => { console.error(e); process.exit(1); });
new file mode 100644
new file mode 100644
@@ -0,0 +1,99 @@
@@ -0,0 +1,99 @@
+'use strict';
+/**
+ * Commit statuses: a checks service reports {context, state, description, target_url} per commit; reads
+ * take the latest per context. A change shows them and is mergeable only when its required contexts are
+ * green; an empty required list changes nothing.
+ */
+const assert = require('assert');
+const { boot } = require('./helpers/boot');
+
+(async () => {
+ const t = await boot();
+ try {
+ const sam = t.asUser('sam'), pat = t.asUser('pat');
+ await t.ctx.repos.create({ actor: sam, name: 'ci', starter: 'static', visibility: 'public' });
+ const head = await t.ctx.git.commitFiles('sam', 'ci', {
+ branch: 'agent/feat', base: 'main', message: 'A feature', files: [{ path: 'feat.txt', content: 'yes\n' }],
+ author: { name: 'Sam', email: '[email protected]' },
+ });
+ const root = '/api/v1/repos/sam/ci';
+ const runSvc = t.network.gitToken({ subject: 'svc:run', caps: ['git.checks.write'], type: null });
+ const readOnly = t.network.gitToken({ subject: 'svc:run', caps: ['git.repo.read'], type: null });
+ const post = (body, bearer = runSvc) => t.get(`${root}/commits/${head.sha}/statuses`, { bearer, json: body });
+
+ // Who may report: a checks service, or the owner with the capability. Nobody else.
+ assert.equal((await t.get(`${root}/commits/${head.sha}/statuses`, { json: { context: 'test', state: 'success' } })).status, 401);
+ assert.equal((await post({ context: 'test', state: 'success' }, readOnly)).status, 403);
+ assert.equal((await post({ context: 'test', state: 'success' }, readOnly)).json().code, 'token.scope');
+ assert.equal((await t.get(`${root}/commits/${head.sha}/statuses`, { as: sam, json: { context: 'test', state: 'success' } })).status, 403);
+ assert.equal((await post({ context: 'test', state: 'nope' })).status, 422);
+ assert.equal((await t.get(`${root}/commits/notasha/statuses`, { bearer: runSvc, json: { context: 'test', state: 'success' } })).status, 422);
+ assert.equal((await post({ context: 'test', state: 'success', target_url: 'gopher://x' })).status, 422);
+ assert.equal((await post({ context: '', state: 'success' })).status, 422);
+
+ const first = await post({ context: 'test', state: 'pending', description: 'Running', target_url: 'https://ci.example/runs/1' });
+ assert.equal(first.status, 201, first.text);
+ assert.equal(first.json().context, 'test');
+ assert.equal(first.json().state, 'pending');
+ await post({ context: 'security', state: 'success' });
+ await post({ context: 'test', state: 'success', description: 'Green', target_url: 'https://ci.example/runs/1' });
+
+ // Reads take the latest per context, with the combined state. Public commits read anonymously.
+ const listed = await t.get(`${root}/commits/${head.sha}/statuses`);
+ assert.equal(listed.status, 200);
+ assert.deepStrictEqual(listed.json().statuses.map((s) => [s.context, s.state]), [['security', 'success'], ['test', 'success']]);
+ assert.equal(listed.json().statuses.find((s) => s.context === 'test').description, 'Green');
+ assert.equal(listed.json().state, 'success');
+ assert.equal((await t.get(`${root}/commits/${head.sha}/statuses`)).status, 200);
+
+ // Required contexts: empty by default, the owner sets them, nobody else.
+ assert.deepStrictEqual((await t.get(`${root}/required-checks`)).json().required_checks, []);
+ assert.equal((await t.get(`${root}/required-checks`, { as: pat, json: { contexts: ['test'] } })).status, 403);
+ assert.equal((await t.get(`${root}/required-checks`, { as: sam, json: { contexts: ['x'.repeat(300)] } })).status, 422);
+ const required = await t.get(`${root}/required-checks`, { as: sam, json: { contexts: ['test', 'security'] } });
+ assert.equal(required.status, 200, required.text);
+ assert.deepStrictEqual(required.json().required_checks, ['test', 'security']);
+
+ // A change shows the checks and is mergeable while its required contexts are green.
+ const created = await t.get(`${root}/changes`, { as: sam, json: { title: 'Ship the feature', base_ref: 'main', head_ref: 'agent/feat' } });
+ assert.equal(created.status, 201, created.text);
+ const detail = await t.get(`${root}/changes/${created.json().number}`);
+ assert.equal(detail.json().checks.length, 2);
+ assert.equal(detail.json().checks_label, '2 checks passed');
+ assert.equal(detail.json().state, 'success');
+ assert.equal(detail.json().mergeable, true);
+ assert.deepStrictEqual(detail.json().required.map((r) => [r.context, r.ok]), [['test', true], ['security', true]]);
+ const page = await t.get(`/sam/ci/changes/${created.json().number}`, { as: sam });
+ assert.match(page.text, /2 checks passed/);
+ assert.match(page.text, /git-check--success/);
+ assert.match(page.text, /ci\.example\/runs\/1/);
+
+ // A red required context blocks the merge, on the API and on the page.
+ await post({ context: 'test', state: 'failure', description: 'Broke it' });
+ const red = await t.get(`${root}/changes/${created.json().number}`);
+ assert.equal(red.json().checks_label, '1 of 2 checks failed');
+ assert.equal(red.json().mergeable, false);
+ assert.equal((await t.get(`${root}/changes/${created.json().number}/merge`, { as: sam, json: { head_sha: head.sha, method: 'squash' } })).status, 409);
+ const redPage = await t.get(`/sam/ci/changes/${created.json().number}`, { as: sam });
+ assert.match(redPage.text, /Required checks are not passing/);
+ await post({ context: 'test', state: 'success', description: 'Fixed' });
+ assert.equal((await t.get(`${root}/changes/${created.json().number}`)).json().mergeable, true);
+ assert.equal((await t.get(`${root}/changes/${created.json().number}/merge`, { as: sam, json: { head_sha: head.sha, method: 'squash' } })).status, 200);
+
+ // A missing required context blocks too; clearing the list merges as before.
+ const head2 = await t.ctx.git.commitFiles('sam', 'ci', {
+ branch: 'agent/more', base: 'main', message: 'More', files: [{ path: 'more.txt', content: 'more\n' }],
+ author: { name: 'Sam', email: '[email protected]' },
+ });
+ const created2 = await t.get(`${root}/changes`, { as: sam, json: { title: 'More work', base_ref: 'main', head_ref: 'agent/more' } });
+ const missing = await t.get(`${root}/changes/${created2.json().number}`);
+ assert.equal(missing.json().checks_label, 'No checks');
+ assert.equal(missing.json().mergeable, false, 'required contexts with no report block');
+ assert.equal((await t.get(`${root}/changes/${created2.json().number}/merge`, { as: sam, json: { head_sha: head2.sha, method: 'merge' } })).status, 409);
+ assert.deepStrictEqual((await t.get(`${root}/required-checks`, { as: sam, json: { contexts: [] } })).json().required_checks, []);
+ assert.equal((await t.get(`${root}/changes/${created2.json().number}`)).json().mergeable, true);
+ assert.equal((await t.get(`${root}/changes/${created2.json().number}/merge`, { as: sam, json: { head_sha: head2.sha, method: 'merge' } })).status, 200);
+
+ console.log('checks: report auth, latest-per-context, required contexts, mergeable and the merge gate passed');
+ } finally { await t.close(); }
+})().catch((e) => { console.error(e); process.exit(1); });