openvibers/openvibe-git README.md
README.md
| 1 | # OpenVibe.Git |
| 2 | |
| 3 | Git for OpenVibe: host code with your OpenVibe account, browse it, mirror GitHub, and clone it to your computer with |
| 4 | Link. |
| 5 | |
| 6 | **Live site:** https://git.openvibe.codes · **API docs:** https://openvibe.services · **For AI:** https://git.openvibe.codes/llms.txt |
| 7 | **Run it:** `npm ci && npm run dev` (Node 22 or newer) · **Tests:** `npm test` |
| 8 | |
| 9 | `git.openvibe.codes` is served by this Node service. Repositories are bare git repositories on disk and the wire |
| 10 | protocol is git's own `git http-backend`; the pages, the API and the insights are ours, built with the shared navbar, |
| 11 | footer and kit. |
| 12 | |
| 13 | ## Run |
| 14 | |
| 15 | ```sh |
| 16 | npm install |
| 17 | npm run dev # http://localhost:5250, an embedded PGlite database under data/ |
| 18 | npm test # every test file on a temp PGlite database |
| 19 | npm run test:pg # the same through PostgreSQL + PgBouncer |
| 20 | ``` |
| 21 | |
| 22 | Production reads `/etc/openvibe/git.env`; see `.env.example`. `DATABASE_URL` serves, `DATABASE_DIRECT_URL` migrates, and |
| 23 | `GIT_DATA_DIR` holds the bare repositories (`DATA_DIR/repos/<owner>/<name>.git`). |
| 24 | |
| 25 | ## Official mirrors |
| 26 | |
| 27 | The Forgejo cut-over keeps the mirror list and nothing else, so the service rebuilds it from GitHub itself. When |
| 28 | `GIT_OFFICIAL_ORG` is set, a due-based background job mirrors that organisation's public repositories under |
| 29 | `GIT_OFFICIAL_OWNER` (by default the first of `GIT_OFFICIAL_OWNERS`, `openvibers`): |
| 30 | |
| 31 | - it lists every page of the organisation through GitHub's REST API, with `GIT_GITHUB_TOKEN` when set. When that |
| 32 | listing is refused for any reason (403/429/5xx/network — the host shares GitHub's 60 anonymous calls an hour), it |
| 33 | falls back to the repositories named in `openvibe-contracts` (`manifests/services/*.json` and |
| 34 | `manifests/repositories/*.json`), deduplicated, and the run says which list it used; |
| 35 | - each candidate is checked as public and existing through git's protocol (protocol-v2 `ls-refs` over |
| 36 | `github.com`, which spends no REST quota); private and missing repositories are skipped and counted with the |
| 37 | reason, and the answer names the default branch; |
| 38 | - it skips forks and archived repositories, and creates each missing repository as a GitHub mirror through the |
| 39 | normal repository code, a few clones at a time (`GIT_OFFICIAL_CONCURRENCY`, default 3) and with the mirror |
| 40 | timeout. When the API refuses the metadata for quota, the clone still runs and the card is marked pending; a |
| 41 | later run fills the description, topics and default branch; |
| 42 | - names follow this service's rule, so `OpenVibe.Live` becomes `openvibe.live`; a name that cannot be used here is |
| 43 | skipped and counted as invalid; |
| 44 | - it never deletes anything, and one log line per run reports the list's source, created, existing, skipped (with |
| 45 | each reason) and failed. |
| 46 | |
| 47 | The ten-minute mirror poll reads the remote's refs with `git ls-remote` (also `ls-refs`, also no REST quota) and |
| 48 | fetches only when something changed. |
| 49 | |
| 50 | A run is due when `GIT_OFFICIAL_SYNC_MS` (default six hours) has passed since the last finished run; the last run |
| 51 | is a row in the database, so a deploy does not reset the interval. The first check is shortly after boot and the |
| 52 | job ticks hourly, in the background: `/api/ready` never waits on it, and a failed run waits for the next due time. |
| 53 | An empty `GIT_OFFICIAL_ORG` turns the job off, and so does drill mode. |
| 54 | |
| 55 | ## Tarballs, pins and checks |
| 56 | |
| 57 | Every ref answers a gzip tarball: `GET /<owner>/<name>/archive/<ref>.tar.gz` (a branch may hold slashes) and |
| 58 | the same path under `/api/v1/repos/<owner>/<name>/`. The bytes are git archive of the resolved commit, |
| 59 | streamed, with the commit as the ETag; a tag or a full commit id caches immutably, a branch for a minute. |
| 60 | The top folder is `<name>-<commit>/`, so a tag's bytes never change and a package manager can pin them: |
| 61 | |
| 62 | ```json |
| 63 | "openvibe-shared": "https://git.openvibe.codes/openvibers/openvibe.shared/archive/v4.186.0.tar.gz" |
| 64 | ``` |
| 65 | |
| 66 | instead of the `codeload.github.com` URL it replaces. Public repositories need no token; private ones answer |
| 67 | 404 to anyone who may not see them, like the raw file. |
| 68 | |
| 69 | A commit carries statuses: `POST /api/v1/repos/:owner/:name/commits/:sha/statuses` with |
| 70 | `{context, state: pending|success|failure|error, description?, target_url?}`, reported with a typed Network |
| 71 | service token holding `git.checks.write`; `GET` lists the latest per context. A change shows them and is |
| 72 | mergeable only when its required contexts are green — a per-repository list, empty by default, set with |
| 73 | `POST …/required-checks {contexts}`. For a mirrored repository the checks intake copies GitHub's check-runs |
| 74 | for the open pull-request heads and the default branch (at most every two minutes per repository, and only |
| 75 | when a head moved), so the pages show the truth until our own runners report. |
| 76 | |
| 77 | **Forge CI** runs a repository's tests on the workstation, in a sandbox, for branch pushes to official repositories |
| 78 | by OpenVibers members only; the verdict is the `forge/test` status and the log is at `/<owner>/<name>/ci/<run>`. |
| 79 | Forks, pull requests and non-members keep GitHub Actions. What runs, how it is contained and how to add a repository: |
| 80 | [docs/ci.md](docs/ci.md). |
| 81 | |
| 82 | ## Layout |
| 83 | |
| 84 | - `server/git/` — repositories on disk: creation, refs, trees, blobs, commits, diffs, bunches, and `http-backend`. |
| 85 | - `server/http/` — the pages and `/api/v1`. |
| 86 | - `server/mirror.js` — GitHub mirrors: the push webhook, the ten-minute poll, and the push mirror back. |
| 87 | - `server/official.js` — the due-based seed of the official mirrors from a GitHub organisation or the contracts. |
| 88 | - `server/contracts.js` — the repository list named by openvibe-contracts when GitHub's listing is refused. |
| 89 | - `server/insights.js` — views, unique visitors, clones, stars and commits per week. |
| 90 | - `server/ci.js`, `server/ci-auth.js` — the forge CI queue, the member rule, supersede and the `forge/test` status. |
| 91 | - `runner/` — the workstation runner: poll, clone, sandbox phases, allowlisted install proxy, log and verdict. |
| 92 | - `deploy/ci/`, `deploy/systemd-user/` — the runner's install script, example config and user unit. |
| 93 | - `contracts` — `git.repo@1`, `git.commit@1`, `git.tree@1`, `git.insights@1` (OpenVibe.Contracts). |
| 94 | - `deploy/` — the systemd unit and the nginx vhost for `git.openvibe.codes`. |
| 95 | |
| 96 | Sign-in is OpenVibe only. Git over HTTPS uses a short-lived Git token made on the repository page (`git.repo.read` / |
| 97 | `git.repo.write`) or a typed Network token for the same audience; no second account system exists. |
| 98 | |
| 99 | A first-party service (Codes, Run, Link) holds a typed Network token for `openvibe.git` and acts for the person it |
| 100 | names in `X-OV-Subject`: `POST /api/v1/repos` (with `owner_username`), `GET /api/v1/profile/:username` and |
| 101 | `POST /api/v1/tokens` then work as that person, and the Git token it mints is the person's own for clone and push. |
| 102 | A service never holds a person's password. |
| 103 | |
| 104 | G2 adds changes and review, compare, issues, blame and code search. See [the API and integration notes](docs/g2-review.md) for routes, capability grants, the AI-choices dependency, and recovery details. |
| 105 | |
| 106 | ## Link: clone here, and publish your computer |
| 107 | |
| 108 | G3 adds Link's outbound relay (contracts `git.clone-request@1` and `git.machine@1`): |
| 109 | |
| 110 | - **Clone to my computer** on a repository page makes a `clr_…` request for two minutes and shows |
| 111 | `openvibe-link clone '<link>'` (and an `openvibe-link://clone?url=…` deep link). Link reads the request over |
| 112 | HTTPS with the one-time token, asks the person there, clones into a folder inside the root they chose, and posts |
| 113 | approved / progress / done / refused / failed back; the page's progress line polls the same row. The clone URL's |
| 114 | read token reaches only the Link, is kept in this process, and is revoked when the request ends. |
| 115 | - **Clone to one of my computers:** the Get it panel lists the person's own Link computers, one plain button each. |
| 116 | The button makes a `clr_…` request addressed to that computer and sends it down the watch the computer already |
| 117 | holds (`kind: clone`, the repository, the https clone URL with a short read token, who asked, and the end of the two |
| 118 | minutes). The computer answers `waiting` (it is asking its owner), `approved`, `cloning` with git's percentage, |
| 119 | then `done` with the folder, or `refused` / `failed`; the panel's progress line follows the same row. Nothing is |
| 120 | written until the person answers there, and only the computer the request went to can move it. With no computer |
| 121 | online the panel says so and keeps the terminal-command link above. |
| 122 | - **Your computers** (`/machines`) makes an `mch_…` link for `openvibe-link git connect '<link>'`. Link publishes |
| 123 | the repositories under its root, read-only: the page lists them with the branch, head and date, and browses tree, |
| 124 | file and log through the poll the machine opened. A machine that has not checked in for thirty seconds is |
| 125 | **offline since …**; the only write it ever does is a clone the person approved there. |
| 126 | |
| 127 | The routes are `POST /api/v1/clone-requests`, `GET /api/v1/clone-requests/:id`, |
| 128 | `POST /api/v1/repos/:owner/:name/clone-to-machine` and `GET …/clone-to-machine/:id`, `POST|GET /api/v1/machines`, and |
| 129 | the Link's own `/api/v1/link/clone-requests/:id`, `/api/v1/link/machines/:id/{hello,watch,results}` and |
| 130 | `POST /api/v1/link/machines/:id/clone-requests/:clr`. Tests: `test/link-relay.test.js`, `test/clone-to-machine.test.js`. |
| 131 | |
| 132 | <!-- versions:start --> |
| 133 | - openvibe-contracts: v0.419.0 |
| 134 | - openvibe-sdk: v0.161.0 |
| 135 | <!-- versions:end --> |
| 136 |