openvibers/openvibe.codes Commits

Codes: OpenVibe Git, Forgejo at openvibe.codes/git with OpenVibe sign-in and GitHub mirrors (#85)

OP OpenVibers committed 29 days ago · f6bf96b258 · 8 files, +304 −0

Changes

ddiff --git a/README.md b/README.md
iindex 0894cdc..c3e2ac4 100644
---- a/README.md
++++ b/README.md
@@@ -7,6 +7,8 @@
  **Plan:** plan T16 (the coding-harness fabric); the harness boundaries are [ADR-050](docs/adr/050-codes-harness-fabric.md) (proposed).
  **License:** AGPL-3.0 (same as every OpenVibe service).
  
++
++**OpenVibe Git** (since 2026-10-15): Forgejo at [openvibe.codes/git](https://openvibe.codes/git/), signed in with your OpenVibe account, with read mirrors of the OpenVibers GitHub repositories. Install and operations: [deploy/forgejo](deploy/forgejo/README.md).
  ## Purpose
  
  One place to send a coding task to the agent that fits it — by what the task needs first, then by cost — hand it over when one agent gets stuck, and keep the run in one place; and the front door for anyone who wants to improve OpenVibe itself. The agents run on people's own machines or OpenVibe's; Codes routes, records and hands off, and never holds a person's provider keys in the clear.
ddiff --git a/deploy/forgejo/README.md b/deploy/forgejo/README.md
nnew file mode 100644
iindex 0000000..17b9ec3
---- /dev/null
++++ b/deploy/forgejo/README.md
@@@ -0,0 +1,39 @@
++# OpenVibe Git (Forgejo)
++
++The OpenVibe projects' own Git host, at **https://openvibe.codes/git/**. It runs Forgejo, which is lighter than GitLab
++on the 15 GB host. It moves to `git.openvibe.codes` once that DNS name exists: change `ROOT_URL`, `DOMAIN` and the
++redirect URI.
++
++- **Sign-in:** people sign in with their OpenVibe account through OpenVibe.Network's OpenID Connect (the OAuth client
++  `forgejo`, which Network seeds). There is no local sign-up; the first sign-in creates the account with the OpenVibe
++  username.
++- **Mirrors:** the `OpenVibers` organisation holds read mirrors of the public GitHub repositories, pulled hourly.
++  GitHub stays canonical until a repository is moved here, one at a time. Then the direction flips to a push mirror
++  to GitHub.
++- **Cloning:** HTTPS only. `openvibe.codes` is behind Cloudflare's proxy, which carries no SSH.
++- **Off:** mail, Actions (CI) and federation are off for now; turn them on in `app.ini` when they have a job.
++
++## Files
++
++| File | What it does |
++|---|---|
++| `install.sh` | Binary (sha256-checked), the `forgejo` user, the PostgreSQL role and the `ov_forgejo` database, `/etc/forgejo/app.ini` with secrets generated on the host, and the systemd unit. Idempotent; `FORGEJO_VERSION=x.y.z` upgrades the binary. |
++| `app.ini.template` | The config; `@…@` values are filled in by `install.sh`. |
++| `openvibe-forgejo.service` | The unit: `127.0.0.1:5060`, hardened, 1 GB of memory at most. |
++| `setup-sso.sh` | The site admin (a random password in `/root/forgejo-admin.txt`) and the OpenVibe sign-in. The client secret is read from Network's database. |
++| `mirror-github.sh [file]` | The `OpenVibers` organisation and one pull mirror per public repository: names from `gh`, or one per line in `file`. |
++
++nginx: `deploy/nginx/openvibe.codes.conf` proxies `^~ /git/` to `127.0.0.1:5060`, streamed, with a 512 MB limit.
++
++## Install (on the host, from the Codes checkout)
++
++```bash
++sudo bash deploy/forgejo/install.sh
++sudo ov-vhost-install deploy/nginx/openvibe.codes.conf openvibe.codes.conf
++sudo bash deploy/forgejo/setup-sso.sh                    # after Network seeds the forgejo client
++gh repo list OpenVibers --visibility public --limit 200 --json name -q '.[].name' > /tmp/repos.txt   # where gh is signed in
++sudo bash deploy/forgejo/mirror-github.sh /tmp/repos.txt
++```
++
++Backups: `ov_forgejo` is a PostgreSQL database like the others (add it to the host inventory's databases), and the
++repositories live in `/var/lib/forgejo/repositories`.
ddiff --git a/deploy/forgejo/app.ini.template b/deploy/forgejo/app.ini.template
nnew file mode 100644
iindex 0000000..053bcd2
---- /dev/null
++++ b/deploy/forgejo/app.ini.template
@@@ -0,0 +1,111 @@
++; OpenVibe Git: Forgejo at https://openvibe.codes/git/ (deploy/forgejo/install.sh writes /etc/forgejo/app.ini from this).
++; @VALUES@ are filled in on the host; the secrets are generated there and never leave it.
++APP_NAME = OpenVibe Git
++RUN_USER = forgejo
++RUN_MODE = prod
++WORK_PATH = /var/lib/forgejo
++
++[server]
++PROTOCOL = http
++HTTP_ADDR = 127.0.0.1
++HTTP_PORT = 5060
++DOMAIN = openvibe.codes
++ROOT_URL = https://openvibe.codes/git/
++; Cloning and pushing are over HTTPS only: openvibe.codes is behind Cloudflare's proxy, which carries no SSH.
++DISABLE_SSH = true
++START_SSH_SERVER = false
++LANDING_PAGE = explore
++OFFLINE_MODE = true
++LFS_START_SERVER = false
++APP_DATA_PATH = /var/lib/forgejo/data
++
++[database]
++DB_TYPE = postgres
++HOST = 127.0.0.1:5432
++NAME = ov_forgejo
++USER = forgejo
++PASSWD = @DB_PASSWORD@
++SSL_MODE = disable
++MAX_OPEN_CONNS = 10
++MAX_IDLE_CONNS = 4
++
++[repository]
++ROOT = /var/lib/forgejo/repositories
++DEFAULT_BRANCH = main
++DEFAULT_PRIVATE = public
++
++[service]
++; People join by signing in with their OpenVibe account (the OpenVibe authentication source); there is no local sign-up.
++DISABLE_REGISTRATION = false
++ALLOW_ONLY_EXTERNAL_REGISTRATION = true
++SHOW_REGISTRATION_BUTTON = false
++REQUIRE_SIGNIN_VIEW = false
++DEFAULT_KEEP_EMAIL_PRIVATE = true
++DEFAULT_ALLOW_CREATE_ORGANIZATION = false
++ENABLE_NOTIFY_MAIL = false
++NO_REPLY_ADDRESS = noreply.openvibe.codes
++
++[oauth2_client]
++ENABLE_AUTO_REGISTRATION = true
++; goth's OpenID Connect provider puts the preferred_username claim (the OpenVibe username) in the nickname.
++USERNAME = nickname
++UPDATE_AVATAR = true
++ACCOUNT_LINKING = login
++OPENID_CONNECT_SCOPES = openid profile email
++
++[openid]
++ENABLE_OPENID_SIGNIN = false
++ENABLE_OPENID_SIGNUP = false
++
++[oauth2]
++; Forgejo as an OAuth provider is not needed: OpenVibe.Network is the identity provider.
++ENABLED = false
++
++[security]
++INSTALL_LOCK = true
++SECRET_KEY = @SECRET_KEY@
++INTERNAL_TOKEN = @INTERNAL_TOKEN@
++PASSWORD_HASH_ALGO = argon2
++MIN_PASSWORD_LENGTH = 16
++
++[session]
++PROVIDER = db
++COOKIE_SECURE = true
++COOKIE_NAME = ov_git_session
++
++[mailer]
++ENABLED = false
++
++[picture]
++DISABLE_GRAVATAR = true
++ENABLE_FEDERATED_AVATAR = false
++
++[ui]
++DEFAULT_THEME = forgejo-auto
++THEMES = forgejo-auto,forgejo-light,forgejo-dark
++
++[ui.meta]
++AUTHOR = OpenVibers
++DESCRIPTION = OpenVibe Git: the OpenVibe projects' own source, with your OpenVibe account.
++
++[mirror]
++ENABLED = true
++DEFAULT_INTERVAL = 8h
++MIN_INTERVAL = 10m
++
++[migrations]
++ALLOWED_DOMAINS = github.com,*.github.com,codeberg.org
++ALLOW_LOCALNETWORKS = false
++
++[actions]
++ENABLED = false
++
++[federation]
++ENABLED = false
++
++[log]
++MODE = console
++LEVEL = Info
++
++[cron.update_mirrors]
++SCHEDULE = @every 1h
ddiff --git a/deploy/forgejo/install.sh b/deploy/forgejo/install.sh
nnew file mode 100644
iindex 0000000..28ea4b8
---- /dev/null
++++ b/deploy/forgejo/install.sh
@@@ -0,0 +1,59 @@
++#!/bin/bash
++# OpenVibe Git (Forgejo) on the host: the binary (checked against the release's sha256), its user and directories, a
++# PostgreSQL role and database, /etc/forgejo/app.ini from app.ini.template with secrets generated here, and the
++# systemd unit. Idempotent: a second run upgrades the binary to FORGEJO_VERSION and leaves the config and data alone.
++#
++#   sudo bash deploy/forgejo/install.sh          (from the Codes checkout on the host)
++#
++# Prints no secret. The database password and Forgejo's own keys exist only in /etc/forgejo/app.ini (root:forgejo 0660).
++set -euo pipefail
++VERSION="${FORGEJO_VERSION:-16.0.6}"
++KIT="$(cd "$(dirname "$0")" && pwd)"
++LIB=/usr/local/lib/forgejo
++URL="https://codeberg.org/forgejo/forgejo/releases/download/v${VERSION}/forgejo-${VERSION}-linux-amd64"
++[ "$(id -u)" = 0 ] || { echo "install.sh: run as root" >&2; exit 1; }
++
++# 1. The binary, verified.
++mkdir -p "$LIB"
++if [ ! -x "$LIB/forgejo-$VERSION" ]; then
++    tmp=$(mktemp -d)
++    curl -fsSL -o "$tmp/forgejo" "$URL"
++    curl -fsSL -o "$tmp/forgejo.sha256" "$URL.sha256"
++    want=$(cut -d' ' -f1 "$tmp/forgejo.sha256")
++    have=$(sha256sum "$tmp/forgejo" | cut -d' ' -f1)
++    [ -n "$want" ] && [ "$want" = "$have" ] || { echo "install.sh: sha256 mismatch for forgejo $VERSION" >&2; rm -rf "$tmp"; exit 1; }
++    install -m 0755 "$tmp/forgejo" "$LIB/forgejo-$VERSION"
++    rm -rf "$tmp"
++    echo "forgejo $VERSION installed (sha256 verified)"
++fi
++ln -sfn "$LIB/forgejo-$VERSION" /usr/local/bin/forgejo
++
++# 2. Its user and directories.
++id forgejo >/dev/null 2>&1 || adduser --system --group --home /var/lib/forgejo --shell /bin/bash --disabled-password --gecos 'OpenVibe Git' forgejo
++install -d -o forgejo -g forgejo -m 0750 /var/lib/forgejo /var/lib/forgejo/custom /var/lib/forgejo/data /var/lib/forgejo/log /var/lib/forgejo/repositories
++install -d -o root -g forgejo -m 0770 /etc/forgejo
++
++# 3 and 4. The database and the config, once.
++if [ ! -f /etc/forgejo/app.ini ]; then
++    pw=$(openssl rand -hex 24)
++    printf "DO \$\$ BEGIN IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'forgejo') THEN CREATE ROLE forgejo LOGIN PASSWORD '%s'; ELSE ALTER ROLE forgejo LOGIN PASSWORD '%s'; END IF; END \$\$;\n" "$pw" "$pw" \
++        | sudo -u postgres psql -q -v ON_ERROR_STOP=1 >/dev/null
++    if ! sudo -u postgres psql -tAc "SELECT 1 FROM pg_database WHERE datname = 'ov_forgejo'" | grep -q 1; then
++        sudo -u postgres createdb -O forgejo -E UTF8 -T template0 ov_forgejo
++    fi
++    key=$(/usr/local/bin/forgejo generate secret SECRET_KEY)
++    tok=$(/usr/local/bin/forgejo generate secret INTERNAL_TOKEN)
++    umask 0007
++    sed -e "s|@DB_PASSWORD@|$pw|" -e "s|@SECRET_KEY@|$key|" -e "s|@INTERNAL_TOKEN@|$tok|" "$KIT/app.ini.template" > /etc/forgejo/app.ini
++    chown root:forgejo /etc/forgejo/app.ini
++    chmod 0660 /etc/forgejo/app.ini
++    echo "config written to /etc/forgejo/app.ini (database ov_forgejo, role forgejo)"
++fi
++
++# 5. The service.
++install -m 0644 "$KIT/openvibe-forgejo.service" /etc/systemd/system/openvibe-forgejo.service
++systemctl daemon-reload
++systemctl enable --now openvibe-forgejo.service >/dev/null
++systemctl restart openvibe-forgejo.service
++for _ in $(seq 1 30); do curl -fsS -o /dev/null http://127.0.0.1:5060/api/healthz && break; read -rt 1 < <(tail -f /dev/null) || true; done
++curl -fsS http://127.0.0.1:5060/api/healthz >/dev/null && echo "openvibe-forgejo is up on 127.0.0.1:5060" || { echo "install.sh: forgejo did not come up; journalctl -u openvibe-forgejo" >&2; exit 1; }
ddiff --git a/deploy/forgejo/mirror-github.sh b/deploy/forgejo/mirror-github.sh
nnew file mode 100644
iindex 0000000..48f028f
---- /dev/null
++++ b/deploy/forgejo/mirror-github.sh
@@@ -0,0 +1,25 @@
++#!/bin/bash
++# Read mirrors of OpenVibers' public GitHub repositories on OpenVibe Git, in the OpenVibers organisation: Forgejo pulls
++# each one every MIN_INTERVAL/DEFAULT_INTERVAL (app.ini [mirror]). GitHub stays canonical until a repository is moved
++# here (plan: one repository at a time); then the direction flips to a push mirror. Uses a short-lived site-admin API
++# token it creates and deletes. Idempotent: a repository already mirrored is skipped.
++#
++#   sudo bash deploy/forgejo/mirror-github.sh            (needs `gh` signed in as someone who can list OpenVibers)
++set -euo pipefail
++[ "$(id -u)" = 0 ] || { echo "mirror-github.sh: run as root" >&2; exit 1; }
++FJ=(sudo -u forgejo /usr/local/bin/forgejo --config /etc/forgejo/app.ini --work-path /var/lib/forgejo)
++API=http://127.0.0.1:5060/api/v1
++REPOS_FILE="${1:-}"
++name="mirror-$(date +%s)"
++token=$("${FJ[@]}" admin user generate-access-token --username openvibe-admin --token-name "$name" --scopes write:organization,write:repository --raw)
++trap 'curl -fsS -o /dev/null -X DELETE -H "Authorization: token $token" "$API/users/openvibe-admin/tokens/$name" || true' EXIT
++call() { curl -fsS -H "Authorization: token $token" -H 'Content-Type: application/json' "$@"; }
++call -o /dev/null "$API/orgs/OpenVibers" 2>/dev/null || call -o /dev/null -X POST "$API/orgs" -d '{"username":"OpenVibers","full_name":"OpenVibers","website":"https://openvibe.network","visibility":"public","description":"The OpenVibe projects."}'
++if [ -n "$REPOS_FILE" ]; then list=$(cat "$REPOS_FILE"); else list=$(gh repo list OpenVibers --visibility public --limit 200 --json name -q '.[].name'); fi
++made=0; kept=0
++while read -r repo; do
++    [ -n "$repo" ] || continue
++    if call -o /dev/null "$API/repos/OpenVibers/$repo" 2>/dev/null; then kept=$((kept + 1)); continue; fi
++    call -o /dev/null -X POST "$API/repos/migrate" -d "{\"clone_addr\":\"https://github.com/OpenVibers/$repo.git\",\"repo_owner\":\"OpenVibers\",\"repo_name\":\"$repo\",\"mirror\":true,\"mirror_interval\":\"1h\",\"service\":\"git\",\"description\":\"Mirror of github.com/OpenVibers/$repo\"}" && made=$((made + 1)) || echo "mirror of $repo failed" >&2
++done <<< "$list"
++echo "mirrors: $made created, $kept already there"
ddiff --git a/deploy/forgejo/openvibe-forgejo.service b/deploy/forgejo/openvibe-forgejo.service
nnew file mode 100644
iindex 0000000..97e9849
---- /dev/null
++++ b/deploy/forgejo/openvibe-forgejo.service
@@@ -0,0 +1,23 @@
++[Unit]
++Description=OpenVibe Git (Forgejo) at https://openvibe.codes/git/
++After=network.target postgresql.service
++Wants=postgresql.service
++
++[Service]
++Type=simple
++User=forgejo
++Group=forgejo
++WorkingDirectory=/var/lib/forgejo
++ExecStart=/usr/local/bin/forgejo web --config /etc/forgejo/app.ini --work-path /var/lib/forgejo
++Restart=always
++RestartSec=3
++Environment=USER=forgejo HOME=/var/lib/forgejo GITEA_WORK_DIR=/var/lib/forgejo
++NoNewPrivileges=true
++PrivateTmp=true
++ProtectSystem=full
++ProtectHome=true
++ReadWritePaths=/var/lib/forgejo /etc/forgejo
++MemoryMax=1G
++
++[Install]
++WantedBy=multi-user.target
ddiff --git a/deploy/forgejo/setup-sso.sh b/deploy/forgejo/setup-sso.sh
nnew file mode 100644
iindex 0000000..235ca5b
---- /dev/null
++++ b/deploy/forgejo/setup-sso.sh
@@@ -0,0 +1,32 @@
++#!/bin/bash
++# After install.sh: the site admin (a random password, written to /root/forgejo-admin.txt, never printed) and the
++# "OpenVibe" sign-in: an OpenID Connect source on OpenVibe.Network's discovery document, with the OAuth client
++# `forgejo` that Network seeds (OpenVibe.Network server/db/database.js). Its secret is read from Network's database
++# here and handed to Forgejo directly. Idempotent: an existing admin or source is updated, not duplicated.
++#
++#   sudo bash deploy/forgejo/setup-sso.sh
++set -euo pipefail
++[ "$(id -u)" = 0 ] || { echo "setup-sso.sh: run as root" >&2; exit 1; }
++FJ=(sudo -u forgejo /usr/local/bin/forgejo --config /etc/forgejo/app.ini --work-path /var/lib/forgejo)
++NETWORK_DB="${NETWORK_DB:-ov_network}"
++
++if ! "${FJ[@]}" admin user list --admin 2>/dev/null | awk 'NR > 1 { print $2 }' | grep -qx openvibe-admin; then
++    umask 0077
++    "${FJ[@]}" admin user create --admin --username openvibe-admin --email [email protected] --random-password --must-change-password=false > /root/forgejo-admin.txt
++    echo "site admin openvibe-admin created (password in /root/forgejo-admin.txt)"
++fi
++
++secret=$(sudo -u postgres psql -d "$NETWORK_DB" -tAc "SELECT client_secret FROM oauth_clients WHERE client_id = 'forgejo'")
++[ -n "$secret" ] || { echo "setup-sso.sh: Network has no OAuth client 'forgejo' yet (deploy Network first)" >&2; exit 1; }
++args=(--name OpenVibe --provider openidConnect --key forgejo --secret "$secret"
++      --auto-discover-url https://openvibe.network/.well-known/openid-configuration
++      --scopes openid --scopes profile --scopes email --skip-local-2fa
++      --icon-url https://openvibe.network/favicon.ico)
++id=$("${FJ[@]}" admin auth list 2>/dev/null | awk '$2 == "OpenVibe" { print $1 }')
++if [ -n "$id" ]; then
++    "${FJ[@]}" admin auth update-oauth --id "$id" "${args[@]}" >/dev/null
++    echo "OpenVibe sign-in updated (source $id)"
++else
++    "${FJ[@]}" admin auth add-oauth "${args[@]}" >/dev/null
++    echo "OpenVibe sign-in added"
++fi
ddiff --git a/deploy/nginx/openvibe.codes.conf b/deploy/nginx/openvibe.codes.conf
iindex eb520ad..9bd37d6 100644
---- a/deploy/nginx/openvibe.codes.conf
++++ b/deploy/nginx/openvibe.codes.conf
@@@ -103,6 +103,19 @@ server {
      # OpenVibe.Events delivers to 127.0.0.1 directly; nothing from outside reaches /internal/.
      location /internal/ { return 404; }
  
++    # ── OpenVibe Git: Forgejo (deploy/forgejo), its ROOT_URL is https://openvibe.codes/git/ ──
++    # ^~ so the dotfile rule below never hides a repository path like /git/OpenVibers/x/src/branch/main/.github/.
++    # Pushes and clones are streamed (no buffering), up to 512 MB a request (Cloudflare's own limit is lower).
++    location = /git { return 301 /git/; }
++    location ^~ /git/ {
++        client_max_body_size 512m;
++        proxy_request_buffering off;
++        proxy_buffering off;
++        proxy_read_timeout 600s;
++        proxy_send_timeout 600s;
++        proxy_pass http://127.0.0.1:5060/;
++    }
++
      # ── Pages, docs, robots.txt, sitemap, static assets ─────
      location / {
          proxy_pass http://127.0.0.1:4900;