hive-ci: Forgejo Actions Runner

The hive-ci module runs a Forgejo Actions runner in a hive-ci nixos-container, executing CI jobs from .forgejo/workflows/ci.yml on every PR.

For operators

Enabling it's one line: services.hyperhive.deploy.forgejo.ci.enable = true in the host NixOS config. No manual token provisioning — hive-c0re registers the runner with the forge automatically.

Two things worth knowing before you rely on it:

Everything below this point is implementation detail: exact autoregistration internals, container design, and the full security writeup.

CI checks

Three jobs run on every PR (and on workflow_dispatch for manual re-triggers), defined in .forgejo/workflows/ci.yml. All three are required checks (forge branch protection) — a hit on any of them blocks merge.

Job What it runs
nix flake check treefmt formatting, cargo clippy -D warnings, cargo test, module evaluation
tracker-tag lint flags #NNN issue tags in source and comments (scripts/check-issue-refs.sh)
comment-block lint flags contiguous comment blocks over 30 lines (scripts/check-comment-blocks.sh)

hive-forge ci-rerun --pr N dispatches a workflow_dispatch retrigger without an empty commit.

ci-rerun --pr verifies the code but doesn't reliably move the PR's own status checks. Measured directly (raw GET /repos/.../actions/tasks JSON): a genuine PR-triggered run carries event: pull_request with the #<n> pseudo-ref as its head_branch; a ci-rerun --pr-dispatched run carries event: workflow_dispatch with the real branch name instead. Forgejo's PR commit-status tracking appears scoped to pull_request-event runs specifically, so a dispatched run — even a genuinely successful one — never writes to that status slot. Reproduced with three dispatches on one PR over 20+ minutes, all genuinely green, the PR's own status unmoved throughout. Always re-check pr-status after a dispatch and believe what it says — don't push a commit just to unwedge it, that's the empty-commit anti-pattern this verb exists to avoid. If the code is verified green (hive-forge ci-log --run <n> --job 0, verdict is the last line) but the status is stuck, ask the operator to select "rerun" in the forge web UI — it's CSRF-gated, so only they can do it.

Running checks locally

Don't run nix flake check directly — it dispatches to the shared build farm and wastes a remote-builder slot. Use the devshell equivalents instead:

nix develop -c cargo clippy --all-targets -- -D warnings
nix develop -c cargo test
nix fmt                         # authoritative; the formatter set is nix/treefmt.nix
sh scripts/check-issue-refs.sh  # tracker-tag lint
sh scripts/check-comment-blocks.sh  # comment-block lint

A git pre-push hook that automates the two lint checks is provided at scripts/pre-push. Install it once per clone:

ln -sf ../../scripts/pre-push .git/hooks/pre-push

After that, any git push automatically runs both lints and aborts with a diagnostic if either fails — catching the issue locally before CI sees it. Note that the hook does not run cargo clippy or cargo test (those are slow); run those manually before pushing Rust changes.

Configuration reference

The internal forge is always present (mandatory), so the runner always has a hive-forge instance to register against — nothing extra to enable beyond services.hyperhive.deploy.forgejo.ci.enable = true (see For operators above).

Optional tuning: services.hyperhive.deploy.forgejo.ci.name (runner name in forge admin panel), concurrency (parallel job capacity), labels (workflow targeting), jobTimeout (per-job wall-clock cap, default "1h", Go duration string for example "3h" — a job that exceeds it's killed so a hung or runaway build can't hold the runner's single slot indefinitely).

Container design

Autoregistration flow

Registration is off the container's boot-critical path — hive-c0re owns it and runs it out of band, so a slow forge or core-token never delays the container's start. Gotcha: don't gate the container's own start on a forge round-trip (a host-side unit that did this could exceed the nspawn start timeout and trip a restart loop) — registration must stay something hive-c0re drives after the container is already up. The core admin token is held only by hive-c0re on the host; only the runner registration token reaches the container.

hive-c0re side (forge/ci_runner.rs, run during the startup sweep)

Gated on HYPERHIVE_FORGE_CI_ENABLED (the nix module sets it on hive-c0re.service when deploy.forgejo.ci.enable). Best-effort — failures are logged and never abort the sweep; a healthy runner is never restarted.

  1. If .runner exists at /var/lib/nixos-containers/hive-ci/var/lib/gitea-runner/hive/.runner, validate its id against GET /api/v1/admin/runners/{id} with the core admin token:
    • 200: still registered — done, no restart.
    • 404 / other non-200 / malformed: stale — re-register (below).
    • transport error (forge unreachable): keep the existing creds; a network blip must not wipe a valid runner.
  2. If absent or stale: mint a fresh token from GET /api/v1/admin/runners/registration-token, then hand it to hive-priv's RegisterCiRunner, which (as root) writes TOKEN=<real> in place to the host env-file /run/hive-ci/runner-token (preserving the inode nspawn pinned into the container at start) and restarts gitea-runner-hive.service inside the container so it picks up the credential and registers.

Container side

Actions checkout mirror

When deploy.forgejo.ci.enable is set, hive-c0re autoseeds an actions/checkout pull-mirror on the local forge and sets Forgejo's DEFAULT_ACTIONS_URL to point at the local instance. This means CI uses: actions/checkout@vN steps resolve entirely on loopback — no external DNS on the CI critical path.

The mirror is seeded by hive-c0re itself during its forge provisioning sweep (forge/repos.rs::ensure_mirrors). The nix module forwards the effective mirror list as HYPERHIVE_FORGE_MIRRORS in the hive-c0re service environment (JSON-encoded [{upstream, dest}] list). hive-c0re already holds the admin token for the rest of the forge provisioning sweep (orgs, agent accounts, etc.), so mirror seeding lives in the same place rather than a separate host-side unit.

General-purpose mirrors: you can pre-seed any external repo as a pull-mirror via services.hyperhive.deploy.forgejo.mirrors:

services.hyperhive.deploy.forgejo.mirrors = [
  { upstream = "https://github.com/actions/checkout"; dest = "actions/checkout"; }
  { upstream = "https://github.com/example/tool";    dest = "mirrors/tool"; }
];

Each entry is created as a real Forgejo pull-mirror — not a one-off clone. Forgejo re-syncs the mirror on every pull (git-upload-pack request), so a DNS blip during that sync will propagate back to the runner as a hard git clone failure. The <owner> org in dest is autocreated. Keep mirror dests out of the hive-c0re-managed namespaces (config/, shared/, agents/, core/) to avoid provisioning collisions.

Security: unsandboxed builds and trusted contributors

hive-ci should only run CI for trusted contributors. The security boundary is weaker than it looks:

What unsandboxed builds mean

nspawn containers can't create user-namespaces, so nix.settings.sandbox-fallback = true is set in the container. This means every nix build (and nix flake check) runs without a build sandbox — the build process has full access to the container filesystem, network, and any bind-mounts during the build phase.

A malicious default.nix or build script in a PR can therefore:

The core admin token (forge-core-token) is not bind-mounted into the container. it's held and used only by hive-c0re on the host (forge/ci_runner.rs), which mints per-runner registration tokens; only that registration token reaches the container's env-file. A build process can still reach forge over the network, but can't use the admin token to issue privileged API calls.

Note: nix flake check --no-build (eval-only) reduces the attack surface but doesn't eliminate it — builtins.fetchGit, builtins.fetchurl, and import-from-derivation can reach the network and filesystem during evaluation. The default CI workflow runs full nix flake check (builds derivations), which is the higher-risk path.

Mitigation

For a hive used by a single operator or a small trusted team, the risk is low — all contributors are already trusted with forge access anyway.

For repos with external contributors or fork PRs:

The current design is appropriate for a trusted-team hive where all contributors have implicit forge access.

The CI runner builds derivations through the host nix-daemon — the hive-ci container shares the host store and has no daemon of its own. Build outputs accumulate in /nix/store with no automatic collection, and a busy CI day can fill the disk until every job fails fast with ENOSPC.

Store GC is a host-level concern, so it belongs in the host's own NixOS configuration, not in the hyperhive service modules — a single service should not reach out and change the host's global nix-daemon options. Add the following to your host config:

{
  # Daily GC: delete store paths not referenced by a live root and older
  # than a day. Keeps the store bounded between builds.
  #
  # `randomizedDelaySec` because a fixed daily time lands on the same wall
  # clock as whatever else runs nightly. A collection that overlaps a CI
  # job does not fail it — the runner builds through the host nix-daemon,
  # so an in-flight build's temp roots are protected — but the first build
  # after a collection rebuilds what the collection deleted, so a run that
  # lands just behind one pays for it in wall time.
  nix.gc = {
    automatic = true;
    dates = "daily";
    randomizedDelaySec = "45min";
    options = "--delete-older-than 1d";
  };

  # Disk-pressure GC: when free space drops below min-free mid-build, the
  # daemon collects garbage up to max-free before continuing. This is the
  # real-time net the daily timer can't provide — a same-day build burst is
  # what fills the disk. Tune to your disk size.
  nix.settings.min-free = 20 * 1024 * 1024 * 1024;  # 20 GiB
  nix.settings.max-free = 50 * 1024 * 1024 * 1024;  # 50 GiB
}

Measure a collection on your own hosts before relying on that stagger. randomizedDelaySec spreads the start, not the run, and the run is dominated by the sweep over unused .links entries — which scales with the size of the store, not with what this collection deletes. On a large store it can take most of an hour, so two hosts both set to daily may overlap however much jitter you give them. journalctl -u nix-gc.service reports each run's wall clock.

Remote builders: if CI dispatches builds to a remote builder (for example via nix.buildMachines / ssh-ng://), the build outputs land in that host's store, so the same GC config should be applied wherever the builder runs — GC on the coordinator host won't reclaim space on the builder.

⚠️ Two stores also means the temp-root protection above doesn't span them. Each daemon holds roots only for the builds it runs, so a collection on the builder knows nothing about a coordinator build waiting on a path there. Stagger the two hosts' GC windows rather than giving them the same dates, and prefer min-free/max-free on the builder — disk-pressure collection at least runs while nothing is mid-copy.

References