Config changes + helper events

Config changes are pull requests on the agent's config repo, which an operator merges on the forge; that merge deploys them. Helper events are how agents stay informed about lifecycle changes that need a reaction.

For operators

Everything below this point is the implementation detail behind that flow.

Config changes

Config changes flow through a forge pull request on the agent's agent-configs/<name> repo — the same surface agents use for code PRs. No MCP tool exists for config changes: opening the PR IS the request.

  1. The submitting agent clones agent-configs/<name>, edits it there (any tracked path, but agent.nix is the contract entry point), commits with its own git identity, and pushes a branch + opens a PR with hive-forge — the same way it would change any other repo. The bind-mounted /agents/<name>/config/ is a copy for reading a config, not the tree to edit: authoring in place there produces no PR. (it's currently mounted read-write, which is a defect tracked separately, not an authoring path.) Branch protection (the agent isn't on the main push allowlist; merge and approvals allowlist = operators team; see "Forge config repos" below) makes the agent a write collaborator that can't merge its own config PR.
  2. An operator reviews the PR on the forge (native diff, threaded comments, CI status) and merges it there. Merging needs membership of the operators team in agent-configs. The team starts empty: add each operator by hand in the forge UI.
  3. swarm-controller gets the agent-configs org's pull_request delivery. A closed event with merged: true and base branch main names the commit in merge_commit_sha.
  4. The controller looks up which hive's wanted state places the agent and publishes a deploy request carrying that commit on that hive's deploy subject. With no such hive, or more than one, it deploys nothing and logs a warning naming them.
  5. If the hive's applied/main already is that commit, it does nothing. Otherwise it fetches the config repo's main. A commit that applied/main already contains is a no-op too: requests for two merges can arrive in either order. A commit that descends from applied/main is fast-forwarded to and a rebuild queued. A failure before the rebuild (the fetch, or a commit that has diverged from applied/main) deploys nothing and logs swarm events: rev not applied; nothing queued at error, which fires the Grafana alert config rev refused. An agent with no container on that hive yet ignores the commit; its first deploy builds what it seeds.

This path runs no eval-verify. A merged config that doesn't evaluate or build fails the rebuild, and applied/main stays at that commit, so the agent's rebuilds keep failing until a fix merges. A failed rebuild shows on the hive like any other. A merge whose webhook delivery never reaches the controller deploys nothing.

Creating a brand-new agent is swarm-level (swarmctl agent create, POST /api/agents on the swarm controller), and no hive can originate an agent. The swarm controller's InitAgentConfigRepo job seeds the agent's config repo with a default agent.nix template, then asks the target hive to deploy it. An agent created from an archetype gets no config repo: its hive builds it from the archetype's module.

Changing what the template seeded isn't a special case: like every later change, it's a PR on that config repo, made from a clone and merged by an operator on the forge (see Config changes).

Agent lifecycle

Reminder delivery: file-path semantics

A reminder may carry a file_path (the agent-visible path inside its container, for example /agents/<name>/state/foo.md). On delivery hive-c0re:

  1. Translates the container path to the host path (/var/lib/hyperhive/agents/<name>/state/foo.md) so c0re can write from outside the container.
  2. Validates the path: rejects anything outside the agent's own state subtree, containing .. (path traversal), or with an empty relative tail. On rejection hive-c0re skips the write and delivers the original message inline with a warning — the reminder still fires.
  3. Defends against symlink escape: after create_dir_all, hive-c0re canonicalizes the parent dir and re-verifies it lives under the agent's host state root. hive-c0re opens the final file with O_NOFOLLOW | O_CREAT | O_TRUNC so an existing symlink at the basename can't redirect the write to an arbitrary host path.
  4. Writes the body to disk and delivers a short pointer message in its place, keeping the agent's inbox / wake-prompt small while the agent reads the bulky payload out of band.

Broker::deliver_reminders_batch handles atomicity of the inbox INSERT + reminders.sent_at UPDATE; the scheduler only computes the body strings before calling it.

Destroy semantics

HostRequest::Destroy { name, purge } is the lifecycle tear-down. Stops + removes the nspawn container and drops the systemd drop-in. Persistent state (proposed/applied repos, claude credentials, /state/ notes) is kept by default — recreating the agent with the same name reuses prior config + login. With purge = true the agent's /var/lib/hyperhive/{agents,applied}/<name>/ trees are also wiped (config history + creds + notes gone forever). The root/bootstrap container is destroyable like any other, and stays gone until you create ruth again from the swarm dashboard.

Meta flake

The hive-c0re-owned repo at /var/lib/hyperhive/meta/ declares one flake input per agent (agent-<n>.url = "git+<forge>/agent-configs/<n>.git?rev=<sha>") and one nixosConfigurations.<n> output per agent. Each output wraps inputs.agent-<n>.nixosModules.default with the identity + HIVE_PORT / HIVE_LABEL / HIVE_DASHBOARD_PORT injection module. Containers run against --flake /var/lib/hyperhive/meta#<n>.

The declared input url is the head of main on the agent's forge config repo (agent-configs/<n>, the one config PRs merge into), which sync_agents reads at render time. The flake text depends only on those shas and the agent set, so any machine that can reach the forge resolves it. sync_agents fetches each repo's main with the hive's read token (forge::reader_token()), pins its head, and takes the canonical follows lines from the flake.lock at that commit. If the forge is unreachable or a repo has no main, sync_agents fails for that agent with its name and writes nothing; there is no fallback to a local path. A config that sits only in the hive's applied/<n> mirror, not on forge main, never deploys. The rebuild paths re-lock the input with nix flake update agent-<n> against that same pinned url, so flake.nix and flake.lock name no local path.

Lock updates are single-phase and commit when the lock changed: meta::lock_update_for_rebuild(name) relocks one agent's input for a relocking rebuild (the manual ↻ R3BU1LD button, and the rebuild a merged config commit queues), and meta::lock_update_hyperhive() is the autoupdate flake-rev bump (one shot before per-agent rebuilds).

meta::sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) — hive carries hyperhive_flake, dashboard_port, and the rest of the per-hive config — is the idempotent reconciler called by provision_container, destroy, rebuild, and the startup migration. Renders flake.nix from the agent list; if it differs from disk, or flake.nix or flake.lock has an uncommitted change (a sync stopped before its commit), runs nix flake lock + commits as regenerate meta flake (or seed meta from N agent(s) on the first call). A clean tree whose flake.nix already matches returns without running nix.

The root agent has /meta RO-bound inside its container: git -C /meta log --oneline is the swarm-wide deploy log, cat /meta/flake.lock | jq '.nodes["agent-<n>"].locked' resolves which sha the flake pins each agent at right now. Dashboard surfaces the same info as a deployed:<sha12> chip per container row.

Two repos per agent

/var/lib/hyperhive/agents/<name>/config/    proposed — parent mount is RO
└── <anything>                              # any files the submitting
                                            # agent wants in the commit.
                                            # agent.nix is the
                                            # convention entry
                                            # point; flake.nix is
                                            # tracked boilerplate
                                            # (submitting agent doesn't
                                            # edit it).

/var/lib/hyperhive/applied/<name>/          applied — core-only
├── .git/                                   # tag-rich history
├── flake.nix                               # tracked, fixed
│                                           # boilerplate exporting
│                                           # nixosModules.default
├── agent.nix                               # working tree of main
└── <other committed files>                 # also tracked

/var/lib/hyperhive/meta/                    swarm-wide flake — core
├── .git/                                   # one commit per lock
│                                           # change
├── flake.nix                               # generated from agent set
└── flake.lock                              # pins each agent's sha

Why two physical repos: the submitting agent's /agents/<n>/config/ is RW — a buggy or hostile agent can git clean -fdx its own proposed tree. The applied repo is never bind-mounted (except the read-only .git exposure described below) so a destructive move inside the container can't reach it.

The container's --flake ref is /var/lib/hyperhive/meta#<name> (see "Meta flake" above). The agent's own applied/<n>/flake.nix is a fixed boilerplate that exports nixosModules.default = import ./agent.nix; the meta flake imports that module and wraps it with identity + HIVE_PORT / HIVE_LABEL / HIVE_DASHBOARD_PORT.

Tag state machine

hive-c0re plants deployed/0 on the seed commit at first spawn. A merged config commit that deploys plants no tag: the merged PR on the forge and meta's lock commits record it. A config PR nobody merges carries no extra state on the forge side: the PR stays open, and the submitter pushes again (or closes it) to retry.

Forge config repos

The bundled hive-forge container runs on the swarm's forge host (deploy.forgejo.enable, see ../swarm/services.md). Every agent created through the swarm has a repo in the private agent-configs Forgejo org, created and seeded by swarm-controller. hive-c0re pushes nothing to it: the applied repo's tags (deployed/0) exist only locally, and forge main advances only by an operator's merge. hive-c0re reads from it — reseeding a purged applied (forge::fetch_config_main_into_applied), catching applied up, and seeding proposed at first spawn (forge::clone_config_into_proposed).

A legacy agent spawned by the hive alone has no swarm-seeded repo; if its agent-configs/<name> has no main, a first spawn falls back to the local template, and reseeding a purged applied fails with a destroy --purge and re-spawn instruction.

Each agent is a write collaborator on its own agent-configs/<name> repo — so it can push a branch and open a config PR — but not a member of any other agent's, so it can't reach another agent's config through the forge. Branch protection keeps the agent off the main push allowlist and allowlists merging to the operators team only, so an agent can't fast-forward its own config or self-merge its PR (see Config changes above). hive-c0re passes the credential to each git call inline, never writing it into applied/<n>/.git/config; that repo is RO-bind-mounted into the root agent, and a stored token would leak core's admin credential to an agent.

The dashboard deep-links into this org — a config repo link per container row. See docs/web-ui/dashboard.md.

Submitting agent's view of config repos

An agent holding ManageRootAgent has every other agent's config repo bind-mounted read-only (hive-c0re/src/lifecycle/host_config.rs calls bind_child_agent_dirs for each entry in topology::all_agents()). it's a copy to read another agent's current config — not an editing surface. An agent without the permission sees no other agent's config at all.

An agent submits a change to another agent's config the same way it makes any other change: clone the target agent's config repo from the forge into its own state dir, commit on a branch, open a PR, and let the operator review and merge it. By design, no second, mount-shaped path reaches the same file without the review.

Agents holding the manage_root_agent permission (granted per agent in the swarm's agent-permissions record; see docs/process/conventions.md#permissions) get additional host-side bind mounts via set_nspawn_flags:

An agent without the permission only has its direct children's config dirs.

⚠️ nspawn bakes bind flags at container start, so hive-c0re restarts an agent, after its current turn, when its manage_root_agent changes.

No agent holds the permission by default: an agent whose record leaves permissions unset gets the default, which doesn't include manage_root_agent, whatever the agent's name. The swarm UI's setup assistant creates ruth with it. For a ruth that predates that, whose permissions were unset, swarm-controller writes the default plus manage_root_agent into her key once at start, so she keeps her mounts; it never touches a key that's already set.

Each proposed repo (/agents/<n>/config/) is pre-configured with applied as a git remote pointing at /applied/<n>/.git. Useful incantations from inside an agent with the full /applied mount:

git -C /agents/<n>/config fetch applied
git -C /agents/<n>/config log applied/main --oneline
git -C /agents/<n>/config show applied/refs/tags/deployed/0     # the seed commit
git -C /agents/<n>/config show applied/refs/tags/denied/<id>   # body = operator note
git -C /agents/<n>/config rebase applied/main                 # base in-flight work on what's deployed

git -C /meta log --oneline                                    # swarm-wide deploy history
cat /meta/flake.lock | jq '.nodes | with_entries(select(.key | startswith("agent-")))'

The RO binds block push at the kernel level — git plumbing inside the container can't corrupt either authoritative repo.

Startup migrations (older hosts)

hive-c0re runs a couple of idempotent migrations on every startup so a host set up before the tag-driven-deploy + meta-flake scheme (both described above) converges to it automatically. Each phase is a no-op once already applied:

No state loss in either migration: claude creds, /state/ notes, the events DB, and both proposed + applied history all survive. The root agent keeps its session; sub-agents stay logged in.

The root/bootstrap container

The root agent container runs through the same lifecycle as sub-agents. Nothing creates it at hive boot: you create ruth from the swarm dashboard like any agent, and an existing ruth container starts with the hive like every other. The root agent's flake lives at /var/lib/hyperhive/applied/ruth/; its proposed config at /var/lib/hyperhive/agents/ruth/config/. The root agent can edit its own agent.nix (visible inside the container at /agents/ruth/config/) and open a config PR on agent-configs/ruth for operator review, same as any other agent.

Differences from sub-agents:

Migration note (for older hosts): drop any containers.root = { ... } block from your host NixOS config, and any services.hyperhive.ruthless definition, then create ruth from the swarm dashboard.

Root-agent policy

The system prompt (hive-agent/prompts/system.md, rendered by hive-agent/src/prompt.rs) is the same for every agent; what varies is which MCP tools it surfaces (gated by the agent's permissions, which the swarm sets). No role:manager block renders only for the root agent. The root agent's role-specific behaviour comes from its AGENTS.md / agent-specific instructions, not the system prompt template.

Helper events

hive_sh4re::manager::HelperEvent is the wire type for an event urgent enough to drive an immediate claude turn (delivered as a regular system inbox message, same recv path as any other message). It currently has one variant, NeedsUpdate { agent } (sub-agent's recorded flake rev is stale, operator rebuilds from the dashboard) — and nothing in the tree constructs it; this removal dropped the push path (Coordinator::notify_agent) once its last caller (notify_manager, used only for the crash event below) went away. HelperEvent stays as the type for whatever comes along that genuinely needs to interrupt an agent's turn loop, not because anything live uses it today.

A previously-running container crashing used to be exactly that case (ContainerCrash, routed to the root agent via Coordinator::notify_manager) — it no longer drives any agent turn at all. crash_watch now only tracing::warn!s "container crash detected"; a swarm-level grafana alert matches that log line directly and pages the operator, with no agent (and no per-hive dashboard banner) in the loop. See docs/scheduler/observability.md.

The remaining lower-urgency lifecycle notices — Rebuilt, Killed, Destroyed, NeedsLogin, LoggedIn — are "FYI, check when convenient" events with no reason to drive an immediate turn, so they go out via swarm_notices::notify instead of a HelperEvent: subsystem = "core", key = "<event>:<agent>", and a single free-text summary (rebuilt_todo_summary renders Rebuilt's ok/note/sha/tag fields into that string).

To add a new lifecycle notice: if it's "FYI, check when convenient," call swarm_notices::notify — no new wire type needed. If it needs to drive an immediate turn, decide first whether it actually needs an agent in the loop at all (the crash case above didn't, once a swarm-level alert could carry the same signal) — only reach for a new HelperEvent variant + notify_agent-equivalent call site + prompts/system.md update when the answer is yes.

Autoupdate on startup

hive-c0re serve runs auto_update::run in a background task right after opening the coordinator. It enumerates managed containers and rebuilds any whose recorded hyperhive rev differs from the current one — sub-agents and the root agent go through the same job_queue::templates::rebuild DAG.

"Rev" = canonical filesystem path of services.hyperhive.c0re.hyperhiveFlake. Marker file: /var/lib/hyperhive/applied/.<name>.hyperhive-rev. If the flake input has no canonical path (for example a github: URL), autoupdate is a no-op — rebuild manually.

The dashboard surfaces pending updates per agent: a clickable "needs update ↻" badge appears whenever the marker differs from current rev. The badge POSTs /api/rebuild/<name>, which inserts the same job_queue::templates::rebuild DAG so manual triggers and the startup scan can't drift. When at least one container is stale, a top-level ↻ UPD4TE 4LL button appears that loops over every stale container.