| deploy.victoriametrics.agentAuthRate | Default 50r/s per client address, for agent reads checked by swarm-metrics-agent-auth; over it and agentAuthBurst, the gateway answers 403. | | deploy.victoriametrics.agentAuthBurst | Default 500 requests above agentAuthRate before the 403s start. | | deploy.victoriametrics.agentAuthVerdictTtl | Default 60 seconds that swarm-metrics-agent-auth keeps granting an agent secret it verified. A replaced or deleted secret keeps working at the metrics store for up to that long. | | deploy.victoriametrics.agentAuthPackage | The swarm-logs-auth build that runs as swarm-metrics-agent-auth. Set on its own: deploy.victorialogs.authPackage doesn't reach it. |

Swarm-wide services

Some things exist once per swarm rather than once per hive: the forge, the matrix homeserver, SSO, the secret store, the queue, and the metrics and log stack. This page says which host runs them and what a hive that runs none of them configures instead. The all-local quick start sets everything with one line → README.

Two options say where they live, and everything else derives:

services.hyperhive.deploy.singleHostSwarm = true;   # everything on this box
# or, for a dedicated services host with hives elsewhere:
services.hyperhive.deploy.allSwarmServices = true;

deploy.allSwarmServices is what "the swarm's shared services run here" means: every once-per-swarm service takes its enable from it. That's the whole rule, stated once — the per-service sections below don't repeat it, so a service that stops deriving is a visible difference rather than one more paragraph saying the same thing.

singleHostSwarm is the all-on-one-box mode above it. It defaults deploy.allSwarmServices, the swarm controller (deploy.swarm-controller.enable), the host's /etc/hosts entries for the names it serves (gateway.localHostsEntry) and where the secret store's bootstrap token goes (deploy.bao.bootstrapTokenFile). You can still set each derived toggle on its own, and that setting wins.

Both default to off, and that's deliberate: a host can't tell whether it's meant to be the swarm's service host, so this is an operator saying so rather than something inferred. With them off, a hive is a client of those services — it configures how to reach them and runs none of them.

That includes the forge (deploy.forgejo.enable). Every swarm needs one, being the canonical store for the meta flake and every agent's config repo, so exactly one host must turn it on: singleHostSwarm, allSwarmServices, or deploy.forgejo.enable = true set by hand. A host that enables its services one by one without any of those runs no forge.

A hive can't start or stop any of these. hivectl stop, start and restart act on the hive's agents and its gateway only, even on the host that runs the swarm's services. The forge and matrix containers are systemd units on that host (container@hive-forge.service, container@hive-matrix.service), and the operator manages them there with systemctl.

A hive that runs none of these reaches each one by name — the forge at forge.<swarm-domain>, for example. Only the host running a service answers its name from its own resolver, so on a swarm spread over more than one host, the operator's DNS has to resolve those names to that host.

Deployment shapes

Those two options are what makes the difference between deployments, so the shapes worth naming are the ones they produce:

• All-local. Everything on one machine: singleHostSwarm = true, plus deploy.hive-controller.enable = true for a hive to run agents on. After the first switch, the steps in setup.md remain.
• Services on the swarm controller host. Set deploy.allSwarmServices and deploy.swarm-controller.enable there, with hives elsewhere. The controller doesn't derive from allSwarmServices.
• Fully spread out. One container / VM / machine per service, somewhere.

These are a set, not a ladder with a correct top, and hyperhive supports in-between shapes. You can set each derived toggle on its own (see above), which is what makes "all local except X" a configuration rather than an unsupported edge case. Nothing in hyperhive prescribes a deployment model, so a doc that treats one shape as the real one and the others as compromises is wrong about the system rather than opinionated.

The practical consequence is that co-location is normal: in the first two shapes services share a host by design. Where a specific service has something to say about sharing a host --- extra requirements, or a cost worth weighing --- that belongs in the doc for that service, not here.

Services

SSO (authelia)

One authelia per swarm, in a swarm-authelia container, at auth.<swarm-domain>. Operator and agents are both subjects of the same provider, differentiated by roles and claims rather than by mechanism — there is one IdP and one auth path.

• deploy.authelia — run the container here.
• swarm.authelia.url — where clients go to authenticate. Present on every hive and the same value on all of them: https://<swarm.authelia.domain>, whether or not this host runs the container. Set it explicitly when joining a swarm whose IdP is under another name.

Agent subjects come from swarm-controller's agent-creation job, written into the users database by swarm-authelia-bridge; human ones come from swarmctl user add → setup.md § 2.

The bridge listens on loopback on the authelia host, and the gateway publishes it as https://<swarm.authelia.domain>/bridge/, which is where deploy.swarm-controller.authBridgeUrl points by default on every host. It answers only swarm-controller's OIDC client (swarm.controller.queueClientId) and refuses every other token 403.

On first boot this module seeds the users database with a disabled placeholder subject, so authelia has a non-empty store to start against before any real account exists (swarm-authelia.nix:873). Authelia generates session and storage keys in the container on first boot and never rotates them automatically; replacing one invalidates data already written (sessions, the encrypted store), so that's an operator action.

Storage is local sqlite and the notifier writes to a file. Both are small-deployment choices, and the scope is the justification: redis buys shared session state across replicas and there is one instance; SMTP exists to mail humans, and provisioning here is programmatic.

The swarm collector scrapes authelia's own metrics at https://<swarm.authelia.domain>/metrics, whichever host runs the container. The gateway admits only the collector's OAuth2 client there; any other session gets deny.

The container also runs swarm-secret-publish, which copies the client secrets authelia mints into the swarm secret store under the secret publisher's own store leaf, handed in as a system credential; see secrets.md. Authelia doesn't wait for it, so the IdP comes up while the store is down, and the publisher retries every 30 seconds. The container's journal forwarder reads its client secret back from the store under the same leaf, and is off where the container has none.

See sso.md for bootstrapping the first user and the OIDC relying-party flow, and secrets.md for where authelia generates and reads each of its keys.

Metrics (VictoriaMetrics + Grafana)

The swarm's telemetry lands in one VictoriaMetrics, and one Grafana reads it, in two containers at metrics.<swarm-domain> and grafana.<swarm-domain>. Two containers rather than one so you can restart or break Grafana without taking the time-series database with it.

They derive together: a store with no UI is unreadable and a UI with no store is empty. To run one without the other, set it directly:

services.hyperhive.deploy.victoriametrics.enable = true;
services.hyperhive.deploy.grafana.enable = false;

⚠️ This starts a database that grows for as long as the swarm runs. See retentionPeriod below before leaving it at its default.

Logging in. Grafana is behind swarm SSO, so the accounts are the authelia ones, and the local login form is always off. Grafana's API still accepts HTTP basic auth for its built-in admin account, so that account's password is a random one minted into the swarm secret store (swarm/services/<grafana client id>/admin/password) and applied on every Grafana start — see secrets.md.

Dashboards. Grafana provisions these from nix/host-modules/swarm-grafana/dashboards/; edits made in the UI aren't kept.

• agents: per-agent resource usage and Claude Code activity.
• authelia, bao, queue: the identity provider, the secret store and the NATS queue.
• claude-usage: Claude use by model, effort, token type and source. Filter by hive, service (agents or the review classifier) and agent. The service filter separates the classifier's API-key spend from the agents.
• controller: the swarm controller's job graph by state, and the push and commit rates it sees on the forge.
• forge: what the forge holds, and the forge process's CPU, memory and descriptors.
• gateway: the gateway nginx on each hive's host: requests per second, connections by state, and accepted against handled connections.
• collector: what the OTEL collectors accept, refuse, export and queue, with queue depth against capacity.
• logstore, metricstore: health of the log store and the metrics store.

A new panel can read "No data" until its source exists: the push and commit counters on controller appear after the first push webhook, and the refused and failure series on collector appear once something fails.

Where the data comes from. The swarm's OTEL collector, below.

The controller panels come from the controller itself. It exports OTLP through the gateway to the swarm's collector from whatever host it runs on, whether or not that host runs the collector.

Neither container is reachable except through the gateway: each listens on a unix socket only the gateway's nginx can open, and VictoriaMetrics runs in a network namespace of its own with no interface but loopback. VictoriaMetrics authenticates nothing itself; the metrics.<swarm-domain> vhost does it:

• The read API (the Prometheus query, series, label and metadata routes): Grafana's datasource, an agent, or an operator. Grafana sends a bearer that swarm-bao-grafana-metrics-bearer creates in the swarm secret store, and swarm-metrics-auth on the store's host checks it against the same store path. An agent's swarm-metrics sends its swarm-agent.<agent>.<secret> bearer, which swarm-metrics-agent-auth on the store's host checks against swarm/agents/<agent>/queue. A request without a bearer needs an operator's authelia session (group admins); without one it gets a 401, not a login redirect.
• vmui: an operator's authelia session, with a redirect to the login. The bare domain redirects to vmui.
• The collector's OTLP push, and its scrape of the store's own /metrics: only the swarm collector's authelia client. Authelia refuses any other session or client there, operators included.
• Every other path, the store's write and admin API included: 403.

swarm-metrics-auth and swarm-metrics-agent-auth read the store with one leaf, deploy.victoriametrics.authBaoClientCertFile / authBaoClientKeyFile. Where the secret store runs on the same host, these default to the leaf it mints. On any other host you place a leaf whose CN is deploy.bao.metricsAuthCommonName, and evaluation refuses the config until you set both. swarm-metrics-auth reads the stored bearer at most once a minute, so a replaced bearer starts working up to a minute later. After a failed or timed-out read it denies for three seconds without trying again. swarm-metrics-agent-auth is the log store's swarm-logs-auth binary, with the same grant TTL and rate limit, under deploy.victoriametrics.agentAuth* (table above).

The store's container has no network, so its journal forwarder leaves through a relay. Inside the container, swarm-victoriametrics-egress-relay listens on loopback and passes each connection to a socket the host's nginx listens on, /run/swarm-victoriametrics-egress/egress.sock. nginx sends the forwarder's route to the swarm collector and its token request to authelia, both by name and verified, and refuses every other path. On the host, swarm-victoriametrics-forwarder-oidc.service fetches the forwarder's OIDC secret under the same leaf as swarm-metrics-auth, and the container sees it read-only through a bind mount.

The collector scrapes Grafana's own metrics at https://<swarm.grafana.domain>/metrics, so a collector on a host without Grafana still gets them. The gateway admits only the collector's client to that path; any other session gets deny.

The collector scrapes the store's own metrics the same way, at https://<swarm.victoriametrics.domain>/metrics, so a collector on a host that doesn't run the store still gets them. On this name the gateway admits only the collector's client to /metrics and the push route; any other session gets deny.

Logs (VictoriaLogs)

Each hive ships its journals to one VictoriaLogs at logs.<swarm-domain>, behind the same SSO as everything else: the swarm's own service containers, the hive's daemons and infra containers, and the harness units inside every agent container. The collector below is what writes to it.

Reading them. Open Grafana, pick Explore, and choose the VictoriaLogs datasource — it's provisioned for you. Grafana's Logs Drilldown app is deliberately not installed: it only supports Loki, and no setting here changes that, so Explore is the log browser for this swarm.

It binds loopback and takes no credential of its own: the gateway vhost is the only way in, and the collector is the only intended writer.

Who may query. /select/logsql/ takes one of three credentials. An agent presents its swarm-agent.<agent>.<secret> bearer, checked by swarm-logs-auth. Grafana's datasource presents the same bearer as its metrics datasource, which swarm-bao-grafana-metrics-bearer creates, and a second swarm-metrics-auth instance (swarm-logs-grafana-auth) on the log store's host checks it against the same store path; the gateway tells it from the others by its shape, 64 hex digits. Anything else needs an operator's authelia session. Because one bearer opens both read APIs, a rotation (see secrets.md) restarts Grafana once, and the log store accepts the new value within a minute.

The collector also scrapes this store's own metrics, at https://<swarm.victorialogs.domain>/metrics, so a collector on a host without the store still gets them. On this name the gateway admits only the collector's client to /metrics and the ingest route /insert/opentelemetry/v1/logs; any other session gets deny on both. The read side stays open to any authenticated caller.

Telemetry collector (OTEL)

The swarm's collector receives from every hive's own collector and is the only process that decides where telemetry goes: it always writes the stores above, and exports to deploy.swarm-otel.upstream.endpoint as well when that's set. Which signals go there is deploy.swarm-otel.upstream.signals (default [ "metrics" "logs" ]); the stores always get both. It holds the upstream credential, which is why no hive and no agent needs one.

It runs in a swarm-otel container. Its swarm.otel.port defaults to 4319 rather than OTLP's usual 4318, which the hive tier uses — swarm containers share the host's network namespace, so two collectors on one port is a coin toss at runtime rather than an error at build time.

Every hive's own collector reaches this one by its gateway name, swarm.otel.domain (default otel.<swarm domain>) — the same by-domain-through-the-gateway shape every other swarm service uses, not a loopback URL an operator has to redirect. Nothing needs setting on a hive that doesn't run the swarm's services; the name resolves through the gateway either way.

The swarm-bao, swarm-nats, swarm-builder, swarm-claude-login, swarm-review-classifier, swarm-ci-runner, swarm-grafana, swarm-victorialogs, swarm-victoriametrics, swarm-authelia, hive-forge and hive-matrix containers each run a collector of their own that forwards the container's journal here, on a route and receiver per service (https://<swarm.otel.domain>/<container>/), authenticated as an OIDC client of that service's. The queue, builder, login minter and classifier reuse the service's own client; the store, CI runner, Grafana, log store, metrics store, authelia, forge and homeserver have a forwarder client of their own (swarm.<service>.otel.clientId). Their records carry service.name (bao, nats, builder, claude-login, review-classifier, ci-runner, grafana, victorialogs, victoriametrics, authelia, forgejo, tuwunel) and no hive label. The forwarder of all but the store's is nix/container-modules/swarm-journal-forwarder.nix.

The swarm collector's own container, swarm-otel, has no forwarder: its collector reads the container's journal itself (receiver journald/self) and exports it to the log store like the host's journal, stamped service.name swarm-otel and with no hive label. Unlike the host's journal, it goes to the log store only, never the upstream: the collector writes an upstream export failure to this journal, and shipping it upstream would fail again. A filter processor also drops the collector's own failure lines that name the store exporter (otlphttp/victorialogs), which can't reach the store anyway; failures of every other exporter still arrive. The container mounts the host's journal read-only at /run/host-journal, not over its own /var/log/journal.

Both store exporters are unconditional, and deploy.victoriametrics.enable doesn't gate them: that option says this host runs the store, while the swarm has one either way, reached by its swarm name through the gateway.

Agent-side configuration, and what a hive's own collector does, are in ../scheduler/observability.md.

CI runner

One Forgejo Actions runner, in a swarm-ci-runner container on the host that sets deploy.ci-runner.enable. It doesn't derive from allSwarmServices: a runner executes pull-request code, so running one is a decision of its own.

It needs neither the forge nor a hive on its host. It reaches the forge at https://<swarm.forge.domain>, and registers with the token swarm-controller publishes to the secret store, which it reads under its own ci-runner leaf. On a host without the store, that leaf is the one file to copy. Registration, the container and the threat model are in ../scheduler/ci.md.

Builder

The swarm builder, in a swarm-builder container on the host that sets deploy.builder.enable. allSwarmServices defaults it on wherever the host has the builder's store identity, as for the review classifier below: the host that runs the store has it. Set deploy.builder.enable = false to opt out. It evaluates unreviewed config PRs and builds them through its host's nix daemon, from a container with its own network namespace.

When an agent-configs/<agent> PR opens, reopens or gets a new push, swarm-controller queues a build_agent job. Jobs run one at a time, and a newer push replaces its PR's job while that job is still waiting. The builder builds that config at the PR's head on the agent base its own host runs, through the host's nix daemon, and swarm-controller writes the result as a swarm-builder commit status on the head. The job shows on the swarm UI's Jobs page. A red status blocks nothing: the merge still deploys.

It needs neither the forge nor a hive on its host. It reaches the queue, the secret store and the forge by name, under its own builder leaf, and fetches the private config repo with the read-only token of its own builder forge account. On a host without the store, that leaf is the one file to copy, as for the CI runner. What it builds and why are in builder.md.

CI metrics

The CI run poller, a swarm-ci-metrics oneshot on a 5-minute timer on the host that sets deploy.ci-metrics.enable. It runs on the host, not in a container. allSwarmServices defaults it on wherever the host has the poller's store identity, as for the builder above. Set deploy.ci-metrics.enable = false to opt out.

Each pass pushes the run time and queue wait of every newly finished forge Actions run to the metrics store, through its own route on the swarm collector; the first pass pushes every run since the first. It reads the forge with the read-only token of its own ci-metrics forge account, under its own ci-metrics leaf. On a host without the store, that leaf is the one file to copy. What it reads and pushes are in ci-metrics.md.

Review classifier

The review classifier's try-out service, in a swarm-review-classifier container on the host that sets deploy.review-classifier.enable. allSwarmServices defaults it on wherever the host has the classifier's store identity: the host that runs the store has it, and a services host with the store elsewhere gets the classifier only once it holds a copy of the pair (see below). Set deploy.review-classifier.enable directly to override either way.

The operator writes the key of the runtime the swarm settings choose by hand, in the secret store's UI under the OIDC login, field api_key:

• the API key runtime's Anthropic key at secret/swarm/services/review-classifier/anthropic-api-key
• the ACP runtime's provider key at secret/swarm/services/review-classifier/acp-provider

Nothing writes them for you: until the chosen one exists the service runs and every run fails with no API key… or no ACP provider key…. The service reads the key on every run, so writing or removing it takes effect on the next run, with no restart. Every run spends money on that key.

It needs neither the forge nor a hive on its host. It reaches the queue and the secret store by name, under its own review-classifier leaf. On a host without the store, that leaf is the one file to copy, as for the CI runner. The swarm UI's try-out drives it through swarm-controller; what a run is and what it costs are in review-classifier.md.

Claude login minter

The claude login minter, in a swarm-claude-login container on the host that sets deploy.claude-login.enable. allSwarmServices defaults it on wherever the host has the minter's store identity, on the review classifier's terms. Run one: a login in progress lives in the service's memory between its start and its code.

An operator logs an agent in from the log in button on its card in the swarm UI. swarm-controller relays the start and the pasted code over the queue, the minter runs claude auth login and writes the credentials to the agent's swarm/agents/<agent>/claude-credentials, and the agent's harness installs a version newer than the one it installed last.

It needs neither the forge nor a hive on its host. It reaches the queue and the secret store by name, under its own claude-login leaf, and reaches claude's login endpoints over the network. On a host without the store, that leaf is the one file to copy, as for the CI runner. The flow and the version rule are in claude-login.md.