The job queue, for operators

Every container operation — rebuild, first-spawn, a config-PR deploy, power changes — runs through one shared job queue. This page explains what the job queue is, as a general idea, independent of what any one subsystem uses it for. For the hive-c0re-specific step catalogue and the engineering internals (scheduler, leases, resource windows) see coordinator.md instead.

What the job queue is, in the abstract

"jobq" is a generic engine for running many interdependent jobs under limited concurrency — it has no idea what a "container" or a "rebuild" is. Two ideas are all there is to it:

• A job is a small graph of steps, not one opaque blob. Steps can depend on each other (this step only starts once that one finishes), so a big operation is really a short, ordered sequence — not a single black box that's either "done" or "not done."
• A step can need a shared resource, which only so many steps can hold at once (a "slot"). If every currently running step already holds the slots it needs, a new step that wants the same one waits its turn — that's the whole reason things queue instead of all firing at once.

The engine's whole job is: whenever a step's ordering and resource needs are both satisfied, run it. It has no opinion on what the steps do — that's supplied by whoever builds the graph. hive-c0re is the one thing building graphs on it today, but nothing about the engine is specific to containers or rebuilds; there's nothing stopping another subsystem from using the same engine for its own unrelated queue.

Watching it happen

Each row you see in a queue view (the BU1LDS page's R3BU1LD QU3U3 — see web-ui/dashboard.md — and swarm-ui's /jobs page both render the same underlying graph) is one job; the rows nested under it are that job's steps, in order (occasionally a couple run side by side). A step shows one of:

A step that isn't needed for a given run shows as · rather than being left out of the tree entirely, so the same kind of operation keeps a recognizable shape run to run, whichever steps it actually needed.