Architecture
Read this page to understand the source code or why a behaviour works the way it does. It covers the modules dispat is
assembled from, the algorithms behind the plan, the execution model, and the design decisions that shape both.
References like §N.N point into the separately versioned CCME specification, located at
specs/ccme-spec/SPEC.md. The specification document is licensed under
GPL-3.0-or-later. Quotations or adaptations of its text retain those terms.
This page is licensed under GPL-3.0-or-later. Source files that reference the GPL specification carry the same
license in their SPDX notices. The CCME parser and other source files without GPL material or substantive references
remain MIT. Official release binaries have a separate MIT grant; see the
license policy.
Runtime steps
Steps 1 and 2 run in the command-line controller, located in internal/cli behind the thin main.go of the dispat
binary. Everything from discovery onward runs in the app package's Status (steps 3 to 6) and Release (all steps).
You can call these same operations without a command line.
The Release function brackets planning and execution with the release lock.
After checking the Git repository and execution role, it pushes an annotated dispat-release-lock tag unforced to
commit.remote before planning. In a composed workspace it acquires every participating repository's lock before
planning. Normal cleanup deletes the owned locks from the remotes and clones under a detached context, so an interrupt
can still give them back. An authorized worker publication whose outcome cannot be established may instead retain its
repository's lock for operator recovery. If lock acquisition fails, the release stops before planning. Set
unsafeDisableLock: true in the config or DISPAT_UNSAFE_DISABLE_LOCK=true in the environment to skip the lock in a
local release; a distributed run refuses that bypass. The tag name is reserved in internal/gitx, so no tag format can
read it back as a release tag.
-
Parse the command line using pflag and dispatch one of the commands in the CLI reference. An unknown command word acts as shorthand for
run, likedispat lint.dispat reads your environment files (
./.envor the file--env-filenames) into the process environment before the phases start. It adds only what the environment does not already define. This happens first because dispat's own variables live there, including the update check's switch and the GitHub token.The controller runs its work in phases ordered by how much each needs to know. It refuses what the flags alone can refuse first, so a usage mistake never costs you a configuration error. Commands that read no config file run next, then the config loads, and finally everything that needs it runs. The commands that answer before loading any config are
init,self-update,install, and the three manifest commands (scanner,writer,replacer). Theifcommand joins them because a condition checks the environment rather than the repository.Run
initto write a starter config and exit before anything else. It refuses a--rootthat is not a git repository root. Theruncommand computes the plan and stops. It runs the script inside each changed package that has one over the dependency graph, using the build concurrency budget. dispat looks up the script in the package'sscripts, then its space's, then the file's. The--on-errorflag decides whether a failure skips the failed package's dependents.The shared
internal/filterresolver decides which packages this covers in three steps. First it finds a window: the release window, what--since <rev>addresses (scopes first, then changed files for scopeless units per §6.2), or every package for--since all. Next, the--package/--space/--groupterms narrow this window. The invocation folder stands in for any terms you did not type. Finally,--consumersexpands the result downstream.Nothing below step 6 applies to the
runcommand. Thepreviewcommand computes the plan quietly with diagnostics but no graph. It prints the pending release notes for every pending package in publish order under the same filter, then stops. Thescannerandwritercommands also answer before any config loads. They expose thepkg/scannerandpkg/writerlibraries directly (see Manifest tools). They read nothing but the paths named on the command line, making a monorepo root, a plan, and a git history beside the point. -
Resolve the config file in
--rootor by ascending its parent directories. The config's own directory becomes the effective monorepo root. A folder's.dispatexcludechooses between candidate names. A file declaringspacesends the ascent. A file declaring onlypackagesyields to a root above claiming its folder as a space. This tells a space folder's file apart from a monorepo of standalone packages. A file declaring neither is a package's in-folder override and does not end the ascent.dispat parses the file using its own format's parser, so every key keeps the case you wrote it in. It replaces each
$refwith the file it names relative to the file that wrote it, and refuses cycles with the chain. A copy of the result carries the flag bindings, and dispat's own decoder reads that copy into the model and validates it, folding a key to find the field it names, refusing any key the model has no field for, and refusing two keys of one object that differ only by case. Nothing is renamed on the way, so a map key reaches the model as its author wrote it.In source-history mode, workspace composition validates each initialized source head against its control gitlink and retains that exact commit as the plan's initial pin boundary. The later history load cannot silently replace it with a head that moved between composition and planning.
-
Discover packages across every direct sub-folder of each space path not excluded by the space's
.dispatexclude. Package names must be unique across spaces, case included, because every name in dispat is matched case-insensitively. Discovery includes every standalonepackagesentry with apath. Per-package configuration resolves here through a seven-layer ladder. It reads the root file's own defaults, the space, the space folder's config file, then the top-levelpackagesentry. It continues through the space'spackagesentry, the space file'spackagesentry, and the package folder's own file.Each configured package gets a derived space value carrying the merged configuration. A standalone package becomes a single-package space of its own. Everything downstream reads per-package behaviour without knowing these layers exist. dispat collects dependency declarations from every source (the root object, the entries, the in-folder files) into one merged list.
-
Build the dependency graph from the merged relations. dispat topologically sorts this graph. Cycles abort the run and print the named members.
-
Build the plan by resolving baselines, computing pending windows, and parsing their union. dispat applies corrections, cancellations, and holds before computing direct bumps. It runs the three propagation phases, then computes versions. Corrections rewrite the parsed stream in place. Every phase after them reads a stream it cannot distinguish from one authored that way. Every versioning group is versioned as one. This applies whether it is a space with its own shared mode or a declared
versionGroupsentry its members joined. -
Print the diagnostics, then narrow the plan to your
--package/--space/--groupselection. Theplan.Narrowfunction deselects every unselected releasing package. It withholds a selected package whose provider is releasing but unselected for the next run asW230. It reports a versioning group split by the selection asW231. dispat then prints the full graph withold -> newversions and channel transitions in publish order. It marks the deselected packages as such.Pass
--strictto refuse a selection carrying either finding. This refusal happens after printing the graph and before any release work. Thestatuscommand stops here. -
Refuse to release when the plan has a repository-scoped error. dispat also refuses any error at all under
commitErrors: "error". -
Verify remote and API access up front when
commit.pushor GitHub releases are enabled. dispat runsgit ls-remoteandGET /repos/{owner}/{repo}to fail fast before any release work starts. Setcommit.verify: falseto skip the git check. -
Run the gating run-level
beforeAllhooks. A failure here aborts the run before any release work begins. In source-history mode, revalidate every participating repository after all control and imported hooks finish. A relevant head, release-tag, or pin change isE330before the first package task. -
Execute the task graph with per-stage concurrency budgets. This runs a version, build, and publish stage per releasing package, excluding held packages, and a sign stage first for a package whose space configures one. The space's gating hooks bracket each stage (
beforeAll,beforeSign/postSign,beforeVersion/postVersion,beforeBuild/postBuild,beforePublish). A space'sflow.loginruns once before its first publish. Every other publish in that space waits on it. Source-history mode revalidates a package's owner plus the transitive provider and shared-version-group repository closure afterbeforePublishand before its publish command. The same check includes control when applicable control intent was consulted or a checkpoint will participate. Packages from one repository publish and record in a deterministic order; packages from different repositories retain the ordinary concurrency budget. Exact source pins admitted by nested native records are shared through a private coordinator bound to the current run so already-running sibling commands can validate fresh state; this transient data is removed with the run and never becomes planning history. -
Three things run in order after each successful publish. The release recorders run first, writing the changelog file and the GitHub release unless you are in release-commit mode. The annotated tag runs next. dispat defers this tag in release-commit mode, and a
PACKAGE_<KEY>script export pins it to the exported commit instead of HEAD. The warn-onlypostPublishhook and the warn-only announce frame (beforeAnnounce, the announce stage,postAnnounce) run last. Because the publish succeeded, none of this can fail the package anymore. A failure here is a critical. The recorders and the tag run detached from the run's cancellation and are bounded at five minutes, so an interrupt cannot cost a published package its record and a remote that stops answering cannot hold the run. -
Run the warn-only
postAllhook. It receives the run outcome through theDISPAT_RESULT_*variables. From here the durable records run on a context of their own: the run's cancellation does not reach them, and an interrupt gives them five minutes from the moment it arrives. An uninterrupted run is not bounded by it. The hooks stay on the live run: an interrupt stops the hook that is running,postAllincluded, and no later hook starts. -
Run the finalize phase when
commitis enabled. When a package whose version orsyncLockstage ran did not publish while another did, a single history first re-synchronizes thecommit.includepaths: the unpublished packages' tracked files and those paths go back to HEAD, leaving published folders and changelogs alone, and the published packages'syncLockscripts run again with an environment naming only what published (release.SettledEnv). A re-synchronization that does not finish restores the paths again and leaves them out of the commit, withE223. dispat then makes one release commit staging all published packages, then places tags on that commit or on a package's exportedPACKAGE_<KEY>commit. The push follows whencommit.pushis enabled. It pushes the branch first, then the run's tags, skipping and warning about any tag already on the remote. GitHub releases referencing the pushed tags come last. The warn-only commit and push hooks bracket these operations (beforeCommit/afterCommit,postCommitafter tags,beforePush/afterPush). Every package here has published, so nothing in this phase aborts it either. Each step runs, and each failure is a critical. Every Git command of this phase runs on the recording context of step 12, so an interrupt that arrives duringbeforeCommitor during the push stops the hook and leaves the commit, the tags and the push to finish. Recording that outlasts the grace is a critical (E223). -
Print a per-package summary and the totals. dispat exits
1if anything failed or if it recorded any critical.
Planning
The planner acts as a pure function of your history, graph, and configuration. It never consults wall-clock time or the outcome of any previous run. This makes re-running after a partial publish deterministic. The one date-derived input is git's newest-first tag ordering, which decides which tag counts as a package's latest. That ordering remains a fixed property of the repository and stays the same on every re-run.
In source-history mode, the planner passes repository input closures to release validation as immutable, interned bitsets in a shared repository order. Validation reuses those words and copies a set only when it must add an optional control checkpoint. Planning drops its temporary repository-name lists before returning the plan, so those lists do not remain reachable through the plan's ancestry callback.
Windows. dispat answers every question of the form "does this commit still count?" against a pending window. This
window contains the commits from a package's last stable tag to HEAD. Which package's window it consults depends
on the purpose. Conflating the two is the bug that loses releases:
| Question | Window consulted |
|---|---|
Does this unit bump P itself? | P's |
Does this unit set P's own channel? | P's |
Does this unit bump dependant D? | D's |
Does this unit re-channel D? | D's |
Reading the last two questions against the source's window silently orphans consumers after a partial publish. The
commit leaves the provider's window when it releases, making the unit lose its source packages. The consumer then never
releases on this or any future run. The target's window handles the ordinary catch-up. A consumer that has itself
released past the propagating commit, on a change of its own while its provider failed or was held, is asked the finer
question of §13.4a instead: whether some release of the provider carries the commit and the consumer's own release
reached it. Once the provider has published in a run the consumer sat out, no ordinary window holds the commit, so the
planner also reads the owed window of each provider and consumer pair (§13.3): the history after the newest provider
release the consumer's baseline reaches. It is taken for every consumer the provider reaches, not only the direct ones,
and read only where it reaches further back than the union of the ordinary windows; it admits nothing by itself. dispat
uses no repair pass, no second traversal, and no timestamp comparison anywhere in the package. Two releases on one
commit are the one state ancestry cannot order, so a run that would release a provider on the baseline commit of a
consumer it still owes, without releasing that consumer after it, is refused before any hook runs (E201), and a
consumer that failed after its provider was released there is reported with the exact Release-As that delivers it.
On a prerelease train, the window deliberately spans commits the train's prereleases already published. This lets §11.4
recompute the train's target and a graduation's version over the whole train. But published work remains published. A
commit contained in the baseline tag (the newest tag of any kind) still counts toward the bump and is discharged for
every question asking "is this still pending?". It cannot re-release the train, which prevents every run from emitting
beta.1, beta.2 from the same content. A Release-As it carries is consumed, and a cancel cannot discard it
because a prerelease tag is a published tag. This discharge extends to diagnostics. dispat does not re-warn a contained
channel directive as W199 because the tag records that it worked. It does not re-judge contained propagated proposals
(W200/W208). A cancel discharged for every package it names is spent rather than misaimed, so W170 stays for live
cancels only.
Propagation runs in three phases, and they may not be merged. Phase 3 reads what phase 2 produces:
- Channel axis: Each unit's
Propagate-Channel*directives propose a channel per package. dispat admits these against each target's window. - Channel resolution: Settle
channel(P)for every package. A direct directive beats every propagated one regardless of age. Among equals, the newest commit wins, followed by the last unit in it. dispat builds candidates by pushing from units to the packages they resolve to, rather than rescanning the window once per package. - Bump axis: Each unit's
Propagate*directives are admitted only where a source releases on a channel the target can resolve. This meansstableor the target's own channel. A stable consumer is not bumped by a provider's beta release. That would be a republish with no content, and a stable package would ship declaring a prerelease dependency.
There is no circularity in the other direction. Phase 1 reads only the units and the packages' baselines, never a value computed in this run. The channel axis admits a contribution only while its commit remains in the dependant's fresh window. The bump axis admits it there as well, and beyond it for as long as the provider still owes the dependant the commit: until a release of the provider carrying it is one the dependant's own release reached. The channel check also rejects a proposal when the package is already on that channel.
Both axes share the traversal. It runs breadth-first from the unit's source packages with a single-visit, shortest-path depth. dispat measures this from the originating source set and never re-bases on an intermediate. A package republishing as a catch-up does not propagate onward. Across a retry, targets only shrink while the corrected sources, traversal, and channel eligibility remain unchanged. If those inputs move, dispat surfaces any newly eligible targets for review before publishing.
Versioning groups. Packages with shared versioning are grouped by their resolved group key. This key is the space's
own name for a space with its own mode, or the declared versionGroups entry the space or package joined. A group may
span spaces. A mode carries one number called its shared depth. This defines how many leading version components the
group holds equal: 3 for fixed, 2 for fixedMajorMinor, 1 for fixedMajor, and 0 for independent. The group's
depth is the deepest any member declares. Sharing more implies sharing the prefix, and dispat reports a disagreement as
W237.
A declared group carries two further axes beside the depth: whether its members also hold one prerelease counter and one channel in common. Both default to shared. The three of them are one value object built once per group, and every question below is asked of it rather than re-derived: whether the group engages, whether a member is aligned, and what floor a member computes under. A shared counter beside independent channels is refused at load, because one counter counts one train and a train runs on one channel.
dispat versions each group as one virtual package after the per-package pipeline produces bumps, channels, and pins. The group aggregates what the version computation reads. It reads the baselines of every member, including held ones, so the shared version can never fall below a position a member already published. It also reads the bumps, new work, and channel movements of the members that would release. dispat runs that through the ordinary §13.9 computation, including pins, trains, and guards. This produces one shared next version, one prerelease train, and one pin per group. The newest member pin wins. Its scope breadth is one by construction because the group holds one shared prefix.
The engagement rule makes a partial depth work: the group engages when a part of the version it shares moves. A
movement of the shared prefix is that under every setting, the move onto the next train included. Beyond it the axes
decide. A shared counter makes the whole train the group's, so its later prereleases and its graduation engage it too,
and at the full depth every movement does, because the group shares the whole version. An independent counter leaves
the train's own progress to each member and keeps only the channel; independent channels keep nothing else at all.
dispat admits a pin to the group computation under the same rule. A pin naming the prefix the group already carries is
left to its own package's applyPin. This ensures a member's local Release-As is never measured against the group's
aggregate.
The channel a group moves on comes only from the members that are themselves moving between channels. A channel is derived from a baseline, so a member resting where its own tags put it proposes nothing: a sparse member left on stable while the rest ride an rc does not graduate the group, and a rider left on rc after the rest graduated does not drag them back. With independent channels the aggregate takes no proposals at all and decides only the core.
The member target floor is what lets a member compute a version its own window does not explain. A group member's §11.4 target and §11.5 graduation version are raised to the core of the group's line before the E185 and E195 guards read them. A ride carries none of the work that set the group's core and a failed leg carries only part of it, so without the floor a retry computes a version below one the group already holds. The floor never reaches past the line, so both guards keep their meaning. One exception: a member releasing on stable whose own baseline is stable takes no floor while the line is a prerelease, because the group has published no stable version of that core.
When the group does not engage, every member goes through the ordinary per-member versionOne with that floor. An
alignment pass then keeps the invariant. A member releasing below the group's prefix adopts it, which is how a sparse
member's first change lands it on the shared part. A non-sparse member with nothing pending whose baseline lags is
released at the prefix with W234. Under an independent counter, a member holding the prefix is aligned whatever its
counter says, so a member one prerelease behind is neither caught up nor raised; the floor does the raising instead,
and it raises only the part the group shares. The one member the floor declines to lift, one on stable while the line
is a prerelease, is brought to the prefix by the alignment pass on the line's channel at its own counter, because the
floor cannot change a channel and staying below the prefix is excused by no axis.
Assignment is where sparseness shows. The mode is each member's own, so a joined group can mix them. A plain mode
releases every non-held member at the group version. It marks members with no cause of their own as rides using W234.
This is non-suppressible and writes a "no changes" changelog entry naming the shared part. A sparse mode assigns the
group version only to members with a cause of their own. With independent channels the assignment is each member's own
computation instead, floored at the group's new core and left on its own channel, with two safety rules: a ride by a
member on stable follows a prerelease group version onto that line, and a ride never graduates a member. Scope
resolution remains untouched either way, so each member keeps its own units for changelog and release notes.
dispat preserves two convergence properties. A quiet group whose members agree on the prefix releases nothing. A
non-sparse member left behind by a failed ride or a mid-life adoption is re-released on the next run. It releases at
exactly the group's published version under fixed, and at the start of its own line under a partial mode.
The group's aggregate measures each member's pending work against the tag holding the group's baseline rather than the member's own. Work that tag already contains never moves the shared prefix a second time: a member whose leg failed after another member published their shared work catches up at the version that carries it, as its own release rather than a ride, and the member that published is not dragged into an empty re-release.
Module map
Read one line per package below. Each package's doc comment carries the full story, and the deeper design notes live in the sections below.
| Package | Role |
|---|---|
pkg/ccme | The commit-message parser: units, headers, directives, footers, scope terms, semver. Regex-free, single-pass, immutable; knows nothing of git or workspaces. Spec in SPEC.md. |
pkg/config | The generic configuration loader (own module): json/yaml/toml into one tree, $ref composition, the upward root ascent, a reflection-free setter-table decode that keeps a key's written case and refuses unknown keys and two spellings of one name, env layering and ref-aware format-preserving edits. Knows nothing of dispat's own model. |
pkg/models | The public configuration model (own module): the structs a dispat.json/.yaml decodes into, so external tooling and the integration suite author configs as typed values. |
pkg/manifest | Shared manifest vocabulary (own module): dependency kinds, the requirements-file name rule, PEP 503 normalisation; definitions the scanner and writer must apply identically. |
pkg/scanner | Deliberately lightweight manifest reader (own module): npm, Go, Cargo, Python, Composer, Maven, .NET, pub, RubyGems and the mobile formats parsed into one Manifest shape with declared names, versions, dependencies and local paths; bounded reads, partial results with joined errors. No lockfile resolution, no network. Exposed on its own as dispat scanner. |
pkg/writer | Format-preserving manifest writer (own module): byte-precise range and version rewrites for every manifest the scanner reads, atomic writes, and a result separating applied edits from ones deliberately left alone (a Maven property, a workspace inheritance). Exposed on its own as dispat writer. |
internal/cli | The command-line controller: flags, dispatch, exit-code mapping, logger construction. |
internal/app | The application layer: Status, Release, RunScript, AutoWriter, AutoReplacer, Preview, Compute, ScanManifests, WriteManifests, the step commands, the finalize phase and run-level hooks; wires every other package together. Its package sweep is the shared half of every command that covers a set of packages: the selection, the task graph, the concurrency budget and the skip cascade live there once, and each command supplies only what one package's work means. |
internal/config | Everything about dispat's configuration that is dispat's rather than configuration's: the fields tables, validation, package discovery, the space and per-package override merging, .dispatexclude over folder and config names, and the domain half of compute --write. Reading a file, following its $refs, the root ascent and the decode itself are pkg/config's; this layer supplies the model and the wording. |
internal/plan | The planner: windows, scopes, directives, propagation, channels, versioning groups; a pure function of history, graph and configuration. Plus Narrow, which restricts a computed plan to part of the graph for a filtered release (publish order withholds, versioning-group splits reported). |
internal/graph | Deterministic topological sort and the generic Scheduler/Drain pump described below. |
internal/release | The executor: the task graph, stage frames, hooks, login gates, native auto-versioning, DISPAT_* environment rendering, script outputs. Plus the release lock, the one-tag mutex a release takes on the remote before it plans. |
internal/execution | Distributed execution: the coordination protocol and its signing, the Git mailbox, the node pool and placement, output manifests with their capture, verification and installation, the worker's serving loop, and the run summary. It knows nothing about releases: the release path hands it stage frames and asks it where they ran. |
internal/changelog | Changelog rendering and the per-package record dispatcher. |
internal/github | The GitHub release recorder: REST calls, asset uploads, up-front verification, and the already-published probe that makes recording repeatable. |
internal/gitx | Git behind an interface: tags, baselines, commits, ancestry, tag formats; the CLI implementation shells out to git. |
internal/script | Shell script execution with process-group cancellation and bounded pipe waits. Traces the resolved shell, folder and command of everything it runs, which is the answer to "what did that stage actually execute". |
internal/selfupdate | Replacing the running binary with one downloaded from a release: version resolution, checksum verification against the published sums, the kept backup and its expiry, and the background check that prints the notice on the way out. |
internal/model | Resolved domain types (Space, Package, AutoVersion, record specs) shared by config, plan and release. |
internal/filter | The one selection resolver every package-selecting command shares, release and status included: --package / --space / --group terms, their globs, and the invocation folder that stands in for the first two. |
internal/globx | The one glob matcher scope terms, autoVersion.match, .dispatexclude and the selection terms share. |
internal/ignore | The change-scope ignore rules: ignore patterns and .dispatignore files compiled into one chain per package (repository, space, package), asked once per changed file after ownership is decided. Built on globx, so there is still one glob dialect. |
Graph algorithms
A handful of small graph algorithms and one graph-shaped structure carry the whole tool. This section maps what runs where.
Topological sort: the publish order
The internal/graph.TopoSort function uses Kahn's algorithm over package names with one refinement. The zero-in-degree
frontier is a min-heap. Ties always break alphabetically, so the same graph yields the same order on every machine
(§17.2). The heap makes the sort O (V log V + E): every package enters and leaves the heap once, and every edge is
relaxed once. dispat also reads reachable tags once and shares each history window between packages whose stable
tags point to the same commit. A package without a stable baseline requires the full reachable history.
a b frontier = nodes with no pending providers: {a, b}
\ / \ the heap pops alphabetically -> a, b, then c, d, then e
c d every provider precedes every consumer
\ /
e a cycle leaves nodes stranded with the frontier empty:
TopoSort returns a typed *CycleError naming them, which
the planner reports as E200 with the edges' manifest kinds
The cost is O (V log V + E): a heap operation per package on the way in and out, one relaxation per edge. dispat runs this once per plan.
The task graph: what a release actually schedules
Each releasing package contributes up to five task nodes. The sign node exists when the package's space enables
autoSign or configures any of flow.sign, flow.beforeSign and flow.postSign,
and it is the package's first node: it writes the package's own version before anything else of the release runs. A
package without that configuration has no sign node, and its graph is the graph it always was. The version node
exists when any provider of the package
moved in this run or its space auto-versions. Section §9.4 reconciles against every workspace dependency, so the
stage cannot be conditional on this run's updates. "Moved" means Release.Updates, which is deliberately wider than
DueTo. A provider releasing beside its consumer with no propagation between them still hands it a version to pick up.
Gating on DueTo made a scripted version stage and a native autoVersion block disagree about the same run. The
syncLock node exists when an autoVersion space configures the scripts. The build and publish nodes always exist.
core: build ─────────────► publish
│ │ │
│ (isBuildWaiting)──┘ │ (always)
▼ ▼
web: version ─► syncLock ─► build ─► publish
(The two provider edges land on web's first task, its version, and on its publish. With a sign stage, web's
chain starts sign ─► version, and the first provider edge lands on sign instead. Read the bullets below for the
precise rule.)
- What a consumer's first task (sign when present, then version, build otherwise) waits for on each changed
provider is that provider's relation: nothing under
{build: none}, the provider'sbuildunderisBuildWaitingPublish: false, and itsbuildand itspublishundertrue. The last covers the Docker case, where the consumer can only build after the base image is pushed; the first covers the deployment case, where nothing the provider builds reaches the consumer's build at all. - Build order is taken over the whole dependency graph and then restricted to the packages that build, exactly as
the publish order is. A provider reached only through a package with nothing to release still builds first, because
the consumer can read it through that package. A
{build: none}hop ends the constraint of every path through it: beyond it nothing is read. - A consumer's
publishalways waits for its providers' publishes. Publishing against an unpublished provider version would be broken under any relation. - Publish order is taken over the whole dependency graph as well, and no relation ends a path through it. A
provider reached only through a package with nothing to release still publishes first, because the consumer resolves
it through that package at install time whatever either build read.
{build: none}says a build reads nothing of the provider, never that a publication may overtake it. - When a changed provider fails or is skipped, its consumers are skipped unless they have a fresh release reason of
their own. A provider under a blocking relation skips them unconditionally: under
publishtheir builds consume the publish that never happened, and undernonetheir publications are the whole of what was meant to follow it, so a reason of their own cannot proceed them either way. - That cascade follows the same whole-graph reach as the publish order, so a provider that failed skips the consumers
that reach it through packages with nothing to release. Each skipped package's
blockedBynames the provider that actually failed rather than whatever stands between them. - The
syncLocknode sits betweenversionandbuildin a scheduling class of its own.
Drain: the one scheduling pump
The graph.Drain function consumes a Scheduler with bounded parallelism. This is the incremental core of Kahn's
algorithm. One coordinating goroutine owns all scheduler state. Every ready node gets a goroutine of its own.
Completions come back over one channel and cascade new ready nodes.
ready queues, one per class budgets
build: [web:version, ...] ◄── sign/version/build share BuildConcurrency
publish: [core:publish] ◄── PublishConcurrency
syncLock: [web:syncLock] ◄── min over the voting spaces (default 1)
loop: launch while budgets allow ──► goroutine per node ──► done channel
▲ │
└── completion cascades newly-ready nodes ◄──────────────┘
Per-class queues ensure a stalled class never blocks another class's budget. A node occupies as many of its class's
slots as its package's configured weight. dispat clamps this per-package concurrency override to [1, budget], so a
weight past the budget simply runs alone. A node that does not fit waits at the head of its queue, and nothing behind
it overtakes it. This head-of-line discipline makes a heavy node's wait finite instead of starvable. The syncLock node
keeps the ordinary cost of one. Its budget exists to serialise lock-file writers, not to price packages.
There are no locks in the scheduling hot path. A mutex guards only the shared result map that handles skip decisions, provider filtering, and the syncLock skip read. Every task finishes exactly once, including no-op completions for packages already failed or skipped. This lets the counting terminate without special cases. A cancelled context stops new launches, awaits in-flight nodes, and reports the rest as cancelled. dispat diagnoses a frontier that empties with nodes remaining as a cycle instead of deadlocking.
Two things drain. The release executor drains over (package, stage) nodes and the budgets above. The package sweep
in internal/app drains over package names and one budget. Every command covering a set of packages runs on this
sweep, including run, autowriter, autoreplacer, and the four step commands. It decides who runs, in what order,
under which budget, and what a failure does to the dependents. A command supplies only a packageWork. Given one
package's release, it hands back the work to do or nothing when there is none. The commit and github commands
declare themselves serial because a repository has one index and one HEAD. The rest ride the build budget. Adding a
command that covers packages requires only one small file rather than a copy of the scheduling.
The stage seam: where a frame may leave this machine
The task graph is the same graph whether a release runs on one machine or several. Only build and publish frames can leave this machine: the sign, version and syncLock frames write the working tree every delegated frame is snapshotted from, so they run on the orchestrator under the snapshot guard, as every manifest reconciliation must (§28.3 of the specification). Distributed execution adds no barrier nodes and no edges, because a second scheduler would be a second answer to the order a release already has. What it adds is one seam inside the executor: the call that runs a stage frame asks a coordinator first, and the coordinator either runs the frame here or places it on a node.
Three properties follow from putting the seam there. A run-wide stage budget is never multiplied by the number of nodes, because a task holds its budget slot from the moment it is ready and asks for a node afterwards. The orchestrator is a node of the same pool, so a frame pinned to it or placed on it for want of room takes the same path as a delegated one, outputs included. And the login gate is the seam's one exception: a space that authenticates publishes on the orchestrator, so nothing about a login ever travels.
Publication is serialized per repository whenever nodes are configured, by setting the publish group a single history would otherwise not need. The lock, the plan, the tags and the records stay where they were, on the machine the release was started on.
dispat run reaches the same coordinator from the package sweep rather than from the executor. With worker links the
task a sweep's packageWork hands back asks the coordinator to place it, as a frame of the kind run with the script's
commands and nothing around them, so the sweep keeps its own order, budget and skip cascade and only the place a task
runs changes. A sweep takes no release lock: its messages are bound to a generation derived from its own run identity.
What its tasks declare they write under runOutputs is admitted through the same validator as a build output set and
merged into the orchestrator's checkout once the drain is over, because a merge that ran as each task answered would
let arrival order decide a path two tasks disagree about.
The worker's answered-work record has one serving process, and worker.lock holds that process's id. A lock naming a
process that is gone is renamed aside, checked and replaced, and a lock naming a live process refuses the newcomer with
E225. Processes started together can each finish a takeover they read as a win, so every claim settles for a second
and is trusted only if the lock still names its process. A serving worker reads the lock again before each claim and
stops with E225 when another process's id is there, and release removes the lock only while it names the releasing
process. The answered-work tuples remain for 48 hours after acceptance, covering the full validity of a message
accepted with an issue time up to 24 hours ahead of the worker's clock. Loading and recording both prune only older
tuples.
The HTTP calls used for GitHub records, webhooks and self-update keep their request deadline through response-body
reads and Close, not just through the response headers. TinyGo's transport can ignore cancellation; a timed-out
call returns to its caller, but its underlying network operation may remain blocked. A process-wide limit of 64 such
exchanges holds a slot until each operation actually ends, bounding stranded work and refusing new work at its own
deadline when no slot is available.
Propagation: bounded BFS, three phases
Propagation (§9.2) walks the dependency graph outward from each unit's source packages. It moves along kind-filtered
edges and admits commits against each target's own pending window. dispat expands every package at most once per
phase. The walk remains O (V+E) whatever the directive's depth says. The symbols ^ (one edge), +N, and ^^/+*
(the closure) only change where the expansion stops, never how often a node is visited.
feat(core)^ feat(core)^^ / +*
core ─► api core ─► api ─► app bumps merge by max();
└──► cli └──► cli a package never re-propagates
(depth 1: (closure: api, a bump it merely received
api, cli) cli, app)
The three phases (channel proposals, channel resolution, bump propagation) may not be merged. Phase 3 reads what phase 2 settled. Phase 1 reads only units and baselines, which makes the channel axis converge.
Ancestry: three sources, memoised
Cancellation (§10.4) and prerelease-train containment ask "is commit a an ancestor of commit b?". The answer comes from
the strongest available source. dispat tries git itself (merge-base --is-ancestor), the commits' parent pointers
(BFS), or plain history order for linear fakes. It memoises every (a, b) answer on the computation. The same pair is
asked from several nested phases, and each uncached git answer spawns a subprocess.
c1 ── c2 ── c3 ── c4 ancestor(c2, c4)? git first;
\ parent-pointer BFS when git has
└─ c5 no answer (ErrNoAncestry); history
rank as the linear-case fallback
A real git failure aborts planning rather than silently degrading to the weaker fallbacks. A wrong ancestry answer changes which releases get cancelled or contained.
Failure semantics
Once the release work starts, one package error does not abort the run. Run-wide checks include the
release lock, a blocked plan, the branch guard, the behind-remote check, the
remote and GitHub verification, the beforeAll hooks, and the source-history fleet revalidation after those hooks.
Those refuse while nothing has happened yet. From the first build script onward, the task graph continues: a package
can fail, its consumers can be skipped behind it, and independent packages still run. The finalize phase records what
published. Only an interrupt or a task's internal error stops the graph early, and even then dispat records what
already published.
Source-history mode also has a package-local refusal point after work begins. Immediately before a package's publish
command, dispat revalidates the repositories whose history supplies that package's plan. Relevant drift is E330: the
package fails before publishing, its consumers remain gated, and independent work continues. This catches Git changes
made by build or hook commands after planning without claiming that dispat can exclude arbitrary external writers.
Inside that, there is a second and stronger line. Once a package's publish succeeds, nothing can fail that package. See After the point of no return below.
A failed script marks the package failed, but nothing aborts the run. The Result.FailedStage records where it failed
for the summary. At the start of every task, the package re-evaluates the skip rule. It skips if some changed provider
failed or was skipped AND the package has no fresh own commits, no channel change, and no successfully published
changed provider. A bump in the changeset its baseline has not published counts as fresh own commits, but own work an
earlier prerelease shipped on a prerelease train does not.
A consumer's terminal outcome is deterministic under every relation. Its publish always waits for its providers'
publishes, so a provider's publish failure is guaranteed to be seen at the latest there. With
isBuildWaitingPublish: true, provider outcomes are already final before the consumer's version stage. With false
or {build: none}, the consumer may spend a version or build on a release that its publish then skips. This is the
trade-off the weaker relations opt into. The version stage filters failed
or skipped providers out of the DISPAT_UPDATED_* variables. It skips its script entirely when it had providers to
pick up and none of them survive.
Failed or skipped consumers are not lost across runs. Nothing about this depends on tag creation times, because those
are not stable under merges, rebases, or equal timestamps. dispat uses them for reporting only. A commit propagates to
a dependant while the dependant's fresh window still contains it. A consumer that missed a run is simply still owed its
release. The DueTo field then contains an unchanged provider. It gets no task nodes, and the version stage passes
its released version. This case makes Updates a union of DueTo and this run's releasing providers rather than
either alone. A consumer that was never released while a provider has been is the same case, using the whole history as
its window. dispat labels such a release a catch-up and reports it with the origin's published version.
Admission is a release-ledger test. A successful baseline tag removes a contribution from the dependant's fresh window, so dispat does not re-admit that ledger entry. A catch-up keeps its reviewed version and target set while its baseline, corrected sources, traversal, and resolved channels remain unchanged. If those inputs change, the recomputed plan may expose a finite follow-up; dispat must surface it for review before publishing.
dispat records a skipped package as blocked with the responsible dependency. It is never silently absent from the summary. A package that was in the plan and produced nothing has to be accounted for.
A task that panics does not take the process with it. The executor contains the panic where the task starts and
turns it into that task's outcome: a package whose publish frame had not returned success fails at the stage the task
was running, with internal error and the panic's value as its reason (the stack is logged at debug level). A package
whose publish frame had returned success stays published, the panic becomes a critical of its record, and its
consumers are blocked as they are behind any incomplete record. The panic then stops the graph the way an interrupt
does: tasks in flight end cancelled and nothing new starts. The run itself is not interrupted, so postAll, the
finalize phase and the lock cleanup still run, and the run ends failed with exit 1. No onFail or onSkip script
runs for a panicked package, and its folder is reverted only under the ordinary revertOnFail rule, on a bounded
context of its own.
For spaces with revertOnFail: true, a failing package has its folder rolled back via the Reverter interface. dispat
runs git checkout -- <dir> and git clean -fd <dir> using gitx.CLI. This restores tracked files from HEAD and
removes untracked files, scoped to the package folder. The same rollback runs when a package is skipped after its
version stage already modified files. In commit mode that skipped case applies without revertOnFail as well, to
every folder the run owns: the start-of-run check proved it clean, and it is neither the root of the repository that
owns it nor a folder holding another package's folder (Executor.IsRunOwnedFolder). Reverts in one checkout take the
repository's mutation lock in turn, because two git checkout processes there would race on its index lock. dispat
logs a revert error, but the package keeps its original failure status.
Nothing is ever reverted after a successful publish, for the reason the next section gives.
After the point of no return
A package's publish script succeeding marks the point of no return. The artefact is on its registry, and no later step can take it back. From there, dispat has nothing left to decide and only things left to record. It writes the tag, the changelog entry, the GitHub release, the release commit, and the push.
When one of those fails, it is a critical. dispat logs it with its own code, counts it in the summary, and
otherwise ignores it. The run carries on to everything else it owed. The command still exits 1 at the end, so a
release that lost part of its record never looks green in CI.
| Code | What failed |
|---|---|
E220 | A release tag could not be created |
E221 | A release tag already exists at a different commit |
E222 | A release record (changelog entry, GitHub release) could not be written |
E223 | The release commit could not be made, or its shared commit.include paths could not be re-synchronized and were left out of it |
E224 | The push failed |
None of these fails the package because failing it would be a lie with consequences. The package is published, so
marking it failed would do three wrong things. It would revert the package folder, throwing away the version bump of
something already on a registry. It would run the onFail script. It would also skip every consumer, none of which has
anything wrong with it, and all of which have a real published provider to build against. None of that un-publishes
anything. The status stays published, and the summary line for that package turns into an error carrying what went
missing. You get told exactly what to go and repair.
For the same reason, nothing stops at the first failure. A changelog that could not be written must not cost the tag too. One package's tag failing says nothing about the next package's tag. A failed push must not withhold the GitHub releases that document the same work. Each step runs, each failure is recorded, and the exit code adds them up.
The E221 error is the one case that also declines to act. A tag already sitting at another commit is left exactly
where it is rather than moved onto this release. It is a record some earlier run made. With
force on, a tag moved here would overwrite the copy on the remote, turning one
local mistake into everyone's.
Design decisions
Interfaces decouple every side effect. This keeps the planner and executor unit-testable with in-memory fakes:
| Interface | Implementations | Used by |
|---|---|---|
gitx.Git | gitx.CLI (shells out to git) | plan |
script.Runner | script.ShellRunner | release |
release.Tagger | gitx.CLI | release |
release.Reverter | gitx.CLI | release |
release.ReleaseRecorder | changelog.FileWriter, github.Releaser | release |
The ReleaseRecorder interface is the extension point for publishing release data anywhere. Both current
implementations render the same changelog sections using the shared changelog.Format. Zero-value fields fall back to
defaults. They differ only in the destination, choosing a file prepend versus a REST call. Recorders run in order after
each successful publish. A recorder error is a critical (E222), and the remaining recorders and the tag still follow.
Other choices:
- Versions live in git tags only, so you have no version files to commit.
- dispat reads reachable tags once and runs one
git logper distinct stable baseline commit. Packages without a stable baseline share a full-history read. Different tags on the same commit share the same window. - It shells out to the git binary to match CI byte-for-byte.
- Script output streams line-by-line into the structured logger, keeping parallel package logs attributable.
- Ordering is deterministic everywhere. dispat uses alphabetical tie-breaks in the toposort and sorted space iteration in discovery.
Two diagnostics are deliberately not suppressible. Each explains a release outcome a reader of the commit log alone cannot account for. A catch-up and a channel-only release explain a package's presence in a plan. A blocked package and a suppressed propagation explain an absence.
Discovery runs in O (packages) time. Planning takes O (U·(V+E)) graph work per propagation phase, one bounded walk per unit, plus one tag inventory and one history read per distinct stable baseline commit. Execution adds O (V log V + E) scheduling overhead on top of script runtime.
That bound is easy to lose in two places, and both are guarded by construction rather than by care. A package needs two baselines, and asking for them separately runs the same tag query twice. The listing is the primitive, and both baselines are selections over it. The propagation phase split costs a second pass over the unit list and a second set of graph traversals. It does not cost a second pass over history. Commit walking, window computation, parsing, and scope resolution all happen once before either phase runs. Going to the object store is the expensive part of a run. In a repository running no prerelease train, no unit carries a channel directive at all, and the channel phase does nothing.
Deliberately out of scope
dispat implements the release computation. Some parts of the specification assume an engine that also reads package
manifests and knows about registries. The manifest half is native when a space opts into
autoVersion, whose parsing strategy covers every format the scanner reads; the
registry half is delegated to the version and publish scripts. This keeps dispat language-agnostic:
| Not implemented | Delegated to |
|---|---|
| Rewriting dependency ranges in manifests | native under autoVersion for every scanner-read format, npm and Go through the game engines (Supported formats), reported as W197/W203; indirections and versions no manifest holds: replace rules or flow.version, via the DISPAT_WORKSPACE_* variables |
| Manifest-vs-baseline version checks | native under autoVersion, or under autoSign where the sign stage owns the version (W192); otherwise nothing |
| Publish targets, registries, adopting published versions | flow.publish |
initialVersion / preserveMajorZero remapping | current behaviour: first release from 0.0.0, ordinary bumps |
| Per-run safety limits (max packages, majors, channel moves) | (nothing; the exact-pin major-jump guard is enforced, with a default of 1) |
| Post-run convergence verification | (nothing; re-run status) |
requireCodeownerFor, the CODEOWNER gate on directives | (nothing; the specification's own configuration for it is unmodelled, so no directive can be gated on approval) |
| Rolling a published release back (§26) | (nothing; a release is corrected by releasing again, and a record is restated with Edits or Deletes) |
| Reusing a build output across runs | (nothing; every distributed run builds what it needs, and an output set is bound to the run, the plan and the attempt that produced it) |
| Registry reconciliation after an unknown publication (§19.4) | the next ordinary run, which plans what is still owed. A publication whose outcome is unknown is reported as E228 and never re-authorized inside the run that lost it |
| A bundle service beside the Git transport | (nothing; a coordination branch in a Git repository is the whole transport) |
Distributing the package sweeps other than dispat run | (nothing; only a release's build and publish frames and a dispat run sweep's tasks are delegated) |
This has a consequence for the diagnostics registry. dispat never emits the specification codes that belong to these
registry-aware and audit-aware features. Errors like E197 (publish-order violation), E198 (registry identity
unverifiable), and E199 (convergence check failed) assume an engine that queries registries and audits its own runs.
Warnings like W195 (staleness audit) and W196 (published version adopted from the registry) belong to the same
features. dispat implements and emits every other code of the registry, including the repository-scoped bucket (E182,
E185, E191, E195, E196, E200) and the run-scoped E201, which refuses a run that would release a provider at
the baseline commit of a consumer it still owes and reports a consumer that failed after its provider was released
there. This includes the codes the registry leaves to the engine rather than to the parser. dispat raises E210-E213
and W209-W215 here. These resolve an Edits or Deletes target against history and report what the correction did
(see Correcting a record). Read Diagnostic codes to see
what each of those six means and what to do about it.
The optional polyrepository Git profile adds its own implemented diagnostic family. E330 rejects an invalid or
changed fleet repository snapshot, E331 rejects ownership crossings and unlisted nested repositories, and E332
rejects composition and override conflicts,
E333 reports a missing or ambiguous cross-repository boundary, or a control directive whose own gitlink snapshot pins
a source revision the active checkout of that source does not contain, and E334 reports semantics that would require
an order between incomparable source revisions. Publication and coordination failures use E335, E336, and E337.
A fleet with no control repository adds two more: E338 rejects links that do not form a tree, and E339 rejects a
repository identity that cannot be trusted to name one participant. Its two recoverable findings are W332, a link
only one of its two repositories declares, and W333, a roster that does not name every member of the composed fleet.
An explicit unsafe lock bypass emits W331, naming every repository it applies to.
An absent provider explicitly declared with external: true emits W330 and leaves that edge inactive for the current
snapshot. The diagnostic reference
gives the recovery for each code.
In the other direction, forty codes are dispat's own. They sit outside the specification's registry, attached to
features the specification predates or does not have. They are numbered from W220 and E215 upward. This clears the
registry, which ends at W215 and E213. It also clears W195 and W196, which the specification reserves for the
audit features above. The six codes of the distributed execution profile carry a second name beside the number, the
outcome category the specification's §28.9 requires, so a reader can switch on the class without knowing the numbers.
| Codes | Feature |
|---|---|
W234-W237 | Versioning groups: a ride, and the three conflicts a shared version can produce |
W220-W222, W225 | Manifest-derived: an ambiguous manifest name, a rewritten dependency with no configured edge, a replace rule that matched nothing, one package's manifests declaring different versions for it |
W223, W224, W226 | A record that is already there: a release tag, a GitHub release, a changelog entry. What makes the step commands re-runnable |
W227 | A commit.include path that names nothing on disk: the path is not staged, and the miss is reported so a typo cannot silently cost the release commit its artifact |
W228 | A step command invoked inside a run whose own replan drifted from the run's DISPAT_* environment: the record is aligned to the run and the correction is reported |
E219 | A step command inside a run it cannot align to, because the package is missing from its plan or the run's version renders a different tag. Nothing is written: a refused leg is re-runnable, a drifted record is not |
W229 | A wired dispat github running before the run's tag exists: GitHub would invent the tag at the default branch head, so the ordering is reported. The commit step belongs first |
W230, W231 | Releasing part of the graph: a package the publish order cannot reach yet, a selection splitting a versioning group |
W232 | An alias tag that could not be written |
W233 | A versioning group whose members sit on different major versions, so the newest one decides where they all land |
W238 | A Release-As footer naming a package whose versioning is none: the directive moves nothing |
W239 | A webhook delivery that did not get through: the endpoint kept refusing, never answered, or the queue was full. Warn-only by design, like W232: a listener that missed a notification is never a reason to fail the release it was watching |
W240 | Commit references are configured but some entry lines have no commit id to point at, so those lines render without one. The planner has a sha for every commit a Git implementation reports one for; a window key that stands in for a missing sha names nothing a reader could open |
W241 | A configured noChangesText that expanded to nothing, or to whitespace alone, so the entry carries the built-in line instead. The record is written and what it says is true, which is exactly why the substitution has to be reported: nothing in the file shows it happened |
W242 | The release push was refused because commits landed on the branch while the run was working, and the release recovered: it pulled them and merged them with its release commit rather than rewriting anything, so the tag still names the tree the run recorded. Warned rather than logged plainly because the branch the release went out on is not the one it was planned against, and the commits that arrived sit outside the tag's ancestry with no record of their own until the next run |
W243 | The release push was refused because commits landed on the branch while the run was working, and those commits changed the same content the release did. The release still completes: this release's side of every conflicting file is what the branch keeps, the other side is pushed to a release-conflicts/... branch of its own, and the changelog and the GitHub release both name the files and that branch. Somebody has to reconcile the two sides |
E220-E224 | After the point of no return: a tag, a record, the release commit or the push failing once a release is already out |
E215-E218 | The scanner command's gates: a local link still present under --verify-unlinked, none present under --verify-linked, a range --forbid-range matched, a range --require-range did not find |
E225-E229, W244 | Distributed execution: a configuration no distributed run could be executed under (E225), work refused for who asked for it (E226), input or output data that cannot be used as it stands (E227), a publication whose outcome the run cannot establish (E228), transport state the run could not leave in a safe place (E229), and the harmless half of that subject, leftover refs and writes outside the declared outputs (W244) |
All of these codes follow the registry's numbering conventions and blast-radius rules. dispat documents them where
their features are. The auto-versioning narrations W192, W197, and W203 are the specification's own §9.4/§12.4
codes.
Testing
The workspace is a set of modules under go.work with no module at the root. You run tests per module using
go test -C services/dispat ./..., and the same for each pkg/ module. Those are testify-based unit tests against
in-memory fakes. Every internal package and every pkg/ module has its own suite.
The integration suite is catalogued claim by claim in the test plan. Its coverage matrix maps each of its requirements onto the tests that prove it. The results are summarised per area in test results. The unit suites have no equivalent catalogue. What they assert is stated in each test's own name and doc comment. You can see what the whole suite reaches per package in coverage.
The composition claims live exclusively in the black-box
integration suite. This includes the
compiled binary, real git over a process boundary, exit codes, scheduling under real concurrency, and signal handling.
The suite builds the real binary and asserts on git state, JSON log events, and nanosecond execution timelines. The
services/dispat module itself hosts no end-to-end tests. A test that could only be satisfied by faking what the suite
can witness for real belongs in the integration suite instead.