> Documentation version: 3. [HTML page](index.html)
# The roc-automation manual
This manual describes an opinionated approach for managing a Roc package or
platform repository: how
its tooling is pinned, what its pull-request, release and nightly workflows
prove, and what every release contains. roc-automation holds the shared actions
that implement the parts every repository needs identically.
[Download the standalone PDF manual](roc-automation.pdf) ·
[View roc-automation on GitHub](https://github.com/lukewilliamboswell/roc-automation).
> **How to use this manual**
>
> - **New to the model**: Start with [How a Roc repository is maintained](#how-a-roc-repository-is-maintained) for the
> compilers involved, the three workflows, and what the model leaves out.
>
> - **Roc package**: [A Roc package](#a-pure-roc-package) says which parts of the model a
> package with no host code uses, and which it skips.
>
> - **Package maintainer**: Read [Pin the tooling environment](#pin-the-tooling-environment),
> [Lay out the repository](#lay-out-the-repository), [Examples and the examples archive](#examples-and-the-examples-archive),
> [Test in pull-request CI](#test-in-pull-request-ci) and [The three workflows](#the-three-workflows) in order.
>
> - **Platform maintainer**: Read the same chapters, then
> [Maintain a platform](#maintain-a-platform) for host bindings, target matrices and native build
> inputs, and [Publish linker inputs from a pull request](#publish-linker-inputs-from-a-pull-request).
>
> - **Routine upkeep**: [Manual maintenance tasks](#manual-maintenance-tasks) lists the changes that only a
> maintainer makes, and how to make each one.
>
> - **Adopting the model**: Follow [Adopt the model in a repository](#adopt-the-model-in-a-repository) and its
> acceptance list.
>
> - **Reviewing trust boundaries**: Read [Automation trust model](#automation-trust-model) and
> [Verify released artifacts](#verify-released-artifacts).
>
> - **Understanding the choices**: [Scope and design principles](#scope-and-design-principles) defines the scope;
> [Risk and control reference](#risk-and-control-reference) maps concrete risks to their contextual guidance
> and verification.
>
> - **Publishing component inventories**: [Publish and consume software bills of materials](#publish-and-consume-software-bills-of-materials)
> defines release SBOM scope, generation, validation and downstream use.
>
> - **Writing a project manual**: [Write project documentation](#write-project-documentation) covers the
> conventions this manual itself follows.
>
> - **Building documentation**: [Build documentation with roc-docs](#build-documentation-with-roc-docs) gives the utility
> commands and supported source profile.
>
> - **Changing the renderer**: [Maintain the documentation renderer](#maintain-the-documentation-renderer) covers renderer
> builds, architecture, fonts and verification.
>
> - **Changing roc-automation**: [Contribute to roc-automation](#contribute-to-roc-automation) covers the manual
> and the shared actions.
>
> - **Sharing automation**: [Share automation through executable utilities](#share-automation-through-executable-utilities) gives a
> working consumer starter, producer layout and utility interface conventions.
## How a Roc repository is maintained
This manual covers the shared tooling, workflow boundaries and evidence needed
to build, test and distribute Roc packages and platforms reliably. It explains
which concrete risks an opinionated practice addresses, how its controls work,
and what the resulting evidence can establish.
Product-specific security design and project governance remain with maintainers.
The model supplies reusable automation and integration contracts; it does not
establish that a product’s algorithms, native host or dependencies are secure.
### Scope and design principles
Shared tools own input validation, artifact identities, bounded execution and
diagnostics. Workflows own when those tools run, which source and artifacts
they receive, and the authority each job holds. Evidence connects the selected
inputs, actual execution and published bytes so a maintainer can investigate
failures and decide what to do next.
Each chapter explains its relevant failure or attack scenario, guiding principle,
specific controls, verification and remaining assumptions. The
[risk and control reference](#risk-and-control-reference)
indexes those explanations. It is a navigation aid, not another specification
or a claim that every control has been deployed in every consumer.
Experience and primary technical research inform the choices. A standard can
reveal an omitted risk; satisfying a framework is not itself the reason to add
a control. Prefer a shared, testable mechanism that addresses an identified
failure over a new service, permission or mandatory check with no clear benefit.
### Why a shared baseline helps
Keeping a Roc project in good shape means juggling several things that move at
different speeds. The compiler is still changing, and breaking changes are
routine. A bug can sit in the compiler, in a platform, in a dependency, or in
the project itself. Releases rarely line up: a package may need a platform
release that has not happened, or a fix may be merged long before users can
download it. Each repository that solves these problems alone solves them
slightly differently, and each difference is something a maintainer has to
remember.
A common baseline is the answer this manual offers. It records the practices
that have worked across these repositories and the reasons behind them, so that
a lesson learnt in one place is applied everywhere. The practices are
opinionated, and they are not finished: when experience shows a better way, the
baseline changes here first and the repositories follow.
One decision shapes most of the rest. The package or platform always meets the
newest compiler, because that is what its users will install. The repository’s
own tooling stays on a compiler the maintainer chose and moves only when the
maintainer moves it, so that scripts, checks and release steps keep working on
the day a new compiler breaks something.
### The model in one page
| Concern | Practice |
| --- | --- |
| Package compiler | The latest published Roc nightly. The package, platform and examples carry no pin. |
| Tooling compiler | ` roc-stable `: one release named in ` Blueprint.roc ` and repeated in each script’s ` roc ` header field, changed by a reviewed pull request. |
| Tools and toolchains | Declared in ` Blueprint.roc `, pinned by ` Blueprint.lock `, identical for contributors and CI. |
| Automation | Released utilities through Blueprint tasks; product-specific Roc scripts under ` scripts/ ` run by ` roc-stable ` where needed. |
| Repository examples | Depend on the working tree by relative path. |
| Pull requests | Scripts pass on ` roc-stable `; the working tree passes on the latest nightly; a platform’s committed host bindings match what that nightly generates. |
| A release | One immutable release holding the bundle, an examples archive pinned to that bundle, a documentation archive, and the manual where there is one. Every asset is tested as the exact bytes that are published. |
| Nightly | Runs the latest release’s examples archive on one selected nightly. Example failures require compatibility investigation; incomplete execution requires setup repair or rerun. |
| Upkeep | Blueprint, the lock, the tooling compiler, the scripts and a platform’s host bindings are each changed by hand, in a pull request. |
### Three compilers, three jobs
There are three compiler roles. A consumer of released utilities need not install
their implementation compiler; a repository with local source tooling names it
explicitly. Each role has one job, and they are never interchangeable.
| Compiler | Selected by | Used for |
| --- | --- | --- |
| Blueprint’s own; it has no command name | The ` blueprint ` binary | Evaluating ` Blueprint.roc ` |
| ` roc-stable ` | The release tag named in ` Blueprint.roc ` | Running the repository’s automation scripts |
| ` roc `, the latest nightly | Nothing: it is whatever [` roc-lang/nightlies `](https://github.com/roc-lang/nightlies) published most recently | Checking, testing, bundling and running the package, the platform and their examples |
A maintainer chooses the second and observes the third. The first is hidden:
` blueprint ` fetches it when needed, as described in
[Get ` blueprint `](#get-the-blueprint-binary).
Blueprint has a compiler of its own because Blueprint is itself written in Roc.
It has to evaluate ` Blueprint.roc ` before any environment exists, so it cannot
take its compiler from the environment that file describes: that would be a
bootstrap problem. Each ` blueprint ` release therefore fixes the one compiler it
was built with.
Three compilers is more than anyone wants, and the arrangement is accepted as
temporary. It follows from Roc having only nightly builds. Once Roc 0.1 is
released, Blueprint and most repository tooling are expected to settle on that
release, which leaves two compilers to think about: the released one that
tooling runs on, and the nightly that packages are tested against.
` roc-stable ` exists so that repository tooling keeps working when a nightly
breaks something. The package deliberately floats, because the question a
maintainer needs answered is whether the package works with the compiler users
will install today. "Stable" means only that the repository does not move the
tag without review. There is no shared registry of stable versions, and
repositories need not agree.
"Latest nightly" means the most recent release published by ` roc-lang/nightlies `.
This model treats that as the last good build and does not check upstream CI
separately.
CI installs that compiler as ` roc ` with the official
[` roc-lang/setup-roc `](https://github.com/roc-lang/setup-roc) action, which is
also how it is named on a contributor’s machine. The Blueprint environment
provides ` roc-stable ` and no ` roc `, so the two names never collide.
### What each workflow answers
**The three workflows and the question each one answers**
` ci.yml ` tests the working tree. ` release.yml ` tests and publishes artifacts.
` nightly.yml ` tests what users already have. Each can fail without the others
being wrong: a new nightly can break ` ci.yml ` on an untouched pull request, and
` nightly.yml ` can fail while the default branch is green because the fix has not
been released. [The three workflows](#the-three-workflows)
describes each in full.
### A Roc package
Roc packages contain Roc code. Platforms provide the host implementation and
may include native libraries.
The one-runner testing profile applies to a package repository when all of
these hold:
- it publishes a package and has no host or platform code;
- it fetches no native build inputs;
- none of its code behaves differently from one target to another.
Such a package is expected to work wherever Roc does. A difference between
targets is a compiler defect to report upstream, not something the package’s
own CI is there to find. It therefore tests on one runner.
| Concern | A Roc package |
| --- | --- |
| Runners | One. ` ci.yml `, the test job of ` release.yml ` and ` nightly.yml ` each run on ` ubuntu-latest ` only. |
| Required status checks | The job names themselves. With no matrix there is no aggregate job. |
| Examples | Run on a published platform, such as basic-cli, and are tested on the one runner. |
| Chapters that do not apply | [Maintain a platform](#maintain-a-platform) and [Publish linker inputs from a pull request](#publish-linker-inputs-from-a-pull-request): host bindings, native build inputs, rebuild selection and ` actions/publish-build-inputs `. |
Everything else applies unchanged: the two compilers, the Blueprint
environment, the Roc scripts, the three workflows, and what a release contains.
The cost is stated here so that it is a decision and not an accident: a
compiler defect that appears on one target only is not detected by this
package’s CI.
A package leaves this profile as soon as it gains behaviour that can differ by
target, for example when it starts a process, interprets file paths, or depends
on the width of a pointer. It then adds a runner for each target it claims, and
takes on the matrix and the aggregate job described in
[Continuous integration](#continuous-integration).
### What is shared and what a repository owns
Anything two repositories need identically belongs in roc-automation, referenced
by full commit SHA. A repository keeps only what knows its own package.
| Lives in roc-automation | Lives in the repository |
| --- | --- |
| ` actions/setup-blueprint `: install Nix and the verified ` blueprint ` binary | ` Blueprint.roc ` and ` Blueprint.lock ` |
| ` actions/nightly-report `: keep the nightly tracking issue | The test job of ` nightly.yml `, its matrix and its schedule |
| ` actions/build-docs `: render an AsciiDoc manual as a site and PDF | ` docs/*.adoc ` |
| ` actions/publish-release `: stage a draft release, verify every asset, then publish | What a release contains, and the jobs that build and test it |
| ` actions/publish-build-inputs `: publish linker inputs from a pull request | The producer workflow and its build recipe |
| The ` check-settings ` task: compare a repository’s GitHub settings with this manual | The settings themselves, applied by a maintainer |
| This manual | Thin caller workflows, ` scripts/ `, tests and expected output |
[Shared actions](#shared-actions) is the reference for each
action.
### What the model does not do
- It does not make package builds reproducible. The package compiler floats by
design; only the tooling environment is pinned.
- It does not verify upstream compiler quality beyond using a published nightly.
- It does not support holding a package on an older compiler line. There are no
compiler-compatibility branches: a release works with the compilers named in
its notes, and a patch release follows when a newer compiler breaks it.
- It does not describe application repositories that publish no package. They
can use the environment and script conventions, and have no release or
nightly workflow.
## Pin the tooling environment
` Blueprint.roc ` declares every tool a contributor and CI need. ` Blueprint.lock `
pins them. Both are committed. The ` blueprint ` command from
[roc-blueprint](https://github.com/lukewilliamboswell/roc-blueprint) realises that
environment, the same way on a contributor’s machine and on every CI runner.
The environment holds the tooling compiler, the tools the scripts call, and the
native toolchains a platform host or a test host needs. It does not hold the
package compiler, which is never pinned.
Roc packages can use ` Programs `, ` Fetch ` and ` Expect ` with no Nix source.
Follow [the complete Roc package adoption guide](#adopt-a-pure-package-without-nix). Native
platforms can declare Nix sources for their host toolchains and sandboxed builds.
### Risks and controls for tooling selection
A contributor and CI can run different tool versions, or a new compiler can
break the scripts needed to diagnose the product. Keep tooling identity explicit
and separate from the compiler under test. Blueprint recipes and locks select
the tooling bytes; local source scripts declare the corresponding compiler.
Verify configuration and run the same named tasks locally and in CI. Changing a
lock is an admission decision, not a harmless cache refresh. The lock establishes
selected identities; it does not establish that the selected upstream code is
free of defects.
### Declare the environment
``` roc
app [config] { pf: platform "" }
config = [
Name("roc-example"),
Packages("nix", From(NixPackages("github:NixOS/nixpkgs/nixos-unstable"))),
Input("roc", "github:roc-lang/roc-overlay"),
Environment(
"dev",
[
Overlays(["roc"]),
Command("roc-stable", "nix#rocpkgs.nightly--"),
RocPackages([
"https://github.com/roc-lang/basic-cli/releases/download//.tar.zst",
"https://github.com/roc-lang/http/releases/download//.tar.zst",
]),
Tools(["nix#coreutils", "nix#gh", "nix#git", "nix#unzip", "nix#zip"]),
],
),
Shell("default", [Use("dev")]),
Task("check", [Use("dev"), Run(["scripts/check_all.roc"])]),
Task("bundle", [Use("dev"), Run(["scripts/build_bundle.roc"])]),
Task("test-bundle", [Use("dev"), Run(["scripts/test_bundle.roc"])]),
Task("package-examples", [Use("dev"), Run(["scripts/package_examples.roc"])]),
Task("package-docs", [Use("dev"), Run(["scripts/package_docs.roc"])]),
Task("test-published", [Use("dev"), Run(["scripts/test_published.roc"])]),
]
```
In this example, ` Packages ` declares its Nix package source,
` Input ` declares the flake input and ` Overlays ` selects its role in this environment.
Source-qualified names such as ` nix#gh ` select the declared package source.
Each setting has one job:
- **` Command("roc-stable", ...) `**: Names the exact nightly release used for tooling.
The compiler comes from [roc-overlay](https://github.com/roc-lang/roc-overlay),
which packages recorded nightlies under their release tags. ` Command ` exposes
that one release as ` roc-stable ` without adding the tool’s own executables, so
the environment provides no bare ` roc `.
- **` RocPackages `**: Declares the content-addressed Roc bundles to prepare before
scripts start. For tasks and shells, Blueprint downloads missing bundles and
uses ` roc unbundle ` to verify their URL hashes before publishing them into Roc’s
package cache; these task-only bundles have no entries in ` Blueprint.lock `.
Sandboxed builds instead take their bundles from provider-locked sources. List
every bundle, including dependencies of dependencies;
` roc-stable deps scripts/check_all.roc ` prints them. Blueprint does not read
package headers and cannot add a missing one.
- **` Tools `**: Every program a script starts, and every toolchain a build needs: for
example ` pandoc `, ` zig `, a Rust toolchain or ` go `. A tool that a script calls
and the environment does not declare is a defect, because CI and contributors
then run different versions.
- **` Task `**: A named entry point. CI calls tasks, never script paths, so the
workflow and a contributor run the same command.
The two files divide the work. ` Blueprint.roc ` states which release is the
tooling compiler and which tools are needed, where a reviewer can read it.
` Blueprint.lock ` pins the overlay revision that supplies that release’s download
URL and hash, package sources, fetched programs and bundle sources used by
sandboxed builds. Task-only bundle identities remain in their committed URLs.
Commit the generated lock after review. Derived files and generated Nix stay in
` .blueprint/ `, which the repository’s ` .gitignore ` lists.
### Get ` blueprint `
Blueprint needs Nix only for Nix-sourced tools and sandboxed builds. For
those environments install [Nix](https://nixos.org/download) first. There are two
ways to get the ` blueprint ` command. Whichever you
use, take the same roc-blueprint release as the platform URL in the
repository’s ` Blueprint.roc ` header.
#### With Nix
Nix can run or install ` blueprint ` straight from the release tag, with nothing
to download or verify by hand:
``` sh
# run it once, without installing
nix run github:lukewilliamboswell/roc-blueprint/ -- run check
# or install it into your profile
nix profile install github:lukewilliamboswell/roc-blueprint/
```
Nix builds the command from tagged source on first use.
#### As a release binary
Each roc-blueprint release also attaches prebuilt ` blueprint- ` binaries
and a ` blueprint.sha256 ` file. Download the one for your system, check it
against the published SHA-256, make it executable and put it on ` PATH `.
CI always uses the release binary, through
[` actions/setup-blueprint `](#setup-blueprint),
which checks it against a reviewed SHA-256 on every run. Nobody builds the CLI
from source in CI.
#### What the command needs
The binary needs neither Python nor Roc installed. It embeds its fixed
configuration compiler, the nightly its release notes record. On first use it
installs the embedded archive into its content-addressed store and verifies the
recorded hash before extraction. It reuses verified bytes on later runs and
repairs a damaged archive from its embedded copy. Bootstrapping this compiler
needs no GitHub request; project tools and packages can still need downloads.
A compatible explicit ` ROC ` override takes precedence, then a compatible host
` roc `. An override must match the compiler required by the Blueprint binary.
` blueprint compiler-version ` reports the bundled compiler tag without evaluating
a project, installing a compiler or writing a cache. It describes Blueprint
itself even when a host override is selected. The report does not require the
repository’s ` roc-stable ` pin to match: update that pin separately after testing
the scripts that use it.
This compiler is separate from ` roc-stable ` for a reason: Blueprint is written
in Roc and must run before the environment it describes exists. See
[Three compilers, three jobs](#three-compilers-three-jobs).
### Work locally
A contributor needs ` blueprint `, Nix when their selected tools/builds use it,
and a current nightly installed
as ` roc `, the way Roc is normally installed. Nothing else has to be set up.
``` sh
blueprint run check # run a declared task in the pinned environment
blueprint shell # or enter the environment and run scripts directly
blueprint tasks # list the declared tasks
```
` blueprint ` is the outer command. Run it from an ordinary shell: there is no
` nix develop ` step first and no nested shell.
Inside the environment ` roc-stable ` is the pinned compiler, and the
contributor’s own ` roc ` remains visible for package work.
### Use the environment in CI
Every job that runs repository scripts sets up Blueprint first, then installs
the latest nightly, then calls a task:
1. Set up the verified ` blueprint ` binary, requesting Nix when needed, with
[` actions/setup-blueprint `](#setup-blueprint).
2. Install the latest nightly as ` roc ` with the official ` roc-lang/setup-roc `
action; see [Install the nightly compiler](#install-the-nightly-compiler).
3. Run ` blueprint run `.
Choose runners for which the selected release has reviewed binaries in
` actions/setup-blueprint/releases.json `, and verify that the declared tools also
support those systems. Cross-compiling a host in
[the builder job](#the-builder-job) does not establish that it
runs on the target.
The job needs no cache account and no token for this. A Nix cache is an
optimisation a repository may add; a cache that requires ` id-token: write `
widens a test job’s token and must be an explicit choice.
A change to ` Blueprint.lock ` changes the tools every build uses. Where a
repository fingerprints native build inputs, ` Blueprint.lock ` is one of the
fingerprinted files; see
[Keep rebuild selection narrow and complete](#keep-rebuild-selection-narrow-and-complete).
## Adopt a Roc package without Nix
A Roc package can run its checks and publish HTML, mirrored Markdown, an
` llms.txt ` discovery index and a PDF using
released executables. Blueprint owns the fixed downloads and their reviewed
lock. The package compiler remains the latest nightly supplied by CI. No local
installer, renderer build or Nix environment is needed for this path.
The working example is in ` examples/pure-package/ `. It uses a Blueprint
release, the matching platform URL and a published roc-docs renderer from a
tools release. Upgrade a utility by changing its recipe and reviewing
its new lock entries. The utility release and fixed tooling compiler do not
control which nightly tests the package.
### Risks and controls for utility provisioning
A cache can return damaged bytes, a task can expose a temporary executable
link, or a renderer recipe can accidentally choose the package compiler. Treat
provisioning, persistent handoff and product testing as separate contracts.
Blueprint rechecks locked downloads; the handoff resolves the executable’s real
store path; the floating compiler remains a distinct host input. The cold/warm
adoption jobs exercise those boundaries and compare actual executable identities.
A cache hit proves only that storage returned an entry. Supported provisioning
targets and external runtime programs still need explicit validation.
### Copy the starter
Copy ` examples/pure-package/ ` to the repository root, including ` .github/ ` and
` .gitignore `. Keep its Blueprint configuration and lock together. Replace the
package, example, expected output, notes and manual content, then review and
regenerate the lock when changing tool recipes.
``` text
Blueprint.roc
Blueprint.lock
.github/workflows/ci.yml renders the manual on a pull request
.github/workflows/release.yml the thin caller of the shared release workflow
.gitignore
package/main.roc
package/Greeting.roc
examples/hello.roc
tests/spec.json
docs/index.adoc
docs/releases/0.1.0.adoc
```
` release.yml ` is the whole of the repository’s release and pull-request
checking; see
[The shared release workflow for a Roc package](#the-shared-release-workflow-for-a-pure-package).
The documentation workflow is:
``` yaml
name: Check package and shared utility
on:
pull_request:
workflow_dispatch:
permissions: {}
jobs:
docs:
runs-on: ubuntu-24.04
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
persist-credentials: false
- uses: lukewilliamboswell/roc-automation/actions/setup-blueprint@1f3bc71102cf2d07c324b6009a41823efd21c9aa
with:
cache: 'true'
cache-workload: docs
- run: blueprint check
- run: blueprint run docs
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
with:
name: manual
path: .docs-out/
if-no-files-found: error
package:
runs-on: ubuntu-24.04
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
persist-credentials: false
- uses: lukewilliamboswell/roc-automation/actions/setup-blueprint@1f3bc71102cf2d07c324b6009a41823efd21c9aa
- uses: roc-lang/setup-roc@f5a74bf90aa694b43ab4a7cbfdb19273bc6aa697
with:
version: nightly-new-compiler
- run: blueprint run check-package
```
The documentation job consumes a fixed released executable. The package job
uses the current nightly; this single test job needs no matrix-wide moving-tag
coordination. Uploading a CI artifact grants no publication authority. The
starter’s declared Blueprint targets are Linux and Apple Silicon macOS; its
workflow uses Linux. See
[the producer and interface skeleton](#share-automation-through-executable-utilities)
when adding another shared operation.
### Declare the utility and tasks
Copy the example’s structure and choose your package name and PDF filename.
The renderer recipe names one release; ` {system} ` selects the release asset
for each declared system. ` roc-stable ` is a separately fetched fixed tool.
Bare ` roc ` is expected from the host and is never supplied by that recipe.
``` roc
# Package testing uses the latest nightly supplied by CI as bare `roc`.
# Fixed tooling and released utilities have separate names and locked bytes.
app [config] { pf: platform "https://github.com/lukewilliamboswell/roc-blueprint/releases/download/0.8.0/FbJymjYdCxx9vqMcau7r4tQfeAbnVGqqJoYrLGqPCqKu.tar.zst" }
import pf.Recipes
config = [
Name("roc-package-example"),
Systems(["x86_64-linux", "aarch64-linux", "aarch64-darwin"]),
Environment("docs", [Programs([Fetch("roc-docs", Recipes.file("https://github.com/lukewilliamboswell/roc-automation/releases/download/tools-0.2.0/roc-docs-{system}"))]), Expect(["realpath", "sh"])]),
Environment("fixed-tools", [Programs([Fetch("roc-stable", Recipes.roc_nightly("nightly-2026-10-18-1df370f"))]), Expect(["realpath", "sh"])]),
Environment("package", [Expect(["roc"])]),
# The shared release workflow runs this utility against the `roc` on PATH; a
# maintainer runs the same task locally, such as `-- test-examples --update`.
Environment("release", [Programs([Fetch("release-package", Recipes.file("https://github.com/lukewilliamboswell/roc-automation/releases/download/tools-0.5.0/release-package-{system}"))]), Expect(["roc", "git", "gh", "tar", "unzip", "zip", "chmod", "touch", "env"])]),
Task("release-package", [Use("release"), Run(["release-package"])]),
Task("docs", [Use("docs"), Run(["roc-docs", "build", "--workspace", ".", "--pdf-filename", "example.pdf"])]),
Task("docs-path", [Use("docs"), Run(["sh", "-c", "realpath \"$(command -v roc-docs)\""])]),
Task("tooling-version", [Use("fixed-tools"), Run(["roc-stable", "version"])]),
Task("tooling-path", [Use("fixed-tools"), Run(["sh", "-c", "realpath \"$(command -v roc-stable)\""])]),
Task("check-package", [Use("package"), Run(["roc", "test", "package/main.roc"])]),
]
```
` Expect ` declares ` sh ` and ` realpath `, which the handoff tasks use, and the
package compiler, which CI supplies. Blueprint’s own archive/store operations
also use the host’s ` tar `, ` ln ` and ` chmod `. GitHub’s supported Linux and macOS
runners supply these. The shared setup and optional diagnostic actions use
Python 3.10 or later on the runner; the renderer executable uses none of these
tools to parse or render documents.
Install the release binary matching the platform URL, following
[Blueprint’s installation instructions](https://lukewilliamboswell.github.io/roc-blueprint/getting-started.html).
Then run these commands from the example directory, or from your repository
after adopting its files:
``` sh
blueprint --version
blueprint update
blueprint check
blueprint run docs
blueprint run tooling-version
```
Review and commit ` Blueprint.roc ` and ` Blueprint.lock `. Check every utility URL,
per-system SHA-256, length and newly admitted download host. On an update,
compare the utility’s entries with the upstream release manifest. The setup
action’s full SHA fixes its reviewed Blueprint installer and checksum registry;
it reads the Blueprint release from the platform URL rather than another
workflow version pin.
Blueprint installs its embedded fixed configuration compiler into its
verified store before any Roc CLI is installed globally. No GitHub compiler
bootstrap download is needed. This compiler is separate from the downloaded
renderer and the floating package compiler; the example’s fixed ` roc-stable `
recipe also remains a separate reviewed pin. Roc package bundles occupy Roc’s
package cache and are not covered by the artifact cache below. Task-only
` RocPackages ` bundles are pinned by their URL hashes and verified on initial
unbundling, rather than listed in ` Blueprint.lock `. Existing extracted package
directories are reused without rehashing, as
[the compiler trust model](#compilers) explains.
The adoption checks read ` blueprint lock show --json ` format 2 and select fetched
programs by name and system to compare actual executable lengths and hashes.
This public report avoids depending on the lock’s internal representation.
### Restore downloads before the first task
In a read-only build job, call the shared setup action after checkout:
``` yaml
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
persist-credentials: false
- uses: lukewilliamboswell/roc-automation/actions/setup-blueprint@1f3bc71102cf2d07c324b6009a41823efd21c9aa
with:
cache: 'true'
cache-workload: docs
- run: blueprint run docs
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
with:
name: website-and-manual
path: .docs-out/site
if-no-files-found: error
```
The setup action installs a verified Blueprint binary without Nix by default.
Set ` install-nix: 'true' ` when a selected environment takes tools from a Nix
source or a sandboxed native build needs it. Existing configurations must follow
[the Blueprint upgrade procedure](#upgrade-blueprint), including
a full lock update when moving to Lock 2. Retain Nix while selected tools or
sandboxed builds depend on it. The privileged report/publisher actions retain their own reviewed tooling
and do not run this candidate repository configuration.
The cache contains only
` ${XDG_CACHE_HOME:-$HOME/.cache}/blueprint/v1/artifacts `. It is restored before
tasks and saved by the cache action’s post step after a successful job.
It does not include ` v1/trees `, ` .blueprint/run `, compiler build outputs or
Roc’s package cache. Each restored artifact is rehashed by Blueprint before it
is used; its length and identity must match the lock. A damaged artifact and
its derived trees are removed and the release bytes are downloaded again.
An unavailable or mismatching replacement cannot become a usable tool.
An exact cache hit is a transfer optimization, not verification. Restoring it
still transfers cache bytes from GitHub. The Blueprint executable has a separate,
default-on cache keyed by release, system and reviewed digest; restored executable
bytes are rehashed before use. Disable it with ` cache-binary: false `.
` binary-cache-hit ` describes verified executable reuse; ` cache-hit ` describes
the artifact store. Neither changes a version pin. Ordinary GitHub branch and pull-request cache
visibility applies: a PR can restore an eligible default-branch cache and its
new cache is scoped to that PR; a failed job does not save a successful cache.
Cache keys include system, Blueprint release, installer/registry identity,
workflow, logical tool set and lock digest. ` cache-workload ` is required when
caching is enabled. Give different tool subsets different names, including
matrix variants that fetch different programs. For example, a docs job uses
` docs ` and a tooling-check job uses ` fixed-tools `. Jobs intentionally sharing
the same complete tool set may share that name. GitHub caches are immutable:
a compiler-only job must not claim the key intended to contain the renderer.
A changed lock or installer produces a new key.
The repository’s ` Blueprint adoption ` workflow tests cold docs and fixed-tool
jobs, then restores each exact cache on another fresh runner. It compares actual
executables with the lock and checks that warm tasks do not refetch utility or
fixed compiler archives. A Nix sentinel fails any accidental Nix invocation.
See
[Blueprint’s store rules](https://lukewilliamboswell.github.io/roc-blueprint/no-provider.html#the-store-of-downloads)
for the detailed verification and rebuilding contract.
### Pass an executable to a shared action
Running ` blueprint run docs ` keeps the renderer’s options in the repository’s
task and needs no executable handoff. For a workflow using ` actions/build-docs `,
use the example’s ` docs-path ` task and the shared handoff action:
``` yaml
- id: renderer
uses: lukewilliamboswell/roc-automation/actions/blueprint-tool@1f3bc71102cf2d07c324b6009a41823efd21c9aa
with:
task: docs-path
label: roc-docs
measure: 'true'
output-directory: .docs-out
- uses: lukewilliamboswell/roc-automation/actions/build-docs@1f3bc71102cf2d07c324b6009a41823efd21c9aa
with:
renderer-binary: ${{ steps.renderer.outputs.binary }}
pdf-filename: example.pdf
```
The task must print exactly one absolute executable path. ` command -v ` alone
returns a temporary link removed when the Blueprint step ends. Resolve it with
` realpath ` inside the task, as the example does. The resolved store path and the
optional wrapper remain usable within that job. A new job restores artifact
bytes and runs the task again; it does not restore or reuse the old task link.
The handoff reports the actual executable’s SHA-256, length and provisioning
time. Blueprint’s successful ` Fetch ` task supplies lock verification; the
reported label is descriptive and is not another admission pin. The action
exports ` executable-path ` for the real file and ` binary ` for the optional
measurement wrapper. Renderer options remain task arguments or action inputs;
the handoff does not infer them from the task.
### Measure stages and retain acceptance checks
Optional measurements record wall time, maximum child-process RSS in KiB,
exit status, and output file count/bytes in JSON. Provisioning includes
verification and any downloads. The measured renderer command excludes the
host Python wrapper’s RSS and output-directory scanning time. If the command
starts an API compiler, its children contribute to the reported command cost.
Measure generated-input preparation and API generation as separately labelled
commands when comparing renderer costs. ` blueprint-tool ` exports
` measure-script ` for these stages; invoke it with the host Python interpreter:
``` sh
python3 "$MEASURE_SCRIPT" --label preparation --metrics "$RUNNER_TEMP/preparation.json" -- blueprint run prepare-docs
python3 "$MEASURE_SCRIPT" --label api --metrics "$RUNNER_TEMP/api.json" -- blueprint run api-docs
```
These task names are repository-defined stages, not built-in Blueprint APIs.
Keep browser and independent PDF-reader acceptance outside the measurement.
Review stable routes and anchors, exact source blocks, tables, figures,
release-specific PDF filenames and API assembly before changing publication.
Deploy jobs consume only the accepted site artifact and keep their write tokens
separate from all repository tasks.
### Keep the package nightly floating
The package-check job installs the latest nightly with the official setup action
and then runs ` blueprint run check-package `. It does not fetch a program called
` roc ` from the fixed-tooling environment:
``` yaml
- uses: roc-lang/setup-roc@f5a74bf90aa694b43ab4a7cbfdb19273bc6aa697
with:
version: nightly-new-compiler
- run: blueprint run check-package
```
For multiple package-test jobs, resolve the moving tag once and pass it through,
as [the nightly workflow](#nightly-compatibility) describes.
Caching fixed tooling and released utilities never delays package-nightly
updates. Native platforms can retain Nix toolchains and sandboxed build inputs
while using the same released documentation utility and artifact deployment.
## Share automation through executable utilities
Use released utilities for common repository operations. Blueprint provisions
their reviewed bytes and declares named tasks; the same tasks run locally and
in read-only CI. Actions add GitHub inputs, outputs and artifact handling around
the shared implementation. Keep product-specific cases and metadata in the
consumer repository.
### Risks and controls for shared utility contracts
Copied automation can drift, and a consumer can combine an executable with
an incompatible helper or result parser. Share versioned executable contracts
and distribute the complete declared runtime. Blueprint locks the selected
artifacts; operation-specific schemas identify inputs and results; exact-artifact
tests run the distributed files without a source checkout. Include malformed
inputs, missing companions and target behavior in those tests. An executable
version does not make every future schema compatible, and cross-compilation
alone does not demonstrate native execution support.
### Start from a working consumer
The [consumer walkthrough](#adopt-a-pure-package-without-nix)
owns the copyable ` examples/pure-package/ ` starter, its configuration and workflow,
download verification, caching and executable handoff.
The renderer is the worked example; apply the same provisioning pattern to
other released utilities. Keep the utility’s implementation compiler separate
from the compiler passed to check a product. Consuming a compiled utility does
not require adopting its basic-cli or tooling compiler in local source scripts.
### Give each layer one job
| Layer | Responsibility |
| --- | --- |
| Blueprint | Select and verify executable/runtime artifacts, provision environments and expose named tasks. |
| Utility | Validate operation-specific data, perform the operation, retain diagnostics and return an explicit result. |
| Action or reusable workflow | Supply GitHub context, map documented inputs and outputs, transport inert artifacts and restrict permissions. |
| Product repository | Declare its paths, cases, release metadata, native prerequisites and product-specific behavior. |
Prefer a Blueprint task that invokes an existing utility. A local Roc script is
appropriate when the product needs additional composition or behavior. General
command and filesystem helpers remain implementation details unless a concrete
in-process consumer justifies a public source API. Do not require consumers to
import a shared helper package merely to use an executable operation.
### Structure a utility producer
Use the existing tools build and release pipeline. For a new operation, this
layout separates the executable boundary from pure decisions and contract tests:
``` text
tools//
main.roc # CLI and effects
src/Config.roc # versioned input decoding and validation
src/Plan.roc # pure decisions and state transitions
src/Report.roc # structured results and human diagnostics
test_config.roc
test_plan.roc
build.roc # fixed build compiler and explicit targets
tests/test_cli.py # tests the distributed executable
tests/fixtures/ # small valid and invalid inputs
README.adoc # CLI, configuration, targets and runtime contract
actions//
action.yml # GitHub integration
.github/workflows/release-tools.yml
```
These are responsibility boundaries, not a requirement to relocate established
tools. Reuse ` tools/roc-policy ` for repository policy and ` tools/roc-docs ` for
documentation. Use the shared published-release compatibility runner for
immutable release suites. Build distinct operations only where their behavior
is not already supplied by an existing tool.
Keep build and executable tests read-only. Test the exact release bytes and
companion files on every claimed execution platform without a source checkout,
Nix or the utility’s build compiler. Explicit external runtime programs may
still be required. The existing publisher receives the tested files as inert
bytes; it never runs a candidate utility or consumer Blueprint task.
### Define a small, versioned contract
For a new checking utility, document its version command, operation arguments,
configuration format, output directory and result schema. Use structured argv
without shell evaluation. Keep configuration in the existing operation-specific
format, such as the examples specification; do not duplicate Blueprint’s
provisioning data in a second project manifest.
Checking and updating are distinct operations. A check never rewrites expected
output. An update is explicit maintainer work and refuses immutable published
suites. No utility changes a compiler pin as a side effect of checking a product.
Keep process status and structured evidence consistent. A useful convention for
new checking interfaces is exit 0 for complete success, exit 1 for completed
checks with findings, and exit 2 for invalid inputs or incomplete evidence.
Document existing tools' actual statuses rather than silently translating them
to a new convention. Machine results have a schema version independent of the
tool version and record the operation, outcome, completeness, relevant source,
compiler and artifact identities, counts and safe relative diagnostic paths.
Write a completed result atomically to the requested output directory.
Preserve bounded stdout and stderr on successful as well as failed compiler
calls. Warnings remain visible and nonfatal by default. A timeout, cancellation,
skip or missing report cannot establish a complete pass. Bound child execution
and clean up descendants. Escape workflow commands in console output; raw
diagnostic artifacts retain the original bytes. Validate paths and reject
unsafe aliases or special files before filesystem effects.
Test malformed inputs and failure transitions as well as successful execution.
Use tiny fixtures for warnings, missing tools, wrong identities, timeouts and
cleanup. Fuzz and benchmark smoke workloads stay within the
[one-second budget](#heavy-runs-stay-out-of-pull-request-ci);
sustained campaigns run separately.
### Package the complete runtime
Prefer a self-contained executable with embedded data when practical. A utility
that needs helpers ships a declared runtime set: executable, helpers, licenses
and their manifest. A per-system archive can keep these files together.
Blueprint’s ` Archive(url, TarGz, member) ` recipe selects the executable while
preserving its archive siblings. Lock the archive digest and length, and test
the unpacked runtime as distributed.
Resolve helpers relative to the real executable, following Blueprint’s task
link, or accept explicit helper paths. Declare external programs such as Python
in the environment. A binary that needs a helper is not self-contained, and a
helper version must agree with its executable. Never rely on an undeclared
source checkout to provide runtime files.
When an Action needs the executable after a Blueprint task ends, use the
[shared executable handoff](#pass-an-executable-to-a-shared-action).
Return a persistent resolved store path rather than the task’s temporary link.
Action code and privileged controllers remain pinned to full reviewed SHAs;
candidate tasks and executables run only in jobs without write authority.
## Lay out the repository
A consistent layout lets the same checks, scripts and workflows apply to every
repository, and lets a maintainer who knows one repository find their way in
another.
### Risks and controls for repository source boundaries
A test can accidentally import a published dependency instead of the changed
working tree, while source automation can accidentally depend on the broken
product it must diagnose. Make source roles explicit. Product examples use
relative working-tree dependencies; source tooling uses independently selected
released dependencies and a fixed compiler. Header and source-policy checks
verify those conventions, and behavioral tests verify the product itself.
Static policy establishes declared structure, not which code paths execute or
whether a package is correct.
### The layout
``` text
Blueprint.roc development environment and named tasks
Blueprint.lock pinned inputs
package/main.roc a package; or platform/main.roc for a platform
examples/ applications that depend on the working tree
scripts/*.roc executable automation entry points
scripts/src/*.roc modules shared by those scripts
tests/ package-specific cases and expected output
docs/*.adoc the manual, where the repository has one
```
A repository that publishes several packages or platforms gives each its own
root directory, such as ` package/ ` and ` tzdb/package/ `, or ` platform-web/ ` and
` platform-gui/ `. Each root is bundled, versioned and released together with the
others in one release; see
[Several bundles in one release](#several-bundles-in-one-release).
### Where a compiler pin belongs
The ` roc: "..." ` header field appears in exactly one place: the scripts under
` scripts/ `, where it names the same release as ` roc-stable `. Nothing else is
pinned. The package, a platform, the examples and every other Roc application
carry no ` roc ` field, and the repository has no ` .roc-version ` file and no
` .github/roc-nightly.json `.
The rule for the package applies to platforms as much as packages. ` roc check ` exits with an error
on a warning, and a dependency whose header pins a different nightly produces a
version-mismatch warning. A pinned platform therefore fails every application
that floats, and a package cannot float until every platform its examples use
has dropped the field.
The check script enforces both halves. It fails when a ` roc ` field or
` .roc-version ` appears outside ` scripts/ `, and it fails when a script’s pin
differs from the tooling compiler; see
[Keep the script pins in step](#keep-the-script-pins-in-step).
### Roc applications in the repository
Every Roc application in the repository depends on the working tree by relative
path. Pull-request CI checks them with the latest nightly compiler. That covers
` examples/ ` and also anything outside it: fuzz targets, benchmarks,
documentation examples, test cases, and a site generator.
**In the repository: the package is a relative path**
``` roc
app [main!] {
cli: platform "https://github.com/roc-lang/basic-cli/releases/download//.tar.zst",
ansi: "../package/main.roc",
}
```
The package is a relative path. Every other dependency, including the platform,
is an immutable release URL with its content hash.
These are not what the nightly workflow tests. They change with every pull
request and may use APIs that no release has. The nightly workflow runs the
examples archive attached to the latest release. In the archive the same example
names that release’s exact bundle URL, and nothing else differs:
**In a release’s examples archive: the package is that release’s bundle**
``` roc
app [main!] {
cli: platform "https://github.com/roc-lang/basic-cli/releases/download//.tar.zst",
ansi: "https://github.com//roc-ansi/releases/download//.tar.zst",
}
```
The release workflow writes the second form when it builds the archive; the
repository only ever contains the first. See
[The examples archive](#the-examples-archive).
The single exception is ` scripts/ `. A script never depends on the repository’s
own package by relative path, because a script must keep working while the
package is broken.
An example that needs more than one file is a directory: ` examples/report/ `
holds ` main.roc `, its sibling modules and any data files it reads. A local
helper package that several examples share sits beside them and is referenced by
relative path.
### Use shared utilities through Blueprint tasks
Use a released executable for an operation shared across repositories, with
its recipe and named task in Blueprint. Local use and CI invoke the same task.
The
[shared utility skeleton](#share-automation-through-executable-utilities)
shows a working consumer and the producer boundaries. A consumer of a prebuilt
tool does not need that tool’s source dependencies or implementation compiler.
Write a local script only for behavior or composition specific to the product.
Keep common validation, example execution and release orchestration in their
shared utilities rather than copying helper modules between repositories.
### Write product-specific automation as Roc scripts
A source entry point kept under ` scripts/ ` is an executable Roc application:
``` roc
#!/usr/bin/env roc-stable
app [main!] {
cli: platform "https://github.com/roc-lang/basic-cli/releases/download//.tar.zst",
roc: "nightly--",
}
```
- Start with the ` roc-stable ` shebang and commit the file as executable.
- Set the ` roc ` header field to the release that ` Blueprint.roc ` names as
` roc-stable `.
- Reference every dependency by immutable release URL with its content hash,
and list the same URLs under ` RocPackages ` in ` Blueprint.roc `.
- Run package work with ` roc `, which is the latest nightly in CI and on a
contributor’s machine. Never use ` roc-stable ` to check, test or bundle the
package.
- Put pure logic in ` scripts/src/ ` modules with ` expect ` tests, and run those
tests from the check script.
- Pass arguments after ` -- `, for example ` scripts/test_goldens.roc -- --update `.
- Declare every entry point as a ` Task ` in ` Blueprint.roc `.
Product orchestration scripts are written in Roc. Small platform adapters and
independent test harnesses may use the runtime their documented contract
requires. A script that only wraps one command is replaced by a Blueprint
` Task ` that runs that command directly.
#### Keep the script pins in step
A script is written for one compiler and one set of dependencies. Someone who
runs it with a newer ` roc `, instead of through ` blueprint `, may find it broken
for reasons that have nothing to do with their change. The ` roc ` field makes
the compiler say so: a compiler other than the one the field names refuses the
file and names the release it targets, which points at the cause before anyone
debugs the script.
The field is useful only while it is true, so one cheap check keeps it true.
The check script reads the tag from the ` Command("roc-stable", ...) ` setting in
` Blueprint.roc ` and requires every file under ` scripts/ ` that has an ` app `
header to carry exactly that tag in its ` roc ` field. It reads text and runs no
compiler, so it costs nothing in pull-request CI.
Two things follow from the compiler enforcing the field:
- Format the scripts with ` roc-stable fmt `. ` roc fmt ` on another compiler
rewrites the field to that compiler’s own release, and the check then fails.
- The field belongs only on a file that ` roc-stable ` runs. A program under
` scripts/ ` that the latest nightly builds, or that a compiler the repository
itself produces builds, cannot carry it: that compiler would refuse the file.
Keep such a program outside ` scripts/ ` where that is practical. Where it
stays, the check script lists it by path as an exception and requires that it
has no ` roc ` field at all.
` Blueprint.roc ` remains the authority. The header fields are copies that the
check holds to it, and
[Change the tooling compiler](#change-the-tooling-compiler)
changes them together.
#### The check script
One ` check ` task owns everything a pull request must pass. It uses shared
checking utilities; a product-specific driver, conventionally
` scripts/check_all.roc `, can compose additional checks when needed. Its scope
includes the static checks in this chapter,
` roc fmt --check ` and ` roc-stable fmt --check `, the script tests, ` roc test ` on the package root, every
example and integration case, a smoke run of each benchmark and fuzz target,
and a dry-run build of what a release would publish.
[What runs on every pull request](#what-runs-on-every-pull-request)
describes each.
### What is committed
| Committed | Not committed |
| --- | --- |
| Source, ` Blueprint.roc `, ` Blueprint.lock ` | ` .blueprint/ ` and anything else ` blueprint ` generates |
| Expected output and golden files under ` tests/ ` | Build output, bundles, archives |
| Generated data the package ships, with the recipe and inputs that produced it | Rendered API documentation and the built site |
| A platform’s generated host bindings | Native binaries, which are released independently and consumed through a content lock; see [No binary blobs in the repository](#no-binary-blobs-in-the-repository) |
Generated data that is committed has a script that regenerates it and a check
that fails when the committed copy is stale. Rendered documentation is published
as a release asset and a Pages deployment, never committed.
### What goes in a bundle
A bundle holds what a user of the package needs and nothing else: the package’s
Roc modules and any data files they import. Fuzz targets, benchmarks and test
modules stay outside the package root, so that ` roc bundle ` on the root produces
the release artifact with no exclusion list.
## Examples and the examples archive
Examples are realistic demonstrations of the package for the people who will
use it. They are where most new visitors land, and for many they are the first
Roc code written against the package that they read. This chapter covers what
an example is for, and how the same example is kept correct in two places: the
repository, and each release.
### Risks and controls for example identity
Examples that pass in the working tree can fail after packaging because files
are omitted, dependency URLs are rewritten incorrectly, or a warm cache hides
which bundle was fetched. Test the consumer-facing archive as an artifact.
Stage the declared files, verify that intended dependency substitutions are the
only changes, and run the archive against candidate bundles with an empty
package cache. Inspect and reject unsafe archive members before extraction.
These checks establish the tested archive’s behavior for its recorded cases;
they do not make the cases representative of every product feature.
### What an example is for
The audience of an example is a user of the package who is probably not
familiar with it. It is not a contributor.
An example is not a test, a benchmark or a fixture. Those have their own
directories and their own standards. Code that exists to cover an edge case, to
measure something, or to feed another tool does not belong under ` examples/ `,
however convenient it is to put it there.
Manage examples with the care their position deserves:
- **Realistic.** Each one solves a problem a user would recognise, with input and
output worth looking at.
- **Focused.** Each one demonstrates one thing. Several short examples serve a
reader better than one that shows everything.
- **Documented.** Each one carries ample comments that walk the reader through
the code: what it is about to do, why it is done that way, and what to change
to adapt it. Write them for someone reading top to bottom with no other
context.
- **High quality.** An example is copied into real projects. It should be code
the maintainers would be content to see copied.
- **Few.** Every example is a promise to keep it working. Add one when it teaches
something the others do not.
### Why examples live in two places
The repository and its releases are rarely in step. Pull requests land
continuously; releases happen now and then. At most commits the package in the
repository is ahead of the last release.
An example in the repository must work with the package at that same commit, so
it may use an API that no release has yet. A user reading it beside the latest
release could copy code that does not compile for them. An example published
with a release has the opposite property: it works with that release and will
never change.
One set of files cannot be both. The repository therefore keeps examples that
follow the working tree, and each release publishes a frozen copy of them that
names that release.
### Repository examples test the working tree
Every example names the package by relative path, as described in
[Roc applications in the repository](#roc-applications-in-the-repository).
A platform’s examples name it the same way, ` pf: platform "../../platform/main.roc" `,
and run against the host outputs the repository built under ` platform/ `.
A pull request that changes the package and breaks an example fails on that pull
request.
Committed examples are never rewritten after a release. No release follow-up
pull request exists, and the default branch never names its own release URL.
### Record expected output
An example that only compiles proves little. Record what each example prints,
and compare it on every run. Examples are run by the same driver as the
integration tests; see
[Integration tests](#integration-tests).
- Keep expected output in ` tests/spec.json `, in the same suite format the
examples archive and the nightly runner use, with the release ` working-tree `
and no dependencies: the arguments each example runs with, its standard
input, and the exact standard output, standard error and exit code expected.
- ` release-package test-examples ` runs every example and fails on a difference;
the shared release workflow, or a platform’s own release workflow, runs it on
every pull request. It needs only the compiler, so it runs on Windows too.
- ` release-package test-examples --update ` rewrites the expected output, so
that an intended change is a reviewed diff.
- An example that cannot run in CI, such as one needing a display or a console
runtime, is still checked and built, and the specification says so
explicitly. It is never silently skipped.
### The examples archive
Each release attaches an examples archive named
` -examples-.zip `, where ` ` is the repository, or the
package when the repository publishes several. It contains:
- complete copies of the examples, including sibling modules and data files, in
which the package dependency is that release’s bundle URL;
- the expected-output specification those examples were tested with; and
- a short README naming the release, the bundle URL and how to run an example.
An example is a file ` examples/.roc `, which depends on the package as
` ../package/main.roc `, or a directory ` examples// ` whose ` main.roc `
depends on it as ` ../../package/main.roc ` and whose other files travel with it.
A ` .roc ` file under ` examples/ ` without an ` app ` header is a module the
examples share. An example names at most one platform; one that names none runs
on the compiler’s default platform. Each case runs in its example’s own
directory, as a reader would run it, so an example reads a sibling data file by
relative path; the working tree’s examples run in a scratch copy, so an example
that writes files beside itself leaves the tree as it was. Every file Git
tracks under ` examples/ ` is archived, so a build output that is not committed
is never published.
Publish the versioned ` spec.json ` and ZIP layout in
[Publish the suite contract](#publish-the-suite-contract)
so that the same immutable archive is executable by the central nightly runner.
Nothing else in an example changes. The archived examples differ from the
repository’s at the tested commit only in the package dependency, and the check
script verifies that.
**The one line that differs between the repository and the archive**
``` diff
app [main!] {
cli: platform "https://github.com/roc-lang/basic-cli/releases/download//.tar.zst",
- ansi: "../package/main.roc",
+ ansi: "https://github.com//roc-ansi/releases/download//.tar.zst",
}
```
A user who wants a starter downloads the archive for the release they are
installing. The archive is also the fixed input to
[nightly compatibility](#nightly-compatibility), which is why
it carries its own expected output: the default branch’s specification describes
the default branch, not the release.
Examples of expensive work, such as fuzzing or benchmarking, expose a small,
bounded mode suitable for routine checks. Archive cases use that mode to verify
the published API and executable behavior; they do not launch sustained searches
or produce performance comparisons. Keep campaigns in
[their own scheduled workflow](#fuzzing), outside both
PR dry runs and the shared published-release compatibility verdict.
### Build and test the exact archive
The archive that is tested is the archive that is published. Nothing is rebuilt
between the test and the upload.
1. Build the bundle from the tested commit with ` roc bundle `. Its filename is its
content hash and the release tag is the version being released, so the final
URL, ` .../releases/download//.tar.zst `, is known before anything
is published. A release with several bundles puts a package name before the
hash; see
[Several bundles in one release](#several-bundles-in-one-release).
2. Build the archive with that real release URL in every example.
3. Serve the bundle from a loopback address.
4. Run the archived examples with ` --replace-dep `,
which ` roc check `, ` roc test ` and ` roc build ` accept. The files are unchanged;
only the download location differs for the duration of the test.
5. Compare the output with the archived specification.
6. Publish the same bundle and the same archive.
Run the test with an empty Roc package cache, so that the bundle is fetched and
verified and not found from an earlier run.
### Dry runs
A pull request runs the release steps with a placeholder version and publishes
nothing. Roc rejects ` 0.0.0 ` in a package URL, so the placeholder is
` 0.0.1-dryrun `.
### An archive is never reformatted
An archive is immutable, so it is checked, tested, built and run, and never
passed to ` roc fmt `. A later formatter change would fail the nightly check for a
reason that no patch release can fix.
## Test in pull-request CI
Code that only compiles proves little. That is true of examples, and equally of
tests, benchmarks, fuzz targets and fixtures. Anything in the repository that is
not run goes stale without anyone noticing, and is found broken on the day it is
needed.
The rule is that every Roc program in the repository runs in pull-request CI,
even if only briefly. This chapter says what runs, how much of it, and how the
tests are organised.
Start with ` expect ` tests for concrete behavior and integration cases for the
program boundary. Use roc-fuzz alongside them to check rules over generated
inputs, preserve named regressions and measure memory behavior. Pure Roc code
needs both value correctness and allocation properties: returning the right
value does not show whether an update reused storage or copied it. Keep these
checks fast on pull requests, and explore new inputs in separate campaigns.
### Risks and controls for test evidence
Compilation alone can leave examples or error paths broken, while a long
campaign in routine CI can make feedback too slow to use. Keep fast checks
meaningful and put sustained searches on their own schedule. Build once, run
recorded success and failure cases, and exercise bounded fuzz/benchmark smoke
modes. Require expected outputs and exits, report skips explicitly, retain
command diagnostics and stop hung child trees. The driver tests these failure
states. A smoke pass establishes that the selected workload runs; it does not
establish search depth, performance or complete test coverage.
### What runs on every pull request
| Check | What it runs |
| --- | --- |
| Formatting | ` roc fmt --check ` over the repository’s Roc sources |
| Unit tests | ` roc test ` on the package or platform root |
| Examples | Every example, built once and run against its recorded cases |
| Integration tests | The test driver, over its full list of cases |
| Allocation checks | Allocation properties and recorded examples, checked against reviewed ceilings or accepted exact baselines |
| Benchmarks | A smoke run: each benchmark builds and completes a small, bounded iteration |
| Fuzz targets | A smoke run: each target builds, checks its declared examples and replays a small committed corpus |
| Scripts | The ` expect ` tests of the modules under ` scripts/src/ ` |
The ` check ` task runs all of these, so a contributor runs the same set with
` blueprint run check ` before opening a pull request.
### Heavy runs stay out of pull-request CI
A long fuzzing campaign or a full benchmark run never runs on a pull request,
including a release dry run. They run in separate scheduled or manually
dispatched jobs, as described in
[Fuzzing](#fuzzing) and
[Benchmarks](#benchmarks).
What pull-request CI owes them is a smoke run. A benchmark that has not been
built since the API it measures changed will not run on the night it is
scheduled. Give each benchmark and fuzz target an explicit smoke mode, with a
small input or iteration count and at most one second of workload execution per
target. This budget excludes provisioning and compilation; it is not a
one-second limit for the whole CI job. Keep the committed smoke corpus small
enough to replay within that budget. Build once and reuse the executable across
cases.
The smoke mode exercises startup and the real operation or property, then exits
normally. A benchmark smoke run checks execution, not performance: it produces
no baseline comparison or performance regression verdict. A fuzz smoke run
replays known inputs; where the engine itself needs exercising, add a tiny,
bounded run without turning the check into a campaign. A passing smoke run
makes no claim about sustained search coverage.
Workload bounds and safety deadlines serve different purposes. Set a small
iteration/input budget so normal execution completes within the smoke budget,
and give the driver a separate hard deadline to stop hangs and child processes.
A crash, wrong result or exceeded deadline fails the smoke check; stopping a
process at the deadline does not establish success. Preserve compiler and
workload diagnostics even on successful commands, with warnings nonfatal by
default. Print explicit skips with reasons; skipped work is not passing
compatibility evidence.
Keep the long-running task out of ` blueprint run check ` and every PR release
dry-run path. Moving it into another job is insufficient if that job still
runs on every pull request: use schedule/manual triggers and explicit event
guards. The routine PR aggregate requires the build and smoke jobs, not a
skipped campaign job.
### Build with ` --opt=dev `
Build and run with ` --opt=dev ` unless there is a reason not to. It compiles
quickly, which is what a test run spends most of its time on, and the checks in
this chapter ask whether the program is correct, not how fast it is.
Use ` --opt=speed ` only where the optimised build is the thing under test: a
benchmark’s measured run, a long fuzzing campaign, or a release artifact that
ships optimised. A benchmark’s smoke run in pull-request CI uses ` --opt=dev `
like everything else.
### Unit tests
Run ` roc test ` once, on the root: ` package/main.roc ` for a package,
` platform/main.roc ` for a platform. Roc runs the ` expect ` tests of every module
the root imports, transitively, so there is no need to list files or to run the
command per module. A module that the root does not reach is either dead code or
missing from the package, and either is worth knowing.
A repository with several roots runs ` roc test ` once on each.
### Integration tests
An integration test runs a built program and looks at what it does: its output,
its exit code, the files it writes. Organise these around an external driver
and a list of cases, not as one program per test.
**One build, many cases**
- **Build once.** The driver builds the application a single time and runs the
resulting executable for every case. Compiling dominates the cost of a test
run, so this is the difference between a suite that takes seconds and one
that takes minutes.
- **Cases are data.** Each case names its arguments, its standard input, any
files it needs, and the expected output and exit code. Adding a case is
adding an entry, not writing a program.
- **Cover failure as well as success.** The same executable is run on the error
paths: bad input, missing files, invalid arguments. A case that expects a
particular error message and a non-zero exit code is as ordinary as one that
expects success.
- **The driver owns the environment.** Because it sits outside the program under
test, the driver decides how cases run. It can give each case its own process
and working directory, run cases in parallel, set up and tear down what a case
needs, and apply a timeout. None of that leaks into the application.
- **An update mode.** The driver accepts ` --update ` to rewrite expected results,
so an intended change in behavior is a reviewed diff.
The driver is a Roc script under ` scripts/ `, and the cases live under ` tests/ `.
The expected-output specification for examples, described in
[Record expected output](#record-expected-output), is one such
list of cases run by the same driver.
### Skip a broken case, visibly
Sometimes a case is broken for a reason that cannot be fixed today: a compiler
bug, a dependency that has not been released, a platform limitation. Deleting
the case loses it, and leaving it failing blocks every pull request. Skip it,
and make the skip impossible to forget.
- Mark the skip where the case is defined, with a ` TODO ` note and a link to a
GitHub issue that tracks the cause.
``` roc
# TODO: skipped until the compiler handles nested tag unions here.
# https://github.com/roc-lang/roc/issues/
{ name: "nested-tags", skip: Skip("compiler bug, roc-lang/roc#"), ... }
```
- A skip always carries a reason. The harness rejects a skip with no note.
- The harness prints every skipped case with its note, and counts skips in the
summary. Nobody should have to search the source to learn what is not being
tested or why.
- A skip is temporary. When the linked issue closes, the skip is removed in the
pull request that confirms the case passes again.
The same applies outside the harness: an example, benchmark or fuzz target that
is switched off says why, in a ` TODO ` with an issue link, at the place it is
switched off.
### Report progress and results
A harness is read far more often than it is written, usually by someone waiting
for it or trying to find out what failed. Its output is part of its job.
**What a run looks like**
``` text
Running 20 cases
[ 1/20] parse-empty-input ............ PASS 12ms
[ 2/20] parse-nested-lists ........... PASS 48ms
[ 3/20] render-wide-table ............ FAIL 1_204ms
[ 4/20] nested-tags .................. SKIP compiler bug, roc-lang/roc#1234
...
[20/20] rejects-missing-file ......... PASS 9ms
Failed
render-wide-table: expected exit code 0, got 1
tests/cases/render-wide-table
Skipped
nested-tags: compiler bug, roc-lang/roc#1234
Summary: 18 passed, 1 failed, 1 skipped in 10_312ms
Slowest: render-wide-table 1_204ms, build 6_870ms
```
- **Show progress as it happens.** Print each step with its position, such as
` 3/20 `, and its result as soon as it finishes. A run that prints nothing for
a minute looks the same as a run that has hung.
- **End with a summary.** Report how many cases passed, failed and were skipped,
list each failure with enough detail to act on, and list each skip with its
note. The summary is what a reader sees first in a CI log, so it stands alone.
- **Report timing.** Give the time of each case, the total, and the slowest
steps, including the build.
- **Use one unit for time.** Print every duration in milliseconds, with digit
separators and aligned to the right: ` 12ms `, ` 1_204ms `, ` 10_000ms `. With a
single unit the eye compares lengths, and a slow case stands out. A column
that mixes ` 850ms `, ` 1.2s ` and ` 2m ` hides exactly the outlier worth seeing.
- **Use color.** ` FAIL ` is red, ` PASS ` green, ` SKIP ` yellow. Color is an aid and
never the only signal: the word is always printed, and color is turned off
when output is not a terminal or when ` NO_COLOR ` is set.
- **Exit accurately.** Any failure gives a non-zero exit code. Skips alone do
not.
### Measure allocations
Memory behavior is a quality of Roc code in its own right, and it is invisible
unless it is measured. A change that keeps a reference alive a little longer
can turn an in-place update into a copy of a whole list, on every call. The
result is still correct, every test still passes, and the package is now
several times slower for its users.
Use [roc-fuzz](https://github.com/lukewilliamboswell/roc-fuzz) as a test platform
for pure package code. Its host measures allocations, reallocations,
deallocations, requested bytes and live/peak memory. Test the hot paths and
operations users call in a loop, including empty, boundary-sized, uniquely
owned and shared inputs.
#### Allocation properties and exact baselines
Use two complementary checks:
- **Properties over inputs.** Assert a memory rule inside the target, such as
"this operation allocates nothing" or "this update performs at most one
allocation or reallocation". The assertion runs for declared examples,
replayed inputs and generated inputs. Use ` Fuzz.expect_allocs_at_most! ` for
a ceiling, or use a named ` Fuzz.measure! ` region and inspect its statistics
for a more specific rule. A measurement returns
` { value, stats } `, so correctness and memory behavior can be checked together.
- **Exact observations for named examples.** Declare representative cases with
` Fuzz.example({ name, revision, make: || value }) `, or preserve a witness
with ` Fuzz.saved({ name, revision, bytes }) `. roc-fuzz checks each in a fresh
process and compares its allocation observations with an explicitly accepted
baseline. Named measurements isolate operations within the property.
These checks have different review rules. A ceiling fails only when the count
exceeds the bound; lower it when an improvement should be protected. roc-fuzz’s
baselines match exactly, including example inventory, revisions, actual
input digests, measurement regions, metric version and statistics. Reductions
also require review and explicit acceptance. A compiler improvement should
therefore update a baseline deliberately, rather than silently changing what
CI accepts. Commit baselines for each supported host and named profile; record
the compiler and build options with the evidence.
` check ` verifies examples and accepted baselines without exploration. A missing
entry is reported as ` unbaselined ` and permits normal execution; it is not
allocation regression protection. Review the declared inventory and measurements,
run ` accept ` explicitly, and commit the resulting baseline before treating
checks as evidence of a protected allocation contract. CI runs ` check ` or ` run `,
never ` accept `.
Allocation evidence depends on the measured scope and storage shape. roc-fuzz
measures the property after input generation/decoding and before adapter teardown;
named regions narrow that scope further. Keep aliases alive when testing shared
storage, and observe returned values so the intended operation remains relevant.
A logically equal value can have different capacity, sharing or slices. Preserve
original witness bytes for storage-sensitive failures: reconstructing a value
in an ` expect ` test may lose the allocation behavior that exposed the bug.
Counts are a steadier PR signal than wall-clock timings, but they depend on the
compiler, host, build and lifetime of values. Review baseline diffs alongside
correctness results, and use a deliberate failing assertion or changed baseline
to verify that the harness detects a regression. A pass covers the checked
examples and explored inputs; it does not prove an allocation bound for all
inputs. roc-fuzz also supports logical trace rules (` Before `, ` Count ` and
` Forbidden `) for ordering and event invariants. These are logical observations;
wall-clock performance belongs in the separate benchmark workflow.
### Correctness and memory properties with roc-fuzz
Use property-based tests for invariants across inputs, especially parsers,
encoders, data structures and arithmetic on sizes or indexes.
[roc-fuzz](https://github.com/lukewilliamboswell/roc-fuzz) combines correctness,
allocation and trace checks with coverage-guided exploration in a self-contained
Roc target. The target imports the package; the shipped package has no dependency
on roc-fuzz.
| Part | Role |
| --- | --- |
| Generator | Produces typed Roc values to try: numbers, strings, lists, records, or shapes built from them |
| Operation | Calls the package code under test |
| Property | Checks value correctness, memory behavior or a logical invariant |
| Examples | Named cases and saved witnesses that run independently of corpus retention |
A pure function needs no external state to recreate. Useful properties include
round-trip encoding, formatting idempotence and agreement with a simple reference
implementation. Preserve failing input bytes and the target/compiler/build
configuration, especially for storage-sensitive memory checks. Coverage guides
exploration toward new branches; saved witnesses and named examples preserve
regressions independently of the evolving corpus.
#### What a target looks like
``` roc
app [target] {
fuzz: platform "https://github.com/lukewilliamboswell/roc-fuzz/releases/download//.tar.zst",
pkg: "../../package/main.roc",
}
import fuzz.Fuzz
import pkg.Stack
# After pushing an item, popping must return that item and the original stack.
test : List(U8) -> Fuzz.Outcome
test = |initial| {
item = 0
pushed = Stack.push(initial, item)
match Stack.pop(pushed) {
Ok({ value, rest }) if value == item and rest == initial => Fuzz.keep
Ok(_) => crash "pop did not undo push"
Err(Empty) => crash "the stack was empty after push"
}
}
target = Fuzz.target_with({
examples: [
Fuzz.example({ name: "empty", revision: "1", make: || [] }),
Fuzz.example({ name: "nonempty", revision: "1", make: || [1, 2, 3] }),
],
name: "stack-push-pop",
generator: Fuzz.list(Fuzz.u8, 8),
test,
show: |initial| Str.inspect(initial),
})
```
The target depends on the working tree by relative path, like every other Roc
application in the repository, and on a roc-fuzz release by URL. It lives
under ` fuzz/ `, outside the package root, so it is never bundled. Use the
content-addressed platform URL and compatible compiler documented in the
[release notes](https://github.com/lukewilliamboswell/roc-fuzz/releases/latest).
Build once, then use the commands embedded in the executable:
``` sh
roc build --fuzz fuzz/stack/main.roc --output=stack-fuzz
./stack-fuzz check
# After reviewing the observations, explicitly accept and commit the baseline.
./stack-fuzz accept
./stack-fuzz check
# Local exploration; the default run searches for 60 seconds after checking.
./stack-fuzz run
```
` check ` runs declared examples and baseline checks without searching. ` run `
checks them before exploring, and skips exploration when those checks fail.
Use ` run --report-dir=PATH ` for campaign evidence; the report directory must be
absent or empty. Bare invocation also runs the default workflow, so PR tasks
must select ` check ` and bounded replay explicitly rather than accidentally
starting a 60-second search. The default baseline is
` _baseline.json ` in the working directory; use
` --baseline=PATH ` and ` --profile=NAME ` when needed.
For an allocation-aware target, use an effectful property and
` Fuzz.target_with! `. For example, popping a stack can check the returned
length while asserting that it performs no allocations or reallocations:
``` roc
test! : List(U8) => Fuzz.Outcome
test! = |initial| {
popped = Fuzz.expect_allocs_at_most!(0, || Stack.pop(initial))
match popped {
Ok({ value: _, rest }) if rest.len() + 1 == initial.len() => Fuzz.keep
Err(Empty) if initial.is_empty() => Fuzz.keep
_ => crash "pop returned the wrong length"
}
}
```
Pass ` test! ` in place of ` test ` to ` Fuzz.target_with! `, keeping the generator,
examples and display function. Replace the measured operation and property
with the package’s actual memory contract. The
[roc-fuzz quality guide](https://lukewilliamboswell.github.io/roc-fuzz/quality.html)
explains named measurements, baselines, trace rules and report fields.
#### How targets are managed
Review properties when APIs or implementations change. Strengthen invariants
and generators as the input domain becomes clearer; add named examples for
newly discovered failures.
- Write one operation and one property per target, so that a failure says what
went wrong.
- Add a target when a module has a rule that should hold for all inputs. Every
parser and every encode and decode pair should have one.
- When roc-fuzz finds a bug, replay and minimise it, fix it, and use ` retain `
to preserve the witness as a named ` Fuzz.saved ` example. Add an ordinary
` expect ` test for the value-level regression when useful. Keep original
storage-sensitive witnesses even when a smaller logical example is available.
- Commit a small corpus for each target. Pull-request CI builds every target
and checks its named examples and allocation assertions, accepted baselines
and bounded corpus replay as its smoke run.
- Run the real search on a schedule, as described in
[Fuzzing](#fuzzing).
Validate preconditions for decoded examples as well as generated values;
generator bounds express search bias, not input validation. Report attempted,
accepted and rejected inputs. Exploration with zero accepted inputs is
inconclusive and unsuccessful. Distinguish the outcome from allocation validity,
trace-rule results and capture completeness in the evidence.
A weak property finds nothing, and a passing run is not a proof. The
[roc-fuzz manual](https://lukewilliamboswell.github.io/roc-fuzz/) covers writing
generators and properties, running campaigns, and reading a saved failure.
### Formatting
Run ` roc fmt --check ` in pull-request CI, over every Roc source the package
compiler builds, and ` roc-stable fmt --check ` over the scripts, whose ` roc `
field another compiler’s formatter would rewrite. It fails the pull request that introduces unformatted code, which is
the only moment at which fixing it is cheap.
Do not run it in ` nightly.yml `. A new nightly can change the formatter, and a
formatting difference says nothing about whether a release still works. The
check would raise an alarm that no patch release answers. When a new formatter
does change the output, the next pull request sees it, and reformatting is a
small pull request of its own.
## Check repository source policy
` roc-policy ` is a precompiled, read-only source-policy utility. It checks
repository conventions that compiler diagnostics, formatting and behavioral
tests do not establish. The tools release builds binaries for x86\_64 Linux,
ARM64 Linux and Apple Silicon macOS. The Linux binaries are static musl
executables. Building and testing the implementation happens upstream; source
checking downstream requires no Roc compiler, Python, Git or Nix.
### Risks and controls for source-policy evidence
A rule can inspect the wrong files, an override can hide a violation, or an
incomplete scan can appear successful. Make rule scope and analysis completeness
observable. Explicit roots, validated profiles, explained effective configuration
and stable findings let maintainers inspect what the checker selected. Malformed
configuration and incomplete analysis fail separately from policy findings;
fixtures verify exclusions and conflicting settings. A clean source-policy report
establishes only the rules actually checked. It is not behavioral test coverage
or proof that the product has no security defects.
### Declare policy in Blueprint
The default authority is ` repository_policy ` in ` Blueprint.roc `. Schema 2 composes
shared defaults with repository rules and explicit overrides. The surrounding
file remains normal Roc configuration; add the binding beside its existing
` config `, without replacing its Blueprint platform declaration:
``` roc
tooling = { compiler: "nightly--" }
repository_policy = {
schema_version: 2,
roots: ["package", "examples", "scripts", "docs"],
exclude: ["**/generated/**"],
test_roots: ["package"],
profiles: [{ name: "base", builtin: "roc-package", version: 1 }],
rules: [
{ id: "project/compiler", kind: CompilerPin, include: ["scripts/**"], value: tooling.compiler },
{ id: "project/no_internal_imports", kind: ForbiddenImport, include: ["package/core/**"], name: "support/Internal" },
{ id: "prose/worth_noting", kind: Phrase, include: ["docs/**"], channels: [Prose], phrase: "it is worth noting" },
],
overrides: [
{
rule: "base/spelling",
reason: "Published ecosystem name",
changes: { accepted_words: { add: ["rocpkgs"] } },
},
{
rule: "base/public_docs",
include: ["package/fixtures/**"],
reason: "Imported fixtures preserve upstream source",
changes: { level: Off },
},
{
rule: "prose/worth_noting",
include: ["docs/quotations/**"],
require_match: False,
reason: "Quoted source retains its original wording",
changes: { level: Warning },
},
],
}
```
An environment may reference the same ` tooling.compiler ` when declaring its
compiler. The utility resolves local constants, forward references, record puns
and field access without evaluating Blueprint. It does not prove that arbitrary
Blueprint expressions actually use those facts. Configuration composition is
implemented by the policy resolver, separate from generic constant extraction.
Supported data are literal records, lists, strings, booleans, plain decimal
integers and bare tags, plus references, field access and parentheses. Strings
support Roc escapes including Unicode scalars. Calls, arithmetic, functions,
record updates, interpolation and imported values in the selected dependency
graph are rejected. Unrelated functions and imports may remain in the file.
The complete file must parse without diagnostics. Duplicate definitions/fields,
cycles, unknown fields, irrelevant options and resource-limit violations fail.
The generic constant-data module has no profile loading or policy-specific
behavior. It limits each source to 1 MiB, declarations/work to 10,000, depth to
64 and expanded-value accounting to 1 MiB. Policy composition admits at most
32 profiles, 10,000 final rule instances and 10,000 overrides.
### Shared profiles and rule identities
A rule’s ` kind ` selects an implementation. Its required ` id ` identifies one
configured policy instance. The diagnostic code identifies the finding category.
For example, two ` Phrase ` rules can share diagnostic code ` text/phrase ` while
having independently overridable IDs. IDs use slash-separated components of
letters, digits, dots, underscores and hyphens; empty, ` . ` and ` .. ` components
are invalid. Duplicate IDs fail instead of replacing earlier declarations.
Profiles load in declared order, followed by local rule declarations. Their rule
IDs are namespaced by the profile’s ` name `: rule ` spelling ` in profile ` base `
becomes ` base/spelling `. A local rule cannot accidentally replace that ID. Change
its settings through an explicit override instead.
` roc-policy list-profiles ` lists profiles embedded in the binary. The immutable
` roc-package ` version 1 supplies ` public_docs ` for ` package/**/*.roc ` and
` spelling ` for supported Roc/prose extensions. Both rules are intentionally
optional across repository layouts. Embedded profiles need no sidecar files;
the reviewed binary selects their bytes, and explanations include their SHA-256.
Unknown names and versions fail.
Alternatively, a profile descriptor may contain ` name `, repository-relative
` path `, ` definition ` and the reviewed lowercase 64-character ` sha256 `. The file
must define a bounded record such as:
``` roc
profile = {
schema_version: 2,
rules: [
{ id: "spelling", kind: Spelling, channels: [Comments, Prose] },
],
}
```
The loader verifies the exact file bytes before interpreting the chosen binding.
A missing file, symlink, unsafe path or mismatched digest fails; no download or
external hashing tool is invoked. Profiles contain rules only, without nested
profile loading, discovery roots or overrides. Profile paths are anchored at the
explicit repository root. This permits independent policy-data releases while
keeping profile selection, pins and repository decisions in Blueprint.
Shared profiles are reusable defaults that repository authors may override.
Mandatory organizational requirements require an independently trusted CI
policy/invocation. Selecting a profile in a repository-controlled Blueprint is
not evidence that such external requirements were enforced.
### Ordered overrides
Overrides target one exact rule ID. Every override requires a nonempty ` reason `;
unknown IDs and unknown or irrelevant changes fail. Without ` include `/` exclude `,
an override applies repository-wide. With file selectors, it applies to matching
paths. All matching overrides apply in written order, including global overrides:
later scalar values win, without hidden path-specificity precedence.
` level ` is ` Off `, ` Warning ` or ` Error `, defaulting to ` Error `. There is no separate
` enabled ` flag. Off suppresses the selected rule. Warnings are reported without
failing by default; set repository ` max_warnings: 0 ` to reject any warning, or a
nonnegative number to allow that many. Configuration errors, I/O failures and
incomplete analysis remain exit 2 regardless of rule level.
Collection changes use explicit operations:
``` roc
changes: {
accepted_words: { add: ["rocpkgs"], remove: ["oldterm"] },
channels: { replace: [Comments, Prose] },
corrections: { add: [{ word: "mispeling", replacement: "misspelling" }] },
}
```
` replace ` discards inherited values and cannot be combined with ` add ` or ` remove `.
` add ` extends sets without duplicates; ` remove ` subtracts matching entries.
Vocabulary and correction keys fold ASCII case; paths remain case-sensitive.
Adding a correction replaces the inherited correction with the same normalized
key. Removing a custom correction exposes the bundled correction, if any; use
` accepted_words ` to accept the word instead. Contradictory operations and duplicate
keys in one layer fail. There is no implicit list concatenation or deep merge.
Overrides may change level and the options supported by their rule kind.
Repository-wide overrides may also change rule ` include `/` exclude ` collections
and ` require_match `. File-scoped overrides cannot change rule IDs, kinds, selectors or the
aggregate rule ` require_match ` assertion. Discovery roots and profile pins are not patchable through rules.
Rules require an applicable source file by default. An intentionally optional
rule may declare ` require_match: False `, useful in profiles shared across
different layouts. Overrides likewise require a selected target/file by default;
set their ` require_match: False ` for intentional future or optional exceptions.
This catches renamed paths and stale exceptions without making every shared rule
mandatory. It checks applicability, not whether an exception suppressed a finding.
### Commands and paths
``` sh
roc-policy check --root .
roc-policy check --root . --format json
roc-policy list-profiles
roc-policy explain-config --root .
roc-policy explain-config --root . --path package/Thing.roc
roc-policy manifest --root . > expected-tests.json
roc-policy run-tests --root . --compiler /absolute/path/to/roc
```
` --config ` selects a file relative to the root and ` --definition ` selects a
binding; defaults are ` Blueprint.roc ` and ` repository_policy `. All configured
commands use this same authority. ` manifest ` and ` explain-config ` emit JSON.
Explanation includes profile digests, rule IDs, origins, effective options,
selector decisions and ordered override traces. Without ` --path ` it shows global
settings; a path shows effective settings and why rules are selected, excluded
or disabled. It need not open the source file to explain its configuration.
` roots ` and ` test_roots ` are exact repository-relative files/directories.
Schema 2 rule and override selectors use root-relative slash-separated globs:
` * ` matches within one component, ` ? ` matches one Unicode scalar, and a complete
` ** ` component matches zero or more path components. Thus ` **/*.roc ` includes
root files and ` package/** ` includes the directory and descendants. Literal
patterns match exactly. Dotfiles match normally. Unsupported bracket/brace,
negation and escape syntax, absolute paths, controls and dot segments fail.
Patterns and matched paths are bounded to 4096 bytes and 128 components.
Global ` exclude ` prunes discovery before opening sources or descending into
excluded directories. Rule exclusions affect that rule alone. File overrides do
not resurrect globally excluded files. There is no implicit Git ignore behavior,
parent-configuration cascade or dependence on the current directory once the
explicit root is fixed. Global exclusions also apply to test discovery, but do
not bypass explicit configuration/profile reads or their integrity checks.
Discovery is deterministic and overlapping roots are deduplicated. Sources are
` .roc `, ` .md `, ` .markdown `, ` .adoc `, ` .txt ` and ` .rst `; accounting uses only ` .roc `.
Selected sources reject symlinks and special files. Input files are limited to
8 MiB and inventory traversal to 100,000 paths.
Exit 0 means completed checks within the warning allowance, 1 means error-level
findings or too many warnings, and 2 means invalid input or incomplete analysis.
Diagnostic JSON uses schema 2 and includes ` rule_id `, ` level `, ` origin ` and ` trace `
alongside the category ` rule `. Text locations use one-based Unicode scalar
columns; JSON retains half-open UTF-8 byte spans.
### Rules
` CompilerPin `, ` NoCompilerPin ` and ` Dependency ` inspect app/package/platform
headers, not lookalike fields in bodies, comments or strings. A required
compiler/dependency rule aimed at an exact entrypoint reports a missing header.
Directory-wide required rules apply only to header-bearing entrypoints, allowing
ordinary helper modules. They fail configuration if no entrypoint is applicable.
A dependency value is an exact decoded string; alternate relative spellings
are distinct values.
` ForbiddenImport ` matches the written path or its dot/slash descendants, ignoring
trivia and preserving aliases' original paths. It does not resolve bindings,
reexports, indirect calls or a complete project dependency graph.
` PublicDocs ` identifies explicit module exposures and filename-based nominal
modules, including nested public associated definitions. It requires nonempty
attached ` ## ` documentation, supporting documentation before an annotation or
its definition. Private helpers and local declarations are excluded. Package
uppercase exposure inventories refer to other module files; select those source
files for checking. Unresolved imported/qualified/wildcard public surfaces return
` documentation/incomplete `, never an assertion that they are documented.
` Spelling ` embeds the pinned typos v1.44.0 correction map (93,861 entries)
through a Roc file import. It requires no dictionary file, download or Rust
toolchain at runtime. This is correction lookup rather than unknown-word
detection; it does not reproduce the typos CLI heuristics or dialect rules.
The exact upstream CSV, MIT notice, provenance and offline verification script
are maintained in ` tools/roc-policy/vendor/typos `. ` roc-policy --licenses `
prints the embedded notice, which is also included among release assets.
` channels ` selects ` Comments `,
` Identifiers `, ` Strings ` and ` Prose `; defaults are comments, identifiers and prose.
Identifiers split snake case, camel case, acronym transitions and digits.
` accepted_words ` exempts word components and ` accepted_identifiers ` exempts whole
Roc identifiers, without accepting their components elsewhere. Add corrections
with ` corrections: [{ word: "mispeling", replacement: "misspelling" }] `; entries
replace the same bundled key. Keys must be single words. Case folding is ASCII;
Unicode bytes and source offsets are preserved. String channels inspect raw
string parts rather than evaluating escape sequences.
` Phrase ` requires ` phrase `, optionally ` replacement `, and uses the same channels
as spelling. Exceptions target the particular phrase rule ID. Whole-word matching folds ASCII case and normalizes whitespace
across consecutive comment/documentation lines; internal and surrounding
punctuation must match. Patterns must contain a matchable word. There are no
implicit banned phrases. A policy enforces chosen language conventions, not
inference about authorship or prose quality.
### Test accounting
` manifest ` discovers static file-level and associated-block ` expect ` declarations
under ` test_roots `, defaulting to ` roots `. It emits schema 1 JSON with ` files `,
each containing ` path `, exact ` source `, and ` tests ` with byte ` start ` and ` end `.
It is a source inventory, with no execution statuses. Inline/local expectations
are valid assertions but are not separate required static tests. A static test
sharing its starting line with another expectation is rejected because the
compiler’s line output cannot distinguish them. Empty selected static inventories
fail explicitly.
` run-tests ` is an optional discovery-driven runner requiring an explicit compiler
path. It invokes ` roc test --verbose ` for each inventoried
file, leaving normal compiler caching enabled. Optional ` test_main ` names a
relative application file used to resolve package aliases with ` --main=... `.
For every static test, exactly one ` PASS: : ` record
must be reported. Tests in imported local modules and executed inline assertions
may add records; they do not inflate the selected file’s static count. The total
number of PASS records must also agree with the compiler’s success summary.
Missing or duplicated required locations fail, even when the overall count is
unchanged. A successful-looking output fragment cannot override a failed process.
Normal cached results are accepted. Cache correctness is the compiler’s
responsibility; this utility does not routinely pass ` --no-cache `. Every child
must exit successfully within 60 seconds and uses bounded output capture.
Inventoried sources are checked again after all invocations; changed sources,
ambiguous locations, malformed output and incomplete parses fail. No shell
command is constructed.
This checks compiler-reported source lines, not native test IDs, exact span
receipts, code coverage or authenticated execution provenance. It does not prove
that a different CI entrypoint selects these tests. It discovers and invokes
each selected test-bearing file itself, so large import graphs can cause repeated
compiler invocations; normal caching reduces that cost. The adapter is tested
against the compiler its tools release records and depends on that compiler’s
human-readable verbose output. An incompatible output format fails explicitly.
The runner can execute project code and invoke the compiler’s normal dependency
behavior. Use it in unprivileged jobs. ` check ` and ` manifest ` require no compiler,
execute no project code, and invoke no external helpers.
### Distribution boundary
Install a reviewed tools release and verify its content digest before invoking
it. Blueprint can manage the binary through ` Programs ` and ` Fetch `, with its
bytes pinned by ` Blueprint.lock `; see
[utility provisioning](#adopt-a-pure-package-without-nix).
Content hashes establish expected bytes, not build provenance.
## The three workflows
Every package and platform repository has the same three workflows. Each answers
one question, and none of them changes the repository.
| Workflow | Trigger | What it proves | Token access |
| --- | --- | --- | --- |
| ` ci.yml ` | Pull request; push to the default branch | Scripts pass on ` roc-stable `. The working tree and its examples pass on the latest nightly. A platform’s committed host bindings match that nightly. A Roc package has no ` ci.yml `: its release dry run proves the same things. | ` contents: read ` |
| ` release.yml ` | Manual dispatch; pull request as a dry run | The exact bundle, examples archive and documentation proposed for publication work. A Roc package’s ` release.yml ` is a thin caller of the shared ` release-package.yml `. | ` contents: read `. The publish job alone has ` contents: write `, behind a protected environment. |
| ` nightly.yml ` | Daily schedule; manual dispatch | The latest published release and its examples archive work with the latest nightly. | Test jobs ` contents: read `; the report job ` issues: write ` |
Rules that hold in all three:
- Every ` uses: ` reference is a full commit SHA.
- Top-level ` permissions ` is ` {} `, and each job states what it needs.
- Checkout sets ` persist-credentials: false `.
- A job that runs repository code has no write access. A job with write access
runs no repository code.
- The nightly tag is resolved once, in the first job, and passed to every other
job as the ` nightly-tag ` input of ` roc-lang/setup-roc `. Matrix jobs never
each resolve "latest" for themselves, or one run could test several
compilers.
- Each job calls a ` blueprint run ` and holds no logic of its own.
Workflows beyond these three, such as a documentation site or scheduled fuzzing,
are described in
[Optional workflows](#optional-workflows).
### Risks and controls for release evidence
A workflow can publish a different source revision or rebuild assets after
testing, and a retry can combine files from unrelated attempts. Preserve source
and artifact identity across the release. Build and test the selected commit,
pass exact files through the run, and let a separately authorized publisher
verify the inventory and downloaded asset hashes. Interrupted-publication rules
refuse ambiguous replacement. Exercise dry runs and publisher failure transitions.
These controls bind the delivered bytes to the selected evidence; they do not
prove an independently reproducible build or the trustworthiness of a compiler.
### Continuous integration
` ci.yml ` tests the working tree.
1. Set up Blueprint and install the latest nightly, as in
[Use the environment in CI](#use-the-environment-in-ci).
2. Run ` blueprint run check `.
[What runs on every pull request](#what-runs-on-every-pull-request)
lists what the ` check ` task covers.
Keep routine CI and release dry runs fast: benchmark and fuzz programs build
and run their explicit smoke mode, with at most one second of workload execution
per target. Compilation has its own budget. Sustained searches and measured
benchmarks run only in schedule/manual jobs with explicit event guards; see
[Heavy runs stay out of pull-request CI](#heavy-runs-stay-out-of-pull-request-ci).
A [Roc package](#a-pure-roc-package) has no ` ci.yml `. The
[shared release workflow](#the-shared-release-workflow-for-a-pure-package)
runs ` blueprint run check ` and the examples on every pull request as part of
its dry run, and its two job names are the required status checks.
A repository whose code can behave differently by target runs it as a matrix,
one job per runner, and ends with a single aggregate job that requires every
matrix job to have succeeded. The aggregate job’s name is the required status
check, so the ruleset does not need editing when the matrix changes.
A platform adds the host-bindings check described in
[Host bindings](#host-bindings). It runs here and nowhere else.
A pull request can fail because a new nightly arrived, with no fault in the
change itself. Compare with the latest ` nightly.yml ` run and with the default
branch before treating a failure as a regression in the pull request.
### Release
` release.yml ` builds what a release would contain, tests those exact bytes, and
publishes them once. A maintainer dispatches it from the default branch with a
version. A pull request runs the same build and tests as a dry run and publishes
nothing.
#### The shared release workflow for a Roc package
A Roc package keeps no release logic of its own. Its ` release.yml ` calls
roc-automation’s ` .github/workflows/release-package.yml ` at a full commit SHA:
``` yaml
name: Release
# A pull request builds and tests the release under a version that is never
# published. Dispatch from the default branch with a version to publish it.
on:
pull_request:
workflow_dispatch:
inputs:
version:
description: Version to publish, such as 0.1.0 or 0.1.0-rc1; empty runs the dry run
type: string
default: ''
nightly-tag:
description: Exact Roc nightly for a diagnostic run; empty resolves the latest
type: string
default: ''
permissions: {}
jobs:
Release:
permissions:
contents: write
id-token: write
attestations: write
uses: lukewilliamboswell/roc-automation/.github/workflows/release-package.yml@8f73710f90f942f3097821b189bbf341d305cd55
with:
version: ${{ inputs.version || '' }}
nightly-tag: ${{ inputs.nightly-tag || '' }}
check-task: check-package
```
The optional numeric ` build-timeout-minutes ` input defaults to ` 30 ` and accepts
integers from ` 1 ` to ` 360 `. Set it high enough for the full configured checks and
release assembly, for example ` 75 ` for a large parser conformance suite. The
request guard rejects invalid values before candidate execution. A timeout or
failed build prevents testing and publication; raising the limit changes no
checks or token permissions.
The shared workflow runs three jobs. **Build and test the source** resolves the
latest nightly once, runs the package’s ` check ` task, checks version
sufficiency against the latest release, runs the working-tree examples, and
builds the bundle, the examples archive, the API documentation archive and the
notes. **Test the release candidate** checks out no package source: it downloads
those exact files and runs the archived examples against the bundle over a
loopback address. **Publish the release** runs only for a dispatched version on
the default branch, in the ` release ` environment; it attests the tested files
and hands them to [` actions/publish-release `](#publish-release).
The bundles, the archives, the notes and both example runs are one utility,
[` release-package `](#release-package), a static executable from
a roc-automation tools release that the repository’s Blueprint fetches by
digest and names as the ` release-package ` task. The workflow checks out its
own repository with ` job.workflow_sha ` for the shared actions it uses, so the
caller’s SHA fixes the workflow and the lock fixes the utility; the package
compiler is the latest nightly, installed by the workflow. A maintainer runs
the same task locally with that compiler on ` PATH `:
``` sh
blueprint run release-test -- test-examples # the working tree's examples against tests/spec.json
blueprint run release-test -- test-examples --update # record what they print now, as a reviewable diff
blueprint run release-package -- build 0.0.1-dryrun # the bundles, archives and notes, into dist/
blueprint run release-test -- test-archive # the archive in dist/ against the bundles in dist/
```
The inputs are the version, an exact nightly for a diagnostic run, the name of
the Blueprint task that checks the package and one program to run before it for
a prerequisite the runner lacks, an SBOM author with an optional composition
task and supplier-imports record when the release publishes sidecars, and a
` manual ` switch with an optional task that writes generated manual sources
before the PDF is rendered. With the
calling job named ` Release `, the required status checks are
` Release / Build and test the source ` and ` Release / Test the release candidate `.
The caller declares no concurrency group: the shared workflow holds one release
at a time per repository.
A platform keeps a ` release.yml ` of its own, described in
[A platform’s release workflow](#a-platform-release-workflow).
##### Adopt the shared workflow in an existing repository
The move from a copied release workflow is the same in every package so far.
1. A ` check ` task in ` Blueprint.roc ` that runs the package’s own checks, and the
` release ` and ` release-test ` environments and tasks from
[the utility’s section](#release-package), followed by
` blueprint update ` for the lock. A check that takes longer than a few
minutes, such as fuzzing or conformance suites, stays in the repository’s own
` ci.yml `; pass ` check-task: '' ` then, so that the release dry run stays fast.
2. Every example depends on the package by relative path and names no ` roc `
compiler; the release notes record the compiler instead. A ` .roc ` file under
` examples/ ` without an ` app ` header is a shared module. An example that
prints the clock or anything else that varies takes an argument or input
that makes a case deterministic: a skipped case never passes.
3. ` release-package test-examples --update ` writes ` tests/spec.json ` with one
case per application and records what it prints; review that diff, then add
cases.
4. Notes in ` docs/releases/.adoc ` open with ` = ` and
use ` {package-url} `, plus ` {-url} ` for each further bundle.
5. The thin ` release.yml `, then delete the scripts and workflow jobs that
bundled, archived, tested examples, rendered notes or published, and ` ci.yml `
when ` check-task ` runs the checks.
6. SBOM sidecars need ` sbom/composition.json ` and an author; a
manual PDF needs the roc-docs renderer behind a ` docs-path ` task. A
repository without either leaves those inputs unset and keeps its own
documentation workflow.
7. Update the ruleset’s required checks to the two shared job names, and after
the first green dry run, a maintainer dispatches the next release.
#### A platform’s release workflow
A platform’s host is built with its own toolchain, and its bundle holds that
host for every target, so its ` release.yml ` builds them itself and hands the
rest to the same utility, as
[A platform release](#release-package-platform) describes.
It has the shape of [one builder and a matrix of runners](#cross-compile-once-run-on-every-target):
1. **Build**, on one Linux runner: build the host for every target, then
` release-test -- test-examples `, the repository’s bundle command with
` --output-dir build/platform `, and
` release-package -- build "$VERSION" --platform-bundle build/platform `.
Upload ` dist/ `.
2. **Test the release candidate**, one job per supported system, Windows
included: install Blueprint with
[` actions/setup-blueprint `](#setup-blueprint) and the nightly
the build resolved, download ` dist/ `, and run
` release-test -- test-archive `. Nothing is rebuilt.
3. **Publish**, as for a package: from the default branch, in the ` release `
environment, attest the tested files and hand them to
[` actions/publish-release `](#publish-release), with no
checkout of the platform’s source.
The repository keeps only what is its own: the host build, the target layout,
the linker inputs and the bundle command with its inventory. Its examples,
` tests/spec.json ` and notes follow the package rules, with ` {platform-url} `
for the platform’s URL.
#### What a release contains
One immutable GitHub release per version holds:
| Asset | Purpose |
| --- | --- |
| The bundle, under the filename ` roc bundle ` emits | What users depend on. The filename ends in its content hash, which is never changed. [Several bundles in one release](#several-bundles-in-one-release) says when the filename also carries a name. |
| ` -examples-.zip ` | Examples pinned to this release’s bundle, with their expected output. |
| ` -docs-.zip ` | The generated API reference and site for this version, so the documentation of any release can be restored. |
| ` -.pdf ` | The manual, where the repository has one. |
| Release notes | An authored account of what changed, with the real bundle URLs and the compiler versions used. |
| ` .spdx.json ` or ` .cdx.json ` | Machine-processable component inventory bound to each deliverable’s digest, with target-specific relationships and declared coverage. See [the SBOM contract](#publish-and-consume-software-bills-of-materials). |
| Signature/attestation evidence | Authenticated SBOM statements and build provenance, with expected signer, source and subject identities. These establish different evidence. |
Managing releases carefully is a large part of what makes a package dependable.
A user meets a release through three things: its version number, its notes, and
the URL they paste into a header. Each is handled deliberately.
#### Choose the version with ` roc bump `
A version number is a promise about compatibility, and ` roc bump ` checks the
promise against the code. It compares the public API of the package with a
previous release and reports the semantic-version bump the difference requires:
patch, minor or major.
``` sh
# what bump does this change need, and what is the next version?
roc bump --old package/main.roc
# in release.yml: fail unless the chosen version bumps far enough
roc bump --old --expect 0.13.0 package/main.roc
```
- Run it before choosing a version, so the number follows from the change and
not from habit.
- ` release.yml ` runs it with ` --expect ` in its validate step, against the
latest release for same-major requests, and stops when the requested version understates the change.
A breaking change can then never go out as a patch.
- A requested major greater than the latest final major already satisfies the
lower bound for every possible API change. The released ` release-package `
utility validates both versions and accepts that proof before attempting
extraction. It does not compute an API diff in that case. Same-major releases
still require a successful comparison, including when extraction fails.
- This version proof does not establish that the candidate compiles or works.
The configured package checks, working-tree examples, exact candidate tests
and SBOM checks remain separate release requirements. Malformed or unreadable
latest-release metadata cannot authorize the proof.
- A repository with several packages runs it once per package root.
- It compares only the modules the package header exposes. A change in behavior
behind an unchanged signature is still the maintainer’s call, and belongs in
the notes.
A package version is independent of any compiler version. A changed package is
always a new version. Nothing is ever republished under an existing version.
#### Write the release notes
Each release has its notes in ` docs/releases/.adoc `, for example
` docs/releases/0.12.0.adoc `. They are committed in a pull request before the
release is dispatched, and the workflow refuses to release a version whose
notes file is missing.
The workflow publishes the notes with the release. Keep the manual focused on
the current interface and procedures; link readers to the published releases
for version-specific changes.
The notes are an authored narrative, written for a user deciding whether and
how to upgrade. They are not a list of commits or merged pull requests.
- Open with what this release is for, in a sentence or two.
- Describe what changed in terms of what a user can now do, or must now do
differently.
- For a breaking change, show the code before and after.
- Say what was fixed, naming the visible symptom.
- Thank contributors, and link issues where a reader may want the background.
**docs/releases/0.12.0.adoc**
``` asciidoc
= roc-example 0.12.0
This release adds streaming decoders and removes the deprecated `Table.render`.
== Use this release
[source,roc]
----
app [main!] {
cli: platform "https://github.com/roc-lang/basic-cli/releases/download//.tar.zst",
example: "{package-url}",
}
----
== Streaming decoders
...
```
The bundle URL cannot be written by hand, because it contains the content hash
of a bundle that does not exist until the release builds it. The notes
therefore use a placeholder, and the workflow fills it in:
- ` {package-url} ` stands for the released bundle’s URL, and ` {platform-url} `
for a platform’s. A release with several bundles has one placeholder for
each, named after the package, such as ` {tzdb-url} `.
- After building the bundles, ` release.yml ` renders the notes with each
placeholder set to the real URL, tag and hash. A placeholder that is left
unresolved, or one the notes never use, fails the release.
- The rendered notes become the body of the GitHub release, followed by a
section headed "Compilers" that holds the output of ` roc version ` and
` roc-stable version `. That record is the statement of what the release was
built and tested with; no header pin makes it.
- A dry run renders the notes too, so a mistake in them fails on the pull
request. It renders one file: the notes for the highest version under
` docs/releases/ `, which are the notes of the next release whenever a pull
request adds or edits them. It asks nothing of GitHub to choose that file.
The committed file keeps its placeholders. It is never rewritten after the
release, and no follow-up pull request is needed.
#### The order of a release
**A release tests exact bytes, then publishes once**
1. **Validate.** Require the default branch. Reject a version whose tag or release
already exists. Require ` docs/releases/.adoc `. Check the version
with ` roc bump --expect `. Record the event commit and use that SHA in every
later job.
2. **Check.** Run ` blueprint run check ` on that commit.
3. **Build.** Build the bundle with the latest nightly, then the examples archive
with the real release URL, the documentation archive, and the manual.
Generate and locally validate the SBOM sidecars from the resolved build and
final artifacts. Render the release notes with the real bundle and SBOM URLs.
Upload them as workflow artifacts.
4. **Test.** On every supported target, download those artifacts and test them as
described in
[Build and test the exact archive](#build-and-test-the-exact-archive).
5. **Publish.** A separate job, with ` contents: write ` and a protected
environment, downloads the tested artifacts. It checks out nothing and runs no
repository script. It hands the version, the rendered notes and the asset
files to
[` actions/publish-release `](#publish-release), which stages a
draft at the tested commit, verifies every uploaded asset against the tested
file, and then publishes.
6. **Update the site.** For a final release, rebuild the documentation site so
that it points at the new release, and deploy it; see
[What the site points at](#what-the-site-points-at).
A release candidate does not change the site.
The publish job consumes the artifacts of its own run. It never rebuilds, and
never resolves a moving branch.
SBOM generation and validation belong to the read-only build task and its PR
dry run; [` actions/sbom-sidecars `](#sbom-sidecars) implements
them with the pinned Blueprint tool, and
[` actions/fetch-dependency-sboms `](#fetch-dependency-sboms)
chains the suppliers' documents in. The shared publication action does not
implement those steps; it uploads the supplied SBOM files as inert assets and
verifies their transferred bytes. Before attestation, the shared package
workflow also rechecks SBOM subject/sidecar bindings against the transferred
envelope and supported Blueprint manifests, using reviewed controller code and
the build job’s producing attempt. Pass the asset list the sidecar action
returns to the upload, the attestation and the publisher, so a missing sidecar
cannot hide behind a wildcard. Remote advisory rescans run separately from routine PR checks.
Enable immutable releases in the repository, so that a published release cannot
gain, lose or replace an asset. Hash checks remain necessary even with immutable
hosting.
#### Several bundles in one release
A repository that publishes several packages or platforms builds one bundle for
each, in the same run, under the same version, and one examples archive whose
specification names every bundle URL. The release notes list every bundle URL
by name. All of them are tested before any of them is published.
The shared release workflow derives the bundles from the examples: every
package root an example depends on by relative path is bundled, tested and
published, and a package no example exercises is not released. The repository’s
own package is ` package/main.roc `; another is ` /package/main.roc `, such as
` tzdb/package/main.roc `. A release with one bundle keeps the bare ` .tar.zst `
name. With several, the repository’s own package becomes ` -.tar.zst `
and the other ` --.tar.zst `, and its placeholder in the notes
is ` {-url} ` beside ` {package-url} `. Every tracked file in a package’s
directory goes into its bundle, modules and data alike.
Two bundles under one tag need different names as well as different hashes.
Roc identifies a package by its URL with the version and the content hash
removed. Two files named only ` .tar.zst ` under one tag are therefore one
package with two contents, and Roc refuses the pair as soon as one depends on
the other:
``` text
The same package version is being served with two different content hashes
```
Give each bundle a prefix that names its package and ends in a hyphen, such as
` example-tzdb-.tar.zst `. Roc reads the hash as the run of base58
characters at the end of the filename, so the prefix must end in a character
that is not base58, and a hyphen is one. The hash itself is never changed.
- At most one bundle in a release may go without a prefix. In a repository that
starts with several packages, name every one.
- The prefix is part of the package’s identity. Adding, removing or changing it
makes a different package as far as Roc is concerned, so choose it before the
first release, and treat a change to it as a breaking release.
- ` roc bundle ` writes ` .tar.zst `. The script that builds the release
copies that file to its prefixed name, and the prefixed file is what is
tested and published.
- ` roc bundle ` resolves the dependencies of the package it bundles. A package
that names a sibling by its final release URL is bundled after that sibling,
with the sibling’s bundle already in the package cache, because the URL does
not exist until the release is published.
- ` roc unbundle ` accepts only the bare ` .tar.zst ` name. A script that
unpacks a prefixed bundle copies it to that name first.
#### Release candidates
A release candidate, such as ` 0.12.0-rc1 `, is a prerelease published for users
to test before the maintainers commit to the release. It answers the question
the workflows cannot: does this version work in real projects, and is the API
right?
- Publish a candidate through ` release.yml ` with a prerelease version. It
follows every step above and holds the same assets, so what users test is
built exactly as the final release will be.
- Mark it as a prerelease on GitHub. It is never the repository’s latest
release.
- Announce it where users will see it, and say what feedback is wanted and for
how long.
- Fix what the feedback finds on the default branch, and publish ` -rc2 `. A
candidate is immutable like any other release and is never amended.
- When a candidate has stood without change, release the final version from
the same commit. Its bundle is a new artifact under a new tag, and it goes
through the release workflow in full.
Everything that speaks of "the latest release" means the latest release that is
not a prerelease. ` nightly.yml ` tests that one, and the documentation site
links to that one. A candidate is found from the releases page and from its
announcement.
#### Recover an interrupted publication
If publication stops, inspect exactly what exists before retrying. The publish
job’s error says which of these states it left.
- **A draft and no tag.** Nothing is public. The error names the draft, which
holds whatever was uploaded before the failure. Look at it to see how far the
job got, delete it, and re-run the failed publish job of the same workflow
run. That job publishes the files the run already tested. The publish action
never adopts a draft it finds, so the draft is deleted first.
- **A published release.** It is immutable and is never repaired. If it is
complete, the version is released and nothing is re-run. If it is not, publish
a new version.
- **Unknown.** When the answer to the publish request was lost, look at the
releases page and treat what is there as one of the two states above.
Changed bytes are always a new version. Do not publish a draft by hand, replace
an asset, or rebuild from a moving branch. If the run’s artifacts have expired,
delete the draft and dispatch the release again from the default branch. The
version was never published, so it is still free, and the new run builds and
tests its own files.
### Nightly compatibility
` nightly.yml ` answers one question each day: does the release users download
today still work with today’s compiler? A failure means the package or platform
needs a patch release.
Use the pinned shared workflow in
[Shared published-release compatibility](#shared-published-release-compatibility), rather
than maintaining a local copy of selection, test execution and reporting.
Compiler-pin updates remain manual work during downstream pull requests and do
not belong in this workflow. Warnings from successful compilation stay visible
and nonfatal; skipped or incomplete evidence never establishes a passing result.
**The inputs to one nightly run and its two outcomes**
- Resolve the latest nightly tag and the latest release once. Accept both as
optional dispatch inputs, so a maintainer can re-run a past combination.
- Skip prereleases when choosing the latest release.
- Download that release’s examples archive and run it unchanged, comparing with
the expected output inside the archive. Do not rebind it to the working tree,
and do not test the default branch’s examples here: new APIs on the default
branch are not evidence about a release.
- Use the shared runner to read the archived specification and own every
compiler invocation. Default-branch code may only provision native prerequisites
in the read-only test job.
- Run on every target the release is tested on: the one runner of a
[Roc package](#a-pure-roc-package), or the same matrix
as ` ci.yml `.
- A platform’s host bindings are not checked here. A released host is fixed;
what matters is whether its applications still build and run.
The archive is immutable, so a failure is never a reason to change it. The fix
is a patch release that works with the new compiler.
#### Reporting
A separate job reports the result by calling
[` actions/nightly-report `](#nightly-report), which keeps one
tracking issue: opened on the first failure, commented on by each later
failure, and closed by the next pass. The action defines which issue that is and
what each outcome does. The workflow supplies the rest:
- Reporting is considered after a failed test too, and holds ` issues: write `
and nothing else. Only an established success or failure reaches the issue
action; unknown evidence leaves the issue alone.
- It checks out nothing and runs no repository or archive code.
- The shared read-only aggregator verifies matching evidence from every matrix
target and the current run attempt. It supplies only a constrained outcome,
nightly tag and release tag to the report action, never example output.
- A failed test-job result alone does not establish incompatibility. Setup,
cancellation, missing evidence, skips and timeouts are distinct outcomes and
cannot close an issue. A complete pass requires every target’s tests to pass.
Manual historical override runs do not modify the latest-pair issue.
A repository with no final release or no canonical examples archive has no
compatibility result: selection fails with a diagnostic and does not call the
issue action. A missing test is never displayed as a passing nightly.
This workflow never opens a pull request, commits, merges, or publishes.
## Shared published-release compatibility
The shared nightly answers one question: does the latest published product
release work with the latest Roc nightly on every supported release target?
Consumers use ` .github/workflows/nightly-compatibility.yml ` at a reviewed full
SHA. Their caller supplies the schedule and native prerequisites; the shared
runner owns selection, archive validation, compilation, execution, diagnostics
and reporting.
Compiler-pin changes belong to maintainers during downstream pull-request work.
This nightly does not change pins, open upgrade pull requests, merge changes,
publish releases or test unreleased development examples.
### Risks and controls for compatibility verdicts
Independent moving lookups can test different release/compiler pairs, and one
passing target can hide another target’s failure or missing result. Resolve the
pair once and require complete matching evidence. The shared runner verifies
asset identity, runs the immutable suite, and aggregates every declared target
for the same run and attempt. Missing or interrupted evidence cannot close the
compatibility issue. Launch errors remain setup failures rather than synthetic
exit codes that a case could accept; signal termination cannot match an ordinary
exit expectation. Header admission checks actual dependency fields, ignoring
comments and rejecting compiler constraints or ambiguous syntax. Regression
fixtures cover these states and hosted tests
exercise native execution. The dashboard is a dated snapshot; a stopped publisher
cannot update its colour. A pass covers the declared examples and targets only.
Fuzzing builds may add only the admitted ` --fuzz ` switch; archived metadata cannot
replace the compiler, output path or other compiler arguments. Cases with unstable
fuzzer statistics explicitly opt out of stderr comparison while preserving its raw
bytes. Stdout and the normal exit code remain assertions. Signals, timeouts and
output limits cannot satisfy an expected failure code. Regression tests cover these
boundaries, and hosted Linux and Apple Silicon smoke tests exercise a published
fuzzing platform with clean replay, one-second search and an expected exit 77.
### Configure the caller
Keep one caller for the product release suite. A Roc package uses one Linux
runner. A platform names every supported release runner. One archive can cover
several bundles through its declared release dependencies.
``` yaml
name: Nightly compatibility
on:
schedule:
- cron: '17 13 * * *'
workflow_dispatch:
inputs:
nightly-tag:
type: string
default: ''
release-tag:
type: string
default: ''
permissions: {}
jobs:
compatibility:
permissions:
contents: read
issues: write
uses: lukewilliamboswell/roc-automation/.github/workflows/nightly-compatibility.yml@REVIEWED_FULL_SHA
with:
nightly-tag: ${{ inputs.nightly-tag || '' }}
release-tag: ${{ inputs.release-tag || '' }}
runners: '["ubuntu-24.04"]'
```
The permission ceiling permits the separate report job. Selection, tests and
aggregation reduce it to ` contents: read `; only reporting has ` issues: write `.
No secrets are inherited. The caller runs only on schedule or manual dispatch
on the actual default branch. The shared concurrency group serializes runs, so
an older failure cannot race a newer passing report.
` runners ` accepts a unique, non-empty JSON array drawn from ` ubuntu-22.04 `,
` ubuntu-24.04 `, ` ubuntu-24.04-arm `, ` macos-15 `, ` macos-15-intel `, ` macos-26 `,
` windows-2022 ` and ` windows-2025 `. ` macos-15-intel ` runs the ` x64mac ` target
natively, and the Windows labels run native x64 executables. Name the complete
release matrix: a target no runner executes contributes no evidence, so it is
cross-compiled only. Windows uses the same archive admission, compiler diagnostics,
case deadlines and aggregate evidence contract as Linux and macOS; the runner
adds the native ` .exe ` suffix automatically.
Independent internal pins can select an old engine or an installer without the
consumer’s reviewed Blueprint checksums even when current source tests pass.
The shared workflow and its composite action use ` $/actions/... ` references,
which GitHub resolves from the containing file’s repository and commit, without
checking out workspace files. The consumer’s one full workflow SHA therefore
fixes selection, execution, aggregation, the installer registry and reporting.
A consumer cannot shadow these actions with files in its checkout. External
actions retain full SHA pins. Source regressions enforce this composition;
hosted consumer tests exercise the real nested installation and archived games.
Unknown Blueprint versions and wrong digests still fail admission. This workflow
targets GitHub.com; self-repository references are unavailable on GitHub Enterprise
Server. A complete native runtime verdict is still required for compatibility.
For native prerequisites, ` setup-command ` is a JSON argv array, for example
` ["blueprint", "run", "nightly-prerequisites"] `. It runs from the event commit’s
checkout in the read-only test job; it must provision dependencies, not rebuild
the package or replace archived examples. ` install-blueprint: true ` installs the
consumer’s verified Blueprint; ` install-nix: true ` explicitly enables Nix for
an environment that needs it. Leave both false for a runner that needs neither.
Toolchain installation precedes the prerequisite command. Windows prerequisites
must use native tools through ` setup-command `, such as a checked-in Python setup
script. Nix is not a Windows prerequisite backend. Requesting either shared
installer on Windows fails explicitly.
For a mixed matrix, keep both installers disabled and let a portable setup script
select the native provisioning steps for each operating system. A prerequisite
failure is a setup failure, never a compatibility pass.
` test-environment ` is a JSON array of public name/value records, for example
` [{"name":"DISPLAY","value":":99"}] `, for a headless native test environment.
Reserved compiler, cache and Actions variables cannot be overridden. Names and
duplicates are checked without regard to case, including Windows ` Path `,
` PATHEXT `, ` SystemRoot ` and ` COMSPEC `. These
values are not secrets. Setup scripts must arrange any display server’s
lifetime and redirected output explicitly; exporting a variable inside the
setup child does not change the runner’s environment.
The action has a fixed 300-second compiler deadline, a 600-second prerequisite
deadline and a default 10-second deadline per example case. ` case-timeout-ms `
can select 1–120000 milliseconds. The matrix job has a 60-minute overall limit.
Each command has a 10 MB capture budget; timeout and output-limit results keep
the partial bytes captured before cancellation and terminate child processes.
On Windows, basic-cli manages the complete child tree in a Windows Job Object;
on Unix, it manages a process group. CI tests that a timed-out example cannot
leave a descendant running, alongside successful compilation and execution with
the real compiler on both supported Windows images.
The compiler and every example run without ` GH_TOKEN ` or ` GITHUB_TOKEN `.
### Keep compatibility cases bounded
The archived cases verify published behavior and expected output. A package
that demonstrates fuzzing or benchmarks supplies a small smoke mode for those
examples, with at most one second of workload execution per target. That budget
excludes compiler checks and builds. The runner’s default ten-second case
deadline is a safety limit for ordinary examples, not a recommended fuzzing
duration or permission to put a campaign in the suite. An interrupted case is
incomplete evidence, even if it ran for its allotted wall time.
Routine PR CI verifies working-tree builds and smoke execution. This shared
nightly verifies the published release on the selected compiler. Sustained fuzz
searches and statistically meaningful benchmark runs belong in separate
schedule/manual jobs, as
[Fuzzing](#fuzzing) and
[Benchmarks](#benchmarks) describe. Their failures
remain visible in their own runs; they neither replace nor alter this runner’s
compatibility verdict. In particular, a daily search of Roc builtins can catch
upstream regressions without making release examples run a ten-minute campaign.
### Select one immutable pair
The selection job resolves the latest Roc nightly and the most recently
published final numeric product release, such as ` 0.13.2 ` or ` v1.2.3 `. It
excludes drafts, prereleases and auxiliary tags such as ` tools-* `, ` runtime-* `
and ` linker-inputs-* `. Release lists are paginated; an incomplete listing is an
error. All targets receive the same ` selection.json `, never separate moving
lookups. Compiler execution verifies the installed version names that nightly.
The selected release must contain exactly one
` -examples-.zip `. ` package-name ` defaults to the
repository name. Missing examples on the newest product release are an error,
not permission to silently test an older product release. A repository with no
final release has no compatibility result; its workflow fails selection with
the reason visible in the log and does not modify the compatibility issue.
Selection records the release ID, asset ID, length and GitHub’s SHA-256 digest.
Targets download by that asset ID and check length and digest before extracting
or executing anything. This binds the tested bytes to that selection; GitHub
release metadata is not an independent provenance attestation. Roc verifies
the content identities in the archived dependency URLs when fetching bundles.
Manual ` nightly-tag ` or ` release-tag ` inputs reproduce a historical combination.
A run with either override never opens or closes the issue about the latest
pair. Use ` report-issue: false ` for an explicit diagnostic-only caller.
### Publish the suite contract
Build and test the same archive before publishing it, as
[Build and test the exact archive](#build-and-test-the-exact-archive)
describes. The canonical ZIP has one root named after its filename without
` .zip `, ordinary Unix regular files and directories, and these contents:
``` text
-examples-/
README.md
spec.json
examples/
main.roc
... sibling modules and data ...
```
Every member stays under that root. Links, traversal, duplicates, encrypted
members, file/directory collisions and ambiguous filenames are rejected before
extraction. The ZIP is at most 50 MB, expands to at most 100 MB, and has at most
10000 members. Member paths use ASCII letters, digits, dots, underscores,
hyphens and directory separators; no empty, ` . ` or ` .. ` component is allowed.
` spec.json ` is a versioned, explicit test plan. Required case fields remain
explicit even when their value is empty:
``` json
{
"schema_version": 1,
"release": "1.0.0",
"dependencies": [
"https://github.com/owner/package/releases/download/1.0.0/CONTENT_HASH.tar.zst"
],
"apps": [{
"path": "examples/main.roc",
"cases": [{
"name": "default",
"args": [],
"stdin": "",
"stdout": "Hello!\n",
"stderr": "",
"exit_code": 0
}]
}]
}
```
` release ` must match the selection. Dependencies are immutable published bundle
URLs belonging to that repository and tag. Each application directly names at
least one declared bundle in its header and has no compiler constraint: it must
be able to meet a newer compiler without editing the archived source. The runner
accepts canonical ` app [entrypoints] { alias: platform "URL", package: "URL" } `
headers, including whitespace and line comments. Dependency URLs in comments or
application bodies do not count; duplicate fields, escaped header strings and
unsupported header syntax fail admission. It never changes an archived file, substitutes the working-tree bundle or runs a
formatter. An empty package cache starts each target’s test.
The suite contains 1–100 unique application paths under ` examples/ `, with
1–100 uniquely named cases per application. The runner checks and builds each
application once with the selected compiler, then runs each recorded case and
compares stdout, stderr and exit code exactly. A failed application does not
prevent diagnostics for later applications. Declared cases are the execution
inventory; maintainers review the suite for representative coverage before
publishing it. Passing is not a claim of security compliance or complete API
coverage.
An application may add ` target `, such as ` wasm32 `, and ` run `, a JSON argv array
for its runtime. Native applications omit both. A non-native target requires a
runtime command containing ` {executable} `; ` {suite} ` names the extracted root.
For example, a Wasm platform can supply
` "target":"wasm32", "run":["node","{suite}/runtime/check.mjs","{executable}"] `.
The runtime helper and fixtures belong to the immutable archive; native setup
installs its tools. The shared runner still owns checking, compilation, timeout,
stdout/stderr capture and exact case comparison. The runtime must exercise the
built artifact and exit after its assertions; merely building a cart or opening
a window does not establish behavioral compatibility.
An application may add ` build_args: ["--fuzz"] ` to link a fuzz target. Missing or
empty ` build_args ` keeps the ordinary build; every other argument or duplicate is
rejected. These arguments affect ` roc build ` only, never ` roc check `. Fuzz builds
use ` --opt=speed `, since fuzz instrumentation requires the LLVM backend; ordinary
examples keep ` --opt=dev `.
A case may add ` stderr_match: "ignore" ` for unstable libFuzzer diagnostics, with
` stderr: "" `. Missing ` stderr_match ` or ` "exact" ` retains exact stderr comparison;
other policies are rejected. Both streams are always captured and displayed, and
stdout and the normal exit code are compared exactly in every mode. Seed replay
also goes through libFuzzer and can print varying diagnostics. A target that must
find a failure records ` exit_code: 77 `; a signal or interruption cannot match it.
The working-tree ` test-examples --update ` command preserves the selected policy
and leaves ignored stderr empty instead of recording volatile statistics.
A case may add ` skip: {"reason":"...","issue":"https://github.com/..."} `.
The app is still checked and built, and the skip is visible. Any skip prevents
a complete pass and prevents closing the compatibility issue. Prefer a runnable
headless test where possible. A suite with no executed cases is never compatible.
### Preserve diagnostics on happy and unhappy paths
Compiler stdout and stderr are always displayed, including after exit code 0.
Warnings remain nonfatal by default. Successful compilation does not imply that
it was warning-free. The runner owns compiler execution instead of delegating
it to a driver that might discard captured success output.
Each target artifact, ` compatibility- `, retains raw stdout and stderr,
command arguments, working directory, process status (normal exit, signal or
I/O error), exit code, signal number, launch-error details, interruption flags and
stdin, public environment, deadlines and expected output for failing cases.
The ZIP inventory is retained too. Console rendering escapes workflow-command
markers, carriage returns and terminal escapes; raw files preserve the original
bytes. Compiler diagnostics and example stderr are separate: expected example
stderr is compared with the case’s specification.
The ` nightly-compatibility-summary- ` artifact contains ` summary.json `:
the selected identities, per-target results, counts and aggregate verdict. The
job summary links the meaning of the verdict to the diagnostic artifacts.
Artifacts are retained for 14 days. Download useful evidence before retention
expires; the run link and selected versions allow a later manual reproduction.
| Result | Meaning | Follow-up |
| --- | --- | --- |
| ` compatible ` | Every target ran cases and passed, without skips | No compatibility action; review any compiler warnings |
| ` incompatible ` | A compiler check/build or recorded example assertion failed | Inspect per-target diagnostics; fix the compiler/package issue and publish a compatible patch release when needed |
| ` setup-failed ` | Selection, archive validation, prerequisites or target evidence failed | Repair setup or release packaging; rerun all jobs |
| ` test-timeout ` / ` output-limit ` | Execution was interrupted; compatibility is unknown | Inspect partial logs; repair the hang/noise or adjust a justified deadline |
| ` not-tested ` | Cases were skipped or no complete execution was established | Resolve the documented skips or missing work; rerun all targets |
Test failures keep the workflow red after evidence is uploaded. Reporting a
failure does not turn it green. Matrix fail-fast is disabled, so a failing
target does not cancel the others. A missing, duplicate, stale-attempt or
mismatched result cannot produce a passing aggregate. Rerun all jobs when
retrying a matrix; a successful old target is not current-attempt evidence.
Only an established compatibility failure opens or updates the one tracking
issue. Only a complete current-attempt pass on every target closes it.
Infrastructure failure, cancellation, timeout and skipped cases leave that issue
alone. The privileged report job executes only the pinned shared reporting
action; it receives outcome and selected tags, never archive diagnostics or
repository code.
### Read the downstream dashboard
The README’s compatibility column reads the fixed verdict gate of the newest
eligible shared workflow, checks its immutable caller still uses the configured
pin, and links to the run. Scheduled runs qualify. A default-branch manual run
also qualifies when its successful shared ` Track release compatibility ` job
proves both effective selectors were empty. The pinned job’s guard excludes
historical release/compiler overrides. The dashboard validates its exact name,
shared-workflow origin and run, attempt and source identities, alongside the
verdict; a local lookalike job cannot establish this proof.
The collector scans up to 50 recent runs in API order, skipping historical or
otherwise unproven manual runs so they cannot hide an older eligible result.
If none qualifies, the result stays unknown. A manual run with reporting disabled,
failed reporting or setup failure before a reportable verdict has no selector
proof and cannot replace eligible evidence. Empty selectors mean latest when
selection ran, subject to the same 48-hour freshness window as scheduled runs.
Verification requires the canonical single-job caller
shown above: one literal job-level ` uses `, optional ` with ` and ` permissions `,
and no local jobs, conditional or matrix on that call. Other caller shapes
remain unverified. GitHub must record the exact resolved shared workflow path
and SHA; every verdict job must belong to the same run, attempt and source SHA.
Job and step display names alone cannot establish the origin of a verdict. Each
snapshot marks passes older than 48 hours stale.
This age check runs at publication time: if Documentation stops deploying,
static badges retain their last colour. Check the snapshot date and investigate
publication when it is over 48 hours old before relying on a green result. Development CI,
pin-updater success and weekly floating-input checks are outside this contract.
An adoption or infrastructure message means compatibility has not been verified,
and states the follow-up. Manual historical probes do not replace eligible
results. The independent revision column tells maintainers which automation
SHAs need review and upgrade.
### Run the utility directly
The source entry point is ` scripts/release_compatibility.roc `, behind
` blueprint run release-compatibility -- `. The tools build also produces
static Linux ` release-compatibility- ` executables; adoption of a released
binary follows [Verify released artifacts](#verify-released-artifacts). New
artifacts become available only when a maintainer releases the tools.
The phases are ` select `, ` test ` and ` aggregate `, using the same ` RA_* ` environment
contract as ` actions/release-compatibility/action.yml `. ` RA_DIRECTORY ` must be a
new private directory for each invocation. ` select ` needs ` GH_TOKEN `,
` GITHUB_REPOSITORY `, ` RA_NIGHTLY `, ` RA_RELEASE `, ` RA_PACKAGE ` and ` RA_RUNNERS `.
` test ` needs ` RA_SELECTION `, ` RA_RUNNER `, the actual ` RA_COMPILER ` executable,
` RA_SETUP `, ` RA_ENVIRONMENT `, ` RA_CASE_TIMEOUT_MS `, ` RA_PYTHON `,
` RA_ARCHIVE_HELPER `, ` GITHUB_RUN_ID ` and
` GITHUB_RUN_ATTEMPT `. ` aggregate ` needs ` RA_SELECTION `, ` RA_ARTIFACTS `,
` RA_TEST_RESULT ` and the same run identity. ` GITHUB_OUTPUT ` is an existing output
file; aggregation also writes ` GITHUB_STEP_SUMMARY `.
The utility and action use ` gh ` and Python 3.10 or later on Linux, macOS and
Windows. ` RA_PYTHON ` names the Python executable and ` RA_ARCHIVE_HELPER ` names
the companion ` release-compatibility-archive.py ` artifact from the same tools
release as the utility. Verify both files against that release’s admitted
checksums. When running from source, use ` actions/release-compatibility/archive.py `
from the same reviewed commit instead.
The Python standard library verifies SHA-256, inspects ZIP metadata and extracts
regular bytes, avoiding platform-dependent shell archive tools. Archive admission
rejects Windows device names, alternate data streams, trailing dots and
case-insensitive file or parent-directory collisions on every host. No archived
code is imported by the helper. The action builds the utility with its fixed tooling compiler before
installing the selected floating package compiler; consumer Blueprint evaluation
is optional and limited to native prerequisite setup.
## Optional workflows
Some repositories need more than the three core workflows. Each addition below
has a fixed name and purpose, so that the same file means the same thing
everywhere. All of them follow the rules in
[The three workflows](#the-three-workflows): full-SHA
references, ` permissions: {} ` at the top, and no repository code in a job that
can write.
| Workflow | Trigger | Purpose | Token access |
| --- | --- | --- | --- |
| ` docs.yml ` | Push to the default branch; pull request | Build the site and manual; deploy to GitHub Pages from the default branch. | Build ` contents: read `; deploy ` pages: write `, ` id-token: write ` |
| ` fuzz.yml ` | Schedule; manual dispatch | Run fuzz targets for a bounded time and keep reproducers. | ` contents: read ` |
| ` bench.yml ` | Schedule; manual dispatch | Measure performance against a recorded baseline. | ` contents: read ` |
| ` codeql.yml `, ` scorecard.yml ` | Push, pull request, schedule | Static analysis and repository-practice scoring. | As each tool documents |
| ` link-inputs.yml ` | Pull request touching producer inputs; manual dispatch | Build, test and attest native linker inputs. | ` contents: read `, ` attestations: write `, ` id-token: write ` |
| ` publish-link-inputs.yml ` | Manual dispatch from the default branch | Publish an attested candidate and commit its lock. | See [Trusted publisher contract](#trusted-publisher-contract) |
### Risks and controls for sustained campaigns
A separate campaign job can still run on every PR, and a cancelled search can
look like a successful time-bounded run. Separate workload intent and evidence.
Use explicit schedule/manual event guards, normal bounded search completion,
per-input limits and an overall safety timeout. Preserve reproducers and exact
compiler/source identities on failure. Check the actual triggers and reproduce
a saved failing input. Campaign results describe the selected search; neither
their success nor a short PR smoke run substitutes for published-release
compatibility or a meaningful benchmark measurement.
### Documentation site
` docs.yml ` has two jobs.
The build job checks out the repository, builds the site, and uploads it as a
Pages artifact. A repository with an AsciiDoc manual uses
[` actions/build-docs `](#build-docs), which also produces the PDF.
A pull request runs this job, so a broken manual fails review, and the artifact
can be downloaded to preview the change.
The deploy job runs only for the default branch. It has ` pages: write ` and
` id-token: write `, uses the ` github-pages ` environment, checks out nothing, and
deploys the artifact the build job produced.
#### What the site points at
A visitor to the site is almost always a user, and a user has the latest
release. The site therefore leads with the latest release, not with the default
branch:
- its primary API reference is the ` roc docs ` output generated for the latest
release, restored from that release’s documentation archive;
- its examples link is the examples archive attached to the latest release; and
- its install snippet names the latest release’s bundle URL.
"Latest" excludes prereleases; see
[Release candidates](#release-candidates).
The site may also carry the default branch’s documentation, under a path and a
label that mark it as unreleased, and the documentation of earlier releases,
each restored from its own archive. Nothing rendered is committed: the site is
assembled at build time from the working tree and from release assets.
A new release changes what "latest" means, so ` release.yml ` rebuilds and
deploys the site after it publishes, in a job that holds only the Pages
permissions and deploys an artifact built by a read-only job.
### Fuzzing
Fuzz targets are written with roc-fuzz; see
[Correctness and memory properties with roc-fuzz](#property-based-tests-with-roc-fuzz)
for why and how. Named regression examples, memory assertions and accepted
allocation baselines run in fast PR checks. This workflow adds sustained
exploration; roc-fuzz’s default 60-second ` run ` is not a PR smoke command.
` fuzz.yml ` runs sustained campaigns on a schedule or manual dispatch. It has no
pull-request trigger, and campaign jobs explicitly accept only those two event
types on the default branch:
``` yaml
if: >-
(github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') &&
github.ref == format('refs/heads/{0}', github.event.repository.default_branch)
```
The same guard is required if a campaign job shares a workflow file with PR
jobs. A separate job or runner alone does not keep expensive work out of PR CI.
Campaign jobs have ` contents: read `, disable persisted checkout credentials and
run no publishing or pin-update controller.
Resolve the latest Roc nightly once and record its exact tag and ` roc version `,
the tested source commit, platform release/digest, target, arguments and duration
in the report. Define whether a budget applies per target or to the complete
campaign; ten minutes per target across dozens of targets is not a ten-minute
job. Bound each input’s time and memory as well as the campaign’s search time,
and leave headroom in the job timeout for building, cleanup and uploading logs.
Use ` TARGET run CORPUS --report-dir=PATH ` to check declared examples and exact
baselines before exploration, and write one supervised report. Choose a fresh
report directory for each invocation. Baseline failures skip exploration;
verification never invokes ` accept `. Missing baseline entries are ` unbaselined `,
so review and commit the intended baseline inventory before relying on it.
Restore and save the evolving corpus through the Actions cache, separately
from the small committed PR smoke corpus. Cache absence starts a new search;
it never excuses a missing target. Preserve crash inputs, raw stdout/stderr and
the run metadata as artifacts even when testing fails. Reproduce findings with
the saved input, not the random seed alone. A property failure, crash or
per-input timeout, allocation assertion or baseline change fails the job. A
campaign that reaches its configured search limit normally may pass. Zero
accepted exploration inputs are inconclusive and unsuccessful. Job cancellation,
missing reports and infrastructure timeouts are incomplete work, never a
successful campaign.
` ci.yml ` gives every target a smoke run: it builds the target, checks named
examples and accepted baselines, and replays the small committed corpus within the
[one-second workload budget](#heavy-runs-stay-out-of-pull-request-ci).
That keeps targets building between campaigns. Sustained searches and their
` --opt=speed ` builds stay in the campaign job.
#### Daily checks of Roc builtins
roc-fuzz’s builtin targets exercise compiler-provided operations such as list
sorting, strings and dictionaries. Run a bounded daily campaign against the
latest Roc nightly, for example a ten-minute total search budget distributed
across the selected targets. New nightlies can change these operations without
any change to roc-fuzz, so a daily campaign can find upstream regressions that
PR smoke runs and published examples do not expose.
Keep this as ` fuzz.yml `, with its own status and failure artifacts. It tests
the selected builtin targets and search configuration, while
[shared nightly compatibility](#shared-published-release-compatibility)
tests the immutable published product examples. Neither result substitutes for
the other. Investigate a failure, minimise and preserve its reproducer, and
report an upstream issue when the evidence points to Roc. Compiler-pin changes
remain a separate maintainer decision.
### Benchmarks
` bench.yml ` measures on a schedule and on request, never on a pull request.
` ci.yml ` gives every benchmark a smoke run, one small iteration built with
` --opt=dev ` and at most one second of workload execution, so that a benchmark
cannot fall behind the API it measures. Smoke timings are not performance
measurements and must not update baselines or fail a performance threshold.
Use the same schedule/manual event and default-branch guard as ` fuzz.yml `.
Measured runs use ` --opt=speed `, documented warmups, samples and runner
conditions. Record source and compiler identities beside the baseline and raw
measurements so a comparison is interpretable. Hosted-runner variance needs an
explicit measurement policy; do not turn timing noise into a PR failure.
For a ceiling-based allocation ratchet, fail when the count exceeds the bound
and lower the ceiling when it improves. roc-fuzz’s accepted allocation baselines
use exact matching instead, so both increases and reductions need review. See
[Allocation properties and exact baselines](#allocation-properties-and-exact-baselines)
for choosing and maintaining these checks.
### Linker inputs
Platforms that consume prebuilt native inputs have a producer workflow and a
publisher workflow. These are the only route by which native build inputs are
released. [Maintain a platform](#maintain-a-platform) explains
when a platform needs them, and
[Publish linker inputs from a pull request](#publish-linker-inputs-from-a-pull-request)
specifies both.
## Maintain a platform
A platform follows the same model as a package: the same environment, layout,
examples archive and three workflows, and no compiler pin in the platform or its examples. This
chapter covers what a platform adds, which is a native host. A host brings
generated bindings, a target matrix, and build inputs that are too expensive or
too sensitive to rebuild on every pull request.
### Risks and controls for native targets
A cross-compiled host can link but fail on its advertised OS, and stale glue
can disagree with the compiler’s ABI. Treat native execution and interface
identity as separate evidence. Check generated bindings against the selected
nightly, bind native inputs to their source and target, and run the exact bundles
on every supported runner. Include a consumer link/run test rather than relying
on archive presence. The target matrix establishes those exercised combinations;
it does not establish that native host code or FFI behavior is memory-safe or
that every downstream application is compatible.
### Host bindings
A host is compiled against bindings that ` roc glue ` generates from
` platform/main.roc `: a Zig, Rust or C description of the types and entry points
the Roc side expects. The compiler generates them, so a new nightly can change
them when nothing in the repository changed.
The bindings are committed. The host build reads the committed file and never
runs ` roc glue ` itself, so a host builds the same way on every runner and a
reviewer sees every change to the interface as a diff.
#### Check the bindings in pull-request CI
` ci.yml ` has one job that regenerates the bindings with the latest nightly and
requires the committed file to match:
1. Obtain the glue specification that belongs to the resolved nightly, from that
nightly’s source archive, and verify the archive’s digest.
2. Run ` roc glue