> Documentation version: 0.5.0. [HTML page](https://lukewilliamboswell.github.io/roc-fuzz/index.html)
# The roc-fuzz manual
roc-fuzz checks pure Roc code through correctness assertions, declared examples,
allocation measurements, logical trace rules and coverage-guided exploration.
One target executable checks exact accepted observations and preserves original
findings for inspection, replay, retention and minimization.
[Browse the generated API reference](api/index.html) ·
[Download the standalone PDF manual](roc-fuzz.pdf) ·
[View roc-fuzz on GitHub](https://github.com/lukewilliamboswell/roc-fuzz).
> **How to use this manual**
>
> - **New to fuzzing**: Start with [Why fuzz a Roc application?](#why-fuzz-a-roc-application) to learn what a fuzz
> target is, how guided search differs from random input, and when fuzzing fits.
>
> - **Ready to try it**: Follow [Fuzz your first function](#fuzz-your-first-function) to test a small Roc module
> and work through a saved failure.
>
> - **Longer campaign**: Continue with [Run and maintain fuzzing campaigns](#run-and-maintain-fuzzing-campaigns) for
> bounds, corpora, status output, and failure diagnosis.
>
> - **Advanced use**: Read [Understand roc-fuzz internals and advanced features](#understand-roc-fuzz-internals-and-advanced-features) for the
> runtime model, allocation assertions, generator dispatch, and raw engine options.
>
> - **Downstream maintainer**: Read [Add fuzzing to CI](#add-fuzzing-to-ci) for a bounded GitHub Actions
> pattern and its report contract.
>
> - **Release verifier**: Read [Release bundle provenance](#release-bundle-provenance).
>
>
> - **Platform contributor**: Read [Contributing to roc-fuzz](#contributing-to-roc-fuzz) for toolchains,
> validation, packaging, and release behavior.
## Why fuzz a Roc application?
Suppose a decoder passes every example you wrote, but one unusual combination
of lengths and Unicode text still makes it fail. Adding more example tests helps
only when you can imagine the troublesome input in advance.
Fuzzing repeatedly runs part of your application with varied inputs. It watches
which inputs reach new behavior and uses those discoveries to guide the search.
You describe what valid inputs look like and what must always be true. roc-fuzz
does the repetitive search and saves any input that exposes a failure.
### What a fuzz target is
A **fuzz target** is a small Roc application that connects roc-fuzz to one
behavior in your code. It contains three essential ideas:
- a **generator** that creates the Roc values to try;
- an **operation** that calls the application code you want to exercise; and
- a **property** that decides whether the result is acceptable.
**Target anatomy**
The target is necessary because roc-fuzz cannot infer what correctness means for
your application. It can explore inputs and notice crashes, hangs, leaks, and
new behavior in the compiled program, but only you can state application rules such as:
- encoding and then decoding returns the original value;
- pushing an item and then popping returns that item and the original stack;
- normalizing an already normalized value changes nothing;
- two independent implementations produce the same result; or
- arbitrary input may return an ordinary error but must never crash.
A target is development and test code. It is separate from the application you
ship, and it normally imports the module it exercises. Start with one operation
and one property per target so failures remain easy to understand.
### How fuzzing complements tests
Unit tests and fuzzing answer different questions and work best together.
| Approach | What it does well | What it leaves to you |
| --- | --- | --- |
| Unit tests | Check named examples and preserve known regressions quickly and deterministically. | You must choose every example. |
| A hand-written random loop | Tries more values than a short example list. | You must build generation, search strategy, repeatability, failure storage, and diagnostic tools. |
| roc-fuzz | Uses execution feedback to explore, retains useful inputs, and saves failures that can be shown, replayed, and minimized. | You must define meaningful generators and properties; a weak property can miss real bugs. |
After roc-fuzz finds a bug, fix the code and retain its original bytes with
` TARGET retain `. Add the printed ` Fuzz.saved ` declaration alongside typed source
examples. Those regressions run independently of coverage retention, while
exploration continues searching for new cases. Unit tests can share the same
application invariants.
### What “typed” and “coverage-guided” mean
**Typed** means the test receives ordinary Roc values rather than making every
target decode an arbitrary byte array itself. A generator can produce a ` U64 `,
a ` Str `, or a record such as
` { delimiter : Str, value : Str } `. You choose the useful ranges and shapes.
roc-fuzz does not automatically know your application’s domain rules.
**Coverage-guided** describes how the search chooses where to spend effort.
roc-fuzz observes which parts of the compiled program each input reaches. When
an input reaches behavior not seen before, it retains that input and creates
further variations from it. This feedback makes the search more purposeful than
independent random samples.
**Coverage-guided search and failure workflow**
The retained useful inputs form a **corpus**. A later run can reuse that corpus to
resume from behavior already discovered. Coverage here is search feedback, not
a percentage and not proof that the program is correct.
roc-fuzz embeds a mature native fuzzing engine in the executable built by Roc.
Normal users interact with roc-fuzz commands and Roc values; knowledge of that
engine is needed only for the expert escape hatch described later.
### What roc-fuzz provides
For a Roc application author, the platform supplies:
- typed generators for common Roc values and tools for combining them;
- coverage feedback that directs the search toward new behavior;
- a bounded local runner with sensible defaults;
- saved inputs for crashes, failed properties, hangs, memory limits, and leaks;
- commands to render a saved input as a Roc value, reproduce it, and minimize it;
- a persistent corpus that makes later runs build on earlier discoveries; and
- a supervised CI mode that records configuration, logs, hashes, and failures.
A target builds into one self-contained executable:
``` sh
roc build --fuzz fuzz/stack/main.roc --output=stack-fuzz
./stack-fuzz run
```
The normal workflow needs no separate native project or fuzzing tool
installation.
### Choose a good first target
Good first targets are fast, deterministic operations over in-memory values with
a rule that is easy to state.
| Application behavior | Possible property | Why it fits |
| --- | --- | --- |
| Parser or decoder | Every input returns a value or an ordinary error without crashing or hanging. | Large input space and clear acceptable outcomes. |
| Encoder and decoder | Decoding an encoded value returns the original value. | Strong round-trip property. |
| Collection or state transition | An operation preserves membership, length, ordering, or an independent model. | Many combinations can exercise edge cases. |
| Canonicalization or normalization | Applying the operation twice gives the same result as once. | Simple idempotence property. |
| Two implementations of one rule | Both produce the same result. | Each implementation acts as an oracle for the other. |
A target is a poor fit when one input performs slow network, filesystem, UI, or
database work; depends on nondeterministic external state; can legitimately run
without a practical time or memory bound; or deliberately terminates the
process for ordinary invalid input. Prefer unit, integration, end-to-end, or
benchmark tests for those jobs.
Fuzzing consumes CPU time and stores a corpus and failure artifacts. More time
can explore more behavior, but a clean run means only that no failure was found
during that search. It never proves the absence of bugs.
roc-fuzz currently supports Linux x86-64 with musl and Apple Silicon macOS 11 or
newer. Continue with [Fuzz your first function](#fuzz-your-first-function) to build
a target against a small application module and work through a failure.
## Fuzz your first function
This tutorial adds a fuzz target beside a small Roc module, runs it, inspects a
failure, fixes the bug, and preserves the lesson as a regression test. You do
not need prior fuzzing experience or knowledge of the embedded engine.
The example uses a stack because its rule is easy to state: after pushing an
item, popping must return that item and the original stack. The demo keeps the
application module beside its target only to make the tutorial self-contained;
a real target normally imports an existing module from the application.
### Create the application module
Create ` fuzz/stack/Stack.roc `:
``` roc
Stack := [].{
push : List(a), a -> List(a)
push = |items, item| List.append(items, item)
pop : List(a) -> Try({ value : a, rest : List(a) }, [Empty])
pop = |items|
match items {
[first, .. as rest] => Ok({ value: first, rest })
[] => Err(Empty)
}
}
```
This version contains a plausible implementation mistake: ` push ` appends the
new item to the end, while ` pop ` removes an item from the front. A test using
only an empty stack would pass, so the defect is easy to overlook.
The fuzz target is a separate development application that imports this module;
the shipped application does not need to depend on roc-fuzz.
### Write the target
Create ` fuzz/stack/main.roc `:
``` roc
app [target] { fuzz: platform "PLATFORM_URL" }
import fuzz.Fuzz
import Stack
test : List(U8) -> Fuzz.Outcome
test = |initial| {
item = 0
pushed = Stack.push(initial, item)
match Stack.pop(pushed) {
Ok({ value, rest }) if value == item and rest == initial => Fuzz.keep
Ok(_) => crash "pop did not undo push"
Err(Empty) => crash "the stack was empty after push"
}
}
target = Fuzz.target_with({
examples: [],
name: "stack-push-pop",
generator: Fuzz.list(Fuzz.u8, 8),
test,
show: |initial| Str.inspect(initial),
})
```
Replace ` PLATFORM_URL ` with the platform URL in the notes of the
[latest release](https://github.com/lukewilliamboswell/roc-fuzz/releases/latest).
The target connects the three ideas from the previous chapter:
| Part | Role in this target |
| --- | --- |
| Generator | ` Fuzz.list(Fuzz.u8, 8) ` supplies lists containing up to eight ` U8 ` values. |
| Application operation | The test calls ` Stack.push ` and ` Stack.pop `. |
| Property | After pushing ` 0 `, the popped value must be ` 0 ` and the remaining stack must equal the generated ` initial ` value. |
` Fuzz.keep ` means the generated case completed normally and satisfied the
property. ` crash ` means roc-fuzz must save the case as a failure. ` show ` tells
roc-fuzz how to render the generated value for a person.
The application header selects an immutable roc-fuzz release and its compatible
Roc compiler. For another project, select a release from
[GitHub Releases](https://github.com/lukewilliamboswell/roc-fuzz/releases) and use
the compiler documented with that release. Roc obtains the platform through its
normal package mechanism; there is no separate roc-fuzz installer.
### Build and find the bug
Build a self-contained fuzzing executable:
``` sh
roc build --fuzz fuzz/stack/main.roc --output=stack-fuzz
```
Start with the bounded default run:
``` sh
./stack-fuzz run
```
roc-fuzz generates initial stacks and repeatedly runs the property. An empty
stack passes even with the bug, while a non-empty stack can expose the mismatch
between the two ends. The exact status lines and saved hash vary, but the end of
the output includes commands shaped like these:
``` console
Next steps:
./stack-fuzz show .roc-fuzz/crash-
./stack-fuzz replay .roc-fuzz/crash-
./stack-fuzz minimize .roc-fuzz/crash- .roc-fuzz/minimized-crash
```
The saved file contains the reproducible input used by the generator. It is an
implementation format for roc-fuzz, so inspect it with ` show ` instead of opening
it as text:
``` sh
./stack-fuzz show .roc-fuzz/crash-
```
For this target, ` show ` prints the generated initial stack. Reproduce the failure
in a fresh process:
``` sh
./stack-fuzz replay .roc-fuzz/crash-
```
Then ask roc-fuzz for a smaller reproducing input:
``` sh
./stack-fuzz minimize \
.roc-fuzz/crash- .roc-fuzz/minimized-crash
./stack-fuzz show .roc-fuzz/minimized-crash
./stack-fuzz replay .roc-fuzz/minimized-crash
```
Minimization is most useful for structured or long inputs, but using the same
workflow here establishes the habit.
### Fix and preserve the failure
The two stack operations disagree about which end is the top. Fix ` push ` in
` Stack.roc `:
``` roc
push : List(a), a -> List(a)
push = |items, item| List.prepend(items, item)
```
Rebuild the executable and replay the minimized input. It should now complete
without a failed property:
``` sh
roc build --fuzz fuzz/stack/main.roc --output=stack-fuzz
./stack-fuzz replay .roc-fuzz/minimized-crash
./stack-fuzz run
```
Retain the exact witness and add its printed ` Fuzz.saved ` declaration to the
target’s ` examples ` list:
``` sh
./stack-fuzz retain .roc-fuzz/minimized-crash --name previous-crash --output=fuzz/regressions
./stack-fuzz check
```
You can also append a normal unit test using the minimized non-empty stack to
` Stack.roc `:
``` roc
expect {
initial : List(U8)
initial = [1]
match Stack.pop(Stack.push(initial, 0)) {
Ok({ value, rest }) => value == 0 and rest == initial
Err(Empty) => Bool.False
}
}
```
Run it with the rest of the module’s tests:
``` sh
roc test fuzz/stack/Stack.roc
```
Use the actual value printed by ` show ` if minimization found a different
one-element list. The regression test protects this known case on every test
run; the fuzz target continues exploring other values and future implementation
changes.
A bounded run that finds no failure is still a successful run. It means the
property held for the cases explored during that run, not that the operation is
proved correct.
### Understand a normal run
By default, ` run ` checks every declared example once in a fresh process, then
explores for 60 seconds. It keeps useful inputs under ` .roc-fuzz/corpus ` and
saves failures in the run’s report directory under ` .roc-fuzz/reports/ `. The corpus is a set
of inputs that reached behavior worth revisiting; later runs reuse it and
continue searching from those discoveries.
During a clean run, ` NEW ` means the engine retained an input that reached new
behavior and ` DONE ` means the configured bound was reached. The
[campaign status glossary](#tune-a-run) explains the remaining
fields and tuning options.
### Design the next target
Choose one fast application operation and one rule that should hold for every
input. Common property shapes include:
- **Round trip:** decoding an encoded value returns the original value.
- **Inverse:** an operation followed by its inverse returns the starting value.
- **Idempotence:** normalizing twice gives the same result as normalizing once.
- **Invariant:** an operation preserves ordering, length, membership, or another
rule of the data structure.
- **Differential:** two independent implementations produce the same result.
- **Robustness:** arbitrary input may return success or an ordinary error, but
must not crash or hang.
Generate useful values directly. If a parser accepts bytes of any shape, arbitrary
bytes may be appropriate. If an operation requires a bounded port number or a
non-empty list, express that rule in the generator so most search effort reaches
the behavior you care about. Use ` Fuzz.reject ` only when a generated case lies
outside the property; do not use ` crash ` to reject an ordinary invalid input.
Keep unrelated operations and properties in separate targets. A practical first
set might include one target per parser, encoder/decoder pair, state transition,
or important business invariant. There is no need to fuzz every function.
The [campaign guide](#run-and-maintain-fuzzing-campaigns) explains corpora, tuning, and unstable
failures. The [advanced guide](#understand-roc-fuzz-internals-and-advanced-features) covers generator dispatch,
allocation assertions, and the low-level engine interface. Run ` TARGET --help `
for the command summary and browse the [generated API
reference](api/index.html) for generator, combinator, and target signatures.
### Repository and release examples
The repository’s ` examples/ ` applications use relative paths to the local platform;
prepare its native inputs before building them. Published releases provide a
` roc-fuzz-examples-VERSION.zip ` download containing complete runnable applications
with that release’s content-addressed platform URL. Neither the platform nor
an archived example pins a compiler: a daily compatibility check runs the latest
release’s archived examples, unchanged, on the newest Roc nightly.
## Design effective targets
The first tutorial generated one ` U8 `. Application behavior usually accepts
records, lists, strings, or domain types. Build a generator that produces values
close to the operation’s useful input domain, then keep the property focused on
one behavior.
### Start from the operation
Choose a function that is fast, deterministic, and easy to call in memory. Write
the property in one sentence before choosing a generator. If the sentence needs
several unrelated “and” clauses, make separate targets.
Examples include:
- “Decoding an encoded message returns the original message.”
- “Sorting preserves every item and produces nondecreasing order.”
- “A parser returns ` Ok ` or ` Err ` for every string without crashing.”
- “Applying this state transition agrees with the simple reference model.”
A generator explores only the cases the property can observe. Calling an
operation without checking its result finds crashes and resource failures, but
cannot find an ordinary wrong answer.
### Generate useful values
roc-fuzz provides generators for numeric values, strings, bytes, and collections.
Use a bounded generator when the full type includes values that are irrelevant
or too expensive for this target:
``` roc
port = Fuzz.u8_in(1, 255)
name = Fuzz.str
payload = Fuzz.list(Fuzz.u8, 1024)
```
` Fuzz.list(item_generator, maximum_length) ` generates a list whose elements come
from ` item_generator `. The maximum is a test-design bound, so choose a size that
reaches meaningful behavior without making each case slow.
Use ` Fuzz.map ` when the domain value can be constructed from one generated value,
and ` Fuzz.map2 ` when it needs two:
``` roc
non_empty_bytes = Fuzz.map(
Fuzz.list(Fuzz.u8, 255),
|rest| List.prepend(rest, 1),
)
range = Fuzz.map2(
Fuzz.u8,
Fuzz.u8,
|a, b| if a <= b { start: a, end: b } else { start: b, end: a },
)
```
Constructing valid values directly gives the search more useful executions.
Generation controls bias; property preconditions or validated input types define
validity. Encoded examples and their mutations bypass the generator and must
pass the same validity checks.
### Build record inputs
Roc’s record-builder syntax can combine a generator for each field:
``` roc
Input := { delimiter : Str, value : Str }.{
parser_for : _
encoder_for : _
}
input_generator : Fuzz.Generator(Input)
input_generator = {
delimiter: Fuzz.str,
value: Fuzz.str,
}.Fuzz
```
Use the resulting generator with ` Fuzz.target_with `:
``` roc
target = Fuzz.target_with({
examples: [],
name: "split-round-trip",
generator: input_generator,
test,
show: |input| Str.inspect(input),
})
```
This explicit form is a good default for a generator used by one target. When a
named input type owns a reusable generation policy, define its ` generator_for `
method and use ` Fuzz.target `:
``` roc
Input := { delimiter : Str, value : Str }.{
parser_for : _
encoder_for : _
generator_for = |_| {
{
delimiter: Fuzz.str,
value: Fuzz.str,
}.Fuzz
}
}
target = Fuzz.target({
examples: [],
name: "split-round-trip",
test,
show: |input| Str.inspect(input),
})
```
Both forms produce the same kind of typed target. The
[advanced explanation](#understand-generator-dispatch) describes
how Roc resolves ` generator_for `; using the pattern does not require knowing its
implementation.
### Declare examples and regressions
``` roc
examples: [
Fuzz.example({ name: "empty", revision: "1", make: || Input.{ delimiter: ",", value: "" } }),
Fuzz.saved({ name: "previous-crash", revision: "1", bytes: regression_bytes }),
]
```
A typed example is encoded into real corpus seed bytes; it does not need an
inverse generator. Saved declarations keep original artifacts. Every example
runs independently of corpus retention, exactly once in a fresh process during
checking or acceptance. Increment revisions when an example’s meaning changes.
The baseline also checks the actual input digest, even if the revision is unchanged.
### Handle invalid input deliberately
An ordinary invalid input is not automatically a fuzzing failure. A parser may
return ` Err ` for malformed text and still satisfy a robustness property. Return
` Fuzz.keep ` when the observed result is allowed.
Prefer a generator that creates valid domain values. Return ` Fuzz.reject ` when a
precondition is awkward to encode and the generated case is outside the property
you are testing. Do not use ` crash ` or ` expect ` to reject such a case; those mean
roc-fuzz found a failure worth saving.
Watch the rejection rate as tooling adds support for it. If most inputs are
rejected, redesign the generator so the campaign spends more time in the code
that matters.
### Keep saved inputs meaningful
A saved file is decoded by the target’s generator. Changing that generator can
make an old file produce a different typed value. Keep generators stable when a
corpus or failure matters, and retain the matching executable when exact
historical reproduction is important.
The [generated API reference](api/index.html) lists all generators and
combinators. Continue with [Run and maintain fuzzing campaigns](#run-and-maintain-fuzzing-campaigns)
when the target produces useful values and passes a short smoke run.
### Arctangent builtin properties
The ` tests/targets/builtins/arctangentDec.roc `, ` arctangentF32.roc `, and
` arctangentF64.roc ` targets check ` atan ` and ` atan2 ` with composed numeric
generators. They also assert that the numeric checks allocate no heap memory;
input generation and diagnostic rendering are outside that measurement.
The float targets check NaN propagation, signed zeros, infinities, range,
quadrants, and odd symmetry. F32 results are compared with F64 results rounded
to F32, allowing two ULP. F64 angles are checked against normalized input
coordinates using sine and cosine, with absolute tolerance ` 3e-15 `. Quadrant
checks allow two ULP at the rounded half-pi boundary, while zero and infinity
cases require exact result bits.
The Dec generator composes two U64 values into a signed 128-bit attosecond
payload and varies its magnitude with a shift. It checks axes, range, symmetry,
and agreement between ` atan(y) ` and ` atan2({ x: 1, y }) `, allowing 128 attoseconds.
Comparison with the separate F64 implementation allows ` 4e-15 ` for conversion
and floating-point error. ` Dec.lowest ` is tested but excluded from negation
checks because its negation overflows.
These checks supplement the compiler’s high-precision unit tests; shared
algorithms and finite tolerances mean they do not prove correct rounding.
To run just these inventory entries, with a ` roc ` on ` PATH ` that includes the
` atan2 ` builtins:
``` sh
blueprint run test-targets -- --operation all --max-total-time 60 \
--target arctangentDec --target arctangentF32 --target arctangentF64
```
Use the normal runner’s ` show `, ` replay `, and ` minimize ` commands to investigate
saved failures, as described in the campaign chapter.
## Run and maintain fuzzing campaigns
After the first target works, use these goal-oriented guides to choose bounds,
interpret status output, retain useful inputs, investigate unstable failures,
and prepare longer searches. The [advanced guide](#understand-roc-fuzz-internals-and-advanced-features) covers the
embedded engine and target implementation details.
### Tune a run
Give every new target a short, bounded smoke run before a long campaign:
``` sh
TARGET run \
--time=60 \
--max-input-size=4096 \
--timeout=5 \
--memory-limit=2048 \
--print-final-stats
```
The most useful limits are:
| Option | Meaning | Starting point |
| --- | --- | --- |
| ` --time=N ` | Stop after ` N ` seconds; ` 0 ` is unbounded. | Defaults to ` 60 `. |
| ` --runs=N ` | Stop after ` N ` generated inputs. | Useful for repeatable CI checks. |
| ` --max-input-size=N ` | Limit raw entropy bytes supplied to the generator. | Start small and raise only when needed. |
| ` --timeout=N ` | Treat one input taking longer than ` N ` seconds as a failure. | A few seconds for a normally fast target. |
| ` --memory-limit=N ` | Stop when the process exceeds this many MB. | Set from the expected working set. |
| ` --dictionary=FILE ` | Add important byte tokens to mutations. | Useful for text or binary formats. |
| ` --seed=N ` | Select the engine’s random seed. | Useful for repeating a campaign with the same runner build and configuration. |
| ` --print-final-stats ` | Print final execution and resource counters. | Useful in CI and smoke tests. |
` --max-input-size ` limits the generator’s raw input, not necessarily
every typed list or string it produces. Put bounds in the generator as
well when a typed value could become expensive.
Do not confuse the campaign seed with a saved input. Different engine
builds, coverage layouts, or configurations may generate different
mutation sequences from ` --seed `. Reproduce a failure with its saved
artifact and ` replay `, not with the campaign seed alone.
The runner reports compact status fields:
- ` NEW `: an input reached behavior not represented by the corpus and was
saved.
- ` REDUCE `: a smaller input preserved previously found behavior.
- ` cov `: covered edges or blocks.
- ` corp `: number and total byte size of in-memory corpus entries.
- ` exec/s `: target executions per second.
- ` rss `: current process memory use.
- ` DONE `: the run reached its time or iteration limit.
Coverage growth shows that the fuzzer is exploring. A plateau is a
reason to inspect the target, generator, seeds, and dictionary, not
proof that the code has no bugs. Very low ` exec/s `, steadily growing
` rss `, or repeated timeouts usually calls for a narrower target or
tighter bounds.
### Maintain a corpus
A corpus is a collection of small raw inputs that collectively reach
useful code. It is not a directory of every attempted input. Use a
stable corpus directory across campaigns:
``` sh
TARGET run quality/corpus/my-target --time=3600 --max-input-size=4096
```
Seed a target with a few small examples that reach meaningfully
different behavior, then let the fuzzer extend them. Because roc-fuzz
decodes raw entropy into typed values, a seed file’s bytes may not
resemble the value printed by ` show `. Check manually created seeds with
the target executable.
Reduce a large corpus into a new, initially empty directory while
preserving coverage:
``` sh
TARGET reduce-corpus quality/corpus/my-target quality/corpus/my-target-small
```
Keep valuable seeds and fixed crash inputs with the target in version
control. Use a deliberate directory such as ` fuzz/corpus// `,
document why each seed matters, and replay it in tests. Generated
` .roc-fuzz/ ` directories are ignored at every depth; keep evolving
campaign corpora in local storage or CI caches, and preserve findings as
CI artifacts until reduced to regressions. Run them after code changes
so the target does not silently stop compiling or exercising the
intended behavior.
Dictionaries are most effective when recognizable bytes flow fairly
directly into a parser. They may help less when a structural generator
heavily transforms the raw input.
Changing a generator can make old corpus and crash files decode to
different typed values. The generator is part of the target’s input
format. Keep it stable when practical and use ` show ` to check saved
inputs after a change.
### Diagnose an unstable failure
Always replay a saved failure in a fresh process. If it does not
reproduce, investigate:
- a generator or target change that altered the typed value or property;
- a different or truncated raw input file;
- an executable built from different target code; or
- a compiler or runtime defect.
Treat the failure as actionable once its conditions are understood and,
ideally, it reproduces from a clean process.
### Know what counts as a failure
Coverage feedback tells the engine which inputs explore new behavior; it
does not tell the runner whether a result is correct. The property in
the target is the correctness oracle.
roc-fuzz saves explicit Roc ` crash ` and failed ` expect ` calls, per-input
timeouts, and process memory-limit failures. A wrong answer that does
not break the target’s property is invisible to the fuzzer, which is why
choosing a strong property matters more than simply calling the function
under test.
### Prepare a long campaign
Before committing substantial CPU time, check that:
- the target tests one named behavior with a meaningful property;
- invalid input returns normally unless invalid input crashing is the
bug;
- the generator creates useful values without excessive rejection;
- input sizes and allocations are bounded;
- every generated input terminates or is caught by the timeout;
- a one-minute run gains coverage without immediate hangs or memory
growth;
- a deliberate temporary ` crash ` is saved, shown, replayed, and
minimized as expected; and
- the corpus and fixed failures have a clear place in the repository and
CI.
### Command reference
| Command | Behavior |
| --- | --- |
| ` TARGET ` or ` TARGET run [CORPUS] [OPTIONS] ` | Check examples and baseline, then explore, with one report. |
| ` TARGET check [OPTIONS] ` | Run each declared example once in a fresh process and compare the baseline. |
| ` TARGET accept [OPTIONS] ` | Explicitly accept the complete valid allocation observations. |
| ` TARGET explore [CORPUS] [OPTIONS] ` | Run coverage-guided exploration independently. |
| ` TARGET examples ` | List names, revisions and input digests. |
| ` TARGET show INPUT ` | Display the decoded representative. |
| ` TARGET replay INPUT [--detailed] ` | Reproduce one input under the supervisor. |
| ` TARGET measure INPUT ` | Print one input’s named allocation observations as JSON. |
| ` TARGET retain INPUT --name NAME --output DIR ` | Keep original bytes and print a source declaration. |
| ` TARGET minimize INPUT OUTPUT [OPTIONS] ` | Minimize a reproducing finding within the campaign budget. |
| ` TARGET reduce-corpus INPUT OUTPUT ` | Retain coverage-relevant inputs in OUTPUT. |
| ` TARGET raw ... ` | Advanced libFuzzer controls. |
Friendly execution options include:
| Option | Meaning |
| --- | --- |
| ` --time=N ` | Exploration budget in seconds; default 60, 0 unbounded. |
| ` --runs=N ` | Exploration execution bound. |
| ` --timeout=N ` | Invocation deadline in seconds; default 10. |
| ` --max-input-size=N ` | Bound corpus input bytes, from 1 byte through 1 MiB. |
| ` --memory-limit=N ` | Exploration process RSS limit in MiB. |
| ` --dictionary=FILE ` | Mutation tokens. |
| ` --seed=N ` | Exploration random seed; retain input bytes for reproduction. |
| ` --report-dir=DIR ` | Absent or empty destination for the complete report. |
| ` --baseline=PATH ` | Override ` _baseline.json `. |
| ` --profile=NAME ` | Select the named baseline profile. |
| ` --detailed ` | Capture a detailed logical trace for replay. |
| ` --no-detect-leaks ` | Disable the broader invocation leak check. |
Provenance options are ` --source-revision `, ` --roc-version `,
` --platform-release ` and ` --platform-sha256 `, each supplied as ` --name=value `.
See [Add fuzzing to CI](#add-fuzzing-to-ci) for evidence and baseline workflows.
Minimization and corpus reduction accept ` --time `, ` --runs `, ` --timeout ` and
` --report-dir `. They default to a 60-second campaign budget and require a positive
` --time `; unbounded exploration remains available with ` run --time=0 ` or
` explore --time=0 `. A successful minimization writes the best witness, including
the unchanged original if no reduction was found. Failed minimization preserves
an existing output. Corpus reduction requires an absent or empty output directory
that does not overlap the input directory.
## Add fuzzing to CI
Every target executable contains its supervisor, baseline reader and evidence
writer. A consumer needs Roc and the compiled target. The
[Blueprint consumer template](https://github.com/lukewilliamboswell/roc-fuzz/tree/trunk/templates/quality) supplies pinned
compiler downloads, ordinary tasks and a workflow with no package provider.
It lives outside the release example URL rewriting path.
``` sh
roc build --fuzz fuzz/main.roc --output=fuzz-target
./fuzz-target run .roc-fuzz/corpus \
--report-dir=.roc-fuzz/evidence \
--time=60 --timeout=10 --max-input-size=4096 \
--source-revision="$SOURCE_REVISION" \
--roc-version="$ROC_VERSION" \
--platform-release="$ROC_FUZZ_RELEASE" \
--platform-sha256="$ROC_FUZZ_SHA256"
```
` run ` checks the complete declared example inventory and exact accepted baseline
before exploring. A failed example or baseline change skips exploration and
records the reason in the same report. Each declared example executes once in a
fresh child process. Exploration reuses one instrumented child for efficiency.
The parent writes reports even when a child crashes or exceeds its deadline.
Example preparation also runs in a supervised child: a crashing or nonterminating
maker cannot terminate the supervisor or accept a partial inventory.
Defaults are 60 seconds of exploration, a 10-second per-input deadline and leak
checking. ` check ` runs examples and baseline checks without exploration;
` explore ` runs the search independently. A missing accepted entry reports
` unbaselined ` and allows normal execution. Exploration with zero accepted
executions is ` inconclusive ` and exits unsuccessfully.
### Review and accept allocation observations
``` sh
./fuzz-target check
./fuzz-target accept --profile=default
./fuzz-target check
```
The default file uses the target’s declared name, such as
` my_target_baseline.json `, in the current working directory. Use
` --baseline=PATH ` to override it. Separate entries hold each supported host target
and named profile. Commit the accepted file after reviewing it; CI runs ` check `
or ` run ` and never accepts changes automatically.
Matching is exact: example names, revisions, input digests, region inventory,
metric version and all statistics. A reduction also requires review. Acceptance
runs each declared example once and updates the relevant entry atomically only
when the complete inventory passes. Rejections, assertions, crashes, timeouts,
incomplete observations and write failures cannot accept a partial inventory.
### Read the evidence
An explicit report directory must be absent or empty. Keep it separate from
inputs and the persistent corpus. Without an override, each invocation chooses
a fresh directory under ` .roc-fuzz/reports/ `.
The directory contains:
- ` report.json `, following [` roc-fuzz-quality/1 `](roc-fuzz-ci-report.schema.json);
- ` summary.md ` and the combined child ` run.log `;
- original example inputs, worker observations and original trace snapshots;
- exploration failure artifacts and ` original-input.bin ` for the last attempted
input, with capture completeness and its SHA-256 in the report; and
- bounded original-run logical traces, including unfinished regions on failure.
The report records the metric version, effective execution controls (including
leak checking), preparation status, structured finding kinds and each
worker’s exact span within the combined log. Inputs are capped at 1 MiB in the
friendly commands; the prepared-inventory JSON transport is capped at 32 MiB.
Use the report’s outcome, baseline status, metric validity, trace-rule result
and capture completeness independently. Missing diagnostic events do not make
allocation statistics invalid; online trace rules continue past capture limits.
A killed writer can leave an incomplete capture, which is explicitly identified.
When a worker returns no result, ` metric_valid `, ` trace_rules_passed ` and
` capture_complete ` are ` null ` (unknown). Fatal outcomes remain separate from
partial evidence in the original snapshot.
A detailed replay is separate evidence rather than a reconstruction of the
original run:
``` sh
./fuzz-target replay finding.bin --detailed --report-dir=.roc-fuzz/replay
./fuzz-target measure finding.bin --report-dir=.roc-fuzz/measurement
./fuzz-target retain finding.bin --name previous-crash --output=fuzz/regressions
```
` measure ` prints a JSON array of named allocation observations. ` retain ` keeps
the original bytes and prints a ` Fuzz.saved ` source declaration plus the decoded
representative. Keep the binary witness when storage shape matters.
Exit status ` 0 ` means success, ` 1 ` means a finding, baseline change or inconclusive
exploration, and ` 2 ` means an infrastructure/configuration failure. Native child
termination remains separate in the report.
### Keep project policy in the project
Enumerate your targets in ordinary Blueprint tasks or a CI matrix. Choose valid
input domains in properties or validated types, and use generators for bias.
Keep resource bounds appropriate for the operation. Acceptance remains a
separate explicit task. The repository’s maintainer toolchain can use Nix while
consumer tasks use only fetched compilers and self-contained target executables.
#### GitHub Actions pattern
The template under ` templates/quality/ ` includes a complete pinned-action workflow
for Linux and Apple Silicon macOS. Its verification task builds, checks and
explores with ` run --report-dir=... `. Upload evidence even on failure:
``` yaml
- name: Verify quality
run: blueprint workflow verify
- name: Preserve evidence
if: always()
uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9
with:
name: quality-${{ runner.os }}
path: .roc-fuzz/evidence
if-no-files-found: error
```
### Coverage limits
Coverage feedback guides discovery; it is not a proof of correctness or a claim
that every source branch ran. A retained corpus and a declared regression
inventory serve different purposes: coverage reduction can remove redundant
seeds while source-declared examples still run independently on every check.
Attempted, accepted and rejected execution counts show whether the property
actually exercised useful inputs. A campaign that rejects every input provides
no passing evidence.
## Release bundle provenance
roc-fuzz does not store native archives or objects in Git. The independent
` link-inputs.yml ` producer builds x64-musl and Apple
Silicon inputs on native GitHub-hosted runners, packages them without
` libhost.a `, and tests the exact archives with fresh hosts and final bundles. It
attests a canonical manifest and each archive but cannot publish. The trusted
default-branch publisher verifies that evidence for the exact same-repository
pull-request ref and SHA before creating an immutable release.
Platform builds restore the reviewed ` native-libraries.lock.json ` manifest and
archives, verify sizes and SHA-256 digests on downloads and cache hits, validate
their locked release, source and inventory, and build fresh hosts.
The reviewed content hash is the routine trust anchor, so a valid cache hit uses
zero network and does not contact GitHub’s attestation service. A corrupt cache
entry is discarded and only the exact locked asset may be downloaded again. The
producer attestations are admission evidence checked once by the trusted
publisher. See [linker-input setup](#generate-platform-inputs).
After those consumer tests pass, GitHub’s attestation service creates
signed SLSA build provenance and an SBOM attestation for the exact
` .tar.zst ` bundle. GitHub binds the attestation to the release workflow
and source revision with its OIDC identity, signs it through Sigstore,
and records it in the repository attestations API. No long-lived signing
key is stored in the repository.
Each release also publishes ` .intoto.jsonl ` for offline
attestation verification, and three files beside the bundle that describe
what it holds:
- ` .spdx.json `, an SPDX software bill of materials;
- ` .cdx.json `, the same graph in CycloneDX;
- ` .sbom-manifest.json `, the evidence manifest that names the bundle
and each SBOM by size and SHA-256.
roc-blueprint’s SBOM tool writes them through roc-automation’s
` actions/sbom-sidecars `, from the reviewed ` sbom/composition.json `. The tool
checks that file against ` Blueprint.roc ` and ` Blueprint.lock `: every input of
the environment is either a component or excluded with a reason. It lists
every member of the archive by digest and binds the documents to the bundle’s
final bytes. The components are the platform itself and the libFuzzer, Zig
runtime, musl and LLVM runtime code in the native inputs; Zig is a build
dependency. What the documents do not establish is stated as incomplete
coverage, not left out.
The build pins the Zig release and the SHA-256 of the ` libfuzzer-sys ` source
archive that ` scripts/native-deps.json ` names. Both targets are compiled against the headers Zig bundles; the
build reads no installed Apple SDK. The libraries are byte-reproducible, so
anyone with that Zig release can rebuild them from the recorded source revision
and compare digests. Native-library archives and the
final platform bundle are separate published and attested subjects, with
independent versions.
### Verify a release
Download a bundle from the GitHub release, then run:
``` console
gh attestation verify roc-fuzz-.tar.zst \
--repo lukewilliamboswell/roc-fuzz
```
Verification checks the bundle digest, Sigstore signature, certificate
and repository identity, and transparency information. This project does
not claim a particular SLSA level; the attestation establishes the
workflow and revision that produced the released bundle.
For offline or archived verification, download the matching
` .intoto.jsonl ` asset and pass it to GitHub CLI:
``` console
gh attestation verify roc-fuzz-.tar.zst \
--bundle roc-fuzz-.tar.zst.intoto.jsonl \
--repo lukewilliamboswell/roc-fuzz
```
The SBOM documents are attested with the bundle. Each names the bundle’s
SHA-256, which must equal the SHA-256 of the downloaded file.
## Understand roc-fuzz internals and advanced features
This chapter explains the runtime model, reusable generator dispatch,
allocation assertions, and the low-level libFuzzer escape hatch. Application
authors do not need this material for the normal target workflow.
### Understand the in-process model
The embedded libFuzzer calls a target thousands of times in the same
process. It does not start a clean executable for every generated input.
This is what makes it fast. Roc’s purity means target calls cannot leak
application state or background work into later calls, and determinism
makes saved inputs reliable with the same target and generator.
A reliable target still needs all of the following properties:
- Invalid or unusual inputs return ordinary values such as ` Err `, rather
than triggering an intentional assertion or process exit.
- One input normally finishes in less than 10 milliseconds.
- Inputs and allocations have sensible upper bounds.
- Every input terminates.
The runner is a poor fit when one test takes a significant fraction of a
second, can grow without a practical memory bound, or is expected to
terminate the process for invalid input. If a crash, timeout, or memory
blow-up would be a bug, it remains a useful fuzzing failure.
A saved entropy input produces the same typed value and result when the
target and generator are unchanged. Encoded examples use the binary parser and
the same property instead. Keep the executable alongside
important failure artifacts when exact historical reproduction matters.
### Understand generator dispatch
` generator_for ` uses Roc’s ordinary static dispatch. It is a method
associated with the target’s input type, not a runtime interface,
registry, or callback table. Package APIs can require their own methods
with ` where ` clauses, so the method does not need to be a
compiler-defined name.
The pattern deliberately mirrors ` Json.parser_for `:
- ` Json.parse ` requires its result type to have a compatible
` parser_for ` method and asks that type for a JSON parser.
- ` Fuzz.target ` requires its input type to have a compatible
` generator_for ` method and asks that type for a fuzz generator.
- The checker resolves both to a specific method implementation at
compile time. The compiled call has no dynamic-dispatch overhead.
Conceptually, ` Fuzz.target ` has this requirement:
``` roc
target : { name : Str, examples : List(Example(a)), test : a -> Outcome, show : a -> Str } -> Target
where [a.generator_for : FuzzEncoding -> Generator(a)]
```
` FuzzEncoding ` currently contains the ` Default ` policy. Passing a policy
value follows the same shape as ` JsonEncoding ` and leaves room for
additional generation policies without changing the method pattern.
Use ` Fuzz.target ` when generation is part of a reusable named input
type. Use ` Fuzz.target_with ` with an explicit ` Generator(a) ` for a local
structural input or when choosing between generators for the same type
in different targets.
### Measure allocations and validate traces
Every property callback has an automatic ` $property ` region. Decoding and
entropy generation finish before that region starts. Named regions use one
native engine:
``` roc
Fuzz.measure! : Str, (() => a) => { value : a, stats : Fuzz.Stats }
Fuzz.start_measure! : Str => Fuzz.Measurement
Fuzz.Measurement.finish! : Fuzz.Measurement => Fuzz.Stats
measured = Fuzz.measure!("decode", || decode(input))
check_result(measured.value)
if measured.stats.reallocs != 0 { crash "decode unexpectedly reallocated" }
window = Fuzz.start_measure!("manual")
value = build(input)
stats = window.finish!()
use(value, stats)
```
Finish manual windows exactly once, in nesting order. Invalid handles, abandoned
windows, instrumentation limits and arithmetic overflow fail explicitly. The
returned value is still alive at the measurement boundary. Nested windows are
inclusive: a parent contains its children’s allocator operations.
The platform transfers measured results through an opaque ownership container
before finishing the region. This keeps every returned field alive even when
the caller reads only ` stats ` or discards the result. The private container is
measurement bookkeeping and is excluded from allocator statistics and explicit
counters in every enclosing region. Ordinary application ` Box.box ` allocations
remain counted. The invocation-wide leak check includes both kinds of storage.
#### Understand the statistics
- **` allocs `, ` reallocs `, ` deallocs `**: Separate Roc allocator hook counts. A realloc
counts once as a realloc; its internal allocation and release are not counted
again. ` allocs + reallocs ` gives the number of allocation requests.
- **` requested_bytes `**: Sum of each allocation’s requested length and each realloc’s
full new length. This includes Roc’s own allocation headers and excludes the
host’s bookkeeping header. It measures traffic, rather than RSS or copied bytes.
- **` entry_live_allocs `, ` entry_live_bytes `**: Roc allocations already live at entry,
including input storage. Allocations made before a window contribute its entry
and peak memory, rather than its allocation count.
- **` exit_live_allocs `, ` exit_live_bytes `**: Memory alive when the window finishes.
This can include a returned value and is not itself a leak.
- **` peak_live_bytes `**: Actual maximum live requested bytes during the window,
including entry-live memory and simultaneous old/new blocks during realloc.
The broader invocation leak check runs after the complete target adapter returns.
It includes input preparation and teardown, which the property region excludes.
` --no-detect-leaks ` disables that check. Metrics remain separate from trace rules
and from diagnostic capture completeness.
Ordinary Roc assertions can inspect any statistic. The allocation assertion
helpers ` expect_allocs_at_most! ` and ` expect_allocs_at_least! ` take zero-argument
callbacks (` || operation(input) `) and count allocations plus reallocations.
Low-level process counters remain available for tightly focused assertions;
assert on differences, never their absolute process-wide totals.
#### Declare logical trace rules
``` roc
Fuzz.trace_rules!([
Before("decode", "checked"),
Count("checked", 1),
Forbidden("fallback"),
])
measured = Fuzz.measure!("decode", || decode(input))
check_result(measured.value)
Fuzz.mark!("checked")
```
Register rules before the region’s first event. Events are direct marks and
completed immediate-child regions. ` Before(a, b) ` requires both events and every
` a ` before every ` b `; ` Count(event, n) ` requires exactly ` n `; ` Forbidden(event) `
requires none. A grandchild completion is an event of its immediate parent.
Rules are evaluated online even after diagnostic capture fills. Detailed replay
exports ordered events, explicit gaps and unfinished regions. Parent identifiers
and occurrence numbers distinguish repeated names. Neither traces nor assertions
contain timestamps. Capture truncation does not turn an invalid trace into a
passing one and does not invalidate completed allocation statistics.
#### Accept an exact allocation baseline
Declare examples with names, revisions and ` make: || value `, alongside saved
artifacts. ` TARGET check ` runs each once in a fresh process. ` TARGET accept ` runs
the same complete inventory once and atomically accepts valid observations.
No warm-up or statistical repetition is needed for these deterministic metrics.
The default file is ` _baseline.json ` in the working
directory: a target named ` my_target ` uses ` my_target_baseline.json `. Override it
with ` --baseline=PATH `. Entries are separate by host target and ` --profile=NAME `.
An absent entry reports ` unbaselined ` and permits normal execution.
Every example name, revision, exact input SHA-256, region inventory, metric
version and statistic must match. Reductions and increases both need explicit
acceptance. Failed or rejected examples, incomplete observations and failed
baseline writes preserve the existing accepted file. Check and acceptance never
repeat examples to produce a more convenient result.
#### Binary inputs and witness identity
Structural values use the platform-owned binary format through Roc’s existing
` encoder_for ` and ` parser_for ` derivation. Nominal inputs opt in with both methods.
` Fuzz.encode(value) ` creates a versioned encoded-value input; ` Fuzz.entropy(bytes) `
frames generator entropy. Source examples are encoded into actual mutable seeds.
Host preparation copies bytes and releases temporary Roc values before running
an input. Encoding does not invert arbitrary ` Fuzz.map ` functions.
The frame is ` RFZ `, version byte ` 1 `, form byte (` 0 ` entropy, ` 1 ` value), then
payload. Bare historical bytes without the reserved ` RFZ ` prefix remain entropy.
Artifacts whose interpretation changes across platform versions need their
original executable; cross-version corpus migration is outside this API.
Unsupported versions, malformed
values and exceeded decoding budgets reject. The parser allows at most 1 MiB,
4096 value nodes and 128 total containers, validating counts before allocation.
Generation chooses bias. Preconditions or validated types define validity for
both forms; a range enforced only by a generator cannot protect encoded values.
Logical encoding is not a storage snapshot. Preserve original entropy when
aliasing, capacity or slices are part of a witness. ` retain ` copies original
bytes, then prints a saved declaration and a decoded representative.
Float encoding follows Roc’s floating-point value semantics. Do not rely on
NaN payloads surviving compiler operations; use integer bit inputs for
payload-sensitive properties. With the October 27 compiler, dictionary keys
that are structural records or tuples can crash compiler specialization during
codec derivation. ` --specialize=no ` avoids that compiler limitation; ordinary
scalar and tag keys work with the default backend.
### Use the libFuzzer interface
` TARGET raw [LIBFUZZER_ARG...] ` passes arguments directly to the
embedded libFuzzer CLI. Run this for its complete option list:
``` sh
TARGET raw -help=1
```
Prefer ` run `, ` show `, ` replay `, ` minimize `, and ` reduce-corpus ` for the
normal workflow. Low-level flags such as ` -max_total_time ` and
` -max_len ` belong behind ` raw `; ` run ` accepts the stable friendly
options listed above. ` raw ` is an escape hatch for libFuzzer features
the friendly command layer does not yet expose directly.
### Further reading
- [LLVM libFuzzer documentation](https://llvm.org/docs/LibFuzzer.html)
covers the in-process execution model, options, corpus behavior, output,
and FAQ.
## Contributing to roc-fuzz
The platform has four layers:
- ` platform/Fuzz.roc ` defines typed generators, record-builder
composition, and statically dispatched target construction.
- ` platform/Target.roc ` erases the app’s input type behind closures.
- ` platform/main.roc ` exposes the byte-oriented native boundary and
declares the x64-musl and Apple Silicon macOS targets.
- ` src/main.zig ` adapts generated Roc ABI calls to libFuzzer and
translates the friendly runner commands.
` src/roc_platform_abi.zig ` is generated from Roc’s ` ZigGlue.roc `.
### Toolchain
roc-fuzz follows the
[roc-automation manual](https://lukewilliamboswell.github.io/roc-automation/).
Two Roc compilers are involved, and they are not interchangeable:
- ` roc-stable ` runs the scripts under ` scripts/ ` and nothing else. It is one
nightly release, named in ` Blueprint.roc ` and repeated in the ` roc ` field of
every script header. It changes only through a reviewed pull request.
- ` roc ` checks, tests, builds and bundles the platform and every application
that depends on it. It is the latest published nightly and is not pinned
anywhere: the platform, the examples and the test targets carry no ` roc `
field. Users install the newest compiler, so that is the one the platform has
to work with.
` Blueprint.roc ` declares the tooling environment and ` Blueprint.lock ` pins it:
` roc-stable `, the Roc bundles the scripts depend on, and the programs they
start. [roc-blueprint](https://github.com/lukewilliamboswell/roc-blueprint)
realises that environment with Nix, the same way on a contributor’s machine and
on every CI runner. To work on roc-fuzz you need:
- [Nix](https://nixos.org/download);
- the ` blueprint ` command, from the roc-blueprint release that the platform URL
in ` Blueprint.roc ` names: ` nix profile install github:lukewilliamboswell/roc-blueprint/VERSION `,
or the release’s prebuilt binary checked against its published SHA-256;
- a current Roc nightly installed as ` roc `.
The environment supplies the Zig release that ` Blueprint.roc ` names, from
[zig-overlay](https://github.com/mitchellh/zig-overlay), to build the host and its
linker inputs and to format the host bindings. ` scripts/native-deps.json ` names
the same release, and the supply-chain check holds the two together.
``` sh
blueprint tasks # list the declared tasks
blueprint run check # everything a pull request must pass that needs no host
blueprint shell # enter the environment and run scripts/*.roc directly
```
Run ` blueprint ` from an ordinary shell. Inside the environment ` roc-stable ` is
the pinned compiler and your own ` roc ` remains visible. Arguments to a
task follow ` -- `, for example ` blueprint run build-platform -- --target x64musl `.
Linker inputs have an independent, content-addressed release cycle;
` libhost.a ` is deliberately outside that cycle and is built with each platform
update. This separation means a host or Roc source change does not rebuild large,
stable runtime inputs. The builder uses ` zig ar `.
Repository examples use relative paths to the working-tree platform. Each
release provides ` roc-fuzz-examples-VERSION.zip ` with complete applications,
supporting modules, the cases that describe them, and that release’s exact
platform URL. Nothing else in an archived example differs from the repository.
The ` Nightly compatibility ` workflow runs the latest release’s archive, exactly
as published, on the latest nightly every day.
#### Maintain the tooling
Nothing updates the tooling on a schedule. Each of these is a pull request a
maintainer opens:
- **Update the lock**: Run ` blueprint update `, or ` blueprint update INPUT ` to move
one input, then ` blueprint run check `.
- **Change the tooling compiler**: Change the tag in ` Command("roc-stable", ...) ` in
` Blueprint.roc ` and the ` roc ` field of every script to the same tag. The check
script fails until they match. Run ` blueprint update roc ` if the locked overlay
predates that nightly.
- **Upgrade Blueprint**: Change the platform URL in the ` Blueprint.roc ` header, run
` blueprint check ` and then a full ` blueprint update `, and review the new lock.
No workflow names the version: ` actions/setup-blueprint ` reads it from
` Blueprint.roc ` and verifies the binary against the checksums roc-automation
records, so a release it does not record yet needs that action repinned first.
- **Update a script dependency**: Change the release URL in each script header and
under ` RocPackages ` in ` Blueprint.roc `, adding any new dependency of that
dependency (` roc-stable deps scripts/check_all.roc ` lists them), then run
` blueprint update `.
- **Keep the SBOM composition current**: ` sbom/composition.json ` gives every input
` Blueprint.roc ` declares (each source, tool, fetched program, script package
and task) a disposition. A new task or tool needs an entry there too, or the
release dry run’s SBOM step refuses the composition. A change to what the
native inputs contain, such as a new library or a new Zig release, changes its
components.
- **Regenerate the host bindings**: See [Host bindings](#host-bindings).
### Repository scripting convention
Repository-owned automation is written in Roc. Put executable entry points in
` scripts/*.roc ` and reusable logic in ` scripts/src/*.roc `. Use the existing shared
modules for commands, paths, hashing, source edits, archives, and release metadata.
Do not add Python scripts, shell launchers, or build-before-run wrappers to work
around compiler behavior. Keep workflow YAML limited to job orchestration and
small environment/output plumbing; substantive automation belongs in Roc.
Every entry point starts with the ` #!/usr/bin/env roc-stable ` shebang, is
committed as executable, and is declared as a ` Task ` in ` Blueprint.roc `. CI
calls tasks, never script paths, so a workflow and a contributor run the same
command. A script starts another script through ` Script.roc_stable! ` and runs
package work through ` Script.roc! `, which is the ` roc ` on ` PATH `. Never use
` roc-stable ` to check, test or bundle the platform. A successful script must
return zero, and a failed script must remain a failure.
A script references every dependency by immutable release URL, and the same URL
is listed under ` RocPackages ` in ` Blueprint.roc `. ` scripts/check_supply_chain.roc `
holds each header to ` Blueprint.roc `: the shebang, the ` roc ` field, and every
bundle URL. It also fails when a Roc file outside ` scripts/ ` carries a ` roc `
field, and when a workflow references an action by anything but a full commit
SHA.
Roc scripts may invoke standard tools such as Git, curl, tar, zip, and unzip via
argument lists, without shell evaluation. Roc owns selection, validation, and
control flow. Declare a program a script starts under ` Tools ` in ` Blueprint.roc `.
Pinned external tools and shared actions remain external dependencies;
for example, the shared documentation builder currently uses Python internally.
That does not justify adding repository-owned Python orchestration.
Pure regression checks belong in shared-module ` expect `s imported by
` scripts/tooling_tests.roc `. Filesystem and process checks belong in
` scripts/tooling_integration_tests.roc `; both run from ` scripts/check_all.roc `.
### Generate platform inputs
Generate the current host’s ignored inputs with:
``` sh
blueprint run build-platform
```
Pass ` -- --target x64musl ` or ` -- --target arm64mac ` explicitly in automation.
The default restores the archives selected by ` native-libraries.lock.json `,
rehashes their manifest and target archive even on a cache hit, validates the
complete archive inventory, builds the current host, and writes local
` SHA256SUMS `. The archive digest is the reviewed authorization for routine use;
the cache key only locates likely bytes. This is why cache hits are still hashed
and why a cache miss downloads the exact locked release rather than rebuilding.
It avoids repeat network traffic without allowing a poisoned or truncated cache
entry to become trusted. Routine restoration does not query GitHub’s attestation
service. Never commit generated archives, objects, or manifests.
The ` Linker inputs ` producer, ` link-inputs.yml `, runs on a pull request that
touches the recipe and on dispatch. One Linux job cross-compiles both targets,
and a job on each target runs every case against the candidate. It packages both targets without
` libhost.a `, tests each candidate through a fresh host and final bundle, emits a
canonical aggregate manifest, and attests that manifest and both archives. It has
no publication authority. Release new inputs for changes to their sources,
runtimes/toolchains, flags, targets, security fixes, or the macOS adapter compiled
into ` libfuzzer.a `. The macOS coverage shim stays with ` libhost.a `.
` Publish link inputs ` runs trusted controller code pinned to a reviewed full
` roc-automation ` commit. It dispatches the producer at the exact same-repository
PR head, verifies the workflow/ref/SHA, hashes and attestations, publishes an
immutable release named from the aggregate manifest hash, and adds only the lock
in a lease-guarded GitHub-signed commit. Candidate code cannot publish, while the
privileged job never executes candidate code. The PR-built release remains the
permanent dependency after merge; rebuilding it would break the evidence-to-byte
identity that the review admitted.
GitHub immutable releases must be enabled before the first publication. The
controller checks the resulting release state and refuses to adopt a lock when
the hosting policy does not enforce immutability; a content-derived name alone
cannot prevent an administrator from replacing assets.
Before adopting a release, review its producer changes, exact source revision,
input fingerprint, manifest and archive digests, sizes, target coverage, native
link tests, and workflow attestations. Merge with a merge commit so the attested
producer SHA and signed lock commit remain reachable. Ordinary pull requests
then know only the reviewed lock, cache, verifier, and target files; they
never dispatch or reproduce linker inputs.
The libraries are byte-reproducible: the same sources, Zig release and flags
give the same bytes on any machine, in any directory, at any time. Nothing is
compiled with debug information, which would record the absolute paths of the
Zig installation, its cache and the working directory, and archive members are
named by file name alone where Zig names them by cache path. The macOS target
is compiled against the headers Zig bundles, so it builds on Linux as well as on
macOS and reads no installed Apple SDK. Keep it that way: a change that makes
two builds of the same commit differ is a defect in the recipe.
A routine pull request never rebuilds the libraries. It hashes a short list of
files and compares the result with the ` input_fingerprint ` the lock records;
` LinkInputs.fingerprint_inputs ` is the list:
- ` scripts/native-deps.json `: the Zig release, the libFuzzer source, and the
recipe revision;
- the macOS libFuzzer adapter and the stand-in header it is compiled with; and
- the licence files packed beside the libraries.
No script, workflow or lock file is in it, so editing tooling never demands a
release. The recipe itself lives in scripts, and that is what the recipe
revision is for: raise ` recipe_revision ` when a change to the build alters a
library byte, such as a flag, a target or the list of libraries.
Reproducibility keeps that honest, and three checks guard it:
- The ` Linker-input recipe ` job of ` ci.yml ` is part of ` CI required `. It asks
whether the pull request touches the recipe, which is the list in
` LinkInputs.recipe_paths `: the build script, the target definitions, the
recipe file, the macOS sources, and ` Blueprint.roc ` and ` Blueprint.lock `,
which supply Zig. A routine pull request stops there. One that touches the
recipe rebuilds both targets and, while the lock claims to be current,
requires the result to be byte-identical to the locked release. A change that
alters a library without raising the revision fails, naming the files, and
cannot merge. So does a change that makes the build irreproducible.
- The ` Linker inputs ` producer makes the same comparison when it builds a
candidate, and runs every case against it on both targets.
- The supply-chain check fails when a path in ` LinkInputs.recipe_paths ` is
missing from the producer’s ` paths ` filter, so a file added to the recipe
cannot be left out of the trigger.
Run the comparison locally with:
``` sh
blueprint run build-platform -- --target x64musl --target arm64mac --libraries source --libraries-only
blueprint run link-inputs -- --operation check-source
```
When the fingerprint changes, or that comparison fails, use this sequence:
1. Stage the change in a same-repository PR, raising ` recipe_revision ` if the
build recipe changed. Its ` Platform bundle ` job fails with a stale-lock error
until the new lock is adopted.
2. To inspect a nonpublishing candidate, dispatch ` Linker inputs ` on the PR branch
with ` release_candidate=true ` and ` expected_sha ` set to its exact current head.
For example, run ` gh workflow run link-inputs.yml --ref BRANCH -f release_candidate=true -f expected_sha=SHA `, replacing ` BRANCH ` and ` SHA `.
Review both target tests, the manifest, archive hashes, and attestations. This
run does not create a GitHub release or edit the PR.
3. From ` trunk `, dispatch ` Publish link inputs ` with ` pull-request ` set to that
PR number: ` gh workflow run publish-link-inputs.yml --ref trunk -f pull-request=NUMBER `.
To recover an interrupted publication, also pass ` producer-run ` with the ID of
the successful producer run for the current head.
The trusted publisher reruns the producer at the exact PR head,
publishes the immutable linker-input release, and appends a GitHub-signed
` native-libraries.lock.json `-only commit to the PR. Do not dispatch it from
the PR branch or rebuild the inputs after merging.
4. Review that lock commit and revalidate the updated PR. After its required
checks pass, merge the PR with a merge commit. Publish a new Roc platform
bundle later through the separate ` Release ` workflow, using the adopted lock.
The optional preview in step 2 does not replace the producer run performed by
the trusted publisher in step 3. The publisher is the release operation; stop
before that dispatch when only candidate validation is authorized.
The fully static build excludes ` FuzzerInterceptors.cpp `. Its wrappers
locate libc functions through ` dlsym `, which is not usable in the static
musl executable. The standard libFuzzer scheduler, mutators, corpus
engine, crash handling, minimizer, and merge engine remain unchanged.
Roc’s compare and switch instrumentation still supplies value feedback.
The macOS target includes ` FuzzerInterceptors.cpp `: its ` dlsym ` lookups
work with macOS’s dynamic ` libSystem `. Its deployment target is macOS
11.0, and its only allowed dynamic dependencies are system libraries.
` src/macos_sancov.c ` supplies the public TLS slot required by Roc’s
macOS stack-depth coverage instrumentation.
` src/macos_fuzzer_ext_functions.cpp ` binds the standalone runner hooks
directly because upstream libFuzzer’s Darwin ` dlsym ` lookup requires
` -export_dynamic `, which Roc does not pass. Both files are
repository-owned platform glue covered by the project license; the
native build, ` show `/replay, fuzz, failure-artifact, and packaged-bundle
smoke tests exercise them on Apple Silicon.
### Host bindings
The host is compiled against ` src/roc_platform_abi.zig `, which ` roc glue `
generates from ` platform/main.roc `. The file is committed. The host build reads
it and never generates it, so the host builds the same way on every runner and
a change to the interface is reviewed as a diff.
The compiler generates the bindings, so a new nightly can change them when
nothing in this repository changed. The ` Host bindings ` job of ` ci.yml `
regenerates them with the nightly the run resolved and fails when the committed
file differs. Run the same check locally with:
``` sh
blueprint run check-glue
```
The glue specification comes from the source archive attached to that nightly’s
release, verified against the SHA-256 that GitHub records for the asset. For a
compiler built from source, name its checkout with ` -- --roc-source DIR `.
When the check fails and the change did not touch ` platform/ `, the compiler
changed the interface. Upgrading is a manual task, because a changed layout or
signature usually needs a matching host change:
1. Run ` blueprint run check-glue -- --update ` with the latest nightly as ` roc `.
2. Review the diff and change the host where a layout or signature changed.
3. Build the host for every target and run the targets.
4. Open one pull request with the bindings and the host change.
### Validation
Every bug fix and significant new behavior should include an automated test at
the narrowest useful layer. Changes to public behavior also update the relevant
examples, interface documentation, or report schema. A pull request should
explain the behavior changed and the validation performed.
The scripts use whichever ` roc ` comes first on ` PATH `. To validate with a
compiler under development, put it there:
``` sh
PATH=/path/to/roc/zig-out/bin:$PATH blueprint run check
blueprint run test-targets -- --operation check --verbose
blueprint run test-targets -- --operation build --verbose
blueprint run test-targets -- --operation seed
blueprint run test-targets -- --operation fuzz --max-total-time 2
```
` blueprint run check ` needs no native inputs: it formats, tests the scripts,
runs the pin and supply-chain checks, the native allocation/trace engine and
supervisor lifecycle suite, type-checks the platform and runs its
` expect ` tests from ` platform/main.roc `, and type-checks every case in
` tests/targets.json `. After native inputs are prepared, ` test-targets `
provides the focused operations shown above.
` test-targets ` runs each stage of the operation (check, test, build, seed,
measure, quality, fuzz) over every selected case and prints one line per step as it
finishes, with its position, ` PASS `, ` FAIL ` or ` SKIP `, and its time in
milliseconds. A failed step does not stop the run: the case’s later stages are
skipped and every other case carries on. The run ends with each failure and the
end of its output, each skip with its note, the counts, the total time and the
slowest steps, and exits non-zero if anything failed. A command’s output is
shown only when its step fails; ` --verbose ` shows all of it as it runs. Colour
is used only on a terminal and never when ` NO_COLOR ` is set. ` --target NAME `
and ` --path-prefix PREFIX ` narrow the selection.
Checked-in examples and test targets use the local platform. The nightly
compatibility workflow uses the latest release’s own archived examples and
immutable bundle. Validation
checks the exact example inventory, Roc formatting and types. After
generating the current host inputs, building creates every
self-contained executable and verifies a static x86-64 ELF on Linux or
an arm64 Mach-O with system-only dynamic dependencies on macOS. Seed
validation renders and replays each deterministic input. The fuzz
operation runs short campaigns and verifies that an intentional Roc
failure is saved byte-for-byte with follow-up commands.
Build the HTML manual, PDF, and generated API reference with:
``` sh
blueprint run docs
blueprint run site
```
` docs ` runs ` roc-docs `, the renderer ` actions/build-docs ` runs, at the
roc-automation tools release that ` Blueprint.roc ` names and ` Blueprint.lock `
pins by digest. It writes the working tree’s manual, PDF and API reference
under ` .docs-out/ `. ` site ` assembles what GitHub Pages serves: the latest
release’s documentation, restored from the archive that release published, with
the working tree’s under ` unreleased/ `.
The renderer does not run Mermaid. Each diagram’s PNG is prepared once, checked
in under ` docs/images/ ` with a provenance note saying how it was made, and
registered with its source’s digest in ` docs/prepared-diagrams.json `. Editing a
diagram’s source fails the build until its image is prepared again. The final API audit fails if an exposed module or public entry
has no rendered documentation. The documentation and release workflows run the
same build and audit.
Every case runs the ` check `, ` test `, ` build `, ` seed `, and ` fuzz `
stages by default. To switch off a case’s seed replay or fuzz run, set its
` skip_seed ` or ` skip_fuzz ` and add an entry to the ` skips ` list in
` tests/targets.json ` with the case name, the operation, a concrete reason and a
full issue URL. The supply-chain check rejects a skip without exactly one such
note, and a note left behind once the case runs again. The driver names every
skip and its reason at the start of each run. No case is skipped today.
A file under ` trophy-case/repros/ ` that can run is a case in
` tests/targets.json `. The one kept there is a deliberate type error that the
compiler once panicked on; ` blueprint run check ` requires ` roc check ` to report
it.
Four workflows each answer one question:
- **` ci.yml `**: Does this change work with today’s compiler? One job runs
` blueprint run check ` and audits the API reference. One job regenerates the
host bindings and compares them with the committed file. One builder
cross-compiles the host for both targets around the locked inputs and
assembles a single bundle, and a job on each
target downloads that bundle and runs every case against it. ` CI required `
aggregates them.
- **` release.yml `**: Do these exact bytes work? On a pull request it is a dry run
under the version ` 0.0.1-dryrun `: it builds the bundle, the examples archive,
the SBOM, the documentation archive and the release notes, and tests the
archive against the bundle on both targets. ` Release required ` aggregates it.
- **` nightly.yml `**: Does the latest release still work with today’s compiler? It
runs daily; see [Release bundle](#release-bundle).
- **` fuzz.yml `**: Has today’s compiler broken a builtin? Once a day, and only on
` trunk `, it fuzzes every builtin target in ` tests/targets.json ` on the latest
nightly for a few seconds each, about ten minutes in all. It runs
` blueprint run test-targets -- --operation fuzz --campaign-dir DIR --path-prefix tests/targets/builtins/ `. The corpus
under ` DIR/corpus/ ` carries over between runs through the Actions cache. Each
target’s failing inputs are kept under ` DIR/failures/ ` and uploaded as the
` fuzz-failures ` artifact. Every target runs before the job fails. Reproduce a
failure with the target’s ` replay ` command and the kept input, then report it
upstream if it is the compiler’s.
A new nightly can fail an untouched pull request. Compare with the latest
` Nightly compatibility ` run and with ` trunk ` before treating a failure as a
regression in the change.
Linux CI also rebuilds the checksum-pinned libFuzzer sources with
AddressSanitizer and runs a short native smoke campaign. This detects heap,
stack, use-after-free, and leak errors in the embedded C++ fuzzing engine on
every pull request, every ` trunk ` update, and before release publication. Run
the same check locally with
` blueprint run native-sanitizers `; it requires Clang,
` curl `, and ` tar `. The sanitizer executable is CI-only and does not change the
static release artifacts.
The Roc driver runs on Linux and Apple Silicon without platform skips. Scripts
execute directly with the fixed tooling compiler; compilation errors, warnings,
and application failures retain their normal Roc exit status.
Use ` --suite-root PATH ` to run an unpacked examples archive containing ` examples/ ` and
` tests/targets.json `. Each invocation has fresh temporary executables and, unless
` --campaign-dir ` names a directory to keep them in, fresh corpora.
Fuzz smoke tests pass corpus directories to libFuzzer; passing an individual seed
file would only replay it. Repository formatting is checked separately, not against
an older release’s frozen source.
Nightly compatibility selects the highest stable semantic-version platform
release, ignoring linker-input releases and release candidates. Both targets
consume the same selection and verify asset hashes. Releases through 0.4.1 use
examples from the exact tag commit, rebound to that release’s own bundle (the old
manual URL follow-up could occur after tagging). Newer releases must include the
examples archive. The compatibility artifact records the release, source commit,
compiler and asset identities used; a missing archive or failed download cannot
become a passing test. Archives up to 0.4.3 pin every example to the compiler of
their day, and the driver treats Roc’s version-mismatch warning as a failure, so
those archives pass only with the compiler they name. Later archives carry no
pin.
Every property callback has an automatic allocation observation. Source examples
run independently of the coverage corpus and can be accepted into an exact
baseline. Explicit named regions use ` Fuzz.measure! ` or manual measurement
windows; ordinary Roc assertions inspect their statistics. Existing builtin
regressions also use allocation assertion helpers around specific operations.
The ` quality ` stage exercises the real executable’s unbaselined, acceptance and
exact-match workflow for targets with declared examples. The native suite tests
crashes, deadlines, rejected inputs, baseline drift in both directions, original
shared evidence and atomic write failures.
Assert on a difference between two counter reads, never on a raw value:
the counter is process-wide and libFuzzer reuses one process across
millions of inputs. Only assert zero for a value that is uniquely owned
and pre-sized, because copy-on-write allocation is correct when a value
is aliased.
Separately, the runner checks after every input that the target freed
everything it allocated, and fails the input otherwise. Pass
` --no-detect-leaks ` to a run to turn that off.
The byte-oriented builtin targets live under ` tests/targets/builtins/ `
and use explicit byte generators and ` Fuzz.keep ` outcomes. Their original
byte-valued probe results keep aliases alive and are not rejection statuses.
They are regression tests for the compiler and its
builtins: ` tests/targets.json ` lists them beside the examples, pull-request CI
runs them, and a release’s examples archive leaves them out. The applications
under ` examples/ ` are an end-user gallery and should
prefer ` Fuzz.target `, typed generators, and the ` .Fuzz ` record builder.
A multi-file example must live in its own directory with ` main.roc ` as
its app root; the test driver enforces this so editor tooling can
discover the project naturally.
### Release bundle
Create a Roc platform bundle after generating both target input sets
with:
``` sh
blueprint run bundle -- --output-dir dist
```
` build_bundle.roc ` verifies every generated input checksum, then includes the
archives and license notices in
` roc bundle `. Test the resulting bundle from an external
target with:
``` sh
blueprint run test-bundle -- dist/.tar.zst
```
#### What a release contains
One immutable GitHub release per version holds:
| Asset | Purpose |
| --- | --- |
| The bundle, under the filename ` roc bundle ` emits | What users depend on. The filename is its content hash. |
| ` roc-fuzz-examples-VERSION.zip ` | The examples pinned to this release’s bundle, with the cases that describe them. |
| ` roc-fuzz-docs-VERSION.zip ` | The manual and API reference for this version. The documentation site restores the latest release’s copy. |
| ` roc-fuzz-VERSION.pdf ` | The manual. |
| ` .spdx.json `, ` .cdx.json ` | The bundle’s SBOM, in SPDX and CycloneDX, written from ` sbom/composition.json `. |
| ` .sbom-manifest.json ` | The evidence manifest binding the bundle and its SBOM documents by digest. |
| ` *.intoto.jsonl ` | The exported build-provenance attestations for the bundle and the examples archive. |
#### Write the release notes
Before dispatching a release, commit a non-empty ` docs/releases/X.Y.Z.adoc `, or
` docs/releases/X.Y.Z-rcN.adoc ` for a release candidate, and add an ` include `
for it at the top of ` docs/releases.adoc `. The first heading
must be ` = roc-fuzz VERSION `. Release notes are reviewed as documentation and
should explain the release to users: lead with the theme, show notable features
and compatibility changes, name the supported hosts, link to
verification material, and give upgrade or feedback guidance.
Use ` {platform-url} ` where the copyable application header needs the
content-addressed bundle URL, and ` {manual-url} ` for the release’s PDF. Do not
put a ` roc ` field in that header. The build job fills both in after it has made
the bundle, translates the note’s headings and source blocks to GitHub Markdown
with ` scripts/render_release_notes.roc `, and appends links to the bundle’s SBOM
documents and, under **Compilers**, the output of ` roc version ` and
` roc-stable version `. That record is the release’s statement of what it was
built and tested with. Notes that never use ` {platform-url} `, or that leave a
placeholder unresolved, fail the build.
The committed file keeps its placeholders and is never rewritten after the
release. A pull request renders the highest version under ` docs/releases/ `, so
a mistake in the notes being written fails on the pull request that makes it.
While that version is still the latest release, there is nothing to render.
#### Publish a release
From the repository’s **Actions** tab, run ` Release ` on ` trunk ` with an ` X.Y.Z `
version. The workflow:
1. rejects a version whose tag or release exists, requires ` trunk `, and requires
the notes file;
2. runs ` blueprint run check ` and the sanitizer check on that commit;
3. cross-compiles the host for both targets, bundles once, and checks the requested
version against the public API with ` roc bump --expect `, so a breaking change
cannot go out as a patch;
4. builds the examples archive with the real release URL, the SBOM, the
documentation archive, the manual and the rendered notes;
5. on each target, serves the bundle on a loopback address and runs the archived
examples against it, with an empty package cache;
6. attests the tested bundle, its SBOM documents and the examples archive; and
7. publishes from a job that checks out nothing and runs no repository script.
roc-automation’s ` actions/publish-release ` stages a draft release at the
tested commit, uploads every asset, downloads each one back and compares it
with the tested file, and only then publishes.
After a final release the workflow rebuilds and deploys the documentation site,
which leads with the latest release.
The publish job runs in the ` release ` environment. Use ` roc bump ` before
choosing a version:
``` sh
roc bump --old platform/main.roc
```
A release candidate is the same dispatch with an ` X.Y.Z-rcN ` version. It goes
through every step above, holds the same assets, and is published as a
prerelease that is never the latest release. Nightly compatibility and the
documentation site ignore it. Fix what a candidate turns up on ` trunk ` and
publish ` -rc2 `; a candidate is immutable like any other release.
#### When a new compiler breaks the latest release
` Nightly compatibility ` runs every day. It selects the latest Roc nightly and
the latest stable platform release, and runs that release’s examples archive,
unchanged, on both targets with ` trunk `’s scripts. Its report job keeps one
issue, titled "The latest release fails on the latest Roc nightly": opened by
the first failure, commented on by each later one, and closed by the next pass.
A run in which setup failed before any example ran fails without touching the
issue.
A release is immutable, so the fix is always a patch release that works with
the new compiler. Reproduce a past combination by dispatching the workflow with
a ` nightly-tag ` and a ` release-version `.
#### Recover an interrupted publication
Inspect exactly what exists before retrying. An unfinished draft can be
completed from the same run’s artifacts: upload what is missing, verify the
complete inventory, then publish. A published release that is incomplete cannot
be repaired; publish a new version. Changed bytes are always a new version.
Never replace an existing tag or asset, and never rebuild a release from a
moving branch.
### Design constraints
Static dispatch happens while ` Fuzz.target ` is specialized for the app’s
input type. ` Target ` then erases that type, preventing the native ABI
from depending on every app record or tag-union layout.
The platform supports x64-musl and Apple Silicon macOS. Keep native
calls behind the generated bindings, so that a compiler ABI change is caught by
the bindings check and by compilation.
### Shared examples archive
Starting with 0.4.5, the examples ZIP has one root named
` roc-fuzz-examples-VERSION `, containing ` README.md `, ` spec.json `, and ` examples/ `.
The specification derives from the published examples in ` tests/targets.json `.
Each app requests ` build_args: ["--fuzz"] ` and records replay and one-second
seeded smoke cases. Cases keep exact stdout and exit expectations while
` stderr_match: "ignore" ` allows libFuzzer statistics to vary; raw stderr remains
available for diagnosis. Seeds and local harness metadata live under
` examples/.roc-fuzz/ `. Historical flat archives remain readable.
The release test jobs run the exact ZIP and bundle through the reviewed shared
` release-package test-archive ` utility before publication. Its loopback bundle
replacement preserves the archived sources and verifies the candidate bytes.
The daily ` Nightly compatibility ` workflow uses the shared single-job caller
on Linux x86-64 and Apple Silicon. Empty selectors test the latest final release
against the latest nightly and report through the shared compatibility gate.
Diagnostic selector overrides test historical pairs and do not update the issue
about the latest pair. The separate builtin fuzz campaign retains its own
schedule and corpus; its result does not determine release compatibility.
## Project information
### Support, feedback, and releases
Use [GitHub issues](https://github.com/lukewilliamboswell/roc-fuzz/issues) for
reproducible bugs and enhancement requests. Include the roc-fuzz release, Roc
compiler version, operating system and architecture, target source or a minimal
reproduction, and the relevant command output. Proposed changes use pull
requests and follow the [contributor guide](#contributing-to-roc-fuzz).
Suspected vulnerabilities follow the
[private
security reporting process](https://github.com/lukewilliamboswell/roc-fuzz/blob/trunk/SECURITY.md) and must not be disclosed in a public issue.
Published bundles, PDF manuals, documentation archives, checksums, provenance,
and human-readable release notes are available from
[GitHub Releases](https://github.com/lukewilliamboswell/roc-fuzz/releases).
### Origins and acknowledgments
The foundational and exploratory work for roc-fuzz was created by
[Brendan Hansknecht](https://github.com/bhansconnect). Brendan established the
original Roc fuzzing integration, arbitrary input machinery, quality-target
suite, and the early
[trophy
case](https://github.com/lukewilliamboswell/roc-fuzz/blob/trunk/trophy-case/README.md) demonstrating the bugs this approach could find.
The platform in this repository extends Brendan’s experiment into a modern Roc
workflow with typed generators, reusable target construction, and a
self-contained executable built directly with ` roc build --fuzz `. This work
would not exist without the foundation he developed.
## Release notes
Each release has its notes in ` docs/releases/ `, newest first. They are the
source of the text on that release’s GitHub page.
A release’s bundle URL holds the content hash of the bundle, which does not
exist until the release builds it. The notes therefore write ` {platform-url} `
where an application header names the platform, and the release workflow fills
in the real URL when it publishes. Copy the URL from the
[release page](https://github.com/lukewilliamboswell/roc-fuzz/releases).
### roc-fuzz 0.5.0
roc-fuzz now combines pure Roc correctness, named source examples, exact
allocation baselines, logical traces and coverage-guided exploration in one
self-contained target executable.
Use the platform dependency attached to this release:
``` roc
app [target] { fuzz: platform "{platform-url}" }
```
#### Examples and reproducible inputs
Every target configuration declares ` examples `, using ` [] ` when it has none.
` Fuzz.example({ name, revision, make: || value }) ` creates a typed seed;
` Fuzz.saved({ name, revision, bytes }) ` preserves an original witness. Each
example runs once in a fresh process, independently of coverage retention.
Typed values become actual mutable corpus seeds through the platform’s binary
format, derived using Roc’s existing encoder/parser protocols. Nominal input
types opt in with ` encoder_for : _ ` and ` parser_for : _ `.
Bare entropy witnesses keep their generator interpretation. Logical encoding
is not a snapshot of allocation capacity, sharing or slices. Keep original
bytes for storage-sensitive findings. Preconditions or validated types must
validate both decoded and generated values; generator bounds express bias.
#### Allocation and logical trace evidence
` Fuzz.measure!("region", || operation(input)) ` returns ` { value, stats } ` and
records a named observation. ` start_measure! ` and ` Measurement.finish! ` use the
same scope engine. Statistics distinguish allocations, reallocations,
deallocations, requested bytes, entry/exit live memory and actual peak live
bytes, including old/new overlap during reallocations. The property itself is
measured after generation/decoding and before adapter teardown.
Trace rules support ` Before `, exact ` Count ` and ` Forbidden ` over direct marks
and completed immediate-child regions. Online validation continues after
bounded capture truncates. Detailed replay identifies repeated paths, gaps and
unfinished regions. Timing measurements and timing assertions are excluded.
The earlier measurement/report API is replaced by named measurements. All
zero-argument callbacks use ` || ` and signatures such as ` () => a `. Byte target
adapters now forward nonzero return values as rejection.
#### One supervised workflow
` TARGET ` or ` TARGET run ` checks examples and baselines, then explores and writes
one report. Defaults remain 60 seconds of exploration, with a 10-second
invocation deadline and leak checks. Failed examples skip exploration.
` check `, ` explore `, ` accept `, ` examples `, ` show `, ` replay `, ` measure `, ` retain `,
` minimize `, ` reduce-corpus ` and the advanced ` raw ` boundary are embedded commands.
CI uses ` run --report-dir=... ` instead of the earlier ` ci ` command.
Baselines default to ` _baseline.json ` in the working
directory, with ` --baseline=PATH ` and ` --profile=NAME ` overrides. Every inventory,
revision, actual input digest, metric version and statistic must match. Both
increases and reductions require explicit acceptance. A missing entry is
` unbaselined `. Acceptance is separate from checking and preserves previous data
when the inventory fails or a write cannot complete.
Reports distinguish outcomes from allocation/trace evidence and capture
completeness. Exploration counts attempted, accepted and rejected inputs; zero
accepted inputs is unsuccessful and inconclusive. Original bytes and bounded
trace breadcrumbs remain in supervisor-owned shared storage even when a child
crashes. Diagnostic replay remains separate evidence.
#### Upgrade and verification
The provider-free Blueprint template uses pinned Roc downloads and ordinary
tasks and workflows. Consumers need no Nix, Zig, Python, Git executable or
standalone coordinator. The maintainer environment still uses Nix. Acceptance
is an explicit maintainer task, never part of verification.
Supported hosts are Linux x86-64 with musl and Apple Silicon macOS 11 or newer.
This release is validated with Roc ` nightly-2026-10-27-de79a0f `. Float encoding
follows Roc value semantics; use integer bits for NaN-payload properties. The
current compiler’s default specialization can fail on binary dictionaries with
structural record/tuple keys; ` --specialize=no ` avoids that compiler limitation.
See the [manual](https://lukewilliamboswell.github.io/roc-fuzz/), the generated API
reference, and the release’s checksums, SBOM and provenance. Existing artifacts
are preserved; there is no automatic corpus migration or cross-version baseline
comparison. Build a target, review its declared inventory, explicitly accept its
allocation observations, and commit the resulting baseline for each host/profile.
### roc-fuzz 0.4.5
Published examples now use the shared roc-automation suite format. Each archive
has one named root, an explicit specification, and seeds for replay and a
one-second smoke run of every released example. The shared runner builds them
with ` roc build --fuzz `, checks stdout and exit codes, and retains libFuzzer’s
variable stderr as diagnostics. Existing flat examples archives remain readable
by repository tooling. The platform API is unchanged.
Use this platform dependency in an application header:
``` roc
app [target] { fuzz: platform "{platform-url}" }
```
The discarded-function-result lifetime repro now expects success on the current
nightly. It retains its allocation-balance assertion: compiler fix
[roc\#12652](https://github.com/roc-lang/roc/pull/12652) releases the unused result
before the next effect. The same repro exits 77 on the October 23 compiler and
0 on the October 27 compiler against the same platform bundle.
### roc-fuzz 0.4.4
This release makes the published platform work with current Roc nightlies
again, and lets a package hold allocation ratchets on roc-fuzz without a test
platform of its own. Release 0.4.3 fails the daily compatibility check on
current nightlies, and a release cannot change, so 0.4.4 replaces it. Every
change to the API is an addition; existing targets build and run unchanged.
#### Allocation ratchets
` Fuzz.measure_alloc_counts! ` counts a region’s allocations, reallocations and
deallocations separately, with the bytes it requested, where
` Fuzz.measure_allocs! ` counts allocations and reallocations together.
` Fuzz.realloc_count! `, ` Fuzz.dealloc_count! ` and ` Fuzz.requested_bytes! `
expose the counters underneath.
` Fuzz.report_allocs! ` measures a region and reports it under a label. The new
` TARGET measure [INPUT] ` command runs the target once, outside libFuzzer, and
prints one versioned JSON line per report:
``` json
{"format":"roc-fuzz-allocs/1","label":"grow a unique list","allocs":1,"reallocs":2,"deallocs":1,"requested_bytes":31}
```
A harness compares each line with the ceilings in a reviewed data file, with
no deliberate crash and no input files. A crash, failed ` expect ` or leak under
` measure ` exits non-zero, and every other command discards reports, so the
same target can still be fuzzed. A plain ` roc build ` gives the same counts as
` roc build --fuzz `. The manual’s *Hold allocation counts with a ratchet*
section covers the line format, what each count means, and what makes counts
change.
#### Out of memory is a saved failure
A Roc allocation the host cannot serve now ends the run the way a crash does.
The runner prints ` [roc crashed] out of memory ` and saves the input that asked
for the allocation, so it can be replayed and minimized like any other failure.
#### Other changes
- The host’s bindings are regenerated from the current compiler’s platform ABI.
- The examples archive no longer pins a compiler. Each archived example differs
from the repository’s only in its platform URL, so it builds with whichever
compiler you run, without a version-mismatch warning. The compiler the
release was built with is recorded under **Compilers** below and in the
archive’s ` release.json `. The archive holds the examples a new user reads;
the regression targets that exercise Roc’s builtins stay in the repository
under ` tests/targets/ `.
- The host and the native linker inputs are byte-reproducible: the same
sources, Zig release and flags give identical libraries on any machine. The
macOS libraries are built against the headers Zig bundles, so the build reads
no installed Apple SDK. Nothing is compiled with debug information, so the
bundle is much smaller.
- The bundle comes with SPDX and CycloneDX documents and an evidence manifest
that binds them to the bundle’s exact bytes. They are linked below.
- The online manual at [https://lukewilliamboswell.github.io/roc-fuzz/](https://lukewilliamboswell.github.io/roc-fuzz/) leads
with the latest release; the default branch’s copy is under ` unreleased/ `.
#### Use this release
roc-fuzz 0.4.4 supports Linux x86-64 with musl and Apple Silicon macOS 11 or
newer. Both hosts test the packaged bundle and the archived examples before
publication. Copy this application header into a fuzz target:
``` roc
app [target] { fuzz: platform "{platform-url}" }
```
The header names no compiler. Use a current Roc nightly.
Download ` roc-fuzz-examples-0.4.4.zip ` from the release assets for the complete
example suite, its supporting modules, and the cases that describe them, bound
to this exact bundle. Read or download the versioned PDF manual at
[https://github.com/lukewilliamboswell/roc-fuzz/releases/download/0.4.4/roc-fuzz-0.4.4.pdf](https://github.com/lukewilliamboswell/roc-fuzz/releases/download/0.4.4/roc-fuzz-0.4.4.pdf).
#### Upgrade and verification
Replace the platform URL in each target’s header and remove the ` roc ` field if
the header has one. No other change to a target is needed. A package that reads
allocation counts from a deliberate crash can move to ` report_allocs! ` and
` measure `.
Saved corpus and crash files encode generator input rather than a stable public
format. Keep the matching executable when exact historical reproduction
matters.
The bundle, its SBOM documents and its build-provenance attestations are
published together. Verify a downloaded bundle with:
``` console
gh attestation verify .tar.zst --repo lukewilliamboswell/roc-fuzz
```
For bugs or feedback, open an issue at
[https://github.com/lukewilliamboswell/roc-fuzz/issues](https://github.com/lukewilliamboswell/roc-fuzz/issues) with the compiler
version, target, and reproducing input. The full history since 0.4.3 is at
[https://github.com/lukewilliamboswell/roc-fuzz/compare/0.4.3...0.4.4](https://github.com/lukewilliamboswell/roc-fuzz/compare/0.4.3...0.4.4).
### roc-fuzz 0.4.3
This patch removes redundant type exposures from the platform imports so the
platform compiles with Roc ` nightly-2026-10-04-130536d `. The public API and native
linker inputs are unchanged from 0.4.2. Release validation and the frozen example
suite retain the accepted compiler ` nightly-2026-09-29-7f11a82 `.
The compiler upgrade and basic-cli 0.24.0 tooling migration are validated
separately against this compatibility release.
### roc-fuzz 0.4.2
This release expands builtin fuzz coverage and makes published examples
reproducible independently of the repository’s development branch. The platform
API is unchanged from 0.4.1. Project validation and repository tooling now use Roc
` nightly-2026-09-29-7f11a82 `.
#### Expanded builtin coverage
The registered suite now contains 75 targets, including new checks for:
- ` atan2 ` for decimal and floating-point numbers;
- approximate equality for ` F32 `, ` F64 `, and ` Dec `;
- decimal rounding and rounding to a step;
- custom iterators and streams, including laziness, size hints, and leaks; and
- numeric prefix parsing, round trips, ownership, and JSON number decoding.
These targets run through type checks, unit tests, builds, seed replay, and short
fuzz campaigns. Wide UTF-16/UTF-32 decoding targets remain deferred because their
APIs are not available in the selected compiler.
#### Use this release
``` roc
app [target] { fuzz: platform "{platform-url}", roc: "nightly-2026-09-29-7f11a82" }
```
Download ` roc-fuzz-examples-0.4.2.zip ` from the release assets for the complete
example suite, supporting modules, and test inventory bound to this exact
platform bundle. Repository examples instead use the local working-tree
platform. The platform no longer embeds a compiler pin; the examples record the
compiler used for publication, and workspace metadata selects the compiler for
development and CI.
Linux x86-64 with musl and Apple Silicon macOS remain supported. Both hosts test
the packaged platform and frozen examples before publication. Reviewed native
linker inputs retain their independent content-addressed release and lock.
#### Upgrade and verification
Update your application’s platform URL to the one above and use the selected
compiler. No platform API migration is required from 0.4.1. Compatibility with
other compiler versions is not implied by this release.
The release includes the versioned manual at [https://github.com/lukewilliamboswell/roc-fuzz/releases/download/0.4.2/roc-fuzz-0.4.2.pdf](https://github.com/lukewilliamboswell/roc-fuzz/releases/download/0.4.2/roc-fuzz-0.4.2.pdf), a zipped documentation
site, an SPDX SBOM, and provenance attestations for the platform and examples.
Use these assets to inspect the published inputs and verify the downloaded
artifacts. Report regressions through the repository’s issue tracker with the
compiler version, target, and reproducing input.
### roc-fuzz 0.4.1
This patch pins the platform and project validation to Roc
` nightly-2026-09-26-d6267b4 `. The September 26 compiler reports a warning for
an older compiler pin even inside a downloaded platform dependency, so 0.4.0
cannot be used by warning-free builds with that compiler. This release aligns
the platform pin with those downstream builds without changing the Roc API.
#### Use this release
``` roc
app [target] { fuzz: platform "{platform-url}", roc: "nightly-2026-09-26-d6267b4" }
```
Linux x86-64 with musl and Apple Silicon macOS remain supported. The native
inputs are regenerated and attested under the updated workspace fingerprint;
packaged-platform tests run on both supported hosts before publication.
Repository tooling retains its separate September 18 compiler pin.
The versioned manual is available at [https://github.com/lukewilliamboswell/roc-fuzz/releases/download/0.4.1/roc-fuzz-0.4.1.pdf](https://github.com/lukewilliamboswell/roc-fuzz/releases/download/0.4.1/roc-fuzz-0.4.1.pdf). See the 0.4.0 notes for the
allocation properties, CI evidence command, and migration from 0.3.0.
### roc-fuzz 0.4.0
roc-fuzz 0.4.0 makes fuzzing a better fit for day-to-day Roc development. It
adds allocation-aware properties and a purpose-built CI evidence command,
while keeping the usual workflow small: describe typed inputs, state the
property, and build a self-contained libFuzzer executable with ` roc build --fuzz `.
This is also the first release with a complete manual and a hardened,
reproducible release pipeline. The platform bundle, documentation, checksums,
SBOM, and signed provenance are published together so that a fuzz target can be
pinned and audited later.
#### Use this release
roc-fuzz 0.4.0 supports Linux x86-64 with musl and Apple Silicon macOS 11 or
newer. It was built using ` nightly-2026-09-22-e494788 ` of the Roc compiler but is
expected to work on later compatible versions.
Copy this application header into a fuzz target:
``` roc
app [target] { fuzz: platform "{platform-url}", roc: "nightly-2026-09-22-e494788" }
```
Read or download the versioned PDF manual at [https://github.com/lukewilliamboswell/roc-fuzz/releases/download/0.4.0/roc-fuzz-0.4.0.pdf](https://github.com/lukewilliamboswell/roc-fuzz/releases/download/0.4.0/roc-fuzz-0.4.0.pdf). The online manual at
[https://lukewilliamboswell.github.io/roc-fuzz/](https://lukewilliamboswell.github.io/roc-fuzz/) follows the default branch and
may describe changes made after this release.
#### Allocation properties
Fuzz targets can now turn memory-management and copy-on-write expectations into
properties. ` Fuzz.measure_allocs! ` measures work, ` expect_allocs_at_most! ` and
` expect_allocs_at_least! ` pin allocation behaviour, and ` expect_no_leaks! `
checks that a region leaves no live allocations behind. Effectful
` target_with! ` and ` from_bytes! ` constructors let those checks run inside a
target without making ordinary pure targets more complicated.
These checks have already paid for themselves: the expanded Dict and List
campaigns found aliasing, uniqueness, and reference-counting regressions, with
small reproducers recorded in the repository’s trophy case.
#### CI evidence, not just a pass or fail
Every generated executable now has a ` ci ` command. It supervises a bounded fuzz
run and writes a machine-readable JSON report, a job-summary Markdown file,
logs, corpus hashes, and reproducing failures into one evidence directory. The
report records the downstream revision, Roc version, platform release, and
platform digest, making it much easier to answer what was tested after a CI job
has finished.
The runner also adds explicit artifact directories and optional leak-detection
control. A new model-based Set suite and broader Dict and List sorting targets
exercise the platform itself across Linux and macOS.
#### Documentation and release integrity
The new manual covers the path from a first target through campaign tuning,
corpus maintenance, failure minimization, CI integration, internals, and
release verification. API documentation is generated from the platform source
and ships beside both the PDF and an offline HTML archive.
Release bundles are tested on every supported operating system. Each release
also includes an SPDX SBOM and downloadable GitHub attestations for offline
provenance verification. Native libraries are checksum-locked and released
independently, while the Linux libFuzzer build is exercised under
AddressSanitizer before publication.
#### Upgrading from 0.3.0
The platform’s low-level ` Target.run ` callback is now effectful and is named
` run! `. Applications built through ` Fuzz.target `, ` Fuzz.target_with `, or
` Fuzz.from_bytes ` do not need to change for this API adjustment. Custom code
which constructs or calls ` Target ` directly must rename that field or method.
This release requires ` nightly-2026-09-22-e494788 ` or later, where tag unions in
output positions are implicitly open. ` Arbitrary.bytes ` now declares its error as
` [NotEnoughData(U64)] ` without ` .. `; code that uses ` ? ` to merge that error into
a wider union keeps working on the required compiler, but will not type-check on
older nightlies.
Saved corpus and crash files encode generator input rather than a stable public
format. Keep the matching executable when exact historical reproduction
matters, and start a fresh corpus when changing a target’s generator changes
how bytes are decoded.
For bugs or feedback, please open an issue at
[https://github.com/lukewilliamboswell/roc-fuzz/issues](https://github.com/lukewilliamboswell/roc-fuzz/issues). The full commit history
since 0.3.0 is available in
[the comparison with 0.3.0](https://github.com/lukewilliamboswell/roc-fuzz/compare/0.3.0...0.4.0).