1. roc-parser: A manual for parsing text and data in Roc§

Parse text and data in Roc. roc-parser gives you small parsers that combine into larger ones, and ready-made parsers for CSV, YAML, XML, JSON, Markdown and HTTP/1.1 built from the same pieces.

Browse the API reference · Download this manual as a PDF.

1.1. Choose a path§

New to roc-parser

Read What roc-parser is, then Getting started, then Your first parser.

Parsing a format roc-parser already supports

Go to its guide: Read CSV data, Read YAML configuration and frontmatter, Read XML documents, Read and write JSON, Markdown or HTTP messages.

Writing your own parser

Read Combinators by task, Parse a format of your own and Report parse errors.

Looking something up

Modules, Conformance, Performance, Compatibility and Glossary.

Not sure whether you need a parser

Read Choosing how to read your input.

Contributing to roc-parser

Start with Contributing to roc-parser.

Curious where the design comes from

Read Where the design comes from.

2. Start here§

2.1. What roc-parser is§

roc-parser turns text into typed Roc values. You give it a string, such as the contents of a CSV file, a YAML configuration, or an HTTP request, and it gives back either a Roc value you can use directly or an error that says what was wrong with the input.

It is written in pure Roc, so it works with any platform, and it comes in two layers:

• Ready-made parsers for common formats. You call one function and get a document tree, or, for CSV and YAML, your own record type: the compiler infers the type from your annotation and decodes straight into it.

• The small building blocks those parsers are made from, called parser combinators. You use them to read a format of your own, from a one-line key=value setting to a complete file format.

This manual is for Roc programmers. You do not need to know anything about parsing theory.

2.1.1. The problem it solves§

Most programs start by reading text that someone else wrote. Splitting a line on commas, or matching it with a regular expression, works until the input contains a quoted comma, a missing field, or a value of the wrong kind. Then the code either crashes, silently produces wrong data, or grows a pile of special cases.

A parser describes the shape of valid input once, as a Roc value, and checks every input against it. With roc-parser:

• The result is typed. A CSV row becomes { name : Str, moons : U64 }, not a list of strings you still have to convert.

• Runtime parsing returns invalid input as an Err value. The ready-made parsers follow their published specifications, including the awkward corners, and report where the problem is.

• Checked quotations and imported constants turn invalid source into compiler errors, with configurable checks and typed decoding.

• A parser is built from smaller parsers, so each piece can be tested on its own with expect.

Compared with a regular expression, a combinator parser is longer to write for a one-off match, but it can describe nested structure (a regular expression cannot match balanced brackets), it produces a value rather than a list of captured substrings, and it reads as ordinary Roc code.

2.1.2. What a first success looks like§

This program decodes a CSV file with a header row into typed records and prints them. Getting started walks through it.

import cli.Stdout
import parser.CSV

Planet : { name : Str, moons : U64 }

input =
	\\name,moons
	\\Mercury,0
	\\Earth,1
	\\Mars,2

planets : Str -> Try(List(Planet), [InvalidCsv(CSV.Error), MissingRequiredField(Str)])
planets = |text| CSV.parse(text)

describe : Str -> Str
describe = |text| {
	match planets(text) {
		Ok(rows) => Str.join_with(rows.map(|p| "${p.name}: ${p.moons.to_str()} ${if (p.moons == 1) "moon" else "moons"}"), "\n")
		Err(InvalidCsv(error)) => "line ${error.line.to_str()}: ${error.message}"
		Err(MissingRequiredField(name)) => "no ${name} column"
	}
}

main! = |_args| {
	Stdout.line!(describe(input))
}

Output:

Mercury: 0 moons
Earth: 1 moon
Mars: 2 moons

2.1.3. Supported formats§

Each module follows a published specification. Where a module implements only part of one, its chapter says which part, and input outside that part is rejected with an error rather than misread.

The format chapters describe each module’s exact behaviour. The API reference lists every public function.

2.1.4. How the pieces fit together§

A combinator is a function that takes parsers and returns a new parser. The Utf8 module provides parsers for the smallest pieces of text, such as one byte, a digit, or an exact word. The Parser module combines them: one after another, one of several alternatives, repeated, or with the result transformed. Each format module also exposes its parser as a Parser value, so your own parsers and the library’s can be mixed.

Your first parser teaches the combinators by building a small parser one step at a time.

2.1.5. When to choose something else§

roc-parser fits programs that read text formats of moderate size completely into memory and want typed results with clear errors. Consider something else when:

• You need full YAML, with anchors, aliases, tags or several documents in one stream, or XML with document type declarations, namespaces or schema validation. These modules reject such input on purpose.

• You need to parse a stream incrementally as it arrives, such as a very large log file or a network connection. Every parser here works on a complete input held in memory.

• You want to render Markdown to HTML, or serialize values back to text in a format other than JSON. Only JsonValue writes its format back out.

• You want to decode JSON straight into your own record type. Roc’s builtin Json does that; JsonValue is for JSON whose shape a record cannot describe (see Read and write JSON).

• You need TOML, JSON5, or another format this package does not include. You can write it with the combinators, but a dedicated package may already exist.

2.1.6. Stability§

roc-parser tracks recent nightly builds of the Roc compiler. Because Roc is still changing, the package’s API may change between releases. Each release names the compiler build it was tested with, and the release notes describe any changes you need to make when upgrading.

2.2. Getting started§

In this chapter you add roc-parser to a Roc app and use it to read three lines of CSV into typed records. At the end the app prints one line per record. You need a terminal, an internet connection, and a little Roc: how to write a function, a record, and a match.

2.2.1. Before you start§

You need:

• The Roc compiler, as a recent nightly build. Roc is still changing, so each roc-parser release is tested with one specific nightly, which its release notes name. Download that build from the Roc nightlies page and put the roc executable on your PATH.

• A platform for your app. The examples in this manual use basic-cli 0.26.0, which provides main! and printing to the terminal. roc-parser itself is pure Roc and works with any platform.

Check the compiler:

roc version

It prints the name of the nightly build, which should match the one in the release notes:

Roc compiler version nightly-<date>-<commit>

2.2.2. Add the package§

A Roc app names each package it uses by URL in its header. The compiler downloads the package the first time it builds the app and checks it against the hash in the URL.

1. Open the roc-parser releases page and choose the newest release.

2. Under Assets, find the file ending in .tar.zst and copy its link.

3. Put that link in your app’s header under a short name. This manual uses parser.

Create a file named main.roc with this header, replacing the parser URL with the one you copied:

app [main!] {
	cli: platform "https://github.com/roc-lang/basic-cli/releases/download/0.26.0/GddgvsGjiNXrHFW6au5i5dUDbTs8FinQdrgE2Gqmw53t.tar.zst",
	parser: "https://github.com/lukewilliamboswell/roc-parser/releases/download/<version>/<hash>.tar.zst",
}

The examples in this manual are tested against the package source in the repository, so their headers say parser: "../../package/main.roc" instead. Everything else in them is the same.

For complete programs to start from, download roc-parser-examples-<version>.zip from the same release. Its apps are already pinned to that release; unzip it and follow the README.md inside, which runs them with roc <name>.roc from its examples/ directory.

2.2.3. Read CSV into records§

Add the rest of the program below the header:

import cli.Stdout
import parser.CSV

Planet : { name : Str, moons : U64 }

input =
	\\name,moons
	\\Mercury,0
	\\Earth,1
	\\Mars,2

planets : Str -> Try(List(Planet), [InvalidCsv(CSV.Error), MissingRequiredField(Str)])
planets = |text| CSV.parse(text)

describe : Str -> Str
describe = |text| {
	match planets(text) {
		Ok(rows) => Str.join_with(rows.map(|p| "${p.name}: ${p.moons.to_str()} ${if (p.moons == 1) "moon" else "moons"}"), "\n")
		Err(InvalidCsv(error)) => "line ${error.line.to_str()}: ${error.message}"
		Err(MissingRequiredField(name)) => "no ${name} column"
	}
}

main! = |_args| {
	Stdout.line!(describe(input))
}

Run it:

roc main.roc

Output:

Mercury: 0 moons
Earth: 1 moon
Mars: 2 moons

The first run takes longer while the compiler downloads the platform and the package.

2.2.4. What the program does§

import parser.CSV makes the CSV module from the package named parser available.

Planet is the record type each row becomes. The first line of the input is a header row, and each field of Planet reads the column with the same name.

planets calls CSV.parse, which splits the text into rows and decodes each row into a record. CSV.parse does not take the row type as an argument: the compiler infers it from the type annotation on planets, and finds the decoding code through the static dispatch method parser_for, which every record type has. Change the annotation and the same call decodes a different record.

The result is Ok with a list of Planet records, or Err with one of two tags:

• InvalidCsv(error) when the text is not valid CSV or a cell does not fit its field. error says which record and field failed, where it starts in the text, and why.

• MissingRequiredField(name) when the header has no column for a field. The compiler adds this tag to the error type of every type-directed decoder, so the annotation must name it.

describe handles all three cases with a match. Nothing is printed until every row has been read successfully.

To see the error path, change Mars,2 to Mars,two and run the program again. It prints a message that names line 4 and quotes two. Rename the moons column and it prints no moons column. Report parse errors shows how to report every kind of failure, and Read CSV data shows optional columns, empty cells and hand-built row parsers that match columns by position.

2.2.5. Next steps§

• Your first parser teaches the building blocks by writing a parser for a small settings format.

• The format chapters show how to read CSV, YAML, XML, JSON, Markdown and HTTP, and how to write JSON back out.

• Parse a format of your own shows how to build and test a parser for a format of your own.

2.3. Choosing how to read your input§

Roc gives you four ways to turn text into values, and roc-parser provides two of them. This chapter helps you pick one before you write any code. In short:

1. Decode into your own type when the input is a standard format and you already know the shape you want. The record type is the specification, and there is almost no code to write.

2. Read a document tree with one of roc-parser’s format modules when you do not know the shape in advance, or you need what a record would throw away.

3. Build a parser with combinators when the format is your own, or you need control over exactly what is accepted and what the errors say.

4. Use no parser at all when a Str function does the job, or when the input is too large or too hot for any of these.

The examples come from choosing-approaches.roc; every expect in it passes.

2.3.1. Decode into your own type§

Roc has type-directed decoding built in. A format (such as the builtin Json) knows how to read strings, numbers, lists and records. Your type knows which of those it is made of. The compiler combines the two, so the annotation alone decides what is accepted:

# (a) Type-directed decoding: the record type is the whole specification.
Service : { name : Str, port : U64 }

decode_service : Str -> Try(Service, [InvalidJson(Str), MissingRequiredField(Str)])
decode_service = |json| Json.parse(json)

expect decode_service("{\"name\": \"web\", \"port\": 8080}") == Ok({ name: "web", port: 8080 })
expect decode_service("{\"name\": \"web\"}") == Err(MissingRequiredField("port"))

The parsers chapter of the language reference describes the mechanism. Every type has a parser_for method, which the compiler derives for records, lists and tuples, and a format is a type module whose methods read one value at a time. Static dispatch joins the two. A field of type Try(_, [Missing]) is optional, a missing required field is reported by name, and when the parser is a top-level constant it is assembled at compile time.

Choose this when:

• the input is a format with a parser_for implementation, and

• the mapping from the format to your type is the obvious one: object keys are field names, arrays are lists, and so on.

The builtin Json is such a format, for JSON whose shape you know in advance. For dynamic JSON, whose keys are not Roc identifiers or whose arrays mix shapes, or that must keep member order and every digit of its numbers, read a document tree with roc-parser’s JsonValue instead (see Read and write JSON).

roc-parser’s CSV and Yaml modules are formats for this mechanism. CSV.parse reads a file with a header row into a list of records, matching columns to fields by name, and Yaml.decode reads a document into a record. Neither takes the type as an argument: the annotation chooses it. A required field with no column or key fails with MissingRequiredField(name), a tag the compiler adds to the error type, so the annotation names it alongside the format’s own error:

# roc-parser's CSV and Yaml modules are formats for the same mechanism.
Planet : { name : Str, moons : U64, rings : Try(Bool, [Missing]) }

planets : Str -> Try(List(Planet), [InvalidCsv(CSV.Error), MissingRequiredField(Str)])
planets = |text| CSV.parse(text)

expect planets("name,moons\nMars,2\n") == Ok([{ name: "Mars", moons: 2, rings: Err(Missing) }])
expect planets("name\nMars\n") == Err(MissingRequiredField("moons"))

service : Str -> Try(Service, [InvalidYaml(Yaml.Error), MissingRequiredField(Str)])
service = |text| Yaml.decode(text)

expect service("name: web\nport: 8080\n") == Ok({ name: "web", port: 8080 })
expect service("name: web\n") == Err(MissingRequiredField("port"))

Its limits come from the same design. The format decides the representation, so you cannot ask it to accept yes as a boolean or to read a date in your own layout. Where a wrong value is depends on the format: CSV.parse reports the record, field, line and column, and Yaml.decode the line and column. And the type must be known when you compile.

2.3.2. Read a document tree§

roc-parser’s Yaml, Xml, JsonValue and Markdown modules read a whole document into a tree that mirrors the format, and HTTP reads a message into a record of its parts. You then walk the tree with pattern matching, or with the tree methods such as get_path and as_i64 on a Yaml or JsonValue value and attribute and children_named on an Xml.Node:

# (b) A ready-made format parser: read the whole tree, then decide what to
# do with each key, whatever keys the file happens to contain.
top_level_keys : Str -> Try(List(Str), [NotAMapping, InvalidYaml(Yaml.Error)])
top_level_keys = |text| {
	match Yaml.parse_str(text)? {
		Mapping(entries) => Ok(entries.map(|entry| entry.key))
		_ => Err(NotAMapping)
	}
}

expect top_level_keys("name: web\nport: 8080\nextra: [1, 2]") == Ok(["name", "port", "extra"])

# The tree methods follow a path without a match per level.
port_of : Str -> Try(I64, [Missing, WrongType, InvalidYaml(Yaml.Error)])
port_of = |text| Yaml.parse_str(text)?.get_path(["server", "port"])?.as_i64()

expect port_of("server:\n  port: 8080\n") == Ok(8080)
expect port_of("server: {}\n") == Err(Missing)

Choose this when:

• the shape is not known in advance: any key may appear, keys vary between files, or you are writing a tool that works on every document, such as a linter or a converter;

• you need what a record type cannot hold: the order of mapping keys, the exact text of a JSON number, the mix of text and child elements in XML, or the block structure of a Markdown document;

• you want a precise error with a line and column from a parser that follows the format’s specification, and you check the content yourself afterwards. The format modules drop comments, so a tool that must preserve them needs a parser of its own.

The cost is code: you write the walk, and a missing or mistyped key is something you check, not something the type system checks for you. The format chapters (Read YAML configuration and frontmatter, Read XML documents, Read and write JSON, Markdown and HTTP messages) show helper functions that keep the walk short.

2.3.3. Build a parser with combinators§

When no format module reads your input, write a parser from the Parser and Utf8 combinators. A parser is a value made of smaller parsers, so the code has the shape of the grammar:

# (c) A parser of your own: a legacy "host:port" line, where the port must
# be digits and the grammar is yours to define.
Endpoint : { host : Str, port : U64 }

endpoint : Parser(Utf8.Bytes, Endpoint)
endpoint =
	Parser.const(|host| |port| { host, port })
		.keep(Parser.chomp_while(|b| b != ':').map(Str.from_utf8_lossy))
		.skip(Utf8.codeunit(':'))
		.keep(Utf8.digits)

expect Utf8.parse_str(endpoint, "example.com:443") == Ok({ host: "example.com", port: 443 })
expect Utf8.parse_str(endpoint, "example.com:https").is_err()

Choose this when:

• the format is your own or a legacy one: a log line, a configuration dialect, a wire protocol, or a small language such as a query or a template;

• the grammar is defined by more than a record shape: alternatives, nesting, repetition with separators, or a part whose length is given earlier in the input;

• you want to reject invalid values while parsing, so that a port of 70000 or a date of 2026-02-30 fails at the right place with your own message;

• you want to read only the start of the input and keep the rest, such as one message from a buffer that holds several (Utf8.parse_str_partial).

Combinators are also how you extend a format module. CSV.record and CSV.field are combinators, every format module has a parser value to embed, and you can mix Utf8 parsers into any parser of your own. A combinator parser’s failure is the furthest failure, with a byte offset. Your first parser teaches the method and Parse a format of your own applies it to a real format.

2.3.4. Use no parser at all§

Parser combinators are not always the right tool, and roc-parser’s own format modules show where the line falls.

The input has no structure worth a grammar

One separator, no quoting and no nesting is a job for Str.split_on, Str.trim and friends. A parser would be longer and no clearer:

# (d) No parser at all: one separator and no nesting is a job for `Str`.
fields : Str -> List(Str)
fields = |line| line.split_on("\t")

expect fields("a\tb\tc") == ["a", "b", "c"]

The input is very large or arrives as a stream

Every parser in this package reads a complete input held in memory, and the result holds copies of the text it read. For a multi-gigabyte log or a network connection, read and process one record at a time with a hand-written loop over the bytes, or use a streaming library.

The parser is on a hot path

Each combinator adds a function call and an intermediate Try. That is cheap for configuration files and messages, but Add a format module records that the CSV, XML, HTTP and Markdown modules moved their document-level parsing to byte scanners with an explicit stack, to stay linear and avoid deep recursion; the JSON module was written that way from the start. Do the same when profiling shows the parser matters. Performance compares the built-in parsers with Go, Rust and Python libraries.

The grammar is ambiguous or left-recursive

Combinators take the first alternative that succeeds and cannot handle a rule that starts with itself, such as expr = expr "-" term. A large grammar written for a parser generator, such as an existing yacc or ANTLR grammar, is often better served by a generator, which checks the grammar for conflicts before any input is read. Where the design comes from explains these limits.

2.3.5. Decision table§

2.3.6. Decision flow§

The same choice as a sequence of questions:

The questions are ordered from least code to most. You can also combine answers: read a Markdown file as a tree, read its frontmatter as a YAML tree, and parse one custom field inside it with a combinator.

2.4. Your first parser§

In this tutorial you write a parser for a small settings format, one line per setting:

name="demo"
port=8080
debug=true

By the end you have a parser that turns that text into a list of typed records, and you know how to read its errors. Along the way you meet the combinators that every parser in this package is built from.

You need the setup from Getting started. Each step is a complete program in the repository’s docs/examples directory; the step’s heading names the file. Run a step with roc <file> and its tests with roc test <file>.

2.4.1. Step 1: read a key§

combinator-primer-step1.roc

A key is one or more lowercase letters or underscores. Start with a parser for one such byte, repeat it, and turn the bytes into a Str:

is_key_byte : U8 -> Bool
is_key_byte = |b| (b >= 'a' and b <= 'z') or b == '_'

key : Parser(Utf8.Bytes, Str)
key =
	Utf8.codeunit_satisfies(is_key_byte)
		.one_or_more()
		.map(Str.from_utf8_lossy)

Three ideas appear here:

• A Parser(Utf8.Bytes, Str) reads from UTF-8 bytes (Utf8.Bytes is List(U8)) and produces a Str. Every parser has an input type and a result type.

• Utf8.codeunit_satisfies(is_key_byte) reads one byte if is_key_byte accepts it, and fails otherwise.

• .one_or_more() repeats a parser until it fails and collects the results in a list. .map(f) transforms the result with an ordinary function; here Str.from_utf8_lossy turns the bytes into a Str.

Utf8.parse_str runs a parser on a whole string. These tests pass:

expect Utf8.parse_str(key, "name") == Ok("name")
expect Utf8.parse_str(key, "Name").is_err()

The second fails because N is not a lowercase letter, so not even one key byte can be read.

2.4.2. Step 2: read a whole setting§

combinator-primer-step2.roc

A setting is a key, an =, and a value. For now, the value is everything up to the end of the line:

value : Parser(Utf8.Bytes, Str)
value =
	Parser.chomp_while(|b| b != '\n')
		.map(Str.from_utf8_lossy)

Entry : { key : Str, value : Str }

entry : Parser(Utf8.Bytes, Entry)
entry =
	Parser.const(|k| |v| { key: k, value: v })
		.keep(key)
		.skip(Utf8.codeunit('='))
		.keep(value)

This is the pattern for reading things in sequence:

1. Parser.const(f) is a parser that reads nothing and produces f, the function that builds the result. f takes its arguments one at a time (|k| |v| ...), because they arrive one at a time.

2. .keep(p) runs p next and passes its result to the function.

3. .skip(p) runs p next and throws its result away. The = must be there, but you do not need it in the record.

Parser.chomp_while(pred) reads bytes while pred accepts them. It always succeeds, even when it reads nothing, which is why "empty=" gives an empty value:

expect Utf8.parse_str(entry, "name=roc") == Ok({ key: "name", value: "roc" })
expect Utf8.parse_str(entry, "empty=") == Ok({ key: "empty", value: "" })

Running the step prints:

Ok({ key: "name", value: "roc" })

2.4.3. Step 3: choose between kinds of value§

combinator-primer-step3.roc

A value is a number, true or false, or text in double quotes. Write one parser for each kind, then try them in order with Parser.one_of:

Value : [Number(U64), Flag(Bool), Text(Str)]

number : Parser(Utf8.Bytes, Value)
number = Utf8.digits.map(|n| Number(n))

flag : Parser(Utf8.Bytes, Value)
flag =
	Parser.one_of([
		Parser.const(Flag(Bool.True)).skip(Utf8.string("true")),
		Parser.const(Flag(Bool.False)).skip(Utf8.string("false")),
	])

text : Parser(Utf8.Bytes, Value)
text =
	Parser.chomp_while(|b| b != '"')
		.map(|bytes| Text(Str.from_utf8_lossy(bytes)))
		.between(Utf8.codeunit('"'), Utf8.codeunit('"'))

value : Parser(Utf8.Bytes, Value)
value = Parser.one_of([number, flag, text])

Parser.one_of tries each parser on the same input and returns the first success. If one fails part-way through, the next one starts again from the beginning, so text does not see input already examined by number or flag.

Utf8.digits reads a whole number as a U64, and Utf8.string("true") reads exactly that text. .between(open, close) reads open, the parser, and close, and keeps only the middle result.

expect Utf8.parse_str(value, "8080") == Ok(Number(8080))
expect Utf8.parse_str(value, "true") == Ok(Flag(Bool.True))
expect Utf8.parse_str(value, "\"hello world\"") == Ok(Text("hello world"))
expect Utf8.parse_str(value, "maybe").is_err()

The entry parser from step 2 now uses this value. Running the step prints:

Ok({ key: "port", value: Number(8080) })

2.4.4. Step 4: read every line, and read the errors§

combinator-primer-step4.roc

A settings file is entries separated by line breaks. sep_by reads that shape and drops the separators:

config : Parser(Utf8.Bytes, List(Entry))
config = entry.sep_by(Utf8.codeunit('\n'))

expect
	Utf8.parse_str(config, "port=8080\ndebug=true")
		== Ok([{ key: "port", value: Number(8080) }, { key: "debug", value: Flag(Bool.True) }])

Utf8.parse_str returns one of three results, and a real program should handle each:

report : Str -> Str
report = |source| {
	match Utf8.parse_str(config, source) {
		Ok(entries) => "parsed ${entries.len().to_str()} entries"
		Err(ParseError({ message, offset })) => "failed at byte ${offset.to_str()}: ${message}"
	}
}

report_entry : Str -> Str
report_entry = |source| {
	match Utf8.parse_str(entry, source) {
		Ok(_) => "parsed one entry"
		Err(ParseError({ message, offset })) => "failed at byte ${offset.to_str()}: ${message}"
	}
}

main! = |_args| {
	Stdout.line!(report("name=\"demo\"\nport=8080\ndebug=true"))?
	Stdout.line!(report("name=\"demo\"\nport=eighty"))?
	Stdout.line!(report("name=\"demo\"\n"))?
	Stdout.line!(report_entry("port=eighty"))
}

Output:

parsed 3 entries
failed at byte 17: expected char `"`
failed at byte 12: expected a codeunit satisfying a condition, but input was empty.
failed at byte 5: expected char `"`

Each line of output shows one outcome:

1. The whole input matched: Ok.

2. port=eighty is not a valid entry, so sep_by stopped after the first entry and did not reach the end of the input. parse_str reports the furthest failure instead: none of the value parsers accepted eighty, at byte 17.

3. The trailing line break is a separator with no entry after it, so the parse fails at the end of the input, where a key was expected. Real files often end with a line break; Parse a format of your own shows one way to accept it.

4. Parsing a single bad entry directly gives the same failure, at byte 5.

The second line is worth remembering. A repetition such as sep_by or many stops at the first element it cannot read, but that element’s failure is kept, so a mistake in the middle of a file is reported where it is. Report parse errors explains how to get better messages.

2.4.5. What you have learned§

• A parser has an input type and a result type, and Utf8.parse_str runs one on a whole string.

• Parser.const(f).keep(a).skip(b).keep(c) reads things in sequence.

• .map(f) transforms a result; Parser.one_of chooses between alternatives; one_or_more, many and sep_by repeat.

• A parse ends in Ok or a ParseError with a message and a byte offset.

Combinators by task covers the remaining combinators and the rules for alternatives and repetition in more depth.

3. Guides§

3.1. Read CSV data§

This chapter shows how to turn comma-separated text into Roc values: as records chosen by the type you ask for, as tuples when the file has no header, as raw fields, and with hand-built record parsers when a column needs custom handling. It also shows how to tell the user which row was wrong. It is for Roc programmers who have added parser to their app’s dependencies.

Every example on this page comes from docs/examples/csv-guide.roc, which CI runs, and imports parser.CSV, parser.Parser and parser.Utf8.

3.1.1. Decode rows into records§

When the first row names the columns, CSV.parse decodes every other row into the record type you annotate. Each field reads the column whose header is exactly its name, so the column order does not matter:

Product : { name : Str, quantity : U64, price : Dec }

products : Str -> Try(List(Product), [InvalidCsv(CSV.Error), MissingRequiredField(Str)])
products = |text| CSV.parse(text)

print_products! : Str => Try({}, _)
print_products! = |text| {
	match products(text) {
		Ok(found) =>
			for p in found {
				Stdout.line!("${p.name}: ${p.quantity.to_str()} at ${p.price.to_str()}")?
			}

		Err(InvalidCsv({ line, column, message, record: _, field: _ })) =>
			Stdout.line!("line ${line.to_str()}, column ${column.to_str()}: ${message}")?

		Err(MissingRequiredField(name)) =>
			Stdout.line!("no column named ${name}")?
	}
	Ok({})
}

Running it on a good file, on a file without a quantity column, and on a file whose second data row is short:

print_products!("name,price,quantity\nbolt,0.15,250\nnut,.05,+40\n")?
print_products!("name,price\nbolt,0.15\n")?
print_products!("name,price,quantity\nbolt,0.15,250\nnut,0.05\n")?

bolt: 250 at 0.15
nut: 40 at 0.05
no column named quantity
line 3, column 1: the record has 2 fields, fewer than the 3 columns

MissingRequiredField(name) is not declared by CSV.parse: the compiler adds it to the error type of any decoder whose record has required fields, so include it in your annotation. Columns that no field names are ignored, and when two columns share a name the last one wins.

A cell decodes according to its field’s type:

These match Rust’s from_str for each number type. A field that is a record, a tuple or a tag with a payload fails with an InvalidCsv error when it is decoded; a field that is a list or a dictionary is a compile-time error, because a CSV cell cannot hold one.

3.1.1.1. Optional columns, empty cells and header spelling§

A field of type Try(a, [Missing]) is Err(Missing) when the file has no such column. A field of type Try(a, [Null]) is Err(Null) when its cell is empty. CSV.parse_normalized matches header names after trimming spaces, lowercasing ASCII letters and turning runs of spaces, - and _ into one _, so First Name, first-name and FIRST_NAME all fill first_name:

Contact : {
	name : Str,
	email : Try(Str, [Missing]),
	age : Try(U64, [Null]),
	status : [Active, Inactive],
}

contacts : Str -> Try(List(Contact), [InvalidCsv(CSV.Error), MissingRequiredField(Str)])
contacts = |text| CSV.parse_normalized(text)

describe_contact : Contact -> Str
describe_contact = |{ name, email, age, status }| {
	email_text =
		match email {
			Ok(address) => address
			Err(Missing) => "no email column"
		}
	age_text =
		match age {
			Ok(years) => years.to_str()
			Err(Null) => "age not given"
		}
	status_text =
		match status {
			Active => "active"
			Inactive => "inactive"
		}
	"${name} (${email_text}, ${age_text}, ${status_text})"
}

For "Name,Age,Status\nAda,36,Active\nAlan,,Inactive\n" this prints:

Ada (no email column, 36, active)
Alan (no email column, age not given, inactive)

3.1.1.2. Row width and blank lines§

Blank lines are skipped. Every other row must have exactly as many fields as the header: a shorter row and a longer row are both an InvalidCsv error naming the row, so that a misplaced comma cannot shift values into the wrong columns. A header with no data rows, and empty input, decode to an empty list.

3.1.2. Files without a header§

CSV.parse_headerless decodes every row into a tuple, one element per column, with the same cell rules. Every row must have as many fields as the tuple has elements:

points : Str -> Try(List((Str, F64, F64)), [InvalidCsv(CSV.Error)])
points = |text| CSV.parse_headerless(text)

Ok([("home", -33.87, 151.21), ("work", -33.86, 151.2)])

3.1.3. Read every field as raw bytes§

Use CSV.parse_records when the columns vary or you want to inspect rows yourself. It returns one CSV.Record, a List(List(U8)), per row:

raw_records : Str -> Str
raw_records = |text| {
	match CSV.parse_records(text) {
		Ok(records) =>
			Str.join_with(records.map(|fields| Str.join_with(fields.map(Str.from_utf8_lossy), " | ")), "\n")

		Err(InvalidCsv({ message, line, column, record: _, field: _ })) =>
			"not valid CSV at ${line.to_str()}:${column.to_str()}: ${message}"
	}
}

Given a header row, a quoted field containing "" and ,, and a row whose last field is empty:

Stdout.line!(raw_records("name,note\r\nwidget,\"says \"\"hi\"\", twice\"\nempty,"))?

name | note
widget | says "hi", twice
empty | 

CSV.split_header separates the first record as column names:

column_names : Str -> Str
column_names = |text| {
	match CSV.parse_records(text) {
		Ok(records) => {
			{ header, rows } = CSV.split_header(records)
			"${Str.join_with(header, ", ")}; ${rows.len().to_str()} rows"
		}

		Err(_) => "not valid CSV"
	}
}

sku, count; 2 rows

CSV.parser is the same reader as a Parser(Utf8.Bytes, List(CSV.Record)), for use inside larger parsers. It accepts any bytes.

3.1.4. Hand-built record parsers§

When a column needs custom handling, such as a list stored in one cell, describe one row with CSV.record, giving it a function that takes one argument per column, then add one .keep(CSV.field(...)) per column in order. A field parser is any Parser(Utf8.Bytes, a); CSV.string, CSV.u64 and CSV.f64 read cells with the rules above:

Movie : { title : Str, year : U64, cast : List(Str) }

cast : Parser(Utf8.Bytes, List(Str))
cast = CSV.string.map(|text| Str.split_on(text, ";"))

movie : Parser(CSV.Record, Movie)
movie =
	CSV.record(|title| |year| |actors| { title, year, cast: actors })
		.keep(CSV.field(CSV.string))
		.keep(CSV.field(CSV.u64))
		.keep(CSV.field(cast))

Run it on text with CSV.parse_with, or on records you already have with CSV.decode. Columns are matched by position, and the record parser must read every field in the row: a row with more fields than .keep calls fails, and so does a row with fewer.

3.1.5. Report which row failed§

Every function stops at the first problem and returns InvalidCsv(error). The CSV.Error record says where the problem is:

describe : Str -> Str
describe = |text| {
	match CSV.parse_with(movie, text) {
		Ok(movies) => "${movies.len().to_str()} movies"
		Err(InvalidCsv({ record, field, line, column, message })) =>
			"record ${record.to_str()}, field ${field.to_str()} (line ${line.to_str()}, column ${column.to_str()}): ${message}"
	}
}

Stdout.line!(describe("Airplane!,1980,Robert Hays;Julie Hagerty"))?
Stdout.line!(describe("Airplane!,1980,Robert Hays\nCaddyshack,soon,Bill Murray"))?
Stdout.line!(describe("Airplane!,1980,Robert Hays,extra"))?

1 movies
record 2, field 2 (line 2, column 12): expected a U64, found `soon`
record 1, field 4 (line 1, column 28): the record has 4 fields, but the parser read only 3

3.1.6. Which CSV files are accepted§

The dialect is RFC 4180 with the relaxations that Python’s csv module (in strict mode) and Rust’s csv crate also accept:

• Fields are separated by ,. Records end with CRLF, LF or a lone CR, and the line break after the last record is optional.

• An unquoted field is any run of bytes other than ,, CR and LF. A " inside an unquoted field is an ordinary character.

• A field that starts with " is quoted. It may contain ,, CR and LF, and "" stands for one ". The closing quote must be followed by ,, a line break or the end of input; an unterminated quoted field is an error.

• Spaces and tabs are kept as part of the field.

• Empty input has no records. A blank line in the middle of the input is a record with one empty field.

• Rows may have different numbers of fields (the typed decoders then check the width).

Stdout.line!(raw_records("size,6\" pipe\rcount,"))?
Stdout.line!(raw_records("bolt,\"250"))?
Stdout.line!("empty input: ${Str.inspect(CSV.parse_records(""))}")?

size | 6" pipe
count | 
not valid CSV at 1:6: unterminated quoted field
empty input: Ok([])

3.1.7. Limitations§

• The separator is always ,. Tab- or semicolon-separated files are not supported.

• A UTF-8 byte order mark is not removed; it becomes part of the first field.

• The whole input is parsed before decoding starts, so very large files are held in memory.

For every function and type in the module, see the CSV API reference.

3.1.8. Performance§

Reading every field as a string runs at 120–150 MB/s on an Apple M2, from a 1.6 KB export to a 1 MiB file: on par with Rust’s csv crate, about 2.3 times slower than Go’s encoding/csv and nearly twice as fast as Python’s C csv module. Delimiters and quotes are found 16 bytes at a time with SIMD, and fields without doubled quotes are slices of the input. The parser is a byte scanner and stays linear on very wide rows and long runs of escaped quotes. Decoding typed columns adds the cost of each conversion. The whole file and every field are held in memory: about 12 MiB of peak memory for a 1 MiB file. See Performance for the method and caveats.

3.2. Read YAML configuration and frontmatter§

This chapter shows how to read a YAML configuration file or the frontmatter at the top of a Markdown post into a Roc value, pull settings out of it, and report mistakes with a line and column. It is for Roc programmers who have added parser to their app’s dependencies and know the YAML they want to read.

The parser handles the YAML that configuration files and frontmatter use in practice, and rejects the rest with an error instead of guessing. If your files use anchors, tags or several documents in one file, read what is rejected first.

The decoding examples come from docs/examples/yaml-guide.roc; the source-highlighting example comes from docs/examples/yaml-source-guide.roc. CI checks, tests and runs both against the package beside this manual. Both import parser.Yaml.

3.2.1. Decode a document into a record§

Yaml.decode reads a document straight into your own Roc type. You do not pass the type: it is inferred from how the result is used, usually from a type annotation. This configuration, held in a Roc multi-line string (each \\ starts one line of the file):

config_text =
	\\# deployment settings
	\\name: web
	\\replicas: 3
	\\debug: false
	\\ports: [80, 443]
	\\owners:
	\\  - name: Ada
	\\    email: ada@example.com
	\\  - name: Grace

is read by:

Owner : { name : Str, email : Try(Str, [Missing]) }

Config : {
	name : Str,
	replicas : U8,
	debug : Bool,
	ports : List(U16),
	owners : List(Owner),
}

## The record type tells `Yaml.decode` what to expect.
load_config : Str -> Try(Config, [InvalidYaml(Yaml.Error), MissingRequiredField(Str)])
load_config = |text| Yaml.decode(text)

summarize : Str -> Try(Str, [InvalidYaml(Yaml.Error), MissingRequiredField(Str)])
summarize = |text| {
	config = load_config(text)?
	owner_names = config.owners.map(|owner| owner.name)

	Ok("${config.name}: ${config.replicas.to_str()} replicas, owners ${Str.join_with(owner_names, " and ")}")
}

summarize(config_text) returns:

web: 3 replicas, owners Ada and Grace

The rules that map YAML onto Roc types:

• A mapping decodes into a record, matching keys to field names, or into a Dict with Str or integer keys. Keys the record does not have are skipped.

• A sequence decodes into a List, or into a tuple when it has exactly as many items as the tuple.

• A plain (unquoted) scalar is resolved for the type that reads it, with the YAML 1.2 core schema (see how plain values are typed). A Str field takes any scalar except a null, so version: 1.10 gives "1.10" rather than a float; a U8 field takes 0x1F; a Bool field takes true or FALSE, but not the string "true".

• A tag union without payloads, such as [Debug, Info], takes the tag’s name.

• A null (~, null or an empty value) decodes as an empty list, dict or record, and as Err(Null) in a Try(_, [Null]) field.

• A Try(_, [Missing]) field is Err(Missing) when its key is absent, like email for Grace above.

A value of the wrong shape or out of range fails with InvalidYaml at that value’s line and column, the same error invalid YAML gives. A required field whose key is absent fails with MissingRequiredField(name). Your code does not declare that tag: the compiler adds it to the error type of every decoder of a record with required fields, so name it in the annotation:

Stdout.line!(Str.inspect(load_config("name: web\nreplicas: 300\n")))?
Stdout.line!(Str.inspect(load_config("name: web\nreplicas: 3\ndebug: no\n")))?
Stdout.line!(Str.inspect(load_config("name: web\nreplicas: 3\ndebug: false\nports: []\n")))?

Err(InvalidYaml({ column: 11, line: 2, message: "integer `300` is outside the range of U8" }))
Err(InvalidYaml({ column: 8, line: 3, message: "expected a boolean, found `no`" }))
Err(MissingRequiredField("owners"))

3.2.1.1. Other key spellings and unknown keys§

Yaml.decoder builds a decoder with other conventions. keys says how YAML spells Roc’s snake_case field names: SnakeCase (as written), KebabCase (user-id) or CamelCase (userId). unknown_keys: Reject turns a key the record does not have into an InvalidYaml error, which catches typos in hand-written configuration:

decode_strict = Yaml.decoder({ keys: KebabCase, unknown_keys: Reject })

load : Str -> Try({ user_id : U64 }, [InvalidYaml(Yaml.Error), MissingRequiredField(Str)])
load = |text| decode_strict(text)

Build the decoder once, as a top-level constant, and call it for each document. To write your own generic function over decodable types, require a.Yaml.Parseable(errors) in its where clause, the way Yaml.decode does.

3.2.2. Explore a document as a tree§

When the shape is not known in advance, Yaml.parse_str returns a Yaml tree:

Yaml := [
    Null,
    Bool(Bool),
    Int(I64),
    Float(F64),
    Text(Str),
    Sequence(List(Yaml)),
    Mapping(List({ key : Str, value : Yaml })),
]

A mapping keeps its entries in source order. get looks up a key, at an index, and get_path follows several of them, using decimal strings such as "0" for sequence indexes. Each returns Err(Missing) when there is nothing there. as_str, as_i64, as_bool and as_list unwrap one variant or return Err(WrongType):

## Explore a document without a record type.
first_owner : Str -> Try(Str, [InvalidYaml(Yaml.Error), Missing, WrongType])
first_owner = |text| {
	config = Yaml.parse_str(text)?
	name = config.get_path(["owners", "0", "name"])?
	name.as_str()
}

For config_text, and then for "owners: []":

Ok("Ada")
Err(Missing)

Empty input, or a document holding only comments, parses as Null. To combine YAML with other parsers, Yaml.parser is a Parser(Utf8.Bytes, Yaml) that reads the rest of its input; its ParseError offset is the byte offset of the error’s line and column.

3.2.3. Read Markdown frontmatter§

Frontmatter is a YAML document between two --- lines at the start of a Markdown file. Split the text at the closing --- and parse the first part. The parser accepts the opening --- (and a closing ...) as document markers, so you do not need to strip them:

## Split a Markdown file into its YAML frontmatter and body.
frontmatter : Str -> Try({ meta : Yaml, body : Str }, [NoFrontmatter, InvalidYaml(Yaml.Error)])
frontmatter = |markdown| {
	match Str.split_on(markdown, "\n---\n") {
		[header, .. as rest] if Str.starts_with(header, "---\n") =>
			Ok({ meta: Yaml.parse_str(header)?, body: Str.join_with(rest, "\n---\n") })

		_ => Err(NoFrontmatter)
	}
}

For "---\ntitle: Hello\ntags: [roc, yaml]\n---\n# Hello\n", printing Yaml.to_inspect(meta) and then the body gives:

Mapping([{ key: "title", value: Text("Hello") }, { key: "tags", value: Sequence([Text("roc"), Text("yaml")]) }])
# Hello

Yaml.decode works on the same text when you know the frontmatter’s fields.

3.2.4. How plain values are typed§

In a Yaml tree, an unquoted value is resolved with the YAML 1.2 core schema. Anything that matches none of these forms is Text:

show!("[~, null, true, FALSE, 42, -7, 0x1F, 0o17, 1.5, 6.02e23, .inf, -.Inf, .nan, 1.2.3, 1_000, 'true', 2026-10-01]")

Sequence([Null, Null, Bool(True), Bool(False), Int(42), Int(-7), Int(31), Int(15), Float(1.5), Float(6.02e23), Float(inf), Float(-inf), Float(nan), Text("1.2.3"), Text("1_000"), Text("true"), Text("2026-10-01")])

The last four values show what stays text: a version number such as 1.2.3, a number with _ separators, anything in quotes, and dates (YAML 1.2 has no timestamp type). tRUE and 0X1 are text too, because the core schema allows only the spellings in the table. In a tree, quote a value whenever it must stay text; Yaml.decode reads any non-null plain scalar into a Str field as written.

Yaml.to_inspect prints a tree in Roc notation, escaping line breaks, tabs and other control characters so each value stays on one line.

3.2.5. Write multi-line text§

Use a block scalar for text that spans lines. | (literal) keeps line breaks; > (folded) joins lines with spaces and keeps a blank line as a paragraph break. A chomping indicator controls the final line break: none keeps one, - removes it, and `` keeps every trailing blank line. A digit sets the content indentation, so the text can start with spaces:

show!(
	\\literal: |
	\\  line one
	\\  line two
	\\folded: >
	\\  joined
	\\  into one line
	\\
	\\  new paragraph
	\\stripped: |-
	\\  no final newline
	\\kept: |+
	\\  trailing blank lines kept
	\\
	\\indented: |2
	\\    starts with two spaces
)

Mapping([{ key: "literal", value: Text("line one\nline two\n") }, { key: "folded", value: Text("joined into one line\nnew paragraph\n") }, { key: "stripped", value: Text("no final newline") }, { key: "kept", value: Text("trailing blank lines kept\n\n") }, { key: "indented", value: Text("  starts with two spaces") }])

3.2.6. Write lists§

A sequence item may start with a mapping on the same line as its - (compact), may hold another sequence the same way, and may sit at the same indentation as its parent key (indentless):

show!(
	\\steps:
	\\- run: build
	\\  name: compile
	\\- - nested
	\\  - compact
)

Mapping([{ key: "steps", value: Sequence([Mapping([{ key: "run", value: Text("build") }, { key: "name", value: Text("compile") }]), Sequence([Text("nested"), Text("compact")])]) }])

Short lists and mappings can also use flow style on one line, as in ports: [80, 443] or labels: { tier: web }.

3.2.7. Quote strings§

A single-quoted string has no escapes except '' for one quote. A double-quoted string supports every YAML 1.2 escape, including \t, \n, \xXX, \uXXXX and \UXXXXXXXX. A # inside quotes is text; outside quotes and after a space it starts a comment:

show!(
	\\single: 'it''s # not a comment'
	\\plain: text # this is a comment
)
show!(
	"double: \"tab\\there, \\u00e9, \\x41\"",
)

Mapping([{ key: "single", value: Text("it's # not a comment") }, { key: "plain", value: Text("text") }])
Mapping([{ key: "double", value: Text("tab\there, é, A") }])

3.2.8. Highlight the original source§

Use Yaml.tokenize_str(text) when you need to display YAML as written. Unlike a decoded tree, its result retains whitespace, comments, quotes, line endings and punctuation. Yaml.tokenize_bytes(bytes) also accepts arbitrary bytes, including invalid UTF-8. Both return this shape:

{
    source : List(U8),
    tokens : List(Yaml.Token),
    diagnostics : List(Yaml.Diagnostic),
    status : [Complete, Recovered, Limited],
}

Each token has kind, start, end, role and scalar fields. start and end are U64 byte offsets into source, with start included and end excluded. They are byte coordinates, not character indexes: a UTF-8 é occupies two bytes and CRLF occupies two bytes. Nonempty tokens cover the entire input in order without gaps or overlap; empty input has no tokens. There is no synthetic end-of-file token. Concatenating the slices reproduces the input byte for byte, including malformed bytes.

The scanner distinguishes these token kinds:

role is Key, Value or Other. scalar is Text, Boolean, Null or Number; it describes lexical spelling without constructing a semantic value or imposing the tree parser’s integer range. Quoted and block scalar contents stay text. Plain scalars follow YAML 1.2 spelling: on is text, false is boolean, and null is null. A mapping key’s role takes precedence for highlighting, so on: and true: both get PropertyName.

Yaml.highlight_category(token) maps a token to Trivia, CommentText, PropertyName, StringText, NumericLiteral, BooleanLiteral, NullLiteral, Symbol, Name, TypeName or InvalidText. Use these categories as theme keys in either HTML or PDF output. The helper returns no rendered markup; your output writer copies the original slices and escapes or encodes them for its own output format.

This example uses CRLF and non-ASCII text, a workflow key, a URL fragment, and a shell script in a block scalar:

workflow = "# CI\r\non: [push]\r\nname: 'Check café'\r\nurl: https://example.com/manual#setup\r\ncontinue-on-error: false\r\nrun: |-\r\n  # Shell text, not a YAML comment\r\n  echo ok\r\n"

# Each piece retains bytes from the original source, including whitespace.
# Both an HTML writer and a PDF writer can consume these same pieces.
source_pieces = |tokenized| tokenized.tokens.map(
	|token| {
		category: Yaml.highlight_category(token),
		bytes: tokenized.source.sublist({ start: token.start, len: token.end - token.start }),
	},
)

copy_source = |tokenized| source_pieces(tokenized).fold([], |bytes, piece| bytes.concat(piece.bytes))

Passing each piece to an output writer keeps the same category and byte range in both renderers. copy_source is a minimal sink that discards the theme and joins the original bytes; a production writer can stream each slice directly. Inspecting a few categories:

tokenized = Yaml.tokenize_str(workflow)
Stdout.line!("preserved: ${Str.inspect(copy_source(tokenized) == workflow.to_utf8())}")?
for token in tokenized.tokens {
	bytes = tokenized.source.sublist({ start: token.start, len: token.end - token.start })
	if bytes == "on".to_utf8() or bytes == "false".to_utf8() or bytes == "https://example.com/manual#setup".to_utf8() {
		Stdout.line!("${Str.from_utf8(bytes) ?? ""}: ${Str.inspect(Yaml.highlight_category(token))}")?
	}
}

preserved: True
on: PropertyName
https://example.com/manual#setup: StringText
false: BooleanLiteral

# starts a comment only in a YAML comment position. Quoted #, a URL’s #setup fragment, and the shell comment inside run: |- stay scalar content. Literal |, folded >, and chomping or indentation indicators remain in their original spelling. GitHub Actions ${{ ... }} expressions remain YAML scalar text; an application can add expression colouring as a second overlay within the returned spans.

3.2.8.1. Recovery and semantic validation§

Tokenization does not require successful decoding. It recognizes anchors, aliases, tags and stream markers even where the semantic parser rejects them. Complete means the lexical scan completed without a diagnosed problem; it does not establish that the YAML is semantically valid. Duplicate keys and unsupported decoding features still need a semantic check when you intend to load configuration.

Each diagnostic has a code and half-open start/end byte coordinates. Codes are UnclosedQuote, InvalidEscape, UnclosedCollection, UnexpectedDelimiter, InvalidIndentation, InvalidBlockHeader, InvalidCharacter, InvalidUtf8 and ResourceLimit. Recovered reports lexical problems while keeping usable tokens and all original bytes. Limited reports more than 256 simultaneously open flow collections. The scanner emits an Invalid token covering the remaining input and a ResourceLimit diagnostic, so callers can display those bytes unchanged. This lexical limit is separate from semantic decoding’s 100-level nesting limit. An unclosed quoted scalar retains the bytes through end of input as one quoted token and reports UnclosedQuote. Invalid escapes stay inside the quoted token with diagnostics. Unmatched closing flow delimiters remain punctuation tokens; unclosed collections report their opening spans through end of input. These recovery choices keep a broken example displayable; they do not repair it or supply a valid semantic value.

# Tokenization succeeds as a source operation even when decoding cannot.
broken = Yaml.tokenize_str("name: \"unfinished")

invalid_utf8 = Yaml.tokenize_bytes([0xFF, 0x0D, 0x0A])

expect copy_source(broken) == broken.source
expect !broken.diagnostics.is_empty()
expect copy_source(invalid_utf8) == [0xFF, 0x0D, 0x0A]
expect !invalid_utf8.diagnostics.is_empty()

If a renderer cannot use a token’s category, it can always copy its source slice with the default text style. Keep source as bytes when exact preservation matters: converting invalid UTF-8 to a string with replacement characters changes those bytes. A text output format may instead show a replacement glyph while retaining the original byte coordinates for reporting diagnostics.

3.2.9. What semantic decoding rejects§

The following are outside the supported subset. Each one is an InvalidYaml error, never a silently different value:

• anchors (&name), aliases (*name) and tags (!!str);

• directives (%YAML) and complex keys (? key);

• more than one document in a file;

• a flow collection ([...] or {...}) that continues onto another line;

• a quoted or plain string that continues onto another line (use a block scalar instead);

• tabs used for indentation;

• duplicate keys in one mapping;

• nesting deeper than 100 levels, counting flow collections;

• in a Yaml tree, integers outside the I64 range (decode them into a wider type such as U64, I128 or Str instead);

• control characters other than tab and line breaks.

Mapping keys are kept as their source text, not resolved like values. 1 and 01 are different keys, and true is the string key "true".

3.2.10. Show where an error is§

InvalidYaml carries a Yaml.Error record: a one-based line and column and a message. Columns count bytes from the start of the line:

report : Str -> Str
report = |text| {
	match Yaml.parse_str(text) {
		Ok(value) => Yaml.to_inspect(value)
		Err(InvalidYaml({ line, column, message })) => "line ${line.to_str()}, column ${column.to_str()}: ${message}"
	}
}

Stdout.line!(report("server:\n\tport: 80"))?
Stdout.line!(report("base: &defaults\n  port: 80"))?
Stdout.line!(report("one: 1\n---\ntwo: 2"))?
Stdout.line!(report("name: a\nname: b"))?
Stdout.line!(report("list: [1,\n  2]"))?
Stdout.line!(report("1: one\n01: zero-one"))?

line 2, column 1: tabs may not be used for YAML indentation
line 1, column 7: anchors, aliases, and tags are not supported by this YAML subset
line 2, column 1: multiple YAML documents are not supported
line 2, column 1: duplicate mapping key `name`
line 1, column 7: unterminated flow collection
Mapping([{ key: "1", value: Text("one") }, { key: "01", value: Text("zero-one") }])

For the full module interface, see the Yaml API reference.

3.2.11. Performance§

Parsing takes time linear in the size of the document, at about 51–55 MB/s on an Apple M2 from 4 KiB to 1 MiB: a 4 KiB configuration in under 0.1 ms and a 1 MiB one in about 20 ms. That is about as fast as yaml-rust2 (Rust), 1.6 times faster than serde_yaml, 2.6 times faster than Go’s yaml.v3 and seven times faster than PyYAML with libyaml. Nesting is limited to 100 levels. Decoding into a record builds the same document first, then walks it once. See Performance for the measurements and the comparison with other libraries.

3.3. Read XML documents§

This chapter shows how to parse an XML document into a tree, find elements, attributes and text in it, and report malformed input with a line and column. It is for Roc programmers who have added parser to their app’s dependencies and need data out of XML such as feeds, SVG files or configuration.

The parser checks that a document is well-formed according to XML 1.0 and gives you the tree after the usual XML clean-up: references replaced by their text, line endings normalised, comments dropped. It does not validate against a schema or DTD, and it rejects documents that contain a <!DOCTYPE>; see what is not supported before using it on documents from other tools.

Every example on this page comes from docs/examples/xml-guide.roc, which CI runs, and imports parser.Xml.

3.3.1. Parse a document and find elements§

Xml.parse_str takes the whole document and returns an Xml record with the document’s root element and its declaration. Each node in the tree is one of:

Node := [
    Element({ name : Str, attributes : List(Attribute), children : List(Node) }),
    Text(Str),
]

An Element holds its name, its attributes in source order, and its children. There is no query language, but Xml.Node has four helpers for the common lookups, and you can pattern match on the tree for anything else:

• node.name() is Ok(name) for an element and Err(NotAnElement) for text.

• node.attribute(name) is the attribute’s value, or Err(Missing).

• node.children_named(name) lists the direct child elements with that name.

• node.text() joins all text inside the node, like the DOM’s textContent.

describe_entry : Xml.Node -> Str
describe_entry = |entry| {
	id = entry.attribute("id") ?? "?"
	titles = Str.join_with(entry.children_named("title").map(|title| title.text()), "")

	"entry ${id}: ${titles}"
}

Given this document:

feed_text =
	\\<?xml version="1.0" encoding="UTF-8"?>
	\\<feed lang="en">
	\\  <!-- newest first -->
	\\  <entry id="2"><title>Fish &amp; Chips</title></entry>
	\\  <entry id="1"><title><![CDATA[<Hello>]]> &#x1F600;</title></entry>
	\\</feed>

parse it and describe each entry element under the root:

match Xml.parse_str(text) {
	Ok(xml) =>
		for entry in xml.root.children_named("entry") {
			# Keep the displayed output portable across documentation fonts.
			Stdout.line!(describe_entry(entry).replace_each("\u(1F600)", "\\u(1F600)"))?
		}

	Err(InvalidXml(problem)) => Stdout.line!("invalid XML: ${problem.message}")?
}

entry 2: Fish & Chips
entry 1: <Hello> \u(1F600)

The whitespace between elements is kept as Text nodes; children_named skips them because it only returns elements. &, the CDATA section and the character reference 😀 have already been turned into ordinary text.

3.3.2. What the tree contains§

The tree is what an XML processor reports, not the literal source:

• The five predefined entities < > & ' " and decimal (A) or hexadecimal (B) character references are replaced by their characters, in text and in attribute values.

• CDATA sections become text, and adjacent text, including text on both sides of a comment, is merged into one Text node.

• Line endings (\r\n and lone \r) become \n.

• In attribute values, tabs and line breaks become spaces.

• Comments and processing instructions are checked and then dropped.

• The XML declaration is available as declaration: Ok({ version, encoding }) or Err(Missing). The version is a record such as { major: 1, minor: 0 }, and the encoding is Err(Missing) when the declaration does not name one.

This document has every one of these in a single element:

print_tree!("<p class='a\tb'>x &lt; y &#65;&#x42;\r\n<![CDATA[<raw>]]><!-- gone -->z<br/></p>")?

Printing the root with Str.inspect gives:

Element({ attributes: [{ name: "class", value: "a b" }], children: [Text("x < y AB
<raw>z"), Element({ attributes: [], children: [], name: "br" })], name: "p" })

3.3.3. Report malformed input§

When a document is not well-formed, Xml.parse_str returns InvalidXml({ line, column, message }). Lines and columns are one-based, and columns count UTF-8 bytes, so a column after non-ASCII text is larger than the number of characters a user sees.

check : Str -> Str
check = |text| {
	match Xml.parse_str(text) {
		Ok(_) => "well-formed"
		Err(InvalidXml({ line, column, message })) => "${line.to_str()}:${column.to_str()}: ${message}"
	}
}

Stdout.line!(check("<a>\n  <b>\n</a>"))?
Stdout.line!(check("<a>&nbsp;</a>"))?
Stdout.line!(check("<a x='1' x='2'/>"))?
Stdout.line!(check("<a/><b/>"))?
Stdout.line!(check("<!DOCTYPE html><html/>"))?

3:1: end tag </a> does not match start tag <b>
1:4: undeclared entity &nbsp;
1:10: duplicate attribute x
1:5: unexpected content after the root element
1:1: document type declarations are not supported

If you are combining XML with other parsers, Xml.parser is the same parser as a Parser(Utf8.Bytes, Xml). It leaves any input after the document for the next parser. Its failures are ParseError({ message, offset }), with the same message as Xml.parse_str and the byte offset of the problem.

3.3.4. What is not supported§

These limits are by design:

Document type declarations

A <!DOCTYPE ...> is rejected. As a result only the five predefined entities exist, and any other reference, such as  , is an error. Replace HTML entities with character references ( ) before parsing.

Namespaces

Names are kept exactly as written. svg:path is an element named "svg:path", and xmlns declarations are ordinary attributes; prefixes are not resolved to URIs.

Declared encodings

The input is a Roc Str, which is already UTF-8. An encoding="..." in the declaration is reported in declaration but not used. Convert other encodings to UTF-8 before parsing.

Validation

Only well-formedness is checked. Required elements, attribute types and schemas are not.

For the full module interface, see the Xml API reference.

3.3.5. Performance§

Xml.parse_str builds the tree at about 120–135 MB/s on an Apple M2, and the rate stays flat from 4 KiB to 1 MiB documents: the parser is linear in the input size. On the generated documents that is about 2.2 times as fast as Go’s encoding/xml, and 1.4–2 times slower than Rust’s quick-xml and roxmltree. Character data and attribute values are found 16 bytes at a time with SIMD, the input is validated as UTF-8 once, and names, text and attribute values with nothing to unescape are slices of the input string, so they keep the input alive. Elements are parsed with an explicit stack, so 5,000 levels of nesting are safe, and duplicate-attribute checks are linear: an element with 5,000 attributes parses 19 times faster than with the Rust libraries. See Performance for the measurements and caveats.

3.4. Read and write JSON§

This chapter shows how to read a JSON document into a value whose shape you do not have to know in advance, walk it, report mistakes with a line and column, choose how strict to be, and write a value back out. It is for Roc programmers who have added parser to their app’s dependencies and need to handle JSON that a record type cannot describe: software bills of materials, JSON Schema files, configuration with comments, or data that must pass through unchanged.

By default the parser is strict: it accepts only the JSON of RFC 8259, and also rejects repeated keys and nesting deeper than 128 levels. Options relax it for common variants, such as comments, trailing commas and a byte order mark, and add checks, such as size limits and I-JSON numbers. It reads complete documents held in memory; see what is not supported.

Every example on this page comes from docs/examples/json-guide.roc, which CI checks and runs, and imports parser.JsonValue.

3.4.1. The builtin Json or JsonValue§

Roc auto-imports a builtin module named Json. It decodes a document straight into a record, list or other type that you name in an annotation, in the way Choosing how to read your input describes. Use it when you know the shape of the document and a Roc type can hold all of it.

Use JsonValue when a record type cannot describe the document, or would lose part of it:

• keys that are not Roc identifiers, such as JSON-LD’s @context or CycloneDX’s bom-ref;

• arrays whose items differ in shape, such as a licence that is either an SPDX id or an SPDX expression;

• documents of any shape, such as JSON Schema files, or a tool that works on every document;

• data that must pass through unchanged: the order of members, every digit of a number, and a definite answer for repeated keys.

The module is called JsonValue because the builtin already has the name Json: import parser.Json would leave the alias unbound. Its error tag is InvalidJsonValue for the same reason. The builtin reports invalid input as InvalidJson with a Str, and one tag cannot carry two payload types, so the different name lets one function use ? on both modules.

3.4.2. Parse a document and walk it§

JsonValue.parse_str reads a whole document and returns a JsonValue, or InvalidJsonValue with the position of the first problem. A JsonValue is one of:

JsonValue := [
    Null,
    Bool(Bool),
    Number(Str),
    String(Str),
    Array(List(JsonValue)),
    Object(List({ key : Str, value : JsonValue })),
]

An Object keeps its members in source order. A Number holds the number’s text exactly as written; Numbers keep their text explains why and how to convert it. JsonValue has methods for the common lookups:

• get(key) is the value of the first member named key, at(index) the item at a zero-based index, and get_path(path) follows a list of keys and indexes, with indexes written as decimal strings such as "0". Each returns Err(Missing) when there is nothing there.

• keys() lists an object’s keys in order, and is empty for anything else.

• as_str, as_bool, as_list and as_object unwrap one variant or return Err(WrongType). as_i64, as_u64 and as_dec convert a number.

This CycloneDX bill of materials gives each component’s licence either as an SPDX id or as an expression, and names its components with a bom-ref key that no Roc record field could have:

sbom_text =
	\\{
	\\  "bomFormat": "CycloneDX",
	\\  "specVersion": "1.6",
	\\  "components": [
	\\    {
	\\      "bom-ref": "pkg:roc/parser@2.1.0",
	\\      "name": "roc-parser",
	\\      "version": "2.1.0",
	\\      "licenses": [{ "license": { "id": "UPL-1.0" } }]
	\\    },
	\\    {
	\\      "bom-ref": "pkg:roc/unicode@4.3.0",
	\\      "name": "unicode",
	\\      "version": "4.3.0",
	\\      "licenses": [{ "expression": "UPL-1.0 OR MIT" }]
	\\    }
	\\  ]
	\\}

These functions list the components with their licences, trying both shapes:

## A component's licence: an SPDX id, or an SPDX expression.
licence : JsonValue -> Str
licence = |component| {
	match component.get_path(["licenses", "0", "license", "id"]) {
		Ok(String(id)) => id
		_ =>
			match component.get_path(["licenses", "0", "expression"]) {
				Ok(String(expression)) => expression
				_ => "no licence"
			}
	}
}

## One line per component of a CycloneDX document.
components : Str -> Try(List(Str), [InvalidJsonValue(JsonValue.Error), Missing, WrongType])
components = |text| {
	bom = JsonValue.parse_str(text)?
	found = bom.get("components")?
	list = found.as_list()?
	Ok(
		list.map(
			|component| {
				name = component.get("name") ?? Null
				version = component.get("version") ?? Null
				"${name.as_str() ?? "?"} ${version.as_str() ?? "?"}: ${licence(component)}"
			},
		),
	)
}

print_components!(sbom_text)?
Stdout.line!(Str.inspect(JsonValue.parse_str(sbom_text).map_ok(JsonValue.keys)))?

roc-parser 2.1.0: UPL-1.0
unicode 4.3.0: UPL-1.0 OR MIT
Ok(["bomFormat", "specVersion", "components"])

get_path uses each key as it is written, so a key that contains /, . or ~ needs no escaping. A segment applied to an array must be a decimal index without a sign or leading zeros, as in a JSON Pointer.

For anything the methods do not cover, match on the variants. This function collects every string in a document with the keys and indexes that lead to it:

## Every string in a value, with the keys and indexes that lead to it.
strings : JsonValue, Str -> List(Str)
strings = |value, path| {
	match value {
		String(text) => ["${path} = ${text}"]
		Array(items) => {
			var $found = []
			var $index = 0.U64
			for item in items {
				$found = $found.concat(strings(item, "${path}/${$index.to_str()}"))
				$index = $index + 1
			}
			$found
		}

		Object(members) => {
			var $found = []
			for member in members {
				$found = $found.concat(strings(member.value, "${path}/${member.key}"))
			}
			$found
		}

		Null | Bool(_) | Number(_) => []
	}
}

Keeping only the lines that mention bom-ref prints:

/components/0/bom-ref = pkg:roc/parser@2.1.0
/components/1/bom-ref = pkg:roc/unicode@4.3.0

3.4.3. Show where an error is§

InvalidJsonValue carries a JsonValue.Error record: a one-based line and column and a message. Columns count bytes from the start of the line, and a line ends at LF, CR or CRLF. The position is the start of the problem: the first character of a bad number or literal, the comma before a closing bracket, the slash of a comment, or the opening quote of a repeated key.

report : Str -> Str
report = |text| {
	match JsonValue.parse_str(text) {
		Ok(_) => "valid"
		Err(InvalidJsonValue({ line, column, message })) => "${line.to_str()}:${column.to_str()}: ${message}"
	}
}

Stdout.line!(report("{\n  \"name\": \"web\",\n  \"port\": 08080\n}"))?
Stdout.line!(report("[\"café\", tru]"))?
Stdout.line!(report("{\"name\": 'web'}"))?
Stdout.line!(report("[1, 2,]"))?
Stdout.line!(report("// settings\n{}"))?
Stdout.line!(report("{\"a\": 1} {\"b\": 2}"))?
Stdout.line!(report("\"\\ud800\""))?

3:11: leading zeros are not allowed in numbers
1:11: invalid literal: expected true, false or null
1:10: expected a JSON value
1:6: trailing comma is not allowed
1:1: comments are not allowed
1:10: unexpected content after the JSON value
1:2: unpaired surrogate in \u escape

The second error is at column 11 although tru is the tenth character, because é is two bytes in UTF-8. JSON has no single-quoted strings and no comments, and a document holds exactly one value, so text after the first value is an error rather than a second document; Combine with other parsers shows how to read several values. The last input is a \u escape for half of a UTF-16 surrogate pair, with no other half. RFC 8259 leaves such escapes to the implementation. A Roc Str cannot hold one, so JsonValue rejects it unless you ask for U+FFFD instead (see Choose how strict to be).

3.4.4. Numbers keep their text§

JSON does not limit the size or precision of numbers, and parsers disagree about what 18446744073709551616 or 0.1000000000000000055511151231257827 means: many turn every number into a 64-bit float and lose digits. JsonValue does not decide for you. A Number holds the text from the input, so a number read and written back keeps every digit, its sign and the spelling of its exponent. Convert it when you need arithmetic:

• as_i64 and as_u64 accept only JSON integers, without a fraction or an exponent: 1.0 and 1e3 are Err(WrongType), and an integer outside the type’s range is Err(OutOfRange). -0 is zero.

• as_dec converts any JSON number, with or without a fraction or an exponent, that fits a Dec. A number outside the Dec range, or one that needs more than 18 decimal places, is Err(OutOfRange).

• For a floating-point value, pass the text to F64.from_str, which rounds it to an F64.

## How each accessor reads a number's text.
convert : JsonValue -> Str
convert = |value| {
	text =
		match value {
			Number(lexeme) => lexeme
			_ => "not a number"
		}
	"${text}: as_i64 ${Str.inspect(value.as_i64())}, as_u64 ${Str.inspect(value.as_u64())}, as_dec ${Str.inspect(value.as_dec())}"
}

for text in ["42", "-0", "1.50", "1e3", "18446744073709551616", "1e400"] {
	Stdout.line!(convert(JsonValue.parse_str(text) ?? Null))?
}
Stdout.line!(Str.inspect(JsonValue.parse_str("1.50") == JsonValue.parse_str("1.5")))?
Stdout.line!(Str.inspect(F64.from_str("1.50")))?

42: as_i64 Ok(42), as_u64 Ok(42), as_dec Ok(42.0)
-0: as_i64 Ok(0), as_u64 Ok(0), as_dec Ok(0.0)
1.50: as_i64 Err(WrongType), as_u64 Err(WrongType), as_dec Ok(1.5)
1e3: as_i64 Err(WrongType), as_u64 Err(WrongType), as_dec Ok(1000.0)
18446744073709551616: as_i64 Err(OutOfRange), as_u64 Err(OutOfRange), as_dec Ok(18446744073709551616.0)
1e400: as_i64 Err(WrongType), as_u64 Err(WrongType), as_dec Err(OutOfRange)
False
Ok(1.5)

Because values compare by their text, 1.50 and 1.5 are different JsonValue values, as the last line but one shows. Convert both with as_dec to compare them as numbers.

3.4.5. Repeated keys§

RFC 8259 says that the names within an object should be unique, and leaves open what a repeated name means. Readers differ: most keep the last value, some keep the first, and some reject the document. Two programs can then read different data from the same file, so by default JsonValue rejects a repeated key. Keys are compared after their escapes are decoded, so "extension" and "\u0065xtension" are the same key.

The duplicates option chooses what a repeated key means:

Reject

The default. Any repeated key is an error, reported at the repeated key.

CollapseIdentical

Keep one member when every copy has an equal value, and report an error when the values differ. Values are compared as JsonValue values: numbers by their text and objects member by member in order, so 1 and 1.0, or the same members in another order, differ.

KeepFirst

Keep the first member and ignore later copies.

KeepLast

Keep the first member’s position with the last copy’s value, as JavaScript’s JSON.parse and Python’s json.loads do.

## Read `text` with each duplicate mode, and write what each one kept.
modes : Str -> List(Str)
modes = |text| {
	options : List((Str, JsonValue.Options))
	options = [
		("Reject", {}),
		("CollapseIdentical", { duplicates: CollapseIdentical }),
		("KeepFirst", { duplicates: KeepFirst }),
		("KeepLast", { duplicates: KeepLast }),
	]
	options.map(
		|(name, chosen)| {
			match JsonValue.parse_with(text, chosen) {
				Ok(value) => "${name}: ${JsonValue.to_str(value) ?? "?"}"
				Err(InvalidJsonValue({ line, column, message })) => "${name}: ${line.to_str()}:${column.to_str()}: ${message}"
			}
		},
	)
}

print_lines!(modes("{\"id\": 1, \"name\": \"web\", \"id\": 2}"))?
print_lines!(modes("{\"extension\": {\"a\": 1}, \"\\u0065xtension\": {\"a\": 1}}"))?

Reject: 1:26: duplicate key "id"
CollapseIdentical: 1:26: duplicate key "id" with a different value
KeepFirst: {"id":1,"name":"web"}
KeepLast: {"id":2,"name":"web"}
Reject: 1:25: duplicate key "extension"
CollapseIdentical: {"extension":{"a":1}}
KeepFirst: {"extension":{"a":1}}
KeepLast: {"extension":{"a":1}}

CollapseIdentical exists for documents that repeat a key by mistake without changing its meaning. The published SPDX 3.0.1 JSON schema is one: it defines extension and prop_Element_extension twice in the same object, with equal values each time. Strict parsing rejects it; CollapseIdentical reads it and would still report a copy with a different value. The conformance review reads the pinned schema this way (see JSON).

3.4.6. Choose how strict to be§

JsonValue.parse_with(text, options) takes a JsonValue.Options record whose fields all have defaults, so you pass only the fields you change. parse_str(text) is parse_with(text, {}): strict RFC 8259, no repeated keys, and at most 128 levels of nesting. A limit or check that fails is reported as an InvalidJsonValue error at the offending value, like any other error.

The parser keeps its nesting on an explicit stack, so a higher max_depth cannot overflow the call stack while parsing. Other operations do recurse over a value: comparing values with == or CollapseIdentical, hashing, and Str.inspect. The writer refuses more than 128 levels. Raise the limit only as far as your documents need.

3.4.6.1. Read JSON with comments§

Configuration files such as tsconfig.json and editor settings use JSON with comments and trailing commas, often called JSONC:

tsconfig_text =
	\\{
	\\  // Compiler options for the example project.
	\\  "compilerOptions": {
	\\    "target": "ES2022", /* modern output */
	\\    "strict": true,
	\\  },
	\\  "include": ["src/**/*.ts",],
	\\}

## JSON with comments, as in `tsconfig.json` and editor settings.
read_jsonc : Str -> Try(JsonValue, [InvalidJsonValue(JsonValue.Error)])
read_jsonc = |text| JsonValue.parse_with(text, { comments: True, trailing_commas: True })

Reading tsconfig_text with read_jsonc, and then strictly with parse_str, and writing the first result with to_str:

{"compilerOptions":{"target":"ES2022","strict":true},"include":["src/**/*.ts"]}
2:3: comments are not allowed

/* inside a string, as in "src/**/*.ts", is text, not a comment. An unterminated block comment is reported where it opens. These two options do not make the parser read JSON5; see What is not supported.

3.4.6.2. Read files with a byte order mark§

Some Windows editors start a UTF-8 file with a byte order mark. parse_str rejects it, at line 1, column 1. bom: True skips it, and only at the start of the document. This reader also requires the document to be an object, as a configuration file must be:

## A configuration file that may start with a byte order mark, as files
## saved by some Windows editors do, and must hold an object.
read_config : Str -> Try(JsonValue, [InvalidJsonValue(JsonValue.Error)])
read_config = |text| JsonValue.parse_with(text, { bom: True, top_level: Object })

show!(read_config("\u(FEFF){\"port\": 8080}"))?
show!(JsonValue.parse_str("\u(FEFF){\"port\": 8080}"))?
show!(read_config("[8080]"))?

{"port":8080}
1:1: byte order mark is not allowed
1:1: top-level value must be an object

3.4.6.3. Read NaN and Infinity§

Python’s json.dumps writes NaN, Infinity and -Infinity for floats that JSON cannot express, unless it is told not to. non_finite: True reads them as Number text. They are not numbers that as_dec, as_i64 or as_u64 can convert, and the writer refuses them, so what to_str writes is always strict JSON:

## Read metrics written by Python's `json.dumps`, which writes `NaN` and
## `Infinity` for floats that JSON cannot express.
print_loss! : Str => Try({}, _)
print_loss! = |text| {
	match JsonValue.parse_with(text, { non_finite: True }) {
		Ok(metrics) => {
			loss = metrics.get("loss") ?? Null
			Stdout.line!(Str.inspect(loss))?
			Stdout.line!(Str.inspect(loss.as_dec()))?
			Stdout.line!(Str.inspect(JsonValue.to_str(metrics)))?
		}

		Err(_) => Stdout.line!("invalid JSON")?
	}
	Ok({})
}

For {"loss": NaN, "best": -Infinity} this prints:

Number("NaN")
Err(OutOfRange)
Err(InvalidNumber("NaN"))

3.4.6.4. Limit untrusted input§

A document from the network can be built to make a reader do too much work or hold too much memory. Set limits that suit your documents, require the top-level shape you expect, and keep repeated keys rejected:

## Options for documents from an untrusted source.
untrusted : JsonValue.Options
untrusted = {
	max_depth: 32,
	max_items: 1000,
	max_string_bytes: 4096,
	max_number_digits: 40,
	top_level: Object,
	interoperable_numbers: True,
}

show!(JsonValue.parse_with("{\"ids\": [1, 2, 3]}", untrusted))?
show!(JsonValue.parse_with("{\"id\": 9007199254740993}", untrusted))?
show!(JsonValue.parse_with("{\"a\": \"${Str.repeat("x", 5000)}\"}", untrusted))?
show!(JsonValue.parse_with("[1, 2, 3]", untrusted))?

{"ids":[1,2,3]}
1:8: number 9007199254740993 is not interoperable: it is an integer beyond 2^53-1 in magnitude
1:7: string exceeds the limit of 4096 bytes
1:1: top-level value must be an object

9007199254740993 is a valid JSON number, but a JavaScript program would read it as 9007199254740992; interoperable_numbers refuses it so that every reader agrees. The options do not limit the size of the whole document: check Str.count_utf8_bytes before parsing if you need to.

3.4.7. Write JSON§

JsonValue.to_str writes a value as compact JSON, and JsonValue.to_str_with takes JsonValue.WriteOptions:

• indent greater than zero puts each item and member on its own line, indented by that many spaces per level, with a space after each :. Empty arrays and objects are written [] and {}.

• sort_keys: True writes the members of every object in the byte order of their keys, which is Unicode code point order. Otherwise members keep their order.

Set max_bytes in WriteOptions to bound the exact UTF-8 output size, including escaped strings, keys and indentation. It defaults to no limit. The writer checks the size before allocating its output and returns TooLarge(max_bytes) if the budget would be exceeded. It adds no final newline: reserve one byte from your budget if the calling protocol appends one.

expect JsonValue.parse_with("null", { max_bytes: 4 }) == Ok(Null)
expect JsonValue.to_str_with(String("é"), { max_bytes: 4 }) == Ok("\"é\"")
expect JsonValue.to_str_with(String("é"), { max_bytes: 3 }) == Err(TooLarge(3))

For example, a downstream SBOM tool can use a 16 MiB budget for both parsing and writing while keeping its own depth and component limits. Number text is preserved, so large identifiers and precise decimal values do not pass through floating point.

Numbers are written as their text. In strings, only the quote, the backslash and control characters are escaped: \b, \f, \n, \r and \t for those characters and \u00XX (lowercase) for the rest. Other characters, including / and non-ASCII text, are written as they are. The output is byte for byte what Python’s json.dumps(value, ensure_ascii=False, sort_keys=..., indent=...) writes, with the separators (",", ":") when compact and (",", ": ") when indented, for the same number text; the conformance review checks this.

release : JsonValue
release = Object([
	{ key: "name", value: String("roc-parser \"JSON\" café") },
	{ key: "version", value: String("2.2.0") },
	{ key: "sizes", value: Array([Number("1.50"), Number("2E3")]) },
	{ key: "draft", value: Bool(False) },
	{ key: "assets", value: Object([]) },
])

Stdout.line!(JsonValue.to_str(release) ?? "not writable")?
Stdout.line!(JsonValue.to_str_with(release, { indent: 2, sort_keys: True }) ?? "not writable")?
Stdout.line!(Str.inspect(JsonValue.to_str(Array([Number("0x10")]))))?
Stdout.line!(Str.inspect(JsonValue.to_str(Object([{ key: "a", value: Null }, { key: "a", value: Null }]))))?

{"name":"roc-parser \"JSON\" café","version":"2.2.0","sizes":[1.50,2E3],"draft":false,"assets":{}}
{
  "assets": {},
  "draft": false,
  "name": "roc-parser \"JSON\" café",
  "sizes": [
    1.50,
    2E3
  ],
  "version": "2.2.0"
}
Err(InvalidNumber("0x10"))
Err(DuplicateKey("a"))

A value built in your code can hold something that JSON cannot express, and the writer refuses it rather than write text that would read back as something else:

InvalidNumber(text)

A Number whose text is not a JSON number, such as 0x10, +1, 1. or text with spaces around it, including NaN and Infinity.

DuplicateKey(key)

A key repeated in one object.

TooDeep

More than 128 levels of nesting.

TooLarge(max_bytes)

The serialized output would exceed the byte budget.

So what the writer returns always reads back with parse_str as the value you wrote, or, with sort_keys, as its sorted copy. It does not keep the input’s layout: comments, whitespace and the spelling of escapes are gone, so "\u00e9" is written "é".

3.4.8. Check JSON during compilation§

Annotate a string literal with the JsonValue type, and the compiler calls JsonValue.from_quote on it while compiling your program. The literal must be strict JSON; anything else is a compiler error, and the constant needs no parsing at run time:

lock : JsonValue
lock =
	\\{"name": "app",
	\\ "checksum": 340282366920938463463374607431768211455,
	\\ "greeting": "caf\\u00e9 \\ud83d\\ude00"}

In a Roc string literal a backslash starts a Roc escape, in multi-line strings too, so write each backslash of a JSON escape twice: JSON’s \u00e9 is \\u00e9, and \n is \\n. A single \u00e9 is a compiler error, because Roc’s own escape is written \u(E9); a single \n puts a real line break inside the JSON string, which strict JSON rejects.

For a file, import its text as a Str and require a checked conversion in a top-level declaration, as Parse and validate during compilation describes:

import "assets/json-guide-settings.jsonc" as settings_text : Str

settings : JsonValue
settings = Literal.require(
	JsonValue.validate({
		source: settings_text,
		name: "json-guide-settings.jsonc",
		options: { comments: True, trailing_commas: True },
	}),
)

JsonValue.validate takes source, a name used in errors (default "<literal>"), and options, the same JsonValue.Options that parse_with takes (default {}). JSON has no separate lint checks: its extra checks are these options. There is no validate_as; to decode JSON into a record, use the builtin Json.

Stdout.line!(Str.inspect(lock.get("checksum")))?
# Display the non-BMP scalar as an escape while keeping its decoded value.
Stdout.line!(Str.inspect(lock.get("greeting")).replace_each("\u(1F600)", "\\u(1F600)"))?
Stdout.line!(Str.inspect(settings.get_path(["files.exclude", "**/.roc-parser-tmp"])))?
Stdout.line!(Str.inspect(JsonValue.validate({ source: "{\"a\": [1,]}", name: "list.json" })))?

Ok(Number("340282366920938463463374607431768211455"))
Ok(String("café \u(1F600)"))
Ok(Bool(True))
Err(BadQuotedBytes("list.json:1:9: JSON: trailing comma is not allowed"))

The last line shows the message a failed check gives, which the compiler reports with the name, line and column.

3.4.9. Combine with other parsers§

JsonValue.parser is a Parser(Utf8.Bytes, JsonValue) for use with the combinators. It reads the whitespace before a value, one strict JSON value, and the whitespace after it, and leaves the rest of the input for the next parser. JsonValue.parser_with(options) takes the same options as parse_with. Failures are ParseError({ message, offset }), with the same message as parse_str and the byte offset of the problem.

JSON Lines (newline-delimited JSON) logs hold one value per line, which Parser.many reads:

## Every value of a JSON Lines (newline-delimited JSON) log.
json_lines : Str -> Try(List(JsonValue), [ParseError({ message : Str, offset : U64 })])
json_lines = |text| Utf8.parse_str(Parser.many(JsonValue.parser), text)

Stdout.line!(Str.inspect(json_lines("{\"event\": \"start\"}\n{\"event\": \"stop\"}\n")))?
Stdout.line!(Str.inspect(json_lines("{\"event\": \"start\"}\n{\"event\": }\n")))?

Ok([Object([{ key: "event", value: String("start") }]), Object([{ key: "event", value: String("stop") }])])
Err(ParseError({ message: "expected a JSON value", offset: 29 }))

Because each value takes the whitespace after it, this reader also accepts several values on one line; it does not check that each value has a line of its own.

3.4.10. What is not supported§

These limits are by design:

Streaming

The parser reads a complete document held in memory and returns a tree; there is no event or token interface. Numbers, and strings without escapes, share the input’s memory instead of being copied, so a value keeps the whole input alive while any part of it is in use.

JSON5

Only the relaxations listed in Choose how strict to be are available. Single-quoted strings, unquoted keys, hexadecimal numbers, leading or trailing decimal points, a leading ``, and line breaks escaped inside strings are always errors.

Type-directed decoding

There is no JsonValue.decode or validate_as. To decode JSON into your own record type, use the builtin Json.

Other encodings

The input is UTF-8. JsonValue.parser, which takes bytes, rejects a value that contains bytes that are not UTF-8. Convert UTF-16 and UTF-32 documents first.

Queries and schemas

get_path takes a list of keys and indexes, not a JSON Pointer string, and there is no JSONPath or JSON Schema validation.

Preserved layout

Comments, whitespace and the spelling of escapes are dropped, and the writer has no option to escape non-ASCII text.

For the full module interface, see the JsonValue API reference.

3.4.11. Performance§

The parser scans the input bytes once, keeping its nesting on an explicit stack as the XML parser does, so deep nesting cannot overflow the call stack. Repeated keys are found by comparing keys in objects of up to 16 members and with a hash table in larger ones. The module’s tests parse an object with 20,000 members and an array of 100,000 numbers, and the fuzz targets run with a time limit per input, to catch quadratic behaviour. Number text, and strings without escapes, are slices of the input string rather than copies. The benchmark harness compares JsonValue.parse_str with serde_json, Go’s encoding/json and Python’s json; see Benchmarks and Compare equivalent work.

3.5. Markdown§

This chapter shows how to turn Markdown text into a tree of Roc values and how to walk that tree to produce your own output, such as HTML or a table of contents. It is a set of how-to guides for readers who already know basic Roc and have added roc-parser to an app. By the end you can parse a document, read its blocks and inline content, pull out YAML frontmatter, and write a renderer.

The Markdown module follows CommonMark 0.31.2 plus the GitHub Flavored Markdown (GFM) extensions for tables, task list items, strikethrough and extended autolinks. It also reads a leading --- frontmatter block.

3.5.1. Parse a document into blocks§

Parse a whole document with Markdown.parse_str. It returns a List(Markdown): one value per top-level block, in document order.

Markdown has no syntax errors. Every input is a valid document, so Markdown.parse_str returns the blocks directly rather than a Try; text that looks like broken syntax becomes ordinary paragraph text. To use the document parser inside your own combinators, Markdown.parser is the same parser as a Parser(Utf8.Bytes, List(Markdown)).

The following app parses a document that uses each kind of block and prints each block with Str.inspect, which shows what the parser produced.

source =
	\\# Shopping
	\\
	\\Buy these *today*:
	\\
	\\- [x] apples
	\\- [ ] pears
	\\
	\\> Quoted
	\\
	\\```roc
	\\main = 1
	\\```
	\\
	\\| Item | Qty |
	\\| :--- | --: |
	\\| pear | 2 |
	\\
	\\***
	\\
	\\<div>raw</div>

main! = |_args| {
	blocks = Markdown.parse_str(source)
	for block in blocks {
		Stdout.line!(Str.inspect(block))?
	}
	loose_list!()
}

A list in which no items are separated by blank lines is tight; otherwise it is loose. The loose field records this, which matters when you render HTML: a tight list normally shows its item paragraphs without <p> tags. Ordered lists keep their starting number:

loose_list! = || {
	blocks = Markdown.parse_str("3. first\n\n4. second\n")
	for block in blocks {
		Stdout.line!(Str.inspect(block))?
	}
	Ok({})
}

Output:

Heading({ level: One, content: [Text("Shopping")] })
Paragraph([Text("Buy these "), Emphasis([Text("today")]), Text(":")])
ListBlock({ kind: Unordered, loose: False, items: [{ blocks: [Paragraph([Text("apples")])], task: Checked }, { blocks: [Paragraph([Text("pears")])], task: Unchecked }] })
Blockquote([Paragraph([Text("Quoted")])])
Code({ info: "roc", pre: "main = 1
" })
Table({ header: [[Text("Item")], [Text("Qty")]], align: [Left, Right], rows: [[[Text("pear")], [Text("2")]]] })
ThematicBreak
HtmlBlock("<div>raw</div>
")
ListBlock({ kind: Ordered({ start: 3 }), loose: True, items: [{ blocks: [Paragraph([Text("first")])], task: NoTask }, { blocks: [Paragraph([Text("second")])], task: NoTask }] })

The block variants are:

Block quotes and lists nest at most 1,000 levels deep. Deeper > and list markers are read as paragraph text. Emphasis, strikethrough, links and images also nest at most 1,000 levels deep (counted separately); the delimiters and brackets of deeper ones are read as text. Code that walks the tree recursively therefore cannot run out of stack on hostile input.

The syntax tree types support == and hashing, so blocks and inlines can be Dict keys and Set elements.

Link reference definitions ([label]: /url) do not appear in the tree. The parser resolves them into the links that use them.

3.5.2. Parse inline content§

Inline content is the text inside a paragraph, heading or table cell. When you already have a single line or fragment, such as a title from a database, parse it directly with Markdown.parse_inlines, which returns a List(Inline). Markdown.inline_parser is the same parser for use in combinators.

show_inlines! = |text| {
	inlines = Markdown.parse_inlines(text)
	for inline in inlines {
		Stdout.line!(Str.inspect(inline))?
	}
	Ok({})
}

show_inlines!("*em* **strong** ~~gone~~ `x + 1`")?
show_inlines!("[Roc](https://roc-lang.org \"Home\") and ![a cat](cat.png)")?
show_inlines!("<https://example.com> www.example.com me@example.com")?
show_inlines!("one\ntwo  \nthree")?

Output:

Emphasis([Text("em")])
Text(" ")
Strong([Text("strong")])
Text(" ")
Strikethrough([Text("gone")])
Text(" ")
InlineCode("x + 1")
Link({ label: [Text("Roc")], target: { href: "https://roc-lang.org", title: Ok("Home") } })
Text(" and ")
Image({ alt: [Text("a cat")], target: { href: "cat.png", title: Err(Missing) } })
Link({ label: [Text("https://example.com")], target: { href: "https://example.com", title: Err(Missing) } })
Text(" ")
Link({ label: [Text("www.example.com")], target: { href: "http://www.example.com", title: Err(Missing) } })
Text(" ")
Link({ label: [Text("me@example.com")], target: { href: "mailto:me@example.com", title: Err(Missing) } })
Text("one
two")
HardBreak
Text("three")

The output shows these rules:

• ** and __ produce Strong, * and _ produce Emphasis, and ~ or ~~ produce Strikethrough.

• Autolinks in angle brackets and GFM extended autolinks (www., http://, https:// and email addresses written as plain text) become ordinary Link values. The parser adds http:// to www. links and mailto: to email links.

• A soft line break, a plain newline inside a paragraph, stays as "\n" inside Text. A hard line break, two trailing spaces or a backslash before the newline, becomes HardBreak.

• Backslash escapes and entity references such as & are decoded, so Text holds the characters the reader sees, not the source.

• Inline HTML such as <b> becomes HtmlInline(raw).

• A link’s target is { href, title }, where title is Ok(text) or Err(Missing).

3.5.2.1. Reference links need the whole document§

A reference link such as [the guide][guide] points to a definition elsewhere in the document. Markdown.parse_inlines sees only the fragment you give it, so it cannot resolve the reference and leaves the brackets as text. Parse the whole document with Markdown.parse_str instead:

document =
	\\See [the guide][guide] and ![logo][].
	\\
	\\[guide]: https://example.com/guide "The guide"
	\\[logo]: /logo.png

blocks = Markdown.parse_str(document)
for block in blocks {
	Stdout.line!(Str.inspect(block))?
}

Output:

Paragraph([Text("See "), Link({ label: [Text("the guide")], target: { href: "https://example.com/guide", title: Ok("The guide") } }), Text(" and "), Image({ alt: [Text("logo")], target: { href: "/logo.png", title: Err(Missing) } }), Text(".")])

Reference labels match case-insensitively using Unicode case folding, so [Straße] matches a definition labelled [STRASSE].

3.5.3. Read frontmatter§

Many static-site tools put metadata at the top of a Markdown file between two --- lines. When the document’s first line is exactly --- and a later line is exactly ---, Markdown.parse_str returns the lines between them as the first block, Frontmatter(raw), and parses the rest as Markdown. Markdown.frontmatter(blocks) returns that text, or Err(Missing) when the document has none. The parser does not interpret the text. Pass it to the YAML parser (see Read YAML configuration and frontmatter) to read the fields:

post =
	\\---
	\\title: Hello
	\\draft: false
	\\---
	\\# Hello
	\\
	\\First post.

main! = |_args| show!(post)

show! = |text| {
	blocks = Markdown.parse_str(text)
	match Markdown.frontmatter(blocks) {
		Ok(raw) => {
			Stdout.line!("raw: ${Str.inspect(raw)}")?
			meta = Yaml.parse_str(raw)?
			Stdout.line!("meta: ${meta.to_inspect()}")?
			# The frontmatter is the first block.
			Stdout.line!("body blocks: ${(blocks.len() - 1).to_str()}")?
		}

		Err(Missing) => Stdout.line!("no frontmatter")?
	}
	Ok({})
}

Output:

raw: "title: Hello
draft: false
"
meta: Mapping([{ key: "title", value: Text("Hello") }, { key: "draft", value: Bool(False) }])
body blocks: 2

Without a closing --- line, the opening --- is ordinary Markdown: a thematic break, or a Setext heading underline.

3.5.4. Walk the tree to produce output§

To produce output, write two functions: one that matches each block variant and one that matches each inline variant. Each function calls the other for nested content. This section builds a small HTML renderer and a table-of-contents extractor for the following release notes:

blocks = Markdown.parse_str(source)
Stdout.write!(render_blocks(blocks))?
Stdout.line!("--- contents ---")?
for line in table_of_contents(blocks) {
	Stdout.line!(line)?
}

Escape the characters that are significant in HTML:

escape : Str -> Str
escape = |text| {
	text
		.replace_each("&", "&")
		.replace_each("<", "&lt;")
		.replace_each(">", "&gt;")
		.replace_each("\"", "&quot;")
}

Render inline content:

render_inlines : List(Markdown.Inline) -> Str
render_inlines = |inlines| {
	var $out = ""
	for inline in inlines {
		$out = $out.concat(render_inline(inline))
	}
	$out
}

render_inline : Markdown.Inline -> Str
render_inline = |inline| {
	match inline {
		Text(text) => escape(text)
		Strong(children) => "<strong>${render_inlines(children)}</strong>"
		Emphasis(children) => "<em>${render_inlines(children)}</em>"
		Strikethrough(children) => "<del>${render_inlines(children)}</del>"
		InlineCode(code) => "<code>${escape(code)}</code>"
		Link({ label, target }) => "<a href=\"${escape(target.href)}\"${title_attr(target.title)}>${render_inlines(label)}</a>"
		Image({ alt, target }) => "<img src=\"${escape(target.href)}\" alt=\"${escape(plain_text(alt))}\"${title_attr(target.title)} />"
		HardBreak => "<br />\n"
		# Raw HTML passes through unchanged; see the warning above.
		HtmlInline(raw) => raw
	}
}

title_attr : Try(Str, [Missing]) -> Str
title_attr = |title| {
	match title {
		Ok(text) => " title=\"${escape(text)}\""
		Err(Missing) => ""
	}
}

Render blocks. A tight list item is rendered without its <p> wrapper:

render_blocks : List(Markdown) -> Str
render_blocks = |blocks| {
	var $out = ""
	for block in blocks {
		$out = $out.concat(render_block(block))
	}
	$out
}

render_block : Markdown -> Str
render_block = |block| {
	match block {
		Heading({ level, content }) => {
			n = level.to_str()
			"<h${n}>${render_inlines(content)}</h${n}>\n"
		}

		Paragraph(inlines) => "<p>${render_inlines(inlines)}</p>\n"
		Blockquote(children) => "<blockquote>\n${render_blocks(children)}</blockquote>\n"
		ListBlock({ kind, loose, items }) => {
			tag =
				match kind {
					Unordered => "ul"
					Ordered(_) => "ol"
				}
			var $body = ""
			for item in items {
				# A tight list shows its paragraphs without <p> tags.
				inner =
					match item.blocks {
						[Paragraph(inlines)] if !loose => render_inlines(inlines)
						_ => render_blocks(item.blocks).trim()
					}
				$body = $body.concat("<li>${inner}</li>\n")
			}
			"<${tag}>\n${$body}</${tag}>\n"
		}

		Code({ pre, .. }) => "<pre><code>${escape(pre)}</code></pre>\n"
		ThematicBreak => "<hr />\n"
		HtmlBlock(raw) => raw
		# Tables, frontmatter and the remaining variants are left out of this sketch.
		_ => ""
	}
}