Capture

Screenshots and recordings of the app's rendered output.

Recorded frames are captured from the framebuffer at the end of each frame, and so are the size of the window. Supported outputs are PNG image sequences, GIF, and WebM. A still image can also be exported from an offscreen Draw.RenderTexture of any size, which is how output larger than the window is produced.

Every path here is relative to the output directory set with App.default.with_output_dir, and one that would escape it -- absolute, or containing .. -- is refused rather than rewritten. Capture is the only path-sandboxed writer the platform grants: Files.write_text! and Files.write_bytes! write wherever the process may write, while everything here is confined to the output directory.

Recording starts and stops through Capture.start! and Capture.stop!, a single frame is written by Capture.screenshot!, an offscreen Draw.RenderTexture is written at its own size by Capture.screenshot_texture!, and recording state is available as input.capture each cycle.

Pixels also come back the other way, without becoming a file. pixel_at! reads one pixel and read_region! reads a rectangle, from either the last presented frame or a Draw.RenderTexture; both are refused in render!, and both say what they cost.

The types and pure helpers live in the companion roc-ray-types package so reusable packages can depend on them without depending on this platform. This module re-exports them, so Recording here and in the package are the same nominal type.

screenshot! : Str => Try({  }, ScreenshotError)

Write one PNG of the app's rendered output.

The framebuffer is read back at the end of the frame that asked -- after the draw batch is flushed and before the buffers are swapped, so the pixels are the ones just drawn -- and the PNG is encoded and written off the frame thread. This call waits for that write, so it parks the task until the file exists and answers with the write's own outcome.

Legal only in a task, where it parks the task; refused in init!, update!, and render!. Every other waiting effect also works in init!, where it blocks; a screenshot cannot, because what it waits for is the end of a frame and init! runs before the frame loop has drawn one. Spawn a task from update! instead -- on the first cycle if the shot is meant to be of the first frame:

if input.time.cycle_count == 0 {
    Task.spawn!(input, || Shot(Capture.screenshot!("frame0.png")))
}

A headless run has no framebuffer at all and answers Ok({}) without writing, so a screenshotting app still runs under --host-headless.

Only one screenshot can be in flight: a second one while the first is still waiting for its frame is AlreadyPending.

screenshot_texture! : RenderTexture, Str => Try({  }, TextureExportError)

Write one PNG of what a render target holds, at the target's own size.

This is how an app exports an image larger than its window: draw the composition into a Draw.RenderTexture of the size the output needs and export the target rather than the frame. Transparency survives, unlike a screenshot!, whose framebuffer readback is always opaque.

Legal in init!, where it blocks startup, and in tasks, where it parks the task; refused in update! and render!. Nothing here waits on the frame loop, which is why init! is allowed where a screenshot! cannot be -- but drawing into a target is only possible during render!, so a target no render! has drawn into holds undefined pixels.

The pixels are the ones the last completed render! left in the target, so an app that draws its composition every frame exports what it last showed. The path is resolved under the output directory exactly as screenshot! resolves one.

A headless run has no pixels to read and answers Ok({}) without writing, so an exporting app still runs under --host-headless.

Task.spawn!(input, || Exported(Capture.screenshot_texture!(poster, "poster.png")))
pixel_at! : Source, { x : I32, y : I32 } => Try(Rgba, PixelReadError)

Read the colour of one pixel.

Legal in init!, update!, and tasks; refused in render!.

This is the cheap read. A point needs no allocation and no delivery slot, which a one-pixel read_region! would still take for the same four bytes, so a colour picker should ask for a point.

picked = match Capture.pixel_at!(Screen, { x: 40, y: 24 }) {
    Ok(color) => color
    Err(_) => Color.black
}

Reading Screen costs one full framebuffer readback per frame for as long as the app keeps reading: the host snapshots the frame it presents so that a later update! has defined pixels to look at, and stops snapshotting once a frame goes by with no read. Nothing is snapshotted before the first read, so the first Screen read of a run -- and every Screen read from init!, which runs before any frame -- is Unavailable, and the next cycle's read is not.

Reading Target costs one readback of the whole target per call, however small the point, because that is what the graphics API will give. Read a region once rather than a point many times.

A headless run has no pixels of any kind and answers Unavailable, so an app that reads pixels has to say what it does without them before it can run under --host-headless.

read_region! : Source, Region => Try(List(U8), PixelReadError)

Read a rectangle of a source as packed RGBA8 bytes.

Legal in init!, update!, and tasks; refused in render!.

The bytes are row-major and top-down: four bytes per pixel in red, green, blue, alpha order, width pixels per row, the source's topmost requested row first. A Screen region is always opaque, because reading the framebuffer forces alpha; a Target region keeps the alpha the app drew.

strip = Capture.read_region!(Target(poster), { x: 0, y: 0, width: 64, height: 1 })?

The list is handed over rather than copied, so a large region costs no second buffer -- and, like every other handed-over byte list, it occupies one of a bounded number of delivery slots until the app drops it. A read with no slot free is Busy.

The bound is Capture.max_readback_bytes: a region above it is RegionOutOfBounds rather than something a later frame could take. The per-call and per-frame costs are the ones pixel_at! describes; this is not a per-frame operation on a whole window.

max_readback_bytes : U64

Most RGBA bytes one read_region! may deliver: 128 mebibytes, which is 8192 by 4096 pixels.

screenshot_texture! draws on the same budget, for the same reason: each holds one whole RGBA image in host memory while it works, so the two cannot both be given the whole of it.

default : Recording

A 25 FPS half-scale GIF of at most 300 frames, using fixed-input timing and balanced encoder quality.

start! : Recording => {  }

Begin recording.

Legal in init!, update!, and tasks; refused in render!.

Frames accumulate until the recording hits its frame cap, Capture.stop! is called, or the app exits -- all three finalize the file.

A rejected start appears as Failed in input.capture on the next cycle; the call itself reports nothing, so the recording's outcome is observed the same way whichever phase started it.

stop! : () => {  }

Finish the current recording and write its file.

Legal in init!, update!, and tasks; refused in render!. An encode and a file write would otherwise land in the middle of drawing a frame.

Stopping while idle does nothing. The next input reports the frame count and file size as Finished.

Format : Format

Container and codec written for a capture.

Scale : Scale

How far each captured frame is downscaled from the framebuffer.

Timing : Timing

Whether simulation time follows the wall clock or advances in exact steps.

Cursor : Cursor

Whether the host composites a pointer glyph into captured frames.

Quality : Quality

How hard the encoder works to choose colours for each frame.

Recording : Recording

A validated recording request. Update it through its receivers.

Declared in the roc-ray-types package's Capture and re-exported here, which is also where its receivers are documented.

Status : Status

Live recording state, sampled onto every App.Input as input.capture.

Finished remains observable after automatic finalization at the frame cap.

FailureReason : FailureReason

Why a recording is not running.

PathInvalid, PathEscapesOutputDir, AlreadyRecording, and BudgetExceeded reject a start request before anything is written. The remaining reasons may stop an active recording.

ScreenshotError : [PathInvalid, PathEscapesOutputDir, AlreadyPending, WriteFailed, Busy, Unavailable]

Why a screenshot did not become a file.

PathInvalid is a path the sandbox cannot even resolve: empty, absolute, containing .., or holding a NUL byte. PathEscapesOutputDir is the resolvable path that lands outside the output directory. Both are the app's own string to fix.

Busy is the host at its limit across every capture request at once, so nothing was captured or written. It is about right now: the same screenshot offered on a later frame may well be taken, which is what separates it from Unavailable. AlreadyPending is narrower still -- this app's own previous screenshot is still waiting for its frame.

Unavailable is the host having no capture facility to use at all -- the app is shutting down and the wait was cancelled before the frame ended. It is not what a call from the wrong callback gets: that is a programmer error and stops the app. See screenshot!.

TextureExportError : [
    PathInvalid,
    PathEscapesOutputDir,
    TargetUnavailable,
    BudgetExceeded,
    Busy,
    OutOfMemory,
    ReadbackFailed,
    WriteFailed,
    Unavailable,
]

Why an offscreen export did not become a file.

TargetUnavailable is a render target that no longer resolves to a host resource -- a released one, or the Draw.RenderTexture.stub a pure test holds. BudgetExceeded is an image too large for the host to hold at all, so retrying will not help; Busy is one that would fit were other exports not in flight, so a later frame may take it. Unavailable is the app shutting down before the write started.

Source : [Screen, Target(RenderTexture)]

Where a pixel readback takes its pixels from.

Screen is the last frame the host presented, which is the frame on screen while update! runs, at the framebuffer's own size. Target is an offscreen Draw.RenderTexture, holding whatever the last completed render! drew into it, at the target's own size.

Region : {
    x : I32,
    y : I32,
    width : I32,
    height : I32,
}

A rectangle of pixels, in pixels right and down from the source's top-left corner.

PixelReadError : [RegionOutOfBounds, TargetUnavailable, Busy, ReadbackFailed, Unavailable]

Why a readback produced no pixels.

RegionOutOfBounds is a point or rectangle that is not entirely inside the source, or a rectangle larger than one read may deliver; neither becomes possible on a later frame. TargetUnavailable is a render target that no longer resolves to a host resource -- a released one, or the Draw.RenderTexture.stub a pure test holds. Busy is the readback budget being committed elsewhere -- to still exports in flight, or to the delivery slots that carry byte lists to this app -- or, for a render target bigger than max_readback_bytes on its own, committed for good. ReadbackFailed is the graphics driver declining to hand the pixels over. Unavailable is there being nothing to read at all -- a headless run, or a Screen read before the host has a presented frame to read.