Testing

Fern ships with a pure-Fern unit-test runner under std/test. Tests are ordinary .fern files: write functions that return Option[string], list them in main, and the runner prints TAP-13 to stdout.

The runner doesn’t depend on the Go toolchain, so the same program runs under fern -interp, compiles to a native binary, and (once the compiler is self-hosted) keeps working unchanged.

Hello, test

A test function returns None when the assertion holds and Some(message) when it fails. The auto-prelude makes TestRunner, test_new, and every assert_* helper available by bare name — no imports needed.

▸ live snippet open in playground →

Save as hello_test.fern and run it:

fern -interp hello_test.fern

Output:

TAP version 13
# Suite: hello
ok 1 - addition
ok 2 - strings
1..2
# tests 2
# pass 2
# fail 0

Exit code is 0. A failing assertion flips it to 1, prints a not ok line, and embeds the actual and expected values in the diagnostic block.

Assertions

Each helper returns Option[string]. The failure message names the predicate, so a grep across CI logs finds the regression.

| Family | Helpers | | ------------ | -------------------------------------------------------------------------------------------------------------------- | | Boolean | assert_true, assert_false | | Integers | assert_eq_i32, assert_neq_i32, assert_lt_i32, assert_le_i32, assert_gt_i32, assert_ge_i32 | | Wider ints | assert_eq_i64, assert_neq_i64, assert_eq_u32, assert_neq_u32, assert_eq_u64, assert_neq_u64 | | Wider relational | assert_lt_i64 / _le_i64 / _gt_i64 / _ge_i64 plus matching _u32 / _u64 variants | | Floats | assert_eq_f32_near, assert_eq_f64_near, assert_eq_f64_exact, assert_is_nan_f32, assert_is_nan_f64 | | Range | assert_in_range_i32, assert_in_range_i64 | | Multi-substr | assert_contains_all, assert_contains_any, assert_contains_in_order | | String diff | assert_eq_string_diff — reports first differing line | | File state | assert_file_exists, assert_file_not_exists, assert_file_contains, assert_file_contents, assert_is_file, assert_is_dir, assert_file_size | | JSON | assert_json_eq — parses both sides + walks the trees, JObject key order independent | | Map | assert_map_len_*, assert_map_has_*, assert_map_lacks_* for (i32, i32) + (string, string) K/V pairs | | Golden files | assert_matches_golden(path, actual) (auto-bootstrap) / assert_matches_golden_strict(...) (CI strict) | | Booleans | assert_eq_bool | | Strings | assert_eq_string, assert_neq_string, assert_empty_string, assert_non_empty_string | | Substring | assert_contains, assert_not_contains, assert_starts_with, assert_ends_with | | Array length | assert_len_i32, assert_len_string | | Array eq | assert_eq_i32_array, assert_eq_string_array | | Process | assert_exit, assert_stdout_eq, assert_stderr_eq, assert_stdout_contains, assert_stderr_contains, assert_process | | Free-form | pass(), fail(msg) |

Skipping cases

Toolchain-conditional cases (wasmtime missing, OS-specific behaviour, CI-only checks) skip with (r).skip(name, reason). The TAP output emits # SKIP so consumers can tell a skip from a pass; skipped cases don’t affect the exit code.

function main(): i32 {
    var r: TestRunner = test_new("backends");
    r = r.skip_if(env("WASMTIME_PATH").is_none(),
                  "wasm e2e", "wasmtime not on $PATH",
                  run_wasm_test());
    return r.finish();
}

Nested suites

Group related cases with r.subsuite(name) + r.merge(child). TAP stays flat (one stream, monotonically numbered) but case names print as parent / child / case, so the hierarchy is visible in the log.

var arm: TestRunner = r.subsuite("arm64");
arm = arm.it("exit code", run_arm64_exit());
arm = arm.it("stdout",    run_arm64_stdout());
r = r.merge(arm);   // fold counts back into parent

Spawning subprocesses

subprocess(cmd, args, stdin) runs an external program and returns a ProcessResult { stdout, stderr, exit_code }. The assertion family above (assert_exit / assert_stdout_eq / assert_process) operates on that struct.

function test_fern_hello(): Option[string] {
    var r: ProcessResult = subprocess("./fern",
        ["-interp", "/tmp/hello.fern"], "");
    return assert_process(r, 0, "hello");
}

Spawn failures (binary missing, permission denied) surface as exit_code = 127 with the OS error in stderr, so callers can assert linearly without an extra Result unwrap.

Tempdirs + cleanup hooks

Tests needing a scratch directory call temp_dir(prefix) and register cleanup at the runner level:

match (temp_dir("my-fixture")) {
    Ok(dir) => {
        r = r.defer_cleanup(dir);
        // ... write fixtures into `dir`, spawn subprocesses ...
    },
    Err(_) => { r = r.skip("setup", "temp_dir failed"); }
}

finish() runs remove_dir_all(dir) on every registered path regardless of test outcome. Cleanup errors print as TAP comments and shift the exit code to 2 (a “tests passed but cleanup leaked” sentinel) so CI can tell it apart from a real test failure.

Companion filesystem builtins: read_dir(path) -> Result[string[], IoError] lists immediate children; remove_file(path) and remove_dir_all(path) return Option[IoError] (None on success).

Fuzzing

A fuzz target is a (string) => Option[string] — the same shape as a regular test, called repeatedly with mutated inputs from a seed corpus.

function check_to_upper_idempotent(input: string): Option[string] {
    if (input.to_upper().to_upper() == input.to_upper()) { return None; }
    return Some("to_upper is not idempotent");
}

function main(): i32 {
    var r: TestRunner = test_new("fuzz");
    r = r.fuzz("to_upper idempotent",
               ["", "abc", "Hello"],
               fuzz_default_iterations(),
               check_to_upper_idempotent);
    return r.finish();
}

The harness mutates with byte flip / drop / insert / unchanged. Failures stop the loop and report the input with non-printable bytes escaped to \xNN, so the log doubles as a reproducer.

Filtering test cases

For dev-loop iteration, run a single case (or a substring match) via the runner’s --filter flag:

fern -interp my_test.fern -- --filter addition

The example file lifts the pattern off args():

function main(): i32 {
    var f: string = parse_filter_from_args(args());
    var r: TestRunner = test_new_filtered("my suite", f);
    r = r.it("addition basic",    test_a());
    r = r.it("subtraction basic", test_b());
    return r.finish();
}

Non-matching cases convert to skips with reason "filtered out", so the TAP stream stays a faithful record of the test list.

Golden-file testing

For tests whose expected output is too verbose to inline, compare against a saved file on disk:

r = r.it("renderer matches golden",
         assert_matches_golden("testdata/render.golden",
                               render(input)));

assert_matches_golden bootstraps the file when it doesn’t exist (writes actual to path, passes, prints a comment naming the bootstrap) — the “approve by running” loop. Use assert_matches_golden_strict in CI to catch missing-golden cases that would silently self-approve.

Shrinking

r.fuzz_shrink runs the same loop but, on a failure, minimises the offending input via halving + single-byte drops before reporting. The failure message embeds both the raw input (what the mutator produced) and the shrunk form (the smallest still-failing variant found):

not ok 1 - detect
  ---
  message: seed[0]: forbidden
  raw     (42 bytes): "lots of padding here BAD lots more padding"
  shrunk  (3 bytes): "BAD"
  ---

Use the shrink variant when the property’s trigger is much smaller than the input the corpus + mutator produces — typical for parser fuzz targets where any single malformed byte breaks the property.

Writing your own assertion

A test function can run any logic that returns Option[string] — the assertions above are conveniences. For a hand-rolled check, return fail(...) directly:

function test_modulo_table(): Option[string] {
    var inputs: i32[] = [0, 1, 5, 10, 11];
    var expected: i32[] = [0, 1, 5, 0, 1];
    var i: i32 = 0;
    while (i < len(inputs)) {
        if (inputs[i] % 10 != expected[i]) {
            return fail("table mismatch at " + i.to_string());
        }
        i = i + 1;
    }
    return pass();
}

Why explicit registration?

Fern has no reflection, so main lists the cases it runs. That mirrors Zig’s test blocks and keeps the runner a plain library — no build-system magic, no framework runtime. A future discover_tests() macro would lower to the same r.it("name", func()) chain.

Failing tests in CI

The process exit code is 0 on a clean run and 1 on any failure. Drop it into any CI runner — GitHub Actions, GitLab CI, Buildkite — without further configuration. TAP consumers (prove, tape, tap-junit) parse the stream directly.

Migrating from Go-side tests

Today the compiler’s own tests live in Go (internal/e2e/*.go). The plan (docs/ROADMAP-AND-SELF-HOSTING.md) is to retire them once the compiler is self-hosted; write new tests for stdlib helpers and Fern code against std/test so they’re already in the right shape for the migration.

See the working examples under examples/tests/ in the repo:

• arithmetic_test.fern — basic integer assertions
• strings_test.fern — string-method coverage
• runner_self_test.fern — the runner’s meta-test (verifies every helper on both pass and fail paths)
• skip_and_subsuites_test.fern — skip / skip_if / subsuite / merge end-to-end
• process_assertions_test.fern — subprocess + temp_dir
  • the process assertion family (echo / cat / sh fixtures)
• fuzz_example_test.fern — three benign property fuzzers plus a trim-strips-spaces invariant
• wide_numerics_test.fern — i64 / u32 / u64 assertions
• filesystem_ops_test.fern — read_dir / remove_file / remove_dir_all builtins
• fern_binary_e2e_test.fern — the canonical migration shape: spawn the fern binary itself, drive it through -interp / -check against tempdir fixtures, assert on (exit, stdout, stderr). Gated on $LANG_BIN being set so dev laptops without the env var skip cleanly.
• helpers_test.fern — multi-substring / string-diff / range / file-state / must_temp_dir convenience helpers
• float_test.fern — f32 / f64 tolerance + exact + NaN + f32_bits round-trip coverage
• fuzz_shrink_test.fern — r.fuzz_shrink walkthrough on three benign properties; the failure-path shrinking contract is pinned by the Go-side TestRunnerFuzzShrinkSurfacesMinimisedInput gate
• batch7_test.fern — wider-int relational, f64_bits round-trips, stat() + file-state helpers, and assert_json_eq order-independent JSON equality
• batch8_test.fern — argv passthrough, --filter PATTERN selection, golden-file assertions, Map receiver assertions
• float_math_test.fern — (x: f64).sqrt() / .pow() / .log() etc., the IEEE-754 classification helpers, and the round-trip property tests (exp(log(x)) == x, sin² + cos² = 1)
• timing_test.fern — monotonic_ns() / now_unix_ms() / sleep_ms() builtins + assert_elapsed_lt_ms for benchmark-style “completes within N ms” assertions