fluxbench

Benchmarks that survive panics, with statistics you can gate CI on

Runs each benchmark in a worker process and reports bootstrap confidence intervals and percentiles. Critical #[verify] rules exit 1 on a failed check.

Docs · Crate · Benchmarks · Verification · Documentation · Example · Contributing

FluxBench is a benchmarking framework for Rust. A benchmark is a function with #[bench]. cargo bench runs it in a worker process, so a panic in one benchmark does not stop the suite. Each run reports bootstrap confidence intervals, percentiles, and cycle counts. Attributes add verification rules, derived metrics, comparison tables, and report output.

Quick Start

FluxBench requires Rust 1.85 or later.

1. Add the dependency

cargo add --dev fluxbench

2. Configure the bench target

# Cargo.toml
[[bench]]
name = "my_benchmarks"
harness = false

3. Write benchmarks

Create benches/my_benchmarks.rs:

use fluxbench::prelude::*;
use std::hint::black_box;

#[bench]
fn addition(b: &mut Bencher) {
    b.iter(|| black_box(42) + black_box(17));
}

#[bench(group = "compute")]
fn fibonacci(b: &mut Bencher) {
    fn fib(n: u32) -> u64 {
        if n <= 1 { n as u64 } else { fib(n - 1) + fib(n - 2) }
    }
    b.iter(|| black_box(fib(20)));
}

fn main() {
    fluxbench::run().unwrap();
}

Benchmarks can also live in examples/. See Bench function.

4. Run benchmarks

cargo bench
cargo bench -- --group compute --warmup 5 --measurement 10

Run cargo bench -- --help for all flags. See the CLI reference.

Features

Feature	Description
Crash isolation	A panic, signal, or timeout fails one benchmark. The suite continues.
Bootstrap statistics	BCa confidence intervals, p50 to p999 percentiles, cycle counts, outlier detection
Verification	#[verify(expr = "bench_a < bench_b", severity = "critical")]
Synthetic metrics and comparisons	#[synthetic(...)] derived metrics and #[compare(...)] tables and series charts
Output formats	Human, JSON, HTML, CSV, GitHub summary, and github-action-benchmark JSON
CI integration	Exit code 1 on critical failures. Severity levels in flux.toml.
Allocation tracking	Per-iteration heap bytes and count
Throughput	Bytes per second (MiB/s, GiB/s) or elements per second
Batched inputs and async	iter_batched, iter_batched_ref, BatchSize, iter_custom, and Tokio runtimes
Config	flux.toml with CLI override. Priority: bench > group > CLI > file > default.

Defining Benchmarks

Attributes compose. Add one, several, or all of them. Benchmarks do not need restructuring when you add an attribute.

#[bench(group = "io", tags = "network")]
fn http_request(b: &mut Bencher) {
    // ...
}

Run the tagged benchmarks with cargo bench -- --tag network. Use --skip-tag network to exclude them.

The macro reference lists every #[bench] key. It covers groups, #[group] run config, parameterized benchmarks with args, time budgets, and async benchmarks.

Verify and Compare

#[verify] states a performance rule next to the code it checks:

use fluxbench::verify;

#[verify(expr = "fibonacci < 50000", severity = "critical")]
struct FibUnder50us;

FluxBench checks the rule on every run:

• critical failures exit 1 while ci.fail_on_critical is on, the default.
• warning failures are reported. The run does not fail.
• info results are logged only.

The verification reference lists every metric suffix, the margin option, #[synthetic], and #[compare]. The statistics reference lists every exit status.

Documentation

Topic	File
Benchmark macros, attributes, groups, async	docs/macros.md
Verification, synthetic metrics, comparisons	docs/verification.md
CLI commands and flags	docs/cli.md
flux.toml keys and priority	docs/configuration.md
Statistics, regression detection, exit status, allocation tracking	docs/statistics.md
Crash isolation, workers, failure.kind values	docs/isolation.md
Output formats and HTML charts	docs/output.md
Runnable examples	docs/examples.md

Contributing

Contributions are welcome. See CONTRIBUTING.md for conventions and checks.

Questions and design discussion happen on Discord.

License

Licensed under Apache-2.0. See LICENSE.