Using the `cargo bench` Command (with examples)

This article will explain and provide code examples for different use cases of the cargo bench command in Rust. The cargo bench command is used to compile and execute benchmarks for a Rust package. It is a useful tool for measuring the performance of your code and identifying potential bottlenecks.

Execute all benchmarks of a package

To execute all benchmarks of a package, you can simply run the cargo bench command without any additional arguments:

cargo bench

Motivation: Executing all benchmarks is useful when you want to get an overview of the performance of your codebase. This can help you identify areas that need optimization or compare the performance of different algorithms or implementations.

Example Output:

running 5 tests
test benchmark1  ... bench:         128 ns/iter (+/- 2)
test benchmark2  ... bench:       1,332 ns/iter (+/- 21)
test benchmark3  ... bench:           7 ns/iter (+/- 0)
test benchmark4  ... bench:       1,222 ns/iter (+/- 11)
test benchmark5  ... bench:       6,032 ns/iter (+/- 111)

test result: ok. 0 passed; 0 failed; 0 ignored; 5 measured; 0 filtered out

Don’t stop when a benchmark fails

By default, cargo bench stops executing benchmarks when any of them fail. However, you can use the --no-fail-fast flag to continue running all benchmarks, even if some of them fail:

cargo bench --no-fail-fast

Motivation: In some cases, you may want to continue running all benchmarks to gather more data about the performance of your code, even if some individual benchmarks fail. This can be useful for identifying patterns or trends in performance across different benchmark iterations.

Example Output:

running 5 tests
test benchmark1  ... bench:         128 ns/iter (+/- 2)
test benchmark2  ... bench:       1,332 ns/iter (+/- 21)
test benchmark3  ... bench:           7 ns/iter (+/- 0)
test benchmark4  ... bench:       1,222 ns/iter (+/- 11)
test benchmark5  ... bench:       6,032 ns/iter (+/- 111)

test result: ok. 0 passed; 1 failed; 0 ignored; 5 measured; 0 filtered out

Compile, but don’t run benchmarks

If you only want to compile the benchmark code without executing it, you can use the --no-run flag. This will save time by skipping the execution phase:

cargo bench --no-run

Motivation: Compiling benchmarks can sometimes be time-consuming, especially if you have a large codebase or complex benchmarks. By using the --no-run flag, you can skip the execution phase and focus on quickly iterating and optimizing your code.

Example Output:

Compiling my_project v0.1.0 (/path/to/my_project)
Finished dev [unoptimized + debuginfo] target(s) in 2.98s

Benchmark the specified benchmark

If you want to run a specific benchmark, you can use the --bench flag followed by the name of the benchmark. For example, to run a benchmark named my_benchmark, you would use the following command:

cargo bench --bench my_benchmark

Motivation: Sometimes you may only want to benchmark a specific part of your codebase. By running a specific benchmark, you can isolate and measure the performance of a particular function or module.

Example Output:

running 1 test
test my_benchmark  ... bench:        54 ns/iter (+/- 3)

test result: ok. 0 passed; 0 failed; 0 ignored; 1 measured; 0 filtered out

Benchmark with the given profile

By default, cargo bench uses the bench profile for benchmarking. However, you can specify a different profile using the --profile flag. For example, if you have a custom profile named high_performance, you can benchmark with that profile using the following command:

cargo bench --profile high_performance

Motivation: Using different profiles for benchmarking can be helpful when you want to test the performance of your code with different optimization settings or target platforms. This allows you to fine-tune your code for specific scenarios or use cases.

Example Output:

running 5 tests
test benchmark1  ... bench:         120 ns/iter (+/- 7)
test benchmark2  ... bench:       1,200 ns/iter (+/- 27)
test benchmark3  ... bench:           6 ns/iter (+/- 0)
test benchmark4  ... bench:       1,220 ns/iter (+/- 22)
test benchmark5  ... bench:       6,040 ns/iter (+/- 129)

test result: ok. 0 passed; 0 failed; 0 ignored; 5 measured; 0 filtered out

Benchmark all example targets

If your Rust package contains examples, you can benchmark all of them using the --examples flag:

cargo bench --examples

Motivation: Benchmarking examples can be useful when you want to evaluate the performance of specific usage scenarios or code snippets. By benchmarking all example targets, you can gather insights into the performance characteristics of your code in different contexts.

Example Output:

running 2 tests
test example1  ... bench:         500 ns/iter (+/- 10)
test example2  ... bench:       1,000 ns/iter (+/- 20)

test result: ok. 0 passed; 0 failed; 0 ignored; 2 measured; 0 filtered out

Benchmark all binary targets

If your Rust package contains multiple binary targets, you can benchmark all of them using the --bins flag:

cargo bench --bins

Motivation: Benchmarking all binary targets can be helpful when you want to measure the performance of specific command-line tools or executables within your codebase. This allows you to identify and optimize performance bottlenecks in different parts of your application.

Example Output:

running 3 tests
test binary1  ... bench:         300 ns/iter (+/- 13)
test binary2  ... bench:       1,200 ns/iter (+/- 17)
test binary3  ... bench:       2,000 ns/iter (+/- 33)

test result: ok. 0 passed; 0 failed; 0 ignored; 3 measured; 0 filtered out

Benchmark the package’s library

To benchmark the library of the package, you can use the --lib flag:

cargo bench --lib

Motivation: Benchmarking the package’s library is useful when you want to measure the performance of the core functionality provided by your codebase. This can help you identify performance bottlenecks and optimize critical sections to improve overall performance.

Example Output:

running 1 test
test library_benchmark  ... bench:         100 ns/iter (+/- 5)

test result: ok. 0 passed; 0 failed; 0 ignored; 1 measured; 0 filtered out

In this article, we have explored various use cases of the cargo bench command in Rust. By compiling and executing benchmarks, we can measure the performance of our code and identify areas that need optimization. Whether it’s running all benchmarks, specifying a particular benchmark, or benchmarking different targets, the cargo bench command is a powerful tool for performance analysis and improvement in Rust projects.