Implement a Benchmark with a Custom Output Without Statistics

Implement a Benchmark with a Custom Output Without Statistics#

This section shows how to implement a benchmark with a custom output without statistics using cpp-stat-bench library.

Hint

If your custom output changes in each iteration, you should implement a benchmark with a custom output with statistics.

Prerequisites#

Sample Code with Explanation#

Example of a benchmark with a custom output without statistics.#
#include <stat_bench/benchmark_macros.h>

[[nodiscard]] double approx_exp(double x, int num_terms);

// Define a fixture to define parameters.
class ApproxExpFixture : public stat_bench::FixtureBase {
public:
    ApproxExpFixture() {
        // Define a parameter.
        add_param<int>("num_terms")
            // Add values to be tested.
            ->add(1)
            ->add(2)
            ->add(5)
            ->add(10);
    }
};

// Define a benchmark with a fixture.
// The first argument is the fixture class,
// the second argument is the name of the group of benchmarks,
// and the third argument is the name of the case of the benchmark.
STAT_BENCH_CASE_F(ApproxExpFixture, "ApproxExp", "approx_exp") {
    constexpr double x = 1.0;
    const int num_terms =
        // Get the value of the parameter from the current context object.
        stat_bench::current_invocation_context().get_param<int>("num_terms");

    STAT_BENCH_MEASURE() { return approx_exp(x, num_terms); };

    const double expected = std::exp(x);
    const double approx = approx_exp(x, num_terms);
    const double error = std::abs(approx - expected);
    // Set a custom output.
    stat_bench::current_invocation_context().add_custom_output("error", error);
}

// Define main function. (Required once in an executable.)
STAT_BENCH_MAIN

double approx_exp(double x, int num_terms) {
    double result = 1.0;
    double term = 1.0;
    for (int i = 1; i < num_terms; ++i) {
        term *= x / i;
        result += term;
    }
    return result;
}

Execution#

You can build the above code and run the resulting executable without arguments to see an output similar to the one below.

Example output to the console.#
cpp-stat-bench 0.21.0

Benchmark start at 2025-02-13T13:35:41.832607+0000

Time resolution: 1.000e-09 sec.

========================================================================================================================
Processing Time
========================================================================================================================

>> ApproxExp
                                                                                Time [us]
                                                            Iterations Samples       Mean Std. Err.       Max Custom Outputs (mean)
------------------------------------------------------------------------------------------------------------------------
approx_exp (num_terms=1)                                             1     100     0.0367    0.0005    0.0810 error=1.718e+00,
approx_exp (num_terms=2)                                             1     100     0.0352    0.0141    1.4270 error=7.183e-01,
approx_exp (num_terms=5)                                             1     100     0.0248    0.0002    0.0360 error=9.948e-03,
approx_exp (num_terms=10)                                            1     100     0.0299    0.0002    0.0500 error=3.029e-07,

========================================================================================================================
Mean Processing Time
========================================================================================================================

>> ApproxExp
                                                                                Time [us]
                                                            Iterations Samples       Mean Std. Err.       Max Custom Outputs (mean)
------------------------------------------------------------------------------------------------------------------------
approx_exp (num_terms=1)                                       1000000      30     0.0010    0.0000    0.0012 error=1.718e+00,
approx_exp (num_terms=2)                                       1000000      30     0.0022    0.0000    0.0023 error=7.183e-01,
approx_exp (num_terms=5)                                       1000000      30     0.0040    0.0000    0.0043 error=9.948e-03,
approx_exp (num_terms=10)                                      1000000      30     0.0088    0.0000    0.0095 error=3.029e-07,

Benchmark finished at 2025-02-13T13:35:42.799000+0000

Further Reading#