Configure Plotting in Source Code for More Plots

This section shows how to configure plotting in source code to generate more types of plots.

For plots generated without configurations in source codes, see Generate Basic Plots of Processing Time.

Prerequisites

• One of the followings:

  • Implement a Parameterized Benchmark

  • Implement a Benchmark with a Custom Output with Statistics

  • Implement a Benchmark with a Custom Output Without Statistics

• Generate Basic Plots of Processing Time

Sample Code with Explanation

Example of a benchmark with configurations of plotting.

#include <stat_bench/benchmark_macros.h>

[[nodiscard]] double approx_exp(double x, int num_terms);

// Define a fixture to define parameters.
class ApproxExpFixture : public stat_bench::FixtureBase {
public:
    ApproxExpFixture() {
        // Define a parameter.
        add_param<int>("num_terms")
            // Add values to be tested.
            ->add(1)
            ->add(2)
            ->add(5)
            ->add(10);
    }
};

// Define a benchmark with a fixture.
// The first argument is the fixture class,
// the second argument is the name of the group of benchmarks,
// and the third argument is the name of the case of the benchmark.
STAT_BENCH_CASE_F(ApproxExpFixture, "ApproxExp", "approx_exp") {
    constexpr double x = 1.0;
    const int num_terms =
        // Get the value of the parameter from the current context object.
        stat_bench::current_invocation_context().get_param<int>("num_terms");

    STAT_BENCH_MEASURE() { return approx_exp(x, num_terms); };

    const double expected = std::exp(x);
    const double approx = approx_exp(x, num_terms);
    const double error = std::abs(approx - expected);
    // Set a custom output.
    stat_bench::current_invocation_context().add_custom_output("error", error);
}

// Configure the group of benchmarks.
// This can be placed anywhere in the executable.
STAT_BENCH_GROUP("ApproxExp")
    // Here you can configure plotting.
    // In this example, some of available plots are used.

    // Add a line plot.
    // * X-axis: Parameter "num_terms".
    // * Y-axis: Processing time. (Log scale is used always.)
    .add_parameter_to_time_line_plot("num_terms")

    // Add a line plot.
    // * X-axis: Parameter "num_terms".
    // * Y-axis: Custom output "error" in log scale.
    // Custom outputs without or with statistics can be specified similarly.
    .add_parameter_to_output_line_plot(
        "num_terms", "error", stat_bench::PlotOptions().log_output(true))

    // Add a line plot.
    // * X-axis: Processing time. (Log scale is used always.)
    // * Y-axis: Custom output "error" in log scale.
    // This plot shows the change of the above two values while parameter
    // "num_terms" changes.
    // This is useful to see efficiency of algorithms.
    .add_time_to_output_by_parameter_line_plot(
        "num_terms", "error", stat_bench::PlotOptions().log_output(true));

// Define main function. (Required once in an executable.)
STAT_BENCH_MAIN

double approx_exp(double x, int num_terms) {
    double result = 1.0;
    double term = 1.0;
    for (int i = 1; i < num_terms; ++i) {
        term *= x / i;
        result += term;
    }
    return result;
}

Execution

You can build the above code and run the resulting executable with --plot option similar to Generate Basic Plots of Processing Time:

./configure_plotting --plot results

With the configurations in the source code, more plots are generated in the results directory. Check the generated plots using a web browser.

In the above example, the following plots are generated (plots added by the configurations are highlighted):

Types of Plots

The following types of plots can be generated:

Function	Plot Type	X-axis	Y-axis
add_parameter_to_time_line_plot	Line plot	Parameter	Processing time
add_parameter_to_time_violin_plot	Violin plot	Parameter	Processing time
add_parameter_to_time_box_plot	Box plot	Parameter	Processing time
add_parameter_to_output_line_plot	Line plot	Parameter	Custom output
add_time_to_output_by_parameter_line_plot	Line plot	Processing time	Custom output

Further Reading

• API Reference

  • STAT_BENCH_GROUP()

  • stat_bench::bench_impl::BenchmarkGroupRegister

  • stat_bench::PlotOptions