Writing Benchmarks

Macros to Define and Configure Benchmarks

STAT_BENCH_CASE(GROUP_NAME, CASE_NAME)

Macro to define a case.

Parameters:

• GROUP_NAME – [in] Group name.

• CASE_NAME – [in] Case name.

STAT_BENCH_CASE_F(FIXTURE_NAME, GROUP_NAME, CASE_NAME)

Macro to define a case using a fixture.

Parameters:

• FIXTURE_NAME – [in] Fixture name.

• GROUP_NAME – [in] Group name.

• CASE_NAME – [in] Case name.

STAT_BENCH_MEASURE()

Macro to measure a function.

STAT_BENCH_MEASURE_INDEXED(THREAD_INDEX_VAR, SAMPLE_INDEX_VAR, ITERATION_INDEX_VAR)

Macro to measure a function.

Parameters:

• THREAD_INDEX_VAR – [in] Variable name of the index of the thread.

• SAMPLE_INDEX_VAR – [in] Variable name of the index of the sample.

• ITERATION_INDEX_VAR – [in] Variable name of the index of the iteration.

STAT_BENCH_GROUP(GROUP_NAME)

Macro to define and configure a group.

This macro returns stat_bench::bench_impl::BenchmarkGroupRegister object. You can use this macro to configure a group like following:

STAT_BENCH_GROUP("group_name")
    .add_parameter_to_time_line_plot("parameter_name")
    .add_parameter_to_time_line_plot_log("another_parameter_name");

Parameters:

• GROUP_NAME – [in] Group name.

STAT_BENCH_MAIN

Macro of default main function.

Invocation Context

inline InvocationContext &stat_bench::current_invocation_context()

Get the current invocation context.

Returns:

Reference to the current invocation context.

class InvocationContext

Class of context of invocations of benchmarks.

Public Functions

inline void add_custom_output(const CustomOutputName &name, double value)

Add a custom output without statistics.

Parameters:

• name – [in] Name.

• value – [in] Output value.

inline void add_custom_output(std::string name, double value)

Add a custom output without statistics.

Parameters:

• name – [in] Name.

• value – [in] Output value.

inline std::shared_ptr<stat::CustomStatOutput> add_custom_stat(const CustomOutputName &name, stat::CustomOutputAnalysisType analysis_type = stat::CustomOutputAnalysisType::mean)

Add a custom output with statistics.

Parameters:

• name – [in] Name.

• analysis_type – [in] Type of analysis.

Returns:

Object to add output values.

inline std::shared_ptr<stat::CustomStatOutput> add_custom_stat(std::string name, stat::CustomOutputAnalysisType analysis_type = stat::CustomOutputAnalysisType::mean)

Add a custom output with statistics.

Parameters:

• name – [in] Name.

• analysis_type – [in] Type of analysis.

Returns:

Object to add output values.

template<typename T>
inline const T &get_param(const param::ParameterName &param_name) const

Get a parameter.

Template Parameters:

T – Type of the parameter value.

Parameters:

param_name – [in] Parameter name.

Returns:

Parameter value.

template<typename T>
inline const T &get_param(std::string param_name) const

Get a parameter.

Template Parameters:

T – Type of the parameter value.

Parameters:

param_name – [in] Parameter name.

Returns:

Parameter value.

inline std::size_t iterations() const noexcept

Get the number of iterations.

Returns:

Number of iterations.

inline std::size_t samples() const noexcept

Get the number of samples.

Returns:

Number of samples.

inline std::size_t threads() const noexcept

Get the number of threads.

Returns:

Number of threads.

Group Configuration

class BenchmarkGroupRegister

Class to register benchmark groups.

Public Functions

explicit BenchmarkGroupRegister(const BenchmarkGroupName &name) noexcept

Constructor.

Parameters:

name – [in] Name of the group.

BenchmarkGroupRegister &add_parameter_to_output_line_plot(util::StringView parameter_name, util::StringView custom_output_name, PlotOptions options = PlotOptions()) noexcept

Add a line plot of a custom output with respect to a parameter to the group.

Parameters:

• parameter_name – [in] Parameter name.

• custom_output_name – [in] Custom output name.

• options – [in] Options for the plot.

Returns:

Reference to this object.

BenchmarkGroupRegister &add_parameter_to_time_box_plot(util::StringView parameter_name, PlotOptions options = PlotOptions()) noexcept

Add a box plot of processing time with respect to a parameter to the group.

Parameters:

• parameter_name – [in] Parameter name.

• options – [in] Options for the plot.

Returns:

Reference to this object.

BenchmarkGroupRegister &add_parameter_to_time_line_plot(util::StringView parameter_name, PlotOptions options = PlotOptions()) noexcept

Add a line plot of processing time with respect to a parameter to the group.

Parameters:

• parameter_name – [in] Parameter name.

• options – [in] Options for the plot.

Returns:

Reference to this object.

BenchmarkGroupRegister &add_parameter_to_time_line_plot_log(util::StringView parameter_name) noexcept

Add a line plot of processing time with respect to a parameter in the log scale to the group.

Deprecated:

This function is deprecated in favor of add_parameter_to_time_line_plot() function.

Parameters:

parameter_name – [in] Parameter name.

Returns:

Reference to this object.

BenchmarkGroupRegister &add_parameter_to_time_violin_plot(util::StringView parameter_name, PlotOptions options = PlotOptions()) noexcept

Add a violin plot of processing time with respect to a parameter to the group.

Parameters:

• parameter_name – [in] Parameter name.

• options – [in] Options for the plot.

Returns:

Reference to this object.

BenchmarkGroupRegister &add_time_to_output_by_parameter_line_plot(util::StringView parameter_name, util::StringView custom_output_name, PlotOptions options = PlotOptions()) noexcept

Add a line plot of a custom output with respect to processing time while parameter changes.

Parameters:

• parameter_name – [in] Parameter name.

• custom_output_name – [in] Custom output name.

• options – [in] Options for the plot.

Returns:

Reference to this object.

class PlotOptions

Class of options for plots.

Public Functions

PlotOptions() noexcept

Constructor.

PlotOptions(PlotOption::Value option) noexcept

Constructor.

Deprecated:

This function exists for backward compatibility and will be removed in the future.

Parameters:

option – [in] Option.

bool log_output() const noexcept

Get whether to plot custom outputs in the log scale.

Returns:

Whether to plot custom outputs in the log scale.

PlotOptions &log_output(bool value) noexcept

Set whether to plot custom outputs in the log scale.

Parameters:

value – [in] Value.

Returns:

This object.

bool log_parameter() const noexcept

Get whether to plot parameters in the log scale.

Returns:

Whether to plot parameters in the log scale.

PlotOptions &log_parameter(bool value) noexcept

Set whether to plot parameters in the log scale.

Parameters:

value – [in] Value.

Returns:

This object.

util::StringView subplot_column_parameter_name() const noexcept

Get parameter name used for columns of subplots.

Returns:

Parameter name used for columns of subplots.

PlotOptions &subplot_column_parameter_name(const util::StringView &value) noexcept

Set parameter name used for columns of subplots.

Note

This function is assumed to be used with a string literal, and hold only the pointer to the string literal.

Parameters:

value – [in] Value. (A string literal.)

Returns:

This object.

util::StringView subplot_row_parameter_name() const noexcept

Get parameter name used for rows of subplots.

Returns:

Parameter name used for rows of subplots.

PlotOptions &subplot_row_parameter_name(const util::StringView &value) noexcept

Set parameter name used for rows of subplots.

Note

This function is assumed to be used with a string literal, and hold only the pointer to the string literal.

Parameters:

value – [in] Value. (A string literal.)

Returns:

This object.