cpp-stat-bench

Benchmark library with statistics and plotting in C++.

✨ Features

• Report results in various types of output files.
  • To console.
  • To JSON files.
  • To MessagePack files with or without gzip compression.
  • To HTML files of plots 📈.
    • Processing time for each execution.
    • Cumulative distribution function of processing time.
    • Violin plots.
• Measure processing time in two patterns.
  • Processing time of each execution.
  • Mean processing time of consecutive executions.
• Report custom data with or without statistics.
• Use custom parameters in various types.
• Perform benchmark in multiple threads.
• Use fixtures.

Repositories

• Main in GitLab: https://gitlab.com/MusicScience37Projects/utility-libraries/cpp-stat-bench
• Mirror in GitHub: https://github.com/MusicScience37/cpp-stat-bench

Documentation

• Documentation build on main branch
• Examples are in examples directory.

Installation

Via vcpkg

This library can be installed via vcpkg using following configurations:

• Add a vcpkg registry https://gitlab.com/MusicScience37Projects/vcpkg-registry in vcpkg-configuration.json.

  Example:

  {
    "$schema": "https://raw.githubusercontent.com/microsoft/vcpkg-tool/main/docs/vcpkg-configuration.schema.json",
    "default-registry": {
      "kind": "git",
      "repository": "https://github.com/Microsoft/vcpkg",
      "baseline": "acd5bba5aac8b6573b5f6f463dc0341ac0ee6fa4"
    },
    "registries": [
      {
        "kind": "git",
        "repository": "https://gitlab.com/MusicScience37Projects/vcpkg-registry",
        "baseline": "e9814bf742df3d99e3ef50ca9dcf8ed5d5a578c6",
        "packages": [
          "cpp-msgpack-light",
          "cpp-plotly-plotter",
          "cpp-stat-bench"
        ]
      }
    ]
  }

• Add cpp-stat-bench in vcpkg.json

  Example:

  {
    "$schema": "https://raw.githubusercontent.com/microsoft/vcpkg-tool/main/docs/vcpkg.schema.json",
    "dependencies": ["cpp-stat-bench"]
  }

How to Build in this Repository

To build this repository, you will require following dependencies:

• Python 3.13
  • You may want to use pyenv.
• poetry
  • Required Python packages can be installed using poetry. Execute the command poetry install on this directory.
  • pre-commit will be installed via poetry, and used for some checks. Execute poetry run pre-commit install on this directory if you develop this library.
• CMake
• C++ 17 Compiler
  • Following compilers are tested in CI:
    • GCC 13
    • Clang 19
    • MSVC 19

Additionally, you will require following dependencies to build documentation.

• Doxygen
• Graphviz
• PlantUML
  • Set PLANTUML_JAR_PATH environment variable to specify the place of plantuml.jar file.
• Java runtime environment (JRE) for PlantUML

After installing the above dependencies, execute the following commands:

1. Start a shell of poetry.

   cd <this-directory>
   poetry install
   poetry shell

2. Download vcpkg if you haven't already.

   git submodule update --init

3. Build vcpkg.

   • On Windows:

     .\vcpkg\bootstrap-vcpkg.bat

   • On Linux:

     bash ./vcpkg/bootstrap-vcpkg.sh

4. Configure.

   mkdir build
   cd build
   cmake .. -DCMAKE_TOOLCHAIN_FILE=../vcpkg/scripts/buildsystems/vcpkg.cmake

5. Optionally edit options.

   <some-editor> CMakeCache.txt

6. Build.

   cmake --build .

7. Optionally run tests.

   ctest

License

This project is licensed under Apache License, Version 2.0.