Create Data Files of Benchmark Results#
This section shows how to create data files of benchmark results.
cpp-stat-bench library supports the following formats:
JSON (via
--jsonoption)MessagePack (via
--msgpackoption)Compressed MessagePack (gzip) (via
--compressed-msgpackoption)
Prerequisites#
Implement a benchmark.
Sample Code#
In this section, a benchmark in Implement a Simple Benchmark is used for demonstration purposes.
Create a Data File in JSON Format#
To create a data file in JSON format, use --json option as follows:
./simple_benchmark --json result.json
This command creates a data file named result.json in the current directory.
The created file contains the benchmark results in JSON format as follows:
{
"$schema": "https://cppstatbench.musicscience37.com/schemas/data_file_schema_v3.json",
"finished_at": "2025-02-13T14:30:11.842257+0000",
"measurements": [
{
"case_name": "fibonacci(10)",
"custom_outputs": [],
"custom_stat_outputs": [],
"durations": {
"stat": {
"max": 1.5800000596755126e-07,
"mean": 1.3922000619004393e-07,
"median": 1.3899999373734317e-07,
"min": 1.3599999704183574e-07,
"standard_deviation": 2.897508455390607e-09,
"standard_error": 2.897508621924061e-10,
"variance": 8.39555565389066e-18
},
"values": [
[
1.5800000596755126e-07,
1.479999980347202e-07,
1.4900000167017424e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.370000006772898e-07,
1.3899999373734317e-07,
1.4500000133921276e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.3999999737279722e-07,
1.3800000431274384e-07,
1.3599999704183574e-07,
1.3999999737279722e-07,
1.3800000431274384e-07,
1.3800000431274384e-07,
1.3999999737279722e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.3599999704183574e-07,
1.4100000100825127e-07,
1.370000006772898e-07,
1.3800000431274384e-07,
1.4699999439926614e-07,
1.3800000431274384e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.3999999737279722e-07,
1.3800000431274384e-07,
1.3999999737279722e-07,
1.3999999737279722e-07,
1.370000006772898e-07,
1.3599999704183574e-07,
1.3999999737279722e-07,
1.3800000431274384e-07,
1.3800000431274384e-07,
1.4100000100825127e-07,
1.3800000431274384e-07,
1.3800000431274384e-07,
1.3999999737279722e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.3800000431274384e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.3999999737279722e-07,
1.3899999373734317e-07,
1.3899999373734317e-07,
1.3599999704183574e-07,
1.3999999737279722e-07,
1.3800000431274384e-07,
1.4100000100825127e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.3999999737279722e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.4699999439926614e-07,
1.3800000431274384e-07,
1.370000006772898e-07,
1.3899999373734317e-07,
1.370000006772898e-07,
1.3999999737279722e-07,
1.370000006772898e-07,
1.370000006772898e-07,
1.3999999737279722e-07,
1.370000006772898e-07,
1.3800000431274384e-07,
1.3999999737279722e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.3800000431274384e-07,
1.3999999737279722e-07,
1.3999999737279722e-07,
1.3800000431274384e-07,
1.3899999373734317e-07,
1.370000006772898e-07
]
]
},
"group_name": "Fibonacci",
"iterations": 1,
"measurer_name": "Processing Time",
"params": null,
"samples": 100
},
{
"case_name": "fibonacci(15)",
"custom_outputs": [],
"custom_stat_outputs": [],
"durations": {
"stat": {
"max": 1.4429999737330945e-06,
"mean": 1.3847300124325557e-06,
"median": 1.3840000292475452e-06,
"min": 1.374000021314714e-06,
"standard_deviation": 8.045144106461066e-09,
"standard_error": 8.045143995438764e-10,
"variance": 6.472434183343265e-17
},
"values": [
[
1.4429999737330945e-06,
1.4170000213198364e-06,
1.401999952577171e-06,
1.3909999552197405e-06,
1.3950000266049756e-06,
1.3820000503983465e-06,
1.3940000371803762e-06,
1.3820000503983465e-06,
1.3840000292475452e-06,
1.3850000186721445e-06,
1.374000021314714e-06,
1.3840000292475452e-06,
1.3760000001639128e-06,
1.3820000503983465e-06,
1.3840000292475452e-06,
1.3769999895885121e-06,
1.3869999975213432e-06,
1.3860000080967438e-06,
1.3850000186721445e-06,
1.3850000186721445e-06,
1.3809999472869094e-06,
1.3879999869459425e-06,
1.3860000080967438e-06,
1.3860000080967438e-06,
1.3860000080967438e-06,
1.3889999763705418e-06,
1.3830000398229458e-06,
1.3820000503983465e-06,
1.3789999684377108e-06,
1.3899999657951412e-06,
1.3809999472869094e-06,
1.3869999975213432e-06,
1.3830000398229458e-06,
1.3769999895885121e-06,
1.3809999472869094e-06,
1.3779999790131114e-06,
1.3840000292475452e-06,
1.3840000292475452e-06,
1.3799999578623101e-06,
1.3860000080967438e-06,
1.3809999472869094e-06,
1.3899999657951412e-06,
1.3850000186721445e-06,
1.3830000398229458e-06,
1.3869999975213432e-06,
1.3860000080967438e-06,
1.3889999763705418e-06,
1.3840000292475452e-06,
1.3830000398229458e-06,
1.3889999763705418e-06,
1.3830000398229458e-06,
1.3869999975213432e-06,
1.3830000398229458e-06,
1.3809999472869094e-06,
1.3820000503983465e-06,
1.3779999790131114e-06,
1.3850000186721445e-06,
1.3779999790131114e-06,
1.3789999684377108e-06,
1.3789999684377108e-06,
1.3769999895885121e-06,
1.3860000080967438e-06,
1.3789999684377108e-06,
1.3809999472869094e-06,
1.3879999869459425e-06,
1.3820000503983465e-06,
1.3869999975213432e-06,
1.3840000292475452e-06,
1.3840000292475452e-06,
1.3940000371803762e-06,
1.3840000292475452e-06,
1.3860000080967438e-06,
1.3850000186721445e-06,
1.3840000292475452e-06,
1.3879999869459425e-06,
1.3799999578623101e-06,
1.3860000080967438e-06,
1.3799999578623101e-06,
1.3789999684377108e-06,
1.3820000503983465e-06,
1.3809999472869094e-06,
1.3850000186721445e-06,
1.3750000107393134e-06,
1.3809999472869094e-06,
1.3789999684377108e-06,
1.3830000398229458e-06,
1.3840000292475452e-06,
1.3820000503983465e-06,
1.3879999869459425e-06,
1.3820000503983465e-06,
1.3799999578623101e-06,
1.3869999975213432e-06,
1.3850000186721445e-06,
1.3879999869459425e-06,
1.3850000186721445e-06,
1.3799999578623101e-06,
1.3869999975213432e-06,
1.3840000292475452e-06,
1.3830000398229458e-06,
1.3799999578623101e-06
]
]
},
"group_name": "Fibonacci",
"iterations": 1,
"measurer_name": "Processing Time",
"params": null,
"samples": 100
},
{
"case_name": "fibonacci(10)",
"custom_outputs": [],
"custom_stat_outputs": [],
"durations": {
"stat": {
"max": 1.230868962309728e-09,
"mean": 9.417940871614405e-10,
"median": 9.133014899909142e-10,
"min": 9.113870214072506e-10,
"standard_deviation": 7.730981133047266e-11,
"standard_error": 1.4114775320761552e-11,
"variance": 5.976806844595756e-21
},
"values": [
[
0.0009161920170299709,
0.0009132269769906998,
0.0012112139957025647,
0.0009478270076215267,
0.0012308689765632153,
0.0009474679827690125,
0.0009476090199314058,
0.0009473810205236077,
0.0009477669955231249,
0.0009474880062043667,
0.0009449649951420724,
0.0009132060222327709,
0.0009114100248552859,
0.0009159820037893951,
0.0009133039857260883,
0.0009131859987974167,
0.0009131280239671469,
0.0009131570113822818,
0.0009138790192082524,
0.0009132290142588317,
0.0009142549824900925,
0.0009132989798672497,
0.0009131610277108848,
0.0009131869883276522,
0.0009113869746215641,
0.00091316900216043,
0.0009133359999395907,
0.0009132209816016257,
0.0009131240076385438,
0.0009131939732469618
]
]
},
"group_name": "Fibonacci",
"iterations": 1000000,
"measurer_name": "Mean Processing Time",
"params": null,
"samples": 30
},
{
"case_name": "fibonacci(15)",
"custom_outputs": [],
"custom_stat_outputs": [],
"durations": {
"stat": {
"max": 2.9876860985211806e-09,
"mean": 1.2719898467850044e-09,
"median": 1.170501029434945e-09,
"min": 1.168736996071118e-09,
"standard_deviation": 3.7040920330966287e-10,
"standard_error": 6.762716081576414e-11,
"variance": 1.3720298771340166e-19
},
"values": [
[
0.0011830569710582495,
0.0013660229742527008,
0.0011689079692587256,
0.0011705979704856873,
0.0029876860789954662,
0.002142546931281686,
0.0011750300182029605,
0.001175028970465064,
0.0011705609504133463,
0.0011758999899029732,
0.0011741709895431995,
0.0011704410426318645,
0.0011730949627235532,
0.00121885200496763,
0.001169355004094541,
0.0011688560480251908,
0.0011706360382959247,
0.001168787945061922,
0.001168754999525845,
0.0011688260128721595,
0.0011687539517879486,
0.001170612988062203,
0.0011687369551509619,
0.0011688340455293655,
0.0011687929509207606,
0.001169834053143859,
0.0011688780505210161,
0.0011705970391631126,
0.0011687739752233028,
0.0011687659425660968
]
]
},
"group_name": "Fibonacci",
"iterations": 1000000,
"measurer_name": "Mean Processing Time",
"params": null,
"samples": 30
}
],
"started_at": "2025-02-13T14:30:10.175640+0000"
}
For description of the file format, see Schemas.
Create a Data File in MessagePack Format#
To create a data file in MessagePack format, use --msgpack option as follows:
./simple_benchmark --msgpack result.data
This command creates a data file named result.data in the current directory.
The created file contains the benchmark results in MessagePack format
with data structure same as JSON format.
Create a Data File in MessagePack Format with Compression#
To create a compressed data file in MessagePack format,
use --compressed-msgpack option as follows:
./simple_benchmark --compressed-msgpack result.data
This command creates a data file named result.data in the current directory.
The created file is compressed in gzip format.
The compressed contents is same as the data file in MessagePack format.
Hint
For example, Python’s standard gzip package can decompress the compressed data file.