Skip to content

Commit 75f5182

Browse files
trflynn89trflynn89
committed
Benchmark string formatting
1 parent c953c55 commit 75f5182

File tree

9 files changed

+169
-15
lines changed

9 files changed

+169
-15
lines changed

‎.gitmodules

+3
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,9 @@
44
[submodule "bench/json/extern/nlohmann"]
55
path = bench/json/extern/nlohmann
66
url = https://github.com/nlohmann/json.git
7+
[submodule "bench/string/extern/fmt"]
8+
path = bench/string/extern/fmt
9+
url = https://github.com/fmtlib/fmt.git
710
[submodule "test/Catch2"]
811
path = test/Catch2
912
url = https://github.com/catchorg/Catch2.git

‎bench/README.md

+1
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,4 @@ This directory contains performance benchmarks of various libfly components.
44

55
* [Huffman and Base64 Coders](/bench/coders)
66
* [JSON Parser](/bench/json)
7+
* [String Formatting](/bench/string)

‎bench/files.mk

+2-1
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,8 @@ SRC_DIRS_$(d) := \
1616
# Include the directories containing the benchmark tests.
1717
SRC_DIRS_$(d) += \
1818
bench/coders \
19-
bench/json
19+
bench/json \
20+
bench/string
2021

2122
SRC_$(d) := \
2223
$(d)/main.cpp

‎bench/string/README.md

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# JSON Parser
2+
3+
Benchmark of the libfly JSON parser against the popular {fmt} library and the STL:
4+
5+
* [libfly](/fly/types/string/string.hpp)
6+
* [{fmt}](https://github.com/fmtlib/fmt)
7+
8+
## Results
9+
10+
Results below are the median of 1000001 iterations of creating a formatted string. Obviously, libfly
11+
could use some work :)
12+
13+
| Formatter | Duration (ns) |
14+
| :-- | --: |
15+
| libfly | 4.496 |
16+
| {fmt} | 0.583 |
17+
| STL IO Streams | 1.821 |
18+
19+
## Profile
20+
21+
```
22+
Each sample counts as 0.01 seconds.
23+
% cumulative self self total
24+
time seconds seconds calls ms/call ms/call name
25+
16.09 0.67 0.14 1000001 0.00 0.00 void fly::detail::BasicStringFormatter<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::format_internal<double, int, double, char const (&) [4], decltype(nullptr), char>(std::ostream&, fly::detail::BasicFormatString<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::type_identity<double>::type, std::type_identity<int>::type, std::type_identity<double>::type, std::type_identity<char const (&) [4]>::type, std::type_identity<decltype(nullptr)>::type, std::type_identity<char>::type>&&, fly::detail::BasicFormatParameters<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::type_identity<double>::type, std::type_identity<int>::type, std::type_identity<double>::type, std::type_identity<char const (&) [4]>::type, std::type_identity<decltype(nullptr)>::type, std::type_identity<char>::type>&&)
26+
3.45 0.74 0.03 6000011 0.00 0.00 fly::detail::BasicStreamModifiers<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::~BasicStreamModifiers()
27+
3.45 0.77 0.03 2000002 0.00 0.00 void fly::detail::BasicStringFormatter<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::format_value<double>(std::ostream&, fly::detail::BasicFormatSpecifier<char>&&, double const&)
28+
1.15 0.78 0.01 6000007 0.00 0.00 std::noshowpoint(std::ios_base&)
29+
1.15 0.79 0.01 6000007 0.00 0.00 std::nouppercase(std::ios_base&)
30+
1.15 0.80 0.01 2000002 0.00 0.00 fly::BinaryDecoder::decode_internal(std::istream&, std::ostream&)
31+
1.15 0.81 0.01 2000002 0.00 0.00 void fly::detail::BasicStringFormatter<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::format_internal<unsigned int, char&, char&>(std::ostream&, fly::detail::BasicFormatString<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::type_identity<unsigned int>::type, std::type_identity<char&>::type, std::type_identity<char&>::type>&&, fly::detail::BasicFormatParameters<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::type_identity<unsigned int>::type, std::type_identity<char&>::type, std::type_identity<char&>::type>&&)
32+
1.15 0.82 0.01 1000001 0.00 0.00 void fly::detail::BasicStringFormatter<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::format_value<char>(std::ostream&, fly::detail::BasicFormatSpecifier<char>&&, char const&)
33+
1.15 0.83 0.01 1000001 0.00 0.00 void fly::detail::BasicStringFormatter<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::format_value<int>(std::ostream&, fly::detail::BasicFormatSpecifier<char>&&, int const&)
34+
1.15 0.84 0.01 2 5.00 5.00 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::reserve(unsigned long)
35+
0.57 0.87 0.01 3000004 0.00 0.00 std::left(std::ios_base&)
36+
0.57 0.87 0.01 3000003 0.00 0.00 std::right(std::ios_base&)
37+
0.00 0.87 0.00 6000007 0.00 0.00 std::noshowbase(std::ios_base&)
38+
0.00 0.87 0.00 5000005 0.00 0.00 std::noboolalpha(std::ios_base&)
39+
0.00 0.87 0.00 1000019 0.00 0.00 fly::operator<<(std::ostream&, fly::Styler const&)
40+
0.00 0.87 0.00 1000002 0.00 0.00 std::boolalpha(std::ios_base&)
41+
0.00 0.87 0.00 1000001 0.00 0.00 void fly::detail::BasicStringFormatter<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::format_value<char [4]>(std::ostream&, fly::detail::BasicFormatSpecifier<char>&&, char const (&) [4])
42+
0.00 0.87 0.00 2128 0.00 0.00 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
43+
0.00 0.87 0.00 1196 0.00 0.00 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()
44+
0.00 0.87 0.00 1016 0.00 0.00 std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >::vector(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)
45+
0.00 0.87 0.00 138 0.00 0.00 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose()
46+
0.00 0.87 0.00 120 0.00 0.00 void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>(char const*, char const*, std::forward_iterator_tag)
47+
```

‎bench/string/benchmark_string.cpp

+101
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
#include "bench/util/table.hpp"
2+
3+
#include "fly/types/string/string.hpp"
4+
5+
#include "catch2/catch.hpp"
6+
#include "fmt/format.h"
7+
8+
#include <iostream>
9+
#include <map>
10+
#include <sstream>
11+
#include <string>
12+
13+
namespace {
14+
15+
using StringTable = fly::benchmark::Table<std::string, double>;
16+
17+
class StringBase
18+
{
19+
public:
20+
virtual ~StringBase() = default;
21+
virtual void format() = 0;
22+
};
23+
24+
// libfly - https://github.com/trflynn89/libfly
25+
class LibflyFormat : public StringBase
26+
{
27+
public:
28+
void format() override
29+
{
30+
FLY_UNUSED(fly::String::format(
31+
"{:.10f}:{:04}:{:+}:{}:{}:{}:%\n",
32+
1.234,
33+
42,
34+
3.13,
35+
"str",
36+
nullptr,
37+
'X'));
38+
}
39+
};
40+
41+
// {fmt} - https://github.com/fmtlib/fmt
42+
class FmtFormat : public StringBase
43+
{
44+
public:
45+
void format() override
46+
{
47+
FLY_UNUSED(
48+
fmt::format("{:.10f}:{:04}:{:+}:{}:{}:{}:%\n", 1.234, 42, 3.13, "str", nullptr, 'X'));
49+
}
50+
};
51+
52+
// STL IO Streams
53+
class STLStreamFormat : public StringBase
54+
{
55+
public:
56+
void format() override
57+
{
58+
std::stringstream stream;
59+
stream << std::setprecision(10) << std::fixed << 1.234 << ':'
60+
<< std::resetiosflags(std::ios::floatfield) << std::setw(4) << std::setfill('0')
61+
<< 42 << std::setfill(' ') << ':' << std::setiosflags(std::ios::showpos) << 3.13
62+
<< std::resetiosflags(std::ios::showpos) << ':' << "str" << ':' << nullptr << ':'
63+
<< 'X' << ":%\n";
64+
}
65+
};
66+
67+
} // namespace
68+
69+
CATCH_TEST_CASE("String", "[bench]")
70+
{
71+
static constexpr std::size_t s_iterations = 1000001;
72+
73+
std::map<std::string, std::unique_ptr<StringBase>> formatters;
74+
formatters.emplace("{fmt}", std::make_unique<FmtFormat>());
75+
formatters.emplace("STL IO Streams", std::make_unique<STLStreamFormat>());
76+
formatters.emplace("libfly", std::make_unique<LibflyFormat>());
77+
78+
StringTable table("String Formatting", {"Formatter", "Duration (ns)"});
79+
80+
for (auto &formatter : formatters)
81+
{
82+
std::vector<double> results;
83+
84+
for (std::size_t i = 0; i < s_iterations; ++i)
85+
{
86+
const auto start = std::chrono::system_clock::now();
87+
formatter.second->format();
88+
const auto end = std::chrono::system_clock::now();
89+
90+
const auto duration = std::chrono::duration<double>(end - start);
91+
results.push_back(duration.count());
92+
}
93+
94+
std::sort(results.rbegin(), results.rend());
95+
96+
const auto duration = results[s_iterations / 2];
97+
table.append_row(formatter.first, duration * 1000 * 1000);
98+
}
99+
100+
std::cout << table << '\n';
101+
}

‎bench/string/extern/fmt

Submodule fmt added at 7bdf062

‎bench/string/files.mk

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
SRC_$(d) := \
2+
$(d)/benchmark_string.cpp
3+
4+
# {fmt} - https://github.com/fmtlib/fmt
5+
FMT_FLAGS_$(d) := \
6+
-I$(d)/extern/fmt/include \
7+
-DFMT_HEADER_ONLY
8+
9+
CFLAGS_$(d) += $(FMT_FLAGS_$(d))
10+
CXXFLAGS_$(d) += $(FMT_FLAGS_$(d))

‎build/win/libfly_benchmarks/libfly_benchmarks.vcxproj

+1
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,7 @@
165165
<ItemGroup>
166166
<ClCompile Include="..\..\..\bench\coders\benchmark_coders.cpp" />
167167
<ClCompile Include="..\..\..\bench\json\benchmark_json.cpp" />
168+
<ClCompile Include="..\..\..\bench\string\benchmark_string.cpp" />
168169
<ClCompile Include="..\..\..\bench\main.cpp" />
169170
</ItemGroup>
170171
<ItemGroup>

‎build/win/libfly_benchmarks/libfly_benchmarks.vcxproj.filters

+3-14
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,9 @@
11
<?xml version="1.0" encoding="utf-8"?>
22
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
3-
<ItemGroup>
4-
<Filter Include="coders">
5-
<UniqueIdentifier>{1c950426-e61d-4c3d-bc00-05bb72f98dc1}</UniqueIdentifier>
6-
</Filter>
7-
<Filter Include="json">
8-
<UniqueIdentifier>{7218181a-1f6f-418e-9e43-8d00be882e22}</UniqueIdentifier>
9-
</Filter>
10-
</ItemGroup>
113
<ItemGroup>
124
<ClCompile Include="..\..\..\bench\main.cpp" />
13-
<ClCompile Include="..\..\..\bench\coders\benchmark_coders.cpp">
14-
<Filter>coders</Filter>
15-
</ClCompile>
16-
<ClCompile Include="..\..\..\bench\json\benchmark_json.cpp">
17-
<Filter>json</Filter>
18-
</ClCompile>
5+
<ClCompile Include="..\..\..\bench\coders\benchmark_coders.cpp" />
6+
<ClCompile Include="..\..\..\bench\json\benchmark_json.cpp" />
7+
<ClCompile Include="..\..\..\bench\string\benchmark_string.cpp" />
198
</ItemGroup>
209
</Project>

0 commit comments

Comments
 (0)