Skip to content

Commit 198debe

Browse files
committed
Add some more comments
1 parent 4ad78b4 commit 198debe

File tree

2 files changed

+22
-246
lines changed

2 files changed

+22
-246
lines changed

lib/entries/23_RuntimeCompiled.ex

Lines changed: 0 additions & 245 deletions
This file was deleted.

lib/entries/runtime_compiled.ex

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,33 @@
11
defmodule RuntimeCompiled do
2+
@moduledoc """
3+
Many ideas from what has been tried at https://github.com/gunnarmorling/1brc/discussions/93
4+
Special mentions to @jesperes and @onno-vos-dev for their erlang contributions, I have learned a lot from them.
5+
6+
Ofc, thanks to @IceDragon200 who started the whole conversation and has been running the benchmarks.
7+
8+
The main new idea here is to generate a splitter based on the input file at run time, so we
9+
end up with something like
10+
11+
def split_lines(<<"city1", ?;, ...>>) ...
12+
def split_lines(<<"city2", ?;, ...>>) ...
13+
...
14+
15+
This is a very naive implementation, but it's already faster than the previous ones I had.
16+
I'm sure that there is room for improvement in the parser but I see no point in working on it now.
17+
At any given time, on my machine the producer process has ~16 messages in the queue, so improvements should probably happen there instead...
18+
19+
20+
Note: The `Code.compile_string` part is quite cursed, I do not reccomend anyone to actually do this in real life.
21+
Note2: It was more fun to code than an average macro, though.
22+
"""
223
@measurement_file "./data/measurements.txt"
324

425
@workers_count :erlang.system_info(:schedulers_online)
526
@chunk_size 1024 * 1024 * 8
627

728
defmodule CityMatcher do
829
@moduledoc """
9-
This module must be recompiled at runtime, based on the input file.
30+
This module must be re-compiled at runtime, based on the input file.
1031
The functions are generated based on the cities actually found.
1132
"""
1233
def process_lines(_), do: raise("Not implemented")

0 commit comments

Comments
 (0)