Comparison against alternative crates?

[TheButlah]

Hi, I'm considering using this crate but am unsure whether the performance is any better than the other SPSPC wait free ringbuffers. A comparison to crossbeam_channel might also be merited, since I could see using their bounded queues in a very similar way to the ringbuffer.

[mgeier]

Thanks for bringing this up! This is an interesting but complicated topic ...

---

First, the simple thing: SPSC is always faster than MPSC/SPMC/MPMC.
If not, the SPSC implementation sucks (or the other one is incorrect).
SPSC is just a much simpler problem with a much simpler solution. Not only is it less complex, it also needs much less code and is typically easier to understand. On the flip side, it's also much more limted.

All crossbeam queues are slower than this SPSC (and any reasonable other SPSC implementation).
That's why crossbeam-rs/crossbeam#338 was started, which also includes a few benchmarks.
The rtrb implementation is based on that PR, which was never merged.

---

When it comes to comparing SPSC implementations, you should consider multiple aspects:

• correctness
• API
• performance

Correctness is very hard to check, because all SPSC implementations use a fair amount of unsafe code.
It's basically impossible to test, you'll have to analyze the code to convince yourself that it is correct.

I think that rtrb is correct, but who am I to say?

I think the most important difference between the existing SPSC crates is their API.
Many crates only work for u8 values, some work with arbitrary T.
The simplest API is to push() and pop() single items.
Some crates (e.g. rtrb and https://github.com/agerasev/ringbuf) allow to handle multiple items at once, which is more efficient than repeated push() and/or pop().
The most efficient way to write multiple items will involve an unsafe API (because of uninitialized memory).
IMHO the API of rtrb is nicer than ringbuf, but I'm of course very biased.

What kind of operations are you planning to use?
Which of the provided APIs do you prefer?

If there is something missing in the API of rtrb, please let me know!

Finally, performance ...

It's very hard to create meaningful benchmarks. I've created some in the benches directory, which can be run with cargo bench.
The single-threaded benchmarks are quite reliable and reproducible, but also mostly useless.
The more interesting part is the behavior when two threads are involved, but the benchmarks are potentially less reproducible due to unpredictable OS thread scheduling.

If you want to compare rtrb to other crates, you can copy the benchmarking code and adapt it accordingly.
Some time ago, I've started working on a comparison between crates (https://github.com/mgeier-forks/rtrb/tree/performance-comparison), but I haven't finished it. If you want, you can try it anyway, by running cargo bench in the performance-comparison subdirectory.

All this is probably meaningless, because the benchmark code will most likely not reflect your actual usage pattern, and the real-life performance differences between SPSC implementations are probably negligibly small anyway.

[mgeier]

Keeping in mind that all benchmarks are wrong, here is one result from running the two-threads benchmark from the https://github.com/mgeier-forks/rtrb/tree/performance-comparison branch on my laptop:

You should of course not trust the good result for rtrb, because I created the benchmark (and I selected the result from a few runs).

Changes of about 10% are common between runs, sometimes there are even 20% changes.

If you have any ideas how to make the results more stable (or more meaningful in general), please let me know!

my test result.

with

[profile.bench]
lto = true
opt-level = 3
codegen-units = 1

and tweaked my kernel, resulting in a very amazing speed.

[mgeier]

I've just tried it with the suggested [profile.bench] settings:

The differences are not that big, but it looks like the pop() timings improve in most crates.

Could that mean that maybe an #[inline] is missing in one of the functions in the pop() code path?

tweaked my kernel, resulting in a very amazing speed.

Are you talking about changes to the Linux scheduler?

This might have an influence on how the secondary thread (the one that generates contention on the atomic variables) is scheduled and therefore distort the measurements.

I followed the red hat low latency tuning guide, https://access.redhat.com/sites/default/files/attachments/201501-perf-brief-low-latency-tuning-rhel7-v1.1.pdf, I disable the hyperthreading and turned all the kernel commands mentioned in doc. And use the low latency tuned-adm profile setting. I also use cpuset clear the core I use for benchmark.

[mgeier]

Thanks @zhenpingfeng for the information about latency tuning. This looks very promising and I'll have a closer look when I have more time.

In the meantime, I've modified the benchmark code a bit: #42.
I hope this makes it more reproducible and also more meaningful.

I also modified the performance comparison and created a new branch: https://github.com/mgeier-forks/rtrb/tree/performance-comparison2

[mgeier]

@TheButlah Coming back to your original question about crossbeam-channel ...

I tried my latest benchmark with crossbeam-queue and concurrent-queue:

I must say I'm quite surprised how fast they are!

In the uncontended case they are quite a bit slower than most SPSC implementations but faster than ringbuf!

In the contended case it's much closer. crossbeam-queue is only a little bit slower than the SPSC implementations, and concurrent-queue is even faster than ringbuf.

my benchmark2 result.

use rtrb::RingBuffer;
use std::thread;
use std::time::Instant;


fn main() {
    let (mut producer, mut consumer) = RingBuffer::<Instant>::new(2).split();

    thread::spawn(move || {
        loop {
            match consumer.pop().ok() {
                Some(now) => {
                    println!("{:?}", now.elapsed());
                }
                None => { continue; }
            }
        }
    });

    loop {
        let now = Instant::now();
        let _ = producer.push(now).is_ok();
    }
}

I have some questions. The elapsed time I get by using the above code is about 4us (FIFO scheduler). Is my method of using this library wrong? Or sending an Instant structure does take this amount of time? How can I reduce the latency to the nanosecond level?

New update:

fn main() {

    let num_msg: usize = 1_000_000;

    let (mut producer, mut consumer) = RingBuffer::<Instant>::new(2).split();
    
    thread::spawn(move || {
        let mut n = 0;
        loop {
            match consumer.pop().ok() {
                Some(now) => {
                    let _ = now.elapsed();
                    if n != num_msg {
                        n += 1;
                    } else {
                        break
                    }
                }
                None => { continue; }
            }
        }
    });

    let now = Instant::now();
    for n in 0..num_msg {
        while !producer.push(now).is_ok() {};
    }

    println!("avg {:?}", now.elapsed() / num_msg as u32);
}

After removing the println! marco, it now only cost around 120ns per send. problem solved.

[mgeier]

Thanks @zhenpingfeng for running the benchmarks again, the results seem pretty much consistent with mine, which is good!

The idea of sending an Instant through the ring buffer is brilliant!
However, in your updated code, this isn't really used anymore, right?
You might as well send any other payload since you are ignoring the value obtained from pop().

BTW, I think you could replace your match statement with if let, and the continue statement is superfluous since it is the last statement in the loop anyway.

Actually, the elapsed function call is worked, if I remove it, each sends only cost around 100ns.

[mgeier]

Actually, the elapsed function call is worked, if I remove it, each sends only cost around 100ns.

Yes, sure, the elapsed() method is called, but the result is ignored, so you might as well remove it.

We are not interested in benchmarking the elapsed() method, right?

If it takes 100ns without the elapsed() call, this perfectly matches your result from above (#39 (comment)), doesn't it?