Extremely poor performance??

[benoneal]

Bevy 0.5.0, Windows 10

What you did

1. Ran bevymark and spawned around 2000 birds
2. Ran alien cake addict with board size modified to 50x50 (from 14x21)
3. Built alien cake addict with the release flag (to optimize the build) and re-ran it

What you expected to happen

Blisteringly high frame rates.

What actually happened

1. A woeful 11 FPS in the benchmark
2. Around 1 FPS in both the normal and release builds of the alien cake addict

Additional information

[bjorn3]

We are currently rewriting the renderer. This will result in a huge increase in rendering performance. This is the pipelined-rendering branch. AFAIK it is already usable. Can you try benchmarking the bevymark_pipelined example on this branch instead?

[benoneal]

Sure. Just pulled that branch and ran that benchmark.

Seems around 5.5x faster, so a relatively decent improvement, but still a far cry from what I'd expect to see given what I've been able to get out of Unity's DOTS and even my own Javascript ECS experiments.

Is this normal? Is there something I can run to see a breakdown of timings per frame to isolate the bottleneck?

[Edited to add] * actually it may not be 5x faster... hard to tell, but the non-pipelined branch is rendering exactly 5 different color variants of the sprites, whereas the pipelined is only rendering 1. That may be a coincidence, but at the least it's an apples to oranges comparison right now.

[bjorn3]

Batching is not yet implemented. There is a PR for sprite batching: #2642

As for benchmarking I now there is a tracing infrastructure to get a breakdown per system, but I haven't used it myself, so I don't know exactly how to use it. I believe it is a feature flag and then a profile gets written to a file in the chrome profiler format.

[benoneal]

Hmm. It seems it isn't only a sprite issue. I just ran the spawner example:

Even 1 draw call per object here shouldn't result in 1.3 seconds per frame.

[cart]

Just as a heads up: that spawner example is still using the old renderer, even if you are on the pipelined-rendering branch.

[parasyte]

Another datapoint. I just tried bevymark out of curiosity on the main branch. On my machine, I get about 60 fps with just over 8,600 birbs.

I'm also testing on Windows 10 with a GeForce RTX 3090. The only major difference is my CPU: Ryzen 9 5900X.

Release profile with some extra optimizations in ~/.cargo/config:

[target.x86_64-pc-windows-msvc]
linker = "rust-lld.exe"
rustflags = ["-C", "target-cpu=native"]

Oh, and I'm using a nightly compiler:

$ rustc --version
rustc 1.57.0-nightly (8c2b6ea37 2021-09-11)

[cart]

@benoneal with your computer's specs you should be getting way higher performance, especially using the new renderer. Are you absolutely certain you are running bevymark_pipelined in release mode? My computer is older / weaker than yours and I'm getting ~67,000 sprites (and ~8,000 sprites in the old renderer). The performance you have been reporting across the board is in-line with debug build performance, which makes me question the results a bit. What is the full command you are running on the command line?

[enbugger]

Not bragging, gloating, or anything. Just for comparison, I'll post here the Unity DOTS-related video with timecode so it may be useful: https://youtu.be/ILfUuBLfzGI?t=1245
DOTS with Burst, batching, and some PC is able to move ~100k sprites with playable FPS.

UPD: the specs in the video are shown on task manager dashboard

The screenshot

[sarkahn]

Not bragging, gloating, or anything. Just for comparison, I'll post here the Unity DOTS-related video with timecode so it may be useful: https://youtu.be/ILfUuBLfzGI?t=1245
DOTS with Burst, batching, and some PC is able to move ~100k sprites with playable FPS.

UPD: the specs in the video are shown on task manager dashboard

The screenshot

It's important to note that video is using the previous version of the dots "hybrid renderer". That version used gpu instancing, not batching. A proper implementation of a game using gpu instancing can easily render millions of sprites, even in Unity without dots, without breaking a sweat. However there's a lot of other issues that come along with gpu instancing. Namely compatibility with older hardware, and it doesn't support per instance overrides.

The current version of the hybrid renderer (v2) does use batching and does support per instance material overrides. My expectation is it would still greatly outperform bevy rendering right now as it's been worked on by a team of talented engineers for years at this point. So maybe not a fair comparison.

[cart]

@enbugger

Even without batching the new Bevy renderer currently matches the Unity DOTS performance from the youtube video on my machine (over 100k bunnies at ~41 fps):

My hardware is slightly better, but its still in the same category (just about a year older):

• CPU: Intel i7 7700k 4.2GHz
• GPU: Nvidia GeForce GTX 1070
• RAM: 16GB DDR4

With batching (which there is an open pr out for), I'm hitting 164k bunnies at ~41 fps

I promise the new renderer will have competitive performance.

A proper implementation of a game using gpu instancing can easily render millions of sprites, even in Unity without dots, without breaking a sweat. However there's a lot of other issues that come along with gpu instancing

@sarkahn
I'm gonna go ahead and doubt this until I see an equivalent benchmark (ex: user moves sprites using "game logic", gpu renders instanced sprites). You can have millions of instanced "particles" because they are driven by internal particle logic driven by a shader. If the user was hand-moving each particle, you would need to transfer that data to the gpu, which puts you back into the ballpark of "maybe over 100k if you do everything right". Or if sprites are "static" you don't need to copy data to the gpu every frame and then you can hit the "millions of sprites" mark.