ci: Actually briefly run fuzzer and benches #2559
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Because just building them can hide errors.
larseggert
requested review from
KershawChang,
martinthomson and
mxinden
as code owners
Failed Interop TestsQUIC Interop Runner, client vs. server, differences relative to 182afed. neqo-latest as client
neqo-latest as server
All resultsSucceeded Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
Unsupported Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
|
Benchmark resultsPerformance differences relative to 41036fc. 1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: No change in performance detected. time: [734.22 ms 738.45 ms 742.68 ms]
thrpt: [134.65 MiB/s 135.42 MiB/s 136.20 MiB/s]
change:
time: [-0.2199% +0.5646% +1.4077%] (p = 0.18 > 0.05)
thrpt: [-1.3881% -0.5614% +0.2203%]
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected. time: [348.23 ms 349.81 ms 351.37 ms]
thrpt: [28.460 Kelem/s 28.587 Kelem/s 28.716 Kelem/s]
change:
time: [-0.3455% +0.3043% +0.9673%] (p = 0.37 > 0.05)
thrpt: [-0.9580% -0.3034% +0.3467%]
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: Change within noise threshold. time: [24.764 ms 24.914 ms 25.073 ms]
thrpt: [39.884 elem/s 40.138 elem/s 40.382 elem/s]
change:
time: [-1.8667% -0.9810% -0.0439%] (p = 0.04 < 0.05)
thrpt: [+0.0440% +0.9907% +1.9022%]
1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: No change in performance detected. time: [1.8332 s 1.8538 s 1.8755 s]
thrpt: [53.320 MiB/s 53.942 MiB/s 54.548 MiB/s]
change:
time: [-2.5457% -0.9930% +0.6250%] (p = 0.23 > 0.05)
thrpt: [-0.6211% +1.0030% +2.6122%]
decode 4096 bytes, mask ff: No change in performance detected. time: [12.076 µs 12.112 µs 12.156 µs]
change: [-0.9586% -0.4679% -0.0154%] (p = 0.06 > 0.05)
decode 1048576 bytes, mask ff: No change in performance detected. time: [3.0808 ms 3.0906 ms 3.1020 ms]
change: [-0.4618% +0.0349% +0.5469%] (p = 0.90 > 0.05)
decode 4096 bytes, mask 7f: No change in performance detected. time: [20.142 µs 20.188 µs 20.242 µs]
change: [-0.5397% -0.1359% +0.2516%] (p = 0.51 > 0.05)
decode 1048576 bytes, mask 7f: No change in performance detected. time: [5.2559 ms 5.2689 ms 5.2832 ms]
change: [-0.0870% +0.2367% +0.5625%] (p = 0.16 > 0.05)
decode 4096 bytes, mask 3f: No change in performance detected. time: [7.0161 µs 7.0453 µs 7.0807 µs]
change: [-0.5715% -0.0592% +0.4234%] (p = 0.83 > 0.05)
decode 1048576 bytes, mask 3f: No change in performance detected. time: [1.7924 ms 1.7992 ms 1.8063 ms]
change: [-0.3719% +0.1185% +0.6042%] (p = 0.66 > 0.05)
1 streams of 1 bytes/multistream: No change in performance detected. time: [73.931 µs 74.143 µs 74.359 µs]
change: [-1.4358% +0.1039% +1.0639%] (p = 0.90 > 0.05)
1000 streams of 1 bytes/multistream: Change within noise threshold. time: [25.442 ms 25.478 ms 25.514 ms]
change: [+0.2240% +0.4361% +0.6441%] (p = 0.00 < 0.05)
10000 streams of 1 bytes/multistream: Change within noise threshold. time: [1.7114 s 1.7130 s 1.7147 s]
change: [+0.2057% +0.3543% +0.5034%] (p = 0.00 < 0.05)
1 streams of 1000 bytes/multistream: No change in performance detected. time: [75.365 µs 76.450 µs 77.968 µs]
change: [-0.9164% +1.0766% +3.4559%] (p = 0.38 > 0.05)
100 streams of 1000 bytes/multistream: Change within noise threshold. time: [3.4369 ms 3.4437 ms 3.4511 ms]
change: [+0.2901% +0.5723% +0.8580%] (p = 0.00 < 0.05)
1000 streams of 1000 bytes/multistream: 💔 Performance has regressed. time: [147.95 ms 148.02 ms 148.10 ms]
change: [+2.7942% +2.8781% +2.9533%] (p = 0.00 < 0.05)
coalesce_acked_from_zero 1+1 entries: No change in performance detected. time: [94.690 ns 95.054 ns 95.432 ns]
change: [-2.6964% -0.8954% +0.3177%] (p = 0.32 > 0.05)
coalesce_acked_from_zero 3+1 entries: No change in performance detected. time: [112.43 ns 112.74 ns 113.08 ns]
change: [-0.5025% -0.1864% +0.1152%] (p = 0.24 > 0.05)
coalesce_acked_from_zero 10+1 entries: No change in performance detected. time: [111.99 ns 112.38 ns 112.86 ns]
change: [-0.5451% -0.1237% +0.3279%] (p = 0.57 > 0.05)
coalesce_acked_from_zero 1000+1 entries: No change in performance detected. time: [93.417 ns 93.861 ns 94.290 ns]
change: [-0.9277% +0.1161% +1.1881%] (p = 0.83 > 0.05)
RxStreamOrderer::inbound_frame(): No change in performance detected. time: [115.21 ms 115.27 ms 115.33 ms]
change: [-0.0282% +0.0399% +0.1091%] (p = 0.28 > 0.05)
SentPackets::take_ranges: No change in performance detected. time: [8.3458 µs 8.5898 µs 8.8137 µs]
change: [-5.3496% -1.8959% +1.5433%] (p = 0.31 > 0.05)
transfer/pacing-false/varying-seeds: No change in performance detected. time: [35.343 ms 35.407 ms 35.472 ms]
change: [-0.4659% -0.1842% +0.0565%] (p = 0.18 > 0.05)
transfer/pacing-true/varying-seeds: Change within noise threshold. time: [36.385 ms 36.488 ms 36.592 ms]
change: [+0.7656% +1.1567% +1.5457%] (p = 0.00 < 0.05)
transfer/pacing-false/same-seed: Change within noise threshold. time: [35.304 ms 35.351 ms 35.399 ms]
change: [+0.1283% +0.3161% +0.5102%] (p = 0.00 < 0.05)
transfer/pacing-true/same-seed: Change within noise threshold. time: [37.035 ms 37.105 ms 37.175 ms]
change: [+1.0307% +1.2910% +1.5392%] (p = 0.00 < 0.05)
Client/server transfer resultsPerformance differences relative to 41036fc. Transfer of 33554432 bytes over loopback, 30 runs. All unit-less numbers are in milliseconds.
|
mxinden
approved these changes
Apr 7, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Because just building them can hide errors.