Revert "ci: Enable MC/DC coverage" #2575
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit d93f40c.
larseggert
approved these changes
Apr 9, 2025
|
We can then close #2574 |
Your call. My bad. Sorry for the noise. I am not on top of my GitHub notifications. |
Failed Interop TestsQUIC Interop Runner, client vs. server, differences relative to c1679d8. neqo-latest as client
neqo-latest as server
All resultsSucceeded Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
Unsupported Interop TestsQUIC Interop Runner, client vs. server neqo-latest as client
neqo-latest as server
|
|
I don't care which one we take forward - you pick :-) |
Benchmark resultsPerformance differences relative to c1679d8. 1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: Change within noise threshold. time: [737.70 ms 741.79 ms 745.90 ms]
thrpt: [134.07 MiB/s 134.81 MiB/s 135.56 MiB/s]
change:
time: [+0.4338% +1.2538% +2.1310%] (p = 0.00 < 0.05)
thrpt: [-2.0866% -1.2383% -0.4319%]
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: Change within noise threshold. time: [347.32 ms 348.86 ms 350.43 ms]
thrpt: [28.537 Kelem/s 28.665 Kelem/s 28.792 Kelem/s]
change:
time: [-1.4333% -0.8009% -0.1271%] (p = 0.02 < 0.05)
thrpt: [+0.1273% +0.8074% +1.4541%]
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: No change in performance detected. time: [25.009 ms 25.166 ms 25.325 ms]
thrpt: [39.486 elem/s 39.737 elem/s 39.985 elem/s]
change:
time: [-0.8375% +0.0730% +0.9829%] (p = 0.87 > 0.05)
thrpt: [-0.9733% -0.0729% +0.8445%]
1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: Change within noise threshold. time: [1.8579 s 1.8803 s 1.9046 s]
thrpt: [52.506 MiB/s 53.183 MiB/s 53.824 MiB/s]
change:
time: [-3.8010% -2.1602% -0.3189%] (p = 0.02 < 0.05)
thrpt: [+0.3199% +2.2079% +3.9511%]
decode 4096 bytes, mask ff: No change in performance detected. time: [12.081 µs 12.115 µs 12.158 µs]
change: [-0.1865% +0.3128% +0.9348%] (p = 0.31 > 0.05)
decode 1048576 bytes, mask ff: No change in performance detected. time: [3.1302 ms 3.1397 ms 3.1510 ms]
change: [-0.5345% -0.0517% +0.4423%] (p = 0.85 > 0.05)
decode 4096 bytes, mask 7f: No change in performance detected. time: [20.138 µs 20.183 µs 20.233 µs]
change: [-0.5604% -0.2386% +0.0664%] (p = 0.14 > 0.05)
decode 1048576 bytes, mask 7f: No change in performance detected. time: [5.2481 ms 5.2612 ms 5.2762 ms]
change: [-0.1949% +0.1334% +0.4710%] (p = 0.46 > 0.05)
decode 4096 bytes, mask 3f: No change in performance detected. time: [7.0237 µs 7.0528 µs 7.0891 µs]
change: [-0.0286% +0.6716% +1.5931%] (p = 0.10 > 0.05)
decode 1048576 bytes, mask 3f: No change in performance detected. time: [1.7905 ms 1.7949 ms 1.8006 ms]
change: [-0.6553% -0.1832% +0.2925%] (p = 0.46 > 0.05)
1 streams of 1 bytes/multistream: No change in performance detected. time: [72.934 µs 73.612 µs 74.764 µs]
change: [-0.6669% +0.3262% +1.8755%] (p = 0.74 > 0.05)
1000 streams of 1 bytes/multistream: Change within noise threshold. time: [24.846 ms 24.883 ms 24.920 ms]
change: [+0.0570% +0.2576% +0.4695%] (p = 0.02 < 0.05)
10000 streams of 1 bytes/multistream: No change in performance detected. time: [1.6838 s 1.6854 s 1.6870 s]
change: [-0.1749% -0.0351% +0.1079%] (p = 0.63 > 0.05)
1 streams of 1000 bytes/multistream: No change in performance detected. time: [74.468 µs 74.748 µs 75.028 µs]
change: [-1.8672% -0.3027% +0.7179%] (p = 0.74 > 0.05)
100 streams of 1000 bytes/multistream: No change in performance detected. time: [3.3743 ms 3.3805 ms 3.3872 ms]
change: [-0.1027% +0.1739% +0.4657%] (p = 0.23 > 0.05)
1000 streams of 1000 bytes/multistream: Change within noise threshold. time: [142.87 ms 142.95 ms 143.03 ms]
change: [+0.0597% +0.1365% +0.2124%] (p = 0.00 < 0.05)
coalesce_acked_from_zero 1+1 entries: No change in performance detected. time: [94.754 ns 95.058 ns 95.367 ns]
change: [-0.7094% -0.1846% +0.2983%] (p = 0.47 > 0.05)
coalesce_acked_from_zero 3+1 entries: No change in performance detected. time: [112.86 ns 113.17 ns 113.51 ns]
change: [+0.0205% +0.3773% +0.7689%] (p = 0.05 > 0.05)
coalesce_acked_from_zero 10+1 entries: No change in performance detected. time: [112.32 ns 112.72 ns 113.22 ns]
change: [-0.3408% +0.4778% +1.4445%] (p = 0.30 > 0.05)
coalesce_acked_from_zero 1000+1 entries: No change in performance detected. time: [93.438 ns 93.828 ns 94.262 ns]
change: [-0.6168% +1.3375% +3.5567%] (p = 0.20 > 0.05)
RxStreamOrderer::inbound_frame(): Change within noise threshold. time: [115.68 ms 115.73 ms 115.78 ms]
change: [+0.6957% +0.7607% +0.8267%] (p = 0.00 < 0.05)
SentPackets::take_ranges: No change in performance detected. time: [8.2492 µs 8.5097 µs 8.7575 µs]
change: [-4.4387% -1.6056% +1.1870%] (p = 0.27 > 0.05)
transfer/pacing-false/varying-seeds: Change within noise threshold. time: [36.063 ms 36.130 ms 36.197 ms]
change: [-0.7090% -0.4280% -0.1483%] (p = 0.00 < 0.05)
transfer/pacing-true/varying-seeds: No change in performance detected. time: [37.157 ms 37.263 ms 37.371 ms]
change: [-0.3782% +0.0323% +0.4667%] (p = 0.88 > 0.05)
transfer/pacing-false/same-seed: Change within noise threshold. time: [36.021 ms 36.067 ms 36.113 ms]
change: [+0.4799% +0.6563% +0.8333%] (p = 0.00 < 0.05)
transfer/pacing-true/same-seed: No change in performance detected. time: [37.426 ms 37.484 ms 37.542 ms]
change: [-0.2241% +0.0216% +0.2485%] (p = 0.86 > 0.05)
Client/server transfer resultsPerformance differences relative to c1679d8. Transfer of 33554432 bytes over loopback, 30 runs. All unit-less numbers are in milliseconds.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This reverts #2280.
Suggestion by @larseggert in #2570 (comment). Attempt to fix unrelated code coverage changes.
Additional benefit: Previously we would only do code coverage checking on Rust Nightly. While the Nightly jobs themselves are not marked as
required, the codecov report is. Thus two non-requiredNightly failures (4 total, minimum is 3) would block therequiredcode coverage step, thus blocking a merge. With this change, we are back to code coverage recording on stable.