Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Use samply to convert perf data for the Firefox profiler #2579

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

larseggert
Copy link
Collaborator

@larseggert larseggert commented Apr 10, 2025

This should allow finer-grained stack- and flamegraphs that show inlined functions. Also might lead to smaller artifact sizes.

Copy link

codecov bot commented Apr 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.53%. Comparing base (28c0ea7) to head (62b5f66).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2579      +/-   ##
==========================================
+ Coverage   95.52%   95.53%   +0.01%     
==========================================
  Files         115      115              
  Lines       37275    37275              
  Branches    37275    37275              
==========================================
+ Hits        35607    35612       +5     
+ Misses       1662     1659       -3     
+ Partials        6        4       -2     
Components Coverage Δ
neqo-common 97.52% <ø> (+0.35%) ⬆️
neqo-crypto 90.45% <ø> (ø)
neqo-http3 94.50% <ø> (ø)
neqo-qpack 96.29% <ø> (ø)
neqo-transport 96.50% <ø> (ø)
neqo-udp 90.10% <ø> (ø)

Copy link

github-actions bot commented Apr 10, 2025

Failed Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Copy link

github-actions bot commented Apr 10, 2025

Benchmark results

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client:
       time:   [712.87 ms 717.02 ms 721.21 ms]
       thrpt:  [138.66 MiB/s 139.47 MiB/s 140.28 MiB/s]
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client:
       time:   [351.61 ms 353.22 ms 354.83 ms]
       thrpt:  [28.183 Kelem/s 28.311 Kelem/s 28.440 Kelem/s]
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client:
       time:   [25.045 ms 25.198 ms 25.359 ms]
       thrpt:  [39.434  elem/s 39.685  elem/s 39.927  elem/s]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client:
       time:   [1.9195 s 1.9385 s 1.9588 s]
       thrpt:  [51.052 MiB/s 51.587 MiB/s 52.096 MiB/s]
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild
decode 4096 bytes, mask ff:
       time:   [12.023 µs 12.072 µs 12.125 µs]
Found 18 outliers among 100 measurements (18.00%)
  4 (4.00%) low severe
  2 (2.00%) low mild
  1 (1.00%) high mild
  11 (11.00%) high severe
decode 1048576 bytes, mask ff:
       time:   [2.9529 ms 2.9616 ms 2.9714 ms]
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high severe
decode 4096 bytes, mask 7f:
       time:   [20.010 µs 20.064 µs 20.122 µs]
Found 18 outliers among 100 measurements (18.00%)
  2 (2.00%) low severe
  2 (2.00%) low mild
  2 (2.00%) high mild
  12 (12.00%) high severe
decode 1048576 bytes, mask 7f:
       time:   [4.7983 ms 4.8116 ms 4.8263 ms]
Found 16 outliers among 100 measurements (16.00%)
  16 (16.00%) high severe
decode 4096 bytes, mask 3f:
       time:   [6.3212 µs 6.3495 µs 6.3843 µs]
Found 17 outliers among 100 measurements (17.00%)
  4 (4.00%) low severe
  2 (2.00%) low mild
  4 (4.00%) high mild
  7 (7.00%) high severe
decode 1048576 bytes, mask 3f:
       time:   [2.1483 ms 2.1540 ms 2.1610 ms]
Found 13 outliers among 100 measurements (13.00%)
  6 (6.00%) low mild
  1 (1.00%) high mild
  6 (6.00%) high severe
1 streams of 1 bytes/multistream:
       time:   [72.646 µs 74.054 µs 75.891 µs]
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high severe
1000 streams of 1 bytes/multistream:
       time:   [25.356 ms 25.389 ms 25.422 ms]
10000 streams of 1 bytes/multistream:
       time:   [1.7012 s 1.7030 s 1.7048 s]
Found 27 outliers among 100 measurements (27.00%)
  7 (7.00%) low severe
  7 (7.00%) low mild
  5 (5.00%) high mild
  8 (8.00%) high severe
1 streams of 1000 bytes/multistream:
       time:   [74.305 µs 74.580 µs 74.845 µs]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) low mild
100 streams of 1000 bytes/multistream:
       time:   [3.3801 ms 3.3871 ms 3.3947 ms]
Found 23 outliers among 100 measurements (23.00%)
  23 (23.00%) high severe
1000 streams of 1000 bytes/multistream:
       time:   [143.76 ms 143.83 ms 143.90 ms]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
coalesce_acked_from_zero 1+1 entries:
       time:   [94.623 ns 94.937 ns 95.255 ns]
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe
coalesce_acked_from_zero 3+1 entries:
       time:   [112.54 ns 112.81 ns 113.10 ns]
Found 11 outliers among 100 measurements (11.00%)
  1 (1.00%) low severe
  1 (1.00%) high mild
  9 (9.00%) high severe
coalesce_acked_from_zero 10+1 entries:
       time:   [111.97 ns 112.33 ns 112.79 ns]
Found 18 outliers among 100 measurements (18.00%)
  4 (4.00%) low severe
  2 (2.00%) low mild
  4 (4.00%) high mild
  8 (8.00%) high severe
coalesce_acked_from_zero 1000+1 entries:
       time:   [92.956 ns 93.443 ns 93.992 ns]
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
RxStreamOrderer::inbound_frame():
       time:   [116.48 ms 116.53 ms 116.59 ms]
Found 15 outliers among 100 measurements (15.00%)
  2 (2.00%) low severe
  3 (3.00%) low mild
  7 (7.00%) high mild
  3 (3.00%) high severe
SentPackets::take_ranges:
       time:   [8.4392 µs 8.6735 µs 8.8965 µs]
Found 17 outliers among 100 measurements (17.00%)
  4 (4.00%) low severe
  11 (11.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe
transfer/pacing-false/varying-seeds:
       time:   [36.066 ms 36.131 ms 36.196 ms]
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) low mild
transfer/pacing-true/varying-seeds:
       time:   [37.055 ms 37.156 ms 37.259 ms]
transfer/pacing-false/same-seed:
       time:   [35.990 ms 36.032 ms 36.074 ms]
transfer/pacing-true/same-seed:
       time:   [37.444 ms 37.510 ms 37.576 ms]
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild

Client/server transfer results

Transfer of 33554432 bytes over loopback, 30 runs. All unit-less numbers are in milliseconds.

Client Server CC Pacing Mean ± σ Min Max MiB/s ± σ Δ main Δ main
neqo neqo reno on 337.1 ± 33.5 306.0 438.4
neqo neqo reno 363.6 ± 83.8 303.2 644.5
neqo neqo cubic on 339.5 ± 37.3 307.9 498.1
neqo neqo cubic 329.9 ± 30.2 307.1 468.5
google neqo reno on 776.1 ± 89.3 571.7 957.5
google neqo reno 773.3 ± 84.8 572.2 973.4
google neqo cubic on 769.4 ± 80.0 579.2 859.4
google neqo cubic 772.0 ± 87.3 575.3 913.2
google google 577.5 ± 41.8 546.6 723.3
neqo msquic reno on 265.0 ± 17.7 246.3 310.5
neqo msquic reno 264.1 ± 16.6 228.1 310.2
neqo msquic cubic on 264.6 ± 15.4 243.3 312.2
neqo msquic cubic 261.8 ± 15.2 241.8 308.7
msquic msquic 180.0 ± 42.9 148.4 358.0

⬇️ Download logs

@larseggert
Copy link
Collaborator Author

This is waiting for @mstange to add a --presymbolicate flag as discussed on Slack.

@larseggert larseggert added the blocked Blocked on something else label Apr 11, 2025
@larseggert larseggert changed the title ci: Samply again ci: Use samply to convert perf data for the Firefox profiler Apr 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked Blocked on something else
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant