Skip to content

improvements to the non-clifford functionality #259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 21 commits into from
May 13, 2025
Merged

improvements to the non-clifford functionality #259

merged 21 commits into from
May 13, 2025

Conversation

Krastanov
Copy link
Member

No description provided.

@Krastanov
Copy link
Member Author

@hongyehu , this is what be built yesterday during the meeting, with a bit of polish. I tried to make the commits well separated and easy to read one by one. Hopefully this will make it easier to follow as you do not need to read the entire diff.

I will wait for tests to pass and then merge. I will add a long list of issues to track necessary enhancements.

@Krastanov Krastanov mentioned this pull request Apr 12, 2024
Open
9 tasks
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 12, 2024 •
edited
Loading

Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 12 Apr 2024 - 13:38
  • Baseline: 12 Apr 2024 - 13:45
  • Package commits:
  • Target: 4fef66
  • Baseline: 64f695
  • Julia commits:
  • Target: 1ae41a
  • Baseline: 1ae41a
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.93 (5%) ✅ 0.98 (1%) ✅
["ecc", "evaluate_decoder", "shor_pybp_comm"] 1.10 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 0.75 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 0.83 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "toric8_bp_comm"] 1.03 (5%) 1.04 (1%) ❌
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.03 (5%) 1.03 (1%) ❌
["stabilizer", "canon", "diag_rref500"] 1.06 (5%) ❌ 1.00 (1%)
["stabilizer", "trace", "destabilizer"] 0.91 (5%) ✅ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2544 s 0 s 289 s 6455 s 0 s
 #2 0 MHz 1822 s 0 s 288 s 7164 s 0 s
 #3 0 MHz 2181 s 0 s 292 s 6819 s 0 s
 #4 0 MHz 3122 s 0 s 272 s 5903 s 0 s
 Memory: 15.606498718261719 GB (13358.14453125 MB free)
 Uptime: 935.01 sec
 Load Avg: 1.0 1.09 0.8
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3520 s 0 s 382 s 9741 s 0 s
 #2 0 MHz 2872 s 0 s 360 s 10400 s 0 s
 #3 0 MHz 3337 s 0 s 390 s 9915 s 0 s
 #4 0 MHz 4441 s 0 s 374 s 8831 s 0 s
 Memory: 15.606498718261719 GB (13207.7734375 MB free)
 Uptime: 1371.81 sec
 Load Avg: 1.05 1.1 0.94
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 12 Apr 2024 - 13:38
  • Package commit: 4fef66
  • Julia commit: 1ae41a
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 14.956 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 169.877 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.279 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 117.250 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 12.998 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 116.057 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 62.076 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 173.353 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.102 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.201 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.543 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 157.774 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.084 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.856 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.518 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.622 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 132.046 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.056 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.334 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.823 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.788 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.123 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 560.627 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 50.795 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 24.326 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.519 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.676 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.866 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.783 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 979.950 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 560.638 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.449 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 688.406 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.159 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 569.814 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.729 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.304 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.659 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.295 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.826 ms (5%) 1.78 MiB (1%) 42707
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.093 ms (5%) 3.61 MiB (1%) 81494
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.535 ms (5%) 3.72 MiB (1%) 83014
["ecc", "evaluate_decoder", "shor_pybp_comm"] 58.789 ms (5%) 4.76 MiB (1%) 144207
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 119.038 ms (5%) 9.57 MiB (1%) 284018
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 124.628 ms (5%) 9.66 MiB (1%) 285104
["ecc", "evaluate_decoder", "shor_table_comm"] 257.010 μs (5%) 241.42 KiB (1%) 4593
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 896.275 μs (5%) 334.20 KiB (1%) 4790
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.296 ms (5%) 427.73 KiB (1%) 5876
["ecc", "evaluate_decoder", "toric8_bp_comm"] 706.062 ms (5%) 18.157 ms 174.51 MiB (1%) 1046123
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.429 s (5%) 25.180 ms 357.36 MiB (1%) 2163476
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.412 s (5%) 27.075 ms 350.81 MiB (1%) 2142613
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 459.145 ms (5%) 8.11 MiB (1%) 163696
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 968.377 ms (5%) 18.04 MiB (1%) 360388
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 983.377 ms (5%) 20.56 MiB (1%) 392802
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.156 ms (5%) 1.95 MiB (1%) 23176
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 12.443 ms (5%) 5.58 MiB (1%) 79355
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 21.042 ms (5%) 8.09 MiB (1%) 111769
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.613 ms (5%) 1.64 MiB (1%) 23082
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 10.148 ms (5%) 4.93 MiB (1%) 79160
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 18.707 ms (5%) 7.44 MiB (1%) 111574
["pauli", "mul", "100"] 14.496 ns (5%)
["pauli", "mul", "1000"] 17.628 ns (5%)
["pauli", "mul", "100000"] 730.320 ns (5%)
["pauli", "mul", "20000000"] 192.289 μs (5%)
["stabilizer", "canon", "cano500"] 3.074 ms (5%)
["stabilizer", "canon", "diag_cano500"] 709.174 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.078 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 675.983 μs (5%)
["stabilizer", "canon", "gott500"] 6.480 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.538 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.568 ms (5%)
["stabilizer", "canon", "rref500"] 3.148 ms (5%)
["stabilizer", "project", "destabilizer"] 18.485 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 8.906 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.405 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.948 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 30.918 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 23.854 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2544 s 0 s 289 s 6455 s 0 s
 #2 0 MHz 1822 s 0 s 288 s 7164 s 0 s
 #3 0 MHz 2181 s 0 s 292 s 6819 s 0 s
 #4 0 MHz 3122 s 0 s 272 s 5903 s 0 s
 Memory: 15.606498718261719 GB (13358.14453125 MB free)
 Uptime: 935.01 sec
 Load Avg: 1.0 1.09 0.8
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 12 Apr 2024 - 13:45
  • Package commit: 64f695
  • Julia commit: 1ae41a
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.339 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 171.411 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.385 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 118.872 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 13.479 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 117.109 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 59.482 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 169.527 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.094 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.001 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.292 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 155.120 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.086 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.862 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.489 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.612 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 131.897 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.051 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.334 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.626 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.710 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.122 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 560.507 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 50.956 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 24.385 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.259 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.785 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.632 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.705 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 980.362 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 560.507 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.441 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 685.972 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.161 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 569.925 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.759 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.294 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.668 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.294 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.810 ms (5%) 1.77 MiB (1%) 42567
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.089 ms (5%) 3.61 MiB (1%) 81480
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.890 ms (5%) 3.80 MiB (1%) 84890
["ecc", "evaluate_decoder", "shor_pybp_comm"] 53.401 ms (5%) 4.76 MiB (1%) 144207
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 159.039 ms (5%) 9.57 MiB (1%) 284018
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 150.257 ms (5%) 9.66 MiB (1%) 285104
["ecc", "evaluate_decoder", "shor_table_comm"] 255.052 μs (5%) 241.42 KiB (1%) 4593
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 865.618 μs (5%) 334.20 KiB (1%) 4790
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.264 ms (5%) 427.73 KiB (1%) 5876
["ecc", "evaluate_decoder", "toric8_bp_comm"] 684.192 ms (5%) 13.009 ms 168.33 MiB (1%) 1009765
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.387 s (5%) 23.748 ms 347.70 MiB (1%) 2106685
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.406 s (5%) 32.426 ms 352.19 MiB (1%) 2150726
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 479.316 ms (5%) 8.11 MiB (1%) 163696
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 1.000 s (5%) 18.04 MiB (1%) 360388
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 996.243 ms (5%) 20.56 MiB (1%) 392802
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.263 ms (5%) 1.95 MiB (1%) 23176
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 12.491 ms (5%) 5.58 MiB (1%) 79355
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 20.958 ms (5%) 8.09 MiB (1%) 111769
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.672 ms (5%) 1.64 MiB (1%) 23082
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 10.177 ms (5%) 4.93 MiB (1%) 79160
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 18.742 ms (5%) 7.44 MiB (1%) 111574
["pauli", "mul", "100"] 14.597 ns (5%)
["pauli", "mul", "1000"] 17.899 ns (5%)
["pauli", "mul", "100000"] 731.848 ns (5%)
["pauli", "mul", "20000000"] 202.258 μs (5%)
["stabilizer", "canon", "cano500"] 3.083 ms (5%)
["stabilizer", "canon", "diag_cano500"] 676.415 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.155 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 639.054 μs (5%)
["stabilizer", "canon", "gott500"] 6.540 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.583 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.625 ms (5%)
["stabilizer", "canon", "rref500"] 3.137 ms (5%)
["stabilizer", "project", "destabilizer"] 18.534 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 9.027 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.436 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.829 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 34.014 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 24.235 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3520 s 0 s 382 s 9741 s 0 s
 #2 0 MHz 2872 s 0 s 360 s 10400 s 0 s
 #3 0 MHz 3337 s 0 s 390 s 9915 s 0 s
 #4 0 MHz 4441 s 0 s 374 s 8831 s 0 s
 Memory: 15.606498718261719 GB (13207.7734375 MB free)
 Uptime: 1371.81 sec
 Load Avg: 1.05 1.1 0.94
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
Stepping: 1
BogoMIPS: 4890.85
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 64 KiB (2 instances)
L1i cache: 64 KiB (2 instances)
L2 cache: 1 MiB (2 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft
Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 12 Apr 2024 - 13:48
  • Baseline: 12 Apr 2024 - 13:55
  • Package commits:
  • Target: cb53f3
  • Baseline: 64f695
  • Julia commits:
  • Target: 1ae41a
  • Baseline: 1ae41a
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 0.93 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.90 (5%) ✅ 0.98 (1%) ✅
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 1.08 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 0.88 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "toric8_bp_comm"] 0.99 (5%) 1.01 (1%) ❌
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 0.97 (5%) 0.98 (1%) ✅
["stabilizer", "canon", "diag_cano500"] 1.11 (5%) ❌ 1.00 (1%)
["stabilizer", "canon", "diag_rref500"] 0.87 (5%) ✅ 1.00 (1%)
["stabilizer", "tensor", "diag_pow5_20"] 1.16 (5%) ❌ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2166 s 0 s 308 s 6606 s 0 s
 #2 0 MHz 2714 s 0 s 285 s 6107 s 0 s
 #3 0 MHz 2285 s 0 s 284 s 6498 s 0 s
 #4 0 MHz 2540 s 0 s 296 s 6242 s 0 s
 Memory: 15.606494903564453 GB (13380.4765625 MB free)
 Uptime: 915.57 sec
 Load Avg: 1.1 1.14 0.82
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2989 s 0 s 415 s 9989 s 0 s
 #2 0 MHz 3811 s 0 s 387 s 9224 s 0 s
 #3 0 MHz 3449 s 0 s 368 s 9567 s 0 s
 #4 0 MHz 3914 s 0 s 393 s 9088 s 0 s
 Memory: 15.606494903564453 GB (13298.3828125 MB free)
 Uptime: 1348.76 sec
 Load Avg: 1.0 1.06 0.93
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 12 Apr 2024 - 13:48
  • Package commit: cb53f3
  • Julia commit: 1ae41a
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 14.997 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 170.285 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 13.349 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 117.488 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 13.203 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 116.325 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 59.601 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 169.855 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.100 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 6.001 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.612 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 154.937 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.085 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.871 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.538 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 22.612 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 132.617 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.054 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.334 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.815 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.789 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.122 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 560.616 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 52.897 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 23.132 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.309 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.294 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.816 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.794 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 980.032 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 560.626 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.434 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 687.821 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.162 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 569.535 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.749 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.294 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.669 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.294 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.838 ms (5%) 1.76 MiB (1%) 42455
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.042 ms (5%) 3.62 MiB (1%) 81718
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.486 ms (5%) 3.71 MiB (1%) 82944
["ecc", "evaluate_decoder", "shor_pybp_comm"] 60.832 ms (5%) 4.76 MiB (1%) 144207
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 146.900 ms (5%) 9.57 MiB (1%) 284018
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 128.963 ms (5%) 9.66 MiB (1%) 285104
["ecc", "evaluate_decoder", "shor_table_comm"] 257.289 μs (5%) 241.42 KiB (1%) 4593
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 869.387 μs (5%) 334.20 KiB (1%) 4790
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.265 ms (5%) 427.73 KiB (1%) 5876
["ecc", "evaluate_decoder", "toric8_bp_comm"] 676.413 ms (5%) 10.161 ms 173.16 MiB (1%) 1038164
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.421 s (5%) 24.163 ms 352.70 MiB (1%) 2136078
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.377 s (5%) 28.011 ms 347.48 MiB (1%) 2123041
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 474.298 ms (5%) 8.11 MiB (1%) 163696
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 1.001 s (5%) 18.04 MiB (1%) 360388
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 1.010 s (5%) 20.56 MiB (1%) 392802
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.160 ms (5%) 1.95 MiB (1%) 23176
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 12.361 ms (5%) 5.58 MiB (1%) 79355
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 20.775 ms (5%) 8.09 MiB (1%) 111769
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.604 ms (5%) 1.64 MiB (1%) 23082
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 10.088 ms (5%) 4.93 MiB (1%) 79160
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 18.614 ms (5%) 7.44 MiB (1%) 111574
["pauli", "mul", "100"] 14.204 ns (5%)
["pauli", "mul", "1000"] 17.909 ns (5%)
["pauli", "mul", "100000"] 712.881 ns (5%)
["pauli", "mul", "20000000"] 196.774 μs (5%)
["stabilizer", "canon", "cano500"] 3.095 ms (5%)
["stabilizer", "canon", "diag_cano500"] 752.408 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.069 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 626.931 μs (5%)
["stabilizer", "canon", "gott500"] 6.415 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.548 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.575 ms (5%)
["stabilizer", "canon", "rref500"] 3.100 ms (5%)
["stabilizer", "project", "destabilizer"] 18.525 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 9.377 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.355 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.899 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 31.548 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 24.094 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2166 s 0 s 308 s 6606 s 0 s
 #2 0 MHz 2714 s 0 s 285 s 6107 s 0 s
 #3 0 MHz 2285 s 0 s 284 s 6498 s 0 s
 #4 0 MHz 2540 s 0 s 296 s 6242 s 0 s
 Memory: 15.606494903564453 GB (13380.4765625 MB free)
 Uptime: 915.57 sec
 Load Avg: 1.1 1.14 0.82
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 12 Apr 2024 - 13:55
  • Package commit: 64f695
  • Julia commit: 1ae41a
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 15.324 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 171.759 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 14.359 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 119.102 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 13.265 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 118.039 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 59.681 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 170.397 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.096 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 5.991 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 44.813 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 156.360 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.091 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.822 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.458 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 23.223 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 133.188 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 1.056 ms (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.394 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 11.747 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.819 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.122 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 561.704 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 50.523 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 23.363 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 26.309 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 13.314 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 11.806 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.843 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 980.380 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 561.704 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.434 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 687.427 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.161 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 569.809 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.769 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.294 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.688 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.294 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.826 ms (5%) 1.77 MiB (1%) 42721
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 4.086 ms (5%) 3.61 MiB (1%) 81690
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 5.003 ms (5%) 3.79 MiB (1%) 84792
["ecc", "evaluate_decoder", "shor_pybp_comm"] 59.083 ms (5%) 4.76 MiB (1%) 144207
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 136.484 ms (5%) 9.57 MiB (1%) 284018
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 145.730 ms (5%) 9.66 MiB (1%) 285104
["ecc", "evaluate_decoder", "shor_table_comm"] 255.874 μs (5%) 241.42 KiB (1%) 4593
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 887.918 μs (5%) 334.20 KiB (1%) 4790
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.275 ms (5%) 427.73 KiB (1%) 5876
["ecc", "evaluate_decoder", "toric8_bp_comm"] 684.094 ms (5%) 15.175 ms 170.60 MiB (1%) 1023114
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.390 s (5%) 19.896 ms 350.72 MiB (1%) 2124437
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.427 s (5%) 24.819 ms 356.12 MiB (1%) 2173840
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 488.487 ms (5%) 8.11 MiB (1%) 163696
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 1.006 s (5%) 18.04 MiB (1%) 360388
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 1.002 s (5%) 20.56 MiB (1%) 392802
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 4.279 ms (5%) 1.95 MiB (1%) 23176
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 12.430 ms (5%) 5.58 MiB (1%) 79355
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 20.840 ms (5%) 8.09 MiB (1%) 111769
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.643 ms (5%) 1.64 MiB (1%) 23082
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 10.061 ms (5%) 4.93 MiB (1%) 79160
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 18.486 ms (5%) 7.44 MiB (1%) 111574
["pauli", "mul", "100"] 14.234 ns (5%)
["pauli", "mul", "1000"] 17.748 ns (5%)
["pauli", "mul", "100000"] 721.866 ns (5%)
["pauli", "mul", "20000000"] 203.167 μs (5%)
["stabilizer", "canon", "cano500"] 3.106 ms (5%)
["stabilizer", "canon", "diag_cano500"] 676.426 μs (5%)
["stabilizer", "canon", "diag_gott500"] 4.039 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 721.900 μs (5%)
["stabilizer", "canon", "gott500"] 6.428 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.550 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.564 ms (5%)
["stabilizer", "canon", "rref500"] 3.144 ms (5%)
["stabilizer", "project", "destabilizer"] 18.655 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 9.257 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.171 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.817 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 31.288 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 24.216 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2989 s 0 s 415 s 9989 s 0 s
 #2 0 MHz 3811 s 0 s 387 s 9224 s 0 s
 #3 0 MHz 3449 s 0 s 368 s 9567 s 0 s
 #4 0 MHz 3914 s 0 s 393 s 9088 s 0 s
 Memory: 15.606494903564453 GB (13298.3828125 MB free)
 Uptime: 1348.76 sec
 Load Avg: 1.0 1.06 0.93
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Runtime information

Runtime Info
BLAS #threads 2
BLAS.vendor() lbt
Sys.CPU_THREADS 4

lscpu output:

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: AuthenticAMD
Model name: AMD EPYC 7763 64-Core Processor
CPU family: 25
Model: 1
Thread(s) per core: 2
Core(s) per socket: 2
Socket(s): 1
Stepping: 1
BogoMIPS: 4890.86
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl tsc_reliable nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext invpcid_single vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr rdpru arat npt nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload umip vaes vpclmulqdq rdpid fsrm
Virtualization: AMD-V
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 64 KiB (2 instances)
L1i cache: 64 KiB (2 instances)
L2 cache: 1 MiB (2 instances)
L3 cache: 32 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, STIBP disabled, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Cpu Property Value
Brand AMD EPYC 7763 64-Core Processor
Vendor :AMD
Architecture :Unknown
Model Family: 0xaf, Model: 0x01, Stepping: 0x01, Type: 0x00
Cores 16 physical cores, 16 logical cores (on executing CPU)
No Hyperthreading hardware capability detected
Clock Frequencies Not supported by CPU
Data Cache Level 1:3 : (32, 512, 32768) kbytes
64 byte cache line size
Address Size 48 bits virtual, 48 bits physical
SIMD 256 bit = 32 byte max. SIMD vector size
Time Stamp Counter TSC is accessible via rdtsc
TSC runs at constant rate (invariant from clock frequency)
Perf. Monitoring Performance Monitoring Counters (PMC) are not supported
Hypervisor Yes, Microsoft
Benchmark Result

Judge result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmarks:
  • Target: 12 Apr 2024 - 13:56
  • Baseline: 12 Apr 2024 - 14:04
  • Package commits:
  • Target: 3d58e9
  • Baseline: 64f695
  • Julia commits:
  • Target: 1ae41a
  • Baseline: 1ae41a
  • Julia command flags:
  • Target: None
  • Baseline: None
  • Environment variables:
  • Target: None
  • Baseline: None

Results

A ratio greater than 1.0 denotes a possible regression (marked with ❌), while a ratio less
than 1.0 denotes a possible improvement (marked with ✅). Only significant results - results
that indicate possible regressions or improvements - are shown below (thus, an empty table means that all
benchmark results remained invariant between builds).

ID time ratio memory ratio
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 0.90 (5%) ✅ 1.00 (1%)
["circuitsim", "mctrajectories_union", "q1001_r1"] 0.92 (5%) ✅ 1.00 (1%)
["circuitsim", "pftrajectories_union", "q101_r1"] 0.92 (5%) ✅ 1.00 (1%)
["clifford", "dense", "cnot250_on_dense500_destab"] 1.16 (5%) ❌ 1.00 (1%)
["clifford", "dense", "cnot250_on_diag500_stab"] 0.88 (5%) ✅ 1.00 (1%)
["clifford", "dense", "dense500_on_diag500_stab"] 0.87 (5%) ✅ 1.00 (1%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 0.93 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 0.90 (5%) ✅ 0.98 (1%) ✅
["ecc", "evaluate_decoder", "shor_pybp_comm"] 1.12 (5%) ❌ 1.00 (1%)
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 0.86 (5%) ✅ 1.00 (1%)
["ecc", "evaluate_decoder", "toric8_bp_comm"] 0.94 (5%) ✅ 0.96 (1%) ✅
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.00 (5%) 1.01 (1%) ❌
["pauli", "mul", "100000"] 0.92 (5%) ✅ 1.00 (1%)
["stabilizer", "canon", "diag_cano500"] 1.06 (5%) ❌ 1.00 (1%)
["stabilizer", "canon", "diag_rref500"] 1.15 (5%) ❌ 1.00 (1%)
["stabilizer", "canon", "md_cano500"] 1.06 (5%) ❌ 1.00 (1%)
["stabilizer", "project", "destabilizer"] 0.93 (5%) ✅ 1.00 (1%)
["stabilizer", "project", "stabilizer"] 1.08 (5%) ❌ 1.00 (1%)
["stabilizer", "trace", "destabilizer"] 1.08 (5%) ❌ 1.00 (1%)

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Target

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2543 s 0 s 272 s 5972 s 0 s
 #2 0 MHz 2933 s 0 s 284 s 5595 s 0 s
 #3 0 MHz 2016 s 0 s 298 s 6477 s 0 s
 #4 0 MHz 1869 s 0 s 293 s 6632 s 0 s
 Memory: 15.606498718261719 GB (13355.90625 MB free)
 Uptime: 885.05 sec
 Load Avg: 1.08 1.09 0.76
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Baseline

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 3595 s 0 s 356 s 9082 s 0 s
 #2 0 MHz 3962 s 0 s 394 s 8704 s 0 s
 #3 0 MHz 3468 s 0 s 391 s 9179 s 0 s
 #4 0 MHz 2731 s 0 s 387 s 9923 s 0 s
 Memory: 15.606498718261719 GB (13284.78515625 MB free)
 Uptime: 1311.18 sec
 Load Avg: 1.09 1.13 0.93
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Target result

Benchmark Report for /home/runner/work/QuantumClifford.jl/QuantumClifford.jl

Job Properties

  • Time of benchmark: 12 Apr 2024 - 13:56
  • Package commit: 3d58e9
  • Julia commit: 1ae41a
  • Julia command flags: None
  • Environment variables: None

Results

Below is a table of this job's results, obtained by running the benchmarks.
The values listed in the ID column have the structure [parent_group, child_group, ..., key], and can be used to
index into the BaseBenchmarks suite to retrieve the corresponding benchmarks.
The percentages accompanying time and memory values in the below table are noise tolerances. The "true"
time/memory value for a given benchmark is expected to fall within this percentage of the reported value.
An empty cell means that the value was zero.

ID time GC time memory allocations
["circuitsim", "mctrajectories", "q1001_r1"] 14.300 ms (5%) 500.77 KiB (1%) 18019
["circuitsim", "mctrajectories", "q101_r1"] 159.397 μs (5%) 50.53 KiB (1%) 1818
["circuitsim", "mctrajectories_sumtype", "q1001_r1"] 12.642 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_sumtype", "q101_r1"] 109.774 μs (5%) 256 bytes (1%) 8
["circuitsim", "mctrajectories_union", "q1001_r1"] 12.614 ms (5%) 496 bytes (1%) 9
["circuitsim", "mctrajectories_union", "q101_r1"] 109.143 μs (5%) 256 bytes (1%) 8
["circuitsim", "pftrajectories", "q1001_r1"] 58.399 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r100"] 161.982 μs (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q1001_r10000"] 1.017 ms (5%) 93.80 KiB (1%) 2001
["circuitsim", "pftrajectories", "q101_r1"] 5.861 μs (5%) 9.42 KiB (1%) 201
["circuitsim", "pftrajectories_sumtype", "q1001_r1"] 40.645 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r100"] 145.422 μs (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000"] 1.003 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q1001_r10000_fastrow"] 5.391 ms (5%) 93.84 KiB (1%) 2002
["circuitsim", "pftrajectories_sumtype", "q101_r1"] 4.117 μs (5%) 9.47 KiB (1%) 202
["circuitsim", "pftrajectories_union", "q1001_r1"] 20.858 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r100"] 123.748 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q1001_r10000"] 973.621 μs (5%) 96 bytes (1%) 2
["circuitsim", "pftrajectories_union", "q101_r1"] 2.164 μs (5%) 96 bytes (1%) 2
["clifford", "dense", "cnot250_on_dense500_destab"] 12.768 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_dense500_stab"] 5.441 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_destab"] 1.038 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot250_on_diag500_stab"] 455.187 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "cnot_on_dense500_destab"] 46.927 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_dense500_stab"] 21.681 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_destab"] 24.225 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "cnot_on_diag500_stab"] 12.133 μs (5%) 368 bytes (1%) 10
["clifford", "dense", "dense500_on_dense500_destab"] 10.692 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_dense500_stab"] 5.310 ms (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_destab"] 903.782 μs (5%) 512 bytes (1%) 8
["clifford", "dense", "dense500_on_diag500_stab"] 451.490 μs (5%) 512 bytes (1%) 8
["clifford", "symbolic", "cnot250_on_dense500_destab"] 1.323 ms (5%)
["clifford", "symbolic", "cnot250_on_dense500_stab"] 634.651 μs (5%)
["clifford", "symbolic", "cnot250_on_diag500_destab"] 1.076 ms (5%)
["clifford", "symbolic", "cnot250_on_diag500_stab"] 525.618 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_destab"] 4.388 μs (5%)
["clifford", "symbolic", "cnot_on_dense500_stab"] 2.123 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_destab"] 4.328 μs (5%)
["clifford", "symbolic", "cnot_on_diag500_stab"] 2.113 μs (5%)
["ecc", "evaluate_decoder", "shor_bp_comm"] 1.705 ms (5%) 1.77 MiB (1%) 42553
["ecc", "evaluate_decoder", "shor_bp_naivesyn"] 3.787 ms (5%) 3.61 MiB (1%) 81634
["ecc", "evaluate_decoder", "shor_bp_shorsyn"] 4.164 ms (5%) 3.70 MiB (1%) 82636
["ecc", "evaluate_decoder", "shor_pybp_comm"] 64.280 ms (5%) 4.76 MiB (1%) 144207
["ecc", "evaluate_decoder", "shor_pybp_naivesyn"] 133.041 ms (5%) 9.57 MiB (1%) 284018
["ecc", "evaluate_decoder", "shor_pybp_shorsyn"] 129.201 ms (5%) 9.66 MiB (1%) 285104
["ecc", "evaluate_decoder", "shor_table_comm"] 241.610 μs (5%) 241.42 KiB (1%) 4593
["ecc", "evaluate_decoder", "shor_table_naivesyn"] 798.937 μs (5%) 334.20 KiB (1%) 4790
["ecc", "evaluate_decoder", "shor_table_shorsyn"] 1.172 ms (5%) 427.73 KiB (1%) 5876
["ecc", "evaluate_decoder", "toric8_bp_comm"] 640.512 ms (5%) 10.718 ms 166.49 MiB (1%) 998936
["ecc", "evaluate_decoder", "toric8_bp_naivesyn"] 1.387 s (5%) 24.453 ms 351.19 MiB (1%) 2127202
["ecc", "evaluate_decoder", "toric8_bp_shorsyn"] 1.386 s (5%) 25.639 ms 354.44 MiB (1%) 2163963
["ecc", "evaluate_decoder", "toric8_pybp_comm"] 457.341 ms (5%) 8.11 MiB (1%) 163696
["ecc", "evaluate_decoder", "toric8_pybp_naivesyn"] 927.357 ms (5%) 18.04 MiB (1%) 360388
["ecc", "evaluate_decoder", "toric8_pybp_shorsyn"] 955.391 ms (5%) 20.56 MiB (1%) 392802
["ecc", "evaluate_decoder", "toric8_pymatch_comm"] 3.883 ms (5%) 1.95 MiB (1%) 23176
["ecc", "evaluate_decoder", "toric8_pymatch_naivesyn"] 11.745 ms (5%) 5.58 MiB (1%) 79355
["ecc", "evaluate_decoder", "toric8_pymatch_shorsyn"] 19.647 ms (5%) 8.09 MiB (1%) 111769
["ecc", "evaluate_decoder", "toric8_table_comm"] 2.454 ms (5%) 1.64 MiB (1%) 23082
["ecc", "evaluate_decoder", "toric8_table_naivesyn"] 9.399 ms (5%) 4.93 MiB (1%) 79160
["ecc", "evaluate_decoder", "toric8_table_shorsyn"] 17.538 ms (5%) 7.44 MiB (1%) 111574
["pauli", "mul", "100"] 13.562 ns (5%)
["pauli", "mul", "1000"] 17.344 ns (5%)
["pauli", "mul", "100000"] 681.267 ns (5%)
["pauli", "mul", "20000000"] 163.234 μs (5%)
["stabilizer", "canon", "cano500"] 2.887 ms (5%)
["stabilizer", "canon", "diag_cano500"] 694.102 μs (5%)
["stabilizer", "canon", "diag_gott500"] 3.778 ms (5%) 5.19 MiB (1%) 34521
["stabilizer", "canon", "diag_rref500"] 665.559 μs (5%)
["stabilizer", "canon", "gott500"] 6.071 ms (5%) 5.19 MiB (1%) 34531
["stabilizer", "canon", "md_cano500"] 1.504 ms (5%)
["stabilizer", "canon", "md_rref500"] 1.493 ms (5%)
["stabilizer", "canon", "rref500"] 2.939 ms (5%)
["stabilizer", "project", "destabilizer"] 16.992 μs (5%) 288 bytes (1%) 5
["stabilizer", "project", "stabilizer"] 8.897 μs (5%) 80 bytes (1%) 2
["stabilizer", "tensor", "diag_pow5_20"] 1.080 ms (5%) 23.97 MiB (1%) 34
["stabilizer", "tensor", "pow5_20"] 2.679 μs (5%) 6.44 KiB (1%) 31
["stabilizer", "trace", "destabilizer"] 30.887 μs (5%) 80 bytes (1%) 2
["stabilizer", "trace", "stabilizer"] 22.552 μs (5%) 112 bytes (1%) 3

Benchmark Group List

Here's a list of all the benchmark groups executed by this job:

  • ["circuitsim", "mctrajectories"]
  • ["circuitsim", "mctrajectories_sumtype"]
  • ["circuitsim", "mctrajectories_union"]
  • ["circuitsim", "pftrajectories"]
  • ["circuitsim", "pftrajectories_sumtype"]
  • ["circuitsim", "pftrajectories_union"]
  • ["clifford", "dense"]
  • ["clifford", "symbolic"]
  • ["ecc", "evaluate_decoder"]
  • ["pauli", "mul"]
  • ["stabilizer", "canon"]
  • ["stabilizer", "project"]
  • ["stabilizer", "tensor"]
  • ["stabilizer", "trace"]

Julia versioninfo

Julia Version 1.12.0-DEV.334
Commit 1ae41a2c0a3 (2024-04-12 03:14 UTC)
Build Info:
 Official https://julialang.org/ release
Platform Info:
 OS: Linux (x86_64-linux-gnu)
 Ubuntu 22.04.4 LTS
 uname: Linux 6.5.0-1017-azure #17~22.04.1-Ubuntu SMP Sat Mar 9 04:50:38 UTC 2024 x86_64 x86_64
 CPU: AMD EPYC 7763 64-Core Processor: 
 speed user nice sys idle irq
 #1 0 MHz 2543 s 0 s 272 s 5972 s 0 s
 #2 0 MHz 2933 s 0 s 284 s 5595 s 0 s
 #3 0 MHz 2016 s 0 s 298 s 6477 s 0 s
 #4 0 MHz 1869 s 0 s 293 s 6632 s 0 s
 Memory: 15.606498718261719 GB (13355.90625 MB free)
 Uptime: 885.05 sec
 Load Avg: 1.08 1.09 0.76
 WORD_SIZE: 64
 LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtu...*[Comment body truncated]*

@Krastanov
Copy link
Member Author

@hongyehu , there are some failures related to the expectation value (the tests in test_nonclifford_quantumoptics.jl). I will not have the time to investigate today, but it seems I have messed up a convention somewhere and we are getting expectation values that are occasionally imaginary. Hopefully I will get to it early next week, but let me know if you would like to discuss it together if you have time to investigate it.

@hongyehu
Copy link

@Krastanov Hi Stefan, I can take a look into the expect! function and see whether we missed anything!

@Krastanov
Copy link
Member Author

here is a small reproducers:

using QuantumClifford
using QuantumOpticsBase # which provides Operator and Ket for conversion (kinda like qutip)
using LinearAlgebra

qo_basis = SpinBasis(1//2)
qo_tgate = sparse(identityoperator(qo_basis))
qo_tgate.data[2,2] = exp(im*pi/4)

for s in [S"X", S"Y", S"Z", S"-X", S"-Y", S"-Z"]
    for p in [P"X", P"Y", P"Z", P"-X", P"-Y", P"-Z"]
        gs = GeneralizedStabilizer(s)
        apply!(gs, pcT)
        ρ = dm(qo_tgate*Ket(s))
        @test Operator(gs) ≈ ρ
        if isapprox(expect(p, gs), expect(Operator(p),ρ); atol=1e-5)
        else
            println("failure of expectation calculation for state = T*(", s, ") and observable = ", p)
        end
    end
end

It prints the following failing cases:

failure of expect for state = T*(+ X) and observable = + Y
failure of expect for state = T*(+ X) and observable = - Y
failure of expect for state = T*(+ Y) and observable = + X
failure of expect for state = T*(+ Y) and observable = - X
failure of expect for state = T*(- X) and observable = + Y
failure of expect for state = T*(- X) and observable = - Y
failure of expect for state = T*(- Y) and observable = + X
failure of expect for state = T*(- Y) and observable = - X

@Krastanov Krastanov mentioned this pull request Jul 9, 2024
Open
@Fe-r-oz

This comment has been minimized.

Sign in to view
@Krastanov
Copy link
Member Author

Hi, Feroz. I am not following what you mean with your last comment. If you want to discus some changes to the code, I would suggest making a new branch or even a pull request on which we can talk. Very importantly, though, if you want to discuss specific changes, make sure you have clear separation of a single semantic change per commit. You will need to learn how to use an interactive rebase and potentially do force pushes, to keep the history of your changes easy to work with.

@Fe-r-oz Fe-r-oz mentioned this pull request Jul 11, 2024
Closed
@Fe-r-oz Fe-r-oz mentioned this pull request Aug 10, 2024
Closed
@Krastanov Krastanov marked this pull request as draft August 10, 2024 20:42
Fe-r-oz added a commit to Fe-r-oz/QuantumClifford.jl that referenced this pull request Aug 22, 2024
@Fe-r-oz
Implement first phase of QuantumSavory#309: GeneralizedStabilizer API…
fcf14d2 
… improvements (building on QuantumSavory#259 and addressing QuantumSavory#260 partially)
Fe-r-oz added a commit to Fe-r-oz/QuantumClifford.jl that referenced this pull request Aug 22, 2024
@Fe-r-oz
Implement first phase of QuantumSavory#309: GeneralizedStabilizer API…
46bfaac 
… improvements (building on QuantumSavory#259 and addressing QuantumSavory#260 partially)
This was referenced Aug 22, 2024
Merged
Merged
@Fe-r-oz Fe-r-oz mentioned this pull request Dec 7, 2024
Merged
Fe-r-oz and others added 2 commits April 29, 2025 07:17
@codecov Codecov
Copy link

codecov bot commented Apr 29, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 96.92308% with 4 lines in your changes missing coverage. Please review.

Project coverage is 83.78%. Comparing base (b1079e0) to head (3b7bb0e).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/nonclifford.jl 97.29% 3 Missing ⚠️
src/QuantumClifford.jl 92.30% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master     #259      +/-   ##
==========================================
+ Coverage   83.52%   83.78%   +0.25%     
==========================================
  Files          72       72              
  Lines        4813     4921     +108     
==========================================
+ Hits         4020     4123     +103     
- Misses        793      798       +5

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Fe-r-oz added 2 commits April 29, 2025 19:28
@Fe-r-oz
use testitem for test_nonclifford_quantumoptics so tests actually run…
6a8290d 
… in testing workflow
@Fe-r-oz
improve doctest for projectrand, since it gives random output, we use…
3b7bb0e 
… _projectrand_notnorm to control which eigenstate to project for documentation sake
@Fe-r-oz
Copy link
Collaborator

Fe-r-oz commented Apr 29, 2025

@Krastanov, Hi Stefan, The PR is now ready for review! I’ve resolved the merge conflicts, and the failing tests are unrelated to this PR (they’re from the usual noisy noisycircuit tests). For the projectrand! doctest, since it gives random results, I used _projectrand_notnorm to explicitly control the projection branch (+1 or -1). This ensures the doctest passes reliably for documentation purposes. This documentation change was introduced in 3b7bb0e. Please feel free to adjust this documentation change further if needed.

Thank you so much for your guidance and support throughout this process—I really appreciate it! ✨🙂

@Fe-r-oz Fe-r-oz marked this pull request as ready for review May 1, 2025 04:36
@Krastanov Krastanov merged commit 08c4f9d into master May 13, 2025
18 of 23 checks passed
@Krastanov Krastanov deleted the nonclif branch May 13, 2025 12:28
@Fe-r-oz Fe-r-oz mentioned this pull request May 24, 2025
Open
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
nonclifford
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@Krastanov @hongyehu @Fe-r-oz