[WIP] add submodule for C bindings, library compilation with PackageCompiler

[m-fila]

Adding C bindings and ahead of time compilation to shared library with PackageCompiler.jl or juliac

C components have jetreconstruction_(or JETRECONSTRUCTION_) prefix

For now the C API includes only:

• jet_reconstruct
• exclusive_jets, inclusive_jets
• enums and structs used in their signatures. When needed new structs with C-like layout are introduced (for instance replacing Vector{T} field with Ptr{T} and Csize_t fields for C-facing structs)

The C header file has to be written manually and keep in sync with Julia code. Technically the whole compile could be a separate julia package possibly eveb living in a separate repo, but I think it'd be harder to keep in sync this way

---

To do:

• write to an output pointer from inclusive_jets and exclusive_jets
• add error handling
• support Lorentz vectors
• add missing docstrings
• add precompilation statements
• cache all combinations of strategy and algorithm in precompilation
• use PseudoJet for pp and EEJet for ee algorithms
• have actual test
• test EE algorithms
• install JetReconstructionConfig.cmake
• compilation with upcoming juliac?
  - [ ] add c-style building (make/CMake)?

[codecov]

Codecov Report

Attention: Patch coverage is 69.00000% with 31 lines in your changes missing coverage. Please review.

Project coverage is 73.15%. Comparing base (82f1a80) to head (920b22b).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/C_JetReconstruction/C_JetReconstruction.jl	69.00%	31 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #88      +/-   ##
==========================================
- Coverage   73.81%   73.15%   -0.66%     
==========================================
  Files          18       19       +1     
  Lines        1245     1345     +100     
==========================================
+ Hits          919      984      +65     
- Misses        326      361      +35     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[m-fila]

The CI for Julia 1.9 fails as documenting@ccallable functions was fixed only in 1.11 😢

I couldn't think of a good way of error handling.
What I propose here is to:

• use status codes from enum with:
  • value for success,
  • values for each standard Julia exception,
  • single value for the remaining cases.
• print the exception and backtrace to stderr so it's not lost

I also couldn't find a reasonable way to translate Julia exceptions to C++ exceptions without going through C-bottleneck

[Moelf]

We don't need to support 1.9 I guess, but I hope we can backport JuliaLang/julia#51587 to 1.10.

[graeme-a-stewart]

Yeah, I would agree we can drop support for 1.9 now, but the LTS release (1.10) we should try to support

[graeme-a-stewart]

Just report this crashes on OS X with

[ Info: PackageCompiler: Executing /Users/graemes/.julia/dev/JetReconstruction/compile/precompile_execution.jl => /var/folders/kq/jgwv_wcj3ll2n6qwj06pp6jr0000gn/T/jl_packagecompiler_0hx01x/jl_dsyOUt
julia(84816,0x1f264cf40) malloc: *** error for object 0x134add910: pointer being freed was not allocated
julia(84816,0x1f264cf40) malloc: *** set a breakpoint in malloc_error_break to debug

[84816] signal 6: Abort trap: 6
in expression starting at /Users/graemes/.julia/dev/JetReconstruction/compile/precompile_execution.jl:33

I'm investigating!

[m-fila]

Thanks for checking. I think that was because the results_ptrpointee was managed by Julia and got garbage collected. Should be fixed now

[graeme-a-stewart]

Yes, now it works - thanks for the quick fix @m-fila!

[m-fila]

Added compiling with juliac from Julia nightlies. Unfortunately LoopVectorization isn't compatible with juliac and would have to be removed altogether from the project.
Also any kind of trimming produces a lot of errors and aborts the compilation.

With the nightlies I get a lot of warnings like this. They might be false positives though:

WARNING: Detected access to binding `JetReconstruction.##meta#64` in a world prior to its definition world.
  Julia 1.12 has introduced more strict world age semantics for global bindings.
  !!! This code may malfunction under Revise.
  !!! This code will error in future versions of Julia.
Hint: Add an appropriate `invokelatest` around the access to this binding.

Taking all into account some results

	PackageCompiler	juliac
Compilation time (s)	~730	~90
Lib size (M)	283	247
Deps size (M)	279	0

with juliac the explicit initialization and shutdown of julia runtime is gone. The pre-compilation is also gone (I didn't manage to reduce completely the overheads of first execution with PackageCompiler either with precompilation-statements, precompilations-execution or precompilation-execution on the test suite). Also now the compiled object doesn't bring any exclusive dependencies that should be kept around it.

[Moelf]

sounds good!

Added compiling with juliac from Julia nightlies. Unfortunately LoopVectorization isn't compatible with juliac and would have to be removed altogether from the project.

for this, I guess time to retry #84

[m-fila]

for this, I guess time to retry #84

Thankfully SIMD and its deps seem to be working fine with juliac

[Moelf]

I guess makes sense, it's just injecting LLVM IRs

[m-fila]

PackageCompiler with LoopVectorization compiles and runs but somehow is much slower ( more than x5) even than than a simple loop with no extra optimization