ENH: Replace unsigned long with uint64_t for non-zero Jacobian indices

[N-Dekker]

Aims to ease interaction with LibTorch (specifically when passing such indices as data parameter to torch::from_blob). unsigned long appears problematic, as it is typically 32-bit on Windows, while it's typically 64-bit on Linux.

Issue encountered during the development of the IMPACT Similarity Metric, pull request #1311 by Valentin Boussot (@vboussot) et al.

---

@mstaring @stefanklein @vboussot I still think it would be preferable to ban long and unsigned long from the entire source tree of elastix, in favor of int64_t and uint64_t respectively. But for the interaction with LibTorch I think 🙏 this little pull request should be sufficient.

[vboussot]

@N-Dekker Thanks a lot for the modification! Just a quick note: torch::kLong in PyTorch corresponds to int64_t, so if the data is declared as uint64_t, torch::from_blob(..., torch::kLong) will interpret the underlying memory as signed. In practice, values from NonZeroJacobianIndicesType will likely never exceed INT64_MAX, so it should be fine, but it's something to keep in mind for strict type compatibility.

[N-Dekker]

Thanks @vboussot Would it be possible to use torch::kUInt64 instead? 🤔 As in torch::from_blob(..., torch::kUInt64)?

[vboussot]

Unfortunately, LibTorch doesn’t support unsigned integer types beyond kUInt8 https://pytorch.org/docs/stable/tensor_attributes.html#torch.dtype

[N-Dekker]

Strange... I did find kUInt64 in the LibTorch 2.6.0 distribution:

libtorch-win-shared-with-deps-debug-2.6.0+cpu\libtorch\include\torch\csrc\api\include\torch\types.h from line 54:

constexpr auto kUInt64 = at::kUInt64;
constexpr auto kFloat16 = at::kHalf;
constexpr auto kFloat32 = at::kFloat;
constexpr auto kFloat64 = at::kDouble;

/// Rust-style short dtypes.
constexpr auto kU8 = kUInt8;
constexpr auto kU16 = kUInt16;
constexpr auto kU32 = kUInt32;
constexpr auto kU64 = kUInt64;

[vboussot]

@N-Dekker I tested it because the type torch::kUInt64 does exist, but in practice it’s not fully supported. For example, operations like index_add_() fail with the following error:

index_add_(): Expected dtype int32/int64 for index but got: UInt64

It seems that torch::from_blob(..., torch::kUInt64) works for reading the data, but a cast to torch::kLong is still required to ensure compatibility with indexing operations. I think I’ll go with that approach, since I’d rather keep uint64_t in elastix for types like indices, where unsigned values make more sense.

[N-Dekker]

Casting a uint64 pointer to an int64 pointer, and then using its values "as if they are signed int64" technically sounds like undefined behavior. Right? On the other hand, it should be safe to just copy those Jacobian indices (uint64) into a signed int64 buffer, and pass that to Torch. Is it really a large performance issue?

If it's important enough, maybe we can make a pull request for torch to improve their kUInt64 support 😃

[vboussot]

I found this pytorch/pytorch#58734 it's a fairly recent addition in LibTorch

[vboussot]

I'm not sure when kUInt64 was added, I think it was with PyTorch 2.3 (April 24, 2024). For now, it's reasonable to use torch::from_blob(..., torch::kLong) with uint64_t in this context. Thank you for the change