Add missing thrust headers & Pass dtype objects to cuDF as_column

[alliepiper]

Note by @jakirkham this includes C++ and Python fixes needed for CI. Details below:

---

From @alliepiper :

Discovered in CCCL CI.

/usr/bin/sccache /home/coder/.conda/envs/rapids/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/home/coder/.conda/envs/rapids/bin/x86_64-conda-linux-gnu-c++ -DCUDA_API_PER_THREAD_DEFAULT_STREAM -DCUTLASS_NAMESPACE=raft_cutlass -DLIBCUDACXX_ENABLE_EXPERIMENTAL_MEMORY_RESOURCE -DRAFT_COMPILED -DRAFT_LOG_ACTIVE_LEVEL=RAPIDS_LOGGER_LOG_LEVEL_INFO -DRAFT_SYSTEM_LITTLE_ENDIAN=1 -DTHRUST_DEVICE_SYSTEM=THRUST_DEVICE_SYSTEM_CUDA -DTHRUST_DISABLE_ABI_NAMESPACE -DTHRUST_HOST_SYSTEM=THRUST_HOST_SYSTEM_CPP -DTHRUST_IGNORE_ABI_NAMESPACE_ERROR -Dcugraph_EXPORTS -I/home/coder/cugraph/cpp/../thirdparty -I/home/coder/cugraph/cpp/src -I/home/coder/cugraph/cpp/include -I/home/coder/cugraph/cpp/build/conda/cuda-12.8/release/_deps/cccl-src/lib/cmake/thrust/../../../thrust -I/home/coder/cugraph/cpp/build/conda/cuda-12.8/release/_deps/cccl-src/lib/cmake/libcudacxx/../../../libcudacxx/include -I/home/coder/cugraph/cpp/build/conda/cuda-12.8/release/_deps/cccl-src/lib/cmake/cub/../../../cub -I/home/coder/cugraph/cpp/build/conda/cuda-12.8/release/_deps/cuco-src/include -isystem /home/coder/rmm/include -isystem /home/coder/rmm/build/conda/cuda-12.8/release/include -isystem /home/coder/.conda/envs/rapids/targets/x86_64-linux/include -isystem /home/coder/.conda/envs/rapids/include -isystem /home/coder/raft/cpp/include -isystem /home/coder/raft/cpp/build/conda/cuda-12.8/release/include -isystem /home/coder/raft/cpp/build/conda/cuda-12.8/release/_deps/nvidiacutlass-src/include -isystem /home/coder/raft/cpp/build/conda/cuda-12.8/release/_deps/nvidiacutlass-build/include -t=1 -O3 -DNDEBUG -std=c++17 "--generate-code=arch=compute_70,code=[sm_70]" -Xcompiler=-fPIC --expt-extended-lambda --expt-relaxed-constexpr -Werror=cross-execution-space-call -Wno-deprecated-declarations -DRAFT_HIDE_DEPRECATION_WARNINGS -Xptxas=--disable-warnings -Xcompiler=-Wall,-Wno-error=sign-compare,-Wno-error=unused-but-set-variable -Xfatbin=-compress-all -MD -MT CMakeFiles/cugraph.dir/src/detail/utility_wrappers_64.cu.o -MF CMakeFiles/cugraph.dir/src/detail/utility_wrappers_64.cu.o.d -x cu -c /home/coder/cugraph/cpp/src/detail/utility_wrappers_64.cu -o CMakeFiles/cugraph.dir/src/detail/utility_wrappers_64.cu.o
nvcc warning : Support for offline compilation for architectures prior to '<compute/sm/lto>_75' will be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
/home/coder/cugraph/cpp/src/detail/utility_wrappers_impl.cuh(191): error: namespace "thrust" has no member "equal"
    return thrust::equal(handle.get_thrust_policy(), span1.begin(), span1.end(), span2.begin());

Cugraph CI was failing on this PR with several other missing thrust headers, adding those as I discover them.

---

From @mroeschke :

With rapidsai/cudf#18331, cuDF now requires values passed to dtype in as_column to be supported data type objects recognized by cuDF (cuDF's custom types or np.dtype).

I only found this usage which was passing np.int32 instead of np.dtype(np.int32) which should fix the failures on the Python build

[seunghwak]

LGTM and thanks for the fix!

[alliepiper]

The python errors look unrelated:

AttributeError: type object 'numpy.int32' has no attribute 'kind'

Are these known issues?

[bdice]

@mroeschke Any thoughts on the kind issue?

[jakirkham]

Sounds like the sort of thing where something gets a NumPy type, but expects a dtype

In [1]: import numpy as np

In [2]: np.int32.kind
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 np.int32.kind

AttributeError: type object 'numpy.int32' has no attribute 'kind'

In [3]: np.dtype(np.int32).kind
Out[3]: 'i'

Pretty simple to push whatever type (or dtype) through the constructor for np.dtype

[mroeschke]

Yup, looks like there is a call to cuDF's as_column that is passing np.int32 instead of np.dtype(np.int32) which was a recent change in cuDF Python

[jakirkham]

For context this is the traceback from CI

        File "/opt/conda/envs/test/lib/python3.10/site-packages/cugraph/structure/hypergraph.py", line 633, in _str_scalar_to_category
          children=(cudf.core.column.as_column(0, length=size, dtype=np.int32),),
        File "/opt/conda/envs/test/lib/python3.10/site-packages/cudf/core/column/column.py", line 2843, in as_column
          col = col.astype(dtype)
        File "/opt/conda/envs/test/lib/python3.10/site-packages/cudf/core/column/column.py", line 1635, in astype
          elif dtype.kind == "M":
      AttributeError: type object 'numpy.int32' has no attribute 'kind'

Would it make sense to normalize type's to np.dtype during construction?

Not sure if we will also want to do this elsewhere

[mroeschke]

Would it make sense to normalize type's to np.dtype during construction?

I have been working towards the cudf Python Column level to only accept/deal with dtype objects recently (makes things more straightforward) since it's supposed to be "private".

Type normalization usually is meant to happen in public cuDF public constructors e.g. cudf.Series/cudf.Dataframe

[jakirkham]

Hmm...in that case could we add some validation of input types upstream?

As this discussion shows, the error message is not clear to everyone. Not to mention what should be the developer's next steps

Edit: Or do you mean cuGraph is using private APIs and should move away from those?

[mroeschke]

Hmm...in that case could we add some validation of input types upstream?

Yes definitely - I think this is good to add.

Or do you mean cuGraph is using private APIs and should move away from those?

This is also true. The cuDF Python "column" APIs technically private, but we've seen other RAPIDS projects sometime need to use them for one reason or another. I think some of them might be public eventually since there may not be workarounds for functionality if users start from public APIs

Nonetheless, I can open a cugraph PR to fix these Python tests.

[jakirkham]

Thanks Matthew! 🙏

If there is a good way to use public APIs here, would be interested in your thoughts on how best to do so 🙂

Can see you opened PR: #5025

Given that one will fail due to the CCCL changes that this PR fixes, would it be alright if we merged it into this PR?

[alliepiper]

@jakirkham I've pulled those changes in here. Thanks!

[jakirkham]

Thanks Allison! 🙏

[jakirkham]

Updated the title and description to pull in Matthew's PR info. Hope that is ok

[rlratzel]

/merge

[rlratzel]

Thanks @alliepiper for this!

[jakirkham]

Indeed thanks Allison and Matthew for your fixes! Also thanks to the broader group for helping swarm, review, and reach out to folks for support 🙏

Glad we are getting CI back to green 🟢