Description
Description
Part of the ongoing effort to reduce RAPIDS reliance on downloads.rapids.ai (https://github.com/rapidsai/build-infra/issues/237 - private issue, sorry).
As of branch-25.06, all RAPIDS repos are double-writing conda packages and wheels .... once to GitHub Actions artifact store, once to the S3 bucket serving downloads.rapids.ai.
This documents the work of turning off those uploads to S3 in the 25.08 release.
Benefits of this work
- reduces unnecessary network calls and processing in CI runs
- reduces a bit of AWS cost (no more paying to store these things in S3)
- will help uncover any other dependencies on
downloads.rapids.aithat were missed in the migration
Acceptance Criteria
- RAPIDS projects are not uploading 25.08 conda packages or wheels to
downloads.rapids.ai - no CI processes in RAPIDS are broken as a result of missing artifacts on S3
Approach
First, remove any dependencies on packages being in S3 in shared-workflows workflows:
- use GitHub Actions artifacts for conda and wheel uploads to
anaconda.org(Use GitHub Actions artifacts in Anaconda uploads shared-workflows#364)- test that branch in one RAPIDS repo that no other RAPIDS libraries depend on (stop uploading packages to downloads.rapids.ai cugraph-gnn#215)
Next, remove any uses of rapids-{download,upload}-{conda,wheels}-{from,to}-s3 across RAPIDS ('rapidsai' GitHub search | 'NVIDIA' GitHub search | 'nv-legate' GitHub search | 'nv-morpheus' GitHub search):
-
rapidsai/cucim(stop uploading packages to downloads.rapids.ai cucim#883) -
rapidsai/cudf(stop uploading packages to downloads.rapids.ai cudf#18973) -
rapidsai/cugraph(stop uploading packages to downloads.rapids.ai cugraph#5096) -
rapidsai/cugraph-gnn- initial changes for testing (stop uploading packages to downloads.rapids.ai cugraph-gnn#215)
- revert
shared-workflowsbranch (use branch-25.08 of shared-workflows cugraph-gnn#216)
-
rapidsai/cuml -
rapidsai/cumlprims_mg(https://github.com/rapidsai/cumlprims_mg/pull/255) -
rapidsai/cuopt(stop uploading to downloads.rapids.ai, use rapids-init-pip in wheel CI NVIDIA/cuopt#50) -
rapidsai/cuvs(stop uploading packages to downloads.rapids.ai cuvs#940) -
rapidsai/cuxfilter(stop uploading packages to downloads.rapids.ai cuxfilter#692) -
rapidsai/dask-cuda(stop uploading packages to downloads.rapids.ai dask-cuda#1501) -
rapidsai/docs(document how to use CI artifacts with GitHub Actions artifact store docs#601) -
rapidsai/integration(stop uploading to downloads.rapids.ai, add shellcheck integration#770) -
rapidsai/jupyterlab-nvdashboard(stop uploading packages to downloads.rapids.ai jupyterlab-nvdashboard#236) -
rapidsai/kvikio(stop uploading packages to downloads.rapids.ai kvikio#734) -
rapidsai/nx-cugraph(stop uploading packages to downloads.rapids.ai nx-cugraph#145) -
rapidsai/pynvjitlink(stop uploading packages to downloads.rapids.ai, update to 25.08 workflows pynvjitlink#144) -
rapidsai/raft(stop uploading packages to downloads.rapids.ai raft#2688) -
rapidsai/rapids-build-backend(stop uploading to downloads.rapids.ai, add shellcheck rapids-build-backend#65) -
rapidsai/rapids-cli(stop uploading packages to downloads.rapids.ai, update to 25.08 workflows rapids-cli#98) -
rapidsai/rapids-dask-dependency(stop uploading packages to downloads.rapids.ai rapids-dask-dependency#112) -
rapidsai/rapids-logger(stop uploading packages to downloads.rapids.ai rapids-logger#41) -
rapidsai/rapids-metadata(stop uploading to downloads.rapids.ai, add shellcheck rapids-metadata#54) -
rapidsai/rapids-microbenchmarks(documented in https://github.com/rapidsai/rapids-microbenchmarks/issues/88, folks maintaining that will handle it in the way they think is best) -
rapidsai/rapidsmpf -
rapidsai/rmm(stop uploading packages to downloads.rapids.ai rmm#1929) -
rapidsai/ucx-py(stop uploading packages to downloads.rapids.ai ucx-py#1141) -
rapidsai/ucxx(stop uploading packages to downloads.rapids.ai ucxx#428)
Notes
Repo "folders" will fully disappear from downloads.rapids.ai around 30 days after these changes are applied
The objects in the downloads.rapidsa.ai bucket are automatically deleted after 30 days.
Around 30 days after we fully stop publishing objects for, say, cudf, there will be 0 remaining ci/cudf/* objects left there. Just noting that, so reviewers understand that we might not catch missed dependencies immediately.