1398 pin python version #1405

JoshuaSBrown · 2025-04-30T20:55:13Z

PR Description

Pin the python version to avoid problems with compatibility.

Tasks

- A description of the PR has been provided, and a diagram included if it is a new feature.
- Formatter has been run
- CHANGELOG comment has been added
- Labels have been assigned to the pr
- A reviwer has been added
- A user has been assigned to work on the pr
- If new feature a unit test has been added

Summary by Sourcery

Pin the Python version across installation and dependency scripts to improve compatibility and consistency

Enhancements:

Replace hardcoded 'python3' references with a configurable Python version variable

Chores:

Modify installation scripts to use a dynamic Python version
Update package installation commands to use the versioned Python interpreter

…nterpreter with version number attached

sourcery-ai · 2025-04-30T20:55:17Z

Reviewer's Guide

This pull request pins the Python version used across various shell scripts by replacing hardcoded 'python3' commands and package names with the variable '"python${DATAFED_PYTHON_VERSION}"' to ensure consistent dependency installation and script execution.

File-Level Changes

Change	Details	Files
Replaced hardcoded 'python3' executable calls and package names with the 'DATAFED_PYTHON_VERSION' variable.	Updated Python executable calls (e.g., 'python3 -m pip' to '"python${DATAFED_PYTHON_VERSION}" -m pip'). Modified apt package lists to install specific Python versions (e.g., 'python3-venv' to '"python${DATAFED_PYTHON_VERSION}"-venv').	`scripts/dependency_install_functions.sh` `scripts/install_authz_dependencies.sh` `scripts/install_client_dependencies.sh` `scripts/install_core_dependencies.sh` `scripts/install_docs_dependencies.sh` `scripts/install_end_to_end_test_dependencies.sh` `scripts/install_python_client_dependencies.sh` `scripts/install_repo_dependencies.sh` `repository/docker/entrypoint_authz.sh` `scripts/compose_cleanup_globus_files.sh` `scripts/compose_generate_globus_files.sh` `scripts/install_dependencies.sh` `scripts/install_ws_dependencies.sh`
Ensured pip is installed and upgraded within the virtual environment initialization.	Added '"python${DATAFED_PYTHON_VERSION}" -m ensurepip --upgrade' to the 'init_python' function.	`scripts/dependency_install_functions.sh`
Sourced dependency versions in Globus scripts.	Added 'source "${BUILD_DIR}/dependency_versions.sh"' or 'source "${SOURCE}/dependency_versions.sh"' to ensure the 'DATAFED_PYTHON_VERSION' variable is available.	`repository/docker/entrypoint_authz.sh` `scripts/compose_cleanup_globus_files.sh` `scripts/compose_generate_globus_files.sh`
Corrected a typo in the web Dockerfile.	Fixed 'ake' to 'make' in the 'apt install' command.	`web/docker/Dockerfile`

Possibly linked issues

#0: The PR pins the python version by replacing 'python3' with '${DATAFED_PYTHON_VERSION}' in install scripts, fixing compatibility issues.
#-1: PR pins Python version to fix CI build failure from Python 3.12 incompatibility.

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey @JoshuaSBrown - I've reviewed your changes and found some issues that need to be addressed.

Blocking issues:

Possible typo in Dockerfile package list. (link)
Consider clarifying why python3 is removed from the web/docker/Dockerfile without explicitly adding python${DATAFED_PYTHON_VERSION} as done in other dependency scripts.
The init_python function now upgrades pip using ensurepip --upgrade; the subsequent pip install --upgrade pip calls in various install scripts appear redundant.

Here's what I looked at during the review

🔴 General issues: 1 blocking issue, 1 other issue
🟢 Security: all looks good
🟢 Testing: all looks good
🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

sourcery-ai · 2025-04-30T20:56:39Z

scripts/dependency_install_functions.sh

-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH" PATH="$PATH" python3 -m pip install numpy
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH" PATH="$PATH" python3 setup.py build
-    LD_LIBRARY_PATH="$LD_LIBRARY_PATH" PATH="$PATH" python3 setup.py test
+    LD_LIBRARY_PATH="$LD_LIBRARY_PATH" PATH="$PATH" python${DATAFED_PYTHON_VERSION} -m pip install numpy


suggestion: Consider consistent quoting of the python command.

Some invocations are missing quotes around python${DATAFED_PYTHON_VERSION}. Quote them all to avoid potential word splitting issues.

Suggested implementation:

LD_LIBRARY_PATH="$LD_LIBRARY_PATH" PATH="$PATH" "python${DATAFED_PYTHON_VERSION}" -m pip install numpy

LD_LIBRARY_PATH="$LD_LIBRARY_PATH" PATH="$PATH" "python${DATAFED_PYTHON_VERSION}" setup.py build

LD_LIBRARY_PATH="$LD_LIBRARY_PATH" PATH="$PATH" "python${DATAFED_PYTHON_VERSION}" setup.py test

web/docker/Dockerfile

* updated requirements * updated requirements * updated protobuf submodule * update unit test ci version * updated protobuf version * updated setuptools version * fixed condition * changed where packages are installed * used python3.11 to create the virtual environment instead of latest * reverted protobuf version * testing specificying version * removed pyvenv in ci job for consistency * installed python3.11 in dependencies docker file * added missing -y flag * removed unification of install scripts * added missing python3.11 install * updated ci job to install python3.11 * changed installed python version to 3.9 since protobuf does not officially support 3.11 * changed installed python version to 3.9 since protobuf does not officially support 3.11 * missed version * updated includes for newer gcc versions * updated versions * updated nlohmann json * updated json schema version * changed library search path * hopefully fixed json dep issues * updating function signature to match newer version of base class * updated protobuf * updated protobuf and cmake * downgraded cmake * fixed protobuf version * fixed protobuf version * 1398 pin python version (#1405) * Pin python version * fixed incorrect path * Switch to using .tar file install of libsodium (#1414) * 1413 libsodium build refactor (#1415) * Switch to using .tar file install of libsodium * Switch bad option in wget command from -C to -P * Update scripts/dependency_install_functions.sh * Update scripts/dependency_install_functions.sh * Update dependency_install_functions.sh Libsodium folder version number. * Update dependency_install_functions.sh Make paths explicit. * cleaned up comments * Update scripts/dependency_install_functions.sh Co-authored-by: Joshua S Brown <[email protected]> * Update cmake/JSONSchema.cmake Co-authored-by: Joshua S Brown <[email protected]> * Update scripts/dependency_install_functions.sh Co-authored-by: Joshua S Brown <[email protected]> * Update cmake/JSONSchema.cmake Co-authored-by: Joshua S Brown <[email protected]> * add changes from review * Changed install scripts to be consistent with new python installation method (#1417) * changed install scripts to be consistent * added sudo check * add apt sources check --------- Co-authored-by: Blake Nedved <[email protected]> Co-authored-by: nedvedba <[email protected]>

JoshuaSBrown added 2 commits April 29, 2025 15:59

First commit to pinning python version by explicitly calling python i…

3f3eb02

…nterpreter with version number attached

Pin python version

c5e79e7

JoshuaSBrown requested a review from nedvedba April 30, 2025 20:55

JoshuaSBrown linked an issue Apr 30, 2025 that may be closed by this pull request

[CI, Build, scripts] - Pin Python Version #1398

Open

sourcery-ai bot reviewed Apr 30, 2025

View reviewed changes

AronPerez requested changes May 1, 2025

View reviewed changes

web/docker/Dockerfile Outdated Show resolved Hide resolved

fixed mispelled package name

d9772f4

nedvedba merged commit a758cdb into fix-ci-build May 5, 2025
2 of 4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

1398 pin python version #1405

1398 pin python version #1405

JoshuaSBrown commented Apr 30, 2025 •

edited by sourcery-ai bot

Loading

sourcery-ai bot commented Apr 30, 2025 •

edited

Loading

Interacting with Sourcery

Customizing Your Experience

Getting Help

sourcery-ai bot left a comment

sourcery-ai bot Apr 30, 2025

1398 pin python version #1405

1398 pin python version #1405

Conversation

JoshuaSBrown commented Apr 30, 2025 • edited by sourcery-ai bot Loading

PR Description

Tasks

Summary by Sourcery

sourcery-ai bot commented Apr 30, 2025 • edited Loading

Reviewer's Guide

File-Level Changes

Possibly linked issues

Interacting with Sourcery

Customizing Your Experience

Getting Help

sourcery-ai bot left a comment

Choose a reason for hiding this comment

sourcery-ai bot Apr 30, 2025

Choose a reason for hiding this comment

JoshuaSBrown commented Apr 30, 2025 •

edited by sourcery-ai bot

Loading

sourcery-ai bot commented Apr 30, 2025 •

edited

Loading