Skip to content

Bump min torch to 1.13.1 to mitigate CVE-2022-45907 unsafe usage of eval #8296

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 14, 2025

Conversation

jamesobutler
Copy link
Contributor

@jamesobutler jamesobutler commented Jan 12, 2025

Description

This bumps the minimum required torch version from 1.9.0 to 1.13.1.

See GHSA-47fc-vmwq-366v for more details such as the highest severity scoring of "Critical".

Maintainers will need to update the required status checks for the dev branch to:

  • Remove min-dep-pytorch (1.10.2)
  • Remove min-dep-pytorch (1.11.0)
  • Remove min-dep-pytorch (1.12.1)
  • Remove min-dep-pytorch (1.13)
  • Add min-dep-pytorch (1.13.1)

cc: @KumoLiu

Types of changes

  • Breaking change (fix or new feature that would cause existing functionality to change). (drop of older torch versions)
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.

@jamesobutler jamesobutler force-pushed the bump-torch-minimum branch 3 times, most recently from f406fed to cfac884 Compare January 12, 2025 23:40
@jamesobutler jamesobutler changed the title Bump torch to 1.13.1 to mitigate CVE-2022-45907 unsafe usage of eval Bump min torch to 1.13.1 to mitigate CVE-2022-45907 unsafe usage of eval Jan 12, 2025
@KumoLiu KumoLiu requested review from ericspod and Nic-Ma January 13, 2025 02:56
@KumoLiu
Copy link
Contributor

KumoLiu commented Jan 13, 2025

Thanks @jamesobutler for the contribution!

The PR overall looks good to me. @ericspod and @Nic-Ma, do you have any concern on this pr. For vulnerability reasons, we should indeed drop the pre-1.13.1 version of pytorch, do you have any concerns, because the PR will introduce some compatibility issues and remove some support for earlier versions. If you guys have no problems, I'll help further refine this PR as well as the updated Blossom CI (GPU testing happens over there).
Let me know if you have any concern here! Thanks.

@ericspod
Copy link
Member

Thanks @jamesobutler for the contribution!

The PR overall looks good to me. @ericspod and @Nic-Ma, do you have any concern on this pr. For vulnerability reasons, we should indeed drop the pre-1.13.1 version of pytorch, do you have any concerns, because the PR will introduce some compatibility issues and remove some support for earlier versions. If you guys have no problems, I'll help further refine this PR as well as the updated Blossom CI (GPU testing happens over there). Let me know if you have any concern here! Thanks.

Thanks as well @jamesobutler. I think we should discuss internally what this implies because it's dropping support for a number of Pytorch versions. We should be refining what our policy is about when to drop versions, we do this for Python in that we drop non-supported versions but there's no sunset period defined by Pytorch for theirs (as far as I know). Honestly we could probably drop all of Pytorch 1.* and very few users would be impacted. Let's keep working on this either way.

@jamesobutler
Copy link
Contributor Author

jamesobutler commented Jan 13, 2025

but there's no sunset period defined by Pytorch for theirs (as far as I know).

@ericspod Reviewing the PyTorch release history, the last time maintainers released a patch release out-of-order was when they released version 1.8.2 on August 17th 2021 which was after version 1.9.0 which was released June 15th 2021. This was part of the "PyTorch Enterprise Support Program". However that program only lasted about a 1.5 years as on November 10th 2022 they ended it. See this blog post https://pytorch.org/blog/pytorch-enterprise-support-update/. So therefore PyTorch versions become unmaintained whenever a new major/minor release version comes out. They do not have LTS support or issue patch releases for multiple minor versions.

Honestly we could probably drop all of Pytorch 1.* and very few users would be impacted. Let's keep working on this either way.

I issued this PR to bump the torch version to mitigate "Critical" scored vulnerability issues and see how the monai team would respond. If this was going to be integrated, I was planning to issue a follow-up PR with the proposal to bump the minimum torch version to version 2.2.0 to mitigate "High" scored vulnerabilities (CVE-2024-31580, CVE-2024-31583). It would be my recommendation that monai not use torch versions with vulnerabilities listed as High or Critical. Once the minimum torch version is set to 2.2.0 there would be no more High or Critical published vulnerabilities (or any vulnerabilities listed at any scoring level for that matter) affecting the torch versions used by monai. See Snyk's reporting of each torch version:

https://security.snyk.io/package/pip/torch
{5902F2D5-4381-4F24-A297-55893217D354}

Other unrelated considerations to keep in mind if bumping minimum torch version beyond 2.2.0:
torch 2.3.0 drops binary publishing for macOS x64.
torch 2.3.0 is the first version that is compatible with both numpy 1 and numpy 2 (provides the most flexibility with other monai dependencies)

@KumoLiu
Copy link
Contributor

KumoLiu commented Jan 24, 2025

Hi @aylward, during our development meeting, we discussed the PR to bump the minimum Torch version to 1.13.1 in order to mitigate the critical vulnerabilities identified in PyTorch. We are all in agreement regarding this change, but we would still like to hear your thoughts on the matter.

Thank you!

@jamesobutler
Copy link
Contributor Author

@KumoLiu Any updates on this PR to help proceed with the integration?

@aylward
Copy link
Collaborator

aylward commented Feb 1, 2025

Hi @aylward, during our development meeting, we discussed the PR to bump the minimum Torch version to 1.13.1 in order to mitigate the critical vulnerabilities identified in PyTorch. We are all in agreement regarding this change, but we would still like to hear your thoughts on the matter.

Thank you!

Definitely ok to bump! That version was released in Dec, 2022, and fixed vulnerabilities as you noted.

@KumoLiu
Copy link
Contributor

KumoLiu commented Feb 3, 2025

/build

@jamesobutler
Copy link
Contributor Author

@KumoLiu Why is blossom-ci showing as failed? Is it because certain steps were skipped in https://github.com/Project-MONAI/MONAI/actions/runs/13111088891? Was the authorization step skipped because this branch is in my fork?

@KumoLiu
Copy link
Contributor

KumoLiu commented Feb 3, 2025

@KumoLiu Why is blossom-ci showing as failed? Is it because certain steps were skipped in https://github.com/Project-MONAI/MONAI/actions/runs/13111088891? Was the authorization step skipped because this branch is in my fork?

Hi @jamesobutler, no, it's not related. The issue is due to the mismatched Cuda version.
After #8231 merged, let me trigger the test again.

[2025-02-03T10:09:54.632Z] writing manifest file 'monai.egg-info/SOURCES.txt'
[2025-02-03T10:09:54.632Z] running build_ext
[2025-02-03T10:09:54.632Z] Traceback (most recent call last):
[2025-02-03T10:09:54.632Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/setup.py", line 144, in <module>
[2025-02-03T10:09:54.632Z]     setup(
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/__init__.py", line 87, in setup
[2025-02-03T10:09:54.632Z]     return distutils.core.setup(**attrs)
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/core.py", line 185, in setup
[2025-02-03T10:09:54.632Z]     return run_commands(dist)
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/core.py", line 201, in run_commands
[2025-02-03T10:09:54.632Z]     dist.run_commands()
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/dist.py", line 969, in run_commands
[2025-02-03T10:09:54.632Z]     self.run_command(cmd)
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/dist.py", line 1208, in run_command
[2025-02-03T10:09:54.632Z]     super().run_command(command)
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
[2025-02-03T10:09:54.632Z]     cmd_obj.run()
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/command/develop.py", line 34, in run
[2025-02-03T10:09:54.632Z]     self.install_for_development()
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/command/develop.py", line 114, in install_for_development
[2025-02-03T10:09:54.632Z]     self.run_command('build_ext')
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/cmd.py", line 318, in run_command
[2025-02-03T10:09:54.632Z]     self.distribution.run_command(command)
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/dist.py", line 1208, in run_command
[2025-02-03T10:09:54.632Z]     super().run_command(command)
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/dist.py", line 988, in run_command
[2025-02-03T10:09:54.632Z]     cmd_obj.run()
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/command/build_ext.py", line 84, in run
[2025-02-03T10:09:54.632Z]     _build_ext.run(self)
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/setuptools/_distutils/command/build_ext.py", line 346, in run
[2025-02-03T10:09:54.632Z]     self.build_extensions()
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/torch/utils/cpp_extension.py", line 520, in build_extensions
[2025-02-03T10:09:54.632Z]     _check_cuda_version(compiler_name, compiler_version)
[2025-02-03T10:09:54.632Z]   File "/usr/local/lib/python3.9/dist-packages/torch/utils/cpp_extension.py", line 415, in _check_cuda_version
[2025-02-03T10:09:54.632Z]     raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
[2025-02-03T10:09:54.632Z] RuntimeError: 
[2025-02-03T10:09:54.632Z] The detected CUDA version (11.6) mismatches the version that was used to compile
[2025-02-03T10:09:54.632Z] PyTorch (12.4). Please make sure to use the same CUDA versions.

@ericspod
Copy link
Member

Hi @jamesobutler thanks for the contribution again, we are still looking into this and want to develop a formal policy for dependency support. In the meantime we've done a major refactor of tests which now are causing conflicts with this PR. If you could please sort the conflicts out (I can't through Github), we can try the tests again as @KumoLiu mentioned and go from there.

@jamesobutler
Copy link
Contributor Author

jamesobutler commented Feb 12, 2025

@ericspod I've rebased this branch to put on top of latest dev and handled the merge conflicts.

While fixing merge conflicts I observed that you @ericspod added an upper limit to PyTorch in the requirements.txt in 8dcb9dc, but did not do so in the install_requires that would be used by the monai whl. See below which has no upper pin vs the requirements.txt. Irrelevant to this PR, but wanted to point it out to you if you truly intend to now have an upper pin for PyTorch in monai or if that was a local change that snuck into that commit accidentally.

torch>=1.9

torch>=1.9,<2.6

@ericspod
Copy link
Member

@ericspod I've rebased this branch to put on top of latest dev and handled the merge conflicts.

While fixing merge conflicts I observed that you @ericspod added an upper limit to PyTorch in the requirements.txt in 8dcb9dc, but did not do so in the install_requires that would be used by the monai whl. See below which has no upper pin vs the requirements.txt. Irrelevant to this PR, but wanted to point it out to you if you truly intend to now have an upper pin for PyTorch in monai or if that was a local change that snuck into that commit accidentally.

torch>=1.9

torch>=1.9,<2.6

Good catch on that thanks, the limit is temporary and should go away soon once we figure out a few issues with 2.6 compatibility. If you wanted to add the constraint to this PR, we can merge this first and then move on to improving compatibility.

@jamesobutler
Copy link
Contributor Author

Since you have said it is temporary I'll hold off making any upper pin changes. Just something I saw while updating the min torch version in the same code line.

@jamesobutler
Copy link
Contributor Author

@KumoLiu Are you able to trigger the blossom-ci?

@KumoLiu
Copy link
Contributor

KumoLiu commented Feb 14, 2025

/build

Copy link
Contributor

@KumoLiu KumoLiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!

@KumoLiu KumoLiu merged commit 44add8d into Project-MONAI:dev Feb 14, 2025
28 checks passed
@jamesobutler jamesobutler deleted the bump-torch-minimum branch February 14, 2025 12:41
ericspod pushed a commit that referenced this pull request Mar 4, 2025
…able numpy 2 compatibility (#8368)

This is a follow-up to the comments made in
#8296 (comment).

### Description

This bumps the minimum required `torch` version from 1.13.1 to 2.2.0 in
the first commit.

See GHSA-5pcm-hx3q-hm94 and
GHSA-pg7h-5qx3-wjr3 for more details
regarding the "High" severity scoring.

- https://nvd.nist.gov/vuln/detail/CVE-2024-31580
- https://nvd.nist.gov/vuln/detail/CVE-2024-31583

Additionally, PyTorch added support for numpy 2 starting with PyTorch
2.3.0. The second commit in this PR allows for numpy 1 or numpy 2 to be
used with torch>=2.3.0. I have included this commit in this PR as
upgrading to torch 2.2 means you might as well update to 2.3 to get the
numpy 2 compatibility.

A special case is being handled on Windows as PyTorch Windows binaries
had compatibilities issues with numpy 2 that were fixed in torch 2.4.1
(see
pytorch/pytorch#131668 (comment)).

Maintainers will need to update the required status checks for the
[`dev`](https://github.com/Project-MONAI/MONAI/tree/dev) branch to:
- Remove min-dep-pytorch (2.0.1)

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [X] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.

---------

Signed-off-by: James Butler <[email protected]>
Can-Zhao pushed a commit to Can-Zhao/MONAI that referenced this pull request Mar 10, 2025
…val (Project-MONAI#8296)

### Description

This bumps the minimum required `torch` version from 1.9.0 to 1.13.1.

See GHSA-47fc-vmwq-366v for more details
such as the highest severity scoring of "Critical".

- https://nvd.nist.gov/vuln/detail/CVE-2022-45907
- https://security.snyk.io/vuln/SNYK-PYTHON-TORCH-3149871

Maintainers will need to update the required status checks for the
[`dev`](https://github.com/Project-MONAI/MONAI/tree/dev) branch to:
- Remove min-dep-pytorch (1.10.2)
- Remove min-dep-pytorch (1.11.0)
- Remove min-dep-pytorch (1.12.1)
- Remove min-dep-pytorch (1.13)
- Add min-dep-pytorch (1.13.1)

cc: @KumoLiu

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Breaking change (fix or new feature that would cause existing
functionality to change). (drop of older `torch` versions)
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.

---------

Signed-off-by: James Butler <[email protected]>
Signed-off-by: YunLiu <[email protected]>
Co-authored-by: YunLiu <[email protected]>
Signed-off-by: Can-Zhao <[email protected]>
Can-Zhao pushed a commit to Can-Zhao/MONAI that referenced this pull request Mar 10, 2025
…able numpy 2 compatibility (Project-MONAI#8368)

This is a follow-up to the comments made in
Project-MONAI#8296 (comment).

### Description

This bumps the minimum required `torch` version from 1.13.1 to 2.2.0 in
the first commit.

See GHSA-5pcm-hx3q-hm94 and
GHSA-pg7h-5qx3-wjr3 for more details
regarding the "High" severity scoring.

- https://nvd.nist.gov/vuln/detail/CVE-2024-31580
- https://nvd.nist.gov/vuln/detail/CVE-2024-31583

Additionally, PyTorch added support for numpy 2 starting with PyTorch
2.3.0. The second commit in this PR allows for numpy 1 or numpy 2 to be
used with torch>=2.3.0. I have included this commit in this PR as
upgrading to torch 2.2 means you might as well update to 2.3 to get the
numpy 2 compatibility.

A special case is being handled on Windows as PyTorch Windows binaries
had compatibilities issues with numpy 2 that were fixed in torch 2.4.1
(see
pytorch/pytorch#131668 (comment)).

Maintainers will need to update the required status checks for the
[`dev`](https://github.com/Project-MONAI/MONAI/tree/dev) branch to:
- Remove min-dep-pytorch (2.0.1)

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [X] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.

---------

Signed-off-by: James Butler <[email protected]>
Signed-off-by: Can-Zhao <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants