Skip to content

--sh-boot --venv=... --include-tools pex doesn't respect PEX_TOOLS=1 if venv exists #2725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
huonw opened this issue Mar 27, 2025 · 1 comment · Fixed by #2726
Closed

--sh-boot --venv=... --include-tools pex doesn't respect PEX_TOOLS=1 if venv exists #2725

huonw opened this issue Mar 27, 2025 · 1 comment · Fixed by #2726
Labels

Comments

@huonw
Copy link
Collaborator

huonw commented Mar 27, 2025

If a PEX is built with --sh-boot --venv=... --include-tools, then running PEX_TOOLS=1 ./path/to/pex only works if the venv hasn't already been created in $PEX_ROOT (that is, the cache is cold).

If the venv already exists, then the PEX_TOOLS=1 ./path/to/pex ... invocation is equivalent to ./path/to/pex ..., i.e. just invoking the pex normally without PEX_TOOLS=1.

I'd expect the state of the cache to not change the behaviour of these invocations (only the performance).

Reproducer:

#!/usr/bin/env bash

cd $(mktemp -d)

python3.12 -m venv venv
. venv/bin/activate
pip install pex==2.33.6

pex --include-tools --sh-boot --venv=prepend -o example.pex

export PEX_ROOT=./pex_root

echo "OKAY: PEX_ROOT empty: request for tools works"
PEX_TOOLS=1 ./example.pex -h
# usage: PEX_TOOLS=1 ./example.pex [-h] [-V] ...
# ...

# Seed the venv
./example.pex -c ''

echo "BUG: PEX_ROOT seeded: request for tools ignored (Python help instead)"
PEX_TOOLS=1 ./example.pex -h
# usage: .../python [option] ... [-c cmd | -m mod | file | -] [arg] ...
# ...

The bug stops reproducing after removing any of the options.

@jsirois
Copy link
Member

jsirois commented Mar 27, 2025

Facepalm. Thanks for reporting and fixing @huonw!

jsirois pushed a commit that referenced this issue Mar 28, 2025
This patch stops the "execute venv entrypoint directly" fast-path from
happening when the user has set `PEX_TOOLS=1` with a `--sh-boot` PEX,
mimicking the similar check performed for Python bootstrapping:


https://github.com/pex-tool/pex/blob/17bd416647eaca978e538408f1ec21813ab26335/pex/pex_boot.py#L204-L205

This change ensures that `PEX_TOOLS=1 ./some.pex` always works, no
matter the state of the `PEX_ROOT` cache.

Fixes #2725
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants