Question about project aims given Pex and shiv .pyz generators pre-existing.

[jsirois]

I see this project was started 4 years ago. Pex has been public for at least 11 years and shiv for 7 years. Shiv explicitly aimed to be simpler (it is) and faster (it isn't any longer) than Pex when it launched 4 years after Pex. Was this project aware of those projects when it launched? What distinguishing features are being aimed for? I see the license is GPL3 - maybe just that? Afaict - besides a yaml config file vs command line args, there are no distinguishing features, but as maintainer of Pex, I'm very curious!

[facundobatista]

See the How PyEmpaq Relates to Other Similar Tools? section in the docs, thanks!

Closing the issue as the question is already addressed, but feel free to re-open if you have follow-up questions, thanks!

[jsirois]

Aha - I somehow missed that. Thank you. Pex is multi-platform but it is ahead of time as you say, and so that is a difference with this project. In addition, as you say, Pex currently doesn't support Windows, just Mac and Linux, but Windows is under active development, c.f.: pex-tool/pex#2658

So, to your example, try this:

:; cat << EOF > exe.py                                                                                                         
import charset_normalizer
import cowsay
import requests


print(requests.get("https://echo.free.beeceptor.com").text)
print(charset_normalizer.__file__)
print(cowsay.__file__)
print(requests.__file__)
EOF

# Build a multi-platform PEX: 
:; pex --python python3.9 --python python3.10 requests cowsay -o cowsay.pex --exe exe.py --python-shebang '/usr/bin/env python'

# Run it on a python3.9-only system:
:; docker run --rm -it -v $PWD/cowsay.pex:/cowsay.pex python:3.9 /cowsay.pex                                                   
{
  "method": "GET",
  "protocol": "https",
  "host": "echo.free.beeceptor.com",
  "path": "/",
  "ip": "174.62.125.224:48590",
  "headers": {
    "Host": "echo.free.beeceptor.com",
    "User-Agent": "python-requests/2.32.3",
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate"
  },
  "parsedQueryParams": {}
}
/root/.cache/pex/installed_wheels/0/47e413dc5e5179575551bd24f3c979d5e4030cc5f9dcccdb16b36c408665dbde/charset_normalizer-3.4.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl/charset_normalizer/__init__.py
/root/.cache/pex/installed_wheels/0/fa6f4a84c8762cbb487fd86b7808a641f9909bb3d30eecdc8d63687eef5d1305/cowsay-6.1-py3-none-any.whl/cowsay/__init__.py
/root/.cache/pex/installed_wheels/0/2f57f4ae5920a2f5f55a8f4971fe010b00cb2a30f8ab9c572826150fa309c450/requests-2.32.3-py3-none-any.whl/requests/__init__.py

# Run it on a python3.10-only system: 
:; docker run --rm -it -v $PWD/cowsay.pex:/cowsay.pex python:3.10 /cowsay.pex                                                  
{
  "method": "GET",
  "protocol": "https",
  "host": "echo.free.beeceptor.com",
  "path": "/",
  "ip": "174.62.125.224:36976",
  "headers": {
    "Host": "echo.free.beeceptor.com",
    "User-Agent": "python-requests/2.32.3",
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate"
  },
  "parsedQueryParams": {}
}
/root/.cache/pex/installed_wheels/0/ae654712feab7723a4183556af329995b46cef73ff273ef81f46d5f4bb22a673/charset_normalizer-3.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl/charset_normalizer/__init__.py
/root/.cache/pex/installed_wheels/0/fa6f4a84c8762cbb487fd86b7808a641f9909bb3d30eecdc8d63687eef5d1305/cowsay-6.1-py3-none-any.whl/cowsay/__init__.py
/root/.cache/pex/installed_wheels/0/2f57f4ae5920a2f5f55a8f4971fe010b00cb2a30f8ab9c572826150fa309c450/requests-2.32.3-py3-none-any.whl/requests/__init__.py

The multi-platform can also include wheels for different OSes too (say a Mac/Linux multiplatform PEX - which is popular).

[facundobatista]

IIUC you're building it specifically with python 3.10 and 3.9, right? what happens if you only have python 3.11?

Furthermore, you need both pythons in your environment, right?

And the wheel used is specifically for linux (charset_normalizer-3.4.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl), you say that a .pex will work both in Linux and Windows?

Anyway, I'm not saying that Pex does not work. It is another model. It has its cons and its pros. I know, I worked a lot with a system that tries to put everything inside one file that will run everywhere. PyEmpaq just does it differently.

[jsirois]

Pex does not support Windows yet, just Mac and Linux. You do not need to have the target interpreters available to build for a foreign platform if you generate a "complete platform" description ahead of time (see: pex-tool/pex#2678) and pre-built wheels are available for all targeted platforms.

Yes, these are very different approaches. I just wanted to point out you can in fact target multiple platforms with PEX, even foreign platforms if you have access to pre-built wheels for those platforms.