Using conda-convert to convert from/to noarch to/from specific platform

[jakirkham]

Would be nice to be able to convert noarch packages to a specific platform (e.g. linux-64) and vice versa using conda-convert. Of particular interest is noarch: python packages. Though ideally this could be used for all noarch packages.

[mandeep]

I think this would be an easy change. Just need to change the index.json entries from noarch to the target arch.

[mandeep]

Although I'm not sure how we would want to deal with the files in site-packages.

[jakirkham]

What about .pyc files?

[mandeep]

I don't think there's anything we can do about them. Just let them be created at import time.

[msarahan]

.pyc files are only specific to a given python version - not platform. It should be possible to generate them at conversion time, but we'd require a particular python version to be given (or use the default, which is the python version present in the root env.)

[jakirkham]

That seems like a totally reasonable constraint to me.

Do we need to do anything special for entry points?

[jakirkham]

Maybe you'd be interested in this discussion, @ericdill. ;)

[ericdill]

Thanks for the ping @jakirkham . I recently switched teams at work and so have not been paying enough attention to conda-forge, conda, conda-build and constructor as I should have been. I'll likely just be lurking with little input 😁.

FWIW I really like the idea of noarch: python but I have not been paying enough attention to the future of constructor and noarch. I've got a hard dependency on constructor at my day job so depending on where noarch: python goes I may see if I can carve out some time at work to submit PR(s) to constructor so that it can pull in these new noarch packages

[jakirkham]

No worries. These things happen.

Yeah conda-forge will soon have the same problem. We had been discussing having an installer for some time and there has been work on a constructor-based solution. Pushed on this some myself recently. Once we can get something out that is used in conda-forge, expect there will be a lot more people interested in getting something done on this issue.

[jakirkham]

Where would you advise someone to look if they were to take this on, @msarahan?

[msarahan]

The best approach is probably to do something like creating an env with python plus any deps, then recording that as the set of existing files, then conda-installing the noarch package into that env. That allows you to ignore any implementation details of how noarch is implemented. You may need some cleanup for stuff you don't want, for example the conda-meta folder contents.

search through the codebase for usage of create_env - that should be enough for both of those purposes. For the bundling afterwards, check out the existing convert code.

[mara004]

Could you please revisit this issue? I too would like to convert noarch: python packages to platform specific packages.

For some context, what I'm trying to do is package pypdfium2 for conda upon request of the doctr project. We're using external binaries and ABI-level bindings, so we want to craft platform specific packages that are independent of python version.
I'm new to conda and have tried a lot of different patterns but am unable to achieve this goal within the current capabilities of conda-build.
I think it would work if I could build noarch: python packages as an intermediary and then transform them using conda convert. Without noarch: python, I'm always ending up with packages for a single Python version, which is not what we want.

Thanks!

[mara004]

What about .pyc files?

Putting .pyc files into the packages doesn't seem to make much sense to me because it assumes they are bound to a specific Python version, which would mean unnecessarily complicated packaging. It doesn't sound very reasonable to build for 8 Platforms * 6 Python versions – that would be 48 separate packages.

[mara004]

@mandeep @msarahan @kenodegard @ericdill @p3trus, or anyone else interested:
Do you have any hints on the use case I described? Am I right to assume this is not possible with conda yet?

[github-actions]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   - What OS and version you reproduced the issue on
   - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!