Error with `import pip` in pip 9.0.2

[Miserlou]

Some users are now experiencing an import error when calling import pip in a function with pip 9.0.2. Downgrading to 9.0.1 fixes the issue.

Trace: https://user-images.githubusercontent.com/2273226/37558391-5e41fc94-2a24-11e8-9fdc-5884445e829a.png

More details here:
Miserlou/Zappa#1446

[nx-jackal]

I can confirm this as well. Just about to file same issue.

[alex]

This is a dupe of #5079 -- importing pip is not a supported use case (and never has been).

[Miserlou]

Come on.. making a huge breaking change in a single x.x.PATCH version bump? For a top level method that's called "main"? I think that's pretty poor form..

[FlipperPA]

This also broke Anaconda and I came up with the same solution as @Miserlou:

ContinuumIO/anaconda-issues#8911

Errors are being reported across a lot of projects.

[RayJ414]

This error also bothered me when I was trying to use pip to update my anaconda-navigator.

[Miserlou]

Is there any chance of us getting a 9.0.3 that fixes this - ideally quite soon?

This is already affecting lots of major projects (Anaconda, SpaCy, Zappa), and there are more than 850k+ uses of this on GitHub alone that are now broken by this supposed "patch" version update.

Can you at least revert this massive change in 9.0.3 and then announce to downstreams your intention to change this behavior for a future 10.x.x or at least 9.x.x release?

[RemiCardona]

We don't want to use an older version either, but that's exactly what we ended up doing. 9.0.2 does not exist inside our CI environment, at least for now.

[stmcginnis]

Also seeing this hit in some OpenStack projects.

[pfmoore]

Pip 10 is due out in about a month. As I understand it, the issue is with code that uses import pip to access internal functionality of pip. We've never supported such usage officially, and we have explicitly documented that lack of support for some time now (albeit only in the "latest" version of the docs at https://pip.pypa.io/en/latest/user_guide/#using-pip-from-your-program, because we haven't had a new stable release since that doc section was added). We also announced a reorganisation of the internals to make it clear that use of internal APIs is unsupported, last October (see https://mail.python.org/pipermail/distutils-sig/2017-October/031642.html). That change, which is in pip 10, will break any such usage regardless of what pip a possible pip 9.0.3 release would do. So it's hard to see this as a sudden, unexpected breakage.

If @dstufft wants to make an emergency 9.0.3 release, I'm fine with that. But it will only be a very short term benefit, and I'm a little disappointed that people haven't already moved away from import pip. The people hit by this issue will still need to prepare for pip 10, and should understand that simply switching to import pip._internal is not something we will support or recommend.

Proposals for reintroducing some level of API support are certainly an option (see #5080 for example) but at this stage, it's too late for any such changes to make it into pip 10.

[stmcginnis]

With no warnings being raised by the code for those using the old way, nothing denoting this as "internal" with starting with _, and this just being a bugfix bump, I actually think it's quite easy to see this as a sudden, unexpected breakage.