Test support for PyMongo 4.0 #46

juliusgeo · 2021-12-07T22:58:32Z

No description provided.

blink1073 · 2021-12-08T16:53:04Z

.github/workflows/test.yml

+      - uses: actions/cache@v2
+        with:
+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-${{ hashFiles('**/setup.py') }}


Suggested change

key: ${{ runner.os }}-pip-${{ hashFiles('**/setup.py') }}

key: ${{ runner.os }}-${{ matrix.python-version}}-pip-${{ hashFiles('**/setup.py') }}

The wheels will be different for each version of python

blink1073 · 2021-12-08T16:53:23Z

.github/workflows/test.yml

+          path: ~/.cache/pip
+          key: ${{ runner.os }}-pip-${{ hashFiles('**/setup.py') }}
+          restore-keys: |
+            ${{ runner.os }}-pip-


Suggested change

${{ runner.os }}-pip-

${{ runner.os }}-${{ matrix.python-version}}-pip-

ShaneHarvey · 2021-12-08T20:38:11Z

.github/workflows/test.yml

@@ -9,6 +9,11 @@ jobs:
      matrix:
        python-version: ["3.5", "3.6", "3.7", "3.8", "pypy-3.7"]
        mongodb-version: ["4.4"]
+        pymongo-version: ["v3.11", "v4.0"]
+        # pymongo 4.0 does not support python 3.5


Can you file a new issue to drop support for 3.5 if one doesn't exist already?

ShaneHarvey · 2021-12-08T20:40:18Z

.github/workflows/test.yml

+          path: ~/.cache/pip
+          key: ${{ runner.os }}-${{ matrix.python-version}}-pip-${{ hashFiles('**/setup.py') }}
+          restore-keys: |
+            ${{ runner.os }}-${{ matrix.python-version}}-pip-


Is this needed? How much time are we saving? IIRC these tests run very fast already.

The tests do already run fast. I mainly added this in because Steve suggested it and I thought it would be an easy way to make it run a bit faster.

Okay and what is the impact of this change?

It is a general thing that we add to the Jupyter repos to save time and bandwidth to pypi.org. If we don't want it here that's fine too.

Yeah I get that, I just want to know how much time was actully saved in our case. How slow were the old tests and how much quicker are they now?

In this case I see about 40s -> 30s on the Python 3.8 builds.

Thanks SGTM!

ShaneHarvey · 2021-12-08T20:41:14Z

.github/workflows/test.yml

@@ -9,6 +9,11 @@ jobs:
      matrix:
        python-version: ["3.5", "3.6", "3.7", "3.8", "pypy-3.7"]
        mongodb-version: ["4.4"]
+        pymongo-version: ["v3.11", "v4.0"]


Please remove the v's

Good catch. Fixed.

ShaneHarvey · 2021-12-08T20:41:34Z

.github/workflows/test.yml

+        # pymongo 4.0 does not support python 3.5
+        exclude:
+          - python-version: "3.5"
+            pymongo-version: "v4.0"


Remove the v here too.

ShaneHarvey

So the way I see it, this change so far does ensure that we're now compatible with pymongo 4.0 but is there's still work that needs to be done in the API (new parameters added in 3.12/4.0)?

One issue here is that we're using the v3.11 era CRUD spec tests. With v4.0 we'd ideally want to use the 4.0 era CRUD tests which happen to be in the unified format.

ShaneHarvey · 2021-12-08T20:56:02Z

.github/workflows/test.yml

+          path: ~/.cache/pip
+          key: ${{ runner.os }}-${{ matrix.python-version}}-pip-${{ hashFiles('**/setup.py') }}
+          restore-keys: |
+            ${{ runner.os }}-${{ matrix.python-version}}-pip-


Okay and what is the impact of this change?

ShaneHarvey

LGTM, please open a follow up issue for updating the spec tests themselves. We don't have to work on that ticket now but it would be good to track it. Also it would be good to confirm that there aren't any methods or parameters that need to be updated. I'm not seeing anything notable in the changelog: https://pymongo.readthedocs.io/en/3.12.0/changelog.html

ShaneHarvey · 2021-12-09T20:36:18Z

.github/workflows/test.yml

+          path: ~/.cache/pip
+          key: ${{ runner.os }}-${{ matrix.python-version}}-pip-${{ hashFiles('**/setup.py') }}
+          restore-keys: |
+            ${{ runner.os }}-${{ matrix.python-version}}-pip-


Thanks SGTM!

juliusgeo · 2021-12-10T04:18:00Z

@ShaneHarvey I created an issue to track updating the spec tests. https://jira.mongodb.org/browse/PYTHON-3039

ShaneHarvey

LGTM!

ShaneHarvey · 2021-12-10T20:30:54Z

test/test_crud_v2.py

+        self.run_operations(sessions, collection, test['operations'])
+        if not hasattr(self, "command_logger"):
+            raise Exception("You forgot to add in the bits of code that make "
+                            "utils_spec_runner.py test pymongoexplain!")


juliusgeo added 11 commits December 7, 2021 14:58

initial commit

1b2b5c9

update test/__init__.py

5b6c5b6

update utils files

1b07454

add dynamic vendoring

6bf308c

use http instead of ssh for cloning pymongo

923125c

fix typo in cp

0999eb7

make it explicit

825a14a

exclude 3.5 and 4.0

9b44974

fix typo

780ec65

only clone a single branch

087aa59

fix line that is unnecessary and add comment

2b94244

blink1073 reviewed Dec 8, 2021

View reviewed changes

juliusgeo added 2 commits December 8, 2021 12:26

go to workaround, steve fixes

038cc14

switch to importerror

7b2c640

juliusgeo requested review from blink1073 and ShaneHarvey December 8, 2021 20:32

ShaneHarvey reviewed Dec 8, 2021

View reviewed changes

shane fixes

5074eb9

ShaneHarvey reviewed Dec 8, 2021

View reviewed changes

ShaneHarvey reviewed Dec 9, 2021

View reviewed changes

ShaneHarvey approved these changes Dec 9, 2021

View reviewed changes

update spec runner util to actually test pymongoexplain

1b6d0da

juliusgeo requested a review from ShaneHarvey December 10, 2021 04:06

fix spacing

0fe7eb9

ShaneHarvey approved these changes Dec 10, 2021

View reviewed changes

test to make sure files are vendored correctly

b903c8a

ShaneHarvey approved these changes Dec 10, 2021

View reviewed changes

juliusgeo merged commit 2bce70c into mongodb-labs:master Dec 10, 2021

ShaneHarvey mentioned this pull request Jan 6, 2022

Release 1.2 #50

Open

	key: ${{ runner.os }}-pip-${{ hashFiles('**/setup.py') }}
	key: ${{ runner.os }}-${{ matrix.python-version}}-pip-${{ hashFiles('**/setup.py') }}

	${{ runner.os }}-pip-
	${{ runner.os }}-${{ matrix.python-version}}-pip-

Test support for PyMongo 4.0 #46

Test support for PyMongo 4.0 #46

Uh oh!

Conversation

juliusgeo commented Dec 7, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ShaneHarvey left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ShaneHarvey left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

juliusgeo commented Dec 10, 2021

Uh oh!

ShaneHarvey left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ShaneHarvey left a comment •

edited

Loading

ShaneHarvey left a comment •

edited

Loading