[Python] Svix client v1.34 is broken (again)

[anlambert]

I recently reported #1415 and wanted to check if the issue was resolved svix 1.34.

After bumping svix server and Python client version to 1.34, I got the following errors when executing mypy on the code using the svix Python client I am working on:

swh/webhooks/interface.py:255: error: Argument "schemas" to "EventTypeIn" has incompatible type "dict[str, dict[str, Any]]"; expected "Unset | EventTypeInSchemas | None"  [arg-type]
swh/webhooks/interface.py:265: error: Argument "schemas" to "EventTypeUpdate" has incompatible type "dict[str, dict[str, Any]]"; expected "Unset | EventTypeUpdateSchemas | None"  [arg-type]
swh/webhooks/interface.py:397: error: Argument "metadata" to "EndpointIn" has incompatible type "dict[str, Any]"; expected "Unset | EndpointInMetadata"  [arg-type]
swh/webhooks/interface.py:414: error: Argument "headers" to "EndpointHeadersIn" has incompatible type "dict[str, str]"; expected "EndpointHeadersInHeaders"  [arg-type]
swh/webhooks/interface.py:468: error: "EndpointOutMetadata" has no attribute "pop"  [attr-defined]
swh/webhooks/interface.py:571: error: Argument "payload" to "MessageIn" has incompatible type "dict[str, Any]"; expected "MessageInPayload"  [arg-type]
swh/webhooks/interface.py:718: error: "EndpointOutMetadata" has no attribute "get"  [attr-defined]

New Python types for svix API parameters seem to have been introduced so I updated my code accordingly, below is an example of using the new MessageInPayload type.

diff --git a/swh/webhooks/interface.py b/swh/webhooks/interface.py
index ea00983..4ab418c 100644
--- a/swh/webhooks/interface.py
+++ b/swh/webhooks/interface.py
@@ -31,6 +31,7 @@ from svix.api import (
     MessageAttemptListOptions,
     MessageAttemptOut,
     MessageIn,
+    MessageInPayload,
     MessageListOptions,
     Ordering,
     Svix,
@@ -568,7 +569,7 @@ class Webhooks:
                 app_uid,
                 MessageIn(
                     event_type=event_type_name,
-                    payload=dict(sorted(payload.items())),
+                    payload=MessageInPayload.from_dict(dict(sorted(payload.items()))),
                     channels=[_gen_uuid(channel)] if channel else None,
                     payload_retention_period=self.config.get(
                         "event_retention_period", 90

By the way, the other introduced types: EventTypeInSchemas, EventTypeUpdateSchemas, EndpointInMetadata, EndpointHeadersInHeaders cannot be imported from svix.api module, I had to import them from svix.internal.openapi_client.*.

I could fix all the reported typing errors but when executing the modified code, the following type of error is reported:

swh/webhooks/tests/test_webhooks.py:440: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../.virtualenvs/swh/lib/python3.11/site-packages/tenacity/__init__.py:336: in wrapped_f
    return copy(f, *args, **kw)
../../../.virtualenvs/swh/lib/python3.11/site-packages/tenacity/__init__.py:475: in __call__
    do = self.iter(retry_state=retry_state)
../../../.virtualenvs/swh/lib/python3.11/site-packages/tenacity/__init__.py:376: in iter
    result = action(retry_state)
../../../.virtualenvs/swh/lib/python3.11/site-packages/tenacity/__init__.py:398: in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
/usr/lib/python3.11/concurrent/futures/_base.py:449: in result
    return self.__get_result()
/usr/lib/python3.11/concurrent/futures/_base.py:401: in __get_result
    raise self._exception
../../../.virtualenvs/swh/lib/python3.11/site-packages/tenacity/__init__.py:478: in __call__
    result = fn(*args, **kwargs)
build/__editable__.swh.webhooks-0.3.1.dev3+g9b381c6-py3-none-any/swh/webhooks/interface.py:568: in event_send
    message_out = self.svix_api.message.create(
../../../.virtualenvs/swh/lib/python3.11/site-packages/svix/api.py:1108: in create
    ret = v1_message_create.request_sync(
../../../.virtualenvs/swh/lib/python3.11/site-packages/svix/internal/openapi_client/api/message/v1_message_create.py:198: in request_sync
    return request_sync_detailed(
../../../.virtualenvs/swh/lib/python3.11/site-packages/svix/internal/openapi_client/api/message/v1_message_create.py:138: in request_sync_detailed
    response = httpx.request(
../../../.virtualenvs/swh/lib/python3.11/site-packages/httpx/_api.py:100: in request
    return client.request(
../../../.virtualenvs/swh/lib/python3.11/site-packages/httpx/_client.py:801: in request
    request = self.build_request(
../../../.virtualenvs/swh/lib/python3.11/site-packages/httpx/_client.py:358: in build_request
    return Request(
../../../.virtualenvs/swh/lib/python3.11/site-packages/httpx/_models.py:339: in __init__
    headers, stream = encode_request(
../../../.virtualenvs/swh/lib/python3.11/site-packages/httpx/_content.py:214: in encode_request
    return encode_json(json)
../../../.virtualenvs/swh/lib/python3.11/site-packages/httpx/_content.py:177: in encode_json
    body = json_dumps(json).encode("utf-8")
/usr/lib/python3.11/json/__init__.py:231: in dumps
    return _default_encoder.encode(obj)
/usr/lib/python3.11/json/encoder.py:200: in encode
    chunks = self.iterencode(o, _one_shot=True)
/usr/lib/python3.11/json/encoder.py:258: in iterencode
    return _iterencode(o, 0)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <json.encoder.JSONEncoder object at 0x7fa8945a7150>, o = MessageInPayload(additional_properties={'origin_url': 'https://git.example.org/user/project'})

    def default(self, o):
        """Implement this method in a subclass such that it returns
        a serializable object for ``o``, or calls the base implementation
        (to raise a ``TypeError``).
    
        For example, to support arbitrary iterators, you could
        implement default like this::
    
            def default(self, o):
                try:
                    iterable = iter(o)
                except TypeError:
                    pass
                else:
                    return list(iterable)
                # Let the base class default method raise the TypeError
                return JSONEncoder.default(self, o)
    
        """
>       raise TypeError(f'Object of type {o.__class__.__name__} '
                        f'is not JSON serializable')
E       TypeError: Object of type MessageInPayload is not JSON serializable

/usr/lib/python3.11/json/encoder.py:180: TypeError

I also got the same type of errors for the others newly introduced types.

I think you should add some unit tests to the Python client. I could work on a PR to add those.

PR #1439 is an attempt to fix the same kind of encountered issue but the fix is obviously not the right one.

[anlambert]

I think you should add some unit tests to the Python client. I could work on a PR to add those.

FWIW, I drafted #1441 for setting up unit tests for Svix Python OpenAPI client.

[svix-jplatte]

We're looking into this, should have a fix out soon. Sorry for breaking things again!

[tasn]

Hey @anlambert, I'm so sorry this happened (especially since it seems like it's not the first time). I wanted to let you know that we are taking this very seriously and putting mechanisms in place to prevent this from happening again. Sorry for this.

[urimerhav]

Yup +1 on this, I'm a new user and a very weird exprience to just spend hours figuring out what in the heck I'm doing wrong

[urimerhav]

FYI if it helps someone: 1.33 also doesn't work. 1.30 does.

[svix-aaron1011]

This should be fixed in the newly-released v1.35.0.

[anlambert]

This should be fixed in the newly-released v1.35.0.

I confirm the osberved issues are fixed, I rebased #1441 and tests are now passing. From my point of view, adding a couple of tests for the Python client on the most Svix essential features (application, event type, endpoint, message) would robustify your releases. I can finish that work in #1441 or handle it to you.

[anlambert]

Closing this as it is resolved, we can continue discussion on adding tests for the Python client in #1441.