Skip to content

🐛 Source Monday: handle complexityBudgetExhausted error #36717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 8, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ data:
connectorSubtype: api
connectorType: source
definitionId: 80a54ea2-9959-4040-aac1-eee42423ec9b
dockerImageTag: 2.0.4
dockerImageTag: 2.0.5
releases:
breakingChanges:
2.0.0:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = [ "poetry-core>=1.0.0",]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
version = "2.0.4"
version = "2.0.5"
name = "source-monday"
description = "Source implementation for Monday."
authors = [ "Airbyte <[email protected]>",]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ definitions:
response_filters:
- predicate: "{{ 'error_code' in response and response['error_code'] == 'ComplexityException' }}"
action: RETRY
- predicate: "{{ 'error_code' in response and response['error_code'] == 'complexityBudgetExhausted' }}"
action: RETRY
backoff_strategies:
- type: ConstantBackoffStrategy
backoff_time_in_seconds: 60
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,8 @@ def __init__(self, status_code: int):
def response_with_status(cls, status_code) -> "ErrorResponseBuilder":
return cls(status_code)

def build(self) -> HttpResponse:
return HttpResponse(json.dumps(find_template(str(self._status_code), __file__)), self._status_code)
def build(self, file_path=None) -> HttpResponse:
if not file_path:
return HttpResponse(json.dumps(find_template(str(self._status_code), __file__)), self._status_code)
return HttpResponse(json.dumps(find_template(str(file_path), __file__)), self._status_code)

Original file line number Diff line number Diff line change
Expand Up @@ -44,27 +44,30 @@ def test_given_retryable_error_and_one_page_when_read_teams_then_return_records(
"""
A full refresh sync without pagination completes successfully after one retry
"""
api_token_authenticator = self.get_authenticator(self._config)

http_mocker.get(
TeamsRequestBuilder.teams_endpoint(api_token_authenticator).build(),
[
ErrorResponseBuilder.response_with_status(200).build(),
TeamsResponseBuilder.teams_response().with_record(TeamsRecordBuilder.teams_record()).build(),
],
)

with patch("time.sleep", return_value=None):
output = read_stream("teams", SyncMode.full_refresh, self._config)

assert len(output.records) == 1

error_logs = [
error
for error in get_log_messages_by_log_level(output.logs, LogLevel.INFO)
if f'Response Code: 200, Response Text: {json.dumps({"error_code": "ComplexityException", "status_code": 200})}' in error
]
assert len(error_logs) == 1
test_cases = [("200_ComplexityException", "ComplexityException"),("200_complexityBudgetExhausted", "complexityBudgetExhausted"),]
for test_values in test_cases:
response, error_code = test_values[0], test_values[1]
api_token_authenticator = self.get_authenticator(self._config)

http_mocker.get(
TeamsRequestBuilder.teams_endpoint(api_token_authenticator).build(),
[
ErrorResponseBuilder.response_with_status(200).build(response),
TeamsResponseBuilder.teams_response().with_record(TeamsRecordBuilder.teams_record()).build(),
],
)

with patch("time.sleep", return_value=None):
output = read_stream("teams", SyncMode.full_refresh, self._config)

assert len(output.records) == 1

error_logs = [
error
for error in get_log_messages_by_log_level(output.logs, LogLevel.INFO)
if f'Response Code: 200, Response Text: {json.dumps({"error_code": error_code, "status_code": 200})}' in error
]
assert len(error_logs) == 1

@HttpMocker()
def test_given_retryable_error_when_read_teams_then_stop_syncing(self, http_mocker):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"error_code": "ComplexityException",
"status_code": 200
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"error_code": "complexityBudgetExhausted",
"status_code": 200
}
1 change: 1 addition & 0 deletions docs/integrations/sources/monday.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ The Monday connector should not run into Monday API limitations under normal usa

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:----------------------------------------------------------|:------------------------------------------------------------------------|
| 2.0.5 | 2024-04-01 | [36717](https://github.com/airbytehq/airbyte/pull/36717) | Add handling of complexityBudgetExhausted error. |
| 2.0.4 | 2024-02-28 | [35696](https://github.com/airbytehq/airbyte/pull/35696) | Fix extraction for `null` value in stream `Activity logs` |
| 2.0.3 | 2024-02-21 | [35506](https://github.com/airbytehq/airbyte/pull/35506) | Support for column values of the mirror type for the `Items` stream. |
| 2.0.2 | 2024-02-12 | [35146](https://github.com/airbytehq/airbyte/pull/35146) | Manage dependencies with Poetry. |
Expand Down
Loading