-
Notifications
You must be signed in to change notification settings - Fork 4.6k
[Source-mysql] Increase INITIAL_CDC_WAITING_SECONDS for avoiding timeouts in integration tests. #29854
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Do not run tests in parallel
Before Merging a Connector Pull RequestWow! What a great pull request you have here! 🎉 To merge this PR, ensure the following has been done/considered for each connector added or updated:
If the checklist is complete, but the CI check is failing,
|
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ❌ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
So after removing numThreads the integ tests ran for 81 minutes and still failed (passing on local), so this is not a good change. |
|
Step | Result |
---|---|
Java Connector Unit Tests | ❌ |
Build connector tar | ✅ |
Build source-mysql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ❌ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ❌ |
QA checks | ❌ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
|
Step | Result |
---|---|
Java Connector Unit Tests | ❌ |
Build connector tar | ✅ |
Build source-mysql-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ❌ |
Validate airbyte-integrations/connectors/source-mysql-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ❌ |
QA checks | ❌ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql-strict-encrypt test
Running notes - it seems testcontainers itself does not allow parallel tests at the moment? testcontainers/testcontainers-java#3220 |
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ❌ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
Randomize database names
Revert "remove parallelism" change
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-postgres-strict-encrypt docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-postgres-strict-encrypt/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ❌ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-postgres-strict-encrypt test
Coverage report for source-postgres
|
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-postgres docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ✅ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-postgres/metadata.yaml | ✅ |
Connector version semver check | ✅ |
Connector version increment check | ❌ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-postgres test
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
Set INITIAL_CDC_WAITING_SECONDS to 30s
Revert temporary changes made to these test files
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
@akashkulk running integration tests work locally with this change, CI is blocked due to #29463 |
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
Thanks! This fixes tests locally! BTW the timeout can be reduced as low as 10 seconds and still works |
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
|
Step | Result |
---|---|
Java Connector Unit Tests | ✅ |
Build connector tar | ✅ |
Build source-mysql docker image for platform linux/x86_64 | ✅ |
Java Connector Integration Tests | ❌ |
Acceptance tests | ✅ |
Validate airbyte-integrations/connectors/source-mysql/metadata.yaml | ✅ |
Connector version semver check | ✅ |
QA checks | ✅ |
☁️ View runs for commit in Dagger Cloud
Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command
airbyte-ci connectors --name=source-mysql test
These tests were failing and after some investigation it points to timeouts waiting for Debezium to start so we are trying with increasing CDC Wait time in integration tests.
CdcBinlogsMySqlSourceDatatypeTest. testDataContent() CdcBinlogsMySqlSourceDatatypeTest. testDataTypes() CdcInitialSnapshotMySqlSourceDatatypeTest. testDataContent() CdcMySqlSourceAcceptanceTest. testEmptyStateIncrementalIdenticalToFullRefresh() CdcMySqlSourceAcceptanceTest. testIncrementalSyncShouldNotFailIfBinlogIsDeleted() CdcMySqlSslCaCertificateSourceAcceptanceTest. testEmptyStateIncrementalIdenticalToFullRefresh() CdcMySqlSslCaCertificateSourceAcceptanceTest. testIncrementalSyncShouldNotFailIfBinlogIsDeleted() CdcMySqlSslRequiredSourceAcceptanceTest. testEmptyStateIncrementalIdenticalToFullRefresh() CdcMySqlSslRequiredSourceAcceptanceTest. testIncrementalSyncShouldNotFailIfBinlogIsDeleted()