Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent conversion of null fields #36919

Merged

Conversation

rodireich
Copy link
Contributor

@rodireich rodireich commented Apr 9, 2024

closes https://github.com/airbytehq/airbyte-internal-issues/issues/7055


Ellipsis 🚀 This PR description was created by Ellipsis for commit 44d9beb.

Summary:

This PR addresses a bug in the Airbyte CDK related to null field conversion, updates the CDK version, includes tests for null field conversion, and updates the build.gradle and metadata.yaml files for source-mssql and source-mysql connectors.

Key points:

  • Fixes a bug related to the conversion of null fields in the Airbyte CDK
  • Adds a null check in AbstractJdbcCompatibleSourceOperations.kt
  • Updates the CDK version in version.properties
  • Includes tests for null field conversion in various test files
  • Updates build.gradle and metadata.yaml files for source-mssql and source-mysql connectors

Generated with ❤️ by ellipsis.dev

Copy link

vercel bot commented Apr 9, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Apr 12, 2024 0:15am

@octavia-squidington-iii octavia-squidington-iii added area/connectors Connector related issues CDK Connector Development Kit connectors/source/mysql labels Apr 9, 2024
@octavia-squidington-iii octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Apr 11, 2024
@rodireich rodireich marked this pull request as ready for review April 11, 2024 05:59
@rodireich rodireich requested review from a team as code owners April 11, 2024 05:59
Copy link

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me!

  • Reviewed the entire pull request up to 35204c1
  • Looked at 705 lines of code in 13 files
  • Took 1 minute and 17 seconds to review
More info
  • Skipped 0 files when reviewing.
  • Skipped posting 1 additional comments because they didn't meet confidence threshold of 50%.
1. airbyte-cdk/java/airbyte-cdk/core/src/main/kotlin/io/airbyte/cdk/db/jdbc/AbstractJdbcCompatibleSourceOperations.kt:47:
  • Assessed confidence : 0%
  • Comment:
    Good job on adding a null check before type-specific parsing. This will prevent potential null pointer exceptions.
  • Reasoning:
    The PR author has added a null check before type-specific parsing. This is a good practice as it prevents null pointer exceptions. The code seems logically correct. I don't see any clear violations of best practices, logical bugs, performance bugs, or security bugs. The changes in the test files are also appropriate and they are testing the new functionality of handling null values.

Workflow ID: wflow_BwArMvy2clrTw0h1


Not what you expected? You can customize the content of the reviews using rules. Learn more here.

Copy link

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me!

  • Performed an incremental review on a156070
  • Looked at 487 lines of code in 5 files
  • Took 2 minutes and 8 seconds to review
More info
  • Skipped 0 files when reviewing.
  • Skipped posting 2 additional comments because they didn't meet confidence threshold of 50%.
1. airbyte-integrations/connectors/source-mysql/src/test-integration/java/io/airbyte/integrations/io/airbyte/integration_tests/sources/MySqlSourceAcceptanceTest.java:106:
  • Assessed confidence : 0%
  • Comment:
    The test case for null field conversion is correctly implemented. No changes required.
  • Reasoning:
    The changes in the test files are mostly formatting changes and the addition of a new test case to check for null field conversion. The test case seems to be correctly implemented, with the necessary setup, execution, and assertions. No issues found.
2. airbyte-cdk/java/airbyte-cdk/core/src/main/kotlin/io/airbyte/cdk/db/jdbc/AbstractJdbcCompatibleSourceOperations.kt:45:
  • Assessed confidence : 0%
  • Comment:
    The changes in this file are just formatting changes and do not affect the functionality of the code. No changes required.
  • Reasoning:
    The changes in the AbstractJdbcCompatibleSourceOperations.kt file are just formatting changes and do not affect the functionality of the code. No issues found.

Workflow ID: wflow_zeSUF8s5fridkAzI


Not what you expected? You can customize the content of the reviews using rules. Learn more here.

Copy link

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me!

  • Performed an incremental review on 908d372
  • Looked at 301 lines of code in 5 files
  • Took 2 minutes and 37 seconds to review
More info
  • Skipped 0 files when reviewing.
  • Skipped posting 1 additional comments because they didn't meet confidence threshold of 50%.
1. airbyte-integrations/connectors/source-mssql/src/test-integration/java/io/airbyte/integrations/source/mssql/CdcMssqlSourceAcceptanceTest.java:214:
  • Assessed confidence : 100%
  • Grade: 0%
  • Comment:
    The extractLatestState function is duplicated in all the test files. Consider moving this function to a common utility file to avoid code duplication.
  • Reasoning:
    The extractLatestState function is duplicated in all the test files. It would be better to move this function to a common utility file to avoid code duplication.

Workflow ID: wflow_rhDlAjuMp0O0nsES


Not what you expected? You can customize the content of the reviews using rules. Learn more here.

Copy link

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me!

  • Performed an incremental review on 6314fe9
  • Looked at 93 lines of code in 3 files
  • Took 1 minute and 26 seconds to review
More info
  • Skipped 0 files when reviewing.
  • Skipped posting 0 additional comments because they didn't meet confidence threshold of 50%.

Workflow ID: wflow_2n0N4JiRoUX8UGW1


Not what you expected? You can customize the content of the reviews using rules. Learn more here.

Copy link

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me!

  • Performed an incremental review on 44d9beb
  • Looked at 16 lines of code in 1 files
  • Took 2 minutes and 20 seconds to review
More info
  • Skipped 0 files when reviewing.
  • Skipped posting 1 additional comments because they didn't meet confidence threshold of 50%.
1. /airbyte-integrations/connectors/source-mysql/src/test-integration/java/io/airbyte/integrations/io/airbyte/integration_tests/sources/MySqlSourceAcceptanceTest.java:105:
  • Assessed confidence : 100%
  • Grade: 0%
  • Comment:
    The test method testNullValueConversion should be marked as public instead of protected. Also, the test does not seem to validate the handling of null values. Consider adding assertions to check if null values are correctly handled.
  • Reasoning:
    The test method testNullValueConversion is marked as protected which is not a common practice for test methods. Test methods are usually marked as public to ensure they are accessible to the test runner. Also, the test method does not seem to test the null value conversion as the PR description suggests. It creates a table with a nullable field but does not check if the null value is correctly handled.

Workflow ID: wflow_F829qu6CD5VWiriU


Not what you expected? You can customize the content of the reviews using rules. Learn more here.

Copy link
Contributor

@akashkulk akashkulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick question : why can't we just add a test case to the data type test again?

seems like it's just calling runRead() so should hit this path

@akashkulk
Copy link
Contributor

Quick question : why can't we just add a test case to the data type test again?

seems like it's just calling runRead() so should hit this path

e.g. add 0 to this test case
https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-mysql/src/test-integration/java/io/airbyte/integrations/io/airbyte/integration_tests/sources/AbstractMySqlSourceDatatypeTest.java#L188

@rodireich
Copy link
Contributor Author

rodireich commented Apr 11, 2024

/publish-java-cdk dry-run=true

🕑 https://github.com/airbytehq/airbyte/actions/runs/8653945372
✅ Successfully published Java CDK version=0.30.1!

@rodireich
Copy link
Contributor Author

rodireich commented Apr 11, 2024

/publish-java-cdk

🕑 https://github.com/airbytehq/airbyte/actions/runs/8654179026
✅ Successfully published Java CDK version=0.30.1!

@rodireich rodireich enabled auto-merge (squash) April 11, 2024 22:49
@rodireich rodireich merged commit 1e9ee1d into master Apr 12, 2024
28 of 29 checks passed
@rodireich rodireich deleted the 7055-source-mysql-incorrectly-converts-null-integers-to-0 branch April 12, 2024 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connectors Connector related issues area/documentation Improvements or additions to documentation CDK Connector Development Kit connectors/source/mssql connectors/source/mysql
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants