Skip to content

Clean up jdbc databases #10371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Feb 16, 2022
Merged

Clean up jdbc databases #10371

merged 7 commits into from
Feb 16, 2022

Conversation

tuliren
Copy link
Contributor

@tuliren tuliren commented Feb 16, 2022 •
edited
Loading

Summary

  • The StreamingJdbcDatabase should extend DefaultJdbcDatabase instead of keeping a reference of JdbcDatabase. Otherwise theoretically it is possible to pass in a StreamingJdbcDatabase to construct a StreamingJdbcDatabase and have infinite call loops.
  • Removed some of the redundant interfaces and classes. For example, the ConnectionSupplier interface is actually useless. The DataSource is already a connection supplier.
  • Relate to Ensure every JDBC database use case that returns a stream is closed #10129.

@github-actions github-actions bot added the area/connectors Connector related issues label Feb 16, 2022
@tuliren tuliren temporarily deployed to more-secrets February 16, 2022 06:21 Inactive
@tuliren tuliren temporarily deployed to more-secrets February 16, 2022 06:23 Inactive
@tuliren tuliren temporarily deployed to more-secrets February 16, 2022 06:41 Inactive
@tuliren tuliren temporarily deployed to more-secrets February 16, 2022 06:48 Inactive
@tuliren tuliren temporarily deployed to more-secrets February 16, 2022 06:48 Inactive
@tuliren tuliren changed the title Refactor jdbc database Refactor streaming jdbc database Feb 16, 2022
@tuliren tuliren temporarily deployed to more-secrets February 16, 2022 07:04 Inactive
@tuliren tuliren marked this pull request as ready for review February 16, 2022 07:05
@tuliren tuliren changed the title Refactor streaming jdbc database Clean up jdbc databases Feb 16, 2022
@tuliren tuliren requested a review from edgao February 16, 2022 07:15
@tuliren
Copy link
Contributor Author

tuliren commented Feb 16, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/1851424509
❌ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/1851424509
🐛 https://gradle.com/s/ipve3svwercri

@tuliren
Copy link
Contributor Author

tuliren commented Feb 16, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/1851424735
✅ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/1851424735
No Python unittests run

@tuliren
Copy link
Contributor Author

tuliren commented Feb 16, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/1851425085
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/1851425085
No Python unittests run

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets February 16, 2022 07:18 Inactive
@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets February 16, 2022 07:18 Inactive
@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets February 16, 2022 07:18 Inactive
@tuliren tuliren temporarily deployed to more-secrets February 16, 2022 09:35 Inactive
@tuliren tuliren temporarily deployed to more-secrets February 16, 2022 09:35 Inactive
@tuliren
Copy link
Contributor Author

tuliren commented Feb 16, 2022 •
edited by github-actions bot
Loading

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/1852000958
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/1852000958
No Python unittests run

@octavia-squidington-iii octavia-squidington-iii temporarily deployed to more-secrets February 16, 2022 09:39 Inactive
@tuliren tuliren merged commit d6747ab into master Feb 16, 2022
@tuliren tuliren deleted the liren/refactor-database branch February 16, 2022 20:19
edgao
edgao reviewed Feb 16, 2022
View reviewed changes
Copy link
Contributor

@edgao edgao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one tiny nit but otherwise this makes a lot of sense! (or at least, it makes the jdbc stuff less confusing lol)

airbyte-db/lib/src/main/java/io/airbyte/db/jdbc/DefaultJdbcDatabase.java
@@ -116,7 +113,12 @@ public DatabaseMetaData getMetaData() throws SQLException {

@Override
public void close() throws Exception {
connectionSupplier.close();
if (dataSource instanceof AutoCloseable autoCloseable) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tiny nitpick: it would be nice to keep this comment https://github.com/airbytehq/airbyte/pull/10342/files#diff-c7c618583584a27fb3b21cef812ca82b6d077d8bc779cf9a401bfc7658bb8dbaL145-L146

also I didn't know this instanceof T var syntax existed! it's pretty cool 🚛

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Will add it in a follow-up PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in #10444.

@tuliren tuliren mentioned this pull request Feb 17, 2022
Merged
@octavia-squidington-iii octavia-squidington-iii mentioned this pull request Feb 17, 2022
Merged
@novotl novotl mentioned this pull request Feb 22, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edgao edgao edgao left review comments

Assignees
No one assigned
Labels
area/connectors Connector related issues
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tuliren @edgao @octavia-squidington-iii