Code cleanup in abstract classes #18811

akashkulk · 2022-11-01T20:19:03Z

Increasing readability by

Changing method signature of methods that are marked as public/protected but should be private
Changing method signature of methods that are marked as public but should be protected

Should mainly just make navigating through code in IntelliJ easier, reasoning about which methods can/are actually overriden in subclasses

In addition:

Remove AbstractDbRelationalDbSource class and pull out methods to a utility class so help with the abstract class hierarchy issue

@OverRide

Increasing readability: - Performed audit of methods that are marked as public/protected but should be private - Performed audit of methods that are marked as public but should be protected - Removing @OverRide from methods that don't need them (are the very first)

- Removing AbstractRelationalDbSource abstract class and moving contents of it to a Util class

akashkulk · 2022-11-01T20:19:55Z

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/3372595151
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/3372595151
No Python unittests run

Build Passed

Test summary info:

All Passed

github-actions · 2022-11-01T20:20:00Z

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

Run integration tests
Bump connector version
Add changelog
Publish the new version

⚠ Sources (24)

Connector	Version	Changelog	Publish
`source-alloydb`	`1.0.17`	✅	✅
`source-alloydb-strict-encrypt`	`1.0.17`	✅	⚠ (not in seed)
`source-bigquery`	`0.2.2`	✅	✅
`source-clickhouse`	`0.1.14`	✅	✅
`source-clickhouse-strict-encrypt`	`0.1.14`	✅	⚠ (not in seed)
`source-cockroachdb`	`0.1.18`	✅	✅
`source-cockroachdb-strict-encrypt`	`0.1.18`	✅	⚠ (not in seed)
`source-db2`	`0.1.16`	✅	✅
`source-db2-strict-encrypt`	`0.1.16`	✅	⚠ (not in seed)
`source-jdbc`	`0.3.5`	⚠ (doc not found)	⚠ (not in seed)
`source-mongodb-strict-encrypt`	`0.1.19`	⚠ (doc not found)	⚠ (not in seed)
`source-mongodb-v2`	`0.1.19`	✅	✅
`source-mssql`	`0.4.24`	✅	✅
`source-mssql-strict-encrypt`	`0.4.24`	✅	⚠ (not in seed)
`source-mysql`	`1.0.11`	✅	✅
`source-mysql-strict-encrypt`	`1.0.11`	✅	⚠ (not in seed)
`source-oracle`	`0.3.21`	✅	✅
`source-oracle-strict-encrypt`	`0.3.21`	✅	⚠ (not in seed)
`source-postgres`	`1.0.22`	✅	✅
`source-postgres-strict-encrypt`	`1.0.22`	✅	⚠ (not in seed)
`source-redshift`	`0.3.14`	✅	✅
`source-scaffold-java-jdbc`	`0.1.0`	⚠ (doc not found)	⚠ (not in seed)
`source-snowflake`	`0.1.24`	✅	✅
`source-tidb`	`0.2.1`	✅	✅

See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (0)

Connector	Version	Changelog	Publish

See "Actionable Items" below for how to resolve warnings and errors.

Actionable Items

(click to expand)

Category	Status	Actionable Item
Version	❌ mismatch	The version of the connector is different from its normal variant. Please bump the version of the connector.
	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
Changelog	⚠ doc not found	The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like `source-jdbc` is not published or documented). Please double-check to make sure that it is not a bug.
	❌ changelog missing	There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish	⚠ not in seed	The connector is not in the seed file (e.g. `source_definitions.yaml`), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.
	❌ diff seed version	The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the `/publish` command to publish the latest version.

akashkulk · 2022-11-01T22:17:31Z

/test connector=connectors/source-postgres-strict-encrypt

🕑 connectors/source-postgres-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3373302304
✅ connectors/source-postgres-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3373302304
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-11-01T22:17:39Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3373302711
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3373302711
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-11-01T22:17:49Z

/test connector=connectors/source-mysql-strict-encrypt

🕑 connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3373304240
✅ connectors/source-mysql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3373304240
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-11-01T22:17:56Z

/test connector=connectors/source-mssql-strict-encrypt

🕑 connectors/source-mssql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3373304621
✅ connectors/source-mssql-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3373304621
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-11-01T22:18:02Z

/test connector=connectors/source-mssql

🕑 connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/3373304839
✅ connectors/source-mssql https://github.com/airbytehq/airbyte/actions/runs/3373304839
No Python unittests run

Build Passed

Test summary info:

All Passed

subodh1810 · 2022-11-02T10:52:26Z

...elational-db/src/main/java/io/airbyte/integrations/source/relationaldb/AbstractDbSource.java

@@ -77,7 +78,6 @@
  // TODO: Remove when the flag is not use anymore
  private final FeatureFlags featureFlags = new EnvVariableFeatureFlags();

-  @Override


Why are we removing this Override?

Mainly a stylistic concern.

For here and below, the superclass (BaseConnector) doesn't actually implement check/read/discover. It only implements spec, so there isn't anything to "override". I suppose in the case that these methods are added to the BaseConnector (seems unlikely) that we'd be protected.

I don't feel that strongly on this either way - let me know what you think

I prefer to have the Override keyword! Lets keep these Override keywords cause they help avoid issues by throwing compilation error in case the super class behaviour changes.

Added the override back

...elational-db/src/main/java/io/airbyte/integrations/source/relationaldb/AbstractDbSource.java

airbyte-integrations/bases/base-java/src/main/java/io/airbyte/integrations/BaseConnector.java

akashkulk · 2022-11-03T17:52:29Z

/test connector=connectors/source-postgres

🕑 connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/3388072339
✅ connectors/source-postgres https://github.com/airbytehq/airbyte/actions/runs/3388072339
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-11-03T18:29:53Z

/test connector=connectors/source-mysql

🕑 connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3388317084
✅ connectors/source-mysql https://github.com/airbytehq/airbyte/actions/runs/3388317084
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-11-03T18:30:04Z

/test connector=connectors/source-postgres-strict-encrypt

🕑 connectors/source-postgres-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3388319232
✅ connectors/source-postgres-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3388319232
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-11-03T18:30:14Z

/test connector=connectors/source-bigquery

🕑 connectors/source-bigquery https://github.com/airbytehq/airbyte/actions/runs/3388319289
✅ connectors/source-bigquery https://github.com/airbytehq/airbyte/actions/runs/3388319289
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-11-03T18:57:06Z

/test connector=connectors/source-postgres-strict-encrypt

🕑 connectors/source-postgres-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3388492251
✅ connectors/source-postgres-strict-encrypt https://github.com/airbytehq/airbyte/actions/runs/3388492251
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk · 2022-11-03T18:57:14Z

/test connector=connectors/source-mongodb-v2

🕑 connectors/source-mongodb-v2 https://github.com/airbytehq/airbyte/actions/runs/3388492936
✅ connectors/source-mongodb-v2 https://github.com/airbytehq/airbyte/actions/runs/3388492936
No Python unittests run

Build Passed

Test summary info:

All Passed

akashkulk added 3 commits October 31, 2022 18:49

Refactor attempt 1

c3f963c

Increasing readability: - Performed audit of methods that are marked as public/protected but should be private - Performed audit of methods that are marked as public but should be protected - Removing @OverRide from methods that don't need them (are the very first)

Remove .sqlite files

02e75ce

Remove AbstractRelationDbSource class

54f0960

- Removing AbstractRelationalDbSource abstract class and moving contents of it to a Util class

github-actions bot added the area/connectors Connector related issues label Nov 1, 2022

octavia-squidington-iv added connectors/source/bigquery connectors/source/mssql connectors/source/jdbc connectors/source/relational-db labels Nov 1, 2022

akashkulk temporarily deployed to more-secrets November 1, 2022 20:20 Inactive

akashkulk changed the title ~~More refactoring~~ Remove AbstractDbRelationalDbSource Nov 1, 2022

akashkulk changed the title ~~Remove AbstractDbRelationalDbSource~~ Code cleanup in abstract classes Nov 1, 2022

Rename utility class

baf5dcf

akashkulk temporarily deployed to more-secrets November 1, 2022 22:18 Inactive

akashkulk requested review from subodh1810, tuliren and rodireich November 1, 2022 22:20

akashkulk marked this pull request as ready for review November 1, 2022 22:20

akashkulk requested a review from a team as a code owner November 1, 2022 22:20

subodh1810 reviewed Nov 2, 2022

View reviewed changes

subodh1810 self-requested a review November 3, 2022 09:28

Add overrides back

9969a43

akashkulk temporarily deployed to more-secrets November 3, 2022 15:46 Inactive

Merge branch 'master' into exp2

d456597

akashkulk temporarily deployed to more-secrets November 3, 2022 15:51 Inactive

rodireich approved these changes Nov 3, 2022

View reviewed changes

Merge branch 'master' into exp2

b1edd1e

akashkulk temporarily deployed to more-secrets November 3, 2022 17:01 Inactive

Merge branch 'master' into exp2

c19f4a6

akashkulk temporarily deployed to more-secrets November 3, 2022 17:54 Inactive

akashkulk removed the request for review from tuliren November 3, 2022 18:21

akashkulk merged commit 5b0ed4f into master Nov 3, 2022

akashkulk deleted the exp2 branch November 3, 2022 20:05

github-actions bot mentioned this pull request Nov 15, 2022

Bump helm chart version reference to 0.40.43 #19453

Closed

octavia-squidington-iii mentioned this pull request Nov 18, 2022

Bump Airbyte version from 0.40.18 to 0.40.19 #19579

Merged

Code cleanup in abstract classes #18811

Code cleanup in abstract classes #18811

Uh oh!

Conversation

akashkulk commented Nov 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

akashkulk commented Nov 1, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

github-actions bot commented Nov 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Affected Connector Report

⚠ Sources (24)

✅ Destinations (0)

Actionable Items

Uh oh!

akashkulk commented Nov 1, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

akashkulk commented Nov 1, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

akashkulk commented Nov 1, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

akashkulk commented Nov 1, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

akashkulk commented Nov 1, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

subodh1810 Nov 2, 2022

Choose a reason for hiding this comment

Uh oh!

akashkulk Nov 2, 2022

Choose a reason for hiding this comment

Uh oh!

subodh1810 Nov 3, 2022

Choose a reason for hiding this comment

Uh oh!

akashkulk Nov 3, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

akashkulk commented Nov 3, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

akashkulk commented Nov 3, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

akashkulk commented Nov 3, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

akashkulk commented Nov 3, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

akashkulk commented Nov 3, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

akashkulk commented Nov 3, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

Uh oh!

akashkulk commented Nov 1, 2022 •

edited

Loading

akashkulk commented Nov 1, 2022 •

edited by github-actions bot

Loading

github-actions bot commented Nov 1, 2022 •

edited

Loading

akashkulk commented Nov 1, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 1, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 1, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 1, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 1, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 3, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 3, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 3, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 3, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 3, 2022 •

edited by github-actions bot

Loading

akashkulk commented Nov 3, 2022 •

edited by github-actions bot

Loading