Switch json libraries to jitpack #12531

tuliren · 2022-05-03T03:28:56Z

What

Currently two libraries, json-avro-converter and jsongenerator, depend on git repo directly.
We should depend on maven repositories instead.
Relate to fix gradle --scan for connectors build #10608.

How

The underlying libraries have been published to JitPack.
Switch the dependencies to JitPack repositories directly.

🚨 User Impact 🚨

None expected.

TODO

Run integration tests for all affected connectors.

tuliren · 2022-05-03T22:13:34Z

/test connector=connectors/source-e2e-test

🕑 connectors/source-e2e-test https://github.com/airbytehq/airbyte/actions/runs/2266640981
✅ connectors/source-e2e-test https://github.com/airbytehq/airbyte/actions/runs/2266640981
No Python unittests run

tuliren · 2022-05-03T23:46:19Z

/test connector=connectors/destination-s3

🕑 connectors/destination-s3 https://github.com/airbytehq/airbyte/actions/runs/2266968824
✅ connectors/destination-s3 https://github.com/airbytehq/airbyte/actions/runs/2266968824
No Python unittests run

tuliren · 2022-05-03T23:54:23Z

/test connector=connectors/destination-gcs

🕑 connectors/destination-gcs https://github.com/airbytehq/airbyte/actions/runs/2266992975
✅ connectors/destination-gcs https://github.com/airbytehq/airbyte/actions/runs/2266992975
No Python unittests run

tuliren · 2022-05-03T23:54:29Z

/test connector=connectors/destination-databricks

🕑 connectors/destination-databricks https://github.com/airbytehq/airbyte/actions/runs/2266993143
✅ connectors/destination-databricks https://github.com/airbytehq/airbyte/actions/runs/2266993143
No Python unittests run

tuliren · 2022-05-03T23:54:47Z

/test connector=connectors/destination-bigquery

🕑 connectors/destination-bigquery https://github.com/airbytehq/airbyte/actions/runs/2266994150
✅ connectors/destination-bigquery https://github.com/airbytehq/airbyte/actions/runs/2266994150
Python tests coverage:

Name                                                                                                                            Stmts   Miss  Cover
---------------------------------------------------------------------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                                                                                          2      0   100%
normalization/transform_catalog/reserved_keywords.py                                                                               13      0   100%
normalization/transform_catalog/__init__.py                                                                                         2      0   100%
normalization/destination_type.py                                                                                                  13      0   100%
normalization/__init__.py                                                                                                           4      0   100%
/actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/airbyte-protocol/airbyte_protocol/models/airbyte_protocol.py     124      0   100%
/actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/airbyte-protocol/airbyte_protocol/models/__init__.py               1      0   100%
/actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/airbyte-protocol/airbyte_protocol/__init__.py                      2      0   100%
normalization/transform_catalog/destination_name_transformer.py                                                                   155      8    95%
normalization/transform_config/transform.py                                                                                       159     31    81%
normalization/transform_catalog/table_name_registry.py                                                                            174     34    80%
normalization/transform_catalog/utils.py                                                                                           34      7    79%
normalization/transform_catalog/dbt_macro.py                                                                                       22      7    68%
normalization/transform_catalog/catalog_processor.py                                                                              147     80    46%
normalization/transform_catalog/transform.py                                                                                       61     38    38%
normalization/transform_catalog/stream_processor.py                                                                               534    345    35%
---------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                                            1447    550    62%

tuliren · 2022-05-04T00:07:58Z

/test connector=connectors/destination-bigquery-denormalized

🕑 connectors/destination-bigquery-denormalized https://github.com/airbytehq/airbyte/actions/runs/2267045895
✅ connectors/destination-bigquery-denormalized https://github.com/airbytehq/airbyte/actions/runs/2267045895
Python tests coverage:

Name                                                                                                                            Stmts   Miss  Cover
---------------------------------------------------------------------------------------------------------------------------------------------------
normalization/transform_config/__init__.py                                                                                          2      0   100%
normalization/transform_catalog/reserved_keywords.py                                                                               13      0   100%
normalization/transform_catalog/__init__.py                                                                                         2      0   100%
normalization/destination_type.py                                                                                                  13      0   100%
normalization/__init__.py                                                                                                           4      0   100%
/actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/airbyte-protocol/airbyte_protocol/models/airbyte_protocol.py     124      0   100%
/actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/airbyte-protocol/airbyte_protocol/models/__init__.py               1      0   100%
/actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/airbyte-protocol/airbyte_protocol/__init__.py                      2      0   100%
normalization/transform_catalog/destination_name_transformer.py                                                                   155      8    95%
normalization/transform_config/transform.py                                                                                       159     31    81%
normalization/transform_catalog/table_name_registry.py                                                                            174     34    80%
normalization/transform_catalog/utils.py                                                                                           34      7    79%
normalization/transform_catalog/dbt_macro.py                                                                                       22      7    68%
normalization/transform_catalog/catalog_processor.py                                                                              147     80    46%
normalization/transform_catalog/transform.py                                                                                       61     38    38%
normalization/transform_catalog/stream_processor.py                                                                               534    345    35%
---------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                                            1447    550    62%

* Switch json-avro-converter to jitpack * Switch jsongenerator to jitpack * Fix default config * Fix one more default config use case * Fix jitpack dependency * Move jitpack repo to root build.gradle

Switch json-avro-converter to jitpack

f3c3d7c

github-actions bot added the area/connectors Connector related issues label May 3, 2022

This was referenced May 3, 2022

Pull changes from origin airbytehq/jsongenerator#7

Merged

Log offending union values airbytehq/json-avro-converter#15

Merged

Switch jsongenerator to jitpack

b074e78

tuliren requested a review from girarda May 3, 2022 04:26

girarda approved these changes May 3, 2022

View reviewed changes

This comment was marked as outdated.

Sign in to view

tuliren added 2 commits May 3, 2022 15:04

Fix default config

0248f53

Fix one more default config use case

f4b013e

Fix jitpack dependency

121e2f2

Move jitpack repo to root build.gradle

0682003

tuliren merged commit 0c909de into master May 4, 2022

tuliren deleted the liren/use-jitpack branch May 4, 2022 00:45

tuliren mentioned this pull request May 4, 2022

fix gradle --scan for connectors build #10608

Closed

octavia-squidington-iii mentioned this pull request May 4, 2022

Bump Airbyte version from 0.36.7-alpha to 0.36.8-alpha #12553

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch json libraries to jitpack #12531

Switch json libraries to jitpack #12531

tuliren commented May 3, 2022 •

edited

Loading

This comment was marked as outdated.

This comment was marked as outdated.

tuliren commented May 3, 2022 •

edited by github-actions bot

Loading

tuliren commented May 3, 2022 •

edited by github-actions bot

Loading

tuliren commented May 3, 2022 •

edited by github-actions bot

Loading

tuliren commented May 3, 2022 •

edited by github-actions bot

Loading

tuliren commented May 3, 2022 •

edited by github-actions bot

Loading

tuliren commented May 4, 2022 •

edited by github-actions bot

Loading

Switch json libraries to jitpack #12531

Switch json libraries to jitpack #12531

Conversation

tuliren commented May 3, 2022 • edited Loading

What

How

🚨 User Impact 🚨

TODO

This comment was marked as outdated.

This comment was marked as outdated.

tuliren commented May 3, 2022 • edited by github-actions bot Loading

tuliren commented May 3, 2022 • edited by github-actions bot Loading

tuliren commented May 3, 2022 • edited by github-actions bot Loading

tuliren commented May 3, 2022 • edited by github-actions bot Loading

tuliren commented May 3, 2022 • edited by github-actions bot Loading

tuliren commented May 4, 2022 • edited by github-actions bot Loading

tuliren commented May 3, 2022 •

edited

Loading

tuliren commented May 3, 2022 •

edited by github-actions bot

Loading

tuliren commented May 3, 2022 •

edited by github-actions bot

Loading

tuliren commented May 3, 2022 •

edited by github-actions bot

Loading

tuliren commented May 3, 2022 •

edited by github-actions bot

Loading

tuliren commented May 3, 2022 •

edited by github-actions bot

Loading

tuliren commented May 4, 2022 •

edited by github-actions bot

Loading