Skip to content

Commit 515eced

Browse files
potiukCloud Composer Team
authored and
Cloud Composer Team
committed
Suspend Yandex provider due to protobuf limitation (#30667)
Yandex provider brings protobuf dependency down to <4 and we are gearing up to updating it everywhere else. Protobuf3 support ends in Q2 2023 for Python https://protobuf.dev/support/version-support/#python Yandex is the last provider that we do not closely collaborate with on fixing * Gogle provider dependencies are actively upgraded to latest version by Google led team: #30067 (some of the libraries are already updated) with target to update all dependencies by mid-May * Apache-Beam has already merged protobuf4 support apache/beam#25874 with the target of releasing it in 2.47.0 mid-May * The mysql-connector-python in MySQL provider is already turned into optional dependency: #30487 The only remaining dependency limiting us to protobuf 3 (<3.21) is yandexcloud. We've opened an issue to yandexcloud yandex-cloud/python-sdk#71 3 weeks ago and while there was an initial interest, there is no progress on the issue, therefore - in order to prepare for running all the tests and final migration to protobuf4 we need to suspend Yandex provider - following the suspension process we agreed and got a LAZY CONSENSUS on in the https://lists.apache.org/thread/g8b3k028qhzgw6c3yz4jvmlc67kcr9hj mailing list discussion. The yandex provider can be removed from suspension by a PR reverting this change once yandexcloud dependency removes the protobuf limitation in their release and PR reverting this change (and fixing all tests and static check that will be needed) is the way it can be done. GitOrigin-RevId: b4d6e83686ecfa81d4badbefc4f60a03d9f3c571
1 parent 89401a1 commit 515eced

12 files changed

+184
-201
lines changed

CONTRIBUTING.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -621,7 +621,7 @@ microsoft.mssql, microsoft.psrp, microsoft.winrm, mongo, mssql, mysql, neo4j, od
621621
openlineage, opsgenie, oracle, otel, pagerduty, pandas, papermill, password, pinot, plexus,
622622
postgres, presto, qds, qubole, rabbitmq, redis, s3, salesforce, samba, segment, sendgrid, sentry,
623623
sftp, singularity, slack, smtp, snowflake, spark, sqlite, ssh, statsd, tableau, tabular, telegram,
624-
trino, vertica, virtualenv, webhdfs, winrm, yandex, zendesk
624+
trino, vertica, virtualenv, webhdfs, winrm, zendesk
625625
.. END EXTRAS HERE
626626
627627
Provider packages

INSTALL

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,7 @@ microsoft.mssql, microsoft.psrp, microsoft.winrm, mongo, mssql, mysql, neo4j, od
106106
openlineage, opsgenie, oracle, otel, pagerduty, pandas, papermill, password, pinot, plexus,
107107
postgres, presto, qds, qubole, rabbitmq, redis, s3, salesforce, samba, segment, sendgrid, sentry,
108108
sftp, singularity, slack, smtp, snowflake, spark, sqlite, ssh, statsd, tableau, tabular, telegram,
109-
trino, vertica, virtualenv, webhdfs, winrm, yandex, zendesk
109+
trino, vertica, virtualenv, webhdfs, winrm, zendesk
110110
# END EXTRAS HERE
111111

112112
# For installing Airflow in development environments - see CONTRIBUTING.rst

airflow/providers/yandex/provider.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ package-name: apache-airflow-providers-yandex
2020
name: Yandex
2121
description: |
2222
Yandex including `Yandex.Cloud <https://cloud.yandex.com/>`__
23-
suspended: false
23+
suspended: true # see https://github.com/apache/airflow/pull/30667/
2424
versions:
2525
- 3.3.0
2626
- 3.2.0

docs/apache-airflow/extra-packages-ref.rst

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -214,8 +214,6 @@ These are extras that add dependencies needed for integration with external serv
214214
+---------------------+-----------------------------------------------------+-----------------------------------------------------+
215215
| vertica | ``pip install 'apache-airflow[vertica]'`` | Vertica hook support as an Airflow backend |
216216
+---------------------+-----------------------------------------------------+-----------------------------------------------------+
217-
| yandex | ``pip install 'apache-airflow[yandex]'`` | Yandex.cloud hooks and operators |
218-
+---------------------+-----------------------------------------------------+-----------------------------------------------------+
219217
| zendesk | ``pip install 'apache-airflow[zendesk]'`` | Zendesk hooks |
220218
+---------------------+-----------------------------------------------------+-----------------------------------------------------+
221219

generated/provider_dependencies.json

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -773,13 +773,6 @@
773773
"common.sql"
774774
]
775775
},
776-
"yandex": {
777-
"deps": [
778-
"apache-airflow>=2.3.0",
779-
"yandexcloud>=0.173.0"
780-
],
781-
"cross-providers-deps": []
782-
},
783776
"zendesk": {
784777
"deps": [
785778
"apache-airflow>=2.3.0",

images/breeze/output-commands-hash.txt

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
# Please do not solve it but run `breeze setup regenerate-command-images`.
33
# This command should fix the conflict and regenerate help images that you have conflict with.
44
main:83de6a9bf2b1afecd1f9ce4cd0493733
5-
build-docs:a9ee0b7c62d647cc7562df83076cbc2b
5+
build-docs:d449f8ee7b20545a2c7c46ad65226f94
66
ci:find-newer-dependencies:8fa2b57f5f0523c928743b235ee3ab5a
77
ci:fix-ownership:fee2c9ec9ef19686792002ae054fecdd
88
ci:free-space:47234aa0a60b0efd84972e6e797379f8
@@ -37,15 +37,15 @@ prod-image:verify:31bc5efada1d70a0a31990025db1a093
3737
prod-image:79bd4cc9de03ab7e1d75f025d75eee46
3838
release-management:create-minor-branch:6a01066dce15e09fb269a8385626657c
3939
release-management:generate-constraints:ae30d6ad49a1b2c15b61cb29080fd957
40-
release-management:generate-issue-content-providers:649bd0f56af072754af3e5c2580d7039
40+
release-management:generate-issue-content-providers:767a85195f6e686df63b8f8ea0fb7142
4141
release-management:prepare-airflow-package:3ac14ea6d2b09614959c0ec4fd564789
42-
release-management:prepare-provider-documentation:d2aade2a4f4fb176a89e4db0c1226386
43-
release-management:prepare-provider-packages:7f5ca821bb9c5f80e1473b7b6584adcc
42+
release-management:prepare-provider-documentation:8dab0cba3d0bf3d36ec60d71c8c23d50
43+
release-management:prepare-provider-packages:9d803d0eb5f55d1a178fff2f7951eec8
4444
release-management:release-prod-images:c9bc40938e0efad49e51ef66e83f9527
4545
release-management:start-rc-process:6aafbaceabd7b67b9a1af4c2f59abc4c
4646
release-management:start-release:acb384d86e02ff5fde1bf971897be17c
4747
release-management:verify-provider-packages:88bd609aff6d09d52ab8d80d6e055e7b
48-
release-management:493c6163a3aa0ed6f7dbd4613afc7f3c
48+
release-management:ff43546fa41c3d8a028a32b37036a362
4949
setup:autocomplete:03343478bf1d0cf9c101d454cdb63b68
5050
setup:check-all-params-in-groups:4d0f8c19cbdb56290055d863b08a3376
5151
setup:config:3ffcd35dd24b486ddf1d08b797e3d017

images/breeze/output-commands.svg

Lines changed: 45 additions & 45 deletions
Loading

images/breeze/output_build-docs.svg

Lines changed: 62 additions & 66 deletions
Loading

images/breeze/output_release-management.svg

Lines changed: 15 additions & 15 deletions
Loading

images/breeze/output_release-management_generate-issue-content-providers.svg

Lines changed: 17 additions & 17 deletions
Loading

images/breeze/output_release-management_prepare-provider-documentation.svg

Lines changed: 21 additions & 25 deletions
Loading

images/breeze/output_release-management_prepare-provider-packages.svg

Lines changed: 16 additions & 16 deletions
Loading

0 commit comments

Comments
 (0)