worker receive msg

[jixianyihao]

Apache Airflow version

3.0.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

two different worker receive the same task

What you think should happen instead?

No response

How to reproduce

3.0.2 helm chart deploy， multi scheduler 7， worker： 20， api：3

Operating System

k8s

Versions of Apache Airflow Providers

helm chart

Deployment

Official Apache Airflow Helm Chart

Deployment details

2025-06-20 19:42:21.306
[2025-06-20T19:42:21.305+0800] {scheduler_job_runner.py:842} INFO - Setting external_id for <TaskInstance: dsds.task_run_application scheduled__2025-06-10T04:05:00+00:00 [queued]> to f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71
2025-06-20 19:45:15.518
2025-06-20 11:45:15.518051 [info ] Task execute_workload[f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] received [celery.worker.strategy]

2025-06-20 19:45:15.528
2025-06-20 11:45:15.527881 [info ] [f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] Executing workload in Celery: token='eyJ***' ti=TaskInstance(id=UUID('01978d23-23fc-72fa-9f99-bdd352265f31'), task_id='task_run_application', dag_id='x', run_id='scheduled__2025-06-10T04:05:00+00:00', try_number=1, map_index=-1, pool_slots=1, queue='default', priority_weight=1, executor_config=None, parent_context_carrier={}, context_carrier={}, queued_dttm=None) dag_rel_path=PurePosixPath('flink/airflow_task/dsds/dsds.py') bundle_info=BundleInfo(name='dags-folder', version=None) log_path='dag_id=dsds/run_id=scheduled__2025-06-10T04:05:00+00:00/task_id=task_run_application/attempt=1.log' type='ExecuteTask' [airflow.providers.celery.executors.celery_executor_utils]
2025-06-20 19:45:25.360
2025-06-20 11:45:25.359910 [info ] Task execute_workload[f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] received [celery.worker.strategy]
2025-06-20 19:45:25.369
2025-06-20 11:45:25.368977 [info ] [f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] Executing workload in Celery: token='eyJ***' ti=TaskInstance(id=UUID('01978d23-23fc-72fa-9f99-bdd352265f31'), task_id='task_run_application', dag_id='dsds', run_id='scheduled__2025-06-10T04:05:00+00:00', try_number=1, map_index=-1, pool_slots=1, queue='default', priority_weight=1, executor_config=None, parent_context_carrier={}, context_carrier={}, queued_dttm=None) dag_rel_path=PurePosixPath('flink/airflow_task/dsds/dsds.py') bundle_info=BundleInfo(name='dags-folder', version=None) log_path='dag_id=dsds/run_id=scheduled__2025-06-10T04:05:00+00:00/task_id=task_run_application/attempt=1.log' type='ExecuteTask' [airflow.providers.celery.executors.celery_executor_utils]
2025-06-20 19:45:26.848
2025-06-20 11:45:25.492400 [error ] Task execute_workload[f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] raised unexpected: ServerResponseError('Server returned error') [celery.app.trace]
2025-06-20 19:45:26.849
│ │ 'id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', │ │
2025-06-20 19:45:26.849
│ │ root_id = 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71' │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'shadow': None, │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'parent_id': None, │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'shadow': None, │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'parent_id': None, │ │
2025-06-20 19:45:26.849
│ │ 'correlation_id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', │ │
2025-06-20 19:45:26.849
│ │ 'correlation_id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', │ │
2025-06-20 19:45:26.849
│ │ uuid = 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71' │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'shadow': None, 'eta': None, 'expires': │ │
2025-06-20 19:45:26.849
│ │ 'root_id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'parent_id': None, 'argsrepr': │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'shadow': None, 'eta': None, 'expires': │ │
2025-06-20 19:45:26.849
│ │ 'root_id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'parent_id': None, 'argsrepr': │ │
2025-06-20 19:45:26.849
│ │ 'delivery_mode': 2, 'correlation_id': 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', │ │
2025-06-20 19:45:26.849
│ │ 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71', 'hostname': │ │
2025-06-20 19:45:26.849
│ │ celery_task_id = 'f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71' │ │
2025-06-20 19:45:28.453
2025-06-20 11:45:28.452977 [info ] Task execute_workload[f9b9ac62-41a3-4ccc-af5a-0a2e40c5fd71] succeeded in 12.933839422650635s: None [celery.app.trace]

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR! #51979

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.