-
Notifications
You must be signed in to change notification settings - Fork 15.1k
always push ECS task ARN to xcom in EcsRunTaskOperator
#33703
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
We'll now have to support this new/expanded behaviour (unless/until we deprecate it of course). But on the grand scheme of things I think this is fine 👍 |
I have no objection for returning this value, just my 50 cents This never work as it described airflow/airflow/providers/amazon/aws/operators/ecs.py Lines 652 to 656 in 03fcbcc
Airflow should clear all previous XComs for the task instance before Initial solution might work but it save value in the "fake task_id", which might be a reason for some issues airflow/airflow/providers/amazon/aws/operators/ecs.py Lines 280 to 289 in fc0250f
|
Unit tests are failing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, miss that this operator never return
value as it might be expected. And for work EcsRunTaskOperator -> EcsSensor you should set reattach=True
and wait_for_completion = False
So make sense now
yeah, the "return value" is the last line of the logs, only in some case... I don't think it's a good idea, but we have to live with it. |
that xcom value was removed in #29447 and it broke our system test that relied on that xcom value
While I agree that changing the logic around how to reattach is good, removing the xcom value is potentially a breaking change.
It used to be written only when the task didn't complete, but I think returning it all the time makes sense.