fix(kubernetes): do not fail task when xcom sidecar kill fails after successful XCom read - #71372
fix(kubernetes): do not fail task when xcom sidecar kill fails after successful XCom read#71372waterWang wants to merge 2 commits into
Conversation
|
Closing this as part of a cleanup of a large batch of PRs opened in quick succession from this account. 18 PRs have been opened here in the past two weeks and none have merged. Several show signs of being generated and submitted without review: #71432 and #71433 are the same change across the same five files, opened two minutes apart, and several titles carry a leaked agent identifier that other contributors already flagged as garbled text on #70629 and #71322. Airflow is maintained by volunteers. Every PR costs reviewer time and CI capacity, so a high volume of unvetted submissions has a real cost to the project. You are welcome to keep contributing. Please open one change at a time, run it locally against the tests, and read the contributors' guide before submitting. If you think a specific change here is correct, comment with the reasoning and a maintainer can reopen it. |
|
@waterWang I am testing with the above suggested changes. |
Issue
Closes #71369
Problem
When
KubernetesPodOperatoris used withdo_xcom_push=True, the XCom sidecar container starts and the XCom value is read successfully from/airflow/xcom/return.json. However, the subsequentextract_xcom_killstep attempts to terminate the sidecar viakill -2overkubectl exec, which fails withPermission deniedon certain container runtimes (e.g., k3s/containerd).Because
extract_xcom_killis called in thefinallyblock ofextract_xcom, thePodCommandExceptionpropagates and discards the already-retrieved XCom value, causing the task to be marked asFaileddespite the payload being successfully read.Fix
Wrap
extract_xcom_killin an inner try/except in thefinallyblock. Since the XCom value has already been successfully retrieved, a failure to clean up the sidecar is a non-fatal issue — the sidecar will be cleaned up when the pod is deleted (peron_finish_action). The exception is logged as a warning instead of failing the task.Test
Added
test_extract_xcom_kill_failure_does_not_fail_taskwhich verifies that whenextract_xcom_killraisesPodCommandException, theextract_xcommethod still returns the successfully-read XCom value.