Skip to content

Fix KubernetesExecutor retry losing hostname#67372

Open
kalluripradeep wants to merge 1 commit into
apache:mainfrom
kalluripradeep:fix-ti-history-retry-metadata
Open

Fix KubernetesExecutor retry losing hostname#67372
kalluripradeep wants to merge 1 commit into
apache:mainfrom
kalluripradeep:fix-ti-history-retry-metadata

Conversation

@kalluripradeep
Copy link
Copy Markdown
Contributor

@kalluripradeep kalluripradeep commented May 23, 2026

closes: #65366
closes: #67238

What happened?

When a task instance using KubernetesExecutor (or CeleryExecutor) fails from a non-RUNNING state (e.g. QUEUED, DEFERRED, or killed externally), the task attempt is retried, but the TaskInstanceHistory record is not properly saved because the history logic incorrectly assumed only RUNNING tasks need their execution history recorded. This resulted in the failure attempt losing its hostname and execution metadata, leading to the http://:8793 No host supplied error when retrieving logs in the UI.

What changed?

  • Updated TaskInstanceHistory creation to force the state to FAILED and populate end_date and duration if the task fails from a non-finished state.
  • Updated task_instances.py Execution API logic to correctly update the task instance payload (including end_date and rendered_map_index) when transitioning to a retry state before prepare_db_for_next_try clears the metadata.
  • Added unit tests to ensure that TaskInstanceHistory correctly captures the hostname and other attempt metadata when a task fails and retries.

@boring-cyborg boring-cyborg Bot added area:API Airflow's REST/HTTP API area:Scheduler including HA (high availability) scheduler area:task-sdk labels May 23, 2026
kalluripradeep added a commit to kalluripradeep/airflow that referenced this pull request May 23, 2026
@kalluripradeep kalluripradeep force-pushed the fix-ti-history-retry-metadata branch 2 times, most recently from 82ab137 to f956372 Compare May 23, 2026 13:27
@kalluripradeep kalluripradeep force-pushed the fix-ti-history-retry-metadata branch from 7d3f9d0 to 190c0b6 Compare May 23, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API area:Scheduler including HA (high availability) scheduler area:task-sdk

Projects

None yet

1 participant