Add INFO-level logging to asset scheduling path#63958
Merged
kaxil merged 2 commits intoapache:mainfrom Mar 21, 2026
Merged
Conversation
boring-cyborg
Bot
added
the
area:Scheduler
Lee-W
approved these changes
Mar 20, 2026
The scheduler's asset (dataset) condition evaluation path had zero INFO-level logging, making it impossible to debug why asset-triggered DagRuns were created or skipped. This was a significant gap discovered during a P1 customer incident where the root cause took hours to trace because the scheduler was a complete black box for asset scheduling. Add log lines for: - ADRQ records loaded per DAG (count and DAG IDs) - Asset condition evaluation result (met/not met per DAG) - DAGs deferred due to max_active_runs - Asset-triggered DagRun creation (DAG ID, triggered_date, queued count) - Consumed asset events count per DagRun - ADRQ rows deleted after DagRun creation
- Change "DAGs" to "Dags" in asset-triggered log messages (dag.py) - Fix mypy attr-defined error by casting session.execute() result to CursorResult, matching the existing pattern in delete_dag.py
kaxil
force-pushed
the
add-dataset-scheduling-logging
branch
from
d5581f3 to
2d3f86b
Compare
fat-catTW
pushed a commit
to fat-catTW/airflow
that referenced
this pull request
Mar 22, 2026
The scheduler's asset (dataset) condition evaluation path had zero INFO-level logging, making it impossible to debug why asset-triggered DagRuns were created or skipped. This was a significant gap discovered during a P1 customer incident where the root cause took hours to trace because the scheduler was a complete black box for asset scheduling. Add log lines for: - ADRQ records loaded per DAG (count and DAG IDs) - Asset condition evaluation result (met/not met per DAG) - DAGs deferred due to max_active_runs - Asset-triggered DagRun creation (DAG ID, triggered_date, queued count) - Consumed asset events count per DagRun - ADRQ rows deleted after DagRun creation * Address review feedback: fix casing in log messages and mypy error - Change "DAGs" to "Dags" in asset-triggered log messages (dag.py) - Fix mypy attr-defined error by casting session.execute() result to CursorResult, matching the existing pattern in delete_dag.py
Suraj-kumar00
pushed a commit
to Suraj-kumar00/airflow
that referenced
this pull request
Apr 7, 2026
The scheduler's asset (dataset) condition evaluation path had zero INFO-level logging, making it impossible to debug why asset-triggered DagRuns were created or skipped. This was a significant gap discovered during a P1 customer incident where the root cause took hours to trace because the scheduler was a complete black box for asset scheduling. Add log lines for: - ADRQ records loaded per DAG (count and DAG IDs) - Asset condition evaluation result (met/not met per DAG) - DAGs deferred due to max_active_runs - Asset-triggered DagRun creation (DAG ID, triggered_date, queued count) - Consumed asset events count per DagRun - ADRQ rows deleted after DagRun creation * Address review feedback: fix casing in log messages and mypy error - Change "DAGs" to "Dags" in asset-triggered log messages (dag.py) - Fix mypy attr-defined error by casting session.execute() result to CursorResult, matching the existing pattern in delete_dag.py
abhijeets25012-tech
pushed a commit
to abhijeets25012-tech/airflow
that referenced
this pull request
Apr 9, 2026
The scheduler's asset (dataset) condition evaluation path had zero INFO-level logging, making it impossible to debug why asset-triggered DagRuns were created or skipped. This was a significant gap discovered during a P1 customer incident where the root cause took hours to trace because the scheduler was a complete black box for asset scheduling. Add log lines for: - ADRQ records loaded per DAG (count and DAG IDs) - Asset condition evaluation result (met/not met per DAG) - DAGs deferred due to max_active_runs - Asset-triggered DagRun creation (DAG ID, triggered_date, queued count) - Consumed asset events count per DagRun - ADRQ rows deleted after DagRun creation * Address review feedback: fix casing in log messages and mypy error - Change "DAGs" to "Dags" in asset-triggered log messages (dag.py) - Fix mypy attr-defined error by casting session.execute() result to CursorResult, matching the existing pattern in delete_dag.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The scheduler's asset (dataset) condition evaluation path had zero INFO-level logging, making it impossible to debug why asset-triggered DagRuns were created or skipped. This was a significant gap discovered during a debugging session where the root cause took hours to trace because the scheduler was a complete black box for asset scheduling.
Add log lines for:
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.