Show each Dag only once in airflow dags list - #70934
Conversation
Airflow 3 stores one serialized_dag row per Dag version, so every Dag with more than one version was listed once per version.
90f4247 to
0ec3d95
Compare
henry3260
left a comment
There was a problem hiding this comment.
Well done!
While reviewing this I noticed DBDagBag.iter_all_latest_version_dags()
(airflow-core/src/airflow/models/dagbag.py) has the same issue — the name says
latest-version, but the body is an unfiltered select(SerializedDagModel). Same
origin as this one (#53918 dropped the read_all_dags() call in both places).
Out of scope here. would you mind fixing it in a follow-up PR?
|
Hi maintainer, this PR was merged without a milestone set.
|
Backport successfully created: v3-3-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
Airflow 3 stores one serialized_dag row per Dag version, so every Dag with more than one version was listed once per version. (cherry picked from commit b9ff1d7) Co-authored-by: Y-C <easoneason0905@gmail.com> Co-authored-by: Eason09053360 <185830721+Eason09053360@users.noreply.github.com>
Airflow 3 stores one serialized_dag row per Dag version, so every Dag with more than one version was listed once per version. (cherry picked from commit b9ff1d7) Co-authored-by: Y-C <easoneason0905@gmail.com> Co-authored-by: Eason09053360 <185830721+Eason09053360@users.noreply.github.com>
|
@henry3260 Thanks for reviewing ! |
Airflow 3 stores one serialized_dag row per Dag version, so every Dag with more than one version was listed once per version. Co-authored-by: Eason09053360 <185830721+Eason09053360@users.noreply.github.com>
Airflow 3 stores one
serialized_dagrow per Dag version, butairflow dags listselected every row without filtering by version, so a Dag was printed once per
version it had.
Use the existing
SerializedDagModel.read_all_dags()helper, which selects therow with the highest
version_numberperdag_id.Note that #60868 / #61077 fixed one source of spurious version growth, but
legitimate versions still accumulate whenever a Dag is edited, so the duplicate
rows remain reproducible.
closes: #60594
This supersedes #60637, which reached the same diagnosis but was closed by the
stale bot without review.
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.