Allow upgrades with NUL in serialized Dags - #71442
Open
dhkim1920 wants to merge 1 commit into
Open
Conversation
PostgreSQL rejects U+0000 while converting serialized Dag JSON to JSONB, blocking upgrades from Airflow 3.1.
dhkim1920
marked this pull request as ready for review
August 11, 2026 15:02
dhkim1920
marked this pull request as draft
August 11, 2026 15:03
dhkim1920
marked this pull request as ready for review
August 14, 2026 11:23
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PostgreSQL accepts U+0000 escapes in
JSON, but rejects them when migration 0089 castsserialized_dag.datatoJSONB. This blocks upgrades from Airflow 3.1 when a serialized Dag contains a NUL character.This draft applies the sanitization approach established by #69064 to migration 0089:
Decision needed
#69064 strips U+0000 from historical XCom and DagRun data because PostgreSQL JSONB cannot represent it. Applying the same policy to
serialized_dagis semantically different: a delimiter such as"@@\0@@"becomes"@@@@", changing the Dag definition.This draft asks whether lossy sanitization is acceptable for this migration or whether serialized Dag data needs a different policy. Migration-only sanitization may also not address a later reserialization of the same Dag into JSONB, so the final scope depends on that policy decision.
closes: #65379
Tests
breeze run --backend postgres pytest airflow-core/tests/unit/migrations/test_0089_change_serialized_dag_data_column_to_jsonb.py -xvsprek run --from-ref main --stage pre-commitprek run --from-ref main --stage manual --skip compile-ui-assets-dev --skip view-skill-evalbreeze testing core-tests --run-in-parallelWas generative AI tooling used to co-author this PR?
Generated-by: Codex (GPT-5.6) following the guidelines