fix(arrow-json): validate ListView child nullability - #10486
Conversation
|
run benchmark json_reader |
|
🤖 Arrow criterion benchmark running (GKE) | trigger CPU Details (lscpu)Comparing fix/10476-list-view-non-nullable-child (b0ba059) to 005a2fd (merge-base) diff File an issue against this benchmark runner |
|
🤖 Arrow criterion benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usagebase (merge-base)
branch
File an issue against this benchmark runner |
|
thanks @dk3yyyy |
# Which issue does this PR close? - Closes apache#10476. # Rationale for this change The JSON reader constructs `ListView` arrays without validating child nullability. As a result, JSON containing `null` can be accepted for a non-nullable child field. # What changes are included in this PR? - Replace unchecked `ListView` construction with `GenericListViewArray::try_new`, returning an `ArrowError` when child nullability is invalid. - Add regression coverage for both `ListView` and `LargeListView`. # Are these changes tested? Yes: - `cargo test -p arrow-json reader::tests::test_read_list_view_rejects_null_non_nullable_child -- --exact` - `cargo test -p arrow-json --all-features` - `cargo clippy --workspace --all-targets --all-features -- -D warnings` - `cargo fmt --all -- --check` - `git diff --check` # Are there any user-facing changes? Yes. Invalid JSON containing a null inside a non-nullable `ListView` or `LargeListView` child now returns an error. There is no public API change.
Which issue does this PR close?
Rationale for this change
The JSON reader constructs
ListViewarrays without validating child nullability. As a result, JSON containingnullcan be accepted for a non-nullable child field.What changes are included in this PR?
ListViewconstruction withGenericListViewArray::try_new, returning anArrowErrorwhen child nullability is invalid.ListViewandLargeListView.Are these changes tested?
Yes:
cargo test -p arrow-json reader::tests::test_read_list_view_rejects_null_non_nullable_child -- --exactcargo test -p arrow-json --all-featurescargo clippy --workspace --all-targets --all-features -- -D warningscargo fmt --all -- --checkgit diff --checkAre there any user-facing changes?
Yes. Invalid JSON containing a null inside a non-nullable
ListVieworLargeListViewchild now returns an error. There is no public API change.