[core][flink][spark] Support ARRAY<BLOB> blob files

[leaves12138]

Purpose

Support storing top-level ARRAY columns in dedicated blob files, so one table can use blob file storage for both BLOB and ARRAY fields across core, Flink, and Spark paths.

Changes

• Treat BLOB and ARRAY as blob-file fields in schema validation, data-evolution planning, blob file context, and column directive cleanup.
• Extend BlobFileFormat reader/writer with an ARRAY payload layout that keeps one blob-file record per table row and stores per-element lengths in a compact tail index.
• Preserve support for null arrays, null elements, empty arrays, selection reads, descriptor reads, inline reads, and whole-field placeholders.
• Add Flink catalog conversion and Flink array read conversion so ARRAY blob fields round-trip as ARRAY internally.
• Add Spark catalog conversion and Spark array/data converters so ARRAY blob fields round-trip as ARRAY internally.
• Add format-level, table-level, Flink e2e, and Spark e2e coverage for ARRAY.

[JingsongLi]

Let's add Python together.

[JingsongLi]

1. Inline on schema_manager.py
   [Major] Reject ARRAY partition keys during schema validation
   _validate_blob_fields validates primary keys but does not receive or check partition_keys. As a result, a table partitioned by an ARRAY column can be created successfully, but its first write fails while grouping partitions with TypeError: unhashable type: 'list'.
   Please pass the partition keys into this validation and reject BLOB/ARRAY partition columns during both create_table and commit, consistent with the Java validation.
2. Overall review comment because daft_datasource.py is unchanged
   [Major] ARRAY tables are incorrectly routed to Daft’s native Parquet reader
   Daft only recognizes top-level large_binary fields as blob columns. Since ARRAY is represented as list<large_binary>, _has_blob_columns remains false. An append-only Parquet table is therefore routed to the native reader, which attempts to read the table’s .blob file as Parquet and fails.
   Please detect dedicated blob storage from the Paimon schema and route ARRAY through the Pypaimon reader. If ARRAY conversion is not supported in Daft yet, it should be rejected explicitly instead of scheduling .blob files as Parquet.
3. Inline on format_blob_reader.py
   [Major] ARRAY bypasses blob_parallelism
   Each non-null array element is read using a separate synchronous seek/read operation. Unlike scalar BLOB values, array elements never enter the existing blobs_to_resolve / read_blobs_concurrent path, so blob_parallelism has no effect.
   On object stores, this makes request count and latency scale with the total number of array elements. Please batch the element ranges through the existing concurrent/coalesced resolver and rebuild the arrays afterward. For the serial path, the contiguous element payload could also be read once and sliced in memory.
4. Inline on row_utils.py
   [Minor] Avoid recursively scanning every nested non-BLOB value
   _contains_blob_value now recursively traverses every list, tuple, and dictionary, including ordinary ARRAY/MAP fields. In DedicatedFormatWriter.write_row, normal nested values are checked here and then checked again through row_values_to_arrow_table, causing two full Python traversals before Arrow conversion.
   Please make this validation schema-aware so known non-BLOB fields can bypass the recursive scan, or at least remove the duplicate check in the dedicated row-writing path.

[leaves12138]

Thanks for the thorough review and great catches! I've fixed all four issues and added regression tests.

[JingsongLi]

+1