fix: prevent IVF-RQ merge stalls and speed up priority tracking

[jackye1995]

Large IVF-RQ merge backlogs repeatedly shift sorted priority arrays. Use a shared counted BTreeMap in both I/O schedulers to reduce priority-accounting overhead while preserving duplicate reservations and minimum-priority admission.

Fix a read-backpressure deadlock by loading each scan line's pages concurrently and accepting them in their original order. Add regression coverage for concurrent loading, output order, and priority accounting.

Git history traces the affected implementations to:

• Priority-accounting overhead: #2721 (30b3df75b, August 2024) introduced sorted-vector insertion/removal; #5773 copied it into the lite scheduler. The original assumption of a small backlog breaks down with many completed but unconsumed reads.
• Page-loading deadlock: #2968 (b1abfff9b, October 2024) introduced sequential scan-line page awaits. Later pages can hold the read budget needed by the page being awaited.

The two-file shuffle path originated in #6169. #8894 added offset preloading and coalesced reads, but this 1B workload exceeds the 256 MiB offset-preload cap and takes the per-partition fallback. These are confirmed code origins and the observed triggering conditions; a historical first-failing release has not been established by bisect.

Modern distributed IVF-PQ uses the same shuffle and merge implementation. RQ's additional factor columns and code width change page layout and flush-group count, which can change whether the offset-preload cap is exceeded. The earlier successful PQ workload has not been matched against this run, so an RQ-specific regression has not been established.

A complete merge of 1 billion vectors across 31,623 partitions finished in 1h 02m 44s, including output finalization, with the standard scheduler and a warm cache. The full output audit passed exact unique row-ID and per-partition row-count checks, plus per-partition code/factor checksum checks.

The decoder regression times out before the fix and passes afterward. All 15 decoder tests, 31 scheduler tests, workspace Rust clippy, formatting, and 37 CI checks passed.

[lance-gatekeeper]

✅ Gate recommendation: approve.

The counted ordered map preserves duplicate reservations, minimum-priority admission, and total outstanding counts while eliminating sorted-array shifts on every update. A single shared tracker is preferable to separate or hybrid paths because it addresses the large-backlog cost without adding transition state.

[lance-gatekeeper]

✅ Gate recommendation: approve.

The counted ordered map preserves duplicate reservations and minimum-priority admission while eliminating sorted-array shifts. Polling each scan line’s page futures together breaks the I/O backpressure dependency cycle, and try_join_all retains decoder order; the focused regression covers both progress and output ordering.