Skip to content

ci: shard Rust coverage tests - #8079

Merged
Xuanwo merged 3 commits into
mainfrom
xuanwo/ci-shard-rust-coverage
Jul 30, 2026
Merged

ci: shard Rust coverage tests#8079
Xuanwo merged 3 commits into
mainfrom
xuanwo/ci-shard-rust-coverage

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Rust PR CI's warm-cache Linux coverage job has a 34:04 critical path. In the reference run, the Run tests step took 32:42: compilation accounted for about 3:51, while serial test-binary execution consumed roughly 29 minutes, led by lance at about 18 minutes and lance-encoding at about 7 minutes.

This change builds the canonical ci-profile coverage binaries once, fans the archived tests out across three Linux runners, merges each shard's coverage profile before transfer, and produces the existing single Codecov report in the final linux-build check. It preserves the coverage optimization level and required check name while moving serial test execution off the critical path.

Benchmark

  • Successful warm reference: 34:04.
  • Sharded cold run: 24:46, at least 31.6% faster than the same-SHA control's 36:14 lower bound.
  • Final-head warm run: 20:49, 38.9% faster than the successful warm reference. The stages were 5:02 to build and upload once, 13:10 for the slowest shard, and 2:27 to merge, report, and upload.
  • Three warm sharded runs completed in 20:49, 22:03, and 22:39. The median is 22:03, a 35.3% reduction.

The same-SHA serial controls reached the final lance-linalg test binary at 36:14 and 37:31, then hit the same two existing randomized f16 property-test failures. The successful historical warm run is therefore the primary speed baseline; the same-SHA controls provide conservative lower bounds rather than successful end-to-end samples.

Coverage was compared from the same instrumented archive with three shards versus one unsharded nextest job. Both reports contain exactly 531 files, 334,985 line mappings, and 655,891 segment denominators, with zero missing or changed denominators. The sharded report covered 11 more segments (89.2064% versus 89.2048%, +0.0017 percentage points), smaller than the observed 44-segment variation between two sharded runs.

Aggregate final-head work was 30:54 on 8x runners plus 2:27 on a standard runner, versus 34:04 on one 8x runner for the reference. The speedup comes from parallelism without increasing aggregate large-runner time. The main tradeoff is a 1.49 GB test archive and roughly 1–2 minutes of artifact transfer per stage; artifacts are retained for one day.

Evidence: successful warm reference, cold and warm sharded runs, same-archive coverage validation, and successful final-head workflow.

@github-actions github-actions Bot added ci Github Action or Test issues A-ci CI / build workflows and removed ci Github Action or Test issues labels Jul 29, 2026
@codecov

codecov Bot commented Jul 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions github-actions Bot added the ci Github Action or Test issues label Jul 29, 2026
@Xuanwo
Xuanwo marked this pull request as ready for review July 29, 2026 20:40

@geruh geruh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just one small nit!

cargo +nightly-2026-07-13 llvm-cov nextest \
--no-report \
--archive-file coverage-artifacts/rust-coverage-tests.tar.zst \
--partition slice:${{ matrix.shard }}/3

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Xuanwo
Xuanwo merged commit 5ec11ca into main Jul 30, 2026
29 checks passed
@Xuanwo
Xuanwo deleted the xuanwo/ci-shard-rust-coverage branch July 30, 2026 05:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ci CI / build workflows ci Github Action or Test issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants