Reduce invariant work in local test selection

[Trecek]

Tracking issue: #4956 — #4956

Suggested implementation order: 1 of 3 (soft sequence; see tracker).

Outcome

Make successful local path-filtered test runs spend less work deciding which tests to run, while preserving the current selected items, order, feature/size behavior, and fallback decisions for equivalent inputs.

This is the first recommended implementation slice in the local-test-performance tracking effort. It changes selector bookkeeping, not dependency policy or the number of tests considered relevant.

Investigation

A source investigation, deterministic selector replay, and two independent adversarial reviews of this recommendation established two repeated computations:

1. In the collection hook, every collected item loops over every scope path. sp.is_file() is inside that nested loop. File/type classification is repeated even though the collection snapshot and scope are unchanged.
2. apply_manifest constructs a PathSpec for every manifest pattern on each call. build_test_scope calls it once for each manifest-routed changed file.

At investigation snapshot 5f49d3761, the three-artifact replay (developer contribution document, documentation-count script, and config defaults YAML) made 309 pattern constructions: three calls × 103 patterns. Other replay scopes had 422–440 raw scope paths. These are structural/replay observations, not timing benchmarks. The relevant target files were unchanged through filing checkout 94e3d18e0.

The investigation did not run fresh timing benchmarks because other PR implementations were concurrently using the machine.

Scope and minimal approach

• Compile the invocation's manifest matchers once and reuse them for that invocation's changed-file matches.
• Classify scope paths once after collection, then use efficient membership/ancestor checks while iterating items.
• Retain existing modules/helpers and their contracts; a process-wide cache, registry, filesystem watcher, or new selector subsystem is unnecessary.
• Keep the same immutable input snapshot. Do not cache Git diffs, source-map decisions, or manifest state across runs.
• Preserve unioning of all matching manifest entries, pathspec/glob semantics, file-versus-non-file matching behavior, relative/absolute paths, and collected-item ordering.
• Keep feature gating and aggressive size filtering in their existing semantic order.

The optimization should remove repeated work, not quietly change behavior for missing paths or broaden/narrow a scope.

Coverage boundaries and exclusions

Do not include these independent changes in this ticket:

• pre-collection pruning;
• helper import-closure repair;
• source-map provenance or target-existence policy;
• the independent architectural-hook/full-fallback bug;
• narrowing the contributing.md route;
• worker-count defaults or pytest plugin changes.

Those need their own analysis/acceptance criteria. In particular, preserving today's root fallback is not a claim that the whole current filtering architecture is already sound.

Acceptance and validation

• For representative existing source/test/artifact inputs, the old and optimized selector/matcher produce identical final selected node IDs and order, deselections, and root fallback reason under the same configuration.
• Cover the actual affected edge shapes using existing tests where possible: overlapping directory/file scopes, relative paths, non-file/missing targets, multiple matching manifest rules, and a broad/unmapped fallback.
• Preserve conservative versus aggressive behavior and all always-run guard selections; do not alter marker/feature outcomes to obtain a faster result.
• Demonstrate that scope-path classification no longer grows with collected-item count times scope-path count for a fixed scope.
• Demonstrate that matcher construction grows with manifest size once per selector invocation, rather than once per changed artifact. Do not hard-code 103 in a test: it was the investigation snapshot's count.
• Preserve error/fallback behavior and normal repeated in-process pytest configurations.
• Run the repository's approved task-based checks and the relevant existing filter/collection tests. A multi-part implementation must independently pass task test-check at every part.
• Record controlled paired measurements when the host is suitable; do not present timings from unrelated concurrent PR runs as a clean baseline.

Relevant coverage includes filter plugin tests, core selector tests, tiered always-run tests, and manifest routing tests.

Estimated impact

High confidence in work reduction; wall-time impact unmeasured. This is a seconds-scale candidate, potentially larger for scopes containing hundreds of explicit paths. The filesystem-classification component can move from repeated item × scope checks to one classification per scope entry. Per-invocation matcher compilation changes the three-artifact example from 309 constructions to 103.

Do not promise a suite-wide percentage. Unfiltered and root full-fallback runs bypass the path-matching loop, so they receive little or no benefit from that portion.

Implementation order

Recommended first: it affects the repeated cost of many successful filtered runs and establishes a cleaner baseline for later test-body improvements. This is a soft sequence, not a technical dependency on the other implementation children.

Prior work / provenance

Related predecessor: #4815 (CI performance). This ticket does not repeat its completed production recipe caches or canonical-constant importer optimization. The current investigation artifacts were investigation_local_test_performance_2026-09-10.md and deep-scope-replay/replay-results.json; all information needed to implement this slice is reproduced here, with immutable source links.