Skip to content

perf: reduce acceptance-suite cold-start cost (258 nested ./bashunit runs) #798

Description

@Chemaclass

perf: reduce acceptance-suite cold-start cost (258 nested ./bashunit runs)

Follow-up to #761 (rounds 1+2 complete). The per-test hot path is now largely fork-free; the dominant remaining wall-clock cost is elsewhere.

Problem

The acceptance suite spawns 258 nested ./bashunit subprocess runs. Each pays a full cold start (sourcing all of src/ + env resolution). At ~52s for the full --parallel suite, these cold starts — not per-test overhead — dominate.

Candidate directions (each needs profiling first)

  • Trim cold-start cost — sourcing + env resolution per invocation. Sourcing all of src/ measures only ~28ms, so this alone caps at ~5-8s across 258 runs.
  • Share one built artifact across nested runs instead of re-sourcing each time.
  • Reclassify tests — convert acceptance tests that don't exercise the CLI surface into functional tests (no nested process needed).

Measure before filing sub-work

Profile a representative nested run to attribute cold-start cost (source vs env vs fork) before committing to a direction. Report numbers, then pick.

Also considered, NOT pursued (from #761)

Lazy-sourcing large optional modules (coverage.sh, learn.sh): sourcing all of src/ is only ~28ms, so the win is ~5-8s across 258 runs and it complicates the single-file release build. Revisit only inside the cold-start work above.

Constraints

  • Bash 3.0+ compatible (target machine is bash 3.2 on macOS): no declare -A, [[ ]], ${var,,}, negative array indexing, &>>.
  • TDD; quality gate (./bashunit tests/, ./bashunit --parallel tests/, make sa, make lint, shfmt -w .) on every PR.
  • One issue per PR.

Benchmark protocol

mkdir -p /tmp/bashunit-bench
for i in $(seq 1 100); do echo "function test_noop_$i() { assert_true true; }"; done > /tmp/bashunit-bench/noop100_test.sh
time ./bashunit --no-parallel /tmp/bashunit-bench/noop100_test.sh

# fork census
PS4='+X ${BASH_SOURCE##*/}:$LINENO ' bash -x ./bashunit --no-parallel /tmp/bashunit-bench/noop100_test.sh \
  2>/tmp/bashunit-bench/trace.txt >/dev/null
grep -oE ' (perl|base64|grep|sed|mktemp|rm|date) ' /tmp/bashunit-bench/trace.txt | sort | uniq -c

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions