I was debugging a flaky test. It ended up being due to CPU load and a timeout, but my first thought was it had something to do with the order of tests that were run on a specific process. The ci-queue gem has some behavior that prints what tests were run, I wanted something similar and hacked it like this in GHA:
- name: Integration tests
run: |
mkdir -p tmp/ci-results
set +e # Temporarily disable exit on error
bundle exec parallel_split_test --no-merge --format d --out tmp/ci-results/process spec/
result=$?
set -e # Re-enable exit on error
echo "Capture Result"
echo "Result: $result"
find tmp/ci-results -name "process.*" | sort -V | while read file; do
echo "=== File: $file ==="
cat "$file"
echo
done
exit $resultThis prints the test name and so you get an output like:
5-07-15T00:56:08.7038806Z === File: tmp/ci-results/process.0 ===
2025-07-15T00:56:08.7049351Z
2025-07-15T00:56:08.7049513Z Bugs
2025-07-15T00:56:08.7049850Z database connections
2025-07-15T00:56:08.7050106Z fails with better error message
2025-07-15T00:56:08.7050324Z
2025-07-15T00:56:08.7056434Z LanguagePack::Helpers::FsExtra::RsyncDiff
2025-07-15T00:56:08.7056849Z detects when directories are different (added file in one directory)
2025-07-15T00:56:08.7057127Z
2025-07-15T00:56:08.7057290Z Finished in 1 minute 25.7 seconds (files took 9.65 seconds to load)
2025-07-15T00:56:08.7057631Z 2 examples, 0 failures
Or in the event of a failing test:
25-07-15T00:56:08.7431408Z === File: tmp/ci-results/process.37 ===
2025-07-15T00:56:08.7439600Z
2025-07-15T00:56:08.7439770Z WEB_CONCURRENCY.sh
2025-07-15T00:56:08.7440331Z from a preceding buildpack is overwritten by this buildpack
2025-07-15T00:56:08.7440724Z
2025-07-15T00:56:08.7440923Z Rails Runner
2025-07-15T00:56:08.7441494Z config objects build properly formatted commands (FAILED - 1)
2025-07-15T00:56:08.7441972Z
2025-07-15T00:56:08.7442331Z Failures:
2025-07-15T00:56:08.7442521Z
2025-07-15T00:56:08.7442833Z 1) Rails Runner config objects build properly formatted commands
2025-07-15T00:56:08.7443442Z Failure/Error:
2025-07-15T00:56:08.7443756Z Hatchet::App.new("default_ruby").in_directory_fork do
2025-07-15T00:56:08.7444119Z original_path = ENV["PATH"]
2025-07-15T00:56:08.7444431Z ENV["PATH"] = "./bin/:#{ENV['PATH']}"
2025-07-15T00:56:08.7444730Z
2025-07-15T00:56:08.7444948Z Dir.mktmpdir do |tmpdir|
The ordering was a red-herring for my debugging needs, but I imagine this will come up again. I wanted to leave a note where it was likely to be found next time I (or someone else) needs it.
One unexpected item: It seems that the order of commands matters and flags need to not use the = like --out=name. The README hints at this ability, but I had to dive into the source to figure out how to use it.
I was debugging a flaky test. It ended up being due to CPU load and a timeout, but my first thought was it had something to do with the order of tests that were run on a specific process. The ci-queue gem has some behavior that prints what tests were run, I wanted something similar and hacked it like this in GHA:
This prints the test name and so you get an output like:
Or in the event of a failing test:
The ordering was a red-herring for my debugging needs, but I imagine this will come up again. I wanted to leave a note where it was likely to be found next time I (or someone else) needs it.
One unexpected item: It seems that the order of commands matters and flags need to not use the
=like--out=name. The README hints at this ability, but I had to dive into the source to figure out how to use it.