Skip to content

✅ Fix flaky app_dir and launch tutorial tests under xdist - #1951

Open
YuriiMotov wants to merge 1 commit into
masterfrom
fix-flaky-test
Open

✅ Fix flaky app_dir and launch tutorial tests under xdist#1951
YuriiMotov wants to merge 1 commit into
masterfrom
fix-flaky-test

Conversation

@YuriiMotov

@YuriiMotov YuriiMotov commented Sep 2, 2026

Copy link
Copy Markdown
Member

Description

I noticed that test-redistribute failed on master (see logs), but passed again on later commits.

After investigating it with Claude Code we found a race between xdist workers. Three tests use the real ~/.config/my-super-cli-app/config.json:

  • test_cli_config_doesnt_exist expects the file to not exist
  • test_cli_config_exists creates it in a fixture
  • test_cli in test_launch/test_tutorial002.py creates it via the tutorial code

xdist spreads tests across workers, so these tests run in parallel from time to time and then may fail.

To fix this I added an app_dir fixture in tests/conftest.py. It patches typer.get_app_dir to return tmp_path, so each test gets its own empty dir.
As a bonus, tests no longer create files in the real ~/.config on developers' machines.

To reproduce the flakiness on master:

for i in $(seq 1 15); do pytest tests/test_tutorial/test_app_dir tests/test_tutorial/test_launch -n 4 -q -o addopts="" | grep failed; done

AI Disclaimer

Used Claude Code (Fable 5.1) to investigate and apply the fix. Reviewed manually

Checklist

  • This PR links to a GitHub Discussion for the proposed code change.
  • I added tests for the change.
  • The new or updated tests fail on the main branch and pass on this PR.
  • Coverage stays at 100%.
  • The documentation explains the change if needed.

@YuriiMotov YuriiMotov added the internal Internal changes label Sep 2, 2026
@YuriiMotov
YuriiMotov requested a review from svlandeg September 2, 2026 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal Internal changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants