Skip to content

docs: correct the contributor test command - #155

Open
Hasnain2430 wants to merge 1 commit into
grayhatdevelopers:mainfrom
Hasnain2430:docs/contributor-test-command
Open

docs: correct the contributor test command#155
Hasnain2430 wants to merge 1 commit into
grayhatdevelopers:mainfrom
Hasnain2430:docs/contributor-test-command

Conversation

@Hasnain2430

Copy link
Copy Markdown

Related issue

None.

Summary

The contributor setup and validation steps in docs/CONTRIBUTING.md, and the coding-agent instructions in AGENTS.md, told readers to run uv run --no-sync pytest -q. From a clean checkout that fails during collection, because five root-level tooling tests import the top-level utils package:

ERROR tests/test_ci_scope.py
ERROR tests/test_normalize_sdist.py
ERROR tests/test_published_distribution.py
ERROR tests/test_release_contract.py
ERROR tests/test_release_notes.py
ModuleNotFoundError: No module named 'utils'
!!!! Interrupted: 5 errors during collection !!!!

The repository has no [tool.pytest.ini_options] section and no conftest.py, so nothing places the repository root on the import path. python -m pytest does, and collects the suite cleanly.

CI does not surface this because it runs python -m unittest discover -s tests (.github/workflows/ci.yml), which prepends the working directory for the same reason. docs/benchmarking/runtime_validation.md already records that the automated suite depends on import-path selection.

This is the first command a new contributor runs, so the failure reads as a broken checkout. The change uses python -m pytest in both validation steps and in AGENTS.md, and records why the invocation matters so it is not shortened again.

Documentation only. No product behavior changes. An alternative fix is to add pytest configuration or a root conftest.py so the bare command works; I kept this to the documentation because adding pytest configuration to a repository whose CI standardizes on unittest is a maintainer decision, and I am happy to send that instead if you prefer it.

Validation

  • uv run --no-sync python -m pytest -q --collect-only — 740 tests collected, no errors.
  • uv run --no-sync pytest -q — reproduces the 5 collection errors quoted above, confirming the documented command is the failing one.
  • npx --yes markdownlint-cli2@0.23.2 "AGENTS.md" "docs/CONTRIBUTING.md" — exit 0, no findings.
  • Local link check not run: lychee is not installed here and the change adds no links. Please let the documentation workflow cover it.

For transparency, the corrected command does not produce a clean suite on Windows today: uv run --no-sync python -m pytest -q on upstream/main reports 4 failed, 731 passed, 5 skipped. Two of those are Windows path-resolution assertions I have sent separately; the other two are test_native_ingestion.py autonomous-indexing failures I have not diagnosed. This change only corrects which command to run, and does not claim the suite is green.

One related observation, not changed here: CI's python -m unittest discover -s tests collects 655 tests, while python -m pytest collects 740. The difference is the module-level pytest functions in test_codex_plugin.py and test_native_ingestion.py plus the five root-level tooling tests, none of which unittest discovery collects. So the documented command and the CI command exercise different sets.

The contributor setup and validation steps told readers to run
`uv run --no-sync pytest -q`. From a clean checkout that fails during
collection, because five root-level tooling tests import the top-level
`utils` package:

    ERROR tests/test_ci_scope.py
    ERROR tests/test_normalize_sdist.py
    ERROR tests/test_published_distribution.py
    ERROR tests/test_release_contract.py
    ERROR tests/test_release_notes.py
    ModuleNotFoundError: No module named 'utils'
    !!!! Interrupted: 5 errors during collection !!!!

The repository has no pytest configuration and no `conftest.py`, so
nothing places the repository root on the import path. `python -m pytest`
does, and collects the suite cleanly. CI does not catch this because it
runs `python -m unittest discover -s tests`, which prepends the working
directory for the same reason.

Use `python -m pytest` in both validation steps and in the coding-agent
instructions, and record why the invocation matters so it is not
shortened again. `docs/benchmarking/runtime_validation.md` already notes
that the automated suite depends on import-path selection.

Documentation only. Verified with `python -m pytest -q --collect-only`
(740 tests collected, no errors) and `markdownlint-cli2@0.23.2` on both
changed files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant