Add deep-research provider triage for ecological mechanism/dataset research - #657
Merged
Conversation
…search Ports the fleet's generic deep-research provider-scoring implementation to CommunityMech, completing work the justfile already assumed: `deep-research-providers`/`deep-research-provider` recipes referenced scripts/deep_research_provider.py and conf/deep_research_provider.yaml, but neither file was ever committed, so both recipes were broken on main. research_community.py (the entity runner) was already in place and untouched. Two domain-specific focuses in conf/deep_research_provider.yaml: - ecological_mechanism — exact composition, directional interaction evidence between community members. - datasets_environment — repository-native accessions, ENVO context, cultivation and perturbation metadata. Bumps deep-research-client to 0.2.10, matching the version pinned across CultureMech/TraitMech/MediaIngredientMech/ProteinTraitsMech. Carries two fixes from review rounds on the sibling ports (proteintraitsmech#487, MediaIngredientMech#412): - provider_adjustments keys in the YAML profile are canonicalized and validated against known providers at config-load time, so a typo'd or aliased key no longer silently no-ops instead of applying its intended score bonus. - `--json --provider X` recomputes recommended_available/ fallback_available from the filtered ranking instead of leaving stale values computed from the unfiltered one. Also fixes a latent bug in tests/test_scripts_import.py (#410's import sweep, unrelated to deep-research otherwise): its subprocess probe built a module via importlib.util.module_from_spec() but never registered it in sys.modules before exec_module(). Any script defining a `from __future__ import annotations` frozen dataclass crashes there — dataclasses._is_type does `sys.modules.get(cls.__module__).__dict__` while checking for ClassVar/InitVar, and cls.__module__ ('_probe') was never actually stored in sys.modules, so .get() returns None. The real invocation path (`python scripts/foo.py`) doesn't hit this: running as __main__ registers sys.modules['__main__'] for free. Reproduced in isolation and via mutation test (reverting the one-line fix turns the probe red again on deep_research_provider.py specifically). Verified: full `uv run pytest tests/` — 2523 passed, 89 skipped, 8 deselected (0 failed). `black --check src/ tests/ scripts/`, `ruff check src/ tests/ scripts/`, `mypy src/` — all clean, matching .github/workflows/lint.yaml exactly. `uv sync --frozen --all-extras` — consistent. `scripts/check_vendored_sync.sh` — OK, unaffected. Mutation-tested provider_adjustments and --json filtering fixes independently; both turn their new tests red when reverted. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same generic file, same two gaps found there, fixed identically here before CommunityMech's own review round even landed: - Medium: provider_adjustments canonicalization didn't check for two raw keys resolving to the same provider (e.g. edison: 3 and falcon: 5 in the same focus) — the second silently overwrote the first. Now raises a clear ValueError. - Medium: main()'s CLI-level rejection of an unknown --provider/--focus argument had no test coverage. Verified: mutation test on the duplicate-key guard turns the new test red. Full `uv run pytest tests/` — 2540 passed (+3 from the new tests minus the earlier no-longer-relevant skip count changes), 75 skipped, 8 deselected, 0 failed. `black --check` / `ruff check` / `mypy src/` all clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ments guard, ambiguous issue refs Addresses the remaining findings from this PR's review (the gate failure it also reported — test_a_script_imports[deep_research_provider.py] under "Python 3.14" — does not reproduce: verified directly with the real Python 3.14.6 interpreter installed on this machine, both in isolation and via `uv run --python 3.14 pytest`, both pass. CI itself pins Python 3.10 (.github/workflows/validate-strict.yaml). The fix from this PR's first commit — registering the probe module in sys.modules before exec_module — already covers this; the review's static-gate step likely ran against a stale checkout, a known hazard documented for this fleet's review tooling): - Medium: stage capabilities keys weren't validated against the known capability set, unlike provider_adjustments. A typo'd capability key silently contributed 0 to every provider's score instead of erroring. - Medium: bare "#487"/"#412" review-attribution comments collided with this repo's own unrelated PR numbers (CommunityMech#487 and #412 are both real, different PRs) — qualified as "proteintraitsmech#487" and "mediaingredientmech#412". - Low: `high = max(raw.values()) or 1.0` only guarded an exact-zero max; a large negative provider_adjustments value pushes every score negative, leaving `high` negative too, clamping every fit to 0 and collapsing the ranking to alphabetical order. Guards the sign now, not just falsiness. Reproduced live with all-negative adjustments — fixed ranking keeps a real relative order instead of degenerating. High finding (missing KNOWN_BLOCKED provider check + not registered for vendored-sync) is real but out of scope for this PR — it requires porting CultureMech's evolved implementation and/or deciding the fleet's sharing model, which is what CultureMech#287/#290/#298 already track. Filed CommunityMech#658 to carry it forward rather than expanding this PR indefinitely. Verified: full `uv run pytest tests/` — 2540 passed, 75 skipped, 8 deselected, 0 failed. `black --check`/`ruff check`/`mypy src/` clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/deep_research_provider.py), completing work thejustfilealready assumed:deep-research-providers/deep-research-providerrecipes referenced this script andconf/deep_research_provider.yaml, but neither was ever committed — both recipes are currently broken onmain.research_community.py(the entity runner, plusresearch-community-edison/research-community-causal) was already in place and committed; this PR doesn't touch it.conf/deep_research_provider.yaml:ecological_mechanism— exact composition, directional interaction evidence between community members.datasets_environment— repository-native accessions, ENVO context, cultivation and perturbation metadata.deep-research-clientto0.2.10, matching the version pinned across CultureMech/TraitMech/MediaIngredientMech/ProteinTraitsMech.provider_adjustmentskeys are canonicalized and validated against known providers at config-load time.--json --provider Xrecomputesrecommended_available/fallback_availablefrom the filtered ranking instead of leaving stale values from the unfiltered one.tests/test_scripts_import.py(Six scripts import communitymech.literature_enhanced, which does not exist #410's import sweep — unrelated to deep-research otherwise): its subprocess probe built a module viaimportlib.util.module_from_spec()but never registered it insys.modulesbeforeexec_module(). Any script defining afrom __future__ import annotationsfrozen dataclass crashes there (dataclasses._is_typedoessys.modules.get(cls.__module__).__dict__while checking forClassVar/InitVar, andcls.__module__—'_probe'— was never actually stored insys.modules). The real invocation path (python scripts/foo.py) doesn't hit this, since running as__main__registerssys.modules['__main__']for free. One-line fix; reproduced in isolation and mutation-tested.Test plan
uv run pytest tests/— 2523 passed, 89 skipped, 8 deselected (0 failed)black --check src/ tests/ scripts/,ruff check src/ tests/ scripts/,mypy src/— all clean, matching.github/workflows/lint.yamlexactlyuv sync --frozen --all-extras— consistentscripts/check_vendored_sync.sh— OK, unaffectedprovider_adjustmentsfix, the--jsonfiltering fix, and thetest_scripts_import.pyprobe fix independently — each turns its corresponding test red when reverted🤖 Generated with Claude Code