feat(taint): consolidate cross-file engine into ast_taint_engine + deprecate semantic_engine (closes #49 phase-1) - #140
Merged
Conversation
…precate semantic_engine (closes #49 phase-1) Issue #49 Phase 1 — Taint analysis engine consolidation: 1. Audit & document all 4 taint/dataflow engines (docs/taint-engine-audit.md): - ast_taint_engine.py (3,755 LOC) — v1, AST-based, primary engine - crossfile_taint_engine.py (946 LOC) — v2, cross-file, now compat wrapper - dataflow_engine.py (1,097 LOC) — v3, source->sink, independent (not consolidated) - semantic_engine.py (428 LOC) — regex-based, deprecated 2. Consolidate crossfile_taint_engine into ast_taint_engine: - ast_taint_engine.analyze_workspace() now accepts cross_file=True parameter - ast_taint_engine._analyze_cross_file() method delegates to CrossFileTaintAnalyzer - crossfile_taint_engine.analyze_cross_file_taint() is now a thin compat wrapper that delegates to ast_taint_engine.analyze_workspace(cross_file=True) - Single unified entry point: ast_taint_engine.analyze_workspace() 3. Deprecate semantic_engine with warning: - DeprecationWarning emitted to stderr on first use (once per process) - Module docstring updated with deprecation notice and migration path - TaintAnalyzer class and analyze_workspace() both emit warning - Deprecation path: v8.3 warning -> v8.4 regex fallback in ast_taint -> v9.0 removal 4. Update taint command (scripts/commands/taint.py): - Routes all requests through ast_taint_engine.analyze_workspace() - --cross-file flag sets cross_file=True (no longer calls crossfile_taint_engine directly) - --no-ast flag falls back to semantic_engine (deprecated path) - Simplified engine selection logic Verified: cross_file=True produces identical results to the old compat wrapper. Test suite: 1422 passed, 0 failed (no regressions).
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
7 tasks
|
This was referenced Jul 1, 2026
Wolfvin
added a commit
that referenced
this pull request
Jul 3, 2026
Add a new semantic-query CLI command + codelens_semantic_query MCP tool that ranks symbols by cosine similarity to a natural-language query, using zero-dependency TF-IDF over symbol names + signatures + kinds + file paths. Implements Option A from issue #11 (Option B/C remain future work for embedding-model-based search). New files: - scripts/semantic_search_engine.py — TF-IDF engine: tokenize, build vocabulary + IDF, compute sparse doc vectors, rank by cosine sim. Cached per (db_path, mtime) so re-scans auto-invalidate the index. Named semantic_search_engine (not semantic_engine) to avoid colliding with the pre-existing deprecated taint-analysis semantic_engine.py from PR #140. - scripts/commands/semantic_query.py — CLI command registered as 'semantic-query'. Auto-exposed as codelens_semantic_query MCP tool via the dynamic tool discovery path. Also added an explicit static tool definition in _TOOL_DEFINITIONS so the JSON schema is documented. - tests/test_semantic_search_engine.py — 31 tests covering tokenizer, end-to-end query behavior, cache invalidation, graceful degradation, and TF-IDF ranking signal (rare-vs-common terms, multi-term queries). Modified files (auto-synced by sync_command_count.py --apply): - README.md, SKILL.md, SKILL-QUICK.md, pyproject.toml, skill.json, scripts/graph_model.py, scripts/mcp_server.py — command count 68->70, MCP tool count 66->68. Tests: 1401 passed, 80 skipped, 1 pre-existing failure unrelated (test_codelensignore::test_actual_target_dir_is_ignored — fails on clean HEAD too, environmental issue with the 'target' ignore pattern).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #49 (Phase 1 only — Consolidation)
What
Implements Phase 1 of issue #49: taint analysis engine consolidation.
docs/taint-engine-audit.mdcrossfile_taint_engine.pyintoast_taint_engine.py— single unified entry pointsemantic_engine.pywith warningChanges
ast_taint_engine.pyanalyze_workspace()now acceptscross_file=Trueparameter_analyze_cross_file()method delegates toCrossFileTaintAnalyzer(lazy import avoids circular dep)analyze_workspace()function also acceptscross_filecrossfile_taint_engine.pyanalyze_cross_file_taint()is now a thin compat wrapper that delegates toast_taint_engine.analyze_workspace(cross_file=True)CrossFileTaintAnalyzerclass and CFG/call-graph infrastructure remain as implementation backendsemantic_engine.py_emit_deprecation_warning()prints to stderr once per processTaintAnalyzer.__init__()andanalyze_workspace()both emit warningscripts/commands/taint.pyast_taint_engine.analyze_workspace()--cross-fileflag setscross_file=True(no longer calls crossfile_taint_engine directly)--no-astflag falls back to semantic_engine (deprecated path)docs/taint-engine-audit.md(new)Unified API
Verification
cross_file=Trueproduces identical results to the old compat wrappertest_semantic_engine.py(33 tests) andtest_dataflow_engine.py(9 tests) all passNot in this PR (future phases)
codelens debug-rule)Test plan
analyze_workspace(cross_file=True)matchesanalyze_cross_file_taint()output