[FEATURE] Native LSP server (`codelens lsp`) — consolidate 4 worker proposals (related to #33)

[Wolfvin]

Summary

Add a native LSP 3.17 server (codelens lsp) so CodeLens can push diagnostics to Neovim / Emacs / Helix / Zed / VS Code without requiring the AI agent to call MCP tools explicitly. Currently CodeLens has only lsp_client.py (a client for external LSPs used by --deep).

Worker consensus (4 reports)

Worker	Source	Contribution
Opengrep	update!/CodeLens_Opengrep_Upgrade_Analysis.md #39	LSP 3.17 over stdio: textDocument/publishDiagnostics, textDocument/codeAction (suggest autofix), textDocument/hover (rule doc + taint path), file operation filters. Use pylsp or pygls.
Semgrep	update!/CodeLens_Upgrade_Issues_from_Semgrep.md CL-008	LSP server native: initialize, didOpen, didChange, hover, definition, references, codeLens, codeAction + custom codelens/query, codelens/impact, codelens/guard. Target <50ms hover/definition.
Serena	update!/CodeLens_vs_Serena_Upgrade_Analysis.md S1	Vendor Serena's solidlsp for LSP client integration with 50+ language servers (Pyright, tsserver, gopls, rust-analyzer, clangd, jdtls, OmniSharp). 40+ languages at IDE-level accuracy.
Serena	same file S13	LSP diagnostics: codelens diagnostics [workspace] [--file FILE] [--severity ...] using textDocument/publishDiagnostics. Integrate with check and smell.

Proposed scope (3 phases)

Phase 1 — Native LSP server (P1, 3-4 weeks)

• codelens lsp command, stdio transport
• Use pygls framework (do NOT reimplement JSON-RPC from scratch)
• Methods: initialize, didOpen, didChange, hover, definition, references, codeAction
• Custom methods: codelens/query, codelens/impact, codelens/guard
• Target: <50ms for hover/definition
• New files: scripts/lsp_server.py, scripts/lsp_handlers/

Phase 2 — Vendor solidlsp for multi-language LSP client (P2, 3-4 weeks, depends on Phase 1)

• Vendor Serena's solidlsp/ module (or git submodule)
• Auto-detect language, lazy-start language servers
• Cache at .codelens/lsp_cache/, file watcher for reindex
• Fallback to tree-sitter if LSP unavailable
• Replaces the existing lsp_client.py shim used by --deep

Phase 3 — Diagnostics integration (P2, 1-2 weeks, depends on Phase 1)

• codelens diagnostics command
• Integrate LSP publishDiagnostics into check and smell output
• VS Code extension: push LSP diagnostics to Problems panel

Acceptance criteria

• Phase 1: codelens lsp works with Neovim (lsp-mode), VS Code, Helix
• Phase 1: hover/definition <50ms p95 on 1000-file repo
• Phase 2: solidlsp vendored, 6+ languages with cross-file reference finding
• Phase 3: diagnostics surface in editor Problems panel on file save

Relationship to #33

#33 is the dual-entry-point bug for --lsp-status. That bug should be fixed independently (~1 hour) regardless of this feature. This issue is the larger feature that #33 was a symptom of.

License note

Serena is MIT-licensed — solidlsp can be vendored. Semgrep is LGPL-2.1 — reference only, no verbatim copy.