[M10 correctness] Make diagnostics source-aware and dependency-reactive

[Teakowa]

Parent: #27

Goal

Make diagnostics preserve the same source/project identity guarantees as navigation, and ensure changes to included/open-overlay sources refresh diagnostics for affected documents instead of leaving stale editor state.

Context

M10 remediation made definition/references source-aware and added open-document overlays, but diagnostics are still range-only from the requesting document perspective. A diagnostic originating from an included source can therefore lose its source identity, and didChange currently republishes only the changed URI rather than dependent root/open documents.

This issue should close the source → diagnostic → dependency-refresh contract without introducing a speculative incremental compiler.

Scope

• Introduce an editor-neutral source-aware diagnostic representation (for example SourceDiagnostic) carrying:
  • source/document identity;
  • precise range;
  • severity/code/message;
  • relevant version/provenance information.
• Convert compiler/analyzer spans using the text of the span's actual source, not the requesting document.
• Reuse Wright's project/include/source model to determine which open/root documents depend on a changed include or overlay.
• When an included/open-overlay source changes, recompute and republish diagnostics for affected documents.
• Define and test didClose diagnostic cleanup so previously published diagnostics do not remain visible for a closed document.
• Keep deterministic full recomputation acceptable while current measurements justify it.

Non-goals

• Building a general incremental query engine or complex dependency cache.
• Adding new .opy language features.
• Expanding LSP feature breadth beyond diagnostics/project correctness.

Acceptance criteria

• Diagnostics originating in included files preserve the correct source identity and source-local UTF-16 range.
• Syntax and semantic/static-analysis diagnostics do not map included-file spans through the requesting document text.
• Changing an included open document automatically refreshes diagnostics for affected dependent documents at the LSP boundary.
• A valid → invalid → valid unsaved include edit produces corresponding dependent diagnostic refreshes without reopening the root document.
• Closing a document clears or otherwise correctly retires diagnostics previously published for that document.
• Filesystem-backed includes and open unsaved overlays have regression coverage.
• The implementation reuses Wright-owned project/source semantics and does not create a diagnostics-only project model.

Planning notes

This should be implemented and independently audited before project-wide rename work proceeds. Correct source ownership here is a prerequisite for later multi-document edits.

[Teakowa]

Reopened after post-gate correctness audit

The source-aware diagnostic model itself is now substantially correct, but one LSP ownership/retirement case remains incomplete.

Remaining failed behavior

publish_diagnostics() groups and publishes only diagnostic sources present in the current analysis, plus an always-present entry for the requesting/root URI. It does not retain which included-source URIs received diagnostics from the previous analysis of that root.

Therefore this sequence can leave stale editor diagnostics:

1. main.opy includes shared.opy.
2. shared.opy produces a diagnostic, so Wright publishes diagnostics under shared.opy's URI.
3. main.opy is changed so it no longer includes shared.opy (or otherwise no longer owns diagnostics from that source).
4. Reanalysis of main.opy no longer returns shared.opy in the current diagnostic groups.
5. No empty publishDiagnostics is sent for the previously published shared.opy URI, so the client can retain the obsolete diagnostic.

This is distinct from didClose cleanup of the requesting document and from recomputing a still-included overlay.

Required closure

• Track diagnostic publication ownership/state sufficiently to know which source URIs were published by each root/project analysis.
• On refresh, publish empty diagnostics for previously-owned source URIs that disappear from the new result set.
• Keep behavior deterministic when multiple open roots include the same source: do not clear diagnostics still owned/reported by another current root analysis.
• Add protocol-level regression coverage for at least:
  • included file has error → root removes include → included-source diagnostics are retired;
  • included overlay invalid → valid → diagnostics are retired;
  • shared include used by two roots → removing it from one root does not incorrectly clear diagnostics still produced through the other root.

Do not introduce a general incremental engine. This is a bounded LSP diagnostic ownership/retirement fix. #72 should remain open until the above is verified.

[Teakowa]

Implementation + regression tests for the stale-publication blocker (diagnostic retirement when a source leaves a root analysis).

What changed (crates/wright-lsp, commit 1cd3701 — the semantic model in wright-language is untouched):

• Added a per-root publication-ownership map to the LSP server state: root document URI → set of canonical publication URIs that root last published (BTreeMap<String, BTreeSet<String>>, deterministic iteration).
• Every publish path (didOpen, didChange, dependent-document refresh) now routes through one stateful helper: group current diagnostics by source, publish them, retire previously-owned sources absent from the current set with an empty publishDiagnostics, guarded by other roots' current ownership, then record the new set.
• Keys are the canonical publication URI form (source_to_uri), so an include reached as a resolved path and as an open-document URI collapse to one identity — the same form the wire publishes.
• didClose retires only the closed root's sole-owned sources (its own URI + includes no other root owns); sources still justified by another open root stay published.
• source_version now decodes canonical URIs to paths when looking up open-document versions (needed for the canonical keys).

Required regression tests (all in crates/wright-lsp/tests/lsp.rs, driving the real binary over stdio, asserting exact notification content):

• A lsp_included_diagnostics_retire_when_the_source_disappears — include error published, then publishDiagnostics(bad.opy, []) after the include is removed.
• B lsp_invalid_overlay_clears_diagnostics_when_made_valid — invalid overlay error cleared (empty + current version) when the overlay becomes valid, without reopening the root.
• C lsp_two_roots_keep_shared_diagnostics_until_the_last_owner_drops_them — removing the include from root-a only emits no empty for shared.opy while root-b owns it; the last owner's refresh retires it.
• D lsp_didclose_retires_only_sole_owned_diagnostics — closing root-a keeps shared.opy published (root-b still owns it); closing the final owner retires it.

New fixture crates/wright-language/tests/fixtures/two-root/ (root-a.opy, root-b.opy, shared.opy) for scenarios C/D.

Local verification (CI cannot be run from this sandbox without pushing; the commit is local-only and untagged for CI):

• cargo test -p wright-lsp --test lsp × 2: 17/17 green each run (13 existing + 4 new).
• cargo test --workspace --all-targets --all-features × 2: fully green.
• cargo fmt --check: clean. cargo clippy --workspace --all-targets --all-features -- -D warnings: exit 0. git diff --check: clean.

Existing source-aware tests (filesystem includes, unsaved overlays, source-local UTF-16 ranges, client versions, didClose) remain green.

Note: #72 stays open pending the GitHub CI run on the pushed commit — the local gates above are the locally-verifiable bar.

[Teakowa]

Independent verification after push

Verified against the actual repository and remote CI, not the implementation self-report.

Repository evidence

• Implementation commit is present in repository history: 1cd370190af44d88bb29459c0eeffc16353368f2.
• The patch adds per-root diagnostic publication ownership in wright-lsp, canonical publication URI tracking, cross-root ownership guards, and sole-owner retirement on didClose.
• The patch includes all four required real stdio LSP regression scenarios:
  1. included diagnostic retires when the source disappears from a root analysis;
  2. invalid unsaved overlay becomes valid and its diagnostics are cleared;
  3. two roots sharing one diagnostic source do not clear it until the final owner drops it;
  4. didClose retires only diagnostics solely owned by the closed root.

Remote CI evidence

The current main (de354720b682d96973ba9f4ae360f9d5f2b50ac7) contains the #72 implementation and passed GitHub Actions CI run 31684348750 with all six jobs completed successfully, including stable and Rust 1.85 quality/tests, native-vs-reference differential, OverPy oracle, OverPy-to-HIR adapter, and v1 release gates.

Verdict

The stale included-source diagnostic ownership/retirement blocker is closed. The implementation remains bounded to LSP publication state and does not introduce a second semantic/project graph or speculative incremental engine.

#72 satisfies its current acceptance criteria and can be closed. This does not validate #73, #69, or M10 as a whole.