Skip to content

Azure SQL Database: a monitored master's analysis findings no longer repeat the blocking and deadlocks of databases monitored as their own targets - #4906

Merged
erikdarlingdata merged 32 commits into
devfrom
fix/azure-master-analysis-duplicates
Oct 1, 2026
Merged

erikdarlingdata merged 32 commits into
devfrom
fix/azure-master-analysis-duplicates

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

What a user saw

An Azure SQL Database target on a logical server's master collects blocked-process reports and deadlocks for every database on the server. #4894 stops that master target from ALERTING on the events of databases that are also monitored as their own targets. The analysis findings are a second notifying path: email and webhooks, on by default in Darling. On the master target they still counted those databases' events (BLOCKING_EVENTS, BLOCKING_CHAIN, DEADLOCKS, ANOMALY_BLOCKING_SPIKE, ANOMALY_DEADLOCK_SPIKE), and the drill-down evidence showed them. So the same incident paged twice, and the master copy named no database.

What changed

The rule, shared with the alert sweep:

  • AnalysisContext.SeparatelyMonitoredDatabases carries the same list the alert sweep uses (Azure SQL Database: a monitored master no longer duplicates blocking and deadlock alerts for databases monitored as their own targets #4894's AzureMasterScope): the databases monitored as their own targets on the same server, set only for an Azure SQL Database master target.
  • A blocked-process report or blocking pair is skipped when its database is in the list; a NULL database still counts.
  • A deadlock is skipped only when EVERY process in its graph is in the list (PerformanceMonitor.Common.DeadlockGraphDatabases.AllIn; the alert engine's IsDeadlockExcluded delegates to it).
    • The graph read is bounded: a deadlock whose stored database is named, outside the list and not master is counted in SQL without reading its graph. Graphs are read only for a NULL, master-stamped or in-list database. A row stamped master (it can be the connection's fallback) is always decided by its graph.
  • Both apps, with no list or an empty one: the SQL is today's, byte for byte.

What applies it, in both apps:

  • The facts: BLOCKING_EVENTS, BLOCKING_CHAIN (the reconstructed chains drop pairs in the list, on the blocked-process and DMV arms) and DEADLOCKS.
  • The anomaly spikes: both, on the blocked-process and DMV-snapshot arms. Baselines stay server-wide (below).
  • The drill-down evidence: the top blocking chains, the reconstructed chains and the top deadlocks. The scoped top-deadlocks read streams newest-first, skips deadlocks wholly inside the list until three qualify, and is bounded at 200 rows.
  • The read tools: Lite's CollectAndScoreFactsAsync and ComparePeriodsAsync, and Darling's MCP analyze_server, fact and compare reads and the web endpoints, through DarlingAnalysisService.ScopeForAsync.

Where the list comes from:

  • Lite: filled once per pass from a provider the app sets where its server list lives, keyed by each target's own server id.
  • The Darling worker: the same AzureMasterScope call as the alert sweep, over the live registry by store id. Scheduled passes and analyze_now both pass it.
  • Darling's MCP and web hosts: a target counts as Azure SQL Database when its newest stored server_properties.engine_edition is 5, the value the probe stored, so a private endpoint, a sovereign cloud or a DNS alias agrees with the worker. No row, a NULL edition or any other edition gives no list.
  • A failing scope lookup (anything but cancellation) is logged and gives an unscoped analysis, never a failed one.

Not changed

  • Baselines stay server-wide. A server-wide baseline compared against a filtered count can only make a master spike LESS likely, which is accepted: master isn't those databases' alerting home.
  • The user's own ExcludedDatabases in analysis.
  • Collection.

Pins

  • Lite AzureMasterAnalysisScopeTests: BLOCKING_EVENTS with NULL rows counted; BLOCKING_CHAIN (only the pair outside the list remains); DEADLOCKS (an all-listed graph skipped, a mixed one kept, a master-stamped one decided by its graph); both spikes on both arms; the drill-down evidence; the provider fill keeps an explicit list; a throwing provider degrades to unscoped; null and empty guards.
  • Darling AzureMasterAnalysisScopeLiveTests (live store): the facts, BLOCKING_CHAIN, deadlocks (including master-stamped and unparseable graphs), the spikes, the drill-downs, the worker fill, the MCP/web resolver by stored engine edition (a private-endpoint edition-5 host, an edition-8 host, no row, NULL, newest wins), and a throwing resolver giving an unscoped result while cancellation still propagates. RED before the first fix: 6 of 7 failed (blocking 5 vs 2, deadlocks 3 vs 1, the spikes, the fill).
  • Darling DeadlockGraphDatabasesTests: all-in, mixed, no processes, bad XML, and agreement with the alert engine's check.
  • Test fixtures that follow the change: the analysis-pass worker fixture in PostgresEngineGateBehaviorTests sets the server registry the pass now reads.

CHANGELOG

None: this widens #4894's entry, which the maintainer words to cover the analysis findings too. Shipped in v3.8.0: v3.8.0's fact collectors already counted every blocked-process and deadlock row for the server.

… and deadlocks that belong to databases monitored as their own targets
…s for databases monitored as their own targets
…ases, and the deadlock every-process rule lives in one place
…he sustained-blocking template warns about a master target
…findings skip databases monitored as their own targets
…indings skip databases monitored as their own targets
…ck findings skip databases monitored as their own targets
…k findings skip databases monitored as their own targets
…its closing parenthesis, since another argument now follows it
…ence and fact reads

BLOCKING_CHAIN and the blocking/deadlock drill-downs skip databases monitored as their own targets.
The fact and compare reads fill the list like AnalyzeAsync. The deadlock count counts a deadlock whose
victim database is outside the list in SQL and parses graphs only for the rest. {SCOPE} sits at the end
of its line so an empty list gives the old text. The static provider is internal and a throwing provider
falls back to unscoped.
… master target; bound the deadlock graph read

BLOCKING_CHAIN drops pairs of databases monitored as their own targets. The top-deadlock and top-blocking drill-downs apply the same rule. Deadlocks whose named victim database is not separately monitored are counted in SQL and their graphs are not read; the anomaly path uses the detector's command timeout. The list is passed raw and both sides fold with one lower(). MCP analyze_server and the web read tools resolve the list per call from the live registry.
… and recognises Azure SQL Database by its stored engine edition
…d row is checked by its graph, and the unscoped text is unchanged
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review October 1, 2026 17:51
@erikdarlingdata
erikdarlingdata merged commit 8abdd67 into dev Oct 1, 2026
17 of 18 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/azure-master-analysis-duplicates branch October 1, 2026 17:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant