Fleet overview, daily summary and server summary: an Azure master counts only its own blocking and deadlocks, and its lists say where its databases' events are counted

[erikdarlingdata]

What a user saw

On a fleet with an Azure SQL Database logical server registered at master and its databases registered on their own, Darling counted the databases' blocking events and deadlocks twice. The Fleet Overview header showed Blocking 96 (master 48 + GP 25 + HS 23) and Deadlocks 91 (46 + 23 + 22), and master sat in "Needs attention" as Critical on those same events. Master's daily summary, the Performance Calendar, the fleet sweep's band, MCP get_server_summary and the Darling Viewer's fleet totals and per-server summary did the same. Master's Blocking, Deadlocks and Locking lists showed those rows with nothing saying why the card disagreed.

Why

A master registration reads server-wide blocked-process reports and deadlocks, which include the separately monitored databases' events. Alerts and analysis already skip those databases' events on a master target. These reads didn't: they grouped every stored event by server.

What changed

Every read below takes the separately monitored list analysis uses (AzureMasterScope.SeparatelyMonitoredDatabases). It applies only to an Azure SQL Database master: the server's registration (servers.sql_engine_edition, stamped on every connect) AND its newest stored server_properties.engine_edition must both be 5, on every surface. The service's stored-edition read (DarlingWorker.StoredEngineEditionSql) now tests both, so a pair that disagrees reads unscoped everywhere and #4906's scoped SQL (PgFactCollector.BlockingSqlSkippingSeparate, CountDeadlocksSkippingSeparateAsync: a deadlock counts on master unless every process in its graph is in the list). The DMV blocking arm is unchanged (master's DMV sees only its own sessions).

• The fleet card (DarlingFleetReader.GetFleetOverviewAsync: /api/fleet, the web read mirror, MCP get_fleet_overview): master's blocked-process count and max wait and its deadlock count come from the scoped reads. The cards, header totals, bands and "Needs attention" follow.
• MCP get_server_summary (DarlingHealthReader.GetServerSummaryAsync): the same scoped reads over its one-hour window.
• The daily summary (DailySummarySql.RangeSql via DailySummaryAzureMasterScope: MCP get_daily_summary / get_daily_summary_range, the read mirror, the fleet sweep's band through GetWindowSignalsAsync, and the Viewer's Performance Calendar; a master that is disconnected when the sweep fires gets its list from the stored edition and the registry, as MCP does): the deadlock and blocked-process steps count with a FILTER, and master's graph-carrying deadlocks are counted per day with the every-process rule. The windows, day bucketing and DMV arm are byte-identical. Those steps read the raw tables on every retention tier, so every retained day is scoped. The range cache key carries the list. A day whose only events are siblings' is still a present day, with a count of 0.
• The lists keep their rows (master captures server-wide), with one line above master's Blocking and Deadlocks lists and its Locking & Contention panel, in the Viewer and on the web server page: "Events from databases monitored as their own servers are listed here and counted under those servers." MCP get_blocking and get_deadlocks return it as separately_monitored_note, with the list as separately_monitored_databases; get_object_locking returns the note. The note shows only when the list is non-empty.
• The Darling Viewer computes the same list from the store (the monitored-server registrations and the same two-edition rule). It reads only those columns, never a credential column. It scopes the per-server summary and the calendar, and subtracts the over-count from the fleet totals so they agree with the cards.
• When something fails: a failed list lookup, or a failed scoped read, keeps that master's unscoped counts and logs a warning. The rest of the fleet read succeeds. A failed note lookup shows no note, and the rows still return. A scoped read that times out is followed by the unscoped read, so one call can take up to about twice its read deadline.
• get_server_summary's scoped read ends a day ahead of now, as the Viewer's does, so a row stamped ahead by clock skew counts the same in both.
• Unchanged: every other server, and a master with no separately monitored databases (the same SQL as before); the newest-deadlock time; the fleet SQL constants.
• Cost: for a non-master, the Viewer adds one indexed edition check per card, calendar, Blocking tab or Locking panel refresh. For a master, one list read plus one graph read bounded by the window. A range-cache hit re-reads only the open day.

Pins

• FleetOverviewAzureMasterScopeLiveTests: master + GP + HS on one host: master's card shows Blocking 3 and Deadlocks 1 and the header counts each event once; a sibling's longer wait leaves master's max wait at master's own; a non-Azure server and a sibling-less master are unchanged; with no resolver, a resolver that throws, or a scoped read that throws, the old counts and every card; get_server_summary matches the card, and an edition pair that disagrees either way round reads unscoped.
• ViewerFleetAzureMasterScopeLiveTests: the same arrange through the Viewer's summary and fleet totals, the two-edition rule, a failed scoped read falling back, and the registry read only for masters and once per fleet refresh.
• DailySummaryAzureMasterScopeTests, DailySummaryAzureMasterScopeLiveTests, ViewerDailySummaryAzureMasterScopeLiveTests: the scoped SQL keeps the windows and the DMV arm, every tier is scoped, the cache key carries the list, master's days exclude sibling events, the service and the Viewer agree, and a throwing scoped read falls back.
• SeparatelyMonitoredListNoteTests: the sentence, and when it shows on each surface.
• AzureMasterAnalysisScopeLiveTests: the stored-edition rule (including a registration of 8 with a newest stored edition of 5), and a source pin that the sweep scopes a master with no runtime.
• ServerPageTabsTests: table()'s full signature, now with moreNoteKeys.
• The fleet pins fail on assertions at the tests commit (master blocking expected 3, actual 6), and pass with the fix.

Tests run

• Builds: Lite.Tests and Darling.Tests, 0 warnings and 0 errors.
• Darling.Tests in-process, on a live PostgreSQL store, all passing: FleetOverviewAzureMasterScopeLiveTests (9), AzureMasterAnalysisScopeLiveTests (21), DailySummaryAzureMasterScopeLiveTests (4), ViewerDailySummaryAzureMasterScopeLiveTests (3), ViewerFleetAzureMasterScopeLiveTests (8), DailySummaryAzureMasterScopeTests (6), SeparatelyMonitoredListNoteTests (7), ServerPageTabsTests (19), PgIndexBloatCoverageTests (16), FleetSweepSignalReadLivePostgresTests, FleetSweepCadenceKnobRungTests, DarlingMcpHealthToolsLivePostgresTests, DarlingSelfAlertTests, CollectorMemoryKnobTests, and the T-SQL, doc-comment, repo-file and deadline-scanner guards.
• The new edition and no-runtime pins fail with the worker change reverted, and pass with it.

CHANGELOG

None as its own line: this widens the existing Fixed entry for the fleet overview's Azure master count (#4894) to name the daily summaries, calendars, the sweep band, get_server_summary and the list notes. The double count was present in v3.8.0.