Plan viewer and analyzer: compare row estimates the way PerformanceStudio does (#4627 follow-up) - #4698
Merged
Conversation
…rmanceStudio does (#4627 follow-up) Port PerformanceStudio's RowEstimateHelper (PerformanceStudio#594, fixed in #597) and move rules 5, 16 and 26 onto it. EstimateRows is per execution and ActualRows is a total; ActualExecutions is a real per-execution count only on the inner side of a Nested Loops join, and a thread count in a parallel zone, so the old actual-per-execution division understated the actual by DOP outside a loop. Rule 32 is unchanged: PerformanceStudio#597 left it comparing the raw total to the raw estimate.
…d rows, with just enough decimals to agree with its percentage (#4627) ActualRows is a total, so the label sets it against RowEstimateHelper.GetExpectedRows instead of dividing it by ActualExecutions: an accurate operator at DOP 8 no longer reads 12%, and from DOP 11 it is no longer drawn critical. The new PlanRowAccuracy.FormatActualOfExpected prints N0, adds the fewest decimals (fixed-point, capped at 4) at which the printed numbers give the printed percentage, and never prints a non-zero value as zero. A Key Lookup that ran 117 times for 1 row reads 1 of 1.128 (89%).
…stimateHelper, so a parallel zone stops drawing false Blue edges (#4627 follow-up) PlanEdgeColour.ForChild divided ActualRows by ActualExecutions for every node. In a parallel zone that is not on a Nested Loops inner side, ActualExecutions counts threads, so an accurate operator at DOP 11 or more read as a 1/11 overestimate and drew Blue, and a real underestimate was shrunk by the same factor. ForChild(PlanNode, limit) now takes the child's expected rows from RowEstimateHelper.GetExpectedRows, as PerformanceStudio does after PS#594 / #597, and the numeric overload takes expectedRows instead of an execution count. RowEstimateHelper.GetRowAccuracyRatio(actualRows, expectedRows) holds the ratio arithmetic, and the node overload calls it. GetLinkColorBrush and the minimap's copy pass the node.
…ore (#4627) PlanRowAccuracy.ActualRowsPerExecution, Ratio and FormatActualOfEstimate, and the two members only they used (MaxSignificantDigits and IsFraction), had no caller left once the node label, the edge colour and the minimap moved onto RowEstimateHelper. The stale comment that said the edge and the minimap still called them goes too. Viewer4684RowLabelTests keeps its DoesNotContain pins on the old names in the viewer's source.
…n the accurate parallel operator at DOP 2000 (#4627) Two comments, one test name, one assertion message and two section headers now name RowEstimateHelper or say what they hold instead of calling something "the helper". The accurate-operator edge pin gains a DOP 2000 row: the old division read 1/2000 = 0.0005 there and drew FluoBlue, the third overestimate tier, which no row covered before.
erikdarlingdata
marked this pull request as ready for review
September 29, 2026 02:03
This was referenced Sep 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4627.
Builds on #4632 and #4688 (the row label). Ports erikdarlingdata/PerformanceStudio#594 and erikdarlingdata/PerformanceStudio#597 for the analyzer's row-estimate rules (5, 16 and 26), the plan node's row label, the plan edge colour and the minimap.
Why
EstimateRowsis per execution andActualRowsis a total.ActualExecutionsis a real per-execution count only on the inner side of a Nested Loops join. In a parallel zone it is the thread count. The analyzer's rules 5, 16 and 26, the node label, the edge colour and the minimap all divided the actual rows byActualExecutionsfor every node, so outside a loop they understated the actual by the DOP. PerformanceStudio fixed this in erikdarlingdata/PerformanceStudio#597 withRowEstimateHelper; this ports it. Every one of them now compares the totalActualRowswithRowEstimateHelper.GetExpectedRows(node):EstimateRowstimesActualExecutionson a Nested Loops inner side,EstimateRowseverywhere else.What changes
Analyzer
PerformanceMonitor.PlanAnalysis/RowEstimateHelper.cs, a faithful port with PerformanceStudio's member names:IsInnerSideOfNestedLoops,GetExpectedRows,GetRowAccuracyRatio(zero expected rows: 1.0 for zero actual,double.MaxValueotherwise). The numeric overloadGetRowAccuracyRatio(actualRows, expectedRows)holds the arithmetic and the node overload calls it.PlanAnalyzer.csrules 5, 16 (outer-side line) and 26 useRowEstimateHelper, matching PerformanceStudio rule for rule. Rule 5: the "(N rows x M executions)" text prints only whenisInnerSide && ActualExecutions > 1. Rule 16: the printed actual is the per-execution figure only when the outer child is on an inner side, else the total. Rule 26:rowGoalWorked = GetRowAccuracyRatio(node) <= 1.0.Darling/Darling.Tests/Fixtures/OriginPlans/eager_index_spool_plan.sqlplan(DOP 8), copied from PerformanceStudio.Viewer4627RowEstimateHelperTests(17 facts) andViewer4627RowEstimateRulesTests(5 facts).Node label and formatter
PlanRowAccuracy.FormatActualOfExpected(double actualRows, double expectedRows, IFormatProvider? provider = null).PlanViewerControl.Rendering.cslabel block: the text isPlanRowAccuracy.FormatActualOfExpected(node.ActualRows, RowEstimateHelper.GetExpectedRows(node)); the brush ratio isRowEstimateHelper.GetRowAccuracyRatio(node)with the 0.1 and 10.0 thresholds unchanged. The block's comments say what it does now.PlanRowAccuracy.ActualRowsPerExecution,RatioandFormatActualOfEstimateare deleted with the two members only they used (MaxSignificantDigits,IsFraction), because nothing calls them any more. TheDoesNotContainpins on the old names in the viewer's source stay.Viewer4684RowLabelTestsis rewritten for the totals form (48 tests).NodeWidth - 16(134), so a label like8,042,005 of 8,042,010 (100%)sits near the edge. The node already has a tooltip; none was added.The formatter rule, word for word:
Also: no exponent at any magnitude.
How it is implemented:
(actualRows / expectedRows * 100).ToString("F0", provider), from the unrounded values. It shows only whenexpectedRows > 0; otherwise the label is "X of Y" with no percent. It is a whole percent, so a non-zero ratio can print(0%), for example1 of 1,000,000 (0%). The non-zero rule applies to the row counts, not to the percent.Math.Floor(v) == v) always printsN0; a non-whole number prints"N" + d. The printed text is parsed back withNumberStyles.Numberand the provider (NumberStyles.Floatrejects "2,983"). The search stops at the first d where the whole percent of the printed numbers equals the printed percent. A printed 0 divisor counts as a disagreement (NaN or Infinity never equals the percent). It stops at d = 4 either way, so at the cap a label can still contradict its percent."N" + k, k =-(int)Math.Floor(Math.Log10(Math.Abs(v))). The cap does not apply to k: 0.000005 prints "0.000005".Nformats print row counts.F0prints only the percent. NoG,R,Eor bareToString().The old DOP 8 label for an accurate operator read
12 of 100 (12%): 12.5% is an exact tie, and .NET'sF0rounds a tie to even. At DOP 11 it read9 of 100 (9%)and turned orange (ratio 0.0909). The pins assert these strings.Pinned strings (invariant culture unless noted):
1 of 1.128 (89%)0 of 0.000005 (0%)key_lookup_plan.sqlplanNode 4 (old label0.0085 of 0.0096 (89%))1 of 1.128 (89%)eager_index_spool_plan.sqlplanNode 1 (DOP 8, not inner)609 of 2,983 (20%)609 of 613 (99%)100 of 100 (100%), brush ratio 1.0 (old:12 of 100 (12%),9 of 100 (9%)and orange)400 of 500 (80%)1 of 0.0001 (810045%)1 of 0.00001 (8103728%)0.0001 of 0.0005 (27%)0.0000004 of 1 (0%)1 of 1,128 (89%),609 of 2.983 (20%),0,60 of 0,75 (80%)0.60 of 0.75 (80%),0.25 of 0.30 (83%),0.99 of 1.01 (98%),0.600 of 0.556 (108%),0.0104 of 0.0096 (108%),100,000 of 12.5 (800000%),2.5 of 2 (125%),1 of 0,0 of 0,0.00001 of 01,000,000,000,000,000 of 3,000,000,000,000,000 (33%)Also pinned: a seeded sweep of 2,000 value pairs (1e-6 to 1e6, whole and fractional, a twentieth with 0 actual). Every label agrees with its percent or stopped at 4 decimals, none has an exponent, and no non-zero value prints as zero. The sweep reaches the hard paths (checked in a scratch run): 1,179 labels with decimals, 472 at 4 or more decimals, 222 past the cap by the non-zero rule. Extreme magnitudes (1e300,
double.MaxValue, 5e-324) print no exponent. A source pin checks the label callsFormatActualOfExpected(node.ActualRows, RowEstimateHelper.GetExpectedRows(node)), takes its brush fromGetRowAccuracyRatio(node), and no longer containsFormatActualOfEstimate,ActualRowsPerExecutionorPlanRowAccuracy.Ratio(.Edge colour and minimap
PlanEdgeColour.ForChild(PlanNode child, double divergenceLimit)compares the child's totalActualRowswithRowEstimateHelper.GetExpectedRows(child). The numeric overload is nowForChild(bool hasActualStats, double actualRows, double expectedRows, double divergenceLimit): it takes expected rows instead of an execution count and gets its ratio fromRowEstimateHelper.GetRowAccuracyRatio(actualRows, expectedRows).GetLinkColorBrushinPlanViewerControl.Rendering.csand the minimap's copy inPlanViewerControl.Minimap.cshand the node toForChild(child, limit), so no caller has to invent the expected figure.Viewer4627EdgeColourTestspins the colours;Viewer4627EdgeCallerTestsreads the two viewer files and pins that both pass the node and that no product code calls the numeric overload.What the edge draws, before (the old ratio, actual / executions / estimate, worked out by hand) and after:
eager_index_spool_plan.sqlplanNode 1, DOP 8, not inner (609 actual, estimate 2,983.02)key_lookup_plan.sqlplanNode 4, Nested Loops inner side (1 actual, 0.00964372 x 117 executions)Left alone on purpose
McpPlanAnalysisFormatter(emits rawestimated_rowsandactual_rows; PerformanceStudio'sexpected_rowsaddition has no counterpart here, and a new field would touch the MCP payload census).deprecated/: no copy of this code, not edited.Darling.Tests(PlanSync*and theViewer4627*classes).REDs
Each regression was planted alone, the class run, and the source restored (a clean tree afterwards).
Analyzer (old analyzer restored with
git checkout origin/dev -- PerformanceMonitor.PlanAnalysis/PlanAnalyzer.csonly):Viewer4627RowEstimateRulesTestsTotal 5, Failed 3.Label (
Viewer4684RowLabelTests, 48 tests):Rendering.cs(git checkout 9ef1fb1a7 -- PerformanceMonitor.Ui/PlanViewerControl.Rendering.cs, formatter untouched)NodeLabel_IsBuiltFromTheTotalsAndTheSharedExpectationGformat inPrintRows(value.ToString("G", provider))ExtremeMagnitudes_NeverPrintAnExponent4,LargeNumbers_PrintEveryDigit2),WholeNumbers_PrintN04,Decimals_AreAddedOnlyUntilTheNumbersAgreeWithThePercentage4,Cap_StopsAtFourDecimals_ButANonZeroValueNeverPrintsAsZero5,Culture_FollowsTheCallers3,Culture_DefaultsToTheCurrentCulture,KeyLookupFixture_Node4_PrintsTheTotals,KeyLookupFixture_InGerman_PrintsADecimalComma, both gate vectors,EagerIndexSpoolFixture_PrintsTheTotals(Node 1) and the sweepGformat on the first-significant-digit reprint line aloneCap_...5,ExtremeMagnitudes_NeverPrintAnExponent2,Decimals_...1,GateVector_TinyEstimateWithNoActualRows_KeepsItsFirstSignificantDigitand the sweepCap_...3 (1 of 0.00001234, 0.00001 of 0, 0.0000004 of 1), the tiny-estimate gate vector and the sweepNumberStyles.FloatinPrintedNumbersGiveCulture_FollowsTheCallers(609 of 2983.02 in de-DE),Decimals_...(100000 of 12.5),EagerIndexSpoolFixture_PrintsTheTotals(Node 1's "2,983"),WholeNumbers_PrintN0(1234.5 of 1000)GetExpectedRowsmultiplying byActualExecutionson every node, which is the old per-execution division seen from the expected sideViewer4684RowLabelTestsFailed 3 (AccurateOperatorInAParallelZone_...at DOP 8 and DOP 11,EagerIndexSpoolFixture_PrintsTheTotalsNode 1);Viewer4627*Total 60, Failed 16 (Viewer4627EdgeColourTests11,Viewer4627RowEstimateHelperTests2,Viewer4627RowEstimateRulesTests3);Viewer4579TestsTotal 20, Failed 0The DOP 8 and DOP 11 facts test the label's inputs (
RowEstimateHelperand the formatter), not the WPF block itself; the block is pinned by the source pin (RED 1). A live render pin was not attempted (CreateNodeVisualis private and needs an STA thread and a theme; nothing in either suite renders a node today).Edge colour and minimap (
Viewer4627*, 60 tests when these were run; the DOP 2000 row was added after):ForChild(node, limit)dividesActualRowsbyActualExecutionsand compares withEstimateRows(the old arithmetic)ParallelZoneNode_ThatMetItsEstimate_IsNeutralAtAnyDop5 (DOP 11, 12, 32, 64, 128),ParallelZoneNode_ThatMissedItsEstimate_IsColouredByTheMiss4,EagerIndexSpoolPlan_Node1_AtDop8_IsNeutralWhereDividingByThreadsDrewBlue,SameNumbers_WithNoEnclosingNestedLoops_AreJudgedAgainstThePlainEstimate,NodeOverload_IsTheNumericOverloadFedRowEstimateHelperExpectedRows_ForEveryFixtureNode;Viewer4579TestsFailed 0GetLinkColorBrushinRendering.cshands the numeric overload the barechild.EstimateRowsViewer4627EdgeCallerTestsTotal 4, Failed 2:EdgeCaller_PassesTheNodeToPlanEdgeColour,NoProductCode_CallsTheNumericOverloadGetLinkColorBrushinMinimap.csGetExpectedRowsnever multiplies, so a Nested Loops inner side is judged against the bare estimateViewer4627*Failed 15 (KeyLookupFixture_OnTheNestedLoopsInnerSide_KeepsItsNeutralKey,KeyLookupNumbers_OnTheInnerSideOfANestedLoops_KeepTheirNeutralKey, the 3NestedLoopsWithinLoops_...facts,Viewer4627RowEstimateHelperTests4, rule 5 on an inner side 1,Viewer4627Tests.InnerSideNestedLoopsNode_EdgeRatioMatchesFixtureAndAgreesWithLabel5);Viewer4684RowLabelTestsFailed 6The DOP 2000 row of
ParallelZoneNode_ThatMetItsEstimate_IsNeutralAtAnyDopwas added after E1 ran. It pins the FluoBlue tier, which no earlier row reached; the old division reads 1/2000 = 0.0005 there.Test plan
dotnet buildofDarling.TestsandLite.Testson the merged head: 0 Warning(s), 0 Error(s).Lite.Testson the merged head: Total 5555, Failed 0, Skipped 0.Darling.Testson the merged head, withoutDARLING_TEST_PG(its live PostgreSQL tests skip): Total 16831, Failed 0, Skipped 1051, Not Run 1 (the runner's own count; the log names no test).Viewer4684RowLabelTests: Total 48, Failed 0.Viewer4627*: Total 61, Failed 0. DarlingPlanViewerCapabilityPinTests(source-scansRendering.cs): Total 5, Failed 0. LitePlanViewerCapabilityPinTests: Total 5, Failed 0.git merge origin/devdone twice; the second brought The self-hosted log-events live test drives its own events, so a log rotation cannot hide them #4702 and the WaitRate read-count pin change. Both merges were clean, with nothing inside an analyzer rule.DARLING_TEST_PG): not run. Nothing in this change touches storage.CHANGELOG
This replaces the unshipped #4632 and #4688 entries, which describe the per-execution ratio and label this change removes.