[Validation Approach] Improve ConcurrentStack.TryPopRange performance under contention.

[aw0lid]

Overview

This PR is opened as a validation PR to investigate potential approaches for improving ConcurrentStack<T>.TryPopRange performance under contention, as described in #100083.

The current implementation attempts to pop the requested number of items as a single batch. Under contention, repeated CAS failures can cause the same large portion of the stack to be traversed repeatedly, leading to a significant performance degradation.

Approach 1: Adaptive Batch Sizing

The first approach explored in this PR is adaptive batch sizing.
The batch size starts at the requested count.
If the CAS fails 30 times consecutively, the batch size is reduced by half and the operation is retried. After a successful CAS, the popped batch is copied to the destination array and the remaining items are processed using the current batch size.
If contention increases again and the CAS fails another 30 times, the batch size is reduced again.

For example:

Requested: 1,000,000
Current batch: 1,000,000

        CAS fails ×30
              ↓
Current batch: 500,000

        CAS fails ×30
              ↓
Current batch: 250,000

        CAS succeeds
              ↓
       Pop 250,000
       Remaining: 750,000
              ↓
Next batch: 250,000

        CAS succeeds
              ↓
       Pop 250,000
       Remaining: 500,000
              ↓
Next batch: 250,000

        CAS fails ×30
              ↓
Current batch: 125,000

        CAS succeeds
              ↓
       Pop 125,000
       Remaining: 375,000
              ↓
Next batch: 125,000
              ...
              ↓
Remaining: 0
              ↓
          RETURN 1M

The goal of this approach is to avoid repeatedly traversing a large number of nodes when the stack is highly contended, while still allowing large batches when contention is low.

Initial results

Show benchmark code

public class ConcurrentStackBenchmarks
{
    private readonly ConcurrentStack<int> _stack = new();
    private int[] _popRangeBuffer;
    private bool _running = true;

    [Params(1_000_000)]
    public int TryPopCount;

    [Params(0, 1, 2, 3, 4, 5, 6, 7, 8)]
    public int ParallelThreads;

    [GlobalSetup]
    public void Setup()
    {
        _popRangeBuffer = new int[TryPopCount];

        for (int i = 0; i < TryPopCount; i++)
        {
            _stack.Push(i);
        }

        for (int i = 0; i < ParallelThreads; i++)
        {
            _ = Task.Run(() =>
            {
                while (Volatile.Read(ref _running))
                {
                    _stack.Push(42);
                    _stack.TryPop(out _);
                }
            });
        }
    }

    [GlobalCleanup]
    public void Cleanup() => _running = false;

    [Benchmark]
    public void PopRange()
    {
        int popped = _stack.TryPopRange(_popRangeBuffer);
        if (popped != TryPopCount) throw new Exception("?");
        _stack.PushRange(_popRangeBuffer);
    }
}

Parallel Threads	Mean Before	Mean After	Performance	Allocated Before	Allocated After	Allocation
0	97.08 ms	88.00 ms	-9.4%	30.52 MB	30.52 MB	0%
1	136.92 ms	125.07 ms	-8.7%	52.32 MB	48.39 MB	-7.5%
2	251.16 ms	210.01 ms	-16.4%	86.83 MB	76.98 MB	-11.3%
3	2,056.74 ms	429.75 ms	-79.1%	647.28 MB	247.48 MB	-61.8%
4	11,217.58 ms	470.94 ms	-95.8%	590.65 MB	259.91 MB	-56.0%
5	24,535.96 ms	465.93 ms	-98.1%	567.47 MB	245.29 MB	-56.8%
6	1,065.28 ms	466.86 ms	-56.2%	148.70 MB	255.24 MB	+71.6%
7	10,673.27 ms	477.15 ms	-95.5%	9,368.18 MB	249.82 MB	-97.3%
8	58,015.97 ms	475.71 ms	-99.2%	15,822.80 MB	268.41 MB	-98.3%

The most significant improvement can be seen under high contention, where the benchmark goes from ~58 seconds to ~476 ms with 8 threads, while allocations drop from ~15.8 GB to ~268 MB.

These results are encouraging and show that adaptive batch sizing can significantly reduce the performance degradation under contention.

Further approaches may be explored in this PR to evaluate different strategies and trade-offs.

Open Question

One important aspect that still needs to be evaluated is whether performing multiple successful CAS operations within a single TryPopRange invocation is compatible with the atomicity semantics expected from the API.

Further validation and discussion are needed before considering any approach as a final implementation.

Note

This is a validation implementation intended to evaluate the approach and its performance.
The code may be refined or refactored later.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-collections
See info in area-owners.md if you want to be subscribed.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[aw0lid]

Hi all, just following up on this validation PR when you have a chance.

I’d appreciate any feedback on the approach, especially the open question around TryPopRange atomicity.

[jkotas]

performing multiple successful CAS operations within a single TryPopRange invocation is compatible with the atomicity semantics expected from the API.

This sounds like a problem.

From the API documentation at https://learn.microsoft.com/dotnet/api/system.collections.concurrent.concurrentstack-1.trypoprange : "Attempts to pop and return multiple objects from the top of the ConcurrentStack atomically."

[aw0lid]

Thanks for confirming, @jkotas.

I had already looked at the API documentation before opening the PR, but I wanted to confirm my understanding of the atomicity requirement.

I also noticed that the Returns section describes the return value as:

“The number of objects successfully popped from the top of the stack and inserted in items.”

https://learn.microsoft.com/en-us/dotnet/api/system.collections.concurrent.concurrentstack-1.trypoprange?view=net-10.0#returns

Would this mean that it is valid for TryPopRange to return fewer items than the requested count, provided those items are popped atomically?

For example:

Requested count: 1,000,000

        1,000,000
             │
      30 CAS failures
             ▼
         500,000
             │
             ▼
          RETURN
           500,000

Instead of continuing to perform additional CAS operations until all 1,000,000 items have been popped, the operation would return 500,000 after successfully popping that smaller batch with a single CAS.

I also noticed the existing test TryPopRange_Concurrent_PoppedItemsAreConsecutive in ConcurrentStackTests.cs, which explicitly asserts that the returned count equals the requested numElementsPerThread in a concurrent scenario:

Assert.Equal(numElementsPerThread, res);

So I wanted to make sure I’m not overlooking an intended semantic here.

Would returning a smaller batch in this way be considered valid for TryPopRange when the stack is under contention?

[jkotas]

Would returning a smaller batch in this way be considered valid for TryPopRange when the stack is under contention?

I do not think so. We have many Try... APIs in the .NET class libraries, and the generally expected behavior of such APIs is to perform the complete operation whenever possible. The only valid reason for doing less is when completing the operation is genuinely impossible - for example, when the requested item does not exist.

[aw0lid]

I have another approach I'd like to experiment with.

Instead of traversing the entire range before detecting a changed head, periodically check the head during traversal (e.g. every 1/4 of the range). If it changed, discard the current traversal and restart from the new head; otherwise, keep going.

The idea is to reduce wasted traversal under contention while preserving the existing atomicity semantics.

I'll try it and benchmark it when I get some time.

[aw0lid]

The last push was made by mistake, which resulted in an incorrect diff and caused this PR to be closed.
The work has been continued in a new PR: #134306.