Skip to content

Adjust RouteCache to mitigate contention - #998

Merged
FelixMcFelix merged 4 commits into
masterfrom
routecache-tuning
Jul 24, 2026
Merged

Adjust RouteCache to mitigate contention#998
FelixMcFelix merged 4 commits into
masterfrom
routecache-tuning

Conversation

@FelixMcFelix

Copy link
Copy Markdown
Collaborator

This PR follows up on the steps outlined in #997. The main changes are to increase the table size, and to minimise the amount of time holding the write lock on a miss by fetching a new route before taking the lock. This includes refusing to attempt an insert on a failed lookup in a full table.

One of the steps outlined in the issue, not evicting present entries, was already implemented.

Closes #997.

@FelixMcFelix

FelixMcFelix commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator Author

Some quick testing notes on throughput on glasgow during the implementation of the above:

-P 1 -P 64 -P 128
master 14.0 31.3 31.3
MAX_CACHE_ENTRIES = 32 14.2 27.4 25.9
+ reduced lock scope 14.0 29.1 28.7
+MAX_CACHE_ENTRIES = 8192 14.2 31.7 31.7

My read is that the lock scoping changes (and refusal to take a write at suspected max capacity) do help in the cases that we have more flows than slots. I'll see about getting some lockstat runs.

@FelixMcFelix FelixMcFelix self-assigned this Jun 18, 2026
@FelixMcFelix FelixMcFelix added this to the 21 milestone Jun 18, 2026
@morlandi7 morlandi7 modified the milestones: 21, 22 Jul 9, 2026

@rcgoodfellow rcgoodfellow left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @FelixMcFelix. Just one non-blocking suggestion here.

Comment thread xde/src/route.rs Outdated
Comment thread xde/src/route.rs
@FelixMcFelix
FelixMcFelix merged commit 7eddb50 into master Jul 24, 2026
10 checks passed
@FelixMcFelix
FelixMcFelix deleted the routecache-tuning branch July 24, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Need to better handle contention around RouteCache

3 participants