E2E tests: containerized end-to-end suite for rebalance + dynamic fee engine, in one ordered pass - #559
Open
markettes wants to merge 85 commits into
Open
E2E tests: containerized end-to-end suite for rebalance + dynamic fee engine, in one ordered pass#559markettes wants to merge 85 commits into
markettes wants to merge 85 commits into
Conversation
…e and LightningService
…ting states; add corresponding interfaces and tests
…fee management and automated rebalancing
…s and improve audit logging
…key and check in-flight rebalances by source channel
… repository and remove unused repository interfaces
…lated documentation
…model and related migrations
…ee-engine-phase2a
…tory + no dry-run)
… ChannelFeeOptimizerJob and TargetRatioReevaluationJob
…y/NodeGuard into feat/heuristic-fee-engine-phase2a
…izer and PeerCategorization
…ds for better fee management
…abled for clarity
…FeeOptimizerService
…f int for clarity
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a containerized, ordered end-to-end (E2E) test suite that drives a live NodeGuard instance (gRPC) against real regtest infrastructure (LND + Postgres), covering both channel rebalance and the dynamic fee engine in a single run.
Changes:
- Introduces
E2ESuiteTests(partial class split by scenario) plus an xUnit method orderer to run three dependent scenarios in a fixed order. - Adds an in-process LND gRPC test client (
LndTestClient) and expands shared E2E plumbing (E2ETestBase) to support polling/retry, mining, and common channel-open flow. - Updates NodeGuard Quartz scheduling to share a routing-job cadence helper and updates the docker/CI wiring + docs for the new E2E suite workflow.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/NodeGuard.Tests/E2E/PriorityOrderer.cs | Adds xUnit test-case ordering via [TestPriority] for ordered E2E scenarios within one class. |
| test/NodeGuard.Tests/E2E/LndTestClient.cs | Adds a test-only LND gRPC client to generate controlled payment flow in-process. |
| test/NodeGuard.Tests/E2E/E2ETestBase.cs | Refactors shared E2E plumbing into an abstract base with reusable retry/poll/mining/channel-open helpers. |
| test/NodeGuard.Tests/E2E/E2ESuiteTests.Rebalance.cs | Adds scenario (1): open channel via NodeGuard + circular rebalance, and defines suite-level ordering. |
| test/NodeGuard.Tests/E2E/E2ESuiteTests.FeeEngineSmoke.cs | Adds scenario (2): fee-engine apply/disable lifecycle checks via direct DB reads and helpers. |
| test/NodeGuard.Tests/E2E/E2ESuiteTests.FeeEngineFlow.cs | Adds scenario (3): drives real HTLC flow to validate SINK→SOURCE flip and fee application. |
| src/Program.cs | Refactors routing-job scheduling cadence and adjusts MonitorChannels trigger style (cron-based). |
| docker/e2e/README.md | Updates E2E stack documentation to describe the combined suite and scenarios. |
| docker/e2e/docker-compose.yml | Updates E2E compose profile to support fee-engine convergence knobs and runner DB access. |
| .justfile | Updates test-e2e recipe description/behavior for the new combined E2E suite run. |
| .github/workflows/dotnet.yml | Updates CI step labeling/docs for the new “one stack, one ordered pass” E2E run. |
Suppressed comments (1)
src/Program.cs:327
ChannelFeeOptimizerJobis registered twice (same job identity and trigger identity). Quartz will treat this as a duplicate and may throw at startup or schedule unpredictably. Keep a singleAddJob/trigger pair (preferably the one usingScheduleRoutingJob) and remove the second block.
//Channel Fee Optimizer Job
q.AddJob<ChannelFeeOptimizerJob>(opts =>
{
opts.DisallowConcurrentExecution();
opts.WithIdentity(nameof(ChannelFeeOptimizerJob));
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+280
to
+283
| var routingJobIntervalSeconds = | ||
| int.TryParse(Environment.GetEnvironmentVariable("ROUTING_ENGINE_JOB_INTERVAL_SECONDS"), out var rjs) | ||
| ? rjs | ||
| : (int?)null; |
Comment on lines
+37
to
+38
| [Trait("Category", "E2E")] | ||
| [TestCaseOrderer(PriorityOrderer.TypeName, PriorityOrderer.AssemblyName)] |
Comment on lines
+98
to
+110
| // Idempotent — an "already connected" RpcException is expected and swallowed. | ||
| public async Task ConnectAsync(string peerPubKey, string hostPort) | ||
| { | ||
| try | ||
| { | ||
| await Lightning.ConnectPeerAsync(new ConnectPeerRequest | ||
| { | ||
| Addr = new LightningAddress { Pubkey = peerPubKey, Host = hostPort }, | ||
| Perm = false, | ||
| }, _auth); | ||
| } | ||
| catch (RpcException) { } | ||
| } |
Comment on lines
+8
to
+11
| `just test-e2e` brings the stack up once and runs `E2ESuiteTests` end-to-end, its three scenarios pinned in | ||
| order by `PriorityOrderer`: **(1)** rebalance → **(2)** fee-engine smoke (fee applied, then stopped on | ||
| disable) → **(3)** fee-engine flow (SINK→SOURCE, driving its own LND traffic in-process via `LndTestClient`). | ||
| Ordering matters: (3) reuses (1)'s Alice→Bob, and its traffic would starve (1)'s route if it ran first. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
An end-to-end suite driving a live NodeGuard (gRPC) against real LND (regtest/Polar) + Postgres.
just test-e2ebrings the stack up once and runsE2ESuiteTests— three scenarios pinned in order byPriorityOrderer:OpenChannelgRPC, then circular rebalance Alice→Bob→Carol→Alice.They share one stack and one Alice→Bob channel, so order matters (3 reuses 1's channel). xUnit v2 can't order test classes without killing parallelism, so they're one
partialclass (E2ESuiteTests, split across.Rebalance.cs/.FeeEngineSmoke.cs/.FeeEngineFlow.cs) ordered by[TestPriority].Pieces
E2ESuiteTests+E2ETestBase(shared plumbing) +PriorityOrderer; gated by[E2EFact](skips unless a NodeGuard gRPC is reachable orRUN_E2E_TESTS=1).LndTestClient— in-process LND gRPC driver so scenario 3 generates its own traffic (replaces the oldgenerate-flow.shsidecar); reads creds from env orextract-env'snodeguard-macaroons.env.docker/e2e/:setup-e2e+extract-env→nodeguard→e2e-runner, alle2eprofile-gated. Opt-in CI job (run-e2e-testslabel).Program.cs:MonitorChannelsJobis now cron-only (5s in e2e); the two routing jobs share oneScheduleRoutingJobhelper (seconds-else-minutes).