Skip to content

[Task] Add plugin testing harness and test base #41

Description

@rian-be

Summary

Add thin testing layer for plugin DX in a new, dedicated assembly AuthKit.Plugins.Testing not AuthKit.Plugins.Abstractions. H9 is concrete PluginTestHarness (no interface), H10 is AuthKitPluginTestBase<TPlugin>. The harness is an extraction of the existing Host.IntegrationTests setup (PR #35, AuthKitWebApplicationFactory : WebApplicationFactory<Program>, merged) into reusable shape not second, from scratch in-memory host.

Goal

Remove duplicated plugin test setup by reusing the same host composition the production host uses. A plugin test starts a harness over the real host composition; nothing in the tester reimplements service registration, endpoint mapping, lifecycle, or health.

Precedent (already merged)

tests/Host.IntegrationTests (PR #35, Host Lifecycle) already:

public sealed class AuthKitWebApplicationFactory : WebApplicationFactory<Program>
  • hosts the real Program (production discovery, lifecycle, health) over TestServer,
  • only swaps IKeyStoreRepository for an in-memory stand-in (no PostgreSQL),
  • is used via _factory.CreateClient() (HTTP round-trip) and _factory.Services (DI assertions) by DevTokensHostLoadTests and PluginHealthEndpointIntegrationTests.

This is exactly the "shared composition" the harness needs it is not hypothetical. H9/H10 = extract this into AuthKit.Plugins.Testing.PluginTestHarness, then reuse it from Host.IntegrationTests itself and from new plugin tests.

Architecture

AuthKit.Plugins.Abstractions  runtime plugin contract
        │
        ↓
   IAuthKitPlugin

AuthKit.Plugins.Testing  test support (fully additive, no runtime API)
        │
        ├── PluginTestHarness  concrete, sealed
        └── AuthKitPluginTestBase<TPlugin>
flowchart LR
    P["Program (production composition)"] --> W{"TestServer / WebApplicationFactory"}
    W --> H["PluginTestHarness"]
    H --> B["AuthKitPluginTestBase&lt;TPlugin&gt;"]
    B --> T1["Host.IntegrationTests"]
    B --> T2["plugin unit/integration tests"]
Loading

One composition (Program), reused; no parallel test host code path.

Proposed Contract (H9 concrete class, no interface)

public sealed class PluginTestHarness : IAsyncDisposable
{
    public IServiceProvider Services { get; }
    public HttpClient Client { get; }  // real ASP.NET Core pipeline (TestServer)

    public Task StartAsync();
    public Task StopAsync();
}
  • Client.GetAsync("/hello") HTTP is asserted through the real pipeline, not an abstract "endpoint collection". No Endpoints property, no endpoint mapping resurface.
  • Narrow surface for the first iteration: Services (DI assertions) and Client (HTTP) a direct extraction of what Host.IntegrationTests already uses (_factory.Services, _factory.CreateClient()). No full host mirror.
  • No DiagnosticsSnapshot in the first iteration. Services/Client have a concrete counterpart in PR feat(plugin): add host based plugin configuration, pipeline and lifecycle hooks #35; a diagnostics snapshot has none and would be speculative scaffolding. It is added only as a separate follow-up when the first real test proves it needs a stable, plugin-observable signal. Rule, if it ever lands: exposes only stable plugin-observable signals required by tests; never host internals; never general purpose inspection API.
  • IPluginTestHarness is not defined. Interfaces are justified only by multiple real implementations (TestServerPluginTestHarness vs InMemoryPluginTestHarness) today there is one composition path, so concrete class expresses that better.

Proposed Contract (H10 base, convenience only)

public abstract class AuthKitPluginTestBase<TPlugin>
    where TPlugin : IAuthKitPlugin
{
    protected PluginTestHarness Host { get; }
    protected TPlugin Plugin { get; }
}

Typical usage deletes test setup boilerplate:

public sealed class MyPluginTests : AuthKitPluginTestBase<MyPlugin>
{
    [Fact]
    public async Task Endpoint_Responds() =>
        Assert.Equal(HttpStatusCode.OK, (await Host.Client.GetAsync("/hello")).StatusCode);
}

AuthKitPluginTestBase is convenience wrapper, nothing more: it owns Host + Plugin and handles disposal. It must NOT grow into Test Framework exposing Host shaped API (Services, Client, Metrics, Tracing, Health, Configuration, Manifest, …) that would be second host API for tests.

L1 and L2 are one path, not two

Level Shape Use
L1 lightweight the same WebApplicationFactory<Program> test uses only .Services, never CreateClient()/HTTP fast tests of DI/lifecycle/health without transport
L2 full HTTP the same WebApplicationFactory<Program> + CreateClient() round trip behavior, as in Host.IntegrationTests

L1 is not separate host building method. It is the same composition path with the HTTP call simply omitted at the test's discretion. There is exactly one host-building code path in the harness no alternative service registration or lifecycle implementation exists.

Decisions

  • Harness lives in AuthKit.Plugins.Testing, not AuthKit.Plugins.Abstractions same boundary rule as H6 (tooling/test support outside the runtime contract).
  • PluginTestHarness is concrete, sealed. No interface until second real implementation appears.
  • Extract, do not invent. Start from the merged AuthKitWebApplicationFactory the production Program remains the single composition used by both host and harness. Risk of prod/test drift is eliminated because there is no second implementation to drift away.
  • No DiagnosticsSnapshot until a real test needs it abstracation follows evidence.
  • The composition stays visible (Services/Client reachable): a green harness test does not hide behavior that would differ in the real host.
  • Each test gets a fresh harness. PluginTestHarness instance and its in-memory standins (eg. InMemoryKeyStoreRepository) are per instance, never static/shared no cross test state within the same class unless explicitly shared via IClassFixture.
  • Scope boundary: PluginTestHarness provides reusable host backed plugin tests (Program + DI + lifecycle + middleware + TestServer). It does not eliminate dedicated end to end tests that exercise the production deployment boundary or external infrastructure (real deployment, real DB, Keycloak, network).
  • Code status: neither type exists the assembly and harness extract the existing integration-test setup.

Implementation order

  1. Extract first, prove at zero behavior change: refactor Host.IntegrationTests to consume AuthKit.Plugins.Testing.PluginTestHarness (no Diagnostics, no formal L1) DevTokens load, lifecycle, and health tests stay green without changing assertions. If the extraction needs ~800 lines, that is a signal the shape is wrong; a handful of types (PluginTestHarness, base, a couple of helpers) is the right proportion.
  2. Then add AuthKitPluginTestBase<TPlugin> as the convenience wrapper over the extracted harness.
  3. Then write the first real plugin test using the base class, as the proof of adoption.

Validation

  • PluginTestHarness reuses the real host composition (same Program setup as AuthKitWebApplicationFactory); no duplicated registration/mapping/lifecycle/health code path.
  • No IPluginTestHarness in AuthKit.Plugins.Abstractions; harness/TestBase live only in AuthKit.Plugins.Testing.
  • Host.IntegrationTests refactored to consume AuthKit.Plugins.Testing (DevTokens load, lifecycle, health tests still green extract, don't invent).
  • Client.GetAsync(...) HTTP assertions work through the real pipeline no endpoint abstraction.
  • L1 and L2 share the same composition path: L1 is L2 with the HTTP call omitted both run against the identical Services graph (same registered types, same lifetimes).
  • Each test receives a fresh PluginTestHarness with clean in-memory store no shared state between tests in the same class except via explicit IClassFixture.
  • No DiagnosticsSnapshot in the first iteration added only as follow up driven by real test requirement.
  • AuthKitPluginTestBase<TPlugin> is convenience wrapper only async disposal correct (no leaked containers).

Acceptance Criteria

  • A plugin test runs over the same composition as production (AuthKitPluginTestBase<TPlugin> gives host + plugin with minimal boilerplate).
  • The real host remains the single composition implementation; the harness is thin DX layer over it, proven by the Host.IntegrationTests refactor.
  • No test/runtime API is added to AuthKit.Plugins.Abstractions; no diagnostics snapshot without a real consumer.
  • Existing Host.IntegrationTests behavior is preserved after extraction.

Non Goals

  • No IPluginTestHarness (no interface without second implementation).
  • No test machinery in AuthKit.Plugins.Abstractions.
  • No second, parallel host composition implementation.
  • No separate L1 host building code path L1 is L2 minus the HTTP client call.
  • No DiagnosticsSnapshot / general-purpose host inspection API in the harness contract.
  • No AuthKitPluginTestBase that re-exposes host shaped surface (Services, Metrics, Health, Manifest, …).
  • No full integration/E2E replacement the production deployment boundary and external infrastructure remain the domain of dedicated end to end tests.
  • No premature abstraction: H9/H10 proceeds because the Host.IntegrationTests extraction point already exists, not as speculative scaffolding for future plugins.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1Core operationadditiveAdditive, non-breaking changearea/abstractionsAuthKit.Plugins.Abstractions contractsub-taskChild task of an epic

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions