[Feature] Plugin ccontract documentation, benchmarking and packaging

[rian-be]

Summary

Extend AuthKit.Plugins.Abstractions with complete, first class observability, developer experience, and testing surface covering section H: Observability, DX and tests.

The contract introduces additive observability hooks (GetMetrics, GetOpenTelemetrySources, ConfigureLogging, GetTracingTags), declarative documentation metadata (PluginDocumentation, fed by a marker-driven XML-doc extractor that is build/tooling, not runtime contract), developer scaffolding (samples/ plus dotnet new authkit-plugin template), a focused testing layer (PluginTestHarness / AuthKitPluginTestBase in dedicated AuthKit.Plugins.Testing), and the publishing/operations loop (contract README, Keycloak/OAuth2 example, loading benchmark, NuGet versioning). The host owns sinks, tracers, and doc rendering; plugins only declare their signal surface and own their metadata/DTOs.

Goal

Provide complete, first class implementation of the "Observability, DX and tests" area in the plugin contract, so that plugins are diagnosable, documented, testable, and easy to start from.

The implementation must allow the host to:

• register metrics declared by plugins via GetMetrics(),
• connect plugin OpenTelemetry sources to the host tracer via GetOpenTelemetrySources(),
• honor plugin logging configuration via ConfigureLogging,
• attach plugin provided tags to host spans via GetTracingTags(),
• render plugin documentation from serializable PluginDocumentation,
• produce plugin docs automatically from compiled XML doc,
• reference runnable sample plugin and scaffold new plugin in one command,
• run plugins inside an in-memory test host for assertions,
• read single, example backed contract README and realistic host loaded OAuth2/OIDC sample,
• measure plugin loading cost against repeatable benchmark baseline,
• and ship consistently versioned AuthKit.Plugins.Abstractions package.

Problem

There is no consistent logging/metrics/tracing surface on the plugin contract, no structured documentation model, and no test templates or sample plugin, making onboarding and diagnostics hard. Plugin telemetry, docs, and harnesses are reassembled per project, the host cannot discover what sources to connect or which tags to attach, and starting plugin requires manual project setup.

Scope

Implementation is divided into the following sub issues:

• [Task] Extend IAuthKitPlugin with Observability Hooks #39
• [Task] Add declarative plugin documentation metadata and XML doc extraction #40
• [Task] Add plugin testing harness and test base #41
• [Task] Close plugin contract loop: Documentation, Samples, Benchmarking and NuGet Release #42

---

Specification

ID	Element	Signature / Behavior	Files	Acceptance
H1	GetMetrics() hook	IEnumerable<MetricDefinition> GetMetrics() declares metric descriptors (name, description, unit)	IAuthKitPlugin.cs, new MetricDefinition.cs	host registers the returned metrics
H2	GetOpenTelemetrySources()	IEnumerable<string> GetOpenTelemetrySources() names OTel sources the host connects to its tracer	IAuthKitPlugin.cs	host connects to the tracer using the returned sources
H3	ConfigureLogging	void ConfigureLogging(ILoggingBuilder) registers plugin filters/loggers on the host logging builder	IAuthKitPlugin.cs	filters/loggers registered
H4	GetTracingTags()	IReadOnlyDictionary<string,string> GetTracingTags() supplies tags attached to host spans	IAuthKitPlugin.cs	tags added to spans
H5	Documentation property	PluginDocumentation Documentation { get; } declarative, JSON serializable documentation metadata (accessor only, no runtime generation default new())	IAuthKitPlugin.cs, new PluginDocumentation.cs	metadata shape round trips to JSON
H6	XML doc extraction	build/tooling generator populating PluginDocumentation from compiled XML doc, marker driven (no heuristic mapping)	tool outside AuthKit.Plugins.Abstractions	deterministic extraction feeds catalog/docs, not part of the runtime contract
H7	Sample plugin (samples/HelloAuthKitPlugin)	working plugin exercising A–G (metadata, hooks, middleware, health, security schemes)	samples/ (new project)	builds and loads
H8	dotnet new authkit-plugin template	template emitting compilable plugin skeleton referencing the abstractions	templates/ (new)	dotnet new creates a compilable project
H9	PluginTestHarness	concrete, sealed harness (no interface) reusing the real Program composition via WebApplicationFactory exposes Services + Client (real HTTP pipeline) no DiagnosticsSnapshot until real test needs it	new AuthKit.Plugins.Testing assembly	thin test support layer nothing mirrored in AuthKit.Plugins.Abstractions
H10	AuthKitPluginTestBase<TPlugin>	base class combining harness + plugin under test removes per test setup boilerplate	AuthKit.Plugins.Testing	simplifies writing tests
H11	Contract README	single readable entry point with minimal plugin example (metadata, health, middleware, security scheme, lifecycle) plus pointers to H7/H12; no prose duplication of H7/H6	README.md (new)	primary human readable contract reference
H12	OAuth2/OIDC reference sample	sample plugin under samples/ declaring realistic AuthKitSecuritySchemeDescriptor (Section E: E3/E5/E9) and loading in the host plugin only, not an IdP	samples/	loads via plugin discovery PluginContractValidator passes
H13	Loading benchmark	repeatable benchmark over defined plugin/assembly matrix (N plugins, N assemblies, cold/warm), recording a baseline with documented environment	benchmarks/ (new)	repeatable baseline recorded for the defined matrix
H14	NuGet versioning / release pipeline	exactly one authoritative version source (e.g. Directory.Build.props) -> dotnet pack -> tag -> changelog; no drift standardizes the packaging pipeline, not API surface	.csproj, Directory.Build.props	no version drift consistently versioned package

---

H1. Metrics Hook

Plugins declare the metrics they emit the host registers them. The contract does not choose backend.

Requirements

• GetMetrics() returns IEnumerable<MetricDefinition> with name, optional description, and optional unit.
• Host registers every returned metric at startup and exposes it through its configured sink.
• Default implementation returns an empty collection existing plugins are unaffected.

---

H2. OpenTelemetry Sources

Plugins name the OpenTelemetry sources their instrumentation uses; the host connects them to its tracer.

Requirements

• GetOpenTelemetrySources() returns source/tracer names.
• Host connects each returned source to its tracing pipeline; unknown or absent sources are ignored safely.
• Default implementation returns an empty collection.
• No OpenTelemetry SDK dependency is mandated on the contract itself.

---

H3. Logging Configuration

Plugins configure the host's ILoggingBuilder, not a separate pipeline.

Requirements

• ConfigureLogging(ILoggingBuilder) registers filters, providers, or log level rules.
• Configuration applies to the host logging pipeline and takes effect before plugin runtime work starts.
• Default implementation is no-op existing plugins compile unchanged.

---

H4. Tracing Tags

Plugins contribute tags to host spans without creating or owning spans.

Requirements

• GetTracingTags() returns an immutable IReadOnlyDictionary<string,string>.
• Host attaches the tags to the activity spans it creates for plugin operations.
• Plugins never create spans or parent context only tag contribution.

---

H5. Structured Documentation

Plugins possess structured documentation as declarative metadata the host and catalog can render they do not generate it at runtime.

Requirements

• PluginDocumentation is public record with plain serializable shapes (records, strings, collections).
• Documentation is an accessor-like property (default new()), never method that generates at runtime existing plugins compile and behave unchanged.
• Serializes to/from JSON without loss no doc rendering engine in the contract.
• Parameters documents what configuration entry means (descriptive, MVP dictionary) never actual configuration values.
• Present on IAuthKitPlugin only if the host consumes it as runtime metadata (catalog/admin); otherwise H6 tooling alone is sufficient.

---

H6. XML doc Extraction

A build/tooling concern, not part of the runtime plugin contract.

Requirements

• A tool/generator (outside AuthKit.Plugins.Abstractions) reads the compiled plugin's XML doc and populates PluginDocumentation.
• Mapping from XML doc to product sections (Setup, Security, Configuration, …) is marker-driven and deterministic explicit [PluginDocumentationSection("configuration")] markers (distinct from the PluginDocumentation DTO) or documented XML doc convention never inferred from class/member/parameter names.
• Marker is opt-in: marked element with missing XML doc -> build warning; XML doc without marker -> ignored (normal internal code never enters plugin docs).
• Unmarked code is ignored; output feeds JSON -> catalog / docs / UI.
• PluginDocumentation is consumed as static asset no runtime extraction.

---

H7. Sample Plugin

A runnable reference plugin exercising the full contract surface.

Requirements

• New samples/HelloAuthKitPlugin referencing AuthKit.Plugins.Abstractions.
• Implements A–G: [PluginMetadata] identity (A), configuration + lifecycle hooks (B), middleware/pipeline (C), structured health checks (D), security scheme descriptor + enums (E/F), discovery-compatible manifest behavior (G).
• Builds and loads into the host; passes PluginContractValidator.
• Separate project: never alters the core contract assembly or the host.

---

H8. dotnet new Template

A one command scaffold for compilable plugin.

Requirements

• New templates/ project with a dotnet new authkit-plugin template.
• Emits .csproj referencing the abstractions, [PluginMetadata], and minimal IAuthKitPlugin.
• Scaffolded project compiles and loads out of the box.
• Optional contract hooks appear as comments, not mandatory code.

---

H9. Test Harness

A thin, concrete test-support layer over the existing real host composition an extraction of AuthKitWebApplicationFactory (PR #35), not second host.

Requirements

• PluginTestHarness (concrete, sealed; no IPluginTestHarness interface) reuses the real Program composition through WebApplicationFactory/TestServer; only thin test overrides (e.g. in-memory keystore stand-in).
• Exposes narrow surface for the first iteration: Services (DI assertions) and Client (real HTTP pipeline Client.GetAsync("/hello")). No abstract "endpoint collection", no full host mirror, no DiagnosticsSnapshot (added only as follow up when real test needs stable plugin observable signal).
• IAsyncDisposable with StartAsync/StopAsync; fresh instance (and fresh in-memory store) per test no cross-test shared state unless explicitly via IClassFixture.
• L1 and L2 are one path: L1 is the same WebApplicationFactory<Program> with HTTP omitted (Services only), no separate host building code path.
• Lives in AuthKit.Plugins.Testing, not the runtime abstractions.

---

H10. Test Base

A base class combining harness + plugin under test.

Requirements

• AuthKitPluginTestBase<TPlugin> provides Host + Plugin with minimal boilerplate.
• The composition stays visible (Services/Client reachable) so green harness test does not hide behavior that would differ in the real host.
• Convenience wrapper only never host shaped Test Framework™ surface (Services/Metrics/Health/Manifest…).
• Provides reusable host backed plugin tests (Program + DI + lifecycle + middleware + TestServer) it does not eliminate dedicated end to end tests exercising the production deployment boundary or external infrastructure (real DB, Keycloak, network).

---

H11. Contract README

A single readable entry point for the contract.

Requirements

• New README.md in the abstractions area: one minimal plugin example (metadata, health, middleware, security scheme, lifecycle as applicable) of 30–80 lines a short, copyable snippet, not full per mechanism implementations.
• Pointers to HelloAuthKitPlugin (H7) for the full example and to the Keycloak sample (H12) for realistic scenario.
• Complements (does not duplicate) the XML doc generated API reference (H6) and does not become second H7.
• Examples reflect the implemented contract surface and load correctly.

---

H12. OAuth2/OIDC Reference Sample

A realistic host loaded plugin showing how the contract surface is used for OAuth2/OIDC.

Requirements

• Sample plugin under samples/ declaring realistic OAuth2/OIDC AuthKitSecuritySchemeDescriptor.
• Loads successfully via plugin discovery; passes PluginContractValidator.
• Depends on Section E (OAuth2 flows/URLs/issuer fields E3/E5/E9) ships with or after that descriptor work.
• The sample is plugin, not an IdP no Keycloak server is bundled a full IdP integration (Docker Compose + real token flow) is documented follow up, not part of this task.

---

H13. Loading Benchmark

A repeatable loading baseline for regression detection.

Requirements

• Defined plugin/assembly matrix (N plugins, N assemblies, cold/warm semantics) cold = fresh process/isolated iteration with plugin assemblies not yet loaded warm = subsequent loads under the same process conditions, so baseline read later means the same thing.
• Measures discovery vs assembly loading vs dependency resolution vs activation vs lifecycle independently where possible.
• Repeatable, with documented environment (runtime, OS, hardware) and recorded baseline.
• Regression threshold documented no automated performance gate in this task.
• Measurement only no micro optimization work in this task.

---

H14. NuGet Versioning / Release Pipeline

Single source of truth versioning for the publishable package.

Requirements

• The implementation selects exactly one authoritative version source (e.g. Directory.Build.props <Version>) flowing to .csproj, NuGet package, Git tag, and changelog no drift.
• dotnet pack produces a consistently versioned AuthKit.Plugins.Abstractions.
• Release tagging aligned with package version changelog generation documents per release changes.
• Breaking changes require major version bump (documented convention).
• Standardizes the packaging/release pipeline no API surface change.
• Sits at the boundary with future dedicated publishing section (next available letter: L) a larger packaging scope should move there. API compatibility/breaking change detection is documented follow up issue.

---

Architecture

flowchart TD
    SRC["Plugin implementation<br/>IAuthKitPlugin + hooks"] --> HOOKS["Observability collection<br/>H1–H4 GetMetrics / Sources<br/>ConfigureLogging / TracingTags"]
    HOOKS --> HOST["Host pipeline<br/>registers sinks + tracer + tags"]
    SRC --> DOC{"Documentation metadata<br/>H5 PluginDocumentation (declarative)<br/>+ H6 XML-doc extractor (tool)"}
    DOC --> CAT["Catalog / host rendering"]
    SRC --> TEST["Testability<br/>H9 PluginTestHarness<br/>H10 AuthKitPluginTestBase<br/>AuthKit.Plugins.Testing"]
    TEST --> P["real Program composition<br/>WebApplicationFactory/TestServer"]
    H8["dotnet new authkit-plugin H8"] --> SRC
    H7["samples/HelloAuthKitPlugin H7"] --> SRC
    SRC --> LOAD["Load timing H13<br/>benchmarks/baseline"]
    H12["Keycloak/OAuth2 sample<br/>needs E3/E5/E9"] --> SRC
    H14["NuGet versioning/release<br/>Directory.Build.props → pack → tag"] --> PUBLISH["AuthKit.Plugins.Abstractions package"]
    H11["Contract README"] --> DOCS
    CAT --> DOCS["Consumable docs / catalog"]
    classDef ok fill:#dcfce7,stroke:#22c55e,color:#14532d
    classDef dx fill:#dbeafe,stroke:#3b82f6,color:#1e40af
    classDef ops fill:#fef3c7,stroke:#f59e0b,color:#92400e
    class HOST,CAT,RUN,DOCS,LOAD ok
    class H7,H8,H12,SRC dx
    class H14,PUBLISH ops

Loading

Component	Responsibility
Observability hooks (H1–H4)	declare metrics/sources/logging/tags
Host pipeline	registers sinks, connects tracer, attaches tags
PluginDocumentation + XML-doc (H5–H6)	declarative metadata (contract) + marker driven extractor (tooling)
Sample + template (H7–H8)	reference implementation + one command scaffold
Harness + base (H9–H10)	thin test support layer over the real Program composition (extraction of AuthKitWebApplicationFactory)
README / Keycloak example (H11–H12)	readable entry point + realistic host loaded OAuth2/OIDC sample
Benchmark (H13)	repeatable loading baseline (defined matrix, documented environment)
NuGet versioning/release (H14)	single source of truth version, publishable package

---

Backward Compatibility

• Fully additive: every new hook is default interface implementation ({} / => []), so existing plugins compile and behave unchanged.
• Documentation is declarative metadata (PluginDocumentation, default new()) the XML doc extractor ships as build/tooling, not as an API in AuthKit.Plugins.Abstractions.
• No removal of IAuthKitPlugin or any existing member the four hooks extend it, never replace member.
• DevTokens/DevTools must compile and load without modification the sample/template/benchmark are separate projects and do not alter the core contract assembly.
• No new external dependency on the contract (no OpenTelemetry SDK, no metrics vendor, no doc engine).
• H12 additionally depends on Section E descriptor fields, which are additive on top of the existing AuthKitSecuritySchemeDescriptor.

---

Validation

• dotnet build AuthKit.Plugins.Abstractions (0 errors).
• tools/PluginContractValidator on DevTokens -> [PASS].
• Tests covering each subtask:
  • H1 declared metrics are registered by the host
  • H2 returned OTel sources connect to the tracer
  • H3 logging filters/loggers take effect
  • H4 tracing tags attach to host spans
  • H5 PluginDocumentation JSON round trip default new() object Parameters descriptive only
  • H6 marker driven XML doc extraction outside the abstractions marked+no-doc -> warning, marker is opt-in no naming inference
  • H7 sample builds and loads H8 dotnet new produces compilable project
  • H9 harness reuses the real host composition (no parallel implementation, L1 = L2 minus HTTP, fresh per test instance); H10 base removes boilerplate
  • H11 README is single readable entry point with minimal example + pointers to H7/H12 (no duplication) H12 host loaded OAuth2/OIDC sample loads
  • H13 repeatable benchmark with defined matrix and documented environment; H14 dotnet pack with a single source-of-truth version, no drift
• DevTokens still loads.

---

Acceptance Criteria

• Each subtask in Scope meets its own criteria (Acceptance column).
• Plugins declare metrics, OpenTelemetry sources, logging configuration, and tracing tags through additive default hooks.
• The host wires declared signals the contract never mandates vendor or SDK.
• Plugins possess structured, JSON serializable documentation metadata (new() default) XML doc extraction is tooling that fills it, outside the runtime contract.
• A runnable sample plugin and dotnet new authkit-plugin template exist.
• Plugins are testable through PluginTestHarness / AuthKitPluginTestBase<TPlugin> in AuthKit.Plugins.Testing, running over the real Program composition.
• The contract has single, readable, example backed README that is the primary entry point (no H7/H6 duplication) and realistic host loaded OAuth2/OIDC reference sample.
• A repeatable loading baseline exists (defined plugin/assembly matrix, documented environment) the abstractions package is publishable with single source of truth version.
• Contract compiles and passes validator existing plugins compile unchanged.

---

Non Goals

• No change to host runtime behavior beyond the new contract.
• No breaking changes (fully additive default hooks).
• No metric backend or sink selected by the contract.
• No mandatory OpenTelemetry SDK dependency on the contract.
• No documentation rendering engine in the contract.
• No runtime documentation generation by plugins documentation is declarative metadata.
• No heuristic XML doc -> section mapping extraction is marker driven tooling outside AuthKit.Plugins.Abstractions.
• No configuration values inside PluginDocumentation (descriptive Parameters only).
• No span creation by plugins trace tag contribution only.
• No full integration testing framework replacement; the harness is thin DX layer over the real host (Program stays the single composition), and L1 is L2 minus the HTTP client call (one code path).
• No IPluginTestHarness interface no test machinery or DiagnosticsSnapshot in AuthKit.Plugins.Abstractions.
• No Keycloak server bundled the sample is plugin, not an IdP (full IdP/Docker Compose integration is follow up).
• No micro optimization work H13 measures only (no automated performance gates in this task).
• H14 standardizes the packaging/release pipeline no API surface change, no API compat tooling in this task scope stays small because it sits at the boundary of future dedicated publishing section (L).
• No structured telemetry/metrics infrastructure H1–H4 only declare plugin signals (host pipeline is section C/observability infra).