[feat][pip] PIP-478: Asynchronous v5 client auth plugin interfaces and TLS material provider plugin interface

[lhotari]

Motivation

This PIP proposes, for the Pulsar 5.0 v5 client (PIP-466) plus two new dependency-light SPI
modules — pulsar-tls-factory-api (TLS) and pulsar-http-client-api (HTTP client):

• an asynchronous, capability-segregated v5 client authentication SPI — replacing v4's
  kitchen-sink AuthenticationDataProvider, so credential I/O (OAuth2 / Athenz token
  refresh) never blocks the Netty event loop;
• a framework-managed PulsarHttpClient SPI — so auth plugins share the client's HTTP
  runtime instead of spinning up private AsyncHttpClient instances (issues 24795 and 24944);
• a purpose-driven PulsarTlsFactory TLS SPI that replaces PIP-337 — decoupled from the
  auth SPI, KMS/HSM-friendly (keys never cross a Pulsar API), with rebuild-not-mutate rotation.

It also folds in the full PIP-337 removal: the sslFactoryPlugin / sslFactoryPluginParams
(and brokerClient*) config keys, the v4 ClientBuilder / PulsarAdminBuilder methods, and the
ClusterData fields are removed in 5.0 (a stale config key is rejected at startup), with a
tlsFactoryClassName / tlsFactoryConfig successor — now on the v4 client and admin builders as
well as the server configuration.

TLS transport configuration and FIPS 140-3

This PIP provides the v5 client and server-side (broker / proxy / websocket / functions-worker)
TLS transport configuration, built on the PulsarTlsFactory SPI and the flat TlsPolicy. Beyond
cert/key/trust material, ciphers/protocols, and hostname verification, it makes the TLS engine
(JDK vs native OpenSSL) and — via TlsPolicy.jcaProvider — any JCA security provider
configurable per component. This lets the TLS transport run under FIPS 140-3, including higher
levels:

• Level 1 (software cryptographic module, e.g. BC-FIPS): pin the FIPS-140-validated JCA
  provider via TlsPolicy.jcaProvider, which forces the JDK TLS engine (never the non-validated
  native OpenSSL/BoringSSL engine). File-based PEM keys are handed to the provider — permitted at
  Level 1, where plaintext key (CSP) entry within the operational environment is allowed.
• Level 3 (hardware module / HSM): the decoupled SPI lets a custom PulsarTlsFactory build the
  TLS context against an HSM — e.g. sourcing the key from a PKCS#11 KeyStore — so the key
  material never crosses the module boundary and the private key is never read into Pulsar from a
  file. That satisfies the Level 3 rule that plaintext key (CSP) entry/output use physically-separated
  ports or a trusted channel, by keeping the key inside the module.

The engine selection is wired through every server component and the client, so no listener is pinned
to a non-validated native engine. The broader FIPS-compliance mode — approved algorithms across
message encryption and authentication, a FIPS distribution/packaging variant, a fail-fast validation
switch — is out of scope; this PIP covers the TLS-transport slice.

Links

Rendered PIP document:
https://github.com/lhotari/pulsar/blob/lh-pip-478-v5-client-async-auth-and-tls-material-provider/pip/pip-478.md

Discuss thread:
https://lists.apache.org/thread/s9n9jksr9vqgn9o982zmnnkcxdcncy3f

[lhotari]

The PIP will be simplified (the PulsarTlsMaterialProvider part). I haven't yet updated the PIP document.