Skip to content

feat: add gRPC mutual TLS support for the OAP channel - #146

Merged
wu-sheng merged 2 commits into
apache:masterfrom
songzhendong:feature/mtls-support
Sep 16, 2026
Merged

wu-sheng merged 2 commits into
apache:masterfrom
songzhendong:feature/mtls-support

Conversation

@songzhendong

Copy link
Copy Markdown
Contributor

Summary

Add gRPC mutual TLS (mTLS) support for the Node.js agent → OAP channel.

  • Upgrade gRPC channel credentials from createInsecure() to createSsl() when TLS is enabled.
  • Support one-way TLS (system CA or custom CA) and mTLS (CA + client key + cert chain).
  • Fail closed on incomplete or unreadable client certificate configuration.
  • Optional ssl_target_name_override / default_authority for hostname / SAN mismatch cases.
  • Add unit tests and a real-OAP + BanyanDB mTLS remote e2e suite.
  • Document configuration and local/CI certificate generation (private keys are never committed).

New configuration

Environment variable agent.start() option Description
SW_AGENT_SECURE secure Enable TLS (must be set explicitly)
SW_AGENT_SSL_TRUSTED_CA_PATH sslTrustedCaPath Trusted CA for OAP server verification
SW_AGENT_SSL_KEY_PATH sslKeyPath Client private key (mTLS; with cert chain)
SW_AGENT_SSL_CERT_CHAIN_PATH sslCertChainPath Client certificate chain (mTLS; with key)
SW_AGENT_SSL_TARGET_NAME_OVERRIDE sslTargetNameOverride Override SNI / authority when cert hostname ≠ dial address

Motivation

Align the Node.js agent with Java/Python collector TLS/mTLS capabilities so Agent → OAP traffic can use mutual authentication against OAP receiver-sharing-server.

Implementation notes

  • TLS material is read into memory and passed to grpc.credentials.createSsl(...) at channel build time.
  • Incomplete mTLS config (only key or only cert) throws; certificate paths present with secure=false also throw.
  • PKCS#1 and PKCS#8 private keys are accepted via Node OpenSSL / grpc-js.
  • Trace persistence in e2e is asserted with GraphQL queryTraces (Trace Query V2) against BanyanDB-backed OAP; flush HTTP 200 alone is not treated as proof because flush is best-effort.
  • Negative mTLS e2e accepts only explicit client-certificate-required TLS errors; a bare post-handshake close is treated as failure.

Test plan

  • Unit: tests/remote/TLSChannelBuilder.test.ts (credential branches and error paths)
  • Real OAP mTLS e2e (generate-certs.sh + npm run test tests/remote-e2e/mtls/)
  • CI: TestRemoteE2EMTLS on Node 20 / 22 / 24
  • Lint / License / Build matrix
  • Docs: docs/en/setup/configuration.md, docs/en/advanced/troubleshooting.md, tests/remote-e2e/mtls/README.md

Related

  • Java reference e2e: test/e2e-v2/cases/simple/mtls/ in apache/skywalking

Upgrade gRPC credentials from createInsecure() to createSsl() with
mutual authentication options. New environment variables:

  SW_AGENT_SECURE              enable TLS (fail-closed; must be set
                               explicitly)
  SW_AGENT_SSL_TRUSTED_CA_PATH trusted CA cert for OAP server verification
  SW_AGENT_SSL_KEY_PATH        client private key for mTLS
  SW_AGENT_SSL_CERT_CHAIN_PATH client certificate chain for mTLS
  SW_AGENT_SSL_TARGET_NAME_OVERRIDE
                               override SNI / authority when OAP cert
                               hostname does not match the network address

The implementation:
- Fails closed (throws) on incomplete or contradictory TLS config.
- Supports one-way TLS (system CA), custom CA, and mTLS (CA + key + cert).
- Accepts PKCS#1 and PKCS#8 private keys via Node crypto.
- Exposes grpc.ssl_target_name_override + grpc.default_authority
  for hostname mismatch scenarios.

Tests:
- 11 unit tests covering all credential branches and error paths.
- 2 E2E tests against a real OAP + BanyanDB container:
  (1) OAP rejects TLS without a client certificate — only explicit
      certificate-required alerts/codes pass; a bare post-handshake
      close is treated as failure, not success.
  (2) Agent reports traces through mTLS. listServices proves
      Management gRPC; queryTraces (v2) proves TraceSegment
      client-streaming persisted segments (flush HTTP 200 alone is
      insufficient because flush swallows report failures).
- Local/CI share tests/remote-e2e/mtls/generate-certs.sh so private
  keys are never committed and the suite can be run outside Actions.

CI generates fresh test certificates each run so no private keys are
committed to the repository.
@apache apache deleted a comment from tkNobug Sep 15, 2026
@apache apache deleted a comment from a526672351 Sep 15, 2026
@wu-sheng wu-sheng added this to the 0.10.0 milestone Sep 15, 2026
@wu-sheng wu-sheng added the enhancement New feature or request label Sep 15, 2026
Comment on lines +72 to +77
logger.debug(
'gRPC TLS credentials built: ca=%s key=%s cert=%s',
trustedCaPath ?? '(system)',
keyPath ?? '(none)',
certChainPath ?? '(none)',
);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor logging issue (P3): these new calls use printf-style %s placeholders, but createLogger() intentionally omits winston.format.splat(), so the certificate paths and hostname override are not included in the messages.

For example, with SW_AGENT_LOGGING_LEVEL=debug and sslTargetNameOverride: 'example.test', I reproduced:

gRPC TLS credentials built: ca=%s key=%s cert=%s
gRPC TLS hostname override set to [%s]

The second message should say:

gRPC TLS hostname override set to [example.test]

Could these calls use template strings or structured metadata? For example, the hostname message can be:

logger.debug(`gRPC TLS hostname override set to [${sslTargetNameOverride}]`);

The credential-path message needs the same adjustment. This only affects troubleshooting output; TLS/mTLS behavior is unaffected.

createLogger intentionally omits winston.format.splat(), so printf-style
%s placeholders were logged literally. Use template strings so ca/key/cert
paths and hostname override appear in debug output.
@wu-sheng
wu-sheng merged commit d7f266f into apache:master Sep 16, 2026
43 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants