feat(kernel): JWT private-key M2M auth on useKernel - #504
Conversation
Add JWT private-key client-assertion auth (RFC 7523) to the kernel backend. On `authType: 'databricks-oauth'`, supplying `oauthJwtKeyFile` selects the JWT flow: the kernel signs a short-lived assertion with the private key instead of sending a client secret and owns the token lifecycle (`authMode: 'OAuthM2mJwt'`). - KernelAuth: new JWT branch in buildKernelConnectionOptions (checked before the U2M/M2M-secret split; a private-key file is unambiguous JWT M2M intent), plus the OAuthM2mJwt native option shape. Requires oauthClientId + oauthJwtKid; optional oauthJwtPassphrase / oauthJwtAlgorithm / oauthScopes / tokenUrl. Mutually exclusive with oauthClientSecret. Also threads tokenUrl through the existing M2m branch. - IDBSQLClient: new oauthJwt* + tokenUrl fields on the databricks-oauth ConnectionOptions member. - DBSQLClient: on the useKernel path, do not build the connector's own OAuth provider (it eagerly starts the U2M browser flow / M2M exchange before the kernel is consulted); hand over a minimal PAT provider only when a token is present. Mirrors the Python connector. - tests: 9 unit tests for JWT routing / precedence / validation. Verified end-to-end: SELECT 1 via useKernel against an Azure Databricks warehouse, authenticated by Entra ID with a JWT private-key assertion (tokenUrl pointed at the Entra token endpoint). Requires a @databricks/databricks-sql-kernel build with JWT + tokenUrl support (kernel PRs #249 merged, #275 for tokenUrl). Signed-off-by: Rahul Singhal <rahul.singhal@databricks.com>
There was a problem hiding this comment.
Verdict: 2 Low
Solid, well-documented addition; the JWT branch validation and precedence logic look correct and mirror the existing M2M/U2M handling. Two low-severity notes: telemetry authType misclassifies JWT M2M as external-browser (since it keys off oauthClientSecret), and the connect() auth-provider behavior change lacks direct unit coverage. Minor consistency gap worth noting: the PAT branch's ambiguity guard rejects token + oauthClientId/oauthClientSecret but not token + oauthJwtKeyFile, so a JWT key silently drops when authType: 'access-token' is used.
| // racing — and conflicting with — the kernel's auth. So for `useKernel` we | ||
| // hand over only a minimal PAT provider when a `token` is present, and | ||
| // `undefined` otherwise. Mirrors Python's use_kernel auth-provider handling. | ||
| if (internalOptions.useKernel) { |
There was a problem hiding this comment.
🔵 Low — The new JWT private-key M2M flow is not reflected in telemetry authType. mapAuthType (same file, called unconditionally at the top of connect()) keys off oauthClientSecret === undefined ? 'external-browser' : 'oauth-m2m'. For the JWT path oauthClientSecret is (and must be) undefined, so every JWT M2M kernel connection is reported to telemetry as external-browser (i.e. U2M browser flow) — the opposite of its actual machine-to-machine nature. Consider distinguishing the JWT case (e.g. presence of oauthJwtKeyFile) so telemetry attribution is accurate.
| // hand over only a minimal PAT provider when a `token` is present, and | ||
| // `undefined` otherwise. Mirrors Python's use_kernel auth-provider handling. | ||
| if (internalOptions.useKernel) { | ||
| const { token } = options as { token?: string }; |
There was a problem hiding this comment.
🔵 Low — This is a behavior change to connect() — on the useKernel path the connector now skips createAuthProvider entirely and installs a PAT-only provider (or undefined). The new unit test file only exercises buildKernelConnectionOptions; there is no coverage asserting that (a) a useKernel OAuth/JWT connection ends up with authProvider === undefined (no eager browser flow), and (b) a useKernel connection with a token still gets a PlainHttpAuthentication provider. Since the stated motivation is preventing a spurious browser listener, a regression test guarding that behavior would be valuable.
What
Adds OAuth machine-to-machine auth with a JWT private-key client assertion (RFC 7523) on the kernel backend (
useKernel: true). The kernel signs a short-lived JWT with the service principal's private key instead of sending a client secret, and owns the token lifecycle; the workspace's OAuth IdP verifies it against the SP's registered public key.Companion to the kernel-side feature (databricks-sql-kernel #249; napi
token_urlin #275) and the parallel databricks-sql-python / databricks-sql-go changes.How
lib/kernel/KernelAuth.ts— new JWT branch inbuildKernelConnectionOptions(checked before the U2M/M2M-secret split; a private-key file is unambiguous JWT M2M intent) plus theOAuthM2mJwtnative-option shape. RequiresoauthClientId+oauthJwtKid; optionaloauthJwtPassphrase/oauthJwtAlgorithm/oauthScopes/tokenUrl. Mutually exclusive withoauthClientSecret. Also threadstokenUrlthrough the existing M2M branch.lib/contracts/IDBSQLClient.ts— newoauthJwt*+tokenUrlfields on thedatabricks-oauthConnectionOptionsmember.lib/DBSQLClient.ts— on theuseKernelpath, do not build the connector's own OAuth provider. It eagerly starts the U2M browser flow / M2M token exchange at connect time (a telemetry / feature-flag client callsauthProvider.authenticate()) before the kernel is consulted — which, for the no-secret JWT case, launched a spurious browser listener. Hand over a minimal PAT provider only when atokenis present. Mirrors the Python connector'suse_kernelhandling.Usage
Testing
tests/unit/kernel/auth-m2m-jwt.test.ts(routing, precedence, required-field validation, ambiguity guards); full kernel unit suite 318 passing. prettier + eslint clean.SELECT 1→[{"n":1}], with the client's backend asserted to beKernelBackend(kernel path, not Thrift).Requires a
@databricks/databricks-sql-kernelbuild with JWT +tokenUrlsupport.This pull request and its description were written by Isaac.