feat(node): expose OTLP log and metric APIs - #782
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
6ff9a0f to
913e647
Compare
1308886 to
2c3e3da
Compare
2c3e3da to
aaefe36
Compare
License DiffCompared against Lockfile license changesLockfile License ChangesRustAdded
Removed
Updated/Changed
NodeAdded
Removed
Updated/Changed
PythonAdded
Removed
Updated/Changed
Status output |
aaefe36 to
d0e303e
Compare
d0e303e to
4fa3144
Compare
4fa3144 to
ebe0537
Compare
ebe0537 to
eeea876
Compare
eeea876 to
69b24b0
Compare
69b24b0 to
435e7a4
Compare
|
Stack handoff from #780: this PR owns the Node pending-mark parity for |
435e7a4 to
0f3e918
Compare
0f3e918 to
05bb2f3
Compare
|
Follow-up for Observability V4: please update Node's typed helper surface with the new |
#### Overview Add the Rust source-of-truth APIs and runtime support for independent OTLP log and metric pipelines. This is stack PR 1 of 5. ##### Stack navigation 1. **[#780 — Core runtime, shared event model, config v4, and CLI](#780) — this PR** 2. [#781 — Python and PyO3 bindings](#781) 3. [#782 — Node.js and N-API bindings](#782) 4. [#783 — C FFI and Go bindings](#783) 5. [#779 — Dynamic native/gRPC plugins and consolidated docs](#779) **Position:** 1 of 5 · **GitHub base:** `main` · **Logical predecessor:** `main` · **Layer-only diff:** [compare branches](main...bbednarski/otel-signals-core) · **Next:** [#781](#781) All five PRs target `main`. Their branches are cumulative: this PR includes every preceding layer until those PRs merge and this branch is rebased onto the updated `main`. The GitHub **Files changed** tab therefore shows the cumulative diff. Use the layer-only comparison above to review only the code introduced by this layer. Review and merge in order: [#780](#780) → [#781](#781) → [#782](#782) → [#783](#783) → [#779](#779). After each merge, rebase the next branch onto the updated `main`; its PR remains targeted at `main` and its cumulative diff contracts to the remaining layers. - [x] I confirm this contribution is my own work, or I have the right to submit it under the project's license. - [x] I searched existing issues and open pull requests, and this does not duplicate existing work. #### Details - Add typed log severity, metric measurements, metric envelopes, optional mark `data_schema` and `severity`, and the Rust `metric` API. - Keep `MetricMeasurement` as the serde-facing wire DTO. Each metric envelope is parsed atomically into `ValidatedMetricMeasurement` values before it reaches the OTLP metrics exporter. - Make validation local to the parsed domain types: `InstrumentName` enforces the OTel instrument-name grammar and case-insensitive canonical key; `MetricValue` carries only `U64`, `I64`, or finite `F64`; `HistogramBoundaries` checks finite, strictly increasing, bounded buckets; and `MetricAttributes` owns scalar or homogeneous primitive-array parsing. - Build `InstrumentDescriptor` from the typed fields and enforce supported `kind × value` combinations once. Envelope consistency groups descriptors by canonical name and requires stable kind and unit; description and histogram boundaries are retained as non-identifying advisory fields. - Classify reserved metric marks before signal export. An invalid envelope is rejected atomically with a runtime diagnostic; a valid envelope carries only typed measurements into the metric registry and recorder. - Use the validated descriptor to construct cached OTLP instruments and convert typed attributes directly to OpenTelemetry values. The recorder contains no JSON validation branches; impossible `kind × value` pairs are guarded as internal invariants. - Export sanitized non-metric marks as structured OTLP logs with severity filtering and scope correlation. - Add observability config version 4, signal-specific endpoint derivation and validation, lifecycle handling, diagnostics, layering, generated schema, and CLI editor support. - Continue accepting version 3 as trace-only and preserve existing trace behavior for non-metric marks. - Include minimal internal Python, Node.js, and FFI test bridges so this base commit remains workspace-buildable; their public APIs are reviewed in later stack PRs. ### Metric data-model architecture ```mermaid flowchart LR subgraph Wire["Untrusted wire / serde boundary"] JSON["Metric mark JSON"] Envelope["MetricEnvelope"] WireMeasurement["MetricMeasurement<br/>name · kind · value_type · JSON value<br/>unit · description · boundaries · JSON attributes"] JSON --> Envelope --> WireMeasurement end subgraph Parse["Single parsing and validation boundary"] Convert["ValidatedMetricMeasurement::try_from(&MetricMeasurement)"] Name["InstrumentName<br/>OTel grammar + canonical name"] Value["MetricValue<br/>U64 | I64 | F64(FiniteF64)"] Descriptor["InstrumentDescriptor<br/>name · kind · unit · description · boundaries"] Bounds["HistogramBoundaries<br/>finite · strictly increasing · ≤ limit"] Attributes["MetricAttributes<br/>BTreeMap<String, AttributeValue>"] AttrValue["AttributeValue<br/>scalar or homogeneous typed array"] WireMeasurement --> Convert Convert --> Name Convert --> Value Convert --> Bounds Bounds --> Descriptor Name --> Descriptor Convert --> Attributes Attributes --> AttrValue end subgraph EnvelopePolicy["Envelope-level policy"] Parsed["Vec<ValidatedMetricMeasurement>"] Consistency["Canonical-name descriptor consistency<br/>kind + unit + value type<br/>(description/boundaries advisory)"] Convert --> Parsed --> Consistency end subgraph Export["OTLP exporter: typed inputs only"] Classify["MetricMarkClassification::Valid"] Registry["Instrument registry / cached OTLP instrument"] Record["record_measurement<br/>matches typed MetricValue"] OTLP["OpenTelemetry metrics export"] Consistency --> Classify --> Registry --> Record --> OTLP Attributes --> Record Descriptor --> Registry Value --> Record end Invalid["MetricMarkClassification::Invalid<br/>parse/validation error"] Convert -. failure .-> Invalid Consistency -. failure .-> Invalid ``` Validation: - `cargo fmt --all` - `cargo clippy --workspace --all-targets -- -D warnings` - Focused metric-model and OTLP metrics tests, plus the full Rust workspace, Python, Node.js, and Go/FFI validation matrix, passed during implementation. - `uv run pre-commit run --all-files` passed except the repository's `python-worker-proto-check`, which requires the unavailable `just` executable. Breaking changes: none for version 3 trace configuration or existing trace subscriber APIs. #### Where should the reviewer start? Start with `crates/types/src/api/event.rs`: it contains the wire DTO, parsed-domain types, and atomic envelope parser. Then review `crates/core/src/observability/otel_signal.rs` for classification and `crates/core/src/observability/otel_metrics.rs` for typed instrument registration and recording. Configuration versioning and endpoint derivation live in `crates/core/src/observability/plugin_component.rs`. #### Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to) - Relates to: none ## Summary by CodeRabbit * **New Features** * Added OpenTelemetry log and metric exporting with configurable endpoints, transports, batching, filtering, resource metadata, and delivery controls. * Added validated metric events with measurements, attributes, histograms, limits, and temporality options. * Added typed log severity and data schema support for emitted events. * Observability configuration now defaults to version 4 while retaining version 3 trace-only compatibility. * **Bug Fixes** * Improved handling and diagnostics for invalid events, export failures, queue drops, and shutdown errors. * Preserved metadata and severity when emitting tool and LLM events. Authors: - Bryan Bednarski (https://github.com/bbednarski9) Approvers: - Eric Evans II (https://github.com/ericevans-nv) - Maryam Najafian (https://github.com/mnajafian-nv) - Will Killian (https://github.com/willkill07) URL: #780
05bb2f3 to
4b6d4b6
Compare
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
4b6d4b6 to
fe0b6d4
Compare
Overview
Expose the core OTLP log and metric contracts through the Node.js binding.
This PR is independently rebased on
mainafter #780. It contains only the Node.js/N-API layer and has no branch dependency on the Python, FFI/Go, or dynamic-plugin/docs follow-up PRs. Review and merge it on its own readiness.Details
dataSchemaandseveritymark arguments and themetric()API.Validation:
cargo check -p nemo-relay-node --all-targetsBreaking changes: none.
Where should the reviewer start?
Start with
crates/node/src/types/mod.rsfor the public types andcrates/node/src/api/mod.rsfor mark emission and subscriber lifecycle behavior.Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)