Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions PROGRESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -424,6 +424,25 @@ deploy/scripts/run_e2e_sweep.sh --out-dir /tmp/sweep-$(date +%s) --soak-secs 120
commits and have been intentionally excluded from PRs since
#204. Decision still pending — either commit a clean bump as
its own PR or revert.
9. ~~**`asap/fake-exporter:dev` rebuild broken from upstream drift.**~~
**Done (2026-05-05).** The patched OTLP proto bindings (mpb.DDSketch /
KLLSketch / CountSketch / CountMinSketch / HLLSketch) were never
committed under `opentelemetry-proto-patch/gen/go/...`, so any rebuild
hit `undefined: mpb.*` symbols. Separately, the patch dir's transform
files referenced `metricdata.*EncodingGob` enum names that the metric-
data package had renamed to `*EncodingProto` / `*EncodingDelta`. Fixed:
(a) regenerated the Go bindings via the upstream Makefile recipe and
committed them under `opentelemetry-proto-patch/gen/go/`; (b) added a
`go.opentelemetry.io/proto/otlp` replace to `deploy/fake-exporter/go.mod`
pointing at the patch's gen tree; (c) updated the http+grpc transform
files to use the post-rename `*EncodingProto` / `*EncodingDelta`
metricdata enums and `*_ENCODING_PROTO` / `*_ENCODING_DELTA` mpb enums;
(d) updated `Dockerfile.fake-exporter` to copy `opentelemetry-proto/`
into the build context. Regen recipe lives at
`opentelemetry-proto-patch/REGEN.md`. The companion sketchlib-go PR #53
rename refactor (Add/Insert/InsertValue/EstimateCardinality/
GetValueAtQuantile → Update/UpdateValue/Estimate/Quantile) was already
absorbed into the patch dir before this round; the build verifies it.

---

Expand Down
23 changes: 17 additions & 6 deletions deploy/docker/Dockerfile.fake-exporter
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,19 @@
# aggregators — see docs/sdk-cost-evaluation.md) and
# the sketchlib-go sibling repo.
#
# ## Prereq: combine the patch onto the upstream tree
# ## Prereq: combine the patches onto the upstream trees
#
# ./restore_opentelemetry_go_patches.sh
# ./restore_otel_proto_patches.sh
#
# That walks the `opentelemetry-go-patch/` directory and copies its
# contents on top of `opentelemetry-go/`. The combined tree is what
# the go.mod replaces point at.
# Those walk the `opentelemetry-go-patch/` and `opentelemetry-proto-patch/`
# directories and copy their contents on top of `opentelemetry-go/` and
# `opentelemetry-proto/` respectively. The combined trees are what the
# `go.mod` replaces point at. The proto-patch carries the patched `.proto`
# sources AND the regenerated `gen/go/...` Go bindings (mpb.DDSketch,
# mpb.KLLSketch, mpb.CountSketch, mpb.CountMinSketch, mpb.HLLSketch types
# + the `MetricsService` gRPC client). See
# `opentelemetry-proto-patch/REGEN.md` for the regeneration recipe.
#
# ## Build
#
Expand All @@ -31,9 +37,14 @@
FROM golang:1.25-bookworm AS build
WORKDIR /src

# Main repo tree (contains opentelemetry-go with patches applied and
# deploy/fake-exporter source).
# Main repo tree (contains opentelemetry-go and opentelemetry-proto with
# patches applied and deploy/fake-exporter source). The replace directives
# in opentelemetry-go's exporter go.mod files point at
# ../../../../../opentelemetry-proto/gen/go/go.opentelemetry.io/proto/otlp,
# so opentelemetry-proto/ must be present alongside opentelemetry-go/ in the
# build context root.
COPY opentelemetry-go /src/opentelemetry-go
COPY opentelemetry-proto /src/opentelemetry-proto
COPY deploy/fake-exporter /src/deploy/fake-exporter

# Sketchlib-go pulled via named build context to avoid a submodule
Expand Down
5 changes: 5 additions & 0 deletions deploy/fake-exporter/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,9 @@ replace (
go.opentelemetry.io/otel/sdk => ../../opentelemetry-go/sdk
go.opentelemetry.io/otel/sdk/metric => ../../opentelemetry-go/sdk/metric
go.opentelemetry.io/otel/trace => ../../opentelemetry-go/trace
// The patched OTLP proto bindings (mpb.DDSketch / KLLSketch / CountSketch
// / CountMinSketch / HLLSketch types added on top of upstream v1.9.0)
// are regenerated under opentelemetry-proto/gen/go/... by
// restore_otel_proto_patches.sh. See opentelemetry-proto-patch/REGEN.md.
go.opentelemetry.io/proto/otlp => ../../opentelemetry-proto/gen/go/go.opentelemetry.io/proto/otlp
)

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions opentelemetry-proto-patch/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@
# Vim
.swp

# Generated code
/gen/
# Generated code: track the regenerated Go bindings under
# gen/go/go.opentelemetry.io/proto/otlp/. Other languages' gen/* output is
# ignored. See REGEN.md for the regeneration recipe.
/gen/*
!/gen/go/
/gen/go/*
!/gen/go/go.opentelemetry.io/
!/gen/go/go.opentelemetry.io/proto/
!/gen/go/go.opentelemetry.io/proto/otlp/
!/gen/go/go.opentelemetry.io/proto/otlp/go.mod

node_modules
86 changes: 86 additions & 0 deletions opentelemetry-proto-patch/REGEN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Regenerating the patched OTLP Go bindings

The ASAP-patched `metrics.proto` adds five sketch message types (`DDSketch`,
`KLLSketch`, `CountSketch`, `CountMinSketch`, `HLLSketch`) plus their
`*DataPoint` and `*Encoding` enums on top of upstream
[`opentelemetry-proto` v1.9.0]. The Go bindings under
`gen/go/go.opentelemetry.io/proto/otlp/` are committed to this repo so the
fake-exporter / sketchcol Docker builds don't have to reinvent a protoc
toolchain at build time. The `replace` directives in
`opentelemetry-go-patch/exporters/otlp/otlpmetric/{otlpmetricgrpc,otlpmetrichttp}/go.mod`
and `deploy/fake-exporter/go.mod` point at this gen tree.

## When to regenerate

Whenever any `.proto` file under `opentelemetry-proto-patch/opentelemetry/`
changes. Both the patched files (carried in this directory) AND any
upstream-only files transitively imported by them
(`common.proto`, `resource.proto`, etc.) must be passed to `protoc`.

## Recipe

The repo pins upstream proto to `v1.9.0` (commit
`a8951735f7801e8adfaec5c0ace9262771cfec6e`). The recipe overlays this
patch's `.proto` files on top of a clean v1.9.0 checkout, then runs
`protoc-gen-go` (with `plugins=grpc`) per upstream's Makefile target
`gen-go`.

```bash
# 1. Get a clean v1.9.0 of upstream opentelemetry-proto.
PROTO_BUILD=/tmp/proto-build
rm -rf "$PROTO_BUILD"
git clone --depth 1 --branch v1.9.0 \
https://github.com/open-telemetry/opentelemetry-proto.git "$PROTO_BUILD"

# 2. Overlay the patched .proto files.
REPO_ROOT="$(git rev-parse --show-toplevel)"
cp "$REPO_ROOT/opentelemetry-proto-patch/opentelemetry/proto/metrics/v1/metrics.proto" \
"$PROTO_BUILD/opentelemetry/proto/metrics/v1/metrics.proto"
cp "$REPO_ROOT/opentelemetry-proto-patch/opentelemetry/proto/collector/metrics/v1/metrics_service.proto" \
"$PROTO_BUILD/opentelemetry/proto/collector/metrics/v1/metrics_service.proto"

# 3. Run protoc-gen-go (with grpc plugin) via the otel/build-protobuf
# image — the same image upstream uses for `make gen-go`.
cd "$PROTO_BUILD"
rm -rf gen/go && mkdir -p gen/go
for f in $(find opentelemetry/proto -name '*.proto'); do
docker run --rm -u "$(id -u)" -v "${PWD}:${PWD}" -w "${PWD}" \
otel/build-protobuf:0.9.0 \
--proto_path="${PWD}" \
--go_out=plugins=grpc:./gen/go \
"$f"
done

# 4. Copy the generated tree back into the patch repo.
rm -rf "$REPO_ROOT/opentelemetry-proto-patch/gen/go/go.opentelemetry.io/proto/otlp"/*
cp -r gen/go/go.opentelemetry.io/proto/otlp/* \
"$REPO_ROOT/opentelemetry-proto-patch/gen/go/go.opentelemetry.io/proto/otlp/"

# 5. Confirm the new sketch types are present:
grep -c "DDSketchDataPoint\|KLLSketchDataPoint\|CountSketchDataPoint\|CountMinSketchDataPoint\|HLLSketchDataPoint" \
"$REPO_ROOT/opentelemetry-proto-patch/gen/go/go.opentelemetry.io/proto/otlp/metrics/v1/metrics.pb.go"
```

The `gen/go/go.opentelemetry.io/proto/otlp/go.mod` is hand-maintained
(modeled on upstream) and pins protobuf / gRPC / grpc-gateway versions
that match `deploy/fake-exporter/go.sum`. Update it only when those pins
shift (rare).

## Build wiring

`restore_otel_proto_patches.sh` (in the repo root) copies this entire
directory tree, including `gen/go/...`, on top of the
`opentelemetry-proto/` submodule. The Docker build for fake-exporter
copies `opentelemetry-proto/` into the build context, and
`deploy/fake-exporter/go.mod` has a
`replace go.opentelemetry.io/proto/otlp => ../../opentelemetry-proto/gen/go/go.opentelemetry.io/proto/otlp`
that picks up the regenerated bindings.

## Why the bindings are committed (not generated at build time)

- The Docker build runs `golang:1.25-bookworm` (no protoc, no Docker-in-
Docker for the otel/build-protobuf image). Pulling the toolchain
inside the build would add 200+ MB of layers and a Docker socket
mount. Easier to commit ~10 KLOC of generated `.pb.go`.
- Upstream `opentelemetry-proto` itself commits its `gen/...` outputs,
so we follow the same pattern.
Loading