Skip to content

feat(cli): serve Prometheus metrics from the CLI on a separate admin port #94

Description

@harrydayexe

Summary

Wire the OpenTelemetry SDK and Prometheus exporter into the ./cli module and serve /metrics on a separate admin port, so a containerised GoBlog can be scraped by Prometheus and rendered in Grafana out of the box.

This is the payoff issue for #92 and #93. It depends on both.

Dependencies

Do not start this before both have landed. Adding the exporter while the CLI is still in the root module would pull client_golang, prometheus/common, protobuf, golang.org/x/net, oauth2 and procfs into every library consumer's module graph — the exact outcome #93 exists to prevent.

Design

Dependencies (CLI module only)

go.opentelemetry.io/otel/sdk/metric
go.opentelemetry.io/otel/exporters/prometheus
github.com/prometheus/client_golang   // via the exporter, for promhttp.HandlerFor

Measured at ~48 modules total. All of it confined to cli/go.mod.

Wiring

Construct a sdkmetric.MeterProvider with the Prometheus exporter as its reader, and pass it to the server:

exporter, err := prometheus.New()
if err != nil {
    return err
}
mp := sdkmetric.NewMeterProvider(sdkmetric.WithReader(exporter))
defer mp.Shutdown(ctx)

cfg.Server = append(cfg.Server, config.WithMeterProvider(mp))

No changes to pkg/server should be required. If any turn out to be necessary, that indicates the #92 API was under-specified — raise it there rather than working around it here.

Admin listener

/metrics is served on its own http.Server on a separate port, never on the blog's public listener. Keeping it off the public port avoids exposing operational data to blog readers, avoids any interaction with BlogRoot path prefixing, and matches how this is normally deployed behind Kubernetes or a reverse proxy.

:8080  blog + /healthz/*
:9090  /metrics only

Requirements:

  • Opt-in. Off unless --metrics is passed; no admin listener is bound at all when disabled.
  • The admin listener participates in the same graceful shutdown path as the main server in Server.Run (pkg/server/server.go:293), including the 10s timeout.
  • A bind failure on the admin port must be surfaced clearly, not swallowed. Decide and document whether it is fatal — suggest fatal, since silently serving without metrics an operator believes are running is worse than failing loudly.
  • mp.Shutdown is called on exit so final measurements flush.

CLI flags

Flag Default Notes
--metrics false Enables collection and the admin listener
--metrics-port 9090 Admin listener port
--metrics-host inherit / "" Consider binding to localhost by default; see open question

Follow the existing flag conventions in cli/internal/server/flagConsts.go — a named constant per flag, registered in ServeCommand.

Docker

  • EXPOSE 9090 alongside the existing EXPOSE 8080.
  • Decide whether the image enables --metrics by default. The image already opts into --health-checks in its ENTRYPOINT, so there is precedent for the container being opinionated where the CLI is not. Suggest enabling it, since scraping is the main reason to run the container, and the port is not published unless the operator maps it.
  • Update justfile's run-image recipe to map the port.

Open questions

  1. Default bind address for the admin listener. Binding 0.0.0.0:9090 makes Docker and Kubernetes work with no extra flags, but on a bare-metal goblog serve --metrics it exposes metrics to the whole network. Binding localhost by default is safer but breaks the container case unless overridden. Suggest 0.0.0.0 with a documented warning, matching how most exporters behave, but flagging it as a deliberate choice rather than an oversight.
  2. Metrics in the container by default — see above.
  3. Grafana dashboard. Ship a reference dashboard JSON in-repo, or just document the queries? A dashboard is a nicer experience but becomes a maintenance surface. Suggest documenting queries first, adding a dashboard only if there is demand.

Acceptance criteria

  • --metrics starts an admin listener serving Prometheus-formatted output at /metrics.
  • Without --metrics, no admin port is bound and no measurements are recorded.
  • Exported metric names follow OTel HTTP server semantic conventions, so http_server_request_duration_seconds_count and friends appear as expected.
  • /metrics is never reachable on the blog's public port.
  • Both listeners shut down gracefully on SIGINT/SIGTERM/SIGHUP.
  • Root go.mod is unchanged by this issue — verify explicitly with git diff go.mod.
  • EXPOSE 9090 in the Dockerfile; just run-image maps it.

Testing

  • Integration test in integration/ (testcontainers is already set up there): start the container, request blog pages, scrape /metrics, assert the request count reflects the traffic generated.
  • Assert /metrics returns 404 or is unreachable on :8080.
  • Assert no admin port is bound when --metrics is absent.
  • Assert clean shutdown with both listeners active.

Documentation

Per CLAUDE.md:

Example queries to document

# Page hits per route
sum by (http_route) (rate(http_server_request_duration_seconds_count[5m]))

# Error rate
sum(rate(http_server_request_duration_seconds_count{http_response_status_code=~"5.."}[5m]))
  / sum(rate(http_server_request_duration_seconds_count[5m]))

# p95 latency by route
histogram_quantile(0.95,
  sum by (le, http_route) (rate(http_server_request_duration_seconds_bucket[5m])))

Compatibility

Non-breaking. Purely additive: new opt-in flags, a new port that is only bound when requested, and no change to the library or to default CLI behaviour.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestfeature-requestNew feature or improvement suggested by users

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions