Skip to content

Repository files navigation

alertyG

A small webhook bridge that receives Prometheus Alertmanager notifications and forwards them to Google Chat spaces as richly formatted cards.

Alertmanager POSTs a notification to alertyG, which routes it to the correct Google Chat space (based on an alert label), renders a configurable card template, and delivers it — with retries, throttling, and de-duplication of resolved alerts.

Features

  • Label-based routing — fan out alerts to different Google Chat spaces using the channel group label on each Alertmanager notification.
  • Configurable templates — Go text/template for both the message card and an optional header, with helpers for filtering firing/resolved alerts and formatting timestamps.
  • Asynchronous delivery — the HTTP handler responds 202 Accepted immediately; delivery (with retry + throttle) runs through a bounded per-route queue in the background so Alertmanager is never held open. When a route's queue is full, alertyG responds 429 so Alertmanager retries later instead of alerts being dropped.
  • Resilient sending — automatic retries with backoff and per-webhook minimum-interval throttling to respect Google Chat rate limits; each webhook gets its own delivery worker and queue, so one slow space never blocks the others.
  • Resolved de-duplication — a TTL-bounded tracker avoids re-sending "resolved" notifications for alerts already reported.
  • Structured logginglog/slog with selectable text/json output and configurable level; webhook URLs are redacted in logs.
  • Graceful shutdown — drains in-flight deliveries on SIGINT/SIGTERM.
  • Prometheus metrics — exposed at /metrics, including dispatches received/rejected, delivery outcomes, retries, no-route drops, and delivery queue depth (alertyg_*).

Endpoints

Method Path Description
POST /dispatch Receives an Alertmanager webhook payload. Returns 202, or 429 when the route's delivery queue is full.
GET /health Liveness/readiness check. Returns 503 while draining.
GET /metrics Prometheus metrics.

Metrics

In addition to the default Go/process metrics, alertyG exposes:

Metric Type Labels Description
alertyg_dispatches_received_total counter Alertmanager webhook payloads accepted for delivery.
alertyg_dispatches_rejected_total counter reason Dispatch requests rejected before delivery (queue_full, invalid_payload, unsupported_media_type, body_too_large).
alertyg_deliveries_total counter result Delivery outcomes per alert group (ok, error).
alertyg_delivery_queue_depth gauge Alert groups currently waiting across all per-route delivery queues.
alertyg_groups_dropped_no_route_total counter Alert groups dropped because no route matched.
alertyg_webhook_retries_total counter Webhook send attempts beyond the first.

Configuration

alertyG reads a TOML config file (default config.toml). See config.toml for a complete example.

[application]
addr = "0.0.0.0:7313"
log_level = "debug"            # debug | info | warn | error
logger = "text"                # text | json
resolved_ttl_minutes = 30
proxy_url = ""                 # optional outbound proxy
timezone = "Europe/Sofia"

[routes]
[routes.devops]
webhook_url = "https://chat.googleapis.com/v1/spaces/.../messages?key=...&token=..."

[templates]
header = '''...'''
message = '''...'''

Routing

Each notification is routed by the channel value in its groupLabels to the matching [routes.<name>] entry. For example, a group with channel: devops is delivered to routes.devops.webhook_url. Groups with no matching route are dropped (and logged).

Overrides

Settings can be overridden at runtime. Precedence is flags > environment > config file.

Setting Flag Environment
Listen address -addr ALERTYG_ADDR
Proxy URL -proxy_url ALERTYG_PROXY_URL
Log level -log_level ALERTYG_LOG_LEVEL
Log format -logger ALERTYG_LOGGER
Config path -config

Running

From source

Requires Go 1.25+.

go run ./cmd/app -config config.toml

Or build a binary:

go build -o alertyG ./cmd/app
./alertyG -config config.toml

With Docker

docker build -t alertyg:latest .
docker run --rm -p 7313:7313 \
  -v "$PWD/config.toml:/etc/alertyg/config.toml:ro" \
  alertyg:latest

With Docker Compose

A compose.example.yml is provided that builds the image, mounts your config.toml, publishes port 7313, and adds a /health healthcheck:

docker compose up --build

The server is then reachable at http://localhost:7313. Other containers on the same Docker network can reach it via the service name, e.g. http://alertyg:7313/dispatch.

Wiring up Alertmanager

Point an Alertmanager webhook receiver at alertyG:

receivers:
  - name: alertyg
    webhook_configs:
      - url: http://alertyg:7313/dispatch
        send_resolved: true

Make sure the alerts carry a channel label that matches a configured route. A sample payload is available in api.http for manual testing:

curl -X POST http://localhost:7313/dispatch \
  -H 'Content-Type: application/json' \
  --data-binary @payload.json

Development

go build ./...
go vet ./...
go test ./...

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages