Skip to content

Wire transaction events into devnet#3633

Open
niran wants to merge 3 commits into
mainfrom
niran/txobs-devnet-vector
Open

Wire transaction events into devnet#3633
niran wants to merge 3 commits into
mainfrom
niran/txobs-devnet-vector

Conversation

@niran

@niran niran commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds devnet Vector configuration to tail transaction event JSONL files and ship NDJSON to audit-archiver.
  • Drops parsed devnet transaction events in Vector when data contains unsafe transaction/body/header key names, matching the production sidecar behavior.
  • Enables transaction event journal paths for devnet services and adds a proxyd transaction-events image/config for local verification.
  • Adds a smoke script that exercises devnet transactions and checks audit transaction event ingestion.
  • Documents component_discarded_events_total as the local Vector metric to inspect for malformed JSONL parse drops and unsafe-key validation drops.

Verification

  • docker run --rm -v /Users/niran/workspace/base/.worktrees/txobs-devnet-vector/etc/docker/transaction-events-vector.yaml:/etc/vector/vector.yaml:ro -v /private/tmp/txobs-devnet-vector-data:/vector-data timberio/vector:0.47.0-debian validate /etc/vector/vector.yaml

Stack Context

This PR is based on #3628 and contains only devnet/Vector wiring for local verification. It is intentionally reviewed separately from service producer code.

type=routine
risk=low
impact=sev5

@niran niran force-pushed the niran/txobs-devnet-vector branch from be7562f to 8d22450 Compare June 18, 2026 17:11
@niran niran force-pushed the niran/txobs-builder-events branch from 0887a65 to 1e55997 Compare June 18, 2026 17:11
@niran niran changed the base branch from niran/txobs-builder-events to niran/txobs-event-foundation June 18, 2026 17:11
Comment thread etc/docker/Dockerfile.proxyd.transaction-events Outdated
Comment thread etc/docker/proxyd/proxyd.toml Outdated
Comment thread etc/docker/transaction-events-vector.yaml
@niran niran force-pushed the niran/txobs-devnet-vector branch 2 times, most recently from f36f90d to 0d01740 Compare June 18, 2026 17:33
@niran niran force-pushed the niran/txobs-event-foundation branch 4 times, most recently from 9c11260 to 5f2ba67 Compare June 22, 2026 16:59
Base automatically changed from niran/txobs-event-foundation to main June 22, 2026 18:28
@cb-heimdall

Copy link
Copy Markdown
Collaborator

🟡 Heimdall Review Status

Requirement Status More Info
Reviews 🟡 0/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 0
Sum 1

@niran niran requested a review from wlawt June 22, 2026 20:02
@niran niran force-pushed the niran/txobs-devnet-vector branch from 7fcff07 to 1d27433 Compare June 23, 2026 18:49
Comment thread etc/docker/docker-compose.ingress.yml Outdated
@niran niran force-pushed the niran/txobs-devnet-vector branch from 1d27433 to 567156f Compare June 24, 2026 16:56
Comment thread etc/docker/docker-bake.hcl Outdated
@github-actions

Copy link
Copy Markdown
Contributor

Review Summary

This PR adds devnet infrastructure wiring for transaction event observability: Vector config for tailing JSONL files, Postgres + audit-archiver for persistence, proxyd transaction-events Dockerfile, and a smoke test script. No Rust code changes.

No new issues found. The changes are clean devnet/infrastructure configuration. A few observations:

  • The bootnode IP reassignment (.10/.11.41/.40) to make room for new services at .30-.32 is clean with no collisions in the subnet.
  • The proxyd.toml uses $TRANSACTION_EVENTS_PATH as a TOML string value — this depends on proxyd supporting env var interpolation in config values (standard TOML does not expand variables). This was partially raised in a previous review cycle.
  • Vector's read_from: beginning will re-ingest all JSONL on restart if its data directory is wiped (as happens with just ingress-down), but this is acceptable for devnet.
  • The transaction-events-vector depends on audit-archiver with condition: service_started (not service_healthy), so Vector may attempt sends before audit-archiver is ready. The retry config (retry_attempts: 10, backoff up to 300s) handles this gracefully.
  • The hardcoded port 9100 in transaction-events-vector.yaml is documented with a sync comment, which is the right approach since Vector YAML isn't interpolated by Docker Compose.

@github-actions

Copy link
Copy Markdown
Contributor

✅ base-std fork tests: all 616 passed

base/base is fully in sync with the base-std spec.

Dependency Ref Commit
base-std main 4658f1b7
base-anvil 0092692587d8d064dd2c6923ce26a682c58f3694 00926925

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants