Running ghcr.io/base/node-reth:v1.1.1 (and the ~Jun-5 build for comparison) on a
Base mainnet datadir restored from the official pruned snapshot
(base-mainnet-pruned-reth-1782806250.tar.zst, storage settings absent →
storage_v2:false), fed via base-consensus follow (BASE_NODE_SOURCE_L2_RPC),
host: Linux 6.19 x86_64, 62G RAM, ext4.
Bug 1 — v1.1.1 follow mode sends engine_newPayload only, never any
forkchoiceUpdated. Payloads insert (payload_safety=unsafe, ~3ms each) but
nothing canonicalizes; the in-memory tree grows until cgroup OOM (55G in
minutes), and all progress rewinds on restart. The ~Jun-5 build performs the
expected per-payload lockstep (insert → canonical commit → Forkchoice updated) — but is unusable post-Beryl. Net effect: follow mode cannot EL-sync
on v1.1.1, and the warning ("Running without --proofs; ... does not support EL
sync") suggests this may be intended — if so, what is the supported way to
catch an EL up from a snapshot when devp2p peers are unreachable (residential
NAT; Base EL mesh is effectively unlistable)?
Bug 2 — persistence service dispatches exactly one batch per process, then
never again (observed on the Jun-5 build where follow+FCU works). Kernel-level
evidence while ~300 blocks/process were canonically committed with
--engine.persistence-threshold 16..1024:
persistence and mdbx-rs-txn-mgr threads parked in futex_wait across all
samples, /proc/<pid>/task/<tid>/io lifetime read_bytes=0 write_bytes=0;
- device write counters ~0 the entire time; mdbx.dat mtime bumps only on
graceful shutdown flush;
- durable floor advances by exactly one threshold-batch (e.g. +16) shortly
after start, then never again; memory then accumulates (~150MB/executed
block) until OOM.
Same behavior with --db.sync-mode safe-no-sync, thresholds 16/64/32 and
1024/2048/1024, and paced insertion (--follow.insert-delay-ms 250..2000).
Looks like the engine awaits a persistence ack that never arrives (or the
dispatcher gates on something absent in snapshot-restored v1 datadirs — note
db settings get → "No storage settings found", and the v1-vs-requested-v2
mismatch WARN at startup).
Happy to provide full logs, metric dumps, or run instrumented builds — this
box reproduces both within minutes, every time.
Running
ghcr.io/base/node-reth:v1.1.1(and the ~Jun-5 build for comparison) on aBase mainnet datadir restored from the official pruned snapshot
(
base-mainnet-pruned-reth-1782806250.tar.zst, storage settings absent →storage_v2:false), fed via
base-consensus follow(BASE_NODE_SOURCE_L2_RPC),host: Linux 6.19 x86_64, 62G RAM, ext4.
Bug 1 — v1.1.1 follow mode sends engine_newPayload only, never any
forkchoiceUpdated. Payloads insert (
payload_safety=unsafe, ~3ms each) butnothing canonicalizes; the in-memory tree grows until cgroup OOM (55G in
minutes), and all progress rewinds on restart. The ~Jun-5 build performs the
expected per-payload lockstep (insert → canonical commit →
Forkchoice updated) — but is unusable post-Beryl. Net effect: follow mode cannot EL-syncon v1.1.1, and the warning ("Running without --proofs; ... does not support EL
sync") suggests this may be intended — if so, what is the supported way to
catch an EL up from a snapshot when devp2p peers are unreachable (residential
NAT; Base EL mesh is effectively unlistable)?
Bug 2 — persistence service dispatches exactly one batch per process, then
never again (observed on the Jun-5 build where follow+FCU works). Kernel-level
evidence while ~300 blocks/process were canonically committed with
--engine.persistence-threshold 16..1024:persistenceandmdbx-rs-txn-mgrthreads parked infutex_waitacross allsamples,
/proc/<pid>/task/<tid>/iolifetime read_bytes=0 write_bytes=0;graceful shutdown flush;
after start, then never again; memory then accumulates (~150MB/executed
block) until OOM.
Same behavior with
--db.sync-mode safe-no-sync, thresholds 16/64/32 and1024/2048/1024, and paced insertion (
--follow.insert-delay-ms 250..2000).Looks like the engine awaits a persistence ack that never arrives (or the
dispatcher gates on something absent in snapshot-restored v1 datadirs — note
db settings get→ "No storage settings found", and the v1-vs-requested-v2mismatch WARN at startup).
Happy to provide full logs, metric dumps, or run instrumented builds — this
box reproduces both within minutes, every time.