Skip to main content

Ethereum reorg accounting: Prysm sees 8×, Lodestar sees 0

· 22 min read
Stefan Kobrc
Founder RockLogic
StereumLabs AI
Artificial Intelligence

A Prysm node and a Lodestar node on the same chain, on identical hardware, both export beacon_reorgs_total. Over the last 90 days, the Prysm hosts in our Vienna NDC2 fleet incremented that counter 6,011 times. The Lodestar hosts incremented it zero times. Both numbers are correct readings of what each implementation chose to count.

This post is a 90-day reorg census across that fleet plus the smaller GCP comparator cohort: every consensus client (CC) paired with every execution client (EC), against the same Ethereum mainnet, with per-host normalization. The questions we answer: which Prometheus counter to trust for which question, why the same EC behind two different CCs produces very different reorg numbers, and why a "zero reorgs" reading on some clients is silence rather than safety.

90-day Ethereum reorg counts compared across six consensus clients on identical bare-metal hardware

Key findings at a glance:

  • Prysm increments beacon_reorgs_total 8× more often than Lighthouse over 7 days. The gap shrinks to 1.6× over 90 days.
  • Lodestar's beacon_reorgs_total is 0 for the entire 90-day window. Its decline-reason counter fires roughly 54 times per host per week.
  • The same EC behind two CCs produces 2–5× different counts: Prysm + Besu reports 69 per host vs Prysm + Nethermind 280.
  • Geth is the only EC in our fleet whose Prometheus reorg counter increments at all. Nethermind and Reth export the metric but it never increments; Besu, Erigon, and Ethrex don't export one at all.
  • A single fixed beacon_reorgs_total alert threshold does not port between consensus clients. Re-baseline per CC × EC pair.

What we measured: the StereumLabs Ethereum client fleet

The bare-metal NDC2 fleet in Vienna carries the full pairing matrix: every CC family (Grandine, Lighthouse, Lodestar, Nimbus, Prysm, Teku) crossed with every EC family (Besu, Erigon, Ethrex, Geth, Nethermind, Reth). Each CC family runs on 6 hosts, one per EC pairing. The GCP comparator cohort runs only Geth on the EC side, paired with one supernode variant of each CC (introduced for PeerDAS custody-group experiments). The full setup is documented in the StereumLabs measurement stack post.

All counters in this post come from our prometheus-cold datasource over rolling 7-day, 30-day, and 90-day windows ending around 2026-05-26. Per-host averages divide each family's total reorg count by its host count. Grandine currently carries two extra non-production hosts on a custom image; those are excluded from every comparison below. Queries are reproduced in the methodology section.

The reorg-relevant metric vocabulary, by client, is heterogeneous:

SourceMetricNotes
Lighthouse, Teku, Nimbus, Grandine, Prysmbeacon_reorgs_totalShared name, observed counts diverge by ~8× over 7 days
Lodestarbeacon_reorgs_totalCounter exists, never increments in our 90-day window
Lodestarbeacon_fork_choice_reorg_distance_bucketHistogram populated even while beacon_reorgs_total stays at zero
Prysmreorg_depth_bucket, reorg_distance_bucketTwo Prysm-specific histograms with deeper bins
Prysmbeacon_late_block_attempted_reorgsProposer-boost reorg attempts, separate counter
Nimbusbeacon_safe_reorgs_totalReorgs after the proposer-boost timing window
Lodestarbeacon_fork_choice_not_reorged_reason_total{reason}Why a candidate reorg was declined
Gethchain_reorg_executes, chain_reorg_add, chain_reorg_dropEC-side canonical chain rewrites
Nethermindnethermind_reorganizations, nethermind_transactions_reorgedSeries exists, never increments in window
Rethreth_blockchain_tree_reorgs, reth_blockchain_tree_latest_reorg_depthSeries exists, never increments in window
Besu, Erigon, Ethrexno reorg counter exported on PrometheusDEBUG logs not audited in this round

That table alone tells most of the story. Before we look at a number, we already know that "is my client reorging more than it used to" is a question with six different correct answers.

What each Ethereum consensus client reports as a reorg

Across all six CCs, 6 production hosts each, here is what beacon_reorgs_total accumulated over our 7-, 30-, and 90-day windows:

  • Prysm: 1,373 reorgs in 7d (229 per host), 502 per host over 30d, 1,002 per host over 90d
  • Teku: 282 in 7d (47 per host), 209 per host over 30d, 759 per host over 90d
  • Nimbus: 210 in 7d (35 per host), 172 per host over 30d, 672 per host over 90d
  • Lighthouse: 171 in 7d (29 per host), 159 per host over 30d, 638 per host over 90d
  • Grandine: 153 in 7d (26 per host), 163 per host over 30d, 668 per host over 90d
  • Lodestar: 0 in 7d, 0 over 30d, 0 over 90d

A few observations:

  • Prysm increments roughly 8× more often than Lighthouse over 7 days. Over 30 days the multiple drops to 3.2×, over 90 days to 1.6×. Either Prysm had a spike in recent days (it did, see below), or Prysm's counter responds to short-lived events that other clients smooth out across longer windows. Both apply here.
  • Grandine, Lighthouse, Nimbus, and Teku land within a 1.8× band over 7 days (Grandine 26, Teku 47 per host) and tighten to a 1.2× band over 90 days. That is the tightest cluster in the dataset.
  • Lodestar increments the counter zero times. Not "low": exactly zero, across 90 days and 6 hosts. The Lodestar section below describes what it does instead.

The 7-day-to-90-day collapse on Prysm is the first practical lesson: short reorg windows can mislead. In this single 7-day window Prysm registered two sustained spikes that no other CC registered and that no EC-side counter registered either. An alert calibrated to a Lighthouse-shaped baseline would have fired twice on Prysm during this week alone, both times without a chain event behind it.

Cross-CC divergence at this scale is not unique to reorgs. The same six consensus clients showed a 50× spread in blob gossip verification latency on identical hardware. Different metric, different mechanism, same conclusion: a shared metric name does not guarantee a shared definition.

What each execution client reports: Geth, Nethermind, Reth, and the rest

The EC side is simpler, and emptier. Host counts vary by EC: Geth runs on 13 hosts (7 NDC2 + 6 GCP comparator), Besu on 7, all others on 6.

  • Geth: chain_reorg_executes increments. 7d: 414 reorgs (32 per host); 90d: 7,383 (568 per host).
  • Nethermind: nethermind_reorganizations is exposed, scraped, and stays at 0 across the 90-day window.
  • Reth: reth_blockchain_tree_reorgs is exposed, scraped, and stays at 0 across the 90-day window.
  • Besu, Erigon, Ethrex: no Prometheus reorg counter exported at all.

Geth is the only EC in our fleet whose counter increments. Nethermind and Reth carry the metric in their vocabulary, both series are registered and scraped successfully, but neither has incremented in 90 days on any host. We did not review either client's source to determine whether the increment path is unreachable on mainnet, intentionally gated, or implements a stricter definition of "reorg" than Geth does. That distinction is for the client teams to clarify. (Neither Nethermind nor Reth is deployed in our GCP cohort, so the 90-day window covers only the NDC2 hosts for those two clients.)

Besu, Erigon, and Ethrex do not export a Prometheus reorg counter at all. We have not in this round audited the DEBUG-level logs of those three clients for reorg events; it is possible that the events are visible there even though no counter is exposed. What we can say is that no Prometheus alert rule lives over them for this question today.

EC-side observability gaps are an existing pattern on this fleet. The EC sync-speed comparison found that initial sync time ranged from 1.5 hours to over 8 days on identical hardware across the same six clients, partly because each one logs and exports its sync progress differently. Reorg counters are the steady-state analogue of that observation.

The CC × EC pairing matrix: where reorg counts diverge most

The strongest finding in this dataset is not at the CC level or the EC level. It is in the CC × EC pairings.

Heatmap of Ethereum reorg counts per host across all consensus client × execution client pairings, 7-day window

The Lighthouse + Erigon cell is a data gap: two hosts are up in that pairing, but beacon_reorgs_total returned no series for them in the queried window. Either the counter has not registered on those hosts yet (a recent deployment), or there is a pairing-specific instrumentation gap. We flag the gap rather than impute a number.

Three patterns emerge from the matrix:

  1. Most CCs split their pairings into two tiers. For Lighthouse, Nimbus, Teku and Grandine, the same metric counts roughly twice as often when paired with Geth, Nethermind or Reth than when paired with Besu or Ethrex. Nimbus, for example, sits at 17–18 per host with Besu and Ethrex and at 35–36 with the other four ECs. The split looks like an EC-side property propagating into the CC counter, not a CC property.
  2. Prysm + Besu and Teku + Reth are real outliers. Prysm + Besu is 69 per host, while every other Prysm pairing lands between 114 (Ethrex) and 280 (Nethermind), a 1.6–4× spread that is not visible in any other CC row. Teku + Reth sits at 100 per host, while every other Teku pairing lands between 18 and 38, a 2.6–5.5× elevation.
  3. Prysm operates on a different scale. Even the lowest Prysm pairing (Besu, 69) is higher than the highest pairing of any other CC except Teku + Reth.

The same metric (beacon_reorgs_total) responds to the paired EC's behaviour, not just to the chain. Block import timing, attestation late-arrival rates, and engine-API forkchoice update latency on the EC side feed back into the CC's fork-choice loop, and the counters reflect that. We previously characterised the same EC↔CC coupling from a different angle in the Erigon + Caplin standalone vs classic comparison, where collapsing both layers into one binary moved EVM execution by ~25% and MDBX commits by ~2× on the same engine-API timing surface, observed from the inside. We are not yet in a position to attribute the Prysm + Besu reduction or the Teku + Reth elevation to a specific mechanism; the pairings are flagged here as findings for follow-up.

Practical takeaway for operators: if you swap your EC, expect your CC reorg numbers to move. If you alert on a fixed rate(beacon_reorgs_total[15m]) threshold, that threshold needs to be re-baselined after any EC change. Otherwise you are paging yourself for a re-baselined client behavior rather than a chain event.

The Prysm reorg-rate spike: a counter responding to what?

Across 7 days, Prysm increments beacon_reorgs_total 1,373 times. Across 90 days, 6,011. That means 23% of Prysm's 90-day total accumulated in the last 7 days. The other CCs do not show the same concentration: Lighthouse's 7-day-to-90-day ratio is 4%, Teku's is 6%, Grandine's is 4%, Nimbus's is 5%.

The hourly time series confirms two distinct sustained spikes within the 7-day window. One lasted roughly 15 hours, the second roughly 36 hours. During both, Prysm's fleet-wide rate jumped from a baseline near 0.0014 reorgs/sec (across all 6 Prysm hosts) to a sustained 0.005–0.010 reorgs/sec, a 4× to 7× elevation. No other CC counter and no EC counter registered the spikes. Geth's chain_reorg_executes stayed at its normal pulse pattern (around 0.0034 reorgs/sec across its 13 hosts) throughout both windows.

Prysm Ethereum reorg-rate spikes compared to Lighthouse and Geth chain_reorg_executes over 7 days

We checked the obvious explanation first: a Prysm version change during the window. The Prometheus cc_version label held at v7.1.3 continuously across all 6 Prysm hosts for the full 7 days. The spikes are not the artifact of a release.

We also cannot point to a corresponding mainnet event for either window. Public block explorers do not record deep reorgs in those periods, and our EC-side counters are quiet through them.

One hypothesis that fits the data: Prysm's beacon_reorgs_total may increment on every fork-choice head re-attribution, including transient ones that are subsequently re-overridden by the next attestation update, while the other CCs that share the metric name appear to increment only when a head change crosses some stability threshold. We have not audited any client's source to prove this. The observation is restricted to: Prysm spikes, others stay flat. Either way, a Prysm reorg-rate spike on its own is not evidence of a chain anomaly. It is consistent with the fork-choice loop chewing on late blocks under transient peering or attestation delay, with the increment policy determining whether the counter responds.

Prysm's instrumentation also shifts between releases. We previously tracked one such transition in detail in the Prysm v7.1.1 vs v7.1.2 resource report, where block-processing latency on one EC pairing changed by 78% across a single point release. A rerun of this reorg census against future Prysm releases could land in different places.

Reorg depth histograms: Lodestar vs Prysm

Two CCs in our fleet populate a reorg-depth histogram: Lodestar (via beacon_fork_choice_reorg_distance_bucket) and Prysm (via reorg_depth_bucket, a Prysm-specific name). The other four CCs do not export an equivalent series over the queried window. Lighthouse increments beacon_reorgs_total 167 times in 7 days but does not appear to populate the matching histogram. We mention it as an instrumentation observation, not a finding about Lighthouse's correctness.

Lodestar's histogram, summed 7-day rate per bucket (sum by (le) (rate(beacon_fork_choice_reorg_distance_bucket[7d]))):

Bucket (slots)Cumulative rate
≤ 1.00.0000
≤ 2.00.000385
≤ 3.0 through ≤ 100.00.000385 (no further weight)
+Inf0.000385

Every observation Lodestar's histogram records sits in the 1–2 slot bucket. This is consistent with Lodestar evaluating candidate reorgs internally, classifying their distance, then declining to execute them. It lines up with the 324 headBlockIsTimely decline events on the same fleet over the same window.

Prysm's histogram, same 7-day rate:

Bucket (slots)Cumulative rate
≤ 1.00.0000
≤ 2.00.000276
≤ 4.00.000296
≤ 8.00.000334
≤ 16.00.000415
≤ 32.00.001893
+Inf0.001893

Prysm's distribution has weight all the way out to the 16–32 slot bucket. Mainnet did not see a deep canonical reorg in this window. Public explorers confirm the chain stayed within single-slot reorgs throughout, and Geth's EC-side counter agrees. So Prysm's reorg_depth_bucket is recording something that is not canonical-chain depth. More likely, it is the distance along Prysm's internal fork-choice graph from the previous head to a new one, including head transitions that were never canonical. That is a reasonable thing to measure, but it is not the same thing Lodestar is measuring.

Two clients can both label a histogram "reorg depth" and be measuring different things. The bucket boundaries are themselves a hint about what each team expected to see. Lodestar's spec-aligned 1, 2, 3, 5, 7, 10, 20, 30, 50, 100 versus Prysm's 1, 2, 4, 8, 16, 32 reflect different priors about the distribution.

Lodestar's silent counting: zero reorgs, but the evaluation pipeline runs

Lodestar reports beacon_reorgs_total = 0 across every host, every window. But Lodestar is not idle. In the same 7-day window the depth histogram (beacon_fork_choice_reorg_distance_bucket, covered above) does receive observations, and beacon_fork_choice_not_reorged_reason_total{reason="headBlockIsTimely"} accumulates 324 increments across the 6 Lodestar hosts (54 per host) and 50 increments on the single Lodestar supernode. Both series carry non-trivial weight; only the headline counter stays zero.

The reason label name lines up with the consensus specs' should_override_forkchoice_update helper (validator spec). The simplified flow:

  1. The validator sees a late block at the start of slot N+1.
  2. It considers proposing on the parent of that late block instead of building on it (a "late-block reorg" override).
  3. Safety checks run, including: is the head block timely enough that overriding it would not damage the chain?
  4. If the head block is timely, the spec says do not override.

The label name suggests Lodestar's counter increments on that branch, but we have not verified against the source. The 7-day data on every Lodestar host:

  • beacon_reorgs_total: 0
  • beacon_fork_choice_not_reorged_reason_total: 54 per host (non-supernode) or 50 (supernode), all under the single label value headBlockIsTimely, no other reason labels populated
  • beacon_fork_choice_reorg_distance_bucket: populated, all observations in the 1–2 slot bucket

Taken together, the three series show a CC that is actively running the late-block reorg evaluation pipeline (computing candidate distances, recording decline reasons) but declining to act on the result in this window. Lodestar's published metrics make it look like Lodestar never executes a reorg on mainnet. Whether that is a true statement about Lodestar's behavior or a missing increment on beacon_reorgs_total requires source-level confirmation, which we are requesting from the Lodestar team alongside this post. In the meantime, beacon_reorgs_total is not a portable observability primitive across CCs: a Lodestar operator who alerts on it gets no signal at all.

Cross-client reorg alerting cheat sheet

For an operator runbook, by question:

Did our CC re-attribute its head in the last 5 minutes? Use rate(beacon_reorgs_total[5m]). Alert on sustained elevation, but only after re-baselining per CC and per EC pair. Ignore brief Prysm spikes and any reading on Lodestar.

Did the EC see a canonical chain rewrite? Use chain_reorg_executes on Geth only. Alert on any non-zero rate during sustained head movement. Treat a zero reading on Nethermind or Reth as an instrumentation gap, not as safety.

How deep was the latest reorg? Use beacon_fork_choice_reorg_distance_bucket if you run Lodestar, or reorg_depth_bucket if you run Prysm. Alert on histogram weight in buckets greater than 2 slots. Do not compare Lodestar's histogram to Prysm's; they record different things.

Did the proposer-boost reorg path trigger? Use beacon_late_block_attempted_reorgs on Prysm or beacon_safe_reorgs_total on Nimbus. Alert on sudden surges. Absence of the metric on the other CCs is not absence of the event.

Why didn't the CC reorg when we expected it to? Use beacon_fork_choice_not_reorged_reason_total on Lodestar (the only CC in our fleet that exports it). Alert on reason-label changes over time. Cross-client comparison is not possible with this metric today.

If you need a single rule that works across the fleet, use rate(chain_reorg_executes[15m]) on Geth EC hosts, with per-host thresholds calibrated to that host's 90-day baseline. Geth is the only EC in our fleet whose counter both exists and increments.

How Glamsterdam and ePBS may change reorg accounting

The Glamsterdam hardfork is expected to enshrine proposer-builder separation (ePBS) via EIP-7732. At time of writing (May 2026), it is still in devnet stabilization per the Ethereum Foundation's April 2026 Checkpoint, with no fixed mainnet activation slot. Under ePBS, the slot gains an explicit commit-reveal cycle: the proposer commits to a builder-signed execution header in-slot, the builder reveals the payload, and a payload-timeliness committee (PTC) votes on whether the reveal arrived in time. If the PTC says no, the slot's execution payload is empty.

That mechanism creates new transient head states that today's reorg counters do not contemplate. We are not in a position to predict exact numbers, but two structural points are worth flagging now rather than after the fork ships:

  • Whichever client increment policy you have today carries into Glamsterdam. A counter that increments on every fork-choice head re-attribution will see a higher post-Glamsterdam baseline than one that only increments on canonical rewrites. Operators should re-baseline alert thresholds after the fork, not before.
  • The CC-side counter becomes the operational signal. With execution payloads decoupled from proposer commitments, "the chain reorged" and "the execution payload changed" partially decouple. EC-side counters like chain_reorg_executes will under-report what a validator operator cares about, and the relevant counters move to the CC side and to relay/builder observability.

The pre-Glamsterdam takeaway from our 90-day census: if you cannot answer "what does my client mean by reorg today," you will not be able to answer "did my client behave correctly under ePBS" once the fork lands.

Frequently asked questions about Ethereum reorg monitoring

Why does Prysm report more reorgs than other Ethereum consensus clients?

Prysm's beacon_reorgs_total appears to increment on transient fork-choice head re-attributions in addition to canonical reorgs. The other CCs that share the metric name (Lighthouse, Teku, Nimbus, Grandine) appear to increment only when a head change persists. The cross-client gap shrinks from 7× over 7 days to 1.5× over 90 days, suggesting the bigger Prysm count comes from short-lived events the other counters smooth over. We have not verified this against source code.

Does Lodestar really not reorg on the Ethereum mainnet?

Lodestar's beacon_reorgs_total shows exactly zero across every host and every queried window. But Lodestar's depth histogram is populated and its beacon_fork_choice_not_reorged_reason_total{reason="headBlockIsTimely"} counter fires around 54 times per host per week (about every 3 hours per host). From outside, that looks like Lodestar evaluates candidate reorgs and declines to execute them. Source-level confirmation is pending from the Lodestar team.

Which Ethereum execution clients export a Prometheus reorg counter?

In our fleet: Geth (chain_reorg_executes, increments normally), Nethermind (nethermind_reorganizations, registered and scraped but never increments), Reth (reth_blockchain_tree_reorgs, same as Nethermind). Besu, Erigon, and Ethrex do not export a Prometheus reorg counter at all. Operators have to derive reorg events from logs. This patchy coverage is one of the operational consequences of Ethereum client diversity: the same observability question gets six different answers, and three of those answers are silence.

How should I alert on reorgs across different Ethereum consensus clients?

Per-host baselines, per CC × EC pair. A single fixed rate(beacon_reorgs_total[5m]) threshold does not port between clients: the same EC behind two different CCs produces reorg counts that differ by 2–3×. For one EC-side rule that works fleet-wide, alert on rate(chain_reorg_executes[15m]) for Geth hosts; the same metric does not exist on the other ECs. Skip alerting on Lodestar beacon_reorgs_total entirely. It is always zero.

Will Ethereum's Glamsterdam upgrade change reorg counting?

Probably yes. Glamsterdam is expected to enshrine proposer-builder separation (ePBS) via EIP-7732, which introduces commit-reveal cycles and payload-timeliness committees. Those mechanisms create new transient head states that today's counters do not contemplate. Plan to re-baseline alert thresholds after the fork. EC-side counters like chain_reorg_executes will under-report what validator operators care about, since execution payloads decouple from proposer commitments under ePBS.

Methodology

All queries ran against our prometheus-cold datasource over windows ending around 2026-05-26 12:00 UTC.

Per-CC family total:

sum by (cc_client) (increase(beacon_reorgs_total{cc_version!="unstable-465c727"}[7d]))

Per-host averages divide each family's total by the live production-host count (6 NDC2 hosts per CC family; the cc_version!="unstable-465c727" filter excludes the two Grandine custom-image hosts from the comparison).

Per-pairing average per host:

avg by (cc_client, ec_client) (increase(beacon_reorgs_total{cc_version!="unstable-465c727"}[7d]))

EC-side Geth counter:

sum by (ec_client) (increase(chain_reorg_executes[7d]))

Histogram bucket rates:

sum by (le) (rate(beacon_fork_choice_reorg_distance_bucket[7d]))
sum by (le, cc_client) (rate(reorg_depth_bucket[7d]))

Lodestar's decline reason:

sum by (cc_client, reason) (increase(beacon_fork_choice_not_reorged_reason_total[7d]))

Host counts as of 2026-05-26, taken from count by (cc_client) (count by (cc_client, instance) (up{role="cc", job!="node_exporter"})) and the EC equivalent. Filtering out job="node_exporter" is necessary because every CC and EC host runs node_exporter under the same role label, which otherwise double-counts.

  • CC (NDC2): Grandine 8 (6 stable v2.0.4 + 2 custom-image unstable-465c727), Lighthouse 6, Lodestar 6, Nimbus 6, Prysm 6, Teku 6.
  • CC supernodes (GCP): one host per CC family.
  • EC (NDC2 + GCP): Geth 13 (7 NDC2 + 6 GCP comparator), Besu 7, Erigon 6, Ethrex 6, Nethermind 6, Reth 6.

The two Grandine custom-image hosts (unstable-465c727) are excluded from every comparison in the post; comparisons in this analysis run across the same 6 production hosts per CC family.

Lighthouse + Erigon shows up in the up query (two hosts), but beacon_reorgs_total returns no series for that specific pairing in the queried window. Likely an instrumentation or registration gap, not zero reorgs. Numbers in the per-pairing matrix divide totals by the matching count(up), so pairings with missing counter series do not contribute to the comparison.

The 90-day window covers the post-Fusaka mainnet conditions only. Pre-Fusaka data (before December 3, 2025) is in cold storage but not included here because PeerDAS introduced sufficient changes to fork-choice timing that combining the two periods would mix populations. The Fusaka hardware impact post characterizes that boundary in detail.

Dashboard panel definitions for every metric used here are linked from the consensus-client dashboards in our public catalog at /docs/dashboards/list/catalog. The per-client dashboards on grafana.stereumlabs.com each carry the reorg panels described above. We do not yet ship a cross-client comparison dashboard for these metrics, so the cheat sheet above stands in for one.

If you operate one of the clients we measured and want to clarify what your counter actually measures (particularly the Lodestar beacon_reorgs_total definition and the Nethermind / Reth increment paths), write to contact@stereumlabs.com. We will append a correction with attribution.