Skip to main content

Where the slot goes: Lighthouse and attestation timing

· 14 min read
Stefan Kobrc
Founder RockLogic
StereumLabs AI
Artificial Intelligence

An Ethereum slot is 12 seconds, and your attestation is due 4 seconds in. By the time your execution client sees a block to verify, most of that 4-second budget is already gone: on our fleet the block arrives, on average, 1.7 to 1.9 seconds into the slot, and that number barely moves whichever execution client you run. What the execution client changes is the slice after arrival. Under Lighthouse, that slice runs from about 100 ms (Ethrex) to 460 ms (Erigon) on a normal block. Across the mainstream clients it shifts how often the node lands late enough that attestations would fail by about 1.5x, and by 3.7x once Erigon's disk-bound tail is in the picture, on identical hardware.

This is the first of a series. We run all six execution clients paired with all six consensus clients on identical bare metal, and each consensus client reports slot timing differently. We start with Lighthouse because it instruments the block-delay breakdown more completely than any other CC on the fleet. Later editions take the same question to Prysm, Teku, Nimbus, Lodestar and Grandine.

Read this first
  • "Would fail attestations" is a lateness flag, not a measured miss. Lighthouse derives this counter from block timing alone: it fires when a block lands late enough that a slot's attestation would be late, and it does so on our fleet even though we run no live validators on it. That is the tell that it is a block-processing heuristic, the same with validators or without, not a count of attestations missed or penalised. Read the per-pairing numbers as relative timing risk.
  • Reth is excluded. It has been back-filling across version upgrades through this window (currently v2.3.0) and was not at the tip; whether that is the client or our own deploy cadence is open (see the footprint census). Its numbers are doubly an artifact of that state: the fastest execution time of any client (~39 ms) and the second-highest would-fail count both come from SYNCING-state replay rather than tip blocks, which is why it cannot sit in the comparison.
  • The outcome is a complete count; only the magnitudes are sampled. The would-fail-attestations rate is a monotonic counter, so the per-day numbers and the 3.7x are exact. The per-component execution times are gauges scraped about once a minute, so treat their p99 as a conservative floor. The full sampling detail, and why the tail claim rests on the counter rather than the gauge, is in the methodology.
  • Window and versions. Three days ending 2026-06-23, on Lighthouse v8.1.3 paired with Geth v1.17.3, Nethermind 1.38.1, Besu 26.6.0, Erigon v3.4.3, Ethrex 16.0.0. The fleet has since moved to Lighthouse v8.2.0, so reproducing this window means pinning the version (see methodology).

The 4-second deadline

Attestations for a slot are due one-third of the way through it, at the 4-second mark, and the sync-committee message rides the same clock. Cross that line and the head vote is late: you lose the timely-head reward, and if late production then misses the inclusion windows for the source and target flags, those erode too, down to zero for the slot in the worst case. The attestation can still be included late, so this is a reward-erosion deadline, not a hard cutoff. So the question for an operator is not "did my node import the block" but "did it import the block, verify it, and make it attestable before 4 seconds were up." Everything below is measured against that line.

Where the 12 seconds go

A block does not exist at your node at slot start. It is proposed elsewhere, gossiped across the network, and only then observed locally. Lighthouse records each step. Averaged over the window, the path to attestable looks like this:

Slot timing under Lighthouse: the block is observed about 1.8 s into the slot, consensus verification adds ~0.16 s, execution-layer verification adds 0.1 to 0.46 s depending on the EC, and the block becomes attestable around 2.0 to 2.4 s, against the 4 s attestation deadline

The arithmetic is close to additive: time-to-attestable is roughly arrival plus consensus verification plus execution-layer verification. For Geth that is 1.87 + 0.17 + 0.20 = 2.24 s, against a measured attestable time of 2.28 s; for Erigon, 1.66 + 0.17 + 0.46 = 2.29 s against 2.31 s. The decomposition holds for every client, with a 30-to-40 ms residual for the availability and import steps Lighthouse times separately. The pieces:

StepDurationSet by
Block arrival (observed from slot start)1.66 to 1.87 sthe network and the proposer, not you
Consensus verification~0.17 sthe consensus client, uniform here
Execution-layer verification0.10 to 0.46 sthe execution client you pick
Becomes attestable2.0 to 2.4 sroughly arrival plus verification, with a small residual for availability and import steps Lighthouse times separately

Two of those three rows are effectively fixed. Arrival is a network property: it ranges only from 1.66 to 1.87 s across the six pairings, and it does not track execution-client speed (Erigon, the slowest EC here, sees the earliest arrivals). Consensus verification is the same Lighthouse code in every pairing, around 165 ms throughout. The one row you choose is execution.

Execution is your lever: newPayload latency by client

On a normal block the execution slice is small, 100 to 460 ms, comfortably inside the budget. Arrival is not your problem; this slice is. The risk lives in the tail, and the proof of the tail is the complete counter below, not the sampled magnitudes. For a sense of size, execution-layer verification on a single block reaches a sampled p99 of about 1.2 s on Geth, 1.5 s on Besu, and 2.1 s on Erigon (a once-a-minute sample, so a conservative floor). The mechanism is plain: a slow newPayload tail landing on a block that already arrived late is what tips an otherwise marginal slot past 4 seconds. Stack a 2.1 s Erigon execution tail onto a 1.7 s arrival and you are near 3.8 s with consensus and import still to come.

It shows up in Lighthouse's own counter. beacon_block_delay_head_slot_start_exceeded_total, which Lighthouse describes as firing when "the duration between the start of the block's slot and the current time will result in failed attestations," counts the slots where the node landed too late to attest cleanly:

Execution client attestation timing under Lighthouse: average newPayload verification time alongside the daily would-fail-attestations count, rising from Ethrex and Geth at the low end to Erigon at the high end

Execution clientExecution verify, avgExecution verify, p99*Would-fail-attestations, per day
Ethrex103 ms~1.1 s150
Geth201 ms~1.2 s168
Nethermind165 ms~1.1 s217
Besu297 ms~1.5 s228
Erigon457 ms~2.1 s618

*p99 of the once-a-minute samples, a conservative floor. The tail claim rests on the complete would-fail counter, not this column.

The extremes line up: the fastest-tail clients (Ethrex, Geth) sit lowest at around 150 to 170 a day, and Erigon, with the heaviest tail, sits at 618, about 3.7x as often, on the same hardware behind the same Lighthouse with the same arrival times. The four mainstream clients span 150 to 228 a day, about 1.5x; Erigon's disk-bound tail is what stretches that to 3.7x. In slot terms, 618 a day is about 8.6% of slots flagged on Erigon against 2.3% on Geth. The middle is not strictly ordered by the average, and that is the point: Nethermind's average execution is below Geth's, yet it lands late more often. The mean is comfortable for every client; the tail is what sets the miss rate, which is why the p99 column matters more than the average. The control holds at the tail too: arrival's p99 is 3.3 to 3.5 s across all six pairings and does not order by execution-client speed (Geth's arrival tail is the highest, Erigon's among the lowest), so the would-fail gap is the execution layer's to own.

Why Erigon has the highest newPayload latency tail

This is the same Erigon that, in our pruning census, runs its pruning on the engine-API hot path, and the same Erigon that in the hardware-footprint census sustains the fleet's highest disk writes (about 1.9 TB a day). Heavy, continuous disk work is the most likely driver of the tail of a newPayload call: most are fast, but the ones that land during a flush or compaction are slow, and those are the blocks that miss. We have not yet lined up individual slow-newPayload timestamps against Erigon's compaction windows; a later edition will. The trade Erigon makes for the smallest disk footprint on the fleet shows up again here, on the timing budget.

To be fair across the board: on a typical block every one of these clients is comfortably attestable around 2 seconds, half the budget to spare. None of them is "too slow" in the average case. The difference is entirely in how often the tail bites, and that is what the per-day counter captures. Besu sits one step down in the same regime: the second-heaviest tail (p99 ~1.5 s, 297 ms average) for the same disk reason, just biting less often than Erigon's. It is a gradient, not one villain.

The honest version of "blame your EL"

Look at a single late block and the execution client usually looks innocent. In Lighthouse's DEBUG log, a delayed head block typically reads like observed_delay_ms: 7194, consensus_time_ms: 236, execution_time_ms: 80: the block showed up 7 seconds into the slot, and the local node handled it in a few hundred milliseconds. The lateness was the proposer's or the network's, and no execution client would have saved that slot.

Anatomy of a single late block under Lighthouse: the block arrives 7.2 seconds into the slot, already past the 4-second deadline, while the node's own consensus and execution verification together take only about 0.3 seconds

So why does the execution client move the daily count by 3.7x? Because it is not about the catastrophically late blocks, it is about the marginal ones. Across thousands of slots, a meaningful fraction arrive late enough that a few hundred extra milliseconds of execution is the difference between attestable-in-time and not. A faster execution layer does not fix a 7-second-late block, but it rescues the slots that were close. That is the lever you hold.

What to measure on your own nodes

  • Pull beacon_block_delay_observed_slot_start per pairing. If your blocks arrive late on average, that is a peering or proposer-connectivity problem, not an execution-client one, and changing EC will not help. Our peering deep dive is the place to start on that.
  • Pull beacon_block_delay_execution_time and watch its p99, not its average. The average is fine for everyone; the tail is where duties are lost.
  • Track beacon_block_delay_head_slot_start_exceeded_total as a rate. It is the closest single number to "how often is my timing budget blown," and it is comparable across your own pairings, as long as you pin one consensus-client version per comparison so a mid-window upgrade does not pool two code paths.

Coming next in the series

Lighthouse gives the cleanest breakdown, which is why it is first. The other consensus clients expose this timing differently, and that difference is itself a finding: the next editions take the same slot-timing question to Prysm, Teku, Nimbus, Lodestar and Grandine, and where their instrumentation hides things Lighthouse shows, we will say so.

This comparison holds the hardware and the consensus client fixed and varies only the execution client, which is the cut that makes the execution layer's contribution legible. Lining clients up on truly equal footing across windows of uneven data quality is an analysis in its own right, not a lookup on a static panel, and it is what StereumLabs AI does on our fleet, on the measurement stack we described here. When one pairing stops matching its siblings, that divergence is a finding, sometimes an upstream bug report. If you run Ethereum infrastructure and want this lens on your own nodes, reach us at stereumlabs.com or contact@stereumlabs.com.

Methodology

Numbers come from Lighthouse's beacon_block_delay_* metrics on our NDC2 deployment (Vienna), queried on the Prometheus-cold datasource (uid aez9ck4wz05q8e), with the fleet labels documented in build your own dashboards. Per-client values are for cc_client="lighthouse-super", cc_version="v8.1.3", deployment="NDC2" grouped by ec_client. The would-fail counter rates are increase(...[3d]) evaluated at 2026-06-23T00:00:00Z, so they are stable and reproducible rather than drifting with the query time. The fleet has since moved to Lighthouse v8.2.0, so pinning one CC version per comparison is what reproduces this window. Pinning deployment="NDC2" also keeps the bare-metal Vienna hosts from being pooled with a second Geth set on GCP, which sees the same block-arrival time (~1.88 s) but lands late about 303 times a day against the NDC2 host's 168, a platform effect of the cloud host rather than the network or the client.

  • Components are the Lighthouse gauges beacon_block_delay_observed_slot_start (arrival), beacon_block_delay_consensus_verification_time, beacon_block_delay_execution_time (execution-layer verification), and beacon_block_delay_attestable_slot_start (time to attestable). All are in milliseconds, confirmed against Lighthouse's DEBUG block-delay logs which label the same fields _ms.
  • Sampling. The component gauges are scraped about once every 66 seconds, so each value is one sampled block rather than a full per-block stream. Averages over three days are unbiased; the sampled quantile_over_time p99 and max_over_time worst-case are conservative floors, since between-scrape spikes are missed. We do not lean on them: the tail is carried by the complete would-fail counter. A precise percentile would need a histogram, which Lighthouse does not expose for these gauges (Nimbus does, so the Nimbus edition can use it).
  • The would-fail-attestations figure is increase(beacon_block_delay_head_slot_start_exceeded_total[3d]) / 3, per ec_client. Lighthouse's own help text defines this counter as triggering "when the duration between the start of the block's slot and the current time will result in failed attestations." Unlike the block-delay gauges, this is a monotonic counter, so the per-day counts are complete rather than sampled; only the execution average and p99 columns are the once-a-minute samples. It fires from block timing regardless of validators (it increments on our fleet with none attached), so it is Lighthouse's lateness heuristic, not an observed miss count, and it is a binary flag rather than an inclusion-or-penalty outcome: a flagged slot can still attest late. Read the counts as relative risk between pairings, not as absolute missed duties. Attaching live validators would not move these numbers: the counter times block import, which is independent of attestation signing, aggregation and local block production.
  • Arrival is EC-independent. beacon_block_delay_observed_slot_start ranges 1.66 to 1.87 s across the six pairings and does not order by execution-client speed, which is why we attribute the would-fail differences to execution rather than to the network or host.
  • Reth is excluded from the comparison: it was resyncing throughout (v2.3.0, finish stage at zero, zero VALID forkchoice responses over 24 h), so its block-delay metrics are not those of a synced node; the sync-speed census covers from-scratch sync times client by client. On identical 12-core hosts; consensus and validator processes run on separate machines.