3 posts tagged with "StereumLabs AI"

Six Besu nodes, same version, same hardware, same config. Five held a flat JVM heap around 1.0 to 1.3 GB. The sixth climbed about 10 GB a day and was on track to be OOM-killed roughly 30 hours after a restart. The one thing different about it was the consensus client on the other side of the engine API.

This is a walkthrough of how StereumLabs AI, reading our fleet's metrics and logs, took that one anomalous node, traced it to a single method, and filed it upstream. Besu shipped a round of mitigations and closed the issue. A later devnet reproduction showed the underlying layers still pile up, the issue was reopened, and the fix that followed is now in review. The bug is operational: recoverable by a restart, no consensus impact, no double-sign, no state-root divergence. It is also the kind of cross-client interaction a single-node test will never surface, because it only appears when a live pairing lands in a specific state.

Ethereum runs on two layers: an execution client (EC) handles the EVM, transactions, and world state, and a consensus client (CC) handles Proof-of-Stake fork choice and validator duties. Six EC implementations and seven CC implementations exist in production today, paired in dozens of combinations across the network. StereumLabs runs all of them, side by side, on identical hardware in our Vienna data center, and publishes the numbers.

This post is the technical introduction to that platform: the bare-metal fleet, the metrics and logs pipeline, the label conventions that make the pairings comparable, and the in-house AI workflow that turns the resulting telemetry into the blog posts you are reading.

RockLogic publishes a separate case study on the business side of this workflow: how the same "AI on own data" pattern keeps customer telemetry inside the perimeter while still producing useful answers. This post is the technical view from the other side of the same workflow.

A recap of our EthCC[9] presentation in Cannes: what StereumLabs is, how we use AI on top of our monitoring data, and what Fusaka actually did to hardware across 36 client pairings.