Measuring mump2p on mainnet: earlier blocks on three clients, none of it significant
We ran Optimum's mump2p gateway, version 1.1.1 at commit 66e391d, on 36 mainnet full nodes in a single datacenter for 11.5 days at the end of August, one gateway on every node, across the full six-by-six matrix of consensus against execution clients. Blocks arrived earlier on three of the six consensus clients. On a fourth about half the move is real, on a fifth the metric cannot say either way, and on the sixth the treated nodes got slower rather than faster. None of the numbers is statistically significant, and the reason is built into the design rather than into the data.
The apparent effect, before we looked hard, was 40 to 110 ms. We then spent a second pass attacking our own control group, because the treated nodes are bare metal in Vienna and the controls are GCP in Frankfurt, and a difference-in-differences absorbs a constant offset between two datacenters but never a drift. What survived that pass is smaller, is carried by fewer clients, and rests on a different kind of evidence than the control group provides. This post is about which parts held.