Satellite altimeters are the most common source of wave measurement in phenomenal sea states, with significant wave heights exceeding 14 m. Unfortunately their data is still considered with skepticism, because there is usually no other data to verify the accuracy of the largest values. Here we investigate the self‐consistency of the measurement, and their small scale variability, in order to define an estimate of satellite altimeter precision. Using numerical simulations of ocean surfaces and the processing involved in satellite retracking, we find that wave groups are responsible for a variance in estimated altimeter wave heights that is proportional to the square of the spectral peakedness parameter and the significant wave height. Additional variance induced by speckle noise is proportional to the wave height. The effect of wave groups generally dominates in the most severe storms. This variability requires a relatively large scale smoothing or filtering to yield accurate wave height estimates. For example, the largest ever reported 1 s average significant wave height from altimeters sampled by Jason‐2 in the North Atlantic in 2011, at m, is now interpreted to correspond to a true wave height Hs = 18.5 ± 0.3 m. The difference between 20.1 and 18.5 m is mostly due to wave group contributions to the raw measurement. We argue that wave group effects should not be included in the definition of the significant wave height, just like the maximum wave height differs from the significant wave height.