optimizer's pathestimation errorapproximation errorf₀ startf̂ your fitf* truth
medium
Grows the active circle. F₁ expands a lot — enough to swallow f*. F₂ starts closer but has a low ceiling and can never quite reach the final destination.
medium
How far the optimizer travels from f₀.
none
Straightens the path and cuts the fuel needed — RMSNorm, AdamW, SWA, etc. Variance reduction and optimizer improvements.