Return to Colloquia & Seminar listing
How much can pairwise statistics explain about (deep) learning?
Joint Math/CS Theory| Speaker: | Dhruva Karkada, UC Berkeley |
| Location: | 1131 Kemper |
| Start time: | Mon, Oct 26 2026, 4:10PM |
Description
Learning from (only) two-point statistics can be a surprisingly powerful strategy in high dimension. I'll discuss some examples in which studying the behavior of simple learning algorithms on "Gaussianized" data turns out to be a predictive proxy for practical learning methods on real distributions. In particular, I'll show how Gaussianization ansatzes can: 1) predict (including order-unity prefactors) the generalization errors and sample complexity curves of lazy neural networks; 2) predict the emergence and geometric structure of representational manifolds in the residual stream of LLMs; 3) qualitatively explain why and how the Muon optimizer provides such stark acceleration over gradient descent in PCA-like problems. My main message is that we should fully understand the Gaussian baseline for our learning algorithms, which may help clarify/identify the "extra magic" that neural networks do in extracting and exploiting non-Gaussianity in natural data.
