[317070]

> Examples where it breaks down (e.g., non-finite variance) are comparatively rare, even if there are "many" of them.

I would beg to differ. They are absolutely not rare. [0] One of the most famous people in probability theory even claimed that it's the ones with finite moments that are rare in practice. (But I couldn't find this quote back, I thought it was Poisson or Laplace)

It's even worse. Many distributions where the CLT does apply, require so many samples for them to actually work, that it does not really apply in practice anymore. Any skew in your data blows up the amount of samples you need to find things like the empirical mean.

[0] Chapter 3.4 https://arxiv.org/ftp/arxiv/papers/2001/2001.10488.pdf

[kgwgk]

> One of the most famous people in probability theory even claimed that it's the ones with finite moments that are rare in practice. (But I couldn't find this quote back, I thought it was Poisson or Laplace)

That sounds more like Kolmogorov than like Laplace. Someone more interested in mathematical issues than in practical questions.