Hacker News new | ask | show | jobs
by mensetmanusman 1982 days ago
It’s an interesting thought experiment to consider models that have more parameters than there are data points being analyzed.

What does that mean?

2 comments

It basically means you need to learn about double descent:

https://twitter.com/hippopedoid/status/1243229024085835779

and regularize your models well, implicitly or explicitly. Your model might otherwise memorize the data instead of learning features.

All networks are like this. In a network of N nodes you have N^2 potential relationships. That’s just simple paired relationships. You can still go to higher order groups.