[crubier]

I came here to say this. This entire article should have been 5 lines, had they known about Knowledge Distillation.

Anecdotally, a similar thing happened to me when I "independently invented" synthetic data generation for deep learning back in 2016... Turns out my brilliant idea was not that original after all

[TechBro8615]

The more I learn, the more I realize how rarely I've thought of a "new idea." In fact, I'm not convinced such a thing even exists. An idea is generally a set of connections between pre-existing ideas, so perhaps a novel subset of those connections would qualify as a "new idea," but one could argue it's more of a rearrangement of existing concepts. (What was the first idea? The first thought? Does thought require language, and if so, is consciousness an emergent property of language?)

When I realized how rare it is to have a unique thought, I reframed my approach to evaluating my own ideas. Instead of worrying about novelty, I worry about relevance.

If I can't find a pre-existing similar idea, I assume I'm on the wrong track unless I can identify a compelling reason why I'm in a unique position to have an idea that nobody has ever thought or at least felt like communicating publicly.

If I can find such a pre-existing idea, then I know I'm on the right track, and often it validates a pattern I identified between a subset of connections in my previous knowledgebase, solidifying the pattern in my mind. Regardless of whether I can label it a "new idea," I can use it to work toward the next step in whatever line of thinking caused me to notice the pattern in the first place.