[zvmaz]

I have met people who think they understand a particular topic I am versed in, but actually don't. Similarly, I am often wary that I get superficial knowledge about a topic I don't know much about through "laymen" resources, and I doubt one can have an appropriate level of understanding mainly through analogies and metaphors. It's a kind of "epistemic anxiety". Of course, there are "laymen" books I stumbled upon which I think go to appropriate levels of depth and do not "dumb down" to shallow levels the topics, yet remain accessible, like Gödel's Proof, by Ernest Nagel. I'd be glad to read about similar books on all topics, including the one discussed in this thread.

Knowledge is hard to attain...

[sainez]

I find the best way to learn technical topics is to build a simplified version of the thing. The trick is to understand the relationship between the high level components without getting lost in the details. This high level understanding then helps inform you when you drill down into specifics.

I think this book is a shining example of that philosophy: https://www.buildyourownlisp.com/. In the book, you implement an extremely bare-bones version of lisp, but it has been invaluable in my career. I found I was able to understand nuanced language features much more quickly because I have a clear model of how programming languages are decomposed into their components.

[joe_the_user]

I find the best way to learn technical topics is to build a simplified version of the thing. The trick is to understand the relationship between the high level components without getting lost in the details. This high level understanding then helps inform you when you drill down into specifics.

I agree but that's a good guide to build a technical understanding of a complex subject, not sufficient-in-itself tool set for considering questions in that complex subject.

Especially, I'll people combining some "non-technical summary" of quantum-mechanics/Newton Gravity/genetic engineer/etc with their personal common sense are constant annoyance to me whenever such topics come here.

[nomel]

> constant annoyance to me whenever such topics come here.

I'll say that thinking about things at the edge of my understanding, where "Eureka!" moments are low hanging fruit, results in the highest dump of dopamine out of any other activity. Having silly fun speculating (and I make it clear when I am) my way through some thought process is literally the most fun I can have. Seeing those types of conversations, full of genuine curiosity, thoughtful speculation, and all the resulting corrections/discussions/insight, etc, is why I love HN so much, and I hope it's always a place to nerd out.

One mans trash is another mans treasure, I suppose. :)

[Chocobean]

Thanks for the link. Could you give an example of something you learned better/easier after having implemented a simplified version?

Side question: Is there a entry level build your own language model or gan type learning tool out there as well?

[somenameforme]

There are a million e.g. number parsing (image to digit) neural network type programs on GitHub. Go pick one in your preferred language and break it apart, and rebuild it, looking up the concepts behind parts you don't understand. After you finish up with the above, look up 'the xor problem' to see a common practical problem (which creating a network to replicate xor illustrates, rather than is) and you'll be well on your way to a nice fundamental understanding, built from the ground up.

One of the most interesting things about this topic is that the fundamental concepts and implementations are all really simple. It's the fact that it actually works that's mind boggling. In any case, the above is not a months like affair - but like one week of dedicated work.

[lxe]

I've noticed that the learning curve stary fairly flat when it comes to understanding weights, and layers, and neural networks, heck, even what gradient descent is for... but then when it comes to actually understanding why optimization algorithms are needed, and how they work, things just spiral into very hard math territory.

I do think that maybe it feels inaccessible because we transition from discrete concepts easily digestible by CS grads into some complicated math with very terse mathematic notation, yet the math might not be as hard if presented in a way that doesn't scare away programmers.

[commandlinefan]

> an appropriate level of understanding mainly through analogies and metaphors

I think it's actually worse than that - somebody who doesn't know actually realizes that he doesn't know, but somebody who _thinks_ he understands through analogies and metaphors will confidently come to the incorrect conclusion and then argue with somebody who actually does understand the topic - often managing to convince innocent bystanders because his reasoning is easier to grasp and the so-called expert seems to be getting more and more flustered (as he tries to explain why the analogy is actually correct, but oversimplified).

[dmreedy]

I am fascinated by this phenomenon, and the double-edged sword that metaphors are.

On the one hand they're jargon used as short hand to technical concepts understood well by domain experts. And the concision they afford can lead to deeper understanding as they transcend their composite or adapted meanings and become base terminology in and of themselves (I think of e.g. Latin in English legal terminology. "Habeas corpus" has a literal meaning when translated, but the understood jargon has a deeper, and more specific meaning). At that point, they are powerful because of the precision of meaning and concision of expression they afford.

On the other hand, they lift intuitive terminology from a base language that is understood in vaguer terms by a broader audience. And this creates invisible disconnects because the abstraction created by these terms leaks like a sieve unless you know the precise semantics and have the model to use them.

By translating a discourse into a higher metaphoric level, we increase precision and efficiency amongst mutual understanders, but at the same time, we increase the level of ambiguity, the number of possible interpretations, and the availability of terms familiar to (and thus, handles to grab on to) non-understanders. And that latter situation allows non-understanders to string together what sound superficially like well-formed thoughts using jargon terms, but based on the base language semantics. But without the deeper knowledge required to understand whether a given utterance scans or not.

That's how I've been trying to wrap my head around it at least. I hope it doesn't sound like moralizing or condescension, I don't mean it to. I know I'm "guilty" of trying to manipulate metaphoric models that I don't actually understand, based on the lay-semantics of their jargon.

[melagonster]

this is how we teach students in all discipline. displaying a model and then offer better one, again and again.

[__loam]

You're basically describing a lot of generative AI developers who are applying their technology to fields they don't really understand

[wanderlust123]

Can you please suggest other books similar in spirit to the Nagel book? Would love to read some over summer