[havercosine]

'Little..' series books are famous for their pedagogical style: small bit of concepts, presented as a dialogue, making you think at each step, in a well thought out order. It will appeal to anyone who wants to build it out themselves, understand the 'big' ideas in the field. Skim through the second chapter to see how this will be done: they define line function in inverted way where slope and intercept are taken as parameters later. Feels weird but exactly the mental model shift we should be doing when thinking about ML.

Read the preface. A very high praise coming from Guy Steele Jr and Peter Norvig.

At the same time, a warning! Scheme based introductions don't appeal to everyone. Some people feel way out of comfort zone in coding with it (which is sad because it is much simpler). Also, the utilitarian appeal is low: it won't right away see a step change in your Pytorch knowledge or whatever. The appeal of these books is to think deeply about fundamental ideas by implementing them in simplest language without too much help.

In short, YMMV. But if you have a long term view it might help you a lot than sort of currently fashionable trends. (Though I must admit that fast.ai is not just a flavor of the season resources but much better!)

[hedgehog0]

So you mean that fast.ai maybe a better resource for people new to general ML?

[havercosine]

Well, hard to say if it is a better resource. But it will appeal to those who are in a hurry (and yes, it is okay to be in a hurry). For example, fast.ai first chapter will have you build a dogs vs cats classifier. But how do they build it? By doing `from fastai.vision import *`. They justify it by claiming to use Teaching the whole game approach. If you are a somewhat experienced engineer who already knows Python, what is Jupyter then this approach will get you started quickly. For someone who feels did I spend my time/money but nothing exciting is happening yet, this is a good start. But for some it is crazy and makes them feel even more scared (what is happening with all this imports, how does it work).

My criticism with fast.ai, (I am part time educator), is that this approach is an information overload and poor sequencing. Their comparison with Teach whole game approach is flawed because a game of, say Football, is essentially simple. So you can say just start kicking around. But we don't teach chess this way. It is accepted that you have to spend some time upfront to learn the rules before you can play even simple game. Sure one need not learn castling or en-passant upfront. But you get the drift.

This book (looking at the preview chapters) is going to follow the lego blocks approach or bottoms up approach to build it. For me, this is correct way to teach supervised ML focussed on neural networks and deep learning. We have a problem of too many library plumbers in the ML field currently. People who can piece together library function calls without knowing why it is working. But this house of cards is not sustainable strategy to build AI based application over long term.

Long story short, the book will need patience but that patience will be worth it!

[hedgehog0]

Thank you for your reply.

I'm a Lisper (used Common Lisp, Racket/Scheme, and Clojure) and a math graduate student, and am interested in learning more about ML and DL so that I could potentially use them in research in the future, to come up with constructions and counter-examples, so I may have some time in my hand. I have always been a fan of Dan Friedman, so I'm definitely thinking about getting this book.

I found out that fast.ai seems to be of this approach that you mentioned. Since you are an educator, if I'm interested in learning more about ML and DL (as well as the math behind and implementation), which book(s) would you recommend me to study (I have some experience with Andrew Ng's coursera course many years ago)? I did some research here and people seem to recommend different books...

Thank you!

[barrenko]

Probably, but "two Python books" mentioned in this thread are even more suited to a general beginner.

[hedgehog0]

Thank you. Maybe it's not modest, but I do not consider myself to be a "general beginner". I majored in math and CS, so I'm thinking about getting something more hard-core and serious...

[barrenko]

Great. Have you tried just jumping in to Karpathy's video series on youtube? Maybe that plus reading some papers?

[havercosine]

I second this. Given your background (lisper) maybe do the Little Learner book once it is out + Karpathy's video series. Follow it up by building a slightly complicated application in your favourite domain (text, images, videos, time series).

Also word of advise from my experience (I'm not an expert in DL either): Think of DL field as a game of lego blocks. The ideas in this book / Karpathy's videos are the basic lego blocks: parameterised linear functions, non-linearities, auto-grad, cross-entropy / KL divergence loss and gradient descent. Then there is entire body of more complex legos discovered simply by practice (alchemy!): transformer blocks, layer norm, max-pooling etc. It is impossible to understand how the second kind were obtained from first principles. The trick is to not beat yourself up about the advanced blocks too much but just play around, read up things in papers. Just focus on fundamental blocks.