Hacker News new | ask | show | jobs
by talles 507 days ago
For deep learning:

1. Linear algebra. Be comfortable with vector transformations in the vector space. This is the framework to understand how data is represented and what is going on inside the model.

2. Calculus. Specifically derivatives, up to partial derivatives and the chain rule. This is needed to later understand backpropagation, the learning. It's fine to skip integrals.

3. Vanilla neural network. Study how a simple feed forward and fully connected neural network works, in detail. Every single bit about it.

I wouldn't worry or plan anything ahead until having those. After number 3 you'll have different branches to follow and will be better equipped to pick a path.

1 comments

Do you have specific courses to recommend? Linear Algebra is probably a good one to be learning from Gilbert Strang.
I prefer learning from textbooks than online courses, here are my picks (in the order of my first post):

1. Sorry, I do not have a suggestion here. I'm Brazilian and I happened to find an amazing PDF of an obscure book (by a not well known professor). In Portuguese though.

2. "Calculus Made Easy" is my book of choice. Don't dismiss it because of the 'cheap' title, it's a very good book.

3. It's super important to get the intuitions first, and not just the math. So I mostly used two books, both heavily illustrated: "Deep Learning - A Visual Approach" and "Deep Learning Illustrated". Those are way better as an introduction to the field than the 'big names' (Goodfellow/Bengio, Bishop, etc).

Normally if I'm on youtube, it means I'm struggling with something. The statquest channel is amazing, it's the most 'baby steps' explanations I've ever seen. If I find a video on what I want to understand there, I don't waste my time trying other channels anymore.