I completed Andrew Ng's Coursera course and one thing it did not do is make me understand neural nets from scratch. Probably, you and I have different interpretation of "from scratch".
I actually tried to implement a neutral network from scratch by following 3blue1Browns videos, and using the same handwritten number data set. But I got stumped when I realized I didn't have a clue how to choose the step size in gradient descent, and it's not covered in the videos. Despite that problem I'd say the 3B1B videos are excellent for learning the fundamentals of neural networks.