[Scea91]

This approach is dangerous in statistics and machine learning. Things often work on your toy examples but it may be for totally different reasons than you think and it can bite you later. Since it looks like it works to you, you will never look into the necessary theory. With the approach 'theory first' you might not get nice results as fast but I would bet that your understanding would be much deeper.

I am not saying that you need to read all the papers before coding a line, but the approach of coding your way into it will give you more shallower knowledge if you don't follow up with the theory.

[nkozyra]

Agreed; I'd actually recommend against this. You should know the math and reasoning behind the specific algorithm rather than dissecting it via code.

[zk00006]

In fact there is no correct answer to that and it depends on your current state of knowledge and needs for your project. It also changes over time.

I would be careful to claim that with theoretical approach you will always get better understanding.

[eftychis]

I agree with posts above. Diving straight into 'practice' in statistics (and other fields e.g. cryptography being the most notorious) leaves you open to a great many of pitfalls. Best case you will be inefficient with your approaches.

Your examples may work, a couple of testing sets giving you high confidence, and then you attempt to use it in the wild and everything falls apart.

At the same time machine learning is a lot about data cleaning, bootstrapping, picking the right algorithm with mininum iteration, minimizing your iteration cycle as much as possible etc which you don't gain until you actually mess around and get your hands dirty. Plus there are little implementation tidbits specific to each project.

[s_ngularity]

That might work for some simpler machine learning algorithms, but in deep learning I think you'll have an even harder time correctly figuring out intuitively what's going on than you would learning the math.