[t-vi]

There is A. Karpathy's recipe for training NNs but it is not a walkthrough with an example:

https://karpathy.github.io/2019/04/25/recipe/

but the general idea of "get something that can overfit first" is probably pretty good.

In my experience getting the data right is probably the most underappreciated thing. Karpathy has data as step one, but in my experience, also data representation and sampling strategy does quite the miracle.

In Part II of our book we do an end-to-end project including e.g. a moment where nothing works until we crop around "regions of interest" to balance the per-pixel classes in the training data for the UNet. This has been something I have pasted into the PyTorch forums every now and then, too.

[leminimal]

Thanks for linking me to that post! Its much better at expressing what I'm trying to say. I'll have a careful read of it now.

I think I'm still at a step before the overfit. It doesn't converge to a solution on its training data (fit or overfit). And all my data is artificially generated so no cleaning is needed (though choosing a representation still matters). I don't know if that's what you mean by getting the data right or something else. Example problems that "don't work": fizzbuzz, reverse all characters in a sentence.