[joe_the_user]

I'm not sure why you say "ignore".

As I read this, the author claims to have created a naive "linear" network akin regular deep learning networks but without the added (explicitly) non-linearity and shows it's trainable. He acknowledges it has to operate through non-linearity (indeed underflow) and so the mechanisms you mention sound compatible with his findings.

The point I'd see for the article isn't some magic non-linear to linear transformation but that for all we know, incidental underflow effects might operating in regular "non-linear" networks as well.