[slashdave]

Orthogonality ensures that each weight has its own, individual importance. In a regular MLP, the weights are naturally correlated.