Hacker News new | ask | show | jobs
by hodgehog11 47 days ago
Both you and the comment above are correct; initializing with iid elements ensures that correlations are not disastrous for training, but strong correlations are baked into the weights during training, so pretty much anything could potentially happen.