Hacker News new | ask | show | jobs
by avidmoon 1799 days ago
Short answer, as I'm on mobile: this is structured pruning, which is hardly comparable to unstructured methods like IMP (algorithm for lottery tickets)
1 comments

Thanks. Can you give an intuition as to the difference between 'structured' and 'unstructured' pruning? What are the pros and cons of each?
In unstructured pruning, individual weights of a weight tensor are pruned with no constraints on their position in the tensor. In structured pruning, there are constraints on which weights are pruned. Think of pruning an entire output channels of a convolutional layer vs pruning arbitrary weights of the same tensor. Unstructured pruning allows for a greater percentage of the weights to be pruned, but unstructured pruning doesn't speed up a network on hardware accelerators, which favor dense matrix multiplications.