[iskander]

Often the tweaks are then used to inspire more solid mathematical footing. An interesting example of this is going on with the recent surge of interest in neural networks and deep learning at machine learning conferences. What used to be hacks and heuristics are being given a more rigorous narrative. Of course, as soon as we have a better model for neural networks, someone immediately finds a non-rigorous tweak to improve its performance. And the cycle goes on...