[Toenex]

I certainly agree with this. The computer vision field is awash with papers proposing a 'new' algorithm which is then poorly compared to some select group of existing techniques under criteria chosen by the author. A paper is a very poor substitute for the code itself and really it should be mandatory for code to be submitted with the paper, especially in a field such a computer vision where the entire experimental apparatus could be packed into a zip file. That way any other group could take the code and independently evaluate the technique without reimplementation. Indeed my own experience is that often the maths described in the paper is not necessarily responsible for all the results! As you say this could even become the start of collaborative improvement.

Unfortunately my experience is that too many academic groups believe that their source code is the route to untold riches.