[felippee]

Author of the post here. I totally agree that negative stuff should be published. But without the fanfares. I think they could have changed the tone of that paper and I would not have an issue with it. It is likely that if they did that they'd never go through some idiot reviewer who expects "a positive result" or some similar silliness. This is not a perfect world. The paper as is makes strong claims about the novelty and usefulness of their gimmick. If it turns out your stuff is at least partially hollow and you take on the pompous tone, you have to be ready to take some heat. Science is not about tapping friends on the back (which BTW is what is happening a lot with the so called "deep learning community"). Science is about fighting to get to some truth, even if that takes some heat. People so fragile that they cannot take criticism should just not do it.

[mlthoughts2018]

I completely agree regarding fanfare in deep learning. There are lots of “incremental improvement” papers, GitHub repos, blog posts, etc. and these are totally fine in principle — but they are without a doubt branded as “state of the art” often with messy or incomplete code and little capability to reproduce results.

An additional frustration point I always have is when network architectures are not even fully specified.

Try reading the MTCNN fave detection paper. How, exactly, is the input image pyramid calculated? By what mechanism, exactly, can the network cascade produce multiple detections (i.e. can it only produce one detection per each input scale? If more than one, how?). In the Inception paper dealing with factorized convolutions, just google around to see the deep, deep confusion over the exact mechanics by which the two-stage, smaller convolutions ends up saving operations ovrr a one-stage larger convolution. The highest upvoted answers on Stack Overflow, reddit, quora are often wrong.

And these examples are from reasonably interesting mainstream papers that deserve some fanfare. Just imagine how much worse it is for extremely incremental engineering papers trying to milk the hype by claiming state of the art performance.

Still though, at the end of the day, I’d rather that more papers are published and negative / incremental results are not penalized, because the alternative file drawer bias would be much worse for science overall.