[fwilliams]

I haven't read the paper so I can't comment on the success of the method, but most applied ML research will show their best results in the publication and leave out failure cases.

These images look impressive, but without doing a proper in-depth analysis, more general claims of improvement on a task are hard to make And while it's totally possible that, in this case, the improvements are significant, it's dangerous to extrapolate from just a few examples in a paper.

[cscurmudgeon]

A bit related: On top of that, papers in top venues frequently don't publish data, let alone their implementations. How hard is to just dump a zip of your data in 2018? (I don't want to single out any particular papers).

I am talking about data that should not be that huge. Does no one else feel frustrated by this?

[gshrikant]

Exactly this. Science is supposed to be reproducible. Computer science is much more easily reproducible than many other disciplines and it is just sad that many papers don't publish their data/code to reinforce their conclusions.

I could understand Facebook/Google/BigCo. doing so when the data in question might be internal implementation/tech but the trend is far more prevalent.

Sharing experiment data/parameters would not only help people verify your results but also help students learning how to design and carry out experiments.

[cs702]

Yes... but no one's making "general claims." This work suggests the technique can significantly improve the quality of the representations learned by existing architectures. Please don't resort to straw-man arguments.

[cs702]

fwilliams: looking at this comment, in hindsight, I wish I could soften it. I did make what can only be described as "general claims," and now feel more than a bit dumb for saying otherwise... sorry about that. (What I was trying to say, but evidently did a poor job of articulating, is that the authors of the paper are not making general claims, even though they do speculate about the potential usefulness of their work.)

[fwilliams]

Okay so I went and read the paper. They discuss generative modeling in section 5 and in the appendix (section 7.2).

Section 5 claims "the corresponding CoordConv GAN model generates objects that better cover the 2D Cartesian space while using 7% of the parameters of the conv GAN". There isn't really quantitative analysis beyond a couple of small graphs discussing this any further. Section 7.2 and 7.3 visually compares the results between the generator's output of interpolated noise vectors in the latent space. The results look good but without quantitative analysis, they are very preliminary.

Generative modeling is tricky and I think in your first comment, the jump from a few nice images to CoordConv can "significantly improve the quality of the representations" is a big one given the sparsity of evidence in the paper. I'm not saying that you're wrong but your original comment seemed a bit misleading to me.

[cs702]

Yes, the evidence is preliminary and not extensive. Yes, generative models can be tricky (to say the least). No one's claiming otherwise.

Visual evidence is important for generative image tasks, given that we can't measure any of these DNN generators against a "true" statistical model that generates the data.

For a DNN to be able to generate more realistic transformations of generated images from low-dimensional representations, it must learn higher quality representations... or are you saying otherwise?