[gcr]

GAN training is still spooky mysterious and can easily fail in nonintuitive ways.

Sometimes GANs converge or not depending on the random number seed, even with the same hyperparameters.

[gwern]

I'm not sure about that. The new GAN models over the past 2-3 months, like LS-GAN or WGAN, all seem to train much more stably. I've beaten up on WGAN with all sorts of strange tweaks and hyperparameter settings and while it may not work well, it's never catastrophically diverged on me the way DCGAN would at the drop of a hat.

[AlexCoventry]

Have you found any good ways to speed it up? The five-fold training on the Critic is very expensive.

[gwern]

No, not yet. I suspect that increasing the discriminator-only learning rate might help but haven't tried.

[igul222]

Try removing BN from the critic :)