[gwern]

> There are a bunch of solutions to this, but adversarial models do a good job of approximating a probability distribution like this.

The problem is GANs on sequence data still stink compared to max-likelihood: they train far more slowly, more unstably, and still don't generate decent sequences compared to a char-rnn with a bit of temperature tuning & beam search. They should be better for precisely the reason you say, but they aren't.