[npr11]

This is a neat paper - it's an interesting empirical result combining known techniques - but machine learning academics should really know better than to contribute to the over-hyping of results. For example, talking about "dreams" and "hallucinations" is not helpful - it doesn't make the work more accessible and only adds unnecessary hype.

[hardmaru]

Hi, thanks for the feedback! Honestly we didn't intend to over-hype the results. We took the terms from existing works that we knew:

1) Alex Graves on Hallucination with Recurrent Neural Networks, a 2015 lecture at the University of Oxford from a course by Nando de Freitas (highly recommended).

http://www.creativeai.net/posts/kp4bTG993JTQcqy2d/alex-grave...

2) Generating Sequences With Recurrent Neural Networks

https://arxiv.org/abs/1308.0850

"Assuming the predictions are probabilistic, novel sequences can be generated from a trained network by iteratively sampling from the network’s output distribution, then feeding in the sample as input at the next step. In other words by making the network treat its inventions as if they were real, much like a person dreaming."

There are other terms, such as Imagination, also used in the literature:

3) Imagination-Augmented Agents for Deep Reinforcement Learning

https://arxiv.org/abs/1707.06203

4) Uncertainty-driven Imagination for Continuous Deep Reinforcement Learning

http://proceedings.mlr.press/v78/kalweit17a/kalweit17a.pdf

In our work, the procedure is closer to the approaches in (1) and (2), rather than the "Imagination" approach in (3) and (4) where there are more subtle differences (i.e. planning), so we followed the terms in (1) and (2).

[ppod]

I completely agree with you. Dreams, imagination, or hallucination are appropriate terms for an agent working through solutions within its own world-model without using new external input. Would we reserve the verb 'to fly' only for birds? As Dijkstra said, "the question of whether a computer can think is no more interesting than whether a submarine can swim".

[therein]

I guess the question is, why did we need to move away from `to generate` or `to permutate` on feedback with no additional input?

It seems to have coincided with the re-emergence of neural networks and the only way I can see it is that it romanticizes the field in the expense of some accuracy of statement.

I however definitely can't claim to be immune to the charm of this romanticization, it surely appeals to something inside me.

[ppod]

'generate' and 'permutate' are more semantically general words. To convey what you mean you have to add "on feedback with no additional input". 'imagine' or 'dream' fully includes this specific meaning: it is more accurate. The only difficulty is that we are not used to applying these verbs to non-animal subjects. It is just like going out of your way to say "the submarine propelled itself through the water" or "the plane propelled itself through the air" because you don't want to use the verbs swim or fly with inanimate subjects. Why the distinction in those two particular cases; I have no idea. Maybe we're used to seeing birds glide without moving while you don't really see fish swimming without that distinctive wriggling-flapping motion.

[Cybiote]

9.9 times out of 10, I'd be in strong agreement with you. This time however, I think the use of the term "dream" is not as egregious as would be typical. When we dream, we often believe we're conscious and multiple senses will seem to corroborate this. It must be the case that we're interacting with a generative world model. Hippocampal replay is hypothesized to, in part, facilitate what their model is doing when they call it dreaming.

Hallucination on the other hand, is a less defensible use here. Hallucination refers to when predictions overwhelm sense input. Since their agent was not behaving in a way uncorrelated with its inputs, such as its predictive model overriding input data, it doesn't qualify as hallucination.

[cowpig]

I find that intuitive words like "dream" and "hallucination," as opposed to exclusively using dry jargon, make papers much more accessible.

I also don't think it's the author of the paper's job to manage hype.

[birthcert]

There is a difference between ML research and AI research. AI, traditionally, has more leeway in using intuitive, abstract, or anthropomorphized terms, over ML, which has established learning and optimization theory and a more solid foundation in applied mathematics.

Basically, the deep learning hype in popular science media, owes this status in large part, because it allows nice pictures to be shown. RL research fares well, because they can show video of the agent playing the game. I bet the choice of Doom was also made with this PR in mind, and of course publications like Wired are going to show this work to their readers, over, say, the RELU-paper (impactful in the field, but not much to write an article around).

[Retra]

It's also just plain easier and more fun to work on something when you can see how it behaves rather than trying to infer it by reading some numbers.