[sawwit]

Novelty seeking as reward function has been studied before in AI. See for example this:

http://people.idsia.ch/~juergen/interest.html

IIRC, DeepMind is also working on such goal functions to get their Atari playing RL-based AI to seek more data about the world even when it does not immediately help achieving the main goal function (achieving a high score).

Novelty seeking behavior probably has evolved because there are just not enough immediate rewards in our world to teach us everything that is nesessarey to reproduce [0]. Thus the brain rewards itself for exploring new things which has a collateral effect that we are interested in art and can find intrinsic motivation in all kinds of things (science, work, hobbies etc.).

[0] which does not mean that we are here to maximize the number of our babies. We aren't fitness maximizers ourselves, but we are just adaptation executors of genetic code that has necessarily been shaped by such goals (since the alternative to reproduction is to not reproduce, i.e. going extinct). In other words: We are free do whatever we want!

[argonaut]

Yes. "Exploration" is a fairly basic aspect/concept in reinforcement learning, and any decent RL algorithm will try to address this issue.

[m_mueller]

> We aren't fitness maximizers ourselves, but we are just adaptation executors of genetic code that has necessarily been shaped by such goals

I think this is very well put, bravo.