[tariqali34]

And since we don't know how humans decide their own goals (because knowing that would be a very revolutionary discovery that would immediately be used in a variety of other fields, including politics and advertising), we can never really establish a road map to building "strong AI"/"AGI"/"Robot Gods" (or even recognizing if we have built one by sheer accident). Clever. I like that.

There are probably ways to "cheat" your criteria though by having AI simulate the idea of discovering goals and acting on them, such as building a bot that searches Tweets on Twitter and then writing Tweets based on those Tweets it discovers. But these are "cheats" and won't be universally accepted. We could argue for instance that this bot really has a higher-level goal of finding new goals and carrying them out, and is only coming up with "lower-level" goals based on its initial "higher-level" goal. So, again, you're probably right. We don't know how to have AI create goals on its own...we can only give it to them.

I would say that "dumb slave[s]" or "capricious, self-serving monster[s]" are still threats to worry about though. Just because robots do what we tell them to doesn't mean that they will do what we want them to. Bugs and edge cases can exist in any exist system, and the more complex the system, the more likely it is for those bugs and edge cases to slip by unnoticed. These bugs/edge cases could lead to pretty catastrophic results. Managing complexity when programming AI would be a good place for "AI Advocates" to focus on.