[LeanderK]

I think machine learning researchers are well aware that successful optimisation is only possible using the right priors. This is explicit in bayesian machine learning but also implicit in neural networks in the choice of the architecture, optimisation algorithm and hyper parameters. It's a well discussed problem and a lot of researchers have a serious background in optimisation, theoretical machine learning and other related areas.

[tarxzvf]

What exactly are the right priors for general intelligence? And keep in mind, whichever prior you choose, I can design learning problem where it will lead you astray.

This paper provides some interesting results on the weakness inherent in universal priors: https://arxiv.org/abs/1510.04931

[ac42dgu]

Related question: What are the adversarial examples for human intelligence? We know some for the visual and auditory systems, but what about the arguably general intelligence of humans?

Maybe we can work our way backwards from the adversarial examples to the inductive biases?

[sdenton4]

'Thinking Fast and Slow' is basically all about the rough edges of human thinking.

The interesting tradeoff with ML systems is that you trade lots of individual human crap for one big pile of machine crap. The advantage of the machine crap is that you can actually go in and find systemic problems and work on fixing them at a 'global' level. On the human side, you're always going to be stuck with an unknown array of individual human biases which are incredibly difficult to correct.

[opwieurposiu]

I think fractional reserve banking has done a pretty good job of fooling everyone.

[someguyorother]

That's for reinforcement learning, right? What is the adversarial learning problem in say, classification based on Solomonoff?

If hypercomputation is possible, then anything based on Kolmogorov complexity would be SOL, but if not... is Solomonoff induction just too expensive in practice?