[skybrian]

It's missing the part where you're not sure what the answer is, so you look it up, do an experiment, or ask someone.

That's a bit much to ask of a fiction generator, though. It doesn't know it's not okay to make things up. All its training is about making things up whether it knows the answer or not.

[goodside]

Look into RETRO, which greatly reduces the model’s tendency to confabulate by teaching it to query a document database known to be truthful, and justify its answers with specific references: https://www.deepmind.com/publications/improving-language-mod...

[skjoldr]

That looks like "if confidence < %some number%, then try to expand the dataset using known related sources". Load a library of short book descriptions into such an AI, and it should reproduce this behavior. Though it also needs the capacity to learn. Otherwise you could put a snippet that automatically queries it for better materiel about the subject when its confidence is low, so it just tells you where it thinks you should look. That seems doable now.

[skybrian]

I'm no machine learning expert, but agree that it seems doable. I think rather than an if statement (which seems like a bit of a hack), it would be more principled to somehow train it on when it's worthwhile to do a search, sort of like a multi-armed bandit problem.

Either way, it's not just scaling up, it's changing the algorithm.

[xboxnolifes]

> It doesn't know it's not okay to make things up. All its training is about making things up whether it knows the answer or not.

Sounds very human to me.

[skybrian]

Humans are sometimes curious enough to look for answers.

[solardev]

This model has already spent more time learning than most humans ever would...

Imagine if we gave it more power and funding. Maybe redirect all of Congress's salary to it?