[swatcoder]

Because of how GPT works, the resources needed for good inference (generating output) grow nonlinearly with respect to tokens involved (more tokens require much more resources) and so there’s a practical wall before you just run out of resources to apply.

It’s not very efficient. It’s like if your calculator could use a little solar power thingie for numbers that were only a few digits, but needed a diesel generator to crunch on 8 digit numbers, and a nuclear plant to crunch on 12 digit ones. Practically, you’d have no choice but to limit yourself to something manageable.

Future models may be more efficient, and future hardware solutions may be more efficient, but those things don’t get sorted out overnight any more than fusion power.

Beyond that, I think it’s important that you understand that Bing Assistant doesn’t express desires. It picks common sequences words based on its training data. It doesn’t know what nuclear codes are. It just knows what it looks like for a message about wanting nuclear codes to follow some other message in a dialog (probably a pattern it picked up on a forum like Reddit) and so it dutifully put that text after the prompt it had been given. There’s no will or consistency to it.

With enough resources, you could drive it through a feedback loop where it kept prompting itself and see what happens, but the feedback loop would just produce noise like any other simple feedback loop because it would just keep either honing in on the most boring and common continuation to the last thing it gave itself or it would start diverging off into nonsense. Because it’s sooooo inefficient, you can’t give it enough resources for it to be stable and interesting for very long.