[revel]

I think you're getting to the heart of what a smart approach to regulation would look like (for me at least). Try as I might, I can't imagine a situation where someone is able to subvert an LLM to come up with a world-ending scenario using its knowledge of onion-based recipes or by painting fingernails. There is no point to regulating every single potential use of an AI: to do so is to cede the future to a less scrupulous nation. We already know the kind of data and functionality that will likely lead to a dystopian hellscape because we already do those things and they are already regulated.

Paradoxically, a heavy-handed approach to regulation could guarantee a bleak future for humanity. We risk increasing the incentives for malicious use at the same time as we make legitimate use prohibitively expensive / regulated out of existance. If the war and crime machine is cheap and easy to monetize, don't be surprised when that's what gets built.

The future could be unimaginably better for everyone, but we won't realize that future unless we get the regulation right.