[coolhand2120]

> But our A.I. systems are still largely inscrutable black boxes, which makes herding them difficult. What we get out of them broadly reflects what we have put in, but no one can predict exactly how. So we observe the results, tinker and try again.

What an absurd thing to say. You don't get an abomination like Gemini without extreme and intentional tampering with the model. IIRC this was demonstrated in the HN thread where it was reported. Someone got Gemini to cough up its special instructions. Real 2001 HAL stuff.

[talldayo]

I don't think the existence of Gemini disproves the author's statement. The model is clearly broken, not only within the definitions of what you or I would consider acceptable but also within the definition set by the prudes on-high. The wildly diverse output seems especially emblematic of a hackjob finetune not dissimilar to what OpenAI does with their instruction-tuning.

The quoted comment seems to align with how Google saw the situation. They wanted a specific desired outcome (neutered AI output), they applied a documented strategy, and got a torrential wave of "observed results" from the audience.

[Terr_]

Isn't that confusing an ineffective treatment (prompt engineering) with the actual disease (insufficiently apolitical training data and use cases)?