[westhanover]

They know their system prompt and they could easily be trained on data that explains their structure. Your dismissal is invalid and I suggest you don’t really know what you are talking about to be speaking in such definitive generalities.

[xanderlewis]

But the original comment was suggesting (implicitly, otherwise it wouldn’t be noteworthy) that asking an LLM about its internal structure is hearing it ‘from the horse’s mouth’. It’s not; it has no direct access or ability to introspect. As you say, it doesn’t know anything more than what’s already out there, so it’s silly to think you’re going to get some sort of uniquely deep insight just because it happens to be talking about itself.

[ShamelessC]

Really what you want is to find out what system prompt the model is using. If the system prompt strongly suggests to include diverse subjects in outputs even when the model might not have originally, you’ve got your culprit. Doesn’t matter that the model can’t assess its own abilities, it’s being prompted a specific way and it just so happens to follow its system prompt (to its own detriment when it comes to appeasing all parties on a divisive and nuanced issue).

It’s a bit frustrating how few of these comments mention that OpenAI has been found to do this _exact_ same thing. Like exactly this. They have a system prompt that strongly suggests outputs should be diverse (a noble effort) and sometimes it makes outputs diverse when it’s entirely inappropriate to do so. As far as I know DALLE3 still does this.

[xanderlewis]

> It’s a bit frustrating how few of these comments mention that OpenAI has been found to do this _exact_ same thing.

I think it might be because Google additionally has a track record of groupthink in this kind of area and is known to have stifled any discussion on ‘diversity’ etc. that doesn’t adhere unfailingly to the dogma.

> (a noble effort)

It is. We have to add these parentheticals in lest we be accused to being members of ‘the other side’. I’ve always been an (at times extreme) advocate for equality and anti-discrimination, and I now find myself, bizarrely, at odds with ideas I would have once thought perfectly sensible. The reason this level of insanity has been able to pervade companies like Google is because diversity and inclusion have been conflated with ideological conformity and the notion of debate itself has been judged to be harmful.