[danaris]

Well, it will give you something in English.

You don't actually know that it's a) asking them the questions you asked, rather than just something it hallucinated that seemed plausible in their language, or b) relaying to you a translation of their answers, rather than just something it hallucinated that seems plausible in English.

Because LLMs will always, always, always hallucinate. They are not reliable, and this is not fixable. And if you do this as a matter of policy, you are populating your surveys with incorrect data.

[comrade1234]

I was on an academic NSA project 10+ years ago for extracting metaphors from text. It was necessary for translations - the metaphor "my lawyer is a shark" has completely different meanings in different cultures/languages. We started out using taxonomies/trees of words. The word "lawyer" is not an animal but the word "shark" is, so it's probably a metaphor. The NSA said that this was the wrong approach so we switched to an ngrams approach which also worked.

However these llm models are surprisingly good at understanding metaphors and how they are interpreted in different cultures. Better than a human.

[archagon]

If there is nuance to the word “shark” in different languages, it’s very likely that LLMs will bias towards the mainstream, Western one. See: https://www.linkedin.com/posts/celesteheadlee_who-gets-cited...

[joquarky]

I can feel the spittle coming off of this comment.

You're not going to convince anyone here. We know they are not reliable, but we can still stochasticity extract value.