[retrac]

> Also, while it's easy to calculate word frequencies for stuff like newspaper articles, there aren't any good statistics (last I checked) around just normal everyday conversation. Because that stuff isn't getting recorded and transcribed.

Linguists have resorted to spying on people in parks and shops and restaurants, to get "natural" language unperturbed by observer effects, audience tailoring, the filter of writing, etc. (In most cases that would not be considered ethical today!)

The gold standard is something like: get acculturated to the language community you wish to study, make some acquaintances and friends, somehow finagle an invite to their homes or workplaces or social venues, and then with their permission, record their conversations as unobtrusively as you can.

There is so little data on how language is really used it is quite bedevilling. As soon as people know you are listening they change how they speak.

[rsynnott]

> Linguists have resorted to spying on people in parks and shops and restaurants, to get "natural" language unperturbed by observer effects, audience tailoring, the filter of writing, etc.

Hence the common phrase, found in most language phrasebooks: “is that a linguist hiding behind that bush?”

[itsamario]

That's what I discovered religion is for, understand the language as it was created.

Ever wonder who decides what's self-explanatory and what resonates?

[HPsquared]

This is one upside of the digital panopticon, I guess.