Hacker News new | ask | show | jobs
by deepsquirrelnet 384 days ago
This is a good call-out, and one that might be fixable by setting a repetition penalty in generation — or better in training.
1 comments

I recall in 1980's, style-analyzing software reporting word counts for words with high frequency in the text but low frequency in general usage - jargon. The suggestion was to re-word, and it was often a relevant clue to re-think by examining why one word was doing so much work. Detecting that sounds like more than repetition, but still feasible probabilistically and relevant at both usage and conceptual levels.