[coolspot]

Interesting that even though GPT-J is supposedly English model, it somehow generated valid Russian text for the prompt:

>>>Привет,<<< друзья. Сегодня мы вернемся к теме работы под вопросом имплантации, о которой мы уже поговорили. Она неизменно вызывает специальное внимание мусульманских людей. Да, эта тема вызыв

[iggldiggl]

It's mentioned somewhere in here (https://arxiv.org/abs/2101.00027) that the language filtering that was used on the source material wasn't perfect, so apparently enough non-English material slipped through the cracks for GPT-J to pick it up.

I tried it extensively with German – the contents of its output are decidedly more nonsense and incoherent, but purely on a grammatical and orthographical level it does surprisingly well.