(Fun fact: After reading that tweet, I tried to prompt ChatGPT with 100s of iteration of the word "Na", and it correctly answered "Batman!". However, prompting it with hundreds of iterations of the word "a" led to some weird poetry.)
Indeed, I just run it (GPT-3.5-turbo) in a loop on increasingly long sequences of "b b b b ....", and it went off the rails somewhere around 200+ 'b's.