[andai]

Fascinating. A few years ago a friend and I finetuned GPT-2 on our WhatsApp chat. So it was just a long text file of:

Mark: wassup

Andy: just chilling

It simulated our conversational style and topics quite well, though GPT-2 reads like a glorified Markov chain. Sometimes the outputs were absolutely hilarious and inappropriate. GPT-2 was peak comedy.

My friend described GPT-2 as "like watching a toddler learning how to walk. When it stumbles it's cute and funny." GPT-3, not so much...

Also, it was oddly (painfully) accurate as far as personality goes... like looking into a mirror. For one thing, I talk way more, and this was reflected in the model's output. For another, I am constantly trying to turn my life around and failing, but ever optimistic... and talking about creative plans endlessly without much execution. (So GPT-2-andai ended up the same way...)

[codelion]

GPT-2 is surprisingly good at fine-tuning such conversations even now. I gave a talk recently on "Sparks of Digital Immortality" that covers a bit about how we did it - https://www.youtube.com/watch?v=F9-Qk86QyMM