[simonw]

There's a video of a previous version of this tool here which I found really helped me understand what it was demonstrating: https://twitter.com/thesephist/status/1617747154231259137

It's really neat to see how this sentence:

> The first time I write this sentence, the model is quite confused about what token is about to come next, especially if I throw in weird words like pumpkin, clown, tweets, alpha, teddy bear.

Shows that the words pumpkin, clown etc are considered really unlikely. But when the sentence is repeated a moment later, all of the words become extremely predictable to the model.

Also worth noting: this demo runs entirely in the browser! It loads a 120MB ONNX version of GPT-2 using Transformers.js.