[jameshart]

Oh, I’m so looking forward to my next coding interview.

“Okay, could you show me on the whiteboard how you might go about writing a program that can reverse a string?”

“Great, so I’m going to start by initializing a simple transformer-based neural network with 175 billion parameters and 96 attention layers, and I’m going to train it on a corpus of 45 terabytes of data tokenized into about 500 billion tokens…”

[oneepic]

"Cool, so what do you think would be the time complexity of that? Do you think we can maybe do better than that?"

[jameshart]

Actually it turns out it's O(n). Which goes to show that constant factors can be more important than you think when looking at raw time complexity big-O.

[nullc]

Ehhhh, it's O(n) when the string to be reversed is bounded under its input window.

Because of self-attention it would grow quadratically for larger inputs, no?

[visarga]

Not if you also want a short poem where each word starts with a letter from the original word, and then a short literary commentary on it.