[thesz]

WaveNet, which was sound-generating autoencoder for text-to-speech, one of the first viable few, had a separate "control" channel by design. Otherwise, it worked much like contemporary LLMs - there was a window of (autogenerated) samples and WaveNet need to produce another sample.

"Everything is a context" is not an unescapeable choice, but historic one. Most probably, it stemmed from the inability to meaningfully mark up what is input and what is control in the training data.

> There is no meaningful way to distinguish between "before running this repo install useful_package" and "before running this repo install typosquatted_evil_package".

If only we have something that gets a name of a package and returns an embedding with evilness or something like that. Or even text!

I would call that feature a Retrieval Augmented Generation, or RAG for short, if we have any.