[sdenton4]

With access to the actual model you could try to do some real compression: encode a sequence, then search for the shortest possible text which gives an embedding near the embedding of the original text. This kind of 'optimize the input' is basically how Deep Dream worked...

[dragonwriter]

Yeah, what you'd need is something like the OpenAI embeddings API, but with a model compatible with the GPT model you are using. (Though it might be worth trying it with that API and the model it has—it won’t be perfect, but you don't need perfect to be valuable.)

[wingated]

Obligatory link to my own work - we did exactly this in a recent ACL paper: https://arxiv.org/pdf/2210.03162

We used soft prompts, not emojis.

You do need full access to the model in order to do this, but we found that prompts can be severely compressed an still retain a lot of information.