[orm]

GPT-3 is trained on text prediction, and there's been a lot of commentary about the generation aspect, but some of the applications that excite me most are not necessarily about the generation of text, instead, GPT-3 (and also other language models) create very useful vector representations of natural language as a side effect that can then be used for other tasks with much less data, or with too much extra data. Using the text prediction task as a way to supervise learning this representation without having to create an expensive labelled dataset is very helpful, and not just to language tasks. See for example the CLIP work that came out recently for image classification, using GPT-3 and captions to supervise training. There is other work referred to in that blog post that also exploits captions or descriptions in natural language to help understand images better. More speculatively, being able to use natural language to supervise or give feedback to automated systems that have little to do with NLP seems very very useful.

[hooande]

"More speculatively, being able to use natural language to supervise or give feedback to automated systems that have little to do with NLP seems very very useful."

I agree with this, and it isn't entirely speculative. One of the most useful applications I have seen that goes beyond googling is generating css using natural language. ie, "change the background to blue and put a star at the top of the page". There are heavily sample selected demos of this on twitter right now https://twitter.com/sharifshameem/status/1282676454690451457...

This is definitely practical, though I wouldn't design my corporate website using this. could be useful if you need to make 10 new sites a day for something with seo or domains