[visarga]

If they can scale it up to GPT-3 like sizes, it would be amazing. Foundation models like GPT-3 will be the operating system of tomorrow. But now they are too expensive to run.

They can be trained once and then frozen and you can develop new skills by learning control codes (prompts), or adding a retrieval subsystem (search engine in the loop).

If you shrink this foundation model to a single chip, something small and energy efficient, then you could have all sorts of smart AI on edge devices.