Surprised this is not bigger on HN, seems like a pretty big deal that Ilya Sutskever is claiming what sounds like a breakthrough. I expect that within the next 12 months they should release something significant with that compute.
It'll be interesting to eventually find out what they've been working on, but this update in of itself doesn't give much to discuss, other than perhaps NVIDIA being their partner.
From the little Ilya has previously said, it sounds as if SSI is working on learning algorithms, perhaps some combination of better generalization / sample-efficiency and continual learning.
It's hard to know how seriously to take the "super-safe" part of SSI's name, although this announcement does talk about them developing "robustly aligned" AI. Ilya has previously talked about the role of emotions in human learning as as part of the human "value function", so perhaps some sort of artificial emotions will be used to assist learning and try to provide "robust alignment" (easier said then done without millions of years of co-evolution).
The NVIDIA announcement also says "The two companies will also collaborate on the technical advancement of NVIDIA’s current and future compute platforms, leveraging SSI’s unique insights into the future of AI". This seems like a bit of clue - that SSI are doing something that may steer the hardware in a bit of a different direction than current pre-trained transformers are doing.
Putting it all together, if I had to speculate, I'm guessing that at least part of what SSI are working on is indeed continual learning, and that they see this as resulting in per-user individualized models that might make most sense to be run locally, a future that that it seems NVIDIA may support.
From the little Ilya has previously said, it sounds as if SSI is working on learning algorithms, perhaps some combination of better generalization / sample-efficiency and continual learning.
It's hard to know how seriously to take the "super-safe" part of SSI's name, although this announcement does talk about them developing "robustly aligned" AI. Ilya has previously talked about the role of emotions in human learning as as part of the human "value function", so perhaps some sort of artificial emotions will be used to assist learning and try to provide "robust alignment" (easier said then done without millions of years of co-evolution).
The NVIDIA announcement also says "The two companies will also collaborate on the technical advancement of NVIDIA’s current and future compute platforms, leveraging SSI’s unique insights into the future of AI". This seems like a bit of clue - that SSI are doing something that may steer the hardware in a bit of a different direction than current pre-trained transformers are doing.
Putting it all together, if I had to speculate, I'm guessing that at least part of what SSI are working on is indeed continual learning, and that they see this as resulting in per-user individualized models that might make most sense to be run locally, a future that that it seems NVIDIA may support.