| If it helps tune your probability weightings at all, almost every aspect of the field of AI is exponential and this has been true for decades. 20 years ago computers basically couldn't do voice or image recognition at all. 10 years ago it worked, but was still very flakey. Today it's super human. I agree that if progress flattens out very rapidly and very soon then we'll at least have more time, but right now everything would suggest progress is still exponential and that flattening will now need to come quick and hard. This is largely why I've significantly increased my doom probability over the last year. I don't think there's any reasonable progress curve you can draw at this point that would suggest ASI isn't coming very soon. I suspect our only hope is nuclear war or a long tail good post-ASI outcome. In regards to Sol, I think it's worth remembering that people were saying just a couple of years ago that AI can't even count the number of letters in a word or do basic maths. The fact the bar has risen so much we're saying they're still meh because they're not one-shotting problems is at least worth noting. > I think one real risk is solo or sleeper cell type nefarious actors having a semi-competent engineering consultant which can enable small-scale weapons deployment. Yep, the problem with AI isn't that there's any one risk that anyone can reasonably predict with high probability, but that there are many potential risks with signal digit probabilities. When you consider the overall risk landscape, that's when things get worrying. > So AGI take over, meh, low chance IMO. Right now I'd put this at like 10-20% probability. It seems unlikely to me, but quite possible if AIs of the future develop their own goals and are sufficiently super human. The biggest risk is collapse I think. I don't think the world is stable in a post-ASI world. When the magic wish box will grant me whatever wish I like, I might wish something bad. In my mind this is the biggest risk by far. |
That's very true. I think LLM's have passed the Turing test and I remeber that used to be a very high bar. I remember being flabbergasted when chatgpt went public.
I guess I'm just not convinced self-improvement is a thing...yet
I think it comes down to an information theoretic question of: Can a LLM create information that is a projection outside it's data subspace? Something like can it increase it's "matrix rank" and not just create a dependent row. What ever metaphor... it's fun
> but that there are many potential risks with signal digit probabilities. When you consider the overall risk landscape, that's when things get worrying.
That's a good point too. The number of possible bad and good things goes up and every gets access to doing more complex things previously contained to someone's skill silo.