[quicknir]

I normally don't bother but this comment is so profoundly ridiculous I had to say something.

Tenured ML professors at the top 100 or so universities in the world aren't "most of us". A very large chunk of these people are geniuses. Those jobs are incredibly hard to get, and most of these people are reading everything that is getting published, on an ongoing basis, and are outputting something novel, on an ongoing basis.

The fact that you think that John Carmack, because he's a name that you've actually heard of, is going to go into ML and suddenly make some giant advance that all the poor plebs in the field weren't able to do, is only a reflection of your misunderstanding of what's already happening in academia, not on Carmack's skills or abilities.

You're acting as though everyone are just low level practitioners using sklearn, and it would be a great idea to have some smart people work on developing something novel. Guess what: that's already happening, with incredibly smart people, on an incredibly large scale. Carmack doing it would just be another drop in the bucket.

[fastball]

  Tenured ML professors at the top 100 or so universities in the world aren't "most of us".

Too bad we're talking about AGI, not ML.

  Those jobs are incredibly hard to get,

You don't need to be a genius in order to land a hard-to-get job, and you thinking academia is somehow better at making the absolute smartest people rise to the top is cute.

  The fact that you think that John Carmack, because he's a name that you've actually heard of, is going to go into ML and suddenly make some giant advance that all the poor plebs in the field weren't able to do, is only a reflection of your misunderstanding of what's already happening in academia, not on Carmack's skills or abilities.

I don't think that. Mostly because we're not talking about ML, but also because I don't expect eureka moments from people that have been trying to solve a problem for a long time as much as I expect them from someone that hasn't properly tried their hand at it. Academia produces consistent results and consistent improvement. That's not what I'm looking for.

  You're acting as though everyone are just low level practitioners using sklearn, and it would be a great idea to have some smart people work on developing something novel. Guess what: that's already happening, with incredibly smart people, on an incredibly large scale. Carmack doing it would just be another drop in the bucket.

sklearn hardly seems relevant to AGI, so I'm not sure why I'd act like everyone in the AGI field merely a novice practitioner of it.

[criddell]

> Carmack doing it would just be another drop in the bucket.

If this research is as compute intensive as it seems to be, Carmack's contribution might be that he increases the rate other researchers can add their drops to the bucket.

Carmack isn't the first techie to take on a big hard problem. Jeff Hawkins, a name many of us also know, did as well.

[quicknir]

Yes, he may well improve some algorithm, or rewrite some commonly used tool to improve efficiency. And researchers are often not incentivized to do that, so it would be great. But a far cry from the picture people are painting about him soaking up the field and using his genius to solve some major problem quickly.

If by "techie" you mean, professional software engineer, that's fine, but there's no reason to assume that a professional software engineer is going to be magically better at AI research than... professional AI researchers? He's probably going to be substantially worse.

Also, your statement below:

> That's probably true. I look at this as Carmack running his own PhD program. I expect he will expand what we know about computation and the AGI problem before he's done.

Makes it clear to me that you don't really get it. Carmack, at best, might know enough right now to be in a PhD program. I doubt that he has anywhere near as much knowledge, insight, or ideas for research, as top graduate students. He's in no position to mentor graduate students.

[criddell]

> If by "techie" you mean, professional software engineer, that's fine

No, I mean technologist. He has a pretty solid history with software, physics, aerospace, optics, etc...

> might know enough right now to be in a PhD program

Yeah, that's what I'm saying. The frontier in AGI or even just AI is enormous and I think I would be more surprised if Carmack were not able to find some place he could expand the border of what we know.

[PaulHoule]

Granted.

But the academic activity is focused around the kind of activities that Kuhn calls "Normal Science".

That is, ML researchers mainly do competitions on the same data sets, trying to put up better numbers.

In some sense that keeps people honest, it also lowers the cost of creating training data, but it only teaches people how to do the same data set over and over again, not how to do a fresh one.

So a lot of this activity is meaningful in terms of the field, but not maybe not meaningful in terms of useful use.

I saw this happen in text retrieval; when I was trying to get my head around with why Google was better than prior search engines, I learned very little from looking at TREC, in fact people in the open literature were having a hard time getting PageRank to improve the performance of a search engine.

A big part of the problems was that the pre-Google (and a few years into the Google age) TREC tasks wouldn't recognize that Google was a better search engine because Google was not optimized around the TREC tasks, rather it was optimized around something different. If you are optimizing for something different, it may matter more what you are optimizing for rather than the specific technology you are using.

Later on I realized that TREC biases were leading to "artificial stupidity" in search engines. IBM Watson was famous for returning a probability score for Jeopardy answers, but linking the score of a search result to a probability is iffy at best with conventional search engines.

It turns out that the TREC tasks were specifically designed not to reward search engines that "know what they don't know" because they'd rather people build search engines that can dig deep into hard-to-find results, and not build ones that stick up their hand really high when they answer something that is dead easy.

[munificent]

> But the academic activity is focused around the kind of activities that Kuhn calls "Normal Science".

True, but even Kuhn would note that most paradigm shifts still come from within the field. You don't need complete outsiders and, as far as I know, outsiders revolutionizing a field are quite rare.

You need someone (a) who can think outside the box, but you also need (b) someone who has all of the relevant background to not just reinvent some ancient discarded bad idea. Outsiders are naturals at (a) but are at a distinct disadvantage for (b).

I think what's really happening in this thread is:

1. Carmack is a well-deserved, beloved genius in his field.

2. He's also a coder, so "one of us".

3. Thus we want him to be a successful genius in some other field because that indirectly makes us feel better about ourselves. "Look what this brilliant coder like me did!"

But the odds of him making some big leap in AGI are very slim. That's not to say he shouldn't give it a try! Society progresses on the back of risky bets that pay off.

[criddell]

> But the odds of him making some big leap in AGI are very slim.

That's probably true. I look at this as Carmack running his own PhD program. I expect he will expand what we know about computation and the AGI problem before he's done.

[TheCoelacanth]

> ML researchers mainly do competitions on the same data sets, trying to put up better numbers.

There are surely a lot of researchers doing that, but do you really think anyone who has a plausible claim at being one of the top 100 researchers in the field in the entire world is doing that? Even if there are only 100 people doing truly novel research, that's still 100 times as many people as are going to be working on Carmack's research.

[fastball]

How many people were working on physics before Einstein came along?

I don't think you understand the desired outcome here. We want eureka moments, and we're hopeful for some. That doesn't mean we expect them to happen. Stop being such a pessimist.