But Tesla isn't a cloud-provider company, nor is it a hardware company. None of the technical specs, assembly language, API, SDKs or whatnot have been released for Dojo.
The model of "someone will find this training computer useful" is... fine. Google TPUs, NVidia DGX, Intel Xe-HPC, AMD MI100, Cerebras wafer scale AI. These are computers that nominally are aiming for the market of selling computers / APIs / SDKs that will make training easier.
Its a pretty crowded field. Someone probably has struck gold (NVidia has a lead but... its still anyone's game IMO)
-------
If Tesla's goal is to compete against everyone else (or make a chip that's cost-competitive with everyone else), Tesla needs more volume than (allegedly) 3000 chips (quoted from the article: I dunno where they got this figure but... there's no way in hell 3k chips is cost-effective).
That's the name of the game: volume. The reason why NVidia leads is because NVidia sells the most GPUs right now, which means their R&D costs are applied to the broadest base, which means those company's engineering costs (aka: CUDA training) is spread across the widest number of programmers, leading to a self-reinforcing cycle of better hardware, lower costs, with a larger community of programmers to learn from.
> many other problems that will totally benefit from a bigger training computer.
I don't really think it's that many.
The industry collectively sank untold billions into the blind belief that neural algorithms will somehow turn into "AI."
10 years later, no "AI," and not even a single money making niche use.
Right now the industry is deep in sank cost falacy, and people who promised this, and that to investors are now desperate, and doubling their bets in hopes that "at least something will come out of it...," a casino mode basically.
There are tons of money making niche uses of neural networks. From the branch predictor on your CPU, to trading on the stock market, to image-search engines.
Yeah, you can make a point that it's a dead end if you want "real" "general AI" or whatever, but Google/Facebook/etc are definitely using it to their advantage in analytics, if nothing else.
Do you think Facebook would rather use Tesla Dojo instead of NVidia DGX A100 computers? Do you think any company, would rather choose this Dojo to build their internal software-stack on top of instead of CUDA / OpenCL / whatever?
I mean, its possible. But Tesla needs to start pumping out Github pages, documents, books, etc. etc. to document how exactly to use Dojo.
I was responding to the idea that neural network-based software/hardware as a whole is an unprofitable industry trend, no comment on Tesla's specific hardware here.
In which case... I'm pretty sure its unproftiable for the vast majority of companies pursuing it.
Just as 80%+ of computer companies went bankrupt in 80s and 90s... I expect 80% of these chips to go bankrupt moving forward.
IBM PC, Amiga, Tandy, Acorn, Commodore, Japan's PC-98... they all were on the hype-train for the personal computer revolution. But very few of those machines ended up commercially successful. (IBM PC arguably did, but it was the knockoff clone Dell that won commercially)
Just because we know its a good idea doesn't mean that the companies putting bets on it will survive. In many ways, entering a hype market that's proven successful (but with no clear market winner yet) is almost more risky due to the huge amount of competition.
As someone who works in the field, if your only conception of AI is a truly sentient intelligence, then yes, we are, as far as I can tell, nowhere near that. However, if by AI, you mean the use of sub domains of AI like machine learning or deep learning (where all the money has been spent), it’s quite literally all around us now, whether you realize it or not. At my company our deployment of “AI” techniques is ramping up year over year, not slowing down, and we’re seeing very good results. (And I work at a very old company, not a tech native. I can only imagine how much of Google’s or Facebook’s workflows and products include some sort of machine learning)
The industry collectively sank untold billions into the blind belief that neural algorithms will somehow turn into "AI."
The huge qualitative differences between GPT-2 and GPT-3 seem to suggest that they will, if you just keep adding orders of magnitude more connections and more data.
What? Pretty much every major Internet company extensively uses NNs. Just recently there was a great published use case by Google using GNNs for travel time predictions
The model of "someone will find this training computer useful" is... fine. Google TPUs, NVidia DGX, Intel Xe-HPC, AMD MI100, Cerebras wafer scale AI. These are computers that nominally are aiming for the market of selling computers / APIs / SDKs that will make training easier.
Its a pretty crowded field. Someone probably has struck gold (NVidia has a lead but... its still anyone's game IMO)
-------
If Tesla's goal is to compete against everyone else (or make a chip that's cost-competitive with everyone else), Tesla needs more volume than (allegedly) 3000 chips (quoted from the article: I dunno where they got this figure but... there's no way in hell 3k chips is cost-effective).
That's the name of the game: volume. The reason why NVidia leads is because NVidia sells the most GPUs right now, which means their R&D costs are applied to the broadest base, which means those company's engineering costs (aka: CUDA training) is spread across the widest number of programmers, leading to a self-reinforcing cycle of better hardware, lower costs, with a larger community of programmers to learn from.