[dragontamer]

Somehow, I'm reminded of the Tsar tank from WW1. The Russians knew that a new weapon of war: an armored car, was necessary to break the stalemate of trench warfare.

This hypothetical armored car needed many features: the most important was that it must be able to move across the muddy no man's land reliably.

Tests have shown that regular sized wheels would get stuck in the mud. A bigger wheel has more surface area and greater contact area. So the Russians built an armored car with the largest wheels possible. Russian tests were outstanding, the Tsar tank rolled over a tree !!!!

https://en.m.wikipedia.org/wiki/Tsar_Tank

The French design was to use caterpillar tracks. We know what works now since we have a century of hindsight.

--------

Spending the most money to make the biggest wheel isn't necessarily the path to victory. I think it's more likely that the tech (aka, caterpillar track equivalent) hasn't been invented yet for robotaxis. Hitting the problem with bigger and more expensive neural network computers doesn't seem to be the right way to solve the problem.

[zaptrem]

I agree with your points on the robotaxi front, but there are many other problems that will totally benefit from a bigger training computer.

[dragontamer]

But Tesla isn't a cloud-provider company, nor is it a hardware company. None of the technical specs, assembly language, API, SDKs or whatnot have been released for Dojo.

The model of "someone will find this training computer useful" is... fine. Google TPUs, NVidia DGX, Intel Xe-HPC, AMD MI100, Cerebras wafer scale AI. These are computers that nominally are aiming for the market of selling computers / APIs / SDKs that will make training easier.

Its a pretty crowded field. Someone probably has struck gold (NVidia has a lead but... its still anyone's game IMO)

-------

If Tesla's goal is to compete against everyone else (or make a chip that's cost-competitive with everyone else), Tesla needs more volume than (allegedly) 3000 chips (quoted from the article: I dunno where they got this figure but... there's no way in hell 3k chips is cost-effective).

That's the name of the game: volume. The reason why NVidia leads is because NVidia sells the most GPUs right now, which means their R&D costs are applied to the broadest base, which means those company's engineering costs (aka: CUDA training) is spread across the widest number of programmers, leading to a self-reinforcing cycle of better hardware, lower costs, with a larger community of programmers to learn from.

[jeffbee]

You forgot the Hauwei Ascend 910. 4096 of them in a rack is an easy exaflop.

[deepnotderp]

https://www.adapteva.com/andreas-blog/adapteva-announces-100...

[baybal2]

> many other problems that will totally benefit from a bigger training computer.

I don't really think it's that many.

The industry collectively sank untold billions into the blind belief that neural algorithms will somehow turn into "AI."

10 years later, no "AI," and not even a single money making niche use.

Right now the industry is deep in sank cost falacy, and people who promised this, and that to investors are now desperate, and doubling their bets in hopes that "at least something will come out of it...," a casino mode basically.

[gpm]

> not even a single money making niche use.

There are tons of money making niche uses of neural networks. From the branch predictor on your CPU, to trading on the stock market, to image-search engines.

[spijdar]

Yeah, you can make a point that it's a dead end if you want "real" "general AI" or whatever, but Google/Facebook/etc are definitely using it to their advantage in analytics, if nothing else.

[dragontamer]

Google will use TPUs. Their own technology.

Do you think Facebook would rather use Tesla Dojo instead of NVidia DGX A100 computers? Do you think any company, would rather choose this Dojo to build their internal software-stack on top of instead of CUDA / OpenCL / whatever?

I mean, its possible. But Tesla needs to start pumping out Github pages, documents, books, etc. etc. to document how exactly to use Dojo.

[spijdar]

I was responding to the idea that neural network-based software/hardware as a whole is an unprofitable industry trend, no comment on Tesla's specific hardware here.

[atty]

As someone who works in the field, if your only conception of AI is a truly sentient intelligence, then yes, we are, as far as I can tell, nowhere near that. However, if by AI, you mean the use of sub domains of AI like machine learning or deep learning (where all the money has been spent), it’s quite literally all around us now, whether you realize it or not. At my company our deployment of “AI” techniques is ramping up year over year, not slowing down, and we’re seeing very good results. (And I work at a very old company, not a tech native. I can only imagine how much of Google’s or Facebook’s workflows and products include some sort of machine learning)

[stcredzero]

The industry collectively sank untold billions into the blind belief that neural algorithms will somehow turn into "AI."

The huge qualitative differences between GPT-2 and GPT-3 seem to suggest that they will, if you just keep adding orders of magnitude more connections and more data.

https://www.youtube.com/watch?v=_8yVOC4ciXc

[deepnotderp]

What? Pretty much every major Internet company extensively uses NNs. Just recently there was a great published use case by Google using GNNs for travel time predictions