Karpathy's career arc feels similar to Jim Keller's; a butterfly flitting from one flower to another, gathering experiences and creating magic everywhere they go.
I can spare a minute :). This isn't exhaustive because this is just stuff I know of, obviously.
- At Stanford, Led research on the first (to my knowledge) crop of joint image/text models. Super widely cited work.
- At Tesla, led their whole self driving effort for a while, came up with critical techniques that allowed them to make progress (e.g., the concept of "auto labelling": using a much larger NN to generate training data with which to train smaller models that could fit in the on-device compute. IIRC, Elon said they would not have been able to make progress without this insight).
I'm not sure his educative efforts for the mold of what you're looking for, but if so, the course he designed at Stanford (and availed online):for neural networks, as well as his blog posts, (most famous of which, to my knowledge, is "the unreasonable effectiveness of LSTMs"), made a huge impact on educating a generation of tinkerers and researchers.
The auto labeling work (which has been partially described/presented at Tesla AI day events) seems more like engineering than research, a grab bag of techniques that I would guess the whole team must have contributed to. For example, they auto label low resolution/indeterminate objects (image segments) by temporal continuity... Something that is a low-res blob in the distance becomes a hi-res and easy to identify object when you drive by it, so by tracking objects backwards across frames you can learn how to more confidently label the lo-res blob. Things like this are useful, but it's the sort of stuff that engineers and developers are coming up with every day.
You don't think that tracking objects from frame to frame is obvious ?!
I can guarantee you this was built-in from day #1
I'm guessing you're not a developer if you don't then automatically think of end cases like "what if car # 1 isn't in the preceding frame" ... (then you look at some relevant test data and see it was there, unlabelled ...)
Obvious in hindsight and obvious at the time are very different things.
You seem to have missed the main point anyway - using a larger model to generate labels for a smaller one is what the parent was highlighting, not the temporal labeling alone. The gold standard at the time was human labeling (eg Waymo). Deep learning was just having its moment, all of this stuff was cutting edge, and there is a lot of work in between a published paper and actually applying that to production vehicles.
Tesla still hasn't achieved their 2016 self-drive goal by their self imposed deadline of 2017, even now a decade later. So, politely, is that accolade merited?
The current vehicles sure seem to come close. I'm not entirely clear on how they've missed this goal, but the current models can do full self driving where I live, including parking.
Sure they have improved but how do we define success? Is success "It can drive a road it has never been on?" Even then I'm not sure because the model (not the physical car) has probably scanned that road before so it is recalling a prior route while being aware of hazards. Is that learning, or rote memorization?
My Tesla drives to walmart, finds a parking spot, comes to me outside walmart and drives me home. I've been driving my model 3 for years, and honestly, i've never had to "Take over" due to a saftey issue.
I could never trust a Tesla to drive safely around people. They seem like death traps. Could you share a link to the coast to coast drive please? How aided was it?
How does Elon's arbitrary deadlines impact whether the accolade is "merited"? Incredible progress was made in a fairly short amount of time. His accolade isn't based on his employer's ability to predict delivery dates, they're based on the quality of the systems that are actively deployed today.
I think an accolade's merit is based on the definition of done for work delivered. Elon certainly told the public a certain vision of self-driving (a definition of done) and it didn't come to fruition despite PR progress; i.e. a washing machine can do a lot of work, but is it the right work?
We can arbitrate about what "self-driving success" means until the cows come home, but my point is I've seen a lot of self-driving failures from the Teslas I've witnessed in person.
I was more looking for signal that him + Anthropic might yield something beyond a step-change from Opus 4.7 (disappointing so far). We have not gotten to use Mythos yet, I wonder if that will become Opus 5 or something.
EDIT: It looks like you deleted the part of your post I quoted below. So feel free to ignore my question about it, I guess.
---------------------
Not sure what you mean by
> Shows how much you know
Do you mean that the fact that I misremembered a word on the title suggests that I know very little about Karpathy's contributions to the field of neural networks?