I dug out an old Motofone (the $40 eink candybar) for this reason, only to discover that it’s 2G, which has been decommissioned. Too bad, would’ve been fun to watch the kid learn T9.
I mean, sure you can find them if you look hard enough, but a phone that plays videos is not a dumbphone in my book. And I wouldn't hand it to my kid. It's probably worse than handing them a smartphone, because I'm guessing it is just dumb enough to have no parental controls.
Is there such a thing? Given it's radio waves in regulated radio spectrum, I'm not sure such a device would even be allowed to operate without a license. This said, there are 3G Nokia dumbphones (e.g. C2-01). I just had to stop using one because my operator tore down the 3G network here, like in most of Europe. (2G was already down since some time).
Edit: there are even 4G-VoLTE dumbphones by the way.