Seeing this on HN, I realize I didn't go into a lot of technical detail in the blog post (we still aren't quite sure who our audience is with the Forecast blog). So if anyone has any specific questions, ask away.
Did you try any other ML algorithms on your data before using neural nets? I know neural nets are getting trendy again but a simple SVM or a linear classifier can provide very good performance.
We did, but they didn't work quite as well. But that could absolutely be my lack of experience with how best to normalize and format the input vectors.
In the end, NNs seemed less "fussy", if that makes any sense.
Could you comment on why all the radar noise in the sample image seems to be almost perfectly confined to the eastern half of the US? Is that just an artifact of the time of day (perhaps the sun was setting the in middle of the US at the time)?
I'd love to know more about the neural network too, but I have no experience with them so I'm not even sure what to ask.
Exactly. Those enormous Bagel Blobs tend to occur after the sun goes down (and only in the warmer months. Two months ago, they weren't there).
The image was generated around 10:30 PM EST, so they hadn't had time to propagate further west.
Regarding neural nets, I have to admit that I didn't have much experience with them either before working on Dark Sky. They do, however, seem to be a little more forgiving than other kinds of classifiers (Support Vector Machines, and the like).
The color scale is reduced, but that seems to be ground clutter caused by a temperature inversion. It forms after dark and usually breaks up after a few hours of daylight.
I am a meteorologist and have used radar for nearly thirty years. I have yet to see a foolproof system -- and I've looked for them.