|
The Global Forecast System (GFS), i.e. the model presently used at NCEP, has a grid resolution of 18 miles (28 km). It is (has been, for years, actually), the second best global forecast system, right behind the European ECMWF (sometimes outperforming it, but on average slightly underperforming it, in terms of accuracy). I don't know how the ECMWF model works, but even as someone who did not study meteorology (but studied electrical engineering, which forms the theoretical basis of weather forecasting via the Kalman filter), I can say the following (having spent a number of years working at NCEP):
1. Initial conditions/parameters are fundamental in setting up a model run.
2. Forecasts have for a long time relied on ensembles, which are repeat model runs with slightly varying parameters. The idea of ensembles is, if you run enough of them, you will frequently notice one or more convergence(s) that various sets of parameters produce, e.g. where some sets of parameters predict one movement pattern for a hurricane, while others produce a different movement pattern. Historically, such discrepancies were resolved by actual forecasters, who decided based on their knowledge and experience which one was more likely. In addition, they also had meetings every morning between scientists (developing the model) and forecasters (who relied more on general knowledge and experience) and involved occasionally heated discussions between the groups. But I digress.
3. Considering it involves a chaotic system, I cannot say how much value something like deep learning might bring to the table that produces consistent value above and beyond what's already obtained by using ensembles of Kalman predictive filtering. It is however noteworthy to point out that if the grid resolution is 28,000 meters, then it may not make much sense to set the resolution of the model itself substantially lower (like 300 meters), because any resulting data is more likely to be an artifact of the model itself, rather than reflective of real life information. Luckily, this issue has been and is being addressed through the development of rigorous testing standards, which inform of the inherent quality of forecasts produced by a particular model (this is how they can assign an objective rank to e.g. the GFS and the ECMWF, when forecast quality is generally very close and the model producing the most accurate prediction varies between the two). To put it plainly, the degree to which the website mentioned above has any value is based not on its best predictions, but on the overall variance (i.e. how close predicted data comes to actual measurements of the same, which is necessarily retrospective).
4. That said, it's worthwhile to point out that just because it doesn't involve a government agency with something like a thousand employees, hundreds of scientists (in the case of NCEP alone), and very powerful supercomputers, does not necessarily mean it's bunk (even if it frequently does). For example, I do recall Panasonic (IIRC) showing up out of the blue, with its own forecasting system, which was shown to be competitive after requisite, rigorous testing. I don't remember many details and this was years ago—and its disappearance alone is suspect, but it's worth adding for completeness. |
Kalman filtering is only one part of the process, and plays a critical role during the data assimilation part. Classical Kalman filtering is optimal for Gaussian-distributed linear dynamical systems, but needs tweaks for non Gaussian distributions and non linear systems.
Classical NWP models for instance will integrate the primitive partial differential equations in time and space and run various parameterizations (which can be in some cases even more expensive than integrating the primitive equations). ECMWF on their end use IFS, which is a spectral method for solving the PDEs.
The whole process of solving these models accurately has definitely been some of the most fascinating science and engineering I’ve had the pleasure to work with. It’s extremely humbling :)