If you own a smart phone, you probably have a weather app installed on it. But do you know where the weather information in the app is coming from? Do you know how its forecasts are being produced? The answers to these questions can often make a big difference in the accuracy of your app's forecast.
While watching radar and forecasting for the 24 Hours of Le Mans last weekend, I came across some forecasts that were posted to Twitter that seemed very inaccurate compared to what I was seeing. Here's the scenario that I was watching Saturday night around 11 pm Le Mans time:
I saw some tweets that talked about rain arriving at the circuit around midnight, just a little over one hour from the time of the above radar image. I'm not sure exactly where those projections were coming from, but I suspect it came from a weather app forecast. A simple look at the radar would have made it clear that the rain was well over an hour away. Some weather apps have an algorithm that takes current radar data and project the movement of rain, but this cannot account for the rain changing speed or direction, dissipating, or new rain areas developing. Perhaps some weather app forecast used model output that projected rain to develop over Le Mans in the next hour, but from my own analysis of the environment, that seemed highly unlikely.
Then I saw a tweet that mentioned a 79% chance of rain at 4 am, and an 82% rain chance at 7 am. I was amazed at the "skill" of this forecast that could predict a 3% increase in rain chances over three hours, followed by a drop of 10% in the next three hours, followed by another rise by 9% in the following three hours! The source for this forecast was World Weather Online.
Another example was a story in April about how the Miami Marlins, who play in a baseball stadium with a retractable roof, had a rain delay during a home game because club executives relied on weather apps for their information rather than trained meteorologists.
Not all weather apps are the same. Some produce forecasts that are manually produced by a meteorologist. These are the forecasts that will most often be the most accurate and of the highest quality. Some produce a "human in the loop" forecast where a meteorologist monitors automated data from a models but modifies is based on meteorological reasoning. Some produce a "human over the loop" forecast that is simply automated data output from a model that is quality controlled by a person (probably not a meteorologist) who may edit them only if there are obvious errors. Many apps take data straight from the weather models and display them on your phone as a forecast with no human intervention at all - these forecasts are strictly automated. The quality and accuracy of the forecast is strongly dependent on how much human intervention there is in creating the forecast - the most accurate forecasts will usually be the ones that are produced by a meteorologist, while the worst accuracy and quality will most often come from the automated forecast apps.
But meteorologists rely on forecast models all the time to create their forecasts, so why are automated forecasts bad (read more about the forecast process here)? Sometimes, they're not. Certain weather patterns are more accurately predicted by models than others. The problem is that automated forecast apps are captive to one particular forecast model and will change every time the model runs and produces new output (up to 4 times a day). If the computer model has a good handle on the forecast, they can be very accurate. If the model is wrong ("garbage in"), the output is wrong ("garbage out"). The farther into the future you look, the more likely you are to see wildly-varying forecasts that are just plain wrong.
Most of the time, the most accurate forecast is one that accounts for the output of multiple weather models, and assimilates data from numerous sources. Trained meteorologists view output from several weather models, and improve on that output by using their personal experience, knowledge of local terrain influences, knowledge of favored regional weather patterns, knowledge of model strengths and weaknesses, analysis of recent model trends and biases, and analysis of the latest observed data.
Anyone can post a forecast from their automated weather app, passing it off as an accurate forecast, and spread it far and wide through social media. Be wary of forecasts that are passed on second-hand through social media. Verify that the source of the forecast is from a trusted source and produced by an experienced meteorologist. I know that forecasts from the National Weather Service are entirely produced by meteorologists with local expertise and experience.
At Racecast Weather, we use our experience combined with analysis of numerous data sources to produce our forecasts. In addition to our forecasts graphics, we put a lot of effort into producing posts on our website to help you better understand the weather pattern that is causing the weather, and we try to let you know when we have lower or higher confidence in our forecasts. We try to tailor our forecasts to the needs of fans, teams, and race officials, which is not possible from a weather app. You can see for yourself how accurate we are with our verification posts. We appreciate your feedback on our forecasts, both good and bad. To learn more about Scott and I, visit our About page.