Racecast Weather - Your Source for Race Weather Information
  • Home
  • Racecasts
  • 2025 Schedule
  • Services
  • Contact
  • Bios
  • Links

Can your weather app be trusted?

6/19/2015

Comments

 
By Doug Schneider

If you own a smart phone, you probably have a weather app installed on it. But do you know where the weather information in the app is coming from? Do you know how its forecasts are being produced? The answers to these questions can often make a big difference in the accuracy of your app's forecast. 

While watching radar and forecasting for the 24 Hours of Le Mans last weekend, I came across some forecasts that were posted to Twitter that seemed very inaccurate compared to what I was seeing. Here's the scenario that I was watching Saturday night around 11 pm Le Mans time:

Rain in central France will be dissipating as it moves N. Should not be an issue until near sunrise. @radiolemans pic.twitter.com/uTSvvOVbmu

— Doug Schneider (@Race4caster) June 13, 2015
I expected that the rain would be decreasing in coverage and intensity as it moved north because of my analysis of observed weather data across northern France. The projections of some models that indicated the rain would dissipate added confidence to what I expected. My forecast that I posted just before the race began had a 20% chance of rain Sunday morning, with a 40% chance in the afternoon before the end of the race.

I saw some tweets that talked about rain arriving at the circuit around midnight, just a little over one hour from the time of the above radar image. I'm not sure exactly where those projections were coming from, but I suspect it came from a weather app forecast. A simple look at the radar would have made it clear that the rain was well over an hour away. Some weather apps have an algorithm that takes current radar data and project the movement of rain, but this cannot account for the rain changing speed or direction, dissipating, or new rain areas developing. Perhaps some weather app forecast used model output that projected rain to develop over Le Mans in the next hour, but from my own analysis of the environment, that seemed highly unlikely.

Then I saw a tweet that mentioned a 79% chance of rain at 4 am, and an 82% rain chance at 7 am. I was amazed at the "skill" of this forecast that could predict a 3% increase in rain chances over three hours, followed by a drop of 10% in the next three hours, followed by another rise by 9% in the following three hours! The source for this forecast was World Weather Online. 
Picture
PictureSource: http://blog.memphisweather.net/
If a forecast calls for an 80% chance of rain within the next six hours, that implies a high level of confidence that it's going to happen. And in this case, it didn't. There were some light sprinkles at the circuit in the morning after sunrise, but there was no measurable rain recorded until after 2 pm, during the final hour of the race. The measurable rain arrived just when the rain chance in this forecast was falling from 83% to 55%! I don't mean to pick on WWO (they may very well be accurate most of the time, outside of this particular example), but I think this example demonstrates the problems with many weather apps. 

Another example was a story in April about how the Miami Marlins, who play in a baseball stadium with a retractable roof, had a rain delay during a home game because club executives relied on weather apps for their information rather than trained meteorologists. 

Not all weather apps are the same. Some produce forecasts that are manually produced by a meteorologist. These are the forecasts that will most often be the most accurate and of the highest quality. Some produce a "human in the loop" forecast where a meteorologist monitors automated data from a models but modifies is based on meteorological reasoning. Some produce a "human over the loop" forecast that is simply automated data output from a model that is quality controlled by a person (probably not a meteorologist) who may edit them only if there are obvious errors. Many apps take data straight from the weather models and display them on your phone as a forecast with no human intervention at all - these forecasts are strictly automated. The quality and accuracy of the forecast is strongly dependent on how much human intervention there is in creating the forecast - the most accurate forecasts will usually be the ones that are produced by a meteorologist, while the worst accuracy and quality will most often come from the automated forecast apps.

But meteorologists rely on forecast models all the time to create their forecasts, so why are automated forecasts bad (read more about the forecast process here)? Sometimes, they're not. Certain weather patterns are more accurately predicted by models than others. The problem is that automated forecast apps are captive to one particular forecast model and will change every time the model runs and produces new output (up to 4 times a day). If the computer model has a good handle on the forecast, they can be very accurate.  If the model is wrong ("garbage in"), the output is wrong ("garbage out"). The farther into the future you look, the more likely you are to see wildly-varying forecasts that are just plain wrong. 

Most of the time, the most accurate forecast is one that accounts for the output of multiple weather models, and assimilates data from numerous sources. Trained meteorologists view output from several weather models, and improve on that output by using their personal experience, knowledge of local terrain influences, knowledge of favored regional weather patterns, knowledge of model strengths and weaknesses, analysis of recent model trends and biases, and analysis of the latest observed data. 


PictureA forecast 15 days away from WorldWeatherOnline.com. Is it realistic to expect this level of detail to be accurate 15 days in advance?
Another problem with weather apps is how they present a false expectation of accuracy. Some apps give a forecast over two weeks in advance, with Day 15 of the forecast having nearly the same level of detail as Day 1. While the science of meteorology has advanced tremendously in the past few decades, it is beyond the state of the science to predict the temperature at a certain hour of the day two weeks in advance. The public demands pinpoint, personalized forecasts days or weeks in advance, but sometimes a meteorologist simply has to say "I don't know". Even getting the details of a storm hitting your location in the next hour can be difficult to near impossible due to the variability and complexity of the atmosphere. The detailed forecasts of some weather apps are dangerous because they create absurdly high expectations that ultimately erode overall trust in forecasts because people see “I don’t know” as some sort of proof that all forecasts are guesses or just plain wrong. The graphical presentation of the weather apps leaves no room for forecast uncertainty to be expressed, giving the false impression that the late periods of the forecast have equal confidence to the early periods of the forecast.

Anyone can post a forecast from their automated weather app, passing it off as an accurate forecast, and spread it far and wide through social media. Be wary of forecasts that are passed on second-hand through social media. Verify that the source of the forecast is from a trusted source and produced by an experienced meteorologist. I know that forecasts from the National Weather Service are entirely produced by meteorologists with local expertise and experience.

At Racecast Weather, we use our experience combined with analysis of numerous data sources to produce our forecasts. In addition to our forecasts graphics, we put a lot of effort into producing posts on our website to help you better understand the weather pattern that is causing the weather, and we try to let you know when we have lower or higher confidence in our forecasts. We try to tailor our forecasts to the needs of fans, teams, and race officials, which is not possible from a weather app. You can see for yourself how accurate we are with our verification posts. We appreciate your feedback on our forecasts, both good and bad. To learn more about Scott and I, visit our About page. 

Comments

    Social Feeds


    Instagram


    Authors

    Doug Schneider
    Stephen McCoy

    Harris Cooley 


    Categories

    All
    F1
    IGTC
    IMSA
    Indycar
    Le Mans
    MRTI
    NASA
    Outlooks
    Photos
    PWC
    Racecasts
    SCCA
    SRO
    Trans Am
    TUSC
    Verification
    Weather Education
    WEC


    Archives

    February 2025
    January 2025
    December 2024
    November 2024
    October 2024
    September 2024
    August 2024
    July 2024
    June 2024
    May 2024
    April 2024
    March 2024
    February 2024
    January 2024
    December 2023
    November 2023
    October 2023
    September 2023
    August 2023
    July 2023
    June 2023
    May 2023
    April 2023
    March 2023
    February 2023
    January 2023
    December 2022
    November 2022
    October 2022
    September 2022
    August 2022
    July 2022
    June 2022
    May 2022
    April 2022
    March 2022
    February 2022
    January 2022
    December 2021
    November 2021
    October 2021
    September 2021
    August 2021
    July 2021
    June 2021
    May 2021
    April 2021
    March 2021
    January 2021
    December 2020
    November 2020
    October 2020
    September 2020
    August 2020
    July 2020
    June 2020
    March 2020
    January 2020
    December 2019
    November 2019
    October 2019
    September 2019
    August 2019
    July 2019
    June 2019
    May 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    October 2018
    September 2018
    August 2018
    July 2018
    June 2018
    May 2018
    April 2018
    March 2018
    February 2018
    January 2018
    December 2017
    November 2017
    October 2017
    September 2017
    August 2017
    July 2017
    June 2017
    May 2017
    April 2017
    March 2017
    February 2017
    January 2017
    December 2016
    November 2016
    October 2016
    September 2016
    August 2016
    July 2016
    June 2016
    May 2016
    April 2016
    March 2016
    February 2016
    January 2016
    December 2015
    November 2015
    October 2015
    September 2015
    August 2015
    July 2015
    June 2015
    May 2015
    April 2015
    March 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014



Powered by Create your own unique website with customizable templates.