Forecasting Statistics in Meteorology

(credit: World Weather Research Programme, EUMeTrain)

DISCUSSION: No matter how badly a forecaster wants to be 100% accurate, the unpredictability of the atmosphere will guarantee some missed forecasts in their lifetime. But there are a few tools out there that forecasters can use to quantify and take a closer look at how they forecast. This information is invaluable, because it can be used to ultimately improve one’s forecasting abilities.

There are many different types of forecasts, and different statistical tools for those forecasts, but for simplicity, the focus here will be on dichotomous forecasts. A dichotomous forecast is a forecast that answers a yes or no question in regard to an event happening. For example, answering questions such as “Will a tornado will occur?” “Will fog will be present?” or “Will it rain before noon tomorrow?” are all examples of dichotomous forecasts. To analyze these types of forecasts, a forecaster can construct a “contingency table.” A contingency table combines the four possibilities that could occur during a dichotomous forecast:

• If the event is both forecasted and observed, this is known as a “hit”
• If the event is not forecasted, but observed, this is known as a “miss”
• If the event is forecasted, but not observed, then this is a “false alarm”
• If the event is not forecasted and not observed, this is a “correct negative”

This type of table allows for a forecaster to quickly assess their personal forecasting habits. Below is the basic setup of a contingency table, followed by a filled-out example of a contingency table for gale forecasts (provided by EUMeTrain).

From these tables, different statistics can be calculated to give a forecaster some insight into their own forecasting abilities. For example, if a forecaster wanted to see how often they issue false alarms, they would divide their false alarms by the total times they said “yes, this event will occur.” Using the contingency table, this is just a/(a + b), and this is known as the “False Alarm Ratio”.

These scores can be misleading though, so it is important to clarify which score is being calculated. For example, calculating a forecaster’s accuracy would be the total number of times they got a forecast correct divided by the total number of times they have forecasted. Looking at the contingency table, this is (a + d)/(a+ b + c + d). However, a forecaster usually has many correct negatives, as can be seen in the example contingency table above. Instead of this score, many forecasters use what is called the “threat score” or “critical success index” (TS or CSI). This score is a measure of success, but it does not include correct negatives. Therefore, it is calculated by a/(a + b + c). Since correct negatives are removed from consideration in this score, it is much more sensitive to hits, misses, and false alarms.

As mentioned before, there are many more tools that a forecaster can look at, and a much broader look at forecast verification can be found here!

To learn more about other educational topics regarding the atmospheric and oceanic sciences, click here!

References:
http://www.nws.noaa.gov/oh/rfcdev/docs/Glossary_Verification_Metrics.pdf
http://www.cawcr.gov.au/projects/verification/
http://www.eumetrain.org/data/4/451/english/msg/ver_categ_forec/uos1/uos1_ko1.htm

© 2018 Meteorologist Joseph Fogarty