# Do wind predictions have confidence scores?

I’m a sailing captain in training and my community members asked about confidence in wind predictions. They were unhappy with the large variation between different wind predictions. The speed and directions from different sources were at times in agreement, but at other times they were completely opposite and with a large delta.

I would like to explain that some of the confidence levels of individual wind speed predictions may be very low. Therefore it should be taken in consideration when comparing those wind predictions to other low confidence wind predictions and that both may be entirely wrong. I could not locate any documentation that showed were confidence levels or each individual wind prediction was being calculated or conveyed. Did I miss something?

Wouldn’t it make sense to say something like: we think the wind at this latitude/longitude will be 8.5 knots in the N direction and we are 95% confident of this prediction? That confidence part seems to be missing. Or is it calculated and just not displayed to the end user? It could save a sailor’s life!

• Unfortunately this is something really lacking in meteorology as a whole (for temperature, severe weather, or really even precipitation) I've seen graduate student work into quantifying error like this, but it's really not been a priority. Many argue (perhaps rightly so) that giving such values may make a forecast more complicated and difficult to approach for many in the public. Which may well be so, but leaving it out certainly leaves us short of fully precise forecast details indeed :-( There may be maps available detailing model forecast deviation scores, but not sure where off hand. Jun 4 at 12:32
• The best quick option I can think of for gauging the certainty yourself is to use Pivotal Weather's comparison loop and stepping through by mouseovers or the m\n keys. Or look at the comparison graph for some locations on Iowa State's page. Jun 4 at 16:22
• Though neither really have the depth of ensembles, nor the input of humans on which models are better for specific situations or such. We could use much more on this. Maybe not on the basic public forecast of the NWS, but somewhere at least. The only group that really ever does confidence ranges is Washington Post's snowfall forecasts. But it's so worth it to better convey that to people. If I ever put together an app, consider this something I've got to fix. Jun 4 at 16:25
• @brianray winds do have probabilistic scores - wmolc.org/seasonPmmeUI/plot_PMME# Go there and check out 850 hPa zonal w ind. Jun 8 at 16:46

We've grown accustomed to seeing percentages for probability of precipitation. Why don't they do this for wind?

No single model simulation has confidence/uncertainty associated with it. Confidence is determined after many model simulations are performed with different model configurations, known as an "ensemble". The percentage reported is a probability of rain occurring, which is a binary event (yes vs no) averaged across all ensemble members. The hourly percentage probability is not factoring in the amount of rain, it just has to be higher than 0.01" in the model. Precipitation is cumulative and not really directional in nature, so calculating probability of a yes/no event is actually meaningful. Wind, in contrast, is a vector that really isn't thought of as a binary event. If the forecast said there was an 80% chance of wind over 1 m/s, that wouldn't help anyone.

8.5 knots in the N direction and we are 95% confident of this prediction?

would really look more like:

8.5 knots with standard deviation of 2 knots from NNW with standard deviation of 30 degrees

Wind direction vs wind speed:

Predicting precise wind direction at the surface is extremely difficult, while upper atmosphere winds are less complex. Also consider that a highly confident wind direction prediction could be very inaccurate. The "confidence" of wind direction can be calculated but the ensemble average of hourly wind direction at the surface is often of low accuracy. Thus model "confidence" of wind direction probably isn't of much value to the public. Sometimes surface wind direction predictions can be relatively confident and accurate due to terrain that channels flow in the same direction consistently. However, at sea you have no such terrain to channel the flows into predictable patterns.

Hourly wind direction can regularly be inaccurate for a few reasons:

1. low wind speed conditions causes meandering influenced by localized conditions
2. incorrect terrain parameterization can cause systematic bias in particular locations.
3. the timing of shifting winds can be off due to small perturbations in the pressure systems not captured by the model

In contrast, the occurrence of high wind speeds at the surface can have a lot of confidence and accuracy, which is why weather agencies can issue "high wind alerts" (another binary yes/no event). However, the timing of pressure system movement is not confident, and modeled hourly surface wind direction can vary greatly throughout the event.

Ensembles provide a wealth of information but they are not easily digestible for the public. Imagine looking at a spaghetti plot of wind direction at the surface; it wouldn't look pretty. In general, that type of ensemble forecast analysis is done by the scientists who run the model, so they can asses 3D model performance. However, model performance is driven by the atmosphere aloft, so that is where ensemble analysis is useful (e.g. 850 mb, 500 mb, etc.). See the spaghetti plots on the NCEP ensemble weather model products as an example.

Visualization Tool at www.windy.com:
You can do a quick comparison of winds for several different models by selecting the comparison forecast for a single location on the windy.com site (see example below). This can give an idea of how "confident" the wind predictions at the surface are, in a qualitative sense.

• Seems rough to say that ensembles wouldn't have any confidence, depending on what you mean by the term. You can certainly calculate std & CIs. Certainly there are still error\biases\resolution issues that mean any model-computed confidence value is inherently imperfect... but there's certain to be quantifiable skill, so giving confidence intervals from ensemble averages is useful. And pressure movement can have errors, but theyre usually quite large and well spaced such that wind direction and speed have varying degree of confidence over time & space. But +1 for the windy tool, that's great. Jun 4 at 17:08
• Thanks @JeopardyTempest, I made some edits, let me know if you think it should be changed more. Hourly surface wind direction at specific locations is really not very confident in my experience, especially in low wind situations. And even in windy situations you can have full 180 degree differences quite often, just because of the downstream effect of changed timing/position of pressure systems and interaction of terrain. Do you think showing surface wind direction skill or ensemble spaghetti plots would be worthwhile here?
– f.thorpe
Jun 4 at 17:25
• Windy is gaining popularity within the sailing community too. That exact feature to compare models is exactly the issue. It would be good to not just compare several models but also compare each confidence. There must be a difference between many models that disagree where none are so sure versus one prediction that claims to be pretty sure. Inductively, it would also imply that the larger weather system including that prediction is also correct. Jun 5 at 12:14
• Some comments on inaccuracies in wind directions. Inaccuracy at low wind speed is valid but often not critically important to users. I would be surprised if near-shore effects are captured well, even more so for wind speed of off shore winds. If I recall these effects are generally not included in marine forecasts - you need to have the seamanship to understand where this will be an issue. Timing of shifting winds is a very good point and critically important for firefighters. Often they can track the shift across the landscape. Not too helpful if you don't have real time updates. Jun 8 at 2:17
• OK, but don't sailors need surface winds?
– f.thorpe
Jun 9 at 4:32

Following up on Farrenthrope's answer (which I think is worthy of acceptance more than this answer) I do want to add additional detail that may not be appropriate as a comment.

There is a problem with how we communicate the wind speed and direction that does not fit with ensembles. Often ensembles rely on prior assumptions about the uncertainty distribution of variables like temperature and water vapor (usually the Gaussian distribution for the Ensemble Kalman Filter). These distributions aren't guaranteed to be accurate, let alone translate across transformations. Things like wind speed and wind direction are applicable and sensible, but require some processing. Wind speed and direction aren't exactly the easiest things to express, but here is an example.

Let's consider a wind vector that we are 100% confident to have a northerly component of 10 kts. But let's say that the east-west component has some uncertainty (let's say ± 5 kts. So the wind speed range would be between 10 and 11.2 kts($$\approx\sqrt{(10 \text{ kts})^2+(\pm 5 \text{ kts})^2}$$). Let's take a look at what the wind vector looks like on a compass (blue arrow is the middle with a wind speed of 10 kts and orange has 11.2 knots):

What is the range of the wind direction? It is between 297° and 63°. But is it the purple range or the yellow range? And this is for uncertainty in only one wind component, ignoring the uncertainty in the other component. Things become even more complicated with adding additional uncertainty. But let's move on to another problem with expressing uncertainty with wind forecasts: wind speed.

Farrenthorpe's answer is good, but there is a flaw with reporting the standard deviation. Without reporting additional statistical moments (like skewness) or making assumptions about the uncertainty distribution, the user may make an inference based on faulty assumptions. For example if the prediction is 8.5 knots with a standard deviation of 10 knots, then there could be a perception that the wind speed can be negative, which is unrealistic and unphysical.

• Even when the uncertainty range spans across the 360->0 degree discontinuity, I think it's fairly obvious that the uncertainty angle you report should be the one that includes the possible wind directions. i.e. 3 degress +- 60 as the direction of wind origin. Yes it means you need to get your modular math right, but that's fully doable. Especially if you report wind prediction in a graphical way, not just text angle + range. And if you do need text, you're right that `between 297 and 63" is ambiguous, so 3 deg +- 60 for centre + uncertainty range is standard in e.g. physics measurements. Jun 5 at 1:40
• @PeterCordes while that is the case for the example given, changing the numbers around, say the 10 knots to 1 knot or 0 knots drastically changes the wind direction uncertainty to being almost ±180°, which is almost comically ambiguous. Jun 5 at 18:21
• Sure, so it's useful to distinguish cases where you can confidently predict wind from a general 120 degree arc, or one relatively confident strong component, vs. cases where you can't. Telling people that it's really unpredictable is better than picking a direction that's probably not even close. (I guess you could do that by leaving wind direction out of the forecast entirely when there's nothing predictable about its direction.) I have no experience with making meteorological predictions or sailing so IDK how sensible this idea actually is >.< Jun 5 at 21:45
• I think this is what the forecasts do. In Australia land forecasts at least, light winds are not given a predicted direction or numerical estimate. Directions at higher winds are binned into 8 compass directions. So in some ways what is important is the probability that that bin will encompass the direction. For speed I think a log transform might be useful: +/- 5 knots is more important at 15 knots than at 45 knots. Jun 8 at 2:39
• @PeterCordes Operationally speaking I use ensemble averages of wind speeds and then calculate ensemble anomalies of wind speeds relative to a climatology. That gives an estimate of what the atmosphere is doing. Percentage probabilities have no relevance Jun 8 at 8:12