What is the status of meteorological data exchange standards?

Question

How do the web services exchange meteorological data?

One method is transferring NetCDF and GRIB data and hooking them up to your software.

I read here about the possibility of using WFS and WCS. Here, I read about the proposal of using calls like dataQuery, similar to getCapabilities of WMS servers. I also note that NASA uses JSON output, e.g. for the DCP-30 dataset. In Europe, the INSIPRE committee publishes very complex guidelines . DWD, UK Met office and others recently held a conference on that.

Is there a global standard and what is it? I know the OGC-DWG is working on that since 2009. What are the latest updates?

Addendum (Full forms of Abbreviations)

• WMS : Web Mapping Service, a Server that responds with a map image of a spacific area, or about its ability to serve.
• WCS : Web Coverage Service, similar to above, but serves gridded data, e.g. temperature, covering a continuous area.
• WFS : Web Feature Service, similar to WCS, but serves features on an around a given point (or points) - thus discreet, not continuous.
• DCP : Downsized Climate Projection, a low resolution dataset projecting the future global climate. 30 indicates 30 years in future and so on.
• OGC : Open Geospatial Consortium , a non profit organization standardizing geospatial data.
• DWG : Domain working Group, a workgroup in OGC specializing in a particular field, one in this case for standardizing meteorological and oceanographic data exchange.

Answer 1

I don't think there can be "the single standard" for a range of technical and organisational reasons.

Probably the biggest thing to note is that different formats serve different purposes. For example, comparing WFS, WMS and WCS. WFS is a feature concept - the sort of data you might get as observations from a set of irregularly distributed sites (e.g. automated weather stations at airports). WMS is a picture (e.g. PNG image or JPEG) - the sort of data you might get as an IR image from a satellite, or some particular styling of a gridded forecast such as wave heights (turned into colour bands). WCS is a coverage (so no particular visual representation) - in the gridded forecast of wave heights example, it would be a floating point number of the height (average, peak, whatever is defined) for that "grid area".

Note that these OGC protocols weren't really designed to handle multiple parameters at a time, nor were they designed to handle data represented in more than two dimensions at a time (e.g. you can ask for an elevation and a time, but you can't ask for multiple elevations and times in a single output without some custom (i.e. not standard) approach.

So you'll see observations and forecasts in specialist formats (like BUFR, GRIB and NetCDF). Those formats can be transmitted (e.g. raw files), but they're also able to be subsetted. THREDDS is a particularly good tool for this purpose. Note that GRIB and NetCDF have some overlap in scope. There are also multiple versions of GRIB, and there are non-standard parts to GRIB (e.g. forecast centre specific tables). Adding new standards doesn't help.

Changing formats isn't easy - there are often massive collections of data and complex tools (like numerical forecasting models) that need to be updated or reworked to handle new formats. Not everyone can do that (e.g. for financial reasons, or organisational expertise reasons, or some programmatic reason like there is already another upgrade going on) at the same time.

So you can expect to need to handle lots of formats for a long time to come.

If you have options and need to exchange data, I'd start with NetCDF, mainly because it has a lot of pretty good tools.

Answer 2

For the data files themselves, NetCDF is extremely popular file format for meteorological data which is typically associated with one or more standards like the CF conventions, or ACDD.

There are also standards in the sense of transport standards for data as a service which is what I think you mean. The OGC standards have been mentioned above, and are examples of this (although you might also consider adding OGC SOS or Sensorthings to the list).

The previous answer mentions THREDDS and NetCDF, but I would argue that OPeNDAP is more comparable to the OGC standards as a data transport standard, whereas THREDDS / ERDDAP / Hyrax are the software that can communicate using that standard (ERDDAP can also serve out WMS for instance)

The OPeNDAP standard, which is tightly connected to the NetCDF file format, is used extensively by a number of large national or international meteorological organizations including WMO/GCW and NOAA,

Depending on how broadly you define "Meteorlogical Data", but you could also consider that IOOS and CIOOS use a mix of OGC SOS and OPeNDAP.