1
$\begingroup$

I want to see how well different CMIP6 model realisations did during the model evaluation, but I can't find papers associated with the data presented on the ESGF node. For example, if I was interested in the EC-Earth3 model and specific variant, where can I find out how it performed?

I haven't found an article specifically referring to EC-Earth3, but rather one referring to the AerChemMIP realisation, as opposed to the ScenarioMIP realisation I am interested in. Is it possible that the data from these models contribute to CMIP6 without any peer-reviewed evaluation of the outputs?

$\endgroup$
2
$\begingroup$

Unfortunately, while there's lots of coordination of the CMIP6 experiment protocols across the modelling groups, there's little coordination of the evaluation of outputs, which is done in different ways by different groups. There's a JAMES Special Collection of Special Issues covering some models (CAS-ESM, CESM2, IPSL, GISS, CNRM, E3SM, GFDL, HadGEM3, MPI-ESM1.2) and several GMD Special Issues for individual models (e.g., CanESM, NorESM, ACCESS). These are used differently by each group; some use them just to document the model, some also use them for model evaluation and science results.

EC-Earth seems to be notable for an absence of a special issue. There's a paper on EC-Earth3 historical simulations and an EC-Earth3 report but nothing obviously coordinated. The information is probably scattered across various papers and journals.

It's worth noting that CMIP6 outputs are pushed through various standard benchmarking tools and put on the web, which you might find useful: e.g., https://www.ilamb.org/CMIP6/historical.

Is it possible that the data from these models contribute to CMIP6 without any peer-reviewed evaluation of the outputs?

Yes and no. Pretty much every group will have some form of internal evaluation and sanity check of their model outputs before publishing them to the ESGF. Some groups require a human to approve every single file that gets published. But with each CMIP6 model producing petabytes of data, this internal post-processing, review and publication work flow requires a significant amount of effort by each group (e.g., CESM at NCAR). It's just not practical for all that data to also be formally peer-reviewed just for the purpose of sitting in a database.

In practice though, each science paper that uses those data is a form of peer-review (because none of us should use any data without sanity checking it for ourselves), and those papers will themselves have traditional, formal peer-review before being published. (I've had to contact other modelling groups about oddities in their data I've found while writing a paper and sometimes they have the resources to fix the dataset, sometimes they don't.) So you could say that these datasets have at least three levels of review, and probably more. One of the benefits of making these data on the ESGF available to anyone is that they are scrutinized by far more people, friendly and unfriendly, than happens under traditional peer-review.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.