Skip to content

Commit

Permalink
first pass at addressing #464
Browse files Browse the repository at this point in the history
  • Loading branch information
@durack1
durack1 committed Oct 12, 2022
1 parent 0180f42 commit 498fdd2
Show file tree
Hide file tree
Showing 3 changed files with 40 additions and 42 deletions.
37 changes: 18 additions & 19 deletions CMIP6/Guide/dataUsers.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,25 +20,24 @@ Document overview:
## 1. Experiment design
The CMIP6 protocol and experiments are described in a [special issue][GMDSpecialIssue] of Geoscientific Model Development with an overview of the design and scientific strategy provided in the lead article of that issue by [Eyring et al. (2016)][EyringEtAl16].

Each model participating in CMIP6 will contribute results from the four DECK experiments (piControl, AMIP, abrupt4xCO2, and 1pctCO2) and the CMIP6 historical simulation. These experiments are the only ones directly overseen by the the [CMIP Panel][CMIPPanel], and together these constitute the ongoing (slowly evolving) “CMIP Activity”. They are described in [Eyring et al. (2016)][EyringEtAl16].
Each model participating in CMIP6 will contribute results from the four DECK experiments (piControl, AMIP, abrupt4xCO2, and 1pctCO2) and the CMIP6 historical simulation. These experiments are the only ones directly overseen by the the [CMIP Panel][CMIPPanel], and together these constitute the ongoing (slowly evolving) “CMIP Activity”. They are described in [Eyring et al. (2016)][EyringEtAl16].

In addition to the DECK and historical simulations, each modeling group may choose to contribute to any of the [CMIP6 endorsed MIPs][CMIP6EndorsedMips]. See the [GMD Special CMIP6 Issue][GMDSpecialIssue] for descriptions of each MIP and its experiment specifications. The official names of the currently endorsed CMIP6 MIP activities are recorded in a [“json” file][activityIdJson].

When called for by the experiment protocol, [standard forcing data sets][input4mipsHome] have been used. Any deviation from the standard forcing are supposed to be clearly [documented](#5-model-and-experiment-documentation).
When called for by the experiment protocol, [standard forcing data sets][input4mipsHome] (e.g. [Durack et al. (2018)]) have been used. Any deviation from the standard forcing are supposed to be clearly [documented](#5-model-and-experiment-documentation).

Further documentation about CMIP6 experiments will be available from [ES-DOC][es-docsCmip6], and the reference controlled vocabularies used to define and identify these experiments are available in a [“json” file][experimentIdJson] and are also rendered in [table form][experimentIdhtml].

## 2. Model output specifications
The [CMIP6 Data Request][cmip6DataRequestCog] defines the variables requested from each experiment and specifies the time intervals for which they are supposed to be reported. One option for perusing the lists of variables that should be available from at least some experiments is to display the [excel spreadsheet][variableListXls].
The [CMIP6 Data Request][cmip6DataRequest] defines the variables requested from each experiment and specifies the time intervals for which they are supposed to be reported. One option for perusing the lists of variables that should be available from at least some experiments is to display the [excel spreadsheet][variableListXls].

CMIP6 model output includes metadata and is structured similar to CMIP5 output, but changes have been made to accommodate the more complex structure of CMIP6 and its data request. Some changes have been made to make it easier for users to find the data they need and to enable new services to be established providing, for example, model and experiment documentation and citation information.

As in CMIP5, all CMIP6 output has been written to netCDF files with one variable stored per file. The data have been “cmorized” (i.e., written in conformance with the [CF-conventions][cfConventionsPage] and all the CMIP standards). The CMIP6 data requirements are defined and discussed in the following documents:

* [Definition of CMIP6 netCDF global attributes][cmip6GlobalAttGoogleDoc]
* [Reference “controlled vocabularies” (CV’s) for CMIP6][cmip6Cvs]
* [Specifications][cmip6GlobalAttGoogleDoc] for file names, directory structures,
and CMIP6 Data Reference Syntax (DRS)
* [Specifications][cmip6GlobalAttGoogleDoc] for file names, directory structures, and CMIP6 Data Reference Syntax (DRS)
* Specifications for output file content, structure, and metadata are available in [draft google doc](https://goo.gl/neswPr).
* [Guidance on grid requirements][cmip6GridGoogleDoc]
* [Information on pressure levels][cmip6PressureLevelsPdf] requested
Expand All @@ -55,24 +54,24 @@ See this **[summary table][dataHoldings]** to view available experiments and mod

Here are the currently active CMIP6 CoG sites (all data can be accessed via any one of these):
* USA, PCMDI/LLNL (California) - [https://esgf-node.llnl.gov/projects/cmip6/](https://esgf-node.llnl.gov/projects/cmip6/)
* France, IPSL - [https://esgf-node.ipsl.upmc.fr/projects/cmip6-ipsl/](https://esgf-node.ipsl.upmc.fr/projects/cmip6-ipsl/)
* Germany, DKRZ - [https://esgf-data.dkrz.de/projects/cmip6-dkrz/](https://esgf-data.dkrz.de/projects/cmip6-dkrz/)
* UK, CEDA - [https://esgf-index1.ceda.ac.uk/projects/cmip6-ceda/](https://esgf-index1.ceda.ac.uk/projects/cmip6-ceda/)
* France, IPSL - [https://esgf-node.ipsl.upmc.fr/projects/cmip6-ipsl/](https://esgf-node.ipsl.upmc.fr/projects/cmip6-ipsl/)

To get to the search interface click on “More search options” under the large red text near the center of the page.
There are additional options for searching through the web interface (see “More Search Options” near the top right of the page) and there is also an API that can be used to perform searches. [Tutorials][ESGFTutorial] are available by following the link labeled [“Technical Support”] [ESGFTechSupport] near the top right of the page. Expert users may also want to use the [ESGF Search RESTful API][ESGFSearchapi].

[Globus][globusHome] is available for downloading some datasets and will provide much better performance for large data volumes. With the Globus Download option, ESGF will prepare a python script for batch downloads, or you can monitor transfers for a “Web Download”. You can download an entire “data cart” in one step if all datasets in the cart are served by Globus. The Globus option requires you to establish a user account on ESGF (see “create account” at top right of CoG pages). Note also that a second logon with a Globus-enabled credential is required (nb.: Google ids in addition to many institutions are accepted).

## 4. Terms of use and citation requirements
To enable modeling groups and others who support CMIP6 to demonstrate its impact (and secure ongoing funding), you are required to cite and acknowledge those who have made CMIP6 possible. You also must abide by any licensing restrictions, which are recorded in each file as a global attribute (named “license”).
To enable modeling groups and others who support CMIP6 to demonstrate its impact (and secure ongoing funding), you are required to cite and acknowledge those who have made CMIP6 possible. You also must abide by any licensing restrictions, see below.

<span style="color:red;font-weight:bold">Please carefully read and adhere to</span> the [CMIP6 Terms of Use][termsOfUse].

CMIP6 model output datasets and forcing datasets should, according to the terms of use, be cited by any publication that make use of them (see [Data Citation Guidelines][dataCitationGuide]). It is important to include the version (latest dataset version or if not available the latest data download date) in the data citation of the evolving CMIP6 data. Further information on the data citation concept for CMIP6 is available at [cmip6cite.wdc-climate.de][cmip6cite] and described in [Stockhause and Lautenschlager (2017)][Stockhause2017].

Data references give credit to the data providers and enable the traceability of research findings (see [contribution to the CMIP6 Model Analysis Workshop][citemaws]). They are provided on two granuliarities: fine experiment (contribution by one model to one experiment) and coarse model/MIP (contribution by one model to one MIP). Data references can be found:
* in the CMIP6 CoG sites (see list above): Brief data citation information can be displayed by clicking on “Show citation” next to each dataset. The provided URL or DOI for the finer data reference for the experiment will direct you to the corresponding "landing page" with detailed information including the coarser model/MIP granularity.
* in the CMIP6 CoG sites (see list above): Brief data citation information can be displayed by clicking on “Show citation” next to each dataset. The provided URL or DOI for the finer data reference for the experiment will direct you to the corresponding "landing page" with detailed information including the coarser model/MIP granularity.
* using the [Citation Service search][citesearch]
* searching [DataCite's catalog][datacitecat]
* using [Google's Dataset Search][gdatasetsearch]
Expand All @@ -90,21 +89,21 @@ The [controlled vocabularies][cmip6Cvs] contain basic information about the [mod
## 6. Reporting suspected errors
Information about discovered issues of CMIP6 data is captured by the [ES-DOCs Errata Service][ES-DOCErrataService].
The Errata Service provides the ability to query modifications and/or corrections applied to CMIP6 data in two ways:
* A **[user friendly filtered list of ESGF known issues][errataSearchUIDoc]**.
* A **[search interface that helps retrace a specific dataset/file version history][errataPIDLookupDoc]**.
* A **[user friendly filtered list of ESGF known issues][errataSearchUIDoc]**.
* A **[search interface that helps retrace a specific dataset/file version history][errataPIDLookupDoc]**.

Any ESGF user can report an error to the appropriate modeling group (see "contact" attribute in the netCDF files), or through the <a href="mailto:[email protected]">ESGF user mailing list</a>. After a report is received, the corresponding data manager can create a new errata entry using
[an easy and user-friendly form][errataFormCreateDoc]. A [command line client][errataCLCDoc] is also available. The aim is to clearly and concisely document the issue and through the PID integration, this errata service will include all the datasets/files affected when documentation is completed correctly.

## 7. Registering published work based on CMIP6
Please register on the [CMIP6 publication database][CMIP6pubs] any articles you publish that make use of CMIP6 output. Further guidance will be forthcoming.
Please register on the [CMIP6 publication database][CMIP6pubs] any articles you publish that make use of CMIP6 output.

## 8. CMIP6 organization and governance
The [CMIP Panel][CMIPPanel], which is a standing subcommittee of the WCRP’s [Working Group on Climate Modeling][wgcmSite] provides overall guidance and oversight of CMIP activities. Notably it determines which MIPs will participate in each phase of CMIP using the established selection criteria listed in Table 1 of [Eyring et al. (2016)][EyringEtAl16]. On [its webpages][wgcmCmip6] the CMIP Panel provides additional information that may be of interest to CMIP6 participants, but only the CMIP6 Guide (this document) provides definitive documentation of CMIP6 technical requirements.

The [endorsed MIPs][CMIP6EndorsedMips] are managed by independent committees, but acceptance of endorsement obligates them to follow CMIP’s technical requirements. Thus across all MIPs, the modeling groups can prepare their model output following a common procedure.

The CMIP Panel has delegated responsibility for most of the technical requirements of CMIP to the [WGCM Infrastructure Panel (WIP)][wipCog]. The mission, rationale and Terms of Reference for the panel can be found [here][wipMission]. The WIP has drafted a number of position papers summarizing CMIP6 requirements and specifications. Among these is the [CMIP6 reference specifications for global attributes, filenames, directory structure and Data Reference Syntax (DRS)][cmip6GlobalAttGoogleDoc]. The WIP has also set up a [CMIP Data Node Operations Team (CDNOT)][cdnotGoogleDoc] to interface with data node managers responsible for serving CMIP6 data. This team provides a direct link from the panels establishing data node requirements to those implementing the requirements.
The CMIP Panel has delegated responsibility for most of the technical requirements of CMIP to the [WGCM Infrastructure Panel (WIP)][wip]. The mission, rationale and Terms of Reference for the panel can be found [here][wip]. The WIP has drafted a number of position papers summarizing CMIP6 requirements and specifications. Among these is the [CMIP6 reference specifications for global attributes, filenames, directory structure and Data Reference Syntax (DRS)][cmip6GlobalAttGoogleDoc]. The WIP has also set up a [CMIP Data Node Operations Team (CDNOT)][cdnotGoogleDoc] to interface with data node managers responsible for serving CMIP6 data. This team provides a direct link from the panels establishing data node requirements to those implementing the requirements.

Information is under preparation describing the governance of the following:

Expand All @@ -115,7 +114,7 @@ Information is under preparation describing the governance of the following:
* Long-term archival (LTA) and data quality assurance (QA)
* Evaluation activities
* [input4MIPs][input4mipsHome]
* [obs4MIPs][obs4mipsCog]
* [obs4MIPs][obs4mips]

###### Document version: 09 April 2020

Expand All @@ -130,12 +129,12 @@ Information is under preparation describing the governance of the following:
[activityIdJson]: https://github.com/WCRP-CMIP/CMIP6_CVs/blob/master/CMIP6_activity_id.json
[experimentIdJson]: https://github.com/WCRP-CMIP/CMIP6_CVs/blob/master/CMIP6_experiment_id.json
[experimentIdhtml]: https://wcrp-cmip.github.io/CMIP6_CVs/docs/CMIP6_experiment_id.html
[cmip6DataRequestCog]: https://www.earthsystemcog.org/projects/wip/CMIP6DataRequest
[cmip6DataRequest]: https://cmip6dr.github.io/Data_Request_Home
[cfConventionsPage]: http://cfconventions.org/
[cmip6GlobalAttGoogleDoc]: http://goo.gl/v1drZl
[cmip6GridGoogleDoc]: http://goo.gl/1oA7bO
[cmip6PressureLevelsPdf]: https://www.earthsystemcog.org/site_media/projects/wip/CMIP6_pressure_levels.pdf
[cmip6TimeAveragesCog]: https://www.earthsystemcog.org/projects/wip/time_averages
[cmip6PressureLevelsPdf]: https://cmip6dr.github.io/Data_Request_Home/Documents/CMIP6_pressure_levels.pdf
[cmip6TimeAveragesCog]: https://web.archive.org/web/20201020095856/https://www.earthsystemcog.org/projects/wip/time_averages
[datacitecat]: https://search.datacite.org/works?query=prefix:10.22033
[dataHoldings]: https://pcmdi.llnl.gov/CMIP6/ArchiveStatistics/esgf_data_holdings
[esgfSearchRequirementsGoogleDoc]: https://docs.google.com/document/d/1jNBw2am28Hxux_YuCL_mYMi18EEGkJSGrtNntOs3PJo
Expand All @@ -155,13 +154,13 @@ Information is under preparation describing the governance of the following:
[ESGFUsersList]: [email protected]
[wgcmSite]: https://www.wcrp-climate.org/wgcm-overview
[wgcmCmip6]: https://www.wcrp-climate.org/wgcm-cmip/wgcm-cmip6
[wipCog]: https://www.earthsystemcog.org/projects/wip
[wipMission]: https://www.earthsystemcog.org/projects/wip/aboutus/
[wip]: https://wcrp-cmip.github.io/WGCM_Infrastructure_Panel
[cdnotGoogleDoc]: https://docs.google.com/document/d/1oRWqxtWWEfsucTVhk0G3bMqHC0BL4dJwADrOG8Ukj-g
[obs4mipsCog]: https://esgf-node.llnl.gov/projects/obs4mips/
[obs4mips]: https://pcmdi.github.io/obs4MIPs
[CMIP6Citation]: http://cmip6cite.wdc-climate.de
[variableListXls]: http://proj.badc.rl.ac.uk/svn/exarch/CMIP6dreq/tags/latest/dreqPy/docs/CMIP6_MIP_tables.xlsx
[input4mipsHome]: https://esgf-node.llnl.gov/projects/input4mips/
[Durack et al. (2018)]: http://doi.org/10.1029/2018EO101751
[ESGFHome]: https://esgf.llnl.gov/
[BalajiGMD]: https://www.geosci-model-dev-discuss.net/gmd-2018-52/
[ESGFTutorial]: https://esgf.github.io/esgf-user-support/user_guide.html
Expand Down
Loading

0 comments on commit 498fdd2

Please sign in to comment.