-
Notifications
You must be signed in to change notification settings - Fork 3
A. Setting up GEOS ADAS experiments
The easiest why to set up an ADAS experiment is to use an existing experiment template. Such templates are available here. In the initial release of the ADAS v5.27.1_p3 the following experiment templates are available:
- C48f.input - C48 3DVAR
- C90C.input - C90 3DVAR
- C90C_replay.input - C90 hybrid 4DEnVar, with ensemble replayed to the from FP
- C90C_ens.input - C90 hybrid 4DEnVar
- x0043.input - C360 hybrid 4DEnVar; template for actual official x0043 x-test experiment
- x0044.input - C360 hybrid 4DEnVar; template for actual official x0044 x-test experiment
- prePP.input - full resolution, FP-like, Hybrid 4DEnVar
The user should copy a template of interest, make minor editing to adjust userID and location of experiment home (referred to as FVHOME), and then use the program runjob to process the template, e.g.,
FULLPATH/GEOSadas/install/bin/runjob -l path_of_user_adjusted_template_.input
where FULLPATH is the remainder of the path to the GEOSadas bin directory.
The existing experiment templates can be adjusted for setting experiments for a time period other than those set in the template, and the observing system can also easily be edited to accommodate users need.
The more experienced users can run the base GEOSadas script, namely fvsetup, by hand.
Typically, runjob will prompt the user asking whether to submit the main job to the batch system. In case the user chooses to postpone batch submission, this can be done at a later time by
cd FVHOME/run sbatch g5das.j
Note that in the particular case of x-experiment templates, the main job scripts are named according to the experiment number, e.g., x0043 as x43.j, for its main job (i.e., replace g5das.j above with x43.j).
The experiment home directory, referred to as FVHOME, is where key controlling scripts, resource files, and restarts live for a given ADAS experiment. This directory should be place in one of discover's nobackup disks, during the setup setup above, since it typically holds large files as the run gets going.
The main subdirectories under FVHOME are:
- run - controls ADAS cycling (corrector-predictor steps)
- fcst - controls forecast issued from cycled restart (initial conditions)
- anasa - controls running of standalone atmospheric analysis (GSI)
- asens - controls running of FSOI (forecast-based sensitivities observation impact)
The GMAO implementation of a strategy to generate observation impacts combines the work of Langland and Baker (2004) and that of Tremolet (2008) through the so-called Forecast-based Sensitivity Observation Impact (FSOI) methodology; see also Gelaro et al. (2010) and Diniz and Todling (2019). Without getting into the technical aspects of the procedures, a rough understanding is necessary to know what needs to be available for getting GEOS ADAS producing this information. There are basically two main important components to FSOI beyond the regular model (AGCM) and analysis (GSI) pieces, namely, the adjoint for both these components. The measure of impact brought about by assimilating observations is associated with how sensitive the analysis component (GSI) is to minor changes the observations; changes in the observations induce changes in the analysis (IAU tendencies) and therefore involve sensitivities to the model (forecast) integration. For a variety of reasons (beyond the discussion here), it is the impact of observations on the 24-hour forecasts that are derived in GEOS ADAS (changing the lead time in the forecasts and consequently the lead time of impacts is trivial in GEOS DAS, and has been studied to some extent (e.g., Prive et al. (2020), though caution must be exercised in the validity of results beyond 24 hours).
Whenever talking about the sensitivity of the analysis or the sensitivity of the forecasting model, we are basically talking about the adjoint of the linearized operators corresponding to these entities. Sensitivity to changes in the initial conditions are automatically derived in the forecast step of some of the experiment templates introduced earlier. This is the case for the templates of both the x-experiments and prePP. We saw that in some of these cases, 33-hour and 123-hour forecasts are automatically issued. When the machinery of FSOI is being exercised, the 33-hour forecast is used to generate sensitivities to a 30-hour forecast and the 123-hour forecast is used to generate sensitivities to a 24-hour forecast both associated with the same verification time. The generated sensitivities are automatically placed in the directory named FVHOME/asens and they have names like the following
x0044.fsens_twe.eta.17760703_15z+17760705_00z-17760704_00z.nc4
x0044.fsens_twe.eta.17760703_21z+17760705_00z-17760704_00z.nc4
The three date/time tags in the filenames indicate the initial_date/time+final_fcst_date/time-analysis_date/time. In reference to the filenames above, they relate to forecasts started six-hours apart from each other, one at 1500 UTC and another at 2100 UTC on 3 July 1776; both forecasts end at 0000 UTC on 5 July 1776 - therefore, the first is a 33-hour forecast, and the second is a 27-hour forecast; both run the adjoint model (backwards in time) to generate 24-hour sensitivities valid at 0000 UTC on 4 July 1776. These two sensitivities are required by the Langland and Baker (2004) formulation to derive a quantitative assessment of the impact on the 24 hour forecast of assimilating observations at 0000 UTC on 4 July 1776. Notice that the control of how these sensitivities are produced in development-type experiments is all within the g5fcst.j jobs. Therefore, the valid time of these sensitivities is the last date/time in the sequence of time tags; 0000 UTC on 4 July 1776 in the example above. Turning the generation of these forecasts sensitivities off is very easily accomplished by renaming the file FVHOME/fcst/initadj.rc.
The final step in deriving observation impacts involves processing the forecast sensitivities through the adjoint of the analysis solver (GSI). This step is not automatically launched by the forecast or ADAS scripts, but is rather left to the user to do by hand (this might change in upcoming releases). The job script that must be submitted for this step to complete is FVHOME/asens/g5asens.j. It is important to notice that only when a pair of forecast sensitivities is available can this script be submitted to the batch queue. A simple "sbatch g5asens.j" is enough to do the job, however, here too caution must be exercised. By default, when submitted this way, this script will continue to submit itself for as long as there are forecast sensitivity files available in the FVHOME/asens directory. A safer way to submit these jobs is by specifying explicitly the valid date/time of the forecast sensitivities to process. For example:
sbatch --export=this_nymdhh="17760704_00" g5asens.j
Just as seen in previous cases for offline forecasts and standalone analyses, here too, these analysis sensitivity jobs can be submitted in parallel. A simple script place in the FVHOME/asens directory
#!/bin/csh
set yyyymm = 177607
set hh = 00
foreach dd (`seq -f %02g 3 5`)
sbatch --export=this_nymdhh="${yyyymm}${dd}_${hh}" g5asens.j
end
and executed at command line will do the job -- the case above submits three such jobs simultaneously. CAUTION: again must be exercised to not overtake the batch system, especially from users with access to high priority.
# WARNING: working on this section -
By default the verification used for FSOI is given by results from the experiment running FSOI, i.e., self verification. Recall that FSOI involves defining an aspect of the forecast to be evaluated. This is typically represented as a quadratic quantity:
J= < e’ C e >,
with e = xf – xv; xf the vector of forecast fields at given time; and xv the vector of verification fields valid at that same time. By default the file asm.acq under FVHOME/run controls the verification and points to self, that is, assuming user USER runs experiment MYEXP, and output gets stored in the archive, asm.acq looks like:
/archive/u/USER/MYEXP/ana/Y%y4/M%2/MYEXP.asm.eta.%y4%m2%d2_%h2%n2z.nc4
Suppose the users wants to, instead, verify against FP (e.g., f5127_fp). All that needs to be done is for the user to:
- Edit asm.acq so that it looks like:
/home/dao_ops/f5271_fp/run/.../archive/ana/Y%y4/M%2/f5271_fp.asm.eta.%y4%m2%d2_%h2%n2z.nc4
and
- Edit FVHOME/fcst/g5fcst.j and add the following env variable:
setenv VEXPID f5271_fp
Another possibility is that a user might want to calculate FSOI with respect to the analysis rather than the assimilation fields. Recall from the presentation earlier that under IAU there are two states valid at say, a synoptic hour: the analysis and the assimilation. The first one is the state produced by simply adding the GSI increment to the original background field, the second corresponds to the output of the model integration within the IAU 6-hour period. By default, we verify against the assimilation, but changing to verify against the analysis requires simple the definition of the following env variable
setenv FCSTVERIFY ana
in the file FVHOME/fcst/g5fcst.j. This cases the scripts to look at the file ana.acq in FVHOME/run instead of asm.acq. Just as before, it is possible to verify against an alternative analysis (from a unrelated experiment) but in this case editing ana.acq, instead of asm.acq.
Users should know that there has been a lot of experimentation done comparing FSOI verified against the assimilation versus that verified against the analysis. Although the level of forecast errors, say the 24 and 30 hour errors change when the verification changes, the actual total impact - basically the difference between the 24 and 30 hours errors - does not change. The split of the impact into various observation classes also does not change in any significant way. This is not to say that verifying against an independent experiment does not change results; that is a different matter; results certainly change.
Say, sensitivity on the analysis increment at 20130115_00z is to be calculated:
a) touch a file named
standalone.20130115_00z+20130115_00z-20130115_00z
under the FVHOME/asens directory
b) as with regular analysis sensitivity runs, this calculation uses
gsi_sens.rc.tmpl as driver of adjoint-GSI
note: output ODS files will show up under Y2013/M01/D15/H00 with name
type imp1_inc
c) if user wants to apply an initadj-like norm to the increment:
copy an existing initadj.rc to $FVHOME/asens/initadj4inc.rc and edit at will
d) look in fvpsas.rc to properly set reference_eta_filename and
verifcation_eta_filename
entries.
e) make sure ana.acq brings in your reference and verification states