Two small tweaks to thinning

[ADCollard]

This PR addresses GSI #731. It modifies the satthin.F90 routine to allow data to be processed unthinned in the GSI without requiring exorbitant resources and it removes an unnecessary thinning from the read_atms.f90 routine.

The change to satthin reduces the value of itxmax from 1.e9 to 1.e7 which should be sufficient for most sensors. But if a larger number is required for a sensor, this may be passed in via optional argument.

This change will change results for ATMS only as documented in the issue.

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• New and existing tests pass with my changes
• Any dependent changes have been merged and published

[RussTreadon-NOAA]

@ADCollard , from whom can we request peer reviews?

[ADCollard]

ctests on hera:

ctest -j 7
Test project /scratch1/NCEPDEV/da/Andrew.Collard/git/GSI_ADC/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_glbens
    Start 4: netcdf_fv3_regional
    Start 5: hafs_4denvar_glbens
    Start 6: hafs_3denvar_hybens
    Start 7: global_enkf
1/7 Test #4: netcdf_fv3_regional ..............   Passed  486.31 sec
2/7 Test #3: rrfs_3denvar_glbens ..............   Passed  549.84 sec
3/7 Test #7: global_enkf ......................   Passed  807.26 sec
4/7 Test #2: rtma .............................   Passed  1091.24 sec
5/7 Test #6: hafs_3denvar_hybens ..............***Failed  1161.66 sec
6/7 Test #5: hafs_4denvar_glbens ..............***Failed  1222.88 sec

7/7 Test #1: global_4denvar ...................***Failed  1868.28 sec

57% tests passed, 3 tests failed out of 7

Total Test time (real) = 1868.30 sec

The following tests FAILED:
	  1 - global_4denvar (Failed)
	  5 - hafs_4denvar_glbens (Failed)
	  6 - hafs_3denvar_hybens (Failed)
Errors while running CTest

In each case the only differences in the first outer loop was the ATMS statistics, as expected. After the first outer loop, the results diverged. Also as expected.

[ADCollard]

@ADCollard , from whom can we request peer reviews?

@emilyhcliu would be the best as I already told her about this (and she agrees we need these changes!). @xincjin-NOAA, @azadeh-gh or @HaixiaLiu-NOAA would be good too.

Thanks.

[emilyhcliu]

Yes! @ADCollard talked to me about the two proposed changes related to thinning, and I totally agreed with them. Approved!

[RussTreadon-NOAA]

WCOSS2 ctests
Install the authoritative develop @ 29d9d8f and ADCollard:develop @ b401c47 on Dogwood. Run ctests with following results

Test project /lfs/h2/emc/da/noscrub/russ.treadon/git/gsi/pr734/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_glbens
    Start 4: netcdf_fv3_regional
    Start 5: hafs_4denvar_glbens
    Start 6: hafs_3denvar_hybens
    Start 7: global_enkf
1/7 Test #4: netcdf_fv3_regional ..............   Passed  724.86 sec
2/7 Test #3: rrfs_3denvar_glbens ..............   Passed  786.83 sec
3/7 Test #7: global_enkf ......................   Passed  1098.32 sec
4/7 Test #6: hafs_3denvar_hybens ..............***Failed  1399.65 sec
5/7 Test #2: rtma .............................   Passed  1454.41 sec
6/7 Test #5: hafs_4denvar_glbens ..............***Failed  1459.50 sec
7/7 Test #1: global_4denvar ...................***Failed  2043.41 sec

57% tests passed, 3 tests failed out of 7

Total Test time (real) = 2043.43 sec

The following tests FAILED:
          1 - global_4denvar (Failed)
          5 - hafs_4denvar_glbens (Failed)
          6 - hafs_3denvar_hybens (Failed)

All three failed tests are due to differences in how ATMS radiances are assimilated. Both hafs tests see a significant increase in the number of ATMS NOAA-20 observations passing the initial QC as shown below

< o-g 01 rad  n20       atms               2904         2890          506    658.96       658.96       1.3023       1.3023    
---
> o-g 01 rad  n20       atms              68178        16315         3006    3895.9       3895.9       1.2960       1.2960    

< is the contrl (authoritative develop). > is the updat (ADCollard:develop).

Comparison of the contrl and updat initial QC of ATMS radiances show that slightly fewer ATMS observations are assimilated in the updat code

< o-g 01 rad  npp       atms             623436         3432         2470    1365.2       1365.2      0.55271      0.55271    
< o-g 01 rad  n20       atms             623084         3388         2480    1269.4       1269.4      0.51187      0.51187    
< o-g 01 rad  n21       atms             623458         3388         1828    1949.0       1949.0       1.0662       1.0662    
---
> o-g 01 rad  npp       atms           15399934         3454         2398    1313.0       1313.0      0.54753      0.54753    
> o-g 01 rad  n20       atms           15399934         3432         2398    1306.9       1306.9      0.54501      0.54501    
> o-g 01 rad  n21       atms           15399934         3432         1824    2031.8       2031.8       1.1139       1.1139    

[RussTreadon-NOAA]

@ADCollard : the increase in the number of assimilated ATMS observations in the hafs cases is significant. The updat code keeps 16315 ATMS observations whereas the contrl only keeps 2890. This increase is related to removal of lines 549 and 550 from read_atms.f90

     if (5*NINT(REAL(IScan(Iob))/5_r_kind) /= IScan(IOb) .OR. &
          5*NINT(REAL(IFov-3)/5_r_kind) /= IFOV -3 ) CYCLE ObsLoop

from the updat code.

We do not see a similarly large increase in the number of ATMS observations kept by the updat code in the global_4denvar case. The number of ATMS observation read increases significantly but not the number kept..

The global _4denvar case has isfcalc=1 whereas isfcalc=0 in the hafs case. isfcalc=1 turns on the FOV-based surface code. FOV checks must flag many of the extra observations read in the global_4denvar case using the updat code. This screening is not active in the hafs cases.

Does this explanation of the difference between hafs and global behavior make sense?

I don't know if operational regional DA systems assimilate ATMS data with isfcalc=0 or isfcalc=1. If they use isfcalc=0, this PR will significantly increase the number of assimilated ATMS observations.

[ADCollard]

@RussTreadon-NOAA Thanks for highlighting this. I believe this difference is not related to isfcalc which has not changed with this PR. I think the reason is that the thinning distance for HAFS is 45km which is close to the effective resolution of the ATMS (once re-mapping has been done). The change that effectively chooses every 25 scan positions will become much more significant with this much smaller thinning scale.

This change should make ATMS numbers more consistent with AMSU-A. The snapshot below implies that it is a lot closer.

Updat:

 sat       type              penalty    nobs   iland isnoice  icoast ireduce   ivarl nlgross
 n19        amsua       12698.04740480   2292     489       0     119     828    9910       0
                            qcpenalty     qc1     qc2     qc3     qc4     qc5     qc6     qc7
                        12698.04740480    763     597     757       0       0     423      65

  sat       type              penalty    nobs   iland isnoice  icoast ireduce   ivarl nlgross
 n20        atms         5459.90713035    744     744       0       0      90    3363       0
                            qcpenalty     qc1     qc2     qc3     qc4     qc5     qc6     qc7
                         5459.90713035    393      58     228       0       0       0       0

Control:

  sat       type              penalty    nobs   iland isnoice  icoast ireduce   ivarl nlgross
 n19        amsua       12674.39577140   2292     489       0     119     828    3686       0
                            qcpenalty     qc1     qc2     qc3     qc4     qc5     qc6     qc7
                        12674.39577140    736     588     796       0       0     423      66

  sat       type              penalty    nobs   iland isnoice  icoast ireduce   ivarl nlgross
 n20        atms          881.44218749    132     131       0       1      13     301       0
                            qcpenalty     qc1     qc2     qc3     qc4     qc5     qc6     qc7
                          881.44218749     61      12      46       0       0       0       0

[RussTreadon-NOAA]

Thank you @ADCollard for explaining what's going on.

[RussTreadon-NOAA]

Orion ctests
Run ctests on Orion with the following results

    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_glbens
    Start 4: netcdf_fv3_regional
    Start 5: hafs_4denvar_glbens
    Start 6: hafs_3denvar_hybens
    Start 7: global_enkf
1/7 Test #4: netcdf_fv3_regional ..............***Failed  664.09 sec
2/7 Test #3: rrfs_3denvar_glbens ..............***Failed  786.31 sec
3/7 Test #7: global_enkf ......................   Passed  848.70 sec
4/7 Test #2: rtma .............................   Passed  1149.60 sec
5/7 Test #6: hafs_3denvar_hybens ..............***Failed  1582.32 sec
6/7 Test #5: hafs_4denvar_glbens ..............***Failed  1703.04 sec
7/7 Test #1: global_4denvar ...................***Failed  1742.81 sec

29% tests passed, 5 tests failed out of 7

Total Test time (real) = 1742.82 sec

The following tests FAILED:
          1 - global_4denvar (Failed)
          3 - rrfs_3denvar_glbens (Failed)
          4 - netcdf_fv3_regional (Failed)
          5 - hafs_4denvar_glbens (Failed)
          6 - hafs_3denvar_hybens (Failed)

The netcdf_fv3_regional and rrfs_3denvar_glbens failures were due to the thmethresh2 check. For both tests the hiproc_updat wall time exceeded the hiproc_contrl wall time. This is not a fatal fail.

The hafs and global_4denvar failures are for the same reasons as noted above. This PR alters the assimilation of ATMS data. These three tests assimilate ATMS data. These failures are expected.

[RussTreadon-NOAA]

Hercules ctests
Run ctests on Hercules with the following results

Test project /work/noaa/da/rtreadon/git/gsi/pr734/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_glbens
    Start 4: netcdf_fv3_regional
    Start 5: hafs_4denvar_glbens
    Start 6: hafs_3denvar_hybens
    Start 7: global_enkf
1/7 Test #3: rrfs_3denvar_glbens ..............   Passed  1870.11 sec
2/7 Test #4: netcdf_fv3_regional ..............   Passed  1926.39 sec
3/7 Test #7: global_enkf ......................   Passed  1998.10 sec
4/7 Test #6: hafs_3denvar_hybens ..............***Failed  2236.47 sec
5/7 Test #2: rtma .............................   Passed  2287.66 sec
6/7 Test #5: hafs_4denvar_glbens ..............***Failed  2543.76 sec
7/7 Test #1: global_4denvar ...................***Failed  2642.88 sec

57% tests passed, 3 tests failed out of 7

Total Test time (real) = 2642.90 sec

The following tests FAILED:
          1 - global_4denvar (Failed)
          5 - hafs_4denvar_glbens (Failed)
          6 - hafs_3denvar_hybens (Failed)

This behavior is expected and consistent with what is observed on WCOSS2 (Dogwood) and Hera.