Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validation of CRTMv3.1.1 with GFS background fields #167

Open
emilyhcliu opened this issue Aug 19, 2024 · 54 comments
Open

Validation of CRTMv3.1.1 with GFS background fields #167

emilyhcliu opened this issue Aug 19, 2024 · 54 comments

Comments

@emilyhcliu
Copy link

emilyhcliu commented Aug 19, 2024 •
edited
Loading

Validation of CRTMv3.1.1 with GFS background fields from the current operational system (gfs.v16.3)

  • Two experiments for comparison: GSI with CRTMv3.1.1 vs. GSI with CRTMv2.4.0.1 (used in gfs.v16.3)
  • Two experiments use the same input profiles from operational gfs.v16.3 for 2024021900

Issues found will be documented here.
Tagging @ADCollard for awareness.
@emilyhcliu
Copy link
Author

emilyhcliu commented Aug 19, 2024 •
edited
Loading

Issue-1 Users need to load Aerosol Coefficients when Cloud Coefficients are loaded

CRTMv3/src/CRTM_K_Matrix_Module.f90

Lines 710 to 717 in 229ff89

IF( (CloudC%N_PHASE_ELEMENTS /= AeroC%N_PHASE_ELEMENTS) .or. &
(RTV(1)%n_Stokes > 1.and.CloudC%N_PHASE_ELEMENTS < 6) ) THEN
Error_Status = FAILURE
WRITE( Message,'("N_PHASE_ELEMENTS NOT RIGHT FW ",i0)' ) CloudC%N_PHASE_ELEMENTS
CALL Display_Message( ROUTINE_NAME, Message, Error_Status )
RETURN
END IF

The above code block indicates that if the cloud coefficients are loaded, the aerosol coefficients need to be loaded as well. So, I modified GSI to load the aerosol coefficients for all-sky assimilation of ATMS and AMSU-A, for which the cloud coefficients are required. This resolved the issue of n_phase_elenemts difference between cloud and aerosol coefficients.

Action required: Only GSI changes are required -- need to load aerosol coefficients when cloud coefficients are loaded

@emilyhcliu
Copy link
Author

emilyhcliu commented Aug 19, 2024 •
edited
Loading

Issue-2 - ATMS N21 coefficients

GSI develop + CRTMv2.4.0.1 for ATMS N21
GSIobserver_crtm2 4 0 1_v17 atms_n21 tb_omf_nbc channel_2 2024021900

GSI develop + CRTMv3.1.1 for ATMS N21
GSIobserver_v17 atms_n21 tb_omf_nbc channel_2 2024021900

GSI develop + CRTMv2.4.0.1 for ATMS N20
GSIobserver_crtm2 4 0 1_v17 atms_n20 tb_omf_nbc channel_2 2024021900

GSI develop + CRTMv3 for ATMS N20
GSIobserver_v17 atms_n20 tb_omf_nbc channel_2 2024021900

GSI develop + CRTMv2.4.0.1 for ATMS NPP
GSIobserver_crtm2 4 0 1_v17 atms_npp tb_omf_nbc channel_2 2024021900

GSI develop + CRTMv3 for ATMS NPP
GSIobserver_v17 atms_npp tb_omf_nbc channel_2 2024021900

@emilyhcliu emilyhcliu mentioned this issue Aug 19, 2024
Open
@BenjaminTJohnson
Copy link
Contributor

@emilyhcliu can you provide me with the endianness and the shasum of the atms_n21 coefficient that you're using here for v2.4.0 and v3.1.1?

Try copying the v2.4.0 coefficient into the v3.1.1 directory, replacing the one there (assuming it has a different shasum).

Here's a list of the shasums of the various CRTM releases for atms_n21 (big endian only):

8f8c227bb97e9560359363da795e63d090acc7a6  ./fix_REL-2.4.0_emc_07112023/fix/SpcCoeff/Big_Endian/atms_n21.SpcCoeff.bin
8f8c227bb97e9560359363da795e63d090acc7a6  ./fix_REL-3.0.1_20230923/fix/SpcCoeff/Big_Endian/atms_n21.SpcCoeff.bin
8f8c227bb97e9560359363da795e63d090acc7a6  ./fix_REL-3.1.0.1_01252024/fix/SpcCoeff/Big_Endian/atms_n21.SpcCoeff.bin
8f8c227bb97e9560359363da795e63d090acc7a6  ./fix_REL-3.1.0.3/fix/SpcCoeff/Big_Endian/atms_n21.SpcCoeff.bin
8f8c227bb97e9560359363da795e63d090acc7a6  ./fix_REL-3.1.1.0/fix/SpcCoeff/Big_Endian/atms_n21.SpcCoeff.bin
8f8c227bb97e9560359363da795e63d090acc7a6  ./fix_REL-3.1.1.1/fix/SpcCoeff/Big_Endian/atms_n21.SpcCoeff.bin
851124f6ea7773ab9acf931d2f853cd5c5913ebf  ./crtm-internal_REL-2.4.0_emc/fix/SpcCoeff/Big_Endian/atms_n21.SpcCoeff.bin

It's possible I have mistakenly copied the wrong one from to the v3.x tarballs, i.e., I suspect now that you might need the last one in the list above, when I've been copying the first one to the various tarballs.

@emilyhcliu
Copy link
Author

@BenjaminTJohnson Thanks for looking into N21.
I am testing CRTMv3.1.1 and CRTMv2.4.0.1 on ORION
The coefficients used are Little_Endian
Here is the list of shasum for N21 from CRTMv3.1.1

82beb7dc1ccc9b6cb0a5a7e32904514cd79c7c09  atms_n21.SpcCoeff.bin
b587a277f15013a152cbc0748af52c5367f33c0c  atms_n21.TauCoeff.bin

Here is the list of shasum for N21 from CRTMv2.4.0.1 (used in operational GFS)

8f8c227bb97e9560359363da795e63d090acc7a6  atms_n21.SpcCoeff.bin
bcfc9d95e808f11746f81d5d9d325278a4e898b0  atms_n21.TauCoeff.bin

@emilyhcliu
Copy link
Author

@BenjaminTJohnson One question.
Is there a Jacobian-related change in CRTMv3.1.1?

@BenjaminTJohnson
Copy link
Contributor

BenjaminTJohnson commented Aug 19, 2024 •
edited
Loading

Okay, thanks. Is there any reason you're compiling in Little Endian, or was that the default?

I think that the problem is that the LE coefficient isn't the corresponding one to the BE coefficient -- EMC was the only one using the BE coefficient that was updated awhile back, so it looks like the corresponding LE coefficient wasn't made.

Two options (1) compile and run with BE coefficients... you can try the export trick:
export F_UFMTENDIAN=big

No need to recompile, just re-run the code and it will try to read BE instead of LE, but of course you'll need the appropriate soft links to the BE coefficients.

(2) I can create a LE coefficient for you from the BE coefficient. I will do this anyway, but it might take some time.

@emilyhcliu
Copy link
Author

Okay, thanks. Is there any reason you're compiling in Little Endian, or was that the default?

I think that the problem is that the LE coefficient isn't the corresponding one to the BE coefficient -- EMC was the only one using the BE coefficient that was updated awhile back, so it looks like the corresponding LE coefficient wasn't made.

Two options (1) compile and run with BE coefficients... you can try the export trick: export F_UFMTENDIAN=big

No need to recompile, just re-run the code and it will try to read BE instead of LE, but of course you'll need the appropriate soft links to the BE coefficients.

(2) I can create a LE coefficient for you from the BE coefficient. I will do this anyway, but it might take some time

The orion takes little endian by default.
OK. I will try the export trick you suggest and run with big_endians.
Thanks!

@BenjaminTJohnson
Copy link
Contributor

@emilyhcliu Okay, good to know. I will make a point now to verify that all LE files are the same as the BE files, which I should've done before!

I have converted the BE file to the LE file:

Please test it first:
https://bin.ssec.wisc.edu/pub/s4/CRTM/file/atms_n21-LE.SpcCoeff.bin

Rename to atms_n21.SpcCoeff.bin or point softlink with the appropriate name.

If it works, this will go into future tarball releases (3.1.1.2+)

@emilyhcliu
Copy link
Author

emilyhcliu commented Aug 19, 2024 •
edited
Loading

@BenjaminTJohnson The big endian N21 coefficient file is Good.
CRTMv3.1.1 + Big Endian coefficients
GSIobserver_v17_be atms_n21 tb_omf_nbc channel_2 2024021900

CRTMv2.4.0.1
GSIobserver_crtm2 4 0 1_v17 atms_n21 tb_omf_nbc channel_2 2024021900

@emilyhcliu
Copy link
Author

emilyhcliu commented Aug 20, 2024 •
edited
Loading

@BenjaminTJohnson I am using Big_Endian coefficients with export F_UFMTENDIAN=big on ORION
The ATMS N21 looks good now with the Big_Endian coefficients (see my previous comment)

I ran CRTMv3.1.1 wth Big_Endian coefficients for all other radiance types and ran into a problem for amsua_metop-b.
Here is the error message:

SpcCoeff_ReadFile(Binary)(FAILURE) : Error reading channel data. input statement requires too much data, unit 10, file
/work2/noaa/da/eliu/ORION/GSIobserver_v16/allobs/2024021900/gsi/./crtm_coeffs/amsua_metop-b.SpcCoeff.bin

Here are the shasum info for amsua_metop-b (Big_Endian)

5cff45cd262710090729aec6fa69eb393969483e  amsua_metop-b.SpcCoeff.bin 
52217cd06c5bbc8f5d9837cdb027f810d4d60164  amsua_metop-b.TauCoeff.bin

I removed the assimilation of amsua_metop-b and tried again. All other sensors worked OK.
Could you check the amsua_metop-b coefficients?

@BenjaminTJohnson
Copy link
Contributor

I can reproduce the same issue with the SpcCoeff_Inspect tool:

./SpcCoeff_Inspect ../test_data/3.1.1/fix_REL-3.1.1.1/fix/SpcCoeff/Big_Endian/amsua_metop-b.SpcCoeff.bin 

     **********************************************************
                          SpcCoeff_Inspect

      Program to display the contents of a CRTM Binary format 
      R8 SpcCoeff file to stdout.

      $Revision$
     **********************************************************

 SpcCoeff_ReadFile(Binary)(FAILURE) : Error reading channel data. input statement requires too much data, unit 10, file /data/users/bjohnson/CRTM/CRTMv3_REL-3.1.1/build/test/../test_data/3.1.1/fix_REL-3.1.1.1/fix/SpcCoeff/Big_Endian/amsua_metop-b.SpcCoeff.bin

It's likely related to the coefficient version number. I'll dig into it now.

@BenjaminTJohnson
Copy link
Contributor

Andrew found this bug in CRTM v2.4 recently, and the fix wasn't propagated into v3. (JCSDA/crtm#43)

I'll work on a PR now.

@BenjaminTJohnson
Copy link
Contributor

@emilyhcliu if you can check out the branch: hotfix/btj_fix_version_check_spccoeff_binary_io_REL-3.1.1 -- this should fix the error. Please verify that it doesn't break other things. If it works out, I'll update the coefficient tarball with the updated LE file, and update release/REL-3.1.1 with this fix (I also added it to develop).

@emilyhcliu
Copy link
Author

@emilyhcliu if you can check out the branch: hotfix/btj_fix_version_check_spccoeff_binary_io_REL-3.1.1 -- this should fix the error. Please verify that it doesn't break other things. If it works out, I'll update the coefficient tarball with the updated LE file, and update release/REL-3.1.1 with this fix (I also added it to develop).

OK! Working on it and report back here.

@emilyhcliu
Copy link
Author

@BenjaminTJohnson
I checked out the hotfix/btj_fix_version_check_spccoeff_binary_io_REL-3.1.1 and ran two GDAS experiments with all observation types, big and little endian, respectively
Both runs were completed successfully, and the results (O-F and data passed QC) are comparable to those from the operational system.

@BenjaminTJohnson
Copy link
Contributor

@emilyhcliu awesome, thank you for checking!

@emilyhcliu
Copy link
Author

@emilyhcliu awesome, thank you for checking!

@BenjaminTJohnson
You are welcome!
Are you going to update relaese/REL-3.1.1 with the fixes from the hotfix branch and then create a new tag?

@BenjaminTJohnson
Copy link
Contributor

@emilyhcliu yes, I'm working on it now, will let you know when I have a new tag for release/REL-3.1.1 Should be today.

@BenjaminTJohnson BenjaminTJohnson mentioned this issue Aug 20, 2024
Open
14 tasks
@RussTreadon-NOAA RussTreadon-NOAA mentioned this issue Aug 30, 2024
Open
@emilyhcliu
Copy link
Author

emilyhcliu commented Sep 8, 2024 •
edited
Loading

Preliminary validation of TROPICS using CRTMv3.1.1 with the TROPICS coefficients delivery on 08302024. In this coefficient delivery, the naming convention for TROPICS changed to the following:

tms_tropics-01, tms_tropics-02, tms_tropics-03, ..., tms_tropics-07

Currently, only TROPICS-03, TROPICS-05, and TROPICS-06 are in operational status.
So, the validation of TROPICS was performed for these three TROPICS satellites.

Summary:
TROPICS at EMC

  1. The coefficients for TROPICS-03 do not look right.

For example, TROPICS-03 channel 2 simulated BT values look too low (see the first plot below) compared to the same channel from TROPICS-06 (see the second plot below). The emissivity values are comparable. So, something is wrong with the coefficients from TROPICS-03. All other channels from TROPICS-03 have the same problem.

TROPICS at EMC

TROPICS at EMC (1)

  1. There are some minor issues with the moisture channels (G band).
    TROPICS at EMC
    All simulated BT values look OK except for a few FOVs where the background moisture is high (supersaturation). The high humidity and cloud content values cause negative layer optical depth estimation and thus huge, unphysical transmittance (should be between 0 and 1) and brightness temperatures.
    The model moisture values are higher than those that the coefficients can handle.
    This only occurs in a few FOVs (< 5) where the layers are in supersaturation.

Here is the example

 icmask        :  over ocean
 lat lon       :    6.91820000000000        348.506310000000
 channel no.   :            9
 emissivity    :   0.680391304489999       0.000000000000000E+000
 tsim tsim_clr :   -4.855339803624857962E+07   2.388710652256545188E+02
 total_od      :    105.287784254355
 tcc           :   0.999996376457260


   Layer          Pressure              Temperature                  Moisture                      RH                    Transmittance              Layer Optical Depth           Cloud Fraction
    46   0.112368755452404599E+03   0.192948462139890580E+03   0.352237515743650904E-02   0.118441222522142575E+01   0.984267912974268744E+00   0.627289539468096208E-02   0.100000000000000000E+01
    47   0.120159963699790083E+03   0.195010011677196673E+03   0.530941336449387933E-02   0.141851828835153904E+01   0.955737898333287594E+00   0.289904275737285257E-01   0.100000000000000000E+01 
    48   0.128321931001968267E+03   0.198251431525769362E+03   0.913348737092145764E-02   0.155393938316896718E+01   0.893223071250948353E+00   0.666722649776307352E-01   0.100000000000000000E+01  
    49   0.136864180694765338E+03   0.201298209494333463E+03   0.130024834898564215E-01   0.146679308211965598E+01   0.808946417522109718E+00   0.976751478296985082E-01   0.100000000000000000E+01
    50   0.145795486446628814E+03   0.204544775824695705E+03   0.202593913081536951E-01   0.149171737239978541E+01   0.727175642492371765E+00   0.105028569359558038E+00   0.100000000000000000E+01
    51   0.155123746674179529E+03   0.207770956490796891E+03   0.297951400349664650E-01   0.145531473237471753E+01   0.670402782643488404E+00   0.801176104191126115E-01   0.100000000000000000E+01
    52   0.164855893895915301E+03   0.211041778168916323E+03   0.431277934862537432E-01   0.140984322386916183E+01   0.655460166734705307E+00   0.222162489188298361E-01   0.100000000000000000E+01
    53   0.174997710870320589E+03   0.214335535560490712E+03   0.617697275954307012E-01   0.136529859850888324E+01   0.700777499868205256E+00  -0.658892547995999917E-01   0.100000000000000000E+01
    54   0.185553738029519991E+03   0.217665378297667218E+03   0.873332587148814149E-01   0.131748427729100204E+01   0.840514807232376704E+00  -0.179203251470459485E+00   0.100000000000000000E+01
    55   0.196527138400002229E+03   0.220969451105517066E+03   0.122380758562382272E+00   0.127934043459363567E+01   0.115732702643507146E+01  -0.315243267929734561E+00   0.100000000000000000E+01
    56   0.207919540026004995E+03   0.224301246954333465E+03   0.169436993888227888E+00   0.123871120695182291E+01   0.186194766259390554E+01  -0.468655820703007708E+00   0.100000000000000000E+01
    57   0.219730943316417552E+03   0.227568202169135702E+03   0.232739310167697749E+00   0.121250154903814344E+01   0.357072631435203558E+01  -0.641760076684931513E+00   0.100000000000000000E+01
    58   0.231959558755227135E+03   0.230779442822262922E+03   0.312895227513714347E+00   0.118077569218216971E+01   0.824802065410952245E+01  -0.825136430680274202E+00   0.100000000000000000E+01
    59   0.244601743008196053E+03   0.233913359805495134E+03   0.416553205266905036E+00   0.116064179545471702E+01   0.233501022801095814E+02  -0.102562808483912882E+01   0.100000000000000000E+01
    60   0.257651829770476809E+03   0.236939023891545759E+03   0.549843701941331586E+00   0.115420842857638317E+01   0.798369776910072204E+02  -0.121166456058449068E+01   0.100000000000000000E+01
    61   0.271101930362848350E+03   0.239860925678088620E+03   0.706732012915749608E+00   0.113866544491397637E+01   0.320754150461042173E+03  -0.137064223259729134E+01   0.100000000000000000E+01
    62   0.284941893151870829E+03   0.242713826805646534E+03   0.892381160665631779E+00   0.111860643921549818E+01   0.147397354127264407E+04  -0.150305961273278887E+01   0.100000000000000000E+01
    63   0.299159348057764532E+03   0.245432137711676546E+03   0.110551000855024251E+01   0.109928890755727826E+01   0.750818528049870474E+04  -0.160456485087540557E+01   0.100000000000000000E+01
    64   0.313739716647835962E+03   0.248016029335115036E+03   0.134767134920098175E+01   0.108282948046497940E+01   0.408832162437083607E+05  -0.167029736787144123E+01   0.100000000000000000E+01
    65   0.328666163484191941E+03   0.250522622974734190E+03   0.162681992781712004E+01   0.106844222969337732E+01   0.227930660959769506E+06  -0.169355326955701480E+01   0.100000000000000000E+01
    66   0.343919641283895544E+03   0.252847315002393259E+03   0.192513184263021642E+01   0.105325187653739039E+01   0.124592333921261318E+07  -0.167410654382820545E+01   0.100000000000000000E+01
    67   0.359478944240781971E+03   0.255082476083769620E+03   0.225297347959660366E+01   0.102432736104646893E+01   0.641926583191901911E+07  -0.161579550906801606E+01   0.100000000000000000E+01
    68   0.375320792920089048E+03   0.257197508751600026E+03   0.260553149793525041E+01   0.999169290844158398E+00   0.298638691781954020E+08  -0.151518569345517595E+01   0.100000000000000000E+01
    69   0.391419905135327326E+03   0.259146337272336382E+03   0.296857690737864344E+01   0.982301254118353118E+00   0.122443059165241137E+09  -0.139065806795419999E+01   0.100000000000000000E+01
    70   0.407749123703416387E+03   0.261018926395882829E+03   0.335751244740984900E+01   0.970500429095716211E+00   0.440509229986245334E+09  -0.126183075680462564E+01   0.100000000000000000E+01
    71   0.424279601209808447E+03   0.262784035482675449E+03   0.376061611836828780E+01   0.963539118262376371E+00   0.138903963087451196E+10  -0.113188245424781542E+01   0.100000000000000000E+01
    72   0.440980972861858959E+03   0.264379346083455857E+03   0.415567586969260727E+01   0.961722914129143791E+00   0.387680629636699247E+10  -0.101160417383636014E+01   0.100000000000000000E+01
    73   0.457821591633499452E+03   0.265917377094614949E+03   0.456830716609579035E+01   0.962129730297707875E+00   0.980117483328488922E+10  -0.914121345287031706E+00   0.100000000000000000E+01
    74   0.474768715546678322E+03   0.267381158702413131E+03   0.498277929800677377E+01   0.963421132320651519E+00   0.226938099882273064E+11  -0.827487755819400594E+00   0.100000000000000000E+01
    75   0.491788758866018497E+03   0.268787649563512673E+03   0.539948351817316574E+01   0.964951457884979047E+00   0.484555230262334442E+11  -0.747620024871503475E+00   0.100000000000000000E+01
    76   0.508847589934717348E+03   0.270124119805240525E+03   0.580529980387877576E+01   0.966006822778305962E+00   0.965056665614517212E+11  -0.679024523305099326E+00   0.100000000000000000E+01
    77   0.525910778295324690E+03   0.271441168814241792E+03   0.621372992878328478E+01   0.965764804036856050E+00   0.177285967093166321E+12  -0.599396040191106816E+00   0.100000000000000000E+01
    78   0.542943847917869107E+03   0.272500489526112347E+03   0.659475224143999927E+01   0.977993545069821613E+00   0.188201769227405396E+12  -0.588892972646105420E-01   0.100000000000000000E+01
    79   0.559912599245759338E+03   0.273594377667511935E+03   0.701007486679936953E+01   0.989286221853408532E+00   0.119403862496893616E+12   0.448444482369592834E+00   0.100000000000000000E+01
    80   0.576783352437270082E+03   0.274645003015366456E+03   0.742502044782967197E+01   0.999626857295710547E+00   0.362602205045220261E+11   0.117461132289693948E+01   0.100000000000000000E+01
    81   0.593523209689944224E+03   0.276012203908990045E+03   0.794042667483109277E+01   0.997570226589861275E+00   0.719473793743875408E+10   0.159405793363552140E+01   0.100000000000000000E+01
    82   0.610100327012707680E+03   0.277322812592357423E+03   0.843373930293878615E+01   0.992483828266428558E+00   0.766290570243414879E+09   0.220726208631862253E+01   0.100000000000000000E+01
    83   0.626484120454552794E+03   0.278400072353277892E+03   0.884927214360775416E+01   0.991187505647120548E+00   0.104841582215629935E+09   0.196043912343032334E+01   0.100000000000000000E+01
    84   0.642645492819719721E+03   0.279411225869916848E+03   0.925604645008835547E+01   0.990952632778963505E+00   0.331018831988864318E+08   0.113624250691173234E+01   0.100000000000000000E+01
    85   0.658557013433576685E+03   0.280364930560214930E+03   0.965222650284527894E+01   0.991183188847337804E+00   0.144310428791106530E+08   0.818241576263193826E+00   0.100000000000000000E+01
    86   0.674193105364881717E+03   0.281273565598070547E+03   0.100354896321537375E+02   0.991026192619349988E+00   0.639545296894282289E+07   0.802064020741179107E+00   0.100000000000000000E+01
    87   0.689530126870964750E+03   0.282138423751850041E+03   0.104097833948930774E+02   0.990980367926629580E+00   0.288836097268483089E+07   0.783440082775412217E+00   0.100000000000000000E+01
    88   0.704546511519773503E+03   0.282958158395488908E+03   0.107707099066923035E+02   0.990899005258641274E+00   0.117364668070812267E+07   0.887592255667218666E+00   0.100000000000000000E+01
    89   0.719222908020927662E+03   0.283730880754588952E+03   0.111202103088698081E+02   0.991250772613729403E+00   0.396672082618789515E+06   0.106912487220368413E+01   0.100000000000000000E+01
    90   0.733542144544168195E+03   0.284470647772360280E+03   0.114563409839619972E+02   0.991059818854393759E+00   0.177472661241370399E+06   0.792699969594861331E+00   0.100000000000000000E+01
    91   0.747489241315466984E+03   0.285170671444095319E+03   0.117814221692832835E+02   0.991114507536498324E+00   0.881529164654989436E+05   0.689657199207583504E+00   0.100000000000000000E+01
    92   0.761051496050116725E+03   0.285838581413680117E+03   0.120966766878583236E+02   0.991110248754739631E+00   0.449884899505943322E+05   0.662970234788192170E+00   0.100000000000000000E+01
    93   0.774218370525351247E+03   0.286473941211435260E+03   0.124007739670579546E+02   0.991078591638424267E+00   0.236033741603706294E+05   0.635719471090700927E+00   0.100000000000000000E+01
    94   0.786981495816953611E+03   0.287078960309659806E+03   0.126957980621973494E+02   0.991111195500771647E+00   0.127181561745598683E+05   0.609445807667855277E+00   0.100000000000000000E+01
    95   0.799334589345327231E+03   0.287652195558099606E+03   0.129783461492503776E+02   0.991135714796757439E+00   0.703817243637718821E+04   0.583153312081281183E+00   0.100000000000000000E+01
    96   0.811273330505903914E+03   0.288192564407905479E+03   0.132501704873896085E+02   0.991421151924577804E+00   0.399303044425244752E+04   0.558628033191450046E+00   0.100000000000000000E+01
    97   0.822795341343545374E+03   0.288715219637135419E+03   0.135166657223443405E+02   0.991462921341760506E+00   0.174513164622633712E+04   0.815789364840150411E+00   0.100000000000000000E+01
    98   0.833900000154085774E+03   0.289213233584794921E+03   0.137759469649759545E+02   0.991622797558799229E+00   0.589649181915120693E+02   0.333881178169608983E+01   0.100000000000000000E+01
    99   0.844588385462345514E+03   0.289691350356601617E+03   0.140288840976259674E+02   0.991704909301065363E+00   0.236142827774331376E+01   0.317129515605179968E+01   0.100000000000000000E+01
   100   0.854863083660557209E+03   0.290149786650059866E+03   0.142781695318257142E+02   0.991936822485589231E+00   0.261452925285869109E-01   0.443843963807860575E+01   0.100000000000000000E+01
   101   0.864728095870725610E+03   0.290585845774668655E+03   0.145199061429880398E+02   0.992255683627020768E+00   0.205020777698625806E-03   0.477842758757468911E+01   0.100000000000000000E+01 
Layer          Pressure              Temperature                  Moisture                      RH                    Transmittance.               Layer Optical Depth               Cloud Fraction

@BenjaminRuston
Copy link
Contributor

@BenjaminTJohnson and @emilyhcliu we are also seeing something off with the TROPICS-03 coefficients this is using :
crtm/fix_REL-3.1.1.2/fix/TauCoeff/ODPS/Little_Endian/tms_tropics-0*
crtm/fix_REL-3.1.1.2/fix/SpcCoeff/Little_Endian/tms_tropics-0*

image

@BenjaminTJohnson
Copy link
Contributor

BenjaminTJohnson commented Sep 11, 2024 via email

@fabiolrdiniz
Copy link
Collaborator

@BenjaminTJohnson, here is a list of the shasums for tms_tropics-03 & -05 (little endian only):

3a6f136c2c5aa7bcde37365af44e154e4b5b2037  ./fix_REL-3.1.1.2/fix/SpcCoeff/Little_Endian/tms_tropics-05.SpcCoeff.bin
4b5e32105fb5ba127b3907b3021c4b25cf4077e8  ./fix_REL-3.1.1.2/fix/SpcCoeff/Little_Endian/tms_tropics-03.SpcCoeff.bin
cd8d5cffdb9b2fd2d97fc047f5f973a5e96dd3fa  ./fix_REL-3.1.1.2/fix/TauCoeff/ODPS/Little_Endian/tms_tropics-03.TauCoeff.bin
2d8b6e91c048caf23ac9c5aa4b9180cbd5c16614  ./fix_REL-3.1.1.2/fix/TauCoeff/ODPS/Little_Endian/tms_tropics-05.TauCoeff.bin

@BenjaminTJohnson
Copy link
Contributor

I'm also seeing the TB calculations are different when using the netcdf and binary files for tms_tropics-03 only. I will fix this today and provide an update. All of the other coefficients look fine. The problem isn't immediately obvious, so it might take some poking around.

@BenjaminTJohnson
Copy link
Contributor

BenjaminTJohnson commented Sep 12, 2024 •
edited
Loading

@ADCollard @emilyhcliu @fabiolrdiniz @BenjaminRuston

I recreated all of the tms_tropics-03 coefficients -- here's a link to the updated delivery folder, a link to the tropics-only tarball on the ftp (updated only tms_tropics-03.SpcCoeff* and tms_tropics-03.TauCoeff*)

https://bin.ssec.wisc.edu/pub/s4/CRTM/tropics_09122024_delivery.tgz

https://drive.google.com/drive/folders/185UYpr_PhwfckKHtJtWBexlfgo5L41sm?usp=sharing

The next tarball release of CRTM coefficients fix_REL-3.1.1.3 will contain the fixed files, no ETA on that yet.

@fabiolrdiniz
Copy link
Collaborator

@BenjaminTJohnson we've just run a quick test with your new files from the tropics_09122024_delivery.tgz tarball with these shasums:

215805266650adb666740dd5c2e88603e849ff04  tms_tropics-03.TauCoeff.bin
29c6ac573b1e5955b74c30e94cc3087722b17634  tms_tropics-03.SpcCoeff.bin
2d8b6e91c048caf23ac9c5aa4b9180cbd5c16614  tms_tropics-05.TauCoeff.bin
3a6f136c2c5aa7bcde37365af44e154e4b5b2037  tms_tropics-05.SpcCoeff.bin
8cec0aad9e90fdeb82142790e2b04d05336d1ffb  tms_tropics-06.TauCoeff.bin
d0b14718150d21717cd4c123e268d90e0ea9af7d  tms_tropics-06.SpcCoeff.bin
  • h(x) values summary with develop:
tms_tropics-03 nobs= 4177524 Min=52.0596, Max=330.508, RMS=200.854
tms_tropics-05 nobs= 4864116 Min=62.314, Max=303.619, RMS=255.359
tms_tropics-06 nobs= 5049072 Min=64.1236, Max=304.206, RMS=255.965
  • h(x) values summary with the new files (only changes for tms_tropics-03):
tms_tropics-03 nobs= 4177524 Min=53.963, Max=323.349, RMS=253.674
tms_tropics-05 nobs= 4864116 Min=62.314, Max=303.619, RMS=255.359
tms_tropics-06 nobs= 5049072 Min=64.1236, Max=304.206, RMS=255.965

And following @BenjaminRuston's #167 (comment) figure:

Screenshot 2024-09-12 at 12 03 04 PM

@BenjaminTJohnson
Copy link
Contributor

@fabiolrdiniz this looks to be "better". OMB values would be helpful for overall quality of the coefficients. RMS=253.674 for the new tms_tropics-03 coefficient is now consistent with the other sensors.

@emilyhcliu
Copy link
Author

Here are the Channel 1 Tropics-03 results using the updated coefficients (09122024 delivery) --- Look right to me

Observation

tropics tms_tropics-03 observation channel_1 2024072106

Simulated Observation

tropics tms_tropics-03 simulatedobs channel_1 2024072106

O-F Unadjusted

tropics tms_tropics-03 omf_unadjusted channel_1 2024072106

Here is the Channel 1 Tropics-05 results:

Observation
tropics tms_tropics-05 observation channel_1 2024072106

Simulated Observation

tropics tms_tropics-05 simulatedobs channel_1 2024072106

O-F Unadjusted

tropics tms_tropics-05 omf_unadjusted channel_1 2024072106

@emilyhcliu
Copy link
Author

@fabiolrdiniz @BenjaminRuston @BenjaminTJohnson
The following table is from the previous comment:

tms_tropics-03 nobs= 4177524 Min=53.963, Max=323.349, RMS=253.674
tms_tropics-05 nobs= 4864116 Min=62.314, Max=303.619, RMS=255.359
tms_tropics-06 nobs= 5049072 Min=64.1236, Max=304.206, RMS=255.965

The minimum values of HofX are too low. (~50K) Could you identify the channel?
Thanks.

@BenjaminTJohnson
Copy link
Contributor

BenjaminTJohnson commented Sep 13, 2024 via email

@emilyhcliu
Copy link
Author

emilyhcliu commented Sep 13, 2024 •
edited
Loading

I agree that you'll never see TBs that low in reality or in correct simulations. 90K is really the lowest I've ever seen occurring naturally. My guess is that this is an ice cloud, with too much ice, too high in the atmosphere. I'll leave it to Fabio to identify which channel produces these super low TBs, but my guess is the 205 GHz channel (12) Here's an example of simulated TBs that get down to 90K in hurricanes at 183+/-7 (3rd row 3rd column is the obvious example). [image: image.png]

@BenjaminTJohnson The image does not show.
I agree. Channel 12 could depress the BT a lot more than the other channels due to ice particles.

@fabiolrdiniz
Copy link
Collaborator

fabiolrdiniz commented Sep 13, 2024 •
edited
Loading

The minimum values of HofX are too low. (~50K) Could you identify the channel?

@emilyhcliu it's from channel 12. Please, see below a summary for each of the platforms:

  • tms_tropics-03:
                                                             ObsValue                    hofx            
                                                                  min         max         min         max
variable              channel_number expt_name refTime                                                   
brightnessTemperature 1              Skylab    2024-07-21  172.913635  303.001495   61.776646  323.348907
                      2              Skylab    2024-07-21  158.448578  302.457031   68.563171  309.759827
                      3              Skylab    2024-07-21  167.767960  294.705841   81.259605  299.327820
                      4              Skylab    2024-07-21  172.447098  290.377594   94.906532  289.367462
                      5              Skylab    2024-07-21  189.056198  275.273651  116.515747  275.196838
                      6              Skylab    2024-07-21  192.379044  253.519470  158.157806  250.134247
                      7              Skylab    2024-07-21  194.303268  232.227081  193.123016  226.918442
                      8              Skylab    2024-07-21  194.990021  228.500717  209.523651  227.921768
                      9              Skylab    2024-07-21  139.583603  272.068359   91.893646  273.055817
                      10             Skylab    2024-07-21  128.730835  283.815002   63.313141  283.140350
                      11             Skylab    2024-07-21  124.127464  287.819489   55.737331  287.912476
                      12             Skylab    2024-07-21  105.751259  299.623657   53.962959  300.024292
  • tms_tropics-05:
                                                             ObsValue                    hofx            
                                                                  min         max         min         max
variable              channel_number expt_name refTime                                                   
brightnessTemperature 1              Skylab    2024-07-21  168.433609  294.613373   67.484467  300.656982
                      2              Skylab    2024-07-21         NaN         NaN   75.181709  297.418762
                      3              Skylab    2024-07-21  162.248917  289.225006   85.266518  292.059021
                      4              Skylab    2024-07-21  155.938293  291.721008   96.014412  285.808777
                      5              Skylab    2024-07-21  173.629395  276.906006  115.257484  274.202576
                      6              Skylab    2024-07-21  176.872986  254.874588  153.549255  251.098480
                      7              Skylab    2024-07-21  168.738129  235.059357  187.347626  227.579742
                      8              Skylab    2024-07-21  170.504486  230.131027  209.764999  224.949203
                      9              Skylab    2024-07-21  145.359970  274.139282   95.900795  278.354095
                      10             Skylab    2024-07-21  131.273499  287.919220   69.265396  290.216675
                      11             Skylab    2024-07-21  126.429901  297.038177   63.696941  298.992371
                      12             Skylab    2024-07-21  107.792717  303.699432   62.313972  303.619141
  • tms_tropics-06:
                                                             ObsValue                    hofx            
                                                                  min         max         min         max
variable              channel_number expt_name refTime                                                   
brightnessTemperature 1              Skylab    2024-07-21  156.818680  293.861145   72.122696  301.469208
                      2              Skylab    2024-07-21  140.654129  292.907379   78.676331  298.090363
                      3              Skylab    2024-07-21  150.897171  288.585205   89.086189  292.420074
                      4              Skylab    2024-07-21  147.426880  286.750214  100.768776  285.505188
                      5              Skylab    2024-07-21  165.699173  276.232178  119.331680  274.118317
                      6              Skylab    2024-07-21  185.795166  255.912277  157.085403  250.934937
                      7              Skylab    2024-07-21  196.067245  230.805954  188.515778  227.197754
                      8              Skylab    2024-07-21  204.111374  227.799042  206.749329  224.116791
                      9              Skylab    2024-07-21  142.847015  275.644623   96.426811  279.177063
                      10             Skylab    2024-07-21  124.919464  289.249481   71.504639  290.128632
                      11             Skylab    2024-07-21  122.377251  297.351166   66.100174  298.999115
                      12             Skylab    2024-07-21  100.142387  306.118958   64.123611  304.205597

@BenjaminTJohnson
Copy link
Contributor

Yeah, we're producing too much cloud scattering to be able to generate that low of a TB. @gthompsnJCSDA also sees this in visible / IR images, where we have too much brightness / albedo too high. This suggests that the cloud particle sizes (effective radius) or number concentrations are off. Could also be cloud height errors. Could also be the CloudCoeff LUT.

@emilyhcliu
Copy link
Author

@fabiolrdiniz Thanks for the summary.
It looks like the minimum values of HofX for G-band are low, but they are also low for channels 1-3.
What is the background model? GFS or other?

@fabiolrdiniz
Copy link
Collaborator

What is the background model? GFS or other?

It's GFS c768 grabbed from the NOAA's AWS bucket.

@emilyhcliu
Copy link
Author

What is the background model? GFS or other?

It's GFS c768 grabbed from the NOAA's AWS bucket.

So, we are using the same profiles.
What is your Total Cloud Cover?
Are you assuming overcast?

@emilyhcliu
Copy link
Author

emilyhcliu commented Sep 13, 2024 •
edited
Loading

Here are the background fields

tropics tms_tropics-06 clw_guess 2024072106

tropics tms_tropics-06 ciw_guess 2024072106

tropics tms_tropics-06 rain_guess 2024072106

tropics tms_tropics-06 snow_guess 2024072106

tropics tms_tropics-06 graupel_guess 2024072106

tropics tms_tropics-06 cloud_fraction 2024072106

@fabiolrdiniz
Copy link
Collaborator

fabiolrdiniz commented Sep 13, 2024 •
edited
Loading

So, we are using the same profiles.

Somewhat yes... different pre-processings since you are relying on a BUFR file for GSI and we are converting NASA's file into IODA. And GSI might have some QC being applied too, right?

What is your Total Cloud Cover?
Are you assuming overcast?

We are using Cloud_Fraction: 1.0 in our YAMLs. We are not using it from the model at this point.

Another point of difference between our runs is that you might have access to backgrounds throughout the window via GSI (due to the 4D-flavor settings), while we are seeing numbers referred to an approximated 3D case.

On another topic, @BenjaminRuston asked me to share with you the spaghetti plots previously:

tms_tropics-03 brightnessTemperature sensorScanPosition jedi_raw_omb 2024072100

tms_tropics-05 brightnessTemperature sensorScanPosition jedi_raw_omb 2024072100

tms_tropics-06 brightnessTemperature sensorScanPosition jedi_raw_omb 2024072100

@emilyhcliu
Copy link
Author

The ice signals are weaker in my simulation. The location of the signal is right, but much weaker.
But, we are using the same GFS backend. I am wondering why.
One thing to note that the current GFS operational system is using GFDL cloud physics not Thompson
So, I am using GFDL cloud table in my simulation.

tropics tms_tropics-06 simulatedobs channel_12 2024072106

tropics tms_tropics-06 observation channel_12 2024072106

@BenjaminTJohnson
Copy link
Contributor

I am using GFDL cloud table in my simulation.

I think this is the easiest thing for us to change. We're still using the default CRTM "CloudCoeff.bin", which is most likely a strong contributor here.

@fabiolrdiniz
Copy link
Collaborator

One thing to note that the current GFS operational system is using GFDL cloud physics not Thompson
So, I am using GFDL cloud table in my simulation.

For the effective radius method, as far as I remember, @gthompsnJCSDA set fv3-jedi to use GFDL as default. It's worth checking the code though.

@BenjaminTJohnson
Copy link
Contributor

@fabiolrdiniz
./src/ufo/operators/crtm/ufo_crtm_utils_mod.F90: conf%CloudCoeff_File = "CloudCoeff.bin" change to

first we want to try:
./test_data/3.1.1/fix_REL-3.1.1.2/fix/CloudCoeff/Little_Endian/cloudcoeff_GFDLFV3.bin

then we want to try:
./test_data/3.1.1/fix_REL-3.1.1.2/fix/CloudCoeff/Little_Endian/CloudCoeff.GFDLFV3.-109z-1.bin

probably need to make a change in a CMakeLists.txt file somewhere to find the right file? I'm not that well versed with how UFO is picking up the coefficient files.

@emilyhcliu
Copy link
Author

emilyhcliu commented Sep 13, 2024 •
edited
Loading

So, we are using the same profiles.

Somewhat yes... different pre-processings since you are relying on a BUFR file for GSI and we are converting NASA's file into IODA. And GSI might have some QC being applied too, right?

What is your Total Cloud Cover?
Are you assuming overcast?

We are using Cloud_Fraction: 1.0 in our YAMLs. We are not using it from the model at this point.

Another point of difference between our runs is that you might have access to backgrounds throughout the window via GSI (due to the 4D-flavor settings), while we are seeing numbers referred to an approximated 3D case.

On another topic, @BenjaminRuston asked me to share with you the spaghetti plots previously:

tms_tropics-03 brightnessTemperature sensorScanPosition jedi_raw_omb 2024072100

tms_tropics-05 brightnessTemperature sensorScanPosition jedi_raw_omb 2024072100

tms_tropics-06 brightnessTemperature sensorScanPosition jedi_raw_omb 2024072100

@fabiolrdiniz, thanks for sharing more information. It is useful.
The Channel One scan bias is significant in general and at larger scan positions. At the nadir for TROPICS-06 and TROPICS-05, there is an offset of 5K.
Are the samples for the statistics from clear-sky ones?

@fabiolrdiniz
Copy link
Collaborator

For the effective radius method, as far as I remember, @gthompsnJCSDA set fv3-jedi to use GFDL as default. It's worth checking the code though.

Confirmed. Please, see: https://github.com/JCSDA-internal/fv3-jedi/blob/72cb04e5a4338a6104b75c891db62ea5b0c9274d/src/fv3jedi/VariableChange/Model2GeoVaLs/fv3jedi_vc_model2geovals_mod.f90#L65

@emilyhcliu
Copy link
Author

@fabiolrdiniz You mentioned the following:

Somewhat yes... different pre-processings since you are relying on a BUFR file for GSI and we are converting NASA's file into IODA. And GSI might have some QC being applied too, right?

What is NASA's file? Is it BUFR or HDF/NetCDF?
We remap all ATMS channels to a common 3x3 FOV size using a utility from AAPP software.

@emilyhcliu
Copy link
Author

One thing to note that the current GFS operational system is using GFDL cloud physics not Thompson
So, I am using GFDL cloud table in my simulation.

For the effective radius method, as far as I remember, @gthompsnJCSDA set fv3-jedi to use GFDL as default. It's worth checking the code though.

Got it. Thanks.

@emilyhcliu
Copy link
Author

@fabiolrdiniz Could you point to the skylab monitoring webpage with the experiment name and the user name to see the plots for TROPICS?

@fabiolrdiniz
Copy link
Collaborator

What is NASA's file? Is it BUFR or HDF/NetCDF?

It's an HDF/NetCDF from NASA (please see https://disc2.gesdisc.eosdis.nasa.gov/data/TROPICS_L1B).

We remap all ATMS channels to a common 3x3 FOV size using a utility from AAPP software.

We are not remapping anything on our side yet.

@fabiolrdiniz
Copy link
Collaborator

@fabiolrdiniz Could you point to the skylab monitoring webpage with the experiment name and the user name to see the plots for TROPICS?

Sure. We are testing under the username fabiolrdiniz and experiment ID 487b11. However, I wouldn't rely much on the plots over there because I'm running over the same case multiple times. So the plots will change from time to time depending on the tests we are doing.

@emilyhcliu
Copy link
Author

What is NASA's file? Is it BUFR or HDF/NetCDF?

It's an HDF/NetCDF from NASA (please see https://disc2.gesdisc.eosdis.nasa.gov/data/TROPICS_L1B).

We remap all ATMS channels to a common 3x3 FOV size using a utility from AAPP software.

We are not remapping anything on our side yet.

I am sorry. The TROPICS information I gave you was wrong.
We are receiving TROPICS BUFR files from NESDIS. These BUFR files are converted from NASA's HDF.
We do not remap channels to a common FOV size for TROPICS.
So, we should be using the same information for input.
Sorry about the misunderstanding.

@fabiolrdiniz
Copy link
Collaborator

All good! Thanks for checking. That's good information to know.

@emilyhcliu
Copy link
Author

@fabiolrdiniz Could you point to the skylab monitoring webpage with the experiment name and the user name to see the plots for TROPICS?

Sure. We are testing under the username fabiolrdiniz and experiment ID 487b11. However, I wouldn't rely much on the plots over there because I'm running over the same case multiple times. So the plots will change from time to time depending on the tests we are doing.

That's is good enough. Thanks...

@BenjaminTJohnson
Copy link
Contributor

@emilyhcliu which cloud coefficient file are you using? we want to make sure we're using the same one. Thanks!!

@emilyhcliu
Copy link
Author

@emilyhcliu which cloud coefficient file are you using? we want to make sure we're using the same one. Thanks!!

The current GFS operational system uses GFDL cloud physics mode. So, I am using the following cloud table:
CloudCoeff.GFDLFV3.-109z-1.bin

@fabiolrdiniz
Copy link
Collaborator

Are the samples for the statistics from clear-sky ones?

@emilyhcliu, sorry, I missed this one above... no filtering, all the obs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@BenjaminTJohnson @emilyhcliu @fabiolrdiniz @BenjaminRuston