CLIMO data time_interval retrieval error

[craigr3]

This ticket is a continuation of Discussion #1879 (Climo Time Errors). I am reading NCEP climo data (I sent you the file under the #1879) and still cannot get it to read climo data for any other forecast interval thats not a multple of 6. The climo data is the average value for the middle of the month by every 6 hours. My config file settings are listed below:

If I look for climo to go with a 6 hour forecast, it works fine. If I look for climo to go with a 3 hour forecast, I will get the following error:

You were not able to duplicate this issue last time - maybe you used the 6 hour forecast instead of the 3 hour forecast. I was under the impression I could use this climo against any forecast hour.

[j-opatz]

Hello,

And thank you for following up on the previous discussion.

My sincere apologies, I don't know how I was able to get past the issue you were previously describing in the last Discussions; setting up the exact same example this time, I encountered the issue you're describing. Requesting a standard hour in the valid time (0, 6, 12, and 18) provides a match, while any other hour in the valid time causes this error:

ERROR  : 	ime_interp() -> the interpolation time 20220110_030000 must fall between the input times: 20220115_000000 and 20220115_060000.
ERROR  

This shouldn't be resulting in an error for a valid time within the range of the climatology dataset. The issue is in MET's logic; it first checks for entries in the climo data file that have an hour value difference that's less than the hour_interval specified. Then it checks that the entries are within the day_interval specified. The successful climo data entries are then set up as "borders" (think bookends) for the valid time and the requested interpolation method is performed. However, in your situation it finds that the YYMMDD of each border from the climo file is identical, and the requested valid time that should be between them is outside. So it errors out.

We need to update MET's logic for when it finds borders that are both before or after the valid time, but within the cilmatology dataset's timeframe. I'll reply back when an issue is written up that captures the bug.

I did find a workaround of sorts, but it will most likely affect the values you receive from MET. Instead of passing only 1 climatology file, pass 2 or more so that MET has data for both "before" and "after" the valid time. This will get you past the error and should provide output. As mentioned however, these results may not be the statistical values you're after. Because when I put in a valid date of 20221218_030000 and provided the climatology files for 19591215 and 19590115 with a NEAREST interpolation method, I get the following messages concerning interpolation:

DEBUG 3: For forecast valid at 20221218_020000, found 4 climatology field(s) with valid time(s): 20230115_000000, 20230115_060000, 20221215_000000, 20221215_060000
DEBUG 3: Interpolating climatology fields at 20221215_000000 and 20230115_000000 to 20221218_000000 using the NEAREST interpolation method.
DEBUG 3: Interpolating climatology fields at 20221215_060000 and 20230115_060000 to 20221218_060000 using the NEAREST interpolation method.
DEBUG 3: Interpolating climatology fields at 20221218_000000 and 20221218_060000 to 20221218_020000 using the NEAREST interpolation method.

I would hope that these three interpolation steps result in:

• 20221215_000000 being used since abs(diff(20221215_00 - 20221218_02)) < abs(diff(20230115_00 - 20221218_02))
• 20221215_060000 being used since abs(diff(20221215_06 - 20221218_02)) < abs(diff(20230115_06 - 20221218_02))
• 20221215_000000 being used since abs(diff(20221215_00 - 20221218_02)) < abs(diff(20221215_06 - 20221218_02))

But I'm not 100% sure that's the case. The bugfix that addresses this will likely provide debug comments that make this logic more clear.

[craigr3]

Thanks for the response.  Until you have a fix, can I tell MET to ignore the error so that it would skip the forecast hour that doesn't match climo or skip just the climo calculation.  I will also try your work around.Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: j-opatz ***@***.***> Date: 2/3/23 5:39 PM (GMT-06:00) To: dtcenter/METplus ***@***.***> Cc: craigr3 ***@***.***>, Author ***@***.***> Subject: Re: [dtcenter/METplus] CLIMO data time_interval retrieval error (Discussion #2032) Hello, And thank you for following up on the previous discussion. My sincere apologies, I don't know how I was able to get past the issue you were previously describing in the last Discussions; setting up the exact same example this time, I encountered the issue you're describing. Requesting a standard hour in the valid time (0, 6, 12, and 18) provides a match, while any other hour in the valid time causes this error: ERROR : ime_interp() -> the interpolation time 20220110_030000 must fall between the input times: 20220115_000000 and 20220115_060000. ERROR This shouldn't be resulting in an error for a valid time within the range of the climatology dataset. The issue is in MET's logic; it first checks for entries in the climo data file that have an hour value difference that's less than the hour_interval specified. Then it checks that the entries are within the day_interval specified. The successful climo data entries are then set up as "borders" (think bookends) for the valid time and the requested interpolation method is performed. However, in your situation it finds that the YYMMDD of each border from the climo file is identical, and the requested valid time that should be between them is outside. So it errors out. We need to update MET's logic for when it finds borders that are both before or after the valid time, but within the cilmatology dataset's timeframe. I'll reply back when an issue is written up that captures the bug. I did find a workaround of sorts, but it will most likely affect the values you receive from MET. Instead of passing only 1 climatology file, pass 2 or more so that MET has data for both "before" and "after" the valid time. This will get you past the error and should provide output. As mentioned however, these results may not be the statistical values you're after. Because when I put in a valid date of 20221218_030000 and provided the climatology files for 19591215 and 19590115 with a NEAREST interpolation method, I get the following messages concerning interpolation: DEBUG 3: For forecast valid at 20221218_020000, found 4 climatology field(s) with valid time(s): 20230115_000000, 20230115_060000, 20221215_000000, 20221215_060000 DEBUG 3: Interpolating climatology fields at 20221215_000000 and 20230115_000000 to 20221218_000000 using the NEAREST interpolation method. DEBUG 3: Interpolating climatology fields at 20221215_060000 and 20230115_060000 to 20221218_060000 using the NEAREST interpolation method. DEBUG 3: Interpolating climatology fields at 20221218_000000 and 20221218_060000 to 20221218_020000 using the NEAREST interpolation method. I would hope that these three interpolation steps result in: 20221215_000000 being used since abs(diff(20221215_00 - 20221218_02)) < abs(diff(20230115_00 - 20221218_02)) 20221215_060000 being used since abs(diff(20221215_06 - 20221218_02)) < abs(diff(20230115_06 - 20221218_02)) 20221215_000000 being used since abs(diff(20221215_00 - 20221218_02)) < abs(diff(20221215_06 - 20221218_02)) But I'm not 100% sure that's the case. The bugfix that addresses this will likely provide debug comments that make this logic more clear. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[j-opatz]

As far as I know, there's no way to tell MET to skip a particular forecast hour. You can always skip the climatology calculation by not passing in any climatology data, but again, there's no way that I'm aware of to set a control on when MET utilizes climatology data so it can avoid an error out.

[craigr3]

The work around work.  I had one final question on the climo unit conversion issue.ThanksBobSent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: j-opatz ***@***.***> Date: 2/9/23 10:41 AM (GMT-06:00) To: dtcenter/METplus ***@***.***> Cc: craigr3 ***@***.***>, Author ***@***.***> Subject: Re: [dtcenter/METplus] CLIMO data time_interval retrieval error (Discussion #2032) As far as I know, there's no way to tell MET to skip a particular forecast hour. You can always skip the climatology calculation by not passing in any climatology data, but again, there's no way that I'm aware of to set a control on when MET utilizes climatology data so it can avoid an error out. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[j-opatz]

Hello,

I wanted to reach out again and mention that this issue has been resolved. It should be available in the latest bugfix. Take a look here for the specifics: dtcenter/MET#2439

I'm going to close this Discussion as solved. If you have any additional issues, don't hesitate to open a new Discussions topic.