Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for grib idx & reinflate #528

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Add tests for grib idx & reinflate #528

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
669a362
Add tests for discussion
Nov 19, 2024
194da4d
Fix the tests
Dec 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ netcdf3 = ["scipy"]
dev = [
"cftime",
"dask",
"fastparquet",
"fastparquet>=2024.11.0",
"h5netcdf",
"h5py",
"jinja2",
Expand Down
696 changes: 696 additions & 0 deletions tests/grib_idx_fixtures/20221014/gfs.t00z.pgrb2.0p25.f000.idx
View file
Open in desktop

Large diffs are not rendered by default.

Binary file added tests/grib_idx_fixtures/20221014/gfs.t00z.pgrb2.0p25.f000.test-limit-10
View file
Open in desktop
Binary file not shown.
11 changes: 11 additions & 0 deletions tests/grib_idx_fixtures/20221014/gfs.t00z.pgrb2.0p25.f000.test-limit-10.idx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
1:0:d=2022101400:PRMSL:mean sea level:anl:
2:995942:d=2022101400:CLWMR:1 hybrid level:anl:
3:1082992:d=2022101400:ICMR:1 hybrid level:anl:
4:1363134:d=2022101400:RWMR:1 hybrid level:anl:
5:1597235:d=2022101400:SNMR:1 hybrid level:anl:
6:1697225:d=2022101400:GRLE:1 hybrid level:anl:
7:1745644:d=2022101400:REFD:1 hybrid level:anl:
8:2562710:d=2022101400:REFD:2 hybrid level:anl:
9:3380262:d=2022101400:REFC:entire atmosphere:anl:
10:4270093:d=2022101400:VIS:surface:anl:
11:4983458:d=2022101400:UGRD:planetary boundary layer:anl:
743 changes: 743 additions & 0 deletions tests/grib_idx_fixtures/20221014/gfs.t06z.pgrb2.0p25.f021.idx
View file
Open in desktop

Large diffs are not rendered by default.

Binary file added tests/grib_idx_fixtures/20221014/gfs.t06z.pgrb2.0p25.f021.test-limit-10
View file
Open in desktop
Binary file not shown.
11 changes: 11 additions & 0 deletions tests/grib_idx_fixtures/20221014/gfs.t06z.pgrb2.0p25.f021.test-limit-10.idx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
1:0:d=2022101406:PRMSL:mean sea level:21 hour fcst:
2:986627:d=2022101406:CLWMR:1 hybrid level:21 hour fcst:
3:1071190:d=2022101406:ICMR:1 hybrid level:21 hour fcst:
4:1334374:d=2022101406:RWMR:1 hybrid level:21 hour fcst:
5:1576714:d=2022101406:SNMR:1 hybrid level:21 hour fcst:
6:1669840:d=2022101406:GRLE:1 hybrid level:21 hour fcst:
7:1708612:d=2022101406:REFD:1 hybrid level:21 hour fcst:
8:2515402:d=2022101406:REFD:2 hybrid level:21 hour fcst:
9:3322351:d=2022101406:REFC:entire atmosphere:21 hour fcst:
10:4201577:d=2022101406:VIS:surface:21 hour fcst:
11:4895751:d=2022101406:UGRD:planetary boundary layer:21 hour fcst:
170 changes: 170 additions & 0 deletions tests/grib_idx_fixtures/20221014/hrrr.t01z.wrfsfcf00.grib2.idx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
1:0:d=2022101401:REFC:entire atmosphere:anl:
2:464700:d=2022101401:RETOP:cloud top:anl:
3:797270:d=2022101401:var discipline=0 center=7 local_table=1 parmcat=16 parm=201:entire atmosphere:anl:
4:1279418:d=2022101401:VIL:entire atmosphere:anl:
5:1629309:d=2022101401:VIS:surface:anl:
6:3031911:d=2022101401:REFD:1000 m above ground:anl:
7:3324038:d=2022101401:REFD:4000 m above ground:anl:
8:3573815:d=2022101401:REFD:263 K level:anl:
9:3860080:d=2022101401:GUST:surface:anl:
10:5058926:d=2022101401:UGRD:250 mb:anl:
11:5853053:d=2022101401:VGRD:250 mb:anl:
12:6618262:d=2022101401:UGRD:300 mb:anl:
13:7400136:d=2022101401:VGRD:300 mb:anl:
14:8151598:d=2022101401:HGT:500 mb:anl:
15:8885831:d=2022101401:TMP:500 mb:anl:
16:9432910:d=2022101401:DPT:500 mb:anl:
17:10403024:d=2022101401:UGRD:500 mb:anl:
18:10991346:d=2022101401:VGRD:500 mb:anl:
19:11587686:d=2022101401:HGT:700 mb:anl:
20:12305017:d=2022101401:TMP:700 mb:anl:
21:12859067:d=2022101401:DPT:700 mb:anl:
22:13913033:d=2022101401:DZDT:700 mb:anl:
23:14225291:d=2022101401:UGRD:700 mb:anl:
24:14822752:d=2022101401:VGRD:700 mb:anl:
25:15443030:d=2022101401:HGT:850 mb:anl:
26:16150754:d=2022101401:TMP:850 mb:anl:
27:16726305:d=2022101401:DPT:850 mb:anl:
28:17848040:d=2022101401:UGRD:850 mb:anl:
29:18458023:d=2022101401:VGRD:850 mb:anl:
30:19089986:d=2022101401:TMP:925 mb:anl:
31:19688326:d=2022101401:DPT:925 mb:anl:
32:20828612:d=2022101401:UGRD:925 mb:anl:
33:21458121:d=2022101401:VGRD:925 mb:anl:
34:22094352:d=2022101401:TMP:1000 mb:anl:
35:22701465:d=2022101401:DPT:1000 mb:anl:
36:23829721:d=2022101401:UGRD:1000 mb:anl:
37:24453401:d=2022101401:VGRD:1000 mb:anl:
38:25076605:d=2022101401:MAXUVV:100-1000 mb above ground:0-0 day max fcst:
39:25211183:d=2022101401:MAXDVV:100-1000 mb above ground:0-0 day max fcst:
40:25421473:d=2022101401:DZDT:0.5-0.8 sigma layer:0-0 day ave fcst:
41:25589854:d=2022101401:MSLMA:mean sea level:anl:
42:26196823:d=2022101401:HGT:1000 mb:anl:
43:26876848:d=2022101401:MAXREF:1000 m above ground:0-0 day max fcst:
44:27095445:d=2022101401:REFD:263 K level:0-0 day max fcst:
45:27373508:d=2022101401:MXUPHL:5000-2000 m above ground:0-0 day max fcst:
46:27373720:d=2022101401:MNUPHL:5000-2000 m above ground:0-0 day min fcst:
47:27373932:d=2022101401:MXUPHL:2000-0 m above ground:0-0 day max fcst:
48:27374144:d=2022101401:MNUPHL:2000-0 m above ground:0-0 day min fcst:
49:27374356:d=2022101401:MXUPHL:3000-0 m above ground:0-0 day max fcst:
50:27374568:d=2022101401:MNUPHL:3000-0 m above ground:0-0 day min fcst:
51:27374780:d=2022101401:RELV:2000-0 m above ground:0-0 day max fcst:
52:27374992:d=2022101401:RELV:1000-0 m above ground:0-0 day max fcst:
53:27375204:d=2022101401:HAIL:entire atmosphere:0-0 day max fcst:
54:27614124:d=2022101401:HAIL:0.1 sigma level:0-0 day max fcst:
55:27624989:d=2022101401:HAIL:surface:0-0 day max fcst:
56:27625201:d=2022101401:TCOLG:entire atmosphere (considered as a single layer):0-0 day max fcst:
57:27652136:d=2022101401:LTNGSD:1 m above ground:anl:
58:27652324:d=2022101401:LTNGSD:2 m above ground:anl:
59:27737182:d=2022101401:LTNG:entire atmosphere:anl:
60:27737370:d=2022101401:UGRD:80 m above ground:anl:
61:28866120:d=2022101401:VGRD:80 m above ground:anl:
62:29990913:d=2022101401:PRES:surface:anl:
63:31500434:d=2022101401:HGT:surface:anl:
64:33654129:d=2022101401:TMP:surface:anl:
65:34878729:d=2022101401:ASNOW:surface:0-0 day acc fcst:
66:34878941:d=2022101401:MSTAV:0 m underground:anl:
67:36223738:d=2022101401:CNWAT:surface:anl:
68:36301236:d=2022101401:WEASD:surface:anl:
69:36325558:d=2022101401:SNOWC:surface:anl:
70:36345003:d=2022101401:SNOD:surface:anl:
71:36362081:d=2022101401:TMP:2 m above ground:anl:
72:37527637:d=2022101401:POT:2 m above ground:anl:
73:38635438:d=2022101401:SPFH:2 m above ground:anl:
74:40130169:d=2022101401:DPT:2 m above ground:anl:
75:41313369:d=2022101401:RH:2 m above ground:anl:
76:42828342:d=2022101401:MASSDEN:8 m above ground:anl:
77:43192837:d=2022101401:UGRD:10 m above ground:anl:
78:45574452:d=2022101401:VGRD:10 m above ground:anl:
79:47956067:d=2022101401:WIND:10 m above ground:0-0 day max fcst:
80:49153909:d=2022101401:MAXUW:10 m above ground:0-0 day max fcst:
81:50345853:d=2022101401:MAXVW:10 m above ground:0-0 day max fcst:
82:51496752:d=2022101401:CPOFP:surface:anl:
83:51496940:d=2022101401:PRATE:surface:anl:
84:51497128:d=2022101401:APCP:surface:0-0 day acc fcst:
85:51497340:d=2022101401:WEASD:surface:0-0 day acc fcst:
86:51497552:d=2022101401:FROZR:surface:0-0 day acc fcst:
87:51497764:d=2022101401:FRZR:surface:0-0 day acc fcst:
88:51499493:d=2022101401:SSRUN:surface:0-0 day acc fcst:
89:51499705:d=2022101401:BGRUN:surface:0-0 day acc fcst:
90:51499917:d=2022101401:CSNOW:surface:anl:
91:51500105:d=2022101401:CICEP:surface:anl:
92:51500293:d=2022101401:CFRZR:surface:anl:
93:51500481:d=2022101401:CRAIN:surface:anl:
94:51500669:d=2022101401:SFCR:surface:anl:
95:53406318:d=2022101401:FRICV:surface:anl:
96:54420561:d=2022101401:SHTFL:surface:anl:
97:55543177:d=2022101401:LHTFL:surface:anl:
98:56625664:d=2022101401:VEG:surface:anl:
99:58071925:d=2022101401:var discipline=2 center=7 local_table=1 parmcat=0 parm=231:surface:anl:
100:59188062:d=2022101401:var discipline=2 center=7 local_table=1 parmcat=0 parm=232:surface:anl:
101:60064996:d=2022101401:LAI:surface:anl:
102:60839373:d=2022101401:GFLUX:surface:anl:
103:61414201:d=2022101401:VGTYP:surface:anl:
104:62195380:d=2022101401:LFTX:500-1000 mb:anl:
105:63121072:d=2022101401:CAPE:surface:anl:
106:63613994:d=2022101401:CIN:surface:anl:
107:63876006:d=2022101401:PWAT:entire atmosphere (considered as a single layer):anl:
108:64783912:d=2022101401:AOTK:entire atmosphere (considered as a single layer):anl:
109:64784100:d=2022101401:COLMD:entire atmosphere (considered as a single layer):anl:
110:65276443:d=2022101401:TCOLW:entire atmosphere:anl:
111:66249348:d=2022101401:TCOLI:entire atmosphere:anl:
112:67109731:d=2022101401:TCDC:boundary layer cloud layer:anl:
113:67672649:d=2022101401:LCDC:low cloud layer:anl:
114:68299254:d=2022101401:MCDC:middle cloud layer:anl:
115:68635791:d=2022101401:HCDC:high cloud layer:anl:
116:68917612:d=2022101401:TCDC:entire atmosphere:anl:
117:69566611:d=2022101401:HGT:cloud ceiling:anl:
118:70595777:d=2022101401:HGT:cloud base:anl:
119:72458332:d=2022101401:PRES:cloud base:anl:
120:73401628:d=2022101401:PRES:cloud top:anl:
121:74037702:d=2022101401:HGT:cloud top:anl:
122:74944390:d=2022101401:ULWRF:top of atmosphere:anl:
123:76799204:d=2022101401:DSWRF:surface:anl:
124:77176562:d=2022101401:DLWRF:surface:anl:
125:79101732:d=2022101401:USWRF:surface:anl:
126:79345954:d=2022101401:ULWRF:surface:anl:
127:80946255:d=2022101401:CFNSF:surface:anl:
128:80949779:d=2022101401:VBDSF:surface:anl:
129:81330793:d=2022101401:VDDSF:surface:anl:
130:81829061:d=2022101401:USWRF:top of atmosphere:anl:
131:82183499:d=2022101401:HLCY:3000-0 m above ground:anl:
132:83324440:d=2022101401:HLCY:1000-0 m above ground:anl:
133:85189359:d=2022101401:USTM:0-6000 m above ground:anl:
134:86167889:d=2022101401:VSTM:0-6000 m above ground:anl:
135:87147965:d=2022101401:VUCSH:0-1000 m above ground:anl:
136:89529580:d=2022101401:VVCSH:0-1000 m above ground:anl:
137:91911195:d=2022101401:VUCSH:0-6000 m above ground:anl:
138:94530952:d=2022101401:VVCSH:0-6000 m above ground:anl:
139:97150709:d=2022101401:HGT:0C isotherm:anl:
140:99073414:d=2022101401:RH:0C isotherm:anl:
141:99758884:d=2022101401:PRES:0C isotherm:anl:
142:100470470:d=2022101401:HGT:highest tropospheric freezing level:anl:
143:101211423:d=2022101401:RH:highest tropospheric freezing level:anl:
144:101893927:d=2022101401:PRES:highest tropospheric freezing level:anl:
145:102600075:d=2022101401:HGT:263 K level:anl:
146:103272054:d=2022101401:HGT:253 K level:anl:
147:103895875:d=2022101401:4LFTX:180-0 mb above ground:anl:
148:104812154:d=2022101401:CAPE:180-0 mb above ground:anl:
149:105302746:d=2022101401:CIN:180-0 mb above ground:anl:
150:105541725:d=2022101401:HPBL:surface:anl:
151:108514309:d=2022101401:HGT:level of adiabatic condensation from sfc:anl:
152:111393934:d=2022101401:CAPE:90-0 mb above ground:anl:
153:111768089:d=2022101401:CIN:90-0 mb above ground:anl:
154:112001483:d=2022101401:CAPE:255-0 mb above ground:anl:
155:112515612:d=2022101401:CIN:255-0 mb above ground:anl:
156:112771084:d=2022101401:HGT:equilibrium level:anl:
157:115075666:d=2022101401:PLPL:255-0 mb above ground:anl:
158:116228736:d=2022101401:CAPE:0-3000 m above ground:anl:
159:116985258:d=2022101401:HGT:level of free convection:anl:
160:119703278:d=2022101401:var discipline=0 center=7 local_table=1 parmcat=7 parm=204:surface:anl:
161:120520311:d=2022101401:var discipline=0 center=7 local_table=1 parmcat=7 parm=206:0-500 m above ground:anl:
162:122621569:d=2022101401:LAYTH:261 K level - 256 K level:anl:
163:123921507:d=2022101401:var discipline=0 center=7 local_table=1 parmcat=7 parm=205:0-3000 m above ground:anl:
164:124663817:d=2022101401:RHPW:entire atmosphere:anl:
165:125792368:d=2022101401:LAND:surface:anl:
166:125842844:d=2022101401:ICEC:surface:anl:
167:125843077:d=2022101401:SBT123:top of atmosphere:anl:
168:127436698:d=2022101401:SBT124:top of atmosphere:anl:
169:129136789:d=2022101401:SBT113:top of atmosphere:anl:
170:130576022:d=2022101401:SBT114:top of atmosphere:anl:
Binary file added tests/grib_idx_fixtures/20221014/hrrr.t01z.wrfsfcf00.grib2.test-limit-10
View file
Open in desktop
Binary file not shown.
11 changes: 11 additions & 0 deletions tests/grib_idx_fixtures/20221014/hrrr.t01z.wrfsfcf00.grib2.test-limit-10.idx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
1:0:d=2022101401:REFC:entire atmosphere:anl:
2:464700:d=2022101401:RETOP:cloud top:anl:
3:797270:d=2022101401:var discipline=0 center=7 local_table=1 parmcat=16 parm=201:entire atmosphere:anl:
4:1279418:d=2022101401:VIL:entire atmosphere:anl:
5:1629309:d=2022101401:VIS:surface:anl:
6:3031911:d=2022101401:REFD:1000 m above ground:anl:
7:3324038:d=2022101401:REFD:4000 m above ground:anl:
8:3573815:d=2022101401:REFD:263 K level:anl:
9:3860080:d=2022101401:GUST:surface:anl:
10:5058926:d=2022101401:UGRD:250 mb:anl:
11:5853053:d=2022101401:VGRD:250 mb:anl:
49 changes: 49 additions & 0 deletions tests/grib_idx_fixtures/20221014/hrrr.t01z.wrfsubhf00.grib2.idx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
1:0:d=2022101401:REFC:entire atmosphere:anl:
2:464700:d=2022101401:RETOP:cloud top:anl:
3:797270:d=2022101401:var discipline=0 center=7 local_table=1 parmcat=16 parm=201:entire atmosphere:anl:
4:1279418:d=2022101401:VIL:entire atmosphere:anl:
5:1629309:d=2022101401:VIS:surface:anl:
6:3031911:d=2022101401:REFD:1000 m above ground:anl:
7:3324038:d=2022101401:REFD:4000 m above ground:anl:
8:3573815:d=2022101401:GUST:surface:anl:
9:4772661:d=2022101401:UPHL:5000-2000 m above ground:anl:
10:4842120:d=2022101401:UGRD:80 m above ground:anl:
11:5970870:d=2022101401:VGRD:80 m above ground:anl:
12:7095663:d=2022101401:PRES:surface:anl:
13:8605184:d=2022101401:HGT:surface:anl:
14:10758879:d=2022101401:TMP:2 m above ground:anl:
15:11924435:d=2022101401:SPFH:2 m above ground:anl:
16:13419166:d=2022101401:DPT:2 m above ground:anl:
17:14602366:d=2022101401:UGRD:10 m above ground:anl:
18:16983981:d=2022101401:VGRD:10 m above ground:anl:
19:19365596:d=2022101401:WIND:10 m above ground:0-0 day ave fcst:
20:21181306:d=2022101401:UGRD:10 m above ground:0-0 day ave fcst:
21:22994362:d=2022101401:VGRD:10 m above ground:0-0 day ave fcst:
22:24756729:d=2022101401:DSWRF:surface:0-0 day ave fcst:
23:24996008:d=2022101401:VBDSF:surface:0-0 day ave fcst:
24:25267157:d=2022101401:CPOFP:surface:anl:
25:25267345:d=2022101401:PRATE:surface:anl:
26:25267533:d=2022101401:APCP:surface:0-0 day acc fcst:
27:25267745:d=2022101401:WEASD:surface:0-0 day acc fcst:
28:25267957:d=2022101401:FROZR:surface:0-0 day acc fcst:
29:25268169:d=2022101401:CSNOW:surface:anl:
30:25268357:d=2022101401:CICEP:surface:anl:
31:25268545:d=2022101401:CFRZR:surface:anl:
32:25268733:d=2022101401:CRAIN:surface:anl:
33:25268921:d=2022101401:TCOLW:entire atmosphere:anl:
34:26241826:d=2022101401:TCOLI:entire atmosphere:anl:
35:27102209:d=2022101401:HGT:cloud ceiling:anl:
36:28131375:d=2022101401:HGT:cloud base:anl:
37:29993930:d=2022101401:HGT:cloud top:anl:
38:30900618:d=2022101401:ULWRF:top of atmosphere:anl:
39:32755432:d=2022101401:DSWRF:surface:anl:
40:33132790:d=2022101401:DLWRF:surface:anl:
41:35057960:d=2022101401:USWRF:surface:anl:
42:35302182:d=2022101401:ULWRF:surface:anl:
43:36902483:d=2022101401:VBDSF:surface:anl:
44:37283497:d=2022101401:VDDSF:surface:anl:
45:37781765:d=2022101401:USWRF:top of atmosphere:anl:
46:38136203:d=2022101401:SBT123:top of atmosphere:anl:
47:39729824:d=2022101401:SBT124:top of atmosphere:anl:
48:41429915:d=2022101401:SBT113:top of atmosphere:anl:
49:42869148:d=2022101401:SBT114:top of atmosphere:anl:
Binary file added tests/grib_idx_fixtures/20221014/hrrr.t01z.wrfsubhf00.grib2.test-limit-10
View file
Open in desktop
Binary file not shown.
11 changes: 11 additions & 0 deletions tests/grib_idx_fixtures/20221014/hrrr.t01z.wrfsubhf00.grib2.test-limit-10.idx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
1:0:d=2022101401:REFC:entire atmosphere:anl:
2:464700:d=2022101401:RETOP:cloud top:anl:
3:797270:d=2022101401:var discipline=0 center=7 local_table=1 parmcat=16 parm=201:entire atmosphere:anl:
4:1279418:d=2022101401:VIL:entire atmosphere:anl:
5:1629309:d=2022101401:VIS:surface:anl:
6:3031911:d=2022101401:REFD:1000 m above ground:anl:
7:3324038:d=2022101401:REFD:4000 m above ground:anl:
8:3573815:d=2022101401:GUST:surface:anl:
9:4772661:d=2022101401:UPHL:5000-2000 m above ground:anl:
10:4842120:d=2022101401:UGRD:80 m above ground:anl:
11:5970870:d=2022101401:VGRD:80 m above ground:anl:
Loading
Loading