[1pt] Error in evaluation if a benchmark folder is missing

[mluck]

An error occurs in run_test_cases.py if any of the five benchmark (BLE, NWS, USGS, IFC, or ras2fim) folders are missing. This happens if, for example, FIM validation data are downloaded from ESIP S3, which lacks IFC validation data so is likely to affect external users.

aws s3 ls s3://noaa-nws-owp-fim/hand_fim/test_cases/ --profile esip
                           PRE ble_test_cases/
                           PRE nws_test_cases/
                           PRE ras2fim_test_cases/
                           PRE usgs_test_cases/

Current behavior

During synthesize_test_cases.py, an error occurs in run_test_cases.py if a benchmark folder is missing. For example:

Traceback (most recent call last):
  File "/foss_fim/tools/synthesize_test_cases.py", line 540, in <module>
    all_test_cases = Test_Case.list_all_test_cases(
  File "/foss_fim/tools/run_test_case.py", line 178, in list_all_test_cases
    benchmark_data = benchmark_class.huc_data()
  File "/foss_fim/tools/run_test_case.py", line 60, in huc_data
    for huc in os.listdir(self.validation_data):
FileNotFoundError: [Errno 2] No such file or directory: '/data/test_cases/ifc_test_cases/validation_data_ifc'

Additionally, a separate error occurs in synthesize_test_cases.py if an output metrics file is specfied with the -m option.

Traceback (most recent call last):
  File "/foss_fim/tools/synthesize_test_cases.py", line 685, in <module>
    create_master_metrics_csv(
  File "/foss_fim/tools/synthesize_test_cases.py", line 118, in create_master_metrics_csv
    test_cases_list = [d for d in os.listdir(benchmark_test_case_dir) if re.match(r'\d{8}_\w{3,7}', d)]
FileNotFoundError: [Errno 2] No such file or directory: '/data/test_cases/ifc_test_cases'

Steps to replicate behavior (include URLs)

1. Run tools/synthesize_test_cases.py without one of the benchmark data folders.