Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make preprocess_single work even if files are missing #9

Open
3 tasks
2320sharon opened this issue Feb 2, 2024 · 0 comments
Open
3 tasks

Make preprocess_single work even if files are missing #9

2320sharon opened this issue Feb 2, 2024 · 0 comments
Assignees
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed

Comments

@2320sharon
Copy link
Collaborator

If a file is missing in preprocess_single return None and just skip this file.

Tasks

  • Check if each file needed to run preprocess_single exists or not
  • If its missing return None
  • Maybe log why it failed - '{FILENAME} was missing from {EXPECTED_LOCATION}'

Relevant Code

# fn is a list of 3 file names typically is ['ms_filename','mask_filename','pan/swir filename']
# the snippet below is from the landsat 5 code section
        fn_ms = fn[0]
        fn_mask = fn[1]
       # HERE IS WHERE IT SHOULD BE CHECKED IF ALL FILES IN FN EXIST OR NOT
        # read ms bands
        data = gdal.Open(fn_ms, gdal.GA_ReadOnly)
        # if  fn_ms doesn't exist this code will return None
        georef = np.array(data.GetGeoTransform())

Exceptions Caused by this issue

File ~\anaconda3\envs\coastseg_transformers10\lib\site-packages\coastsat\SDS_shoreline.py:249, in extract_shorelines(metadata, settings)
    239 shoreline_date = os.path.basename(fn[0])[:19]
    241 # preprocess image (cloud mask + pansharpening/downsampling)
    242 (
    243     im_ms,
    244     georef,
    245     cloud_mask,
    246     im_extra,
    247     im_QA,
    248     im_nodata,
--> 249 ) = SDS_preprocess.preprocess_single(
    250     fn,
    251     satname,
    252     settings["cloud_mask_issue"],
    253     settings["pan_off"],
    254     collection,
    255     apply_cloud_mask,
    256 )
    257 # get image spatial reference system (epsg code) from metadata dict
    258 image_epsg = metadata[satname]["epsg"][i]

File ~\anaconda3\envs\coastseg_transformers10\lib\site-packages\coastsat\SDS_preprocess.py:462, in preprocess_single(fn, satname, cloud_mask_issue, pan_off, collection, do_cloud_mask)
    460 # read ms bands
    461 data = gdal.Open(fn_ms, gdal.GA_ReadOnly)
--> 462 georef = np.array(data.GetGeoTransform())
    463 bands = read_bands(fn_ms)
    464 im_ms = np.stack(bands, 2)

AttributeError: 'NoneType' object has no attribute 'GetGeoTransform'
@2320sharon 2320sharon added bug Something isn't working help wanted Extra attention is needed good first issue Good for newcomers labels Feb 2, 2024
@2320sharon 2320sharon self-assigned this Feb 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant