Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Download Collection 1 datastructure with AWS en Google-storage #241

Open
wants to merge 16 commits into
base: develop
Choose a base branch
from

Conversation

jmellian
Copy link

Added download Collection 1 landsat 8 data from AWS after 1 may 2017:
Example use: pre-collection en collection 1 Landsat 8 data

landsat download LC81970232017085LGN00 --bands 432

landsat download LC08_L1TP_139045_20170304_20170316_01_T1 --bands 432

landsat download LC81970232017085LGN00

landsat download LC08_L1TP_139045_20170304_20170316_01_T1

added extra fields to dict result for landsat search
import field is the new 'product_id' in the Collection 1 datastructure

example use:

landsat search --limit 2000 --pathrow 199,23
Added to Dict:
result['results'] = [{'sceneID': i['sceneID'],
'sat_type': u'L8',
'path2': three_digit(i['path']),
'row2': three_digit(i['row']),
'download_links' : i['download_links'],
'BPF_NAME_OLI' : i['BPF_NAME_OLI'],
'thumbnail': i['browseURL'],
'date': i['acquisitionDate'],
'GROUND_CONTROL_POINTS_VERSION': i['GROUND_CONTROL_POINTS_VERSION'],
'DATE_L1_GENERATED': i['DATE_L1_GENERATED'],
'NADIR_OFFNADIR': i['NADIR_OFFNADIR'],
'data_geometry': i['data_geometry'],
'sunAzimuth': i['sunAzimuth'],
'cloudCover': i['cloudCover'],
'COLLECTION_NUMBER': i['COLLECTION_NUMBER'],
'sceneCenterLatitude': i['sceneCenterLatitude'],
'cartURL': i['cartURL'],
'sunElevation': i['sunElevation'],
'cloud_coverage': i['cloud_coverage'],
'CLOUD_COVER_LAND': i['CLOUD_COVER_LAND'],
'scene_id': i['scene_id'],
'GROUND_CONTROL_POINTS_MODEL': i['GROUND_CONTROL_POINTS_MODEL'],
'row': i['row'],
'imageQuality1': i['imageQuality1'],
'cloudCoverFull': i['cloudCoverFull'],
'aws_index': i['aws_index'],
'browseURL': i['browseURL'],
'browseAvailable': i['browseAvailable'],
'BPF_NAME_TIRS': i['BPF_NAME_TIRS'],
'dayOrNight': i['dayOrNight'],
'TIRS_SSM_MODEL': i['TIRS_SSM_MODEL'],
'CPF_NAME': i['CPF_NAME'],
'FULL_PARTIAL_SCENE': i['FULL_PARTIAL_SCENE'],
'DATA_TYPE_L1': i['DATA_TYPE_L1'],
'aws_thumbnail': i['aws_thumbnail'],
'google_index': i['google_index'],
'sceneStartTime': i['sceneStartTime'],
'dateUpdated': i['dateUpdated'],
'sensor': i['sensor'],
'lowerRightCornerLatitude': i['lowerRightCornerLatitude'],
'LANDSAT_PRODUCT_ID': i['LANDSAT_PRODUCT_ID'],
'acquisitionDate': i['acquisitionDate'],
'PROCESSING_SOFTWARE_VERSION': i['PROCESSING_SOFTWARE_VERSION'],
'lowerRightCornerLongitude': i['lowerRightCornerLongitude'],
'lowerLeftCornerLatitude': i['lowerLeftCornerLatitude'],
'sceneCenterLongitude': i['sceneCenterLongitude'],
'COLLECTION_CATEGORY': i['COLLECTION_CATEGORY'],
'upperLeftCornerLongitude': i['upperLeftCornerLongitude'],
'path': i['path'],
'lowerLeftCornerLongitude': i['lowerLeftCornerLongitude'],
'GEOMETRIC_RMSE_MODEL_X': i['GEOMETRIC_RMSE_MODEL_X'],
'GEOMETRIC_RMSE_MODEL_Y': i['GEOMETRIC_RMSE_MODEL_Y'],
'sceneStopTime': i['sceneStopTime'],
'upperLeftCornerLatitude': i['upperLeftCornerLatitude'],
'upperRightCornerLongitude': i['upperRightCornerLongitude'],
'product_id': i['product_id'],
'satellite_name': i['satellite_name'],
'GEOMETRIC_RMSE_MODEL': i['GEOMETRIC_RMSE_MODEL'],
'upperRightCornerLatitude': i['upperRightCornerLatitude'],
'receivingStation': i['receivingStation'],
'cloud': i['cloudCoverFull']} for i in r_dict['results']]

Added Landsat Collection 1 data structure download
Changed a Test to pass the download test, because of the changed files from google after 1 may 2017

j08lue and others added 13 commits March 21, 2017 13:11
Added download Collection 1 landsat 8 data from AWS after 1 may 2017:
Example use: pre-collection en collection 1 Landsat 8 data
- landsat download LC81970232017085LGN00 --bands 432
- landsat download LC08_L1TP_139045_20170304_20170316_01_T1 --bands 432

- landsat download LC81970232017085LGN00
- landsat download LC08_L1TP_139045_20170304_20170316_01_T1
added extra fields to dict result for landsat search
import field is the new 'product_id' in the Collection 1 datastructure

example use: 
- landsat search  --limit 2000 --pathrow 199,23

Added to Dict:
                result['results'] = [{'sceneID': i['sceneID'],
                                      'sat_type': u'L8',
                                      'path2': three_digit(i['path']),
                                      'row2': three_digit(i['row']),
                                      'download_links' : i['download_links'],
                                      'BPF_NAME_OLI' : i['BPF_NAME_OLI'],
                                      'thumbnail': i['browseURL'],
                                      'date': i['acquisitionDate'],
                                      'GROUND_CONTROL_POINTS_VERSION': i['GROUND_CONTROL_POINTS_VERSION'],
                                      'DATE_L1_GENERATED': i['DATE_L1_GENERATED'],
removed some forgotten print statements
delete some forgotten print statements
copied in wrong directory
…orage bucket

Add support for download Landsat Collection 1 data from new Google storage bucket
Only the *.TIF, _MTL.txt and _ANG..TXT are downloadable. Files *.TIF.ovr and _wrk.IMD are forbidden to download. On AWS these file are accessable.
Added a comment
Added test with  Collection 1 Data and change a test because of the changed datastructure
@jmellian jmellian closed this Aug 20, 2017
@jmellian jmellian reopened this Aug 20, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants