Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ticket: [PODAAC-6181]
Description
Add subType to a certain image file under RelatedUrls
This PR works together with cumulus-deploy-tf change which add TaskConfig.relatedUrlSubTypeMap to support OPERA
BROWSE IMAGE SOURCE subtye.
https://github.jpl.nasa.gov/podaac/cumulus-deploy-tf/pull/347
Overview of work done
The approach is to add the following structure under collection configuration's meta
"relatedUrlSubTypeMap": [
{
"regex": "^._B01_WTR\.tif$",
"subType": "BROWSE IMAGE SOURCE"
},
{
"regex": "^._B02_BWTR\.tif$",
"subType": "IceBridge Portal"
}
The code logic is to assign ummg.RelatedUrls.RelatedUrl.subtype value based on whatever input.payload.granules[0].files[] items
matching the expression.
Note for Code Reviewer.
** the field representing ummg version has to change : Ex : https://github.com/podaac/cumulus-metadata-aggregator/blob/feature/PODAAC-6181/src/test/resources/ummgResults/additionalAttributes/OPERA_L3_DSWx_HLS_T14RNV_20210906T170251Z_20221026T184342Z_L8_30_v0.0.json#L2-L5
** Since MetadataFilesToEcho.java will return "clean up json" due to this line was added in current branch : createJson()
all the pre-stored umm.json should have a make up by removing any item with empty array, or null value. So, storing the well fomatted json seems to be a better choice for future debugging.
Overview of verification done
Investigate the the UMMG through EarthdataClient search and found the image files matching above configured
subType regex all set to the correct subtype.
{
"identifier": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4",
"collection": "OPERA_L3_DSWX-S1_PROVISIONAL_V0",
"version": "1.6.1",
"submissionTime": "2024-05-02T00:19:53.862065Z",
"product": {
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4",
"files": [
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_BROWSE.tif",
"type": "browse",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_BROWSE.tif",
"size": 86231,
"checksum": "4aff3db9e4aeedddc8b4ca8201ac815b",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_BROWSE.png",
"type": "browse",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_BROWSE.png",
"size": 8003,
"checksum": "6211893f67440111f8fc7c5b822c63ab",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B01_WTR.tif",
"type": "data",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B01_WTR.tif",
"size": 91275,
"checksum": "5fc5b8a7e4afeaabbc2d9065dafdfd57",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B02_BWTR.tif",
"type": "data",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B02_BWTR.tif",
"size": 91283,
"checksum": "0cc7ecb1687e78f76d6c9a3e4069ae6c",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B03_CONF.tif",
"type": "data",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B03_CONF.tif",
"size": 181141,
"checksum": "4a14218136b52438ef40cf17753db643",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B04_DIAG.tif",
"type": "data",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B04_DIAG.tif",
"size": 3088750,
"checksum": "68e274d406084d0054f99ec265e2d3df",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4.iso.xml",
"type": "metadata",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4.iso.xml",
"size": 333642,
"checksum": "ad9f90aa56d12e64da11f4ccbfc0f309",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4.iso.xml.md5",
"type": "metadata",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4.iso.xml.md5",
"size": 32,
"checksum": "56014ccd9783565ccfe3eb22daecdeb7",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B01_WTR.tif.md5",
"type": "metadata",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B01_WTR.tif.md5",
"size": 32,
"checksum": "81abc30194d7c8b7f886fb9ca3c45350",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B02_BWTR.tif.md5",
"type": "metadata",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B02_BWTR.tif.md5",
"size": 32,
"checksum": "59693f63f14c4e17c966218f5fbacb14",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B03_CONF.tif.md5",
"type": "metadata",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B03_CONF.tif.md5",
"size": 32,
"checksum": "d7f363fa88ae16f90e7a4ac5c8a7ebee",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B04_DIAG.tif.md5",
"type": "metadata",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_B04_DIAG.tif.md5",
"size": 32,
"checksum": "1d3013e414bf2cb8f1afa299071738bb",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_BROWSE.png.md5",
"type": "metadata",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_BROWSE.png.md5",
"size": 32,
"checksum": "a86797cda1092900a07f697e5bc286d6",
"checksumType": "md5"
},
{
"name": "OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_BROWSE.tif.md5",
"type": "metadata",
"uri": "s3://podaac-dev-tst/OPERA_L3_DSWX-S1_PROVISIONAL_V0/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4/OPERA_L3_DSWx-S1_T35TPJ_20240214T155240Z_20240502T001754Z_S1A_30_v0.4_BROWSE.tif.md5",
"size": 32,
"checksum": "2e31b0fcdd21aafc4c3b636b2f989a8e",
"checksumType": "md5"
}
],
"dataVersion": "0.0"
},
"provider": "JPL-OPERA",
"trace": "opera-int-fwd"
}
Tested in SIT:
Explain how you tested in SIT, if applicable
PR checklist:
See Pull Request Review Checklist for pointers on reviewing this pull request