User received an error message when no data downloaded but should have got an empty file

[kereid]

There was a failed download yesterday afternoon https://processes.aodn.org.au/wps/jobStatus?jobId=14cdb0fb-0842-45ea-b9bb-4a84373d25f3&format=HTML and after I investigated I can see that it was due to the user drawing a bounding box on the landmass of Australia. So that is justified there is no data and that’s what the error said.

The collection was https://portal.aodn.org.au/search?uuid=d4f0c06d-0dd3-42ed-a0f1-7504e419e59a and the bounding box N: -32.64, E: 136.54 S: -32.89 W: 136.17 as this possibly won't be investigated until next iteration which is 3 weeks away.

However, the user probably should have received a subset that included no data (fill values only)

[jonescc]

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ns2:Execute service="WPS" version="1.0.0" xmlns:ns2="http://www.opengis.net/wps/1.0.0" xmlns:ns1="http://www.opengis.net/ows/1.1" xmlns:ns3="http://www.w3.org/1999/xlink">
    <ns1:Identifier>gs:GoGoDuck</ns1:Identifier>
    <ns2:DataInputs>
        <ns2:Input>
            <ns1:Identifier>filename</ns1:Identifier>
            <ns2:Data>
                <ns2:LiteralData>IMOS_aggregation_20190513T065824Z</ns2:LiteralData>
            </ns2:Data>
        </ns2:Input>
        <ns2:Input>
            <ns1:Identifier>aggregationOutputMime</ns1:Identifier>
            <ns2:Data>
                <ns2:LiteralData>application/x-netcdf</ns2:LiteralData>
            </ns2:Data>
        </ns2:Input>
        <ns2:Input>
            <ns1:Identifier>layer</ns1:Identifier>
            <ns2:Data>
                <ns2:LiteralData>csiro_cars_weekly_url</ns2:LiteralData>
            </ns2:Data>
        </ns2:Input>
        <ns2:Input>
            <ns1:Identifier>subset</ns1:Identifier>
            <ns2:Data>
                <ns2:LiteralData>TIME,2009-01-01T00:00:00.000Z,2009-12-26T00:00:00.000Z;LATITUDE,-32.892669677734,-32.645477294922;LONGITUDE,136.17965698242,136.5641784668;DEPTH,0,5000</ns2:LiteralData>
            </ns2:Data>
        </ns2:Input>
        <ns2:Input>
            <ns1:Identifier>callbackUrl</ns1:Identifier>
            <ns2:Data>
                <ns2:LiteralData>
                    https://portal.aodn.org.au/wps/jobComplete
                </ns2:LiteralData>
            </ns2:Data>
        </ns2:Input>
        <ns2:Input>
            <ns1:Identifier>callbackParams</ns1:Identifier>
            <ns2:Data>
                <ns2:LiteralData>[email protected]</ns2:LiteralData>
            </ns2:Data>
        </ns2:Input>
    </ns2:DataInputs>
    <ns2:ResponseForm>
        <ns2:ResponseDocument storeExecuteResponse="true" status="true">
            <ns2:Output asReference="true" mimeType="application/zip">
                <ns1:Identifier>result</ns1:Identifier>
            </ns2:Output>
        </ns2:ResponseDocument>
    </ns2:ResponseForm>
</ns2:Execute>

[jonescc]

Looks like the aggregator couldn't find lat/long coords in the dataset that match, but looking at the dataset they look fine.

Needs debugging. Is this a VIT?

2019-05-13 07:02:20 INFO [main] : Commencing download of [52] files.
2019-05-13 07:02:21 INFO [pool-2-thread-1] au.org.emii.download.ParallelDownloadManager : Downloaded http://data.aodn.org.au/CSIRO/Climatology/CARS/2009/AODN-product/CARS2009_Australia_weekly.nc
2019-05-13 07:02:21 ERROR [main] au.org.emii.aggregator.NetcdfAggregator : Error adding /mnt/work/14cdb0fb-0842-45ea-b9bb-4a84373d25f33141799749786498402/downloads/4903bcc0-dd25-430e-af6c-047c3639ce7eCARS2009_Australia_weekly.nc to output file.
au.org.emii.aggregator.exception.AggregationException: Bounding box selected no data
at au.org.emii.aggregator.coordsystem.LatLon1D.subsetGrid(LatLon1D.java:54) ~[aggregator-4.39.3.jar:?]
at au.org.emii.aggregator.coordsystem.LatLonCoords.subset(LatLonCoords.java:38) ~[aggregator-4.39.3.jar:?]
at au.org.emii.aggregator.dataset.SubsettedDataset.<init>(SubsettedDataset.java:76) ~[aggregator-4.39.3.jar:?]
at au.org.emii.aggregator.dataset.AbstractNetcdfDataset.subset(AbstractNetcdfDataset.java:75) ~[aggregator-4.39.3.jar:?]
at au.org.emii.aggregator.NetcdfAggregator.add(NetcdfAggregator.java:99) [aggregator-4.39.3.jar:?]

[jonescc]

OK, so the selection did not select any grid points in the dataset (they selected an area with no recorded measurements even though its within the spatial extent of the dataset - too small a subset).

I think this has been discussed in the past - can't remember the details though - it didn't result in any changes. If we go to the next greatest grid point then we include data that wasn't actually requested.

[jonescc]

Requested latitude subset -32.892669677734,-32.645477294922 is between -32.5 and -33 which are the two nearest latitudes as shown by ncdump below:

 LATITUDE = -50, -49.5, -49, -48.5, -48, -47.5, -47, -46.5, -46, -45.5, -45, 
    -44.5, -44, -43.5, -43, -42.5, -42, -41.5, -41, -40.5, -40, -39.5, -39, 
    -38.5, -38, -37.5, -37, -36.5, -36, -35.5, -35, -34.5, -34, -33.5, -33, 
    -32.5, -32, -31.5, -31, -30.5, -30, -29.5, -29, -28.5, -28, -27.5, -27, 
    -26.5, -26, -25.5, -25, -24.5, -24, -23.5, -23, -22.5, -22, -21.5, -21, 
    -20.5, -20, -19.5, -19, -18.5, -18, -17.5, -17, -16.5, -16, -15.5, -15, 
    -14.5, -14, -13.5, -13, -12.5, -12, -11.5, -11, -10.5, -10, -9.5, -9, 
    -8.5, -8, -7.5, -7, -6.5, -6, -5.5, -5, -4.5, -4 ;

 LONGITUDE = 104, 104.5, 105, 105.5, 106, 106.5, 107, 107.5, 108, 108.5, 109, 
    109.5, 110, 110.5, 111, 111.5, 112, 112.5, 113, 113.5, 114, 114.5, 115, 
    115.5, 116, 116.5, 117, 117.5, 118, 118.5, 119, 119.5, 120, 120.5, 121, 
    121.5, 122, 122.5, 123, 123.5, 124, 124.5, 125, 125.5, 126, 126.5, 127, 
    127.5, 128, 128.5, 129, 129.5, 130, 130.5, 131, 131.5, 132, 132.5, 133, 
    133.5, 134, 134.5, 135, 135.5, 136, 136.5, 137, 137.5, 138, 138.5, 139, 
    139.5, 140, 140.5, 141, 141.5, 142, 142.5, 143, 143.5, 144, 144.5, 145, 
    145.5, 146, 146.5, 147, 147.5, 148, 148.5, 149, 149.5, 150, 150.5, 151, 
    151.5, 152, 152.5, 153, 153.5, 154, 154.5, 155, 155.5, 156, 156.5, 157, 
    157.5, 158, 158.5, 159, 159.5, 160, 160.5, 161 ;

[ggalibert]

the user probably should have received a subset that included no data (fill values only)

This is actually difficult to achieve when the dataset itself doesn't have fill values over the requested subset. I think an error message with enough details (could even mention nearest Lat/Lon points available on grid) would be sufficient.

[jonescc]

So looks like the error is correct, but there is probably room for improvement here as the user doesn't know what the grid is. A possible enhancement to be discussed with PO's regarding what would make sense here.

[jonescc]

Thanks @ggalibert