use scaling and offset props by default

[iferencik]

I am building a STAC catalog in Azure Blob. The data are nightly mosaics for nighttime lights from Colorado School of Mines and are converted to COGs before being inserted into the catalog. The original files are GeoTiffs rounded to 100th place and in average every COG occupies approx. 4 to 5 GB on disk. It occured to me that with some preprocessing (setting negative radiance to nodata, thresholding max radiance and using a scale factor ) the data type can be changed to 2 byte unsigned integer halving thus the output COG size (2GB and less).

titiler can display the COGs but ignores the scale factor and shows the data to be of int type not float as opposed to QGIS/GDAL. I think this might be a tricky one but in essence it is relatively easy to use the scale factor in the reader and I was wondering if there is a good/specific reason why this is not happening.

The way I imagine this is through a config setting, either inside the server or as an env variable, or both, something like UNSCALE=TRUE/FALSE

I am looking forward to hear your thoughts on this

[vincentsarago]

TiTiler endpoints have an unscale option

https://developmentseed.org/titiler/endpoints/cog/#tiles

unscale (bool): Apply dataset internal Scale/Offset.

[vincentsarago]

nice catch 🙏

[iferencik]

@vincentsarago do you know if the scale is available inside the algo call method?

https://github.com/UNDP-Data/geo-cogserver/blob/main/src/cogserver/algorithms/rca.py#L52

concretely, is there a way to get this info from ImageData object?

[vincentsarago]

concretely, is there a way to get this info from ImageData object?

Sadly we do not forward scale/offset to the ImageData. This could be added in https://github.com/cogeotiff/rio-tiler/blob/main/rio_tiler/reader.py#L272-L279

Feel free to start an issue/PR in rio-tiler 🙏

[iferencik]

@vincentsarago , i just check the same source and it was cleat the meta are tags.

rasterio reader has the profile attr and I feel this should contain the scale and offset in case they are set.

I would be in favor of adding the profile attr to the ImageData

 return ImageData(
            data,
            bounds=out_bounds,
            crs=dataset.crs,
            band_names=[f"b{idx}" for idx in indexes],
            dataset_statistics=dataset_statistics,
            metadata=dataset.tags(),
            profile=dataset.profile
        )

in this way the algo developer has access to several useful props (nodata, etc)
the use case here is that algos can do sophisticated things, in our case we are identifying the pixels that have chnaged between two sat images and are not clouds, so mask, nodata, and other params are useful.

[vincentsarago]

the problem is that ImageData are not just of single image, it might make sense for the output of rasterio Reader but for STAC or Mosaic it might be a bit more tricky

as I said, feel free to open an issue on rio-tiler and we can discuss more there