Standard names: Point clouds, DEMs and DTMs

[lhmarsden]

I am working on a data flow for storing data in point clouds in CF-NetCDF files. A point cloud is a collection of data points in a three-dimensional space, typically representing the surface of an object or environment, where each point contains spatial coordinates and may include additional attributes such as colour, normals, or intensity. The data I am working on are mostly focused on natural features (valleys, glaciers etc). However, a point clouds can be used to map the surface of man-made objects too. A point cloud could equally include both - a valley may include buildings for example.

I think CF-NetCDF is a suitable format for this type of data; existing commonly-used formats are not FAIR compliant, as far as I am aware. DEMs and DTMs could be stored in the same way, with the featureType being point.

Here is how I envisage the file might look, with a few data variables added as an example:

Example

dimensions:
        point = 9968400 ;
        band = 186 ;
variables:
        float point(point) ;
                point:units = "1" ;
                point:long_name = "Arbitrary counter for number of points in the point cloud" ;
                point:standard_name = "number_of_observations" ;
                point:coverage_content_type = "coordinate" ;
        float band(band) ;
                band:units = "nanometers" ;
                band:long_name = "Spectral band" ;
                band:standard_name = "radiation_wavelength" ;
                band:coverage_content_type = "coordinate" ;
        int crs ;
                crs:crs_wkt = "PROJCRS[\"unknown\",BASEGEOGCRS[\"unknown\",DATUM[\"World Geodetic System 1984\",ELLIPSOID[\"WGS 84\",6378137,298.257223563,LENGTHUNIT[\"metre\",1]],ID[\"EPSG\",6326]],PRIMEM[\"Greenwich\",0,ANGLEUNIT[\"degree\",0.0174532925199433],ID[\"EPSG\",8901]]],CONVERSION[\"UTM zone 33N\",METHOD[\"Transverse Mercator\",ID[\"EPSG\",9807]],PARAMETER[\"Latitude of natural origin\",0,ANGLEUNIT[\"degree\",0.0174532925199433],ID[\"EPSG\",8801]],PARAMETER[\"Longitude of natural origin\",15,ANGLEUNIT[\"degree\",0.0174532925199433],ID[\"EPSG\",8802]],PARAMETER[\"Scale factor at natural origin\",0.9996,SCALEUNIT[\"unity\",1],ID[\"EPSG\",8805]],PARAMETER[\"False easting\",500000,LENGTHUNIT[\"metre\",1],ID[\"EPSG\",8806]],PARAMETER[\"False northing\",0,LENGTHUNIT[\"metre\",1],ID[\"EPSG\",8807]],ID[\"EPSG\",16033]],CS[Cartesian,2],AXIS[\"(E)\",east,ORDER[1],LENGTHUNIT[\"metre\",1,ID[\"EPSG\",9001]]],AXIS[\"(N)\",north,ORDER[2],LENGTHUNIT[\"metre\",1,ID[\"EPSG\",9001]]]]" ;
                crs:semi_major_axis = 6378137. ;
                crs:semi_minor_axis = 6356752.31424518 ;
                crs:inverse_flattening = 298.257223563 ;
                crs:reference_ellipsoid_name = "WGS 84" ;
                crs:longitude_of_prime_meridian = 0. ;
                crs:prime_meridian_name = "Greenwich" ;
                crs:geographic_crs_name = "unknown" ;
                crs:horizontal_datum_name = "World Geodetic System 1984" ;
                crs:projected_crs_name = "unknown" ;
                crs:grid_mapping_name = "transverse_mercator" ;
                crs:latitude_of_projection_origin = 0. ;
                crs:longitude_of_central_meridian = 15. ;
                crs:false_easting = 500000. ;
                crs:false_northing = 0. ;
                crs:scale_factor_at_central_meridian = 0.9996 ;
        float longitude(point) ;
                longitude:units = "degrees_east" ;
                longitude:long_name = "longitude" ;
                longitude:standard_name = "longitude" ;
                longitude:axis = "X" ;
                longitude:valid_range = -180., 180. ;
                longitude:coverage_content_type = "coordinate" ;
        float latitude(point) ;
                latitude:units = "degrees_north" ;
                latitude:long_name = "latitude" ;
                latitude:standard_name = "latitude" ;
                latitude:axis = "Y" ;
                latitude:valid_range = -90., 90. ;
                latitude:coverage_content_type = "coordinate" ;
        float X(point) ;
                X:units = "meters" ;
                X:long_name = "X coordinate" ;
                X:standard_name = "projection_x_coordinate" ;
                X:grid_mapping = "crs" ;
                X:coverage_content_type = "coordinate" ;
        float Y(point) ;
                Y:units = "meters" ;
                Y:long_name = "Y coordinate" ;
                Y:standard_name = "projection_y_coordinate" ;
                Y:grid_mapping = "crs" ;
                Y:coverage_content_type = "coordinate" ;
        float Z(point) ;
                Z:units = "meters" ;
                Z:long_name = "Z coordinate" ;
                Z:standard_name = "height" ;
                Z:grid_mapping = "crs" ;
                Z:coverage_content_type = "coordinate" ;
        float red(point) ;
                red:units = "1" ;
                red:long_name = "red channel" ;
                red:coverage_content_type = "physicalMeasurement" ;
                red:coordinates = "latitude longitude" ;
        float green(point) ;
                green:units = "1" ;
                green:long_name = "green channel" ;
                green:coverage_content_type = "physicalMeasurement" ;
                green:coordinates = "latitude longitude" ;
        float blue(point) ;
                blue:units = "1" ;
                blue:long_name = "blue channel" ;
                blue:coverage_content_type = "physicalMeasurement" ;
                blue:coordinates = "latitude longitude" ;
        float intensity(point, band) ;
                intensity:units = "W m-2 sr-1 nm-1" ;
                intensity:long_name = "Spectral intensity" ;
                intensity:standard_name = "toa_outgoing_radiance_per_unit_wavelength" ;
                intensity:coverage_content_type = "physicalMeasurement" ;
                intensity:coordinates = "latitude longitude" ;

However, I am missing a few standard names.

Points and directions

point_elevation_above_geoid

Definition:
A point is a single spatial measurement on an object's surface as measured in a point cloud or digital terrain model. elevation_above_geoid is the (geometric) height above the geoid, which is the reference geopotential surface. The geoid is similar to mean sea level.

Note: A point could be the ground surface or the surface of a manmade feature. Therefore, I don't think this is covered by existing standard names. One could imagine two points of different elevation at the same lat/lon - e.g. a bridge or an overhanging cliff.

Canonical units: m

---

sensor_to_point_direction_x_component

Definition:
X-component of the vector describing the direction from the sensor to a point of interest, relative to the sensor. "x" indicates a vector component along the grid x-axis, positive with increasing x.

Canonical units: 1 (dimensionless)

I envisage that this and the below 2 terms could be useful in other applications. Anywhere that it is interesting to know the direction between the sensor and a point of interest.

---

sensor_to_point_direction_y_component

Definition:
Y-component of the vector describing the direction from the sensor to a point of interest, relative to the sensor. "y" indicates a vector component along the grid y-axis, positive with increasing y.

Canonical units: 1 (dimensionless)

---

sensor_to_point_direction_z_component

Definition:
Z-component of the vector describing the direction from the sensor to a point of interest, relative to the sensor. "z" indicates a vector component along the grid z-axis, positive with increasing z.

Canonical units: 1 (dimensionless)

---

surface_normal_vector_x_component

Definition:
The X-component of the surface normal vector at a specified location, indicating the orientation of the surface at that point. "x" indicates a vector component along the grid x-axis, positive with increasing x.

Canonical units: 1 (dimensionless)

---

surface_normal_vector_y_component

Definition:
The Y-component of the surface normal vector at a specified location, indicating the orientation of the surface at that point. "y" indicates a vector component along the grid y-axis, positive with increasing y.

Canonical units: 1 (dimensionless)

---

surface_normal_vector_z_component

Definition:
The Z-component of the surface normal vector at a specified location, indicating the orientation of the surface at that point. "z" indicates a vector component along the grid z-axis, positive with increasing y.

Canonical units: 1 (dimensionless)

---

Red, green and blue

Normalised RGB values can be on different scales, with different valid ranges:

• 0 to 1: Used in many programming and graphics systems (e.g., OpenGL, matplotlib in Python).
  0 represents no intensity (black).
  1 represents full intensity of that channel.
• 0 to 255: Standard 8-bit representation used in most digital images (e.g., JPEG, PNG).
  Each channel can represent 256 levels of intensity.
• 0 to 65,535: Used in 16-bit images (higher dynamic range imaging).
  Each channel can represent 65,536 levels of intensity.

I am unsure how to deal with this. Perhaps the user should be required to include valid_min and valid_max variables attributes? Or should CF specify which range the user should use? It is not obvious to me which range is best.

The wavelength that red, green and blue refer to can vary between different applications, and depends on the context. For the sake of interoperability, the wavelength referred to should be stated. I am not sure what is the best approach here. Perhaps a variable attribute? This is usually only used for quick preview purposes as far as I am aware, so maybe this isn't an issue. In my example I give the intensity of each band as a 2D variable, so one could compute their own RGB values if they wanted to. However, I think it is useful to include both.

---

normalised_red_wavelength_intensity_of_point

Definition:
Intensity of electromagnetic radiation in the red wavelength band reflected or emitted from a point. Values are scaled relative to a reference intensity or maximum possible value. red corresponds to the red component of the RGB colour model.

Canonical units: 1

normalised could be scaled

---

normalised_green_wavelength_intensity_of_point

Definition:
Intensity of electromagnetic radiation in the green wavelength band reflected or emitted from a point. Values are scaled relative to a reference intensity or maximum possible value. red corresponds to the green component of the RGB colour model.

Canonical units: 1

normalised could be scaled

---

normalised_blue_wavelength_intensity_of_point

Definition:
Intensity of electromagnetic radiation in the blue wavelength band reflected or emitted from a point. Values are scaled relative to a reference intensity or maximum possible value. red corresponds to the blue component of the RGB colour model.

Canonical units: 1

normalised could be scaled

[github-actions]

Thank you for your proposal. These terms will be added to the cfeditor (http://cfeditor.ceda.ac.uk/proposals/1) shortly. Your proposal will then be reviewed and commented on by the community and Standard Names moderator.

[JonathanGregory]

Dear Luke @lhmarsden

It is certainly CF-compliant to store a field of data at scattered points, as you suggest. Your point dimension and coordinate variable are a "discrete axis" (section 4.5). In addition, you can designate this arrangement with the attribute featureType="point" (sect 9.1, examples in appendix H1). Is this what you mean?

I don't think quantities at points need special standard names. They are the same quantities as in continuous geophysical fields, but sampled discretely. However, most of your quantities probably do need standard names. The first one can use the existing standard name of altitude, which means height above geoid.

Best wishes

Jonathan

[lhmarsden]

Thanks for your reply, Jonathan. Yes, I think the discrete axis section is relevant here, and I agree that most of the variables don't need special standard names. I was not sure about whether altitude was appropriate in this case or not. I was envisioning a scenario where we just have lat, lon and altitude. For example, a DEM. If we use the standard name altitude, is it clear that this is the altitude at the surface, or just some altitude at which measurements are taken? Maybe I am being pedantic here... Luke

…

________________________________ From: JonathanGregory ***@***.***> Sent: Tuesday, November 19, 2024 2:28:02 pm To: cf-convention/vocabularies ***@***.***> Cc: Luke Marsden ***@***.***>; Mention ***@***.***> Subject: Re: [cf-convention/vocabularies] Standard names: Point clouds, DEMs and DTMs (Issue #239) Dear Luke @lhmarsden<https://github.com/lhmarsden> It is certainly CF-compliant to store a field of data at scattered points, as you suggest. Your point dimension and coordinate variable are a "discrete axis" (section 4.5<https://cfconventions.org/cf-conventions/cf-conventions.html#discrete-axis>). In addition, you can designate this arrangement with the attribute featureType="point" (sect 9.1<https://cfconventions.org/cf-conventions/cf-conventions.html#_features_and_feature_types>, examples in appendix H1<https://cfconventions.org/cf-conventions/cf-conventions.html#point-data>). Is this what you mean? I don't think quantities at points need special standard names. They are the same quantities as in continuous geophysical fields, but sampled discretely. However, most of your quantities probably do need standard names. The first one can use the existing standard name of altitude, which means height above geoid. Best wishes Jonathan — Reply to this email directly, view it on GitHub<#239 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AOMNBFKFYT3SFGY6J6UTTLD2BM4F3AVCNFSM6AAAAABSBQZJ4KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBVG4YTIMBXHA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[JonathanGregory]

Sorry, yes. There is a standard name of surface_altitude too, which is presumably the right one?

[lhmarsden]

This is close. Maybe this is okay. Maybe I am being pedantic. The definition is

The surface called "surface" means the lower boundary of the atmosphere. Altitude is the (geometric) height above the geoid, which is the reference geopotential surface. The geoid is similar to mean sea level.

If the point cloud is of a bridge, and includes the underside of the bridge, can that be considered the lower boundary of the atmosphere?

What is the protocol around adjusting definitions for existing standard names? The name itself does seem suitable, and it feels unnecessary to add a new name for this.

'The surface called "surface" ' seems to be a bit clumsy wording to me. Perhaps a more suitable definition would be

"surface" refers to the boundary between the atmosphere and an object. Altitude is the (geometric) height above the geoid, which is the reference geopotential surface. The geoid is similar to mean sea level.

Luke

[ChrisBarker-NOAA]

is Z -- elvation, height, ??? -- a property of a point at a 2D location, or is the point a 3D point?

        float Z(point) ;
                Z:units = "meters" ;
                Z:long_name = "Z coordinate" ;
                Z:standard_name = "height" ;
                Z:grid_mapping = "crs" ;
                Z:coverage_content_type = "coordinate" ;

Calling it a coordinate makes me think it's well, the 3rd coordinate of the point.

If the latter, then I think the properties should have:

            green:coordinates = "Z, latitude longitude" ;

And that makes this different than a DEM, which, IIUC, defines a surface, such that each location in the 2D, X-Y (lat-lon) space has a single elevation value.

I think a point cloud can have more than. one point at the same horizontal location -- i.e. on on top of the other.

Does this have any bearing on the standard name? I"m not sure it does, but we should make sure the examples are clear.

[lhmarsden]

I think this is a 3rd coordinate of a point. You are correct, I should update the coordinates variable attribute to state this.

For a point cloud, this is just the location of a point in space, all three dimensions.

[JonathanGregory]

Dear Luke @lhmarsden

I understand a digital elevation model to be a 2D field of surface_altitude as a function of the horizontal coordinate variables, say X and Y. A gridded DEM is a 2D data variable surface_altitude(X,Y). If you select a number of discrete points, the data variable of surface altitude would be surface_altitude(point) with auxiliary coordinate variables X(point) and Y(point). It's still the surface altitude, though, even if you're not providing the whole surface.

For a collection of 3D points, which aren't necessarily on the surface, the data variables are dimensioned (point), and there's a third auxiliary coordinate variable Z(point), whose standard name would be altitude if the vertical coordinate is the height above the geoid. I think this is the situation if you have a point on the underside of a bridge. That point isn't on "The Surface" (bottom of the atmosphere), I agree. At any given X and Y there is only one surface altitude. It would be the surface of the river or whatever is underneath the bridge. But, as Chris says, you could have any number of 3D points at the same X and Y but with different Z if Z is altitude.

For a collection of 2D points which are all on the surface, I suppose the data variable of altitude, dimensioned point, could be either altitude or surface_altitude. The latter is perhaps better because it's more informative. This data variable could also serve as the auxiliary Z(point) coordinate variable for other quantities that describe the state of the surface, such as its temperature.

Best wishes

Jonathan

[lhmarsden]

Hi @JonathanGregory, thanks and I agree with all of that. I think I can proceed with altitude for my Z values or surface_altitude for a DEM.

Now for the other data variables :)

[JonathanGregory]

Dear Luke

Your next three quantities, sensor_to_point_direction_[xyz]_component, are somewhat like the existing direction_of_radial_vector_away_from_instrument. That's a 2D vector, yours is 3D, but we might follow that pattern with e.g. [xyz]_component_of_vector_away_from_instrument. The away_from is useful in case anyone wants to describe the reverse direction. There are existing standard names with sensor, but more with instrument.

For the following three, we could also follow a similar pattern, such as [xyz]_component_of_surface_outward_normal_vector. Again, it seems useful to specify the sign convention by including outward.

What are your views?

Best wishes

Jonathan

[lhmarsden]

They both sound like good suggestions, I agree!

[JonathanGregory]

Dear Luke

Finally, there are the colours! I'm not sure what geophysical quantities these are, though. You say "emitted from a point". That sounds like a radiant object. If it was the colour of the surface, it would be reflected light. Where is the electromagnetic radiation coming from?

I suppose that if incident radiation is expressed as RGB, each channel must physically be a quantity in W m-2 integrated over some frequency range, perhaps convolved with a weighting function for the characteristics of the instrument measuring the radiation. The units of each channel are some multiple of W m-2. However, you are thinking of an image produced by the instrument, I guess. Does it tell you the maximum power it can measure in each channel, or alternatively the minimum it can measure (represented by 1 in 8- or 16-bit images)?

Best wishes

Jonathan