Feature/785 documentation metadata

documentation/src/main/resources/_data/sidebars/ditto_sidebar.yml

@@ -133,6 +133,9 @@ entries:
                   - title: Namespaces and Names
                     url: /basic-namespaces-and-names.html
                     output: web
+                  - title: Thing Metadata
+                    url: /basic-metadata.html
+                    output: web
                   - title: Errors
                     url: /basic-errors.html
                     output: web

documentation/src/main/resources/jsonschema/thing_v2.json

@@ -71,6 +71,10 @@
     "_modified": {
       "type": "string",
       "description": "_(read-only)_ The modified timestamp of the Thing in ISO-8601 UTC format. The timestamp is set on each modification of a Thing."
+    },
+    "_metadata": {
+      "type": "object",
+      "description": "_(read-only)_ The Metadata of the Thing. This field is not returned by default but must be selected explicitly. The content is a JSON object having the Thing's JSON structure with the difference that the JSON leaves of the Thing are JSON objects containing the metadata."
     }
   },
   "required": [

documentation/src/main/resources/openapi/ditto-api-2.yml

@@ -403,6 +403,7 @@ paths:
         - $ref: '#/components/parameters/thingIdPathParam'
         - $ref: '#/components/parameters/ifMatchHeaderParam'
         - $ref: '#/components/parameters/ifNoneMatchHeaderParam'
+        - $ref: '#/components/parameters/putMetadataParam'
         - $ref: '#/components/parameters/requestedAcksParam'
         - $ref: '#/components/parameters/timeoutParam'
         - $ref: '#/components/parameters/responseRequiredParam'
@@ -4664,6 +4665,10 @@ components:
           description: |-
             _(read-only)_ The modified timestamp of the Thing in ISO-8601 UTC format. The timestamp is set on each
             modification of a Thing. This field is not returned by default but must be selected explicitly.
+        _metadata:
+          type: object
+          description: |-
+            _(read-only)_ The Metadata of the Thing. This field is not returned by default but must be selected explicitly.
     Policy:
       type: object
       description: policy consisting of policy entries
@@ -4858,6 +4863,25 @@ components:
       required: false
       schema:
         type: string
+    putMetadataParam:
+      name: put-metadata
+      in: header
+      description:  >-
+        The `put-metadata` header, which sets Metadata information in the Thing.
+      required: false
+      schema:
+        type: array
+        description: An array of objects containing metadata to apply.
+        items:
+          type: object
+          description: Object containing a `key` where to apply the metadata and a `value` with the metadata value to apply.
+          additionalProperties:
+            properties:
+              key:
+                type: string
+                description: The JsonPointer to set the metadata `value` to. May start with `*/<key>` in order to apply the metadata to all affected JSON leaves.
+              value:
+                description: The arbitrary JSON value to set as metadata.
     featureIdPathPathParam:
       name: featureId
       in: path
@@ -5043,6 +5067,10 @@ components:
 
           Specifically selects the modified timestamp of the thing in ISO-8601 UTC format. The timestamp is set on each modification of a thing.
 
+        * `_metadata`
+
+          Specifically selects the Metadata of the thing. The content is a JSON object having the Thing's JSON structure with the difference that the JSON leaves of the Thing are JSON objects containing the metadata.
+
         * `_policy`
 
           Specifically selects the content of the policy associated to the thing. (By default, only the policyId is returned.)

documentation/src/main/resources/pages/ditto/basic-metadata.md

@@ -0,0 +1,191 @@
+---
+title: Thing Metadata
+keywords: metadata, things, model, semantic
+tags: [model]
+permalink: basic-metadata.html
+---
+
+A Thing in Ditto is also able to store Metadata information, e.g. about single 
+[feature properties](basic-feature.html#feature-properties), complete features and also [attributes](basic-thing.html#attributes)
+or other data stored in a digital twin ([thing](basic-thing.html)).
+
+This metadata can contain additionally information which shall not be treated as part of the twin's value, however may
+be useful to provide some context of the twin's data.
+
+Metadata has not its own API but can only be updated/set while modifying the state of a twin as a side effect.<br/>
+By default, metadata is not returned at API requests, but must be [asked for explicitly](#reading-metadata-information).
+
+An example is the timestamp when the current value of e.g. a feature property was updated for the last time.
+Or the metadata information of a feature property may contain information about its type or its semantics (e.g. a unit 
+of measurement).
+
+
+## Setting Metadata
+
+Setting arbitrary `Metadata` is possible by using the `put-metadata` header 
+(e.g. for HTTP requests, set it as HTTP header, for Ditto Protocol requests, put it in the `"headers"` section of the 
+protocol message), see [here for an overview of the available headers](protocol-specification.html#headers).
+
+The value of the `put-metadata` is a JSON array containing JSON objects with `"key"` and `"value"` parts:
+* `"key"`: describes the hierarchical position in the Thing where the metadata should be placed
+* `"value"`: is an arbitrary JSON value to set as metadata (could also be a nested JSON object)
+
+### Example for setting Metadata
+
+Assume you modify your twin's lamp color with a call: 
+```json
+{
+  "thingId": "org.eclipse.ditto:my-lamp-1",
+  "features": {
+    "lamp": {
+      "properties": {
+        "color": {
+          "r": 100,
+          "g": 0,
+          "b": 255
+        }     
+      } 
+    } 
+  }
+}
+```
+
+You want to specify to set metadata which affects all the changed properties (`"r"`, `"g"` and `"b"`) plus some 
+extra metadata to only set for the `"r"` property.<br/>
+The content of the `put-metadata` in order to do that would look like this:
+
+```json
+[
+  {
+    "key": "*/foo",
+    "value": "bar"
+  },
+  {
+    "key": "/features/lamp/properties/color/r",
+    "value": {
+      "foo": "bar",
+      "issuedAt": "someTimestamp",
+      "issuedBy": {
+        "name":"me",
+        "mail":"[email protected]"
+      }
+    }
+  }
+]
+```
+
+The resulting Thing JSON including its `_metadata` would look like this:
+```json
+{
+    "thingId": "org.eclipse.ditto:meta-test-1",
+    "features": {
+        "lamp": {
+            "properties": {
+                "color": {
+                    "r": 100,
+                    "g": 0,
+                    "b": 255
+                }
+            }
+        }
+    },
+    "_metadata": {
+        "features": {
+            "lamp": {
+                "properties": {
+                    "color": {
+                        "r": {
+                            "foo": "bar",
+                            "issuedAt": "someTimestamp",
+                            "issuedBy": {
+                                "name": "me",
+                                "mail": "[email protected]"
+                            }
+                        },
+                        "g": {
+                            "foo": "bar"
+                        },
+                        "b": {
+                            "foo": "bar"
+                        }
+                    }
+                }
+            }
+        }
+    }
+}
+```
+
+
+### Setting Metadata to all affected JSON leaves
+
+A special syntax for the key is `*/{key}` which means that all affected JSON leaves of the modify operation will
+get the Metadata key `{key}` with the given value. So if, for example, only the affected JSON leaves should 
+get the timestamp where the changed values were recorded, one would set the `put-metadata` header as shown in the 
+following example: 
+
+```json
+[
+  {
+    "key": "*/timestamp",
+    "value": "someTimestamp"
+  }
+]
+```
+
+
+## Reading Metadata information
+
+Metadata of a Thing can be retrieved is by querying a full thing, e.g. via the [HTTP API](http-api-doc.html), and 
+specifying an (additional) [field selector](httpapi-concepts.html#with-field-selector) `_metadata`, 
+e.g.: `?fields=thingId,attributes,_metadata`.
+
+### Example for reading Metadata
+
+For example a `GET` request to 
+`https://{ditto-instance}/api/2/things/{namespace}:{name}?fields=thingId,policyId,features,_created,_modified,_revision,_metadata`
+will yield the Metadata stored for the given Thing, in the following format:
+
+```json
+{
+  "thingId": "org.eclipse.ditto:thing-1",
+  "policyId": "...",
+  "features": {
+    "lamp": {
+      "properties": {
+        "on": true,
+        "color": {
+          "r": 0,
+          "g": 255,          
+          "b": 255
+        }
+      }
+    }
+  },
+  "_created": "2020-06-01T10:00:00Z",
+  "_modified": "2020-06-09T14:30:00Z",
+  "_revision": 42,
+  "_metadata": {
+    "features": {
+      "lamp": {
+        "properties": {
+          "on": {
+            "issuedAt": "2020-06-09T14:30:00Z"
+          },
+          "color": {
+            "r": {
+              "issuedAt": "2020-06-09T14:15:00Z"
+            },
+            "g": {
+              "issuedAt": "2020-06-09T14:15:00Z"
+            },
+            "b": {
+              "issuedAt": "2020-06-09T14:15:00Z"
+            }
+          }
+        }
+      }
+    }
+  }
+}
+``` 

documentation/src/main/resources/pages/ditto/basic-thing.md

@@ -60,6 +60,12 @@ A Thing may contain an arbitrary amount of [Features](basic-feature.html).
 many Features" max-width=100 %}
 
 
+### Metadata
+
+A Thing may contain additional [metadata](basic-metadata.html) for all of its attributes and features describing the
+semantics of the data or adding other useful information about the data points of the twin.
+
+
 ### Model specification
 
 #### API version 1 - Deprecated

documentation/src/main/resources/pages/ditto/protocol-specification.md

@@ -78,6 +78,7 @@ There are some pre-defined headers, which have a special meaning for Ditto:
 | `requested-acks` | Defines which [acknowledgements](basic-acknowledgements.html) are requested for a command processed by Ditto. | `JsonArray` of `String` - default: `["twin-persisted"]` |
 | `timeout` | Defines how long the Ditto server should wait, e.g. applied when waiting for requested acknowledgements. | `String` - e.g.: `42s` or `250ms` or `1m` - default: `60s`|
 | `version` | Determines in which schema version the `payload` should be interpreted. | `Number` - currently: \[1,2\] |
+| `put-metadata` | Determines which Metadata information is stored in the thing. | `JsonArray` of `JsonObject`s containing [metadata](basic-metadata.html) to apply. |
 
 Custom headers of messages through the [live channel](protocol-twinlive.html) are delivered verbatim. When naming 
 custom headers, it is best to attach a prefix specific to your application, that does not conflict with Ditto or
@@ -97,6 +98,7 @@ HTTP protocol, for example the prefix `ditto-*`.
 The interaction between the headers `response-required`, `requested-acks` and `timeout` is documented
 [here](basic-acknowledgements.html#interaction-between-headers).
 
+
 ## Path
 
 Contains a JSON pointer of where to apply the [value](#value) of the protocol message.