Don't apply desired property if "applied" property was changed by device more recently

[thjaeckle]

As an enhancement of #696 it shall be made possible that a local change from a device may be prioritized above the desired change of a property.

Problem statement

A desiredProperties property change could be outdated compared to the reported property which was e.g. directly changed at a device.
In such cases it does not make sense to apply the desiredProperties property.

Example

That is useful in e.g. the following scenario. Take the room thermostat again as an example:

• the property target-temperature is at 20
• the room thermostat looses its internet connectivity (e.g. due to an ISP outage)
• user feels chilly, so the desiredProperties property configuration/target-temperature is set via mobile app to 22
• as the device doesn't get this change right away as it is not online, nothing happens
• after some time the user notices that it's not getting warmer so use stands up and changes the configuration/target-temperature manually at the room thermostat device to 23
• room thermostat re-establishes connectivity and starts to apply missed desiredProperties state changes
• now, the reported/desired feature implemented in [epic] Distinguish between reported and desired state in twins #696 would lead to:
  • desired state (desiredProperties/configuration/target-temperature=22) differs from actual/reported state (properties/configuration/target-temperature=23)
  • the device would apply the desiredProperties configuration/target-temperature of 22 as a result and overwrite the locally set 23
  • the user wonders why the configuration/target-temperature is now at 22

What would have been desirable in this scenario:

• if the actual property configuration/target-temperature was modified more recently, the desiredProperties configuration/target-temperature is no longer valid and deleted
• this is a configurable behavior

Improvement

After introducing #680 where _metadata is added to each property in a thing, it can be found out which revision each property has and e.g. why (e.g. due to "desired-sync" or "manual-overwrite") it was changed.
So both the device but also Ditto itself could find out if the property or the desiredProperties property target-temperature was updated more recently.

Example Ditto Protocol command (where one of the 2 "ditto-metadata:changeReason" values is chosen):

{
  "topic": "org.eclipse.ditto/my-thermostat-1/things/twin/commands/modify",
  "headers": {
    "ditto-metadata:changeReason": "desired-sync",
    "ditto-metadata:changeReason": "manual-overwrite"
  },
  "path": "/features/Thermostat/properties/configuration/target-temperature",
  "value": 20.0
}

Ditto know "changeReson" values:

• desired-sync
• manual-overwrite

A device would have the means to determine by itself whether desiredProperties or actual reported properties are the more recent configuration state and whether the reported properties were e.g. modified based on a manual overwrite.

Ditto could also apply a deletion strategy based on the _revision and the changeReason:

• reported-more-recent-manual-overwrite: would delete the desiredProperties property if there is a more recently changed reported property with a manual-overwrite changeReason.

{
  "definition": [],
  "properties": {
    "status": {
      "measured-temperature": 20
    },
    "configuration": {
      "target-temperature": 23
    }
  },
  "desiredProperties": {
    "configuration": {
      "target-temperature": 23
    }
  },
  "desiredSettings": {
    "desiredDeletionStrategy": [ 
      "reported-reaches-desired",
      "reported-more-recent-manual-overwrite"
    ]
  },
  "_metadata": {
    "policyId": {
      "_modified": "2020-06-09T14:00:00Z",
      "_revision": 1
    },
    "features": {
      "Thermostat": {
        "properties": {
          "configuration": {
            "target-temperature": {
              "_modified": "2020-06-09T14:40:00Z",
              "_revision": 44,
              "changeReason": "manual-overwrite"
            }
          }
        },
        "desiredProperties": {
          "configuration": {
            "target-temperature": {
              "_modified": "2020-06-09T14:30:00Z",
              "_revision": 42
            }
          }
        }
      }
    }
  }
}

[geglock]

I would give the full control to the devices, based on the last-updated timestamp an/or revision information for each "desired" property.
Based on this, the device can decide if a pending desired change is outdated (but was delivered late), compared to some newer local device-side reconfigurations.

[ffendt]

I agree with @geglock to leave this to the device itself. In my opinion we will add unneeded complexity by also adding settings to the "desired" functionality.

[thjaeckle]

Closed as the device can and should find out by itself with the implicitly created metadata: #778