Auto serialization + encoding for Python values

[wyfo]

Describe the feature

The feature already exists in the Python API

zenoh-python/zenoh/value.py

Lines 44 to 67 in 1516153

	def __new__(cls, payload: IntoValue, encoding: Encoding=None):
	if encoding is None:
	if isinstance(payload, Value):
	return payload
	return Value.autoencode(payload)
	else:
	if not isinstance(payload, bytes):
	raise TypeError("`encoding` was passed, but `payload` is not of type `bytes`")
	return Value.new(payload, encoding)
	@staticmethod
	def autoencode(value: IntoValue) -> 'Value':
	"Automatically encodes the value based on its type"
	if isinstance(value, IValue):
	return Value.new(value.payload, value.encoding)
	if isinstance(value, bytes):
	return Value.new(value, Encoding.APP_OCTET_STREAM())
	if isinstance(value, str):
	return Value.new(value.encode(), Encoding.TEXT_PLAIN())
	if isinstance(value, int):
	return Value.new(f"{value}".encode(), Encoding.APP_INTEGER())
	if isinstance(value, float):
	return Value.new(f"{value}".encode(), Encoding.APP_FLOAT())
	return Value.new(json.dumps(value).encode(), Encoding.APP_JSON())

.
However, in protocol_changes, the encoding has been formally detached from payload. Quoting @milyin:

As far as I understand, the difference between python and rust in this aspect is that in Rust we know the type on compilation stage. So in Rust it's normal to think in this paradigm: pass the value of known type to payload set the encoding corresponding to the type if necessary

But in Python the type of variable is stored inside the variable. So we need a way to serialize arbitrary variable and inform the caller which serializer was used

Also despite that encoding type is detached from serialization operation, there are still set of predefined encoding values whcih actually corresponds to our provided serializers (these ZENOH_INT. ZENOH_STRING, ZENOH_BYTES, etc). So it seems natual to use these values for identifyting serializer.

Maybe it makes sense to update our rust serialization API by adding some function/trait method which associates encoding value to specific serializer. I think unless we have an association between serializer and encoding on Rust we can't correctly provide automatic serialization in Python.

For my part, I agree with @milyin. @Mallets, do you think we should simply drop the autoencode feature in the new Zenoh Python API? This breaking change would be mitigated by Python typing annotations, allowing users to spot and fix the impact before running into a runtime check error.

[Mallets]

payload should be IntoPayload and we should keep the autoserialization (let's not call it encode) in the same way as done in Rust. Corresponding serializer/deserializer should be implemented as well for primitive types. Associated encoding value are out of scope.

[wyfo]

Associated encoding value are out of scope.

That's the answer I wanted.

Corresponding serializer/deserializer should be implemented as well for primitive types.

I imagine that list/dict should keep the actual behavior and serialize to JSON, similar to serde_json::Value implementation. Should we raise an error for other types, as it's the case today, or use pickle?