Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update Rust/Python migration guide for serialization #71

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 45 additions & 27 deletions content/docs/migration_1.0/Python.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,50 +37,68 @@ with zenoh.open(zenoh.Config()) as session:
session.declare_subscriber("my/keyepxr", lambda s: print(s))
sleep(10) # subscriber stays in background and its callback can be called
# `session.close()` will be called at the end of the block, and it will undeclare the subscriber
```¬
```

## ZBytes, encoding, and (de)serialization
## Value is gone, long live ZBytes

### Encoding
`Value` has been split into `ZBytes` and `Encoding`. `put` and other operations now requires a `ZBytes` payload, and builders accept an optional `Encoding` parameter.

`zenoh.Value` has been split into `zenoh.ZBytes` and `zenoh.Encoding`. Put and other operations now require a `ZBytes` payload, and accept an optional `Encoding`; the encoding is no longer automatically deduced from the payload type.
`ZBytes` is a raw bytes container. It can be created directly from raw bytes/strings using `ZBytes` constructor. Then bytes can be retrieved using `ZBytes.to_bytes` or `ZBytes.to_string`. Sample payload is now a `ZBytes` instead of `bytes`.

```python
session.put("my/keyexpr", 42) # default encoding `zenoh/bytes`session.put("my/keyexpr", 42, encoding=zenoh.Encoding.ZENOH_INT64)
- Zenoh 0.11.x

```rust
sample = subscriber.recv()
my_string = sample.payload.decode("utf-8")
```

Publishers can be declared with a default encoding, which will be used for each put operation.
- Zenoh 1.0.0

```rust
sample = subscriber.recv()
my_string = sample.payload.to_string()
```

You can look at a full set of examples in `examples/z_bytes.py`.

### Serialization

Zenoh does provide serialization for convenience as an extension in `zenoh.ext` module. Serialization is implemented for a bunch of standard types like `int`, `float`, `list`, `dict`, `tuple`, etc. and is used through functions `z_serialize`/`z_deserialize`.

```python
import json
publisher = session.declare_publisher("my/keyepxr", encoding=zenoh.Encoding.APPLICATION_JSON)
publisher.put(json.dumps({"key", "value"})) # default encoding from publisher `application/json`
input = b"raw bytes"
payload = ZBytes(input)
output = payload.to_bytes()
```

### (De)serialization
`zenoh.ext` serialization doesn't pretend to cover every use cases, as it is just one available choice among other serialization format like JSON, Protobuf, CBOR, etc. In the end, Zenoh will just send and receive payload raw bytes independently of the serialization used.

NOTE: ⚠️ Serialization of `bytes` is not the same as passing `bytes` to `ZBytes` constructor.

## Encoding

Arbitrary types can be serialized to and deserialized from `ZBytes`. Default (de)serializers are provided for builtin types; `list`/`dict` are **no longer** serialized to JSON, they use instead the builtin serializer of Zenoh, which is compatible with other Zenoh bindings.
`Encoding` has been reworked.
Zenoh does not impose any encoding requirement on the user, nor does it operate on it.
It can be thought of as optional metadata, carried over by Zenoh in such a way that the end user’s application may perform different operations based on encoding.

NOTE: ⚠️ The encoding is no longer automatically deduced from the payload type.

```python
payload = zenoh.ZBytes(42)
assert payload.deserialize(int) == 42# `ZBytes.deserialize` accepts generic `list`/`dict` typepayload = zenoh.ZBytes([0.5, 37.1])
assert payload.deserialize(list[float]) == [0.5, 37.1]
session.put(json.dumps({"key", "value"}), encoding=Encoding.APPLICATION_JSON)
```

(De)serializers can be registered for custom types:
Users can also define their own encoding scheme that does not need to be based on the pre-defined variants.

```python
from dataclasses import dataclass
import zenoh
@dataclassclass RGB:
red: int green: int blue: [email protected] serialize_rgb(rgb: RGB) -> zenoh.ZBytes:
return zenoh.ZBytes(rgb.red | (rgb.green << 8) | (rgb.blue << 16))
@zenoh.deserializerdef deserialize_rgb(payload: zenoh.ZBytes) -> RGB:
compact = payload.deserialize(int)
return RGB(compact & 255, (compact >> 8) & 255, (compact >> 16) & 255)
color = RGB(61, 67, 97)
assert zenoh.ZBytes(color).deserialize(RGB) == color
# types with a registered serializer can be used directly with `put`session.put("my/keyexpr", color)
encoding = Encoding("pointcloud/LAS")
```

Because encoding is now optional for `put`, `Publisher` can be declared with a default encoding, which will be used in every `Publisher.put`.

```python
publisher = session.declare_publisher("my/keyepxr", encoding=Encoding.APPLICATION_JSON)
// default encoding from publisher `application/json`
publisher.put(json.dumps({"key", "value"}))
```

## Handlers
Expand Down
38 changes: 28 additions & 10 deletions content/docs/migration_1.0/Rust.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,32 +142,42 @@ However, the normal user would rarely need to call this method directly.*

## Value is gone, long live ZBytes

We have replaced `Value` with `ZBytes` and `Encoding` , and added a number of conversion implementations such that user structs can be serialized into `ZBytes`, sent via Zenoh, and de-serialized from `ZBytes` with ease.
`Value` has been split into `ZBytes` and `Encoding`. `put` and other operations now requires a `ZBytes` payload, and builders accept an optional `Encoding` parameter. The encoding is no longer automatically deduced from the payload type.

This is facilitated through the `ZSerde` struct, and implementations of the traits
`zenoh::bytes::Deserialize` and `zenoh::bytes::Serialize`.
`ZBytes` is a raw bytes container, which can also contain non-contiguous region of memory. It can be created directly from raw bytes/strings using `ZBytes::from`. Then bytes can be retrieved using `ZBytes::to_bytes`, which returns a `Cow<[u8]>`, as a copy may have to be done if the underlying bytes are not contiguous.

We provide implementations of Zenoh’s aforementioned `Deserialize` and `Serialize` traits for primitive Rust types, Rust’s `Vec`, the `Value` type exposed by `Serde`'s various libraries as well as an example of `Protobuf` ’s `prost::Message` type.

You can look at a full set of examples in `examples/examples/z_bytes.rs`.

NOTE: ⚠️ `ZSerde` is not the only serializer/deserializer users can make use of, nor a limitation to the types supported by Zenoh. Users are free to use whichever serializer/deserializer they wish!
- Zenoh 0.11.x

```rust
let sample = subscriber.recv_async().await.unwrap();
let value: Value = sample.value;
let the_string: String = value.try_into().unwrap();
let raw_bytes: Vec<u8> = value.try_into().unwrap();
```

- Zenoh 1.0.0

```rust
let sample = subscriber.recv_async().await.unwrap();
let zbytes: ZBytes = sample.payload();
let the_string: String = zbytes.deserialize::<String>().unwrap();
let raw_bytes: Cow<[u8]> = zbytes.as_bytes();
```

You can look at a full set of examples in `examples/examples/z_bytes.rs`.

### Serialization

Zenoh does provide serialization for convenience as an extension in `zenoh-ext`. Serialization is implemented for a bunch of standard types like integers, floats, `Vec`, `HashMap`, etc. and is used through functions `z_serialize`/`z_deserialize`.

```rust
let input: Vec<f32> = vec![0.0, 1.5, 42.0];
let payload: ZBytes = z_serialize(&input);
let output: Vec<f32> = z_deserialize(&payload).unwrap();
```

`zenoh-ext` serialization doesn't pretend to cover every use cases, as it is just one available choice among other serialization format like JSON, Protobuf, CBOR, etc. In the end, Zenoh will just send and receive payload raw bytes independently of the serialization used.

NOTE: ⚠️ Serialization of `Vec<u8>` is not the same as creating a `ZBytes` from a `Vec<u8>`: the resulting `ZBytes` are different, and serialization doesn't take ownership of the bytes.

## Encoding

`Encoding` has been reworked.
Expand Down Expand Up @@ -206,6 +216,14 @@ Users can also define their own encoding scheme that does not need to be based o
let encoding = Encoding::from("pointcloud/LAS");
```

Because encoding is now optional for `put`, `Publisher` can be declared with a default encoding, which will be used in every `Publisher::put`.

```rust
let publisher = session.declare_publisher("my/keyepxr").encoding(Encoding::APPLICATION_JSON).await.unwrap();
// default encoding from publisher `application/json`
publisher.put(serde_json::to_vec(json!({"key", "value"})).unwrap()).await.unwrap();
```

## Attachment

In Zenoh 0.11.x, the `AttachmentBuilder` was required to create an attachment.
Expand Down