Support for collections.abc

[TommyDuc]

Hi!
I was interested in using your library which simply a lot of our serialization, notably because we use frozen dataclasses extensively. We also use the abstract base classes from collections.abc so that the collections are also readonly from a typing perspective.

However it doesn't seem that ABCs are supported. I also tried defining (de)serializer functions in both the @serde decorator and using the field function but I still get the same error. IMO ABCs should be work because they can be instantiated using builtin types and deserialized using the same methods as builtins.

Here's a reference code:

from collections.abc import Mapping, Sequence
from dataclasses import dataclass

from serde import serde
from serde.json import to_json


@serde
@dataclass(frozen=True, kw_only=True, slots=True)
class Foo:
    builtin_dict: dict[str, int]
    builtin_list: list[int]
    collections_abc_mapping: Mapping[str, int]  # Remove this property to make it work
    collections_abc_sequence: Sequence[int]  # Remove this property to make it work


def main() -> None:
    print("start")
    foo = Foo(
        builtin_dict={"a": 1},
        builtin_list=[1, 2, 3],
        collections_abc_mapping={"b": 2},
        collections_abc_sequence=[1, 2, 3],
    )
    print(foo)
    print(to_json(foo))


if __name__ == "__main__":
    main()

start
Foo(builtin_dict={'a': 1}, builtin_list=[1, 2, 3], collections_abc_mapping={'b': 2}, collections_abc_sequence=[1, 2, 3])
Traceback (most recent call last):
  File "/home/tommy/projects/misc/serde_test/src/example_package/serde_exemple.py", line 30, in <module>
    main()
  File "/home/tommy/projects/misc/serde_test/src/example_package/serde_exemple.py", line 26, in main
    print(to_json(foo))
          ^^^^^^^^^^^^
  File "/home/tommy/projects/misc/serde_test/.venv/lib/python3.11/site-packages/serde/json.py", line 70, in to_json
    return se.serialize(to_dict(obj, c=cls, reuse_instances=False, convert_sets=True), **opts)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tommy/projects/misc/serde_test/.venv/lib/python3.11/site-packages/serde/se.py", line 450, in to_dict
    return to_obj(  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/tommy/projects/misc/serde_test/.venv/lib/python3.11/site-packages/serde/se.py", line 378, in to_obj
    raise SerdeError(e) from None
serde.compat.SerdeError: Unsupported type: Dict

[yukinarit]

Hi @TommyDuc
Thanks for being interested in pyserde. Yes, those generic classes are not yet supported, but definitely nice features!
Unfortunately, I am unable to work on this right now. If you're interested in contributing, I am happy to assist you, so please let me know.

[yukinarit]

I feel it could be fairly easily implemented if you tweak is_dict and is_list 🤔

pyserde/serde/compat.py

Lines 685 to 701 in 76e919c

	def is_dict(typ: Type[Any]) -> bool:
	"""
	Test if the type is `typing.Dict`.
	>>> from typing import Dict
	>>> is_dict(Dict[int, int])
	True
	>>> is_dict(Dict)
	True
	>>> is_dict(DefaultDict[int, int])
	True
	"""
	try:
	return issubclass(get_origin(typ), (dict, defaultdict)) # type: ignore
	except TypeError:
	return typ in (Dict, dict, DefaultDict, defaultdict)

pyserde/serde/compat.py

Lines 569 to 582 in 76e919c

	def is_list(typ: Type[Any]) -> bool:
	"""
	Test if the type is `typing.List`.
	>>> from typing import List
	>>> is_list(List[int])
	True
	>>> is_list(List)
	True
	"""
	try:
	return issubclass(get_origin(typ), list) # type: ignore
	except TypeError:
	return typ in (List, list)

[TommyDuc]

I can look into it this week :)

[TommyDuc]

I began working on this issue and there will be a name conflict between the types from typing and those from collections.abc. Given this is non-trivial work I would like to have your input before. There are some options to approach this problem:

From example:

import typing as t
import collections.abc as abc

@serde
class Foo:
    mt: t.Mapping[int, int]
    ma: abc.Mapping[int, int]

Another option is to create a module to aliases each types (also non-trivial work):

# types.py
from collections.abc import Mapping as AbcMapping
from typing import Mapping as TMapping

__all__ = (...)

But personally I don't really like type aliases being used to create other types names.

Yet another option that would be more generic but would require more work would be make use of Python's protocols paradigm.

Looking forward for your input :)

[yukinarit]

Hi @TommyDuc

Do you mean a name conflict in generated code?

Perhaps you could try absolute path e.g. typing.Mapping? I guess it works because typing module is included in the scope.
If that works, you can add collections.abc in the scope and do the same for collections.abc.Mapping? 🤔