[BUG] [flytekit] cannot pass dataclass with Optional[list] to FlyteRemote.execute

[gvogel-hh]

Describe the bug

When trying to pass a dataclass containing an optional list as input to FlyteRemote.execute(), a KeyError: 'items' is raised at flytekit/core/type_engine.py:839 in generate_attribute_list_from_dataclass_json_mixin (flytekit==1.13.14).

Expected behavior

Workflow is started with the given dataclass instance.

Additional context to reproduce

from dataclasses import dataclass
from typing import Optional

from flytekit import task, workflow
from flytekit.configuration import Config
from flytekit.remote import FlyteRemote
from mashumaro.mixins.json import DataClassJSONMixin


@dataclass
class MyConfig(DataClassJSONMixin):
    op_list: Optional[list[str]]


@task
def t() -> None:
    pass


@workflow
def wf(config: MyConfig) -> None:
    t()


if __name__ == "__main__":
    config = MyConfig(op_list=["a", "b"])
    remote = FlyteRemote(
        config=Config.auto(config_file="../config.yaml"),
        default_project="flytesnacks",
        default_domain="development",
    )

    remote_wf = remote.fetch_workflow(name="mywf.wf")
    execution = remote.execute(remote_wf, inputs={"config": config})

Screenshots

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[welcome]

Thank you for opening your first issue here! 🛠

[kumare3]

@gvogel-hh the entire JSON subsystem is getting a major update in 1.14. cc @Future-Outlier
Most of these issues will be fixed.

[gvogel-hh]

It seems the problem persists in 1.14.0. I could work around it by patching like

diff --git a/flytekit/core/type_engine.py b/flytekit/core/type_engine.py
index 7c84191b1..1afc9e661 100644
--- a/flytekit/core/type_engine.py
+++ b/flytekit/core/type_engine.py
@@ -1094,7 +1094,8 @@ def generate_attribute_list_from_dataclass_json_mixin(schema: dict, schema_name:
             property_type = property_val["type"]
         # Handle list
         if property_type == "array":
-            attribute_list.append((property_key, typing.List[_get_element_type(property_val["items"])]))  # type: ignore
+            val = property_val["anyOf"][0] if "anyOf" in property_val else property_val
+            attribute_list.append((property_key, typing.List[_get_element_type(val["items"])]))  # type: ignore
         # Handle dataclass and dict
         elif property_type == "object":
             if property_val.get("anyOf"):

(I also had to change list to List in order to avoid TypeTransformerFailedError: Type of Val 'list[str]' is not an instance of typing.List[str].)

[eapolinario]

@Future-Outlier , can you take a look?