Adding HLG to MAP #8740
Adding HLG to MAP #8740
Conversation
Unit Test ResultsSee test report for an extended history of previous test failures. This is useful for diagnosing flaky tests. 29 files ±0 29 suites ±0 11h 18m 6s ⏱️ - 1m 50s For more details on these failures, see this check. Results for commit 2eb19b7. ± Comparison against base commit 670fbfb. ♻️ This comment has been updated with latest results. |
The MapLayer constructor has been refactored to handle nested iterables. This allows for more flexibility when passing in iterables to the MapLayer class. The changes include: - Checking if the iterables are nested and converting them into a list of tuples if necessary. - Updating the logic for generating keys based on the nested iterables. - Modifying the construction of the low-level graph to handle the nested iterables. These changes improve the usability and versatility of the MapLayer class.
There was a problem hiding this comment.
Codecov is very unhappy, this could be related to this error: https://github.com/dask/distributed/actions/runs/9828037658/job/27131436761?pr=8740#step:20:76
| @@ -807,6 +809,116 @@ class VersionsDict(TypedDict): | |||
| client: dict[str, dict[str, Any]] | |||
|
|
|||
|
|
|||
| _T_LowLevelGraph: TypeAlias = dict[Key, tuple] | |||
There was a problem hiding this comment.
Out of scope: We should probably move this into dask.typing now that we've added it in multiple places.
There was a problem hiding this comment.
would that be adding here? There is a FIXME here, seems like we'd want to add a _T_LowLevelGraph and _T_HighLevelGraph types?
There was a problem hiding this comment.
This should match the dask.typing.Graph type but I'm not very keen of this type alias. I suggest to not do anything for now
| @@ -807,6 +809,116 @@ class VersionsDict(TypedDict): | |||
| client: dict[str, dict[str, Any]] | |||
|
|
|||
|
|
|||
| _T_LowLevelGraph: TypeAlias = dict[Key, tuple] | |||
There was a problem hiding this comment.
This should match the dask.typing.Graph type but I'm not very keen of this type alias. I suggest to not do anything for now
Co-authored-by: Florian Jetter <[email protected]>
Co-authored-by: Florian Jetter <[email protected]>
Closes #8706
pre-commit run --all-files