Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pretty printing catalog #3990

Merged
merged 48 commits into from
Jul 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
502f982
Implemented basic __repr__
ElenaKhaustova Jul 3, 2024
9aa704f
Updated __repr__
ElenaKhaustova Jul 3, 2024
45b51e8
Removed __str__
ElenaKhaustova Jul 3, 2024
94e6465
Updated _describe() for CachedDataset
ElenaKhaustova Jul 3, 2024
e2884e7
Made pretty_repr protected
ElenaKhaustova Jul 3, 2024
248a2c0
Merge branch 'main' into feature/3980-pretty-printing-dataset
ElenaKhaustova Jul 4, 2024
aea8ec0
Reverted width parameter to default
ElenaKhaustova Jul 4, 2024
413481a
Implemented repr for catalog
ElenaKhaustova Jul 4, 2024
2c9a7bb
Disable sorting
ElenaKhaustova Jul 4, 2024
7f3019f
Replace set with dict to keep original datasets order when printing
ElenaKhaustova Jul 4, 2024
659de36
Updated printing params
ElenaKhaustova Jul 4, 2024
d1cb03f
Updated printing width
ElenaKhaustova Jul 4, 2024
a3a0c38
Merge branch 'feature/3980-pretty-printing-dataset' into feature/3981…
ElenaKhaustova Jul 4, 2024
c85c92e
Removed params_repr
ElenaKhaustova Jul 4, 2024
4de32c4
Disable sorting
ElenaKhaustova Jul 4, 2024
9e6bae9
Disable sorting
ElenaKhaustova Jul 4, 2024
eb6f148
Merge branch 'feature/3980-pretty-printing-dataset' into feature/3981…
ElenaKhaustova Jul 4, 2024
87dd387
Merge branch 'main' into feature/3981-pretty-printing-catalog
ElenaKhaustova Jul 8, 2024
478df5c
Disabled compact
ElenaKhaustova Jul 8, 2024
57f9a3f
Merge branch 'main' into feature/3980-pretty-printing-dataset
ElenaKhaustova Jul 8, 2024
e61a775
Updated test_str_representation
ElenaKhaustova Jul 8, 2024
0f84735
Updated cached dataset tests
ElenaKhaustova Jul 8, 2024
3a7e748
Updated data catalog tests
ElenaKhaustova Jul 8, 2024
216cb42
Updated core tests
ElenaKhaustova Jul 8, 2024
ca42da1
Updated versioned dataset tests
ElenaKhaustova Jul 8, 2024
924b53e
Updated tests for lambda dataset
ElenaKhaustova Jul 8, 2024
f529c4b
Updated tests for memory dataset
ElenaKhaustova Jul 8, 2024
08a5f0f
Merge branch 'feature/3980-pretty-printing-dataset' into feature/3981…
ElenaKhaustova Jul 8, 2024
967c43f
Updated release notes
ElenaKhaustova Jul 8, 2024
7754078
Set width to maxsize
ElenaKhaustova Jul 9, 2024
14f237f
Removed top-level keys sorting
ElenaKhaustova Jul 9, 2024
cfac9d6
Updated tests
ElenaKhaustova Jul 9, 2024
bfc0841
Merge branch 'main' into feature/3980-pretty-printing-dataset
ElenaKhaustova Jul 9, 2024
69a2229
Merge branch 'feature/3980-pretty-printing-dataset' into feature/3981…
ElenaKhaustova Jul 9, 2024
7190b3c
Merge branch 'main' into feature/3980-pretty-printing-dataset
ElenaKhaustova Jul 9, 2024
54964f8
Updated release notes
ElenaKhaustova Jul 9, 2024
410c150
Merge branch 'feature/3980-pretty-printing-dataset' into feature/3981…
ElenaKhaustova Jul 9, 2024
1b0ae7a
Decoupled describe from pretty printing
ElenaKhaustova Jul 12, 2024
c6a8a31
Merge branch 'main' into feature/3980-pretty-printing-dataset
ElenaKhaustova Jul 12, 2024
22aec50
Returned old __str__ to avoid a breaking change
ElenaKhaustova Jul 12, 2024
28a961d
Updated tests
ElenaKhaustova Jul 12, 2024
6edc5b2
Merge branch 'feature/3980-pretty-printing-dataset' into feature/3981…
ElenaKhaustova Jul 12, 2024
d5cd26b
Merge branch 'main' into feature/3980-pretty-printing-dataset
ElenaKhaustova Jul 16, 2024
d7f50aa
Replaced deprecation comment with TODO
ElenaKhaustova Jul 16, 2024
f1b1435
Merge branch 'feature/3980-pretty-printing-dataset' into feature/3981…
ElenaKhaustova Jul 16, 2024
16f04b4
Merge branch 'main' into feature/3981-pretty-printing-catalog
ElenaKhaustova Jul 17, 2024
43d569d
Merge branch 'main' into feature/3981-pretty-printing-catalog
ElenaKhaustova Jul 18, 2024
ef3b2c0
Merge branch 'main' into feature/3981-pretty-printing-catalog
ElenaKhaustova Jul 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
* Implemented key completion support for accessing datasets in the `DataCatalog`.
* Made [kedro-telemetry](https://github.com/kedro-org/kedro-plugins/tree/main/kedro-telemetry) a core dependency.
* Implemented dataset pretty printing.
* Implemented `DataCatalog` pretty printing.

## Breaking changes to the API

Expand Down
19 changes: 16 additions & 3 deletions kedro/io/data_catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import copy
import difflib
import logging
import pprint
import re
from typing import Any, Dict

Expand Down Expand Up @@ -106,7 +107,7 @@ def __init__(
"""Return a _FrozenDatasets instance from some datasets collections.
Each collection could either be another _FrozenDatasets or a dictionary.
"""
self._original_names: set[str] = set()
self._original_names: dict[str, str] = {}
for collection in datasets_collections:
if isinstance(collection, _FrozenDatasets):
self.__dict__.update(collection.__dict__)
Expand All @@ -116,7 +117,7 @@ def __init__(
# for easy access to transcoded/prefixed datasets.
for dataset_name, dataset in collection.items():
self.__dict__[_sub_nonword_chars(dataset_name)] = dataset
self._original_names.add(dataset_name)
self._original_names[dataset_name] = ""

# Don't allow users to add/change attributes on the fly
def __setattr__(self, key: str, value: Any) -> None:
Expand All @@ -131,11 +132,20 @@ def __setattr__(self, key: str, value: Any) -> None:
raise AttributeError(msg)

def _ipython_key_completions_(self) -> list[str]:
return list(self._original_names)
return list(self._original_names.keys())

def __getitem__(self, key: str) -> Any:
return self.__dict__[_sub_nonword_chars(key)]

def __repr__(self) -> str:
datasets_repr = {}
for ds_name in self._original_names.keys():
datasets_repr[ds_name] = self.__dict__[
_sub_nonword_chars(ds_name)
].__repr__()

return pprint.pformat(datasets_repr, sort_dicts=False)


class DataCatalog:
"""``DataCatalog`` stores instances of ``AbstractDataset`` implementations
Expand Down Expand Up @@ -207,6 +217,9 @@ def __init__( # noqa: PLR0913
if feed_dict:
self.add_feed_dict(feed_dict)

def __repr__(self) -> str:
return self.datasets.__repr__()

@property
def _logger(self) -> logging.Logger:
return logging.getLogger(__name__)
Expand Down