Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mepo3 - hierarchical mepo #274

Closed
wants to merge 31 commits into from
Closed

mepo3 - hierarchical mepo #274

wants to merge 31 commits into from

Conversation

pchakraborty
Copy link
Collaborator

@pchakraborty pchakraborty commented Apr 9, 2024 •
edited
Loading

@tclune @amdasilva @mathomp4

The first draft of hierarchical mepo (mepo3) is ready. The main difference is how clone works - instead of a flat components.yaml, mepo3 reads components.yaml of each sub-repository, when available, and does a recursive cloning.

This codebase also addresses some of the concerns mentioned in #273.

Installation

Either

git clone -b feature/pchakrab/hierarchical-mepo [email protected]:GEOS-ESM/mepo.git mepo3
export PATH=$PATH:/path/to/mepo3/bin

Or, in your python3 virtualenv

pip install -i https://test.pypi.org/simple/ mepo3

Testing

mepo3 clone https://github.com/pchakraborty/GEOSfvdycore.git

This fork of GEOSfvdycore implements hierarchical components.yaml that mepo3 can read and clone. Most of the other mepo commands work as before.

TODO

  • Update mepo3 save.

@pchakraborty
Maybe we don't need the MepoState dependency
2450ba9
@pchakraborty
Merge remote-tracking branch 'origin/develop' into feature/pchakrab/h…
ddd9860 
…ierarchical-mepo
@pchakraborty
Towards hierarchical mepo
82aa552 
`mepo clone` reads components.yaml recursively and clones all the sub-repos. MepoState().initialize() and consequently `mepo init` do not exist any more. Mepo state is created during the execution of `mepo clone`.
@pchakraborty
Renamed __recursive_clone__ -> __recursive_clone
df2cb07
@pchakraborty
Added argument 'partial' to the recursive cloning routine
6d04470
@pchakraborty
Tests work with hierarchical cloning
cca5030
@pchakraborty
Towards a more standard Python module
84089e7 
Added src/mepo/__init__.py
Fewer directories, e.g. renamed command/compare/compare.py -> command/compare.py
Using relative imports inside package
@pchakraborty
Some more renaming: command/commit/commit.py -> command/commit.py
c9941f0
@pchakraborty
Using Python3's relative import
131f67c
@pchakraborty
Some more renaming, e.g command/config/config.py -> command/config.py
4157b17
@pchakraborty
Last round of renaming, e.g. save/save.py -> save.py
42e3d33
@pchakraborty
Renamed config/config_file.py -> registry.py
5a7eef1
@pchakraborty
Using Python3's relative import
43c20f2
@pchakraborty
Using Python3's relative import
e9144fe
@pchakraborty
Removed utilities/__init__.py
22a64c6
@pchakraborty
Fixed tests
239ba1f
@pchakraborty
Updated required Python version in README.md
5668fa7
@pchakraborty
Renamed main.py as __main__.py, and related changes
af8ea63
@pchakraborty
Moved state/exceptions.py to utilities
0264e98
@pchakraborty
Renamed repository/git.py -> git.py, and associated changes
27ca42f
@pchakraborty
Moved state.py and component.py one level up, and associated changes
d91e884
@pchakraborty
Fixed tests
c16bbec
@pchakraborty
Fixed the helper script
3c19f0e
@pchakraborty
Fixed doc generation
8c9a8a9
@pchakraborty
Switched from https to ssh for git cloning
fd9921c
@pchakraborty
Back to https but with correct address this time
4324e08
@pchakraborty
Updated stored test outputs
7d71a03
@pchakraborty
Renamed top level script as mepo3
eb24932
@pchakraborty
Completed mepo to mepo3 conversion
412d4e1 
Renamed src/mepo -> src/mepo3
Updated doc generation
Updated pyproject.toml
@pchakraborty
In code, use mepo state dir as specified my MepoState, instead of har…
df051fe 
…dcoding the directory name
@tclune tclune self-requested a review April 10, 2024 14:39
tclune
tclune reviewed Apr 10, 2024
View reviewed changes
Copy link
Contributor

@tclune tclune left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we merge this into develop, then it will prevent further development of "old-mepo".

Needs thought, or else we need a hepo-dev branch for the interim.

tclune
tclune reviewed Apr 10, 2024
View reviewed changes
src/mepo3/cmdline/parser.py
nargs = '?',
default = None,
help = 'Configuration file (ignored if init already called)')
default = 'components.yaml',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies if this has been handled, but I fear it may be hard to glean directly from staring at the code changes. For hepo, we want the config file to be hidden in a subdirectory. E.g., .hepo/config.yml.

tclune
tclune reviewed Apr 10, 2024
View reviewed changes
Copy link
Contributor

@tclune tclune left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better to have a separate PR for just the python packaging changes. This can safely go into mepo dev. The logic for config files in sub-repositories can/should be a separate PR.

@pchakraborty
Copy link
Collaborator Author

Closing it in favor of #278

@pchakraborty pchakraborty deleted the feature/pchakrab/hierarchical-mepo branch April 16, 2024 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tclune tclune tclune left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@pchakraborty @tclune