-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update packaging tools and logic #398
Comments
Seems benchcab is having similar issues Would be good to have a consistent approach. Any opinions @SeanBryan51? |
Another option for the versioning is to use the hatchling build system and hatch-vcs, which |
This is an interesting read that explains some of the differences between packaging for PyPI and conda https://pyopensci.discourse.group/t/suggestions-for-a-conda-user-preparing-a-package-for-pypi/324/2 |
If changing packaging tools is in scope, this is a helpful resource comparing a number of modern ones (including Hatch).
Maybe also worth mentioning |
Unassigned myself from this for now, but will come back to it later! |
Sorry for being dense, but what is it exactly that you want to achieve here? Consolidate the
When I've done this for other projects, I've still needed both a Or is this idea to have the dependencies defined in one place and have the |
Regarding versioning, the obvious thing to use is |
No the fault is mine. The issue is not well formulated and adds in additional problems/information in an unhelpful way. It became apparent that the So the initial motivation was to fix this issue, which seemed to imply some coalescing of the PyPI and
Yes I was a bit naive and just assumed this was possible. So good to know. So the goal should be to make as much as possible shared by PyPI (effectively
It seems like a decent idea, but I'd prefer something standard and well understood, over something bespoke that might be more fragile or more difficult to maintain. So number 1 problem to solve is consistent versioning between PyPI and It seems @SeanBryan51 has opted for https://github.com/CABLE-LSM/benchcab/pull/229/files Does this imply moving to |
That will be simplest/fastest if I'm the one doing it - see my comment above.
It's not needed, but now seems like a good time to do it. I'm not sure what you mean by "specify things like entry points twice"? So, to do:
Anything else? |
Re: Versioneer
If you have time to do it that would be fantastic. Thanks. @jo-basevi is interested to learn more about python packaging so would be happy to assist, review code etc.
Entry points are currently defined in https://github.com/payu-org/payu/blob/master/conda/meta.yaml#L9-L17 and https://github.com/payu-org/payu/blob/master/setup.py#L59-L70C7 Which seems like useless (and potentially harmful) duplication.
See above. Is it possible for Also if it isn't possible I don't think it is worth delaying for. Entry points are changed relatively infrequently, so we can just live with it. Specifying build dependencies in a single location is a "nice to have", but there are arguments that this is necessary, and it may indeed be advantageous to be able to separate them. Not sure it is applicable in this case. If we can't manage it, I'd settle for some CI that checks they're consistent. |
Maybe not today, but probably tomorrow and definitely some time this week.
Great!
Ah right, of course. I think we can avoid this if we conda build from PyPI dist. |
Awesome! Someone give this man a pay-rise. |
Damn. PyPI publishing failed: https://github.com/payu-org/payu/actions/runs/7606639931/job/20712742002#step:4:15
|
We're getting weird "dirty" tags in readthedocs even though "proper" tags exist: From this issue I think the problem is that even though the
was removed from https://github.com/payu-org/payu/blob/master/payu/_version.py#L54 I really should do a better job reviewing ... |
Rats. This issue just won't stay closed. I think you're right. PR inbound. |
And stay closed! |
Can confirm, still closed. |
payu
deploys topypi
andconda
, but the configuration metadata is stored in separate files:https://github.com/payu-org/payu/blob/master/setup.py
https://github.com/payu-org/payu/blob/master/conda/meta.yaml
It seems this is a well known problem. Some solutions include:
https://github.com/basnijholt/unidep
https://stackoverflow.com/a/76722681
@dougiesquire suggested we could even utilise the PyPI build in
conda
packaging #362 but this left for later. Later has arrived.The "modern" approach is to use a
pyproject.toml
file to specify dependenciesRelated issue, the version is hard-coded:
https://github.com/payu-org/payu/blob/master/payu/__init__.py#L6
and used in a few locations
https://github.com/search?q=repo%3Apayu-org%2Fpayu%20__version__&type=code
so the pypi distribution only installs the version in that file. Hard-coding the version string in a file is a bad approach when using
git
tags for versioning, as it requires putting the version in the file, then tagging with the version in the file as a subsequent step. Nasty.There are a number of options for "single-sourcing" the package version:
https://packaging.python.org/en/latest/guides/single-sourcing-package-version/#single-sourcing-the-version
But option 7 is the one that is most compatible with using
git
tags for versioning, and it uses setuptools_scm.NB: A couple of the places the hard-coded version string is used are in some old install scripts that should be removed (see #396).
The text was updated successfully, but these errors were encountered: