Modify providers release process for transition to trusted publishing #44941
Conversation
|
I'll review tommorow |
| ## Publish the packages to PyPI | ||
|
|
||
| By that time the packages should be in your dist folder. | ||
|
|
||
| ```shell script | ||
| cd ${AIRFLOW_REPO_ROOT} | ||
| git checkout <ONE_OF_THE_RC_TAGS_FOR_ONE_OF_THE_RELEASED_PROVIDERS> | ||
| ``` | ||
|
|
||
| example `git checkout providers-amazon/7.0.0rc2` | ||
|
|
||
| Note you probably will see message `You are in 'detached HEAD' state.` | ||
| This is expected, the RC tag is most likely behind the main branch. | ||
|
|
||
| * Verify the artifacts that would be uploaded: | ||
|
|
||
| ```shell script | ||
| twine check ${AIRFLOW_REPO_ROOT}/dist/*.whl ${AIRFLOW_REPO_ROOT}/dist/*.tar.gz | ||
| ``` | ||
|
|
||
| * Upload the package to PyPi: | ||
|
|
||
| ```shell script | ||
| twine upload -r pypi ${AIRFLOW_REPO_ROOT}/dist/*.whl ${AIRFLOW_REPO_ROOT}/dist/*.tar.gz | ||
| ``` | ||
|
|
||
| * Verify that the packages are available under the links printed. | ||
|
|
||
| Copy links to updated packages, sort it alphabetically and save it on the side. You will need it for the announcement message. | ||
|
|
||
| * Again, confirm that the packages are available under the links printed. |
There was a problem hiding this comment.
I am not sure why this is removed.
Or at least why we are changing the order of the steps in the release. I find it confusing.
There was a problem hiding this comment.
It's not removed - it's moved up. And we are changing the order because we want to also publish the PyPI packages in SVN. So far we have not been doing it - because we did not need them in SVN (and it was a bit problematic as well because RC PyPI packages never went through our SVN and were never available there nor available in the history.
With "publish" workflow we want to make sure that packages are committed to SVN (including the PyPI packages) - that's why we are moving the "prepare" step before.
And since we eventually want to replace the "publish" step from manual to workflow, it makes sense to put the "manual" step next to the "workflow" test.
I think it could be also left below, but I found it cleaner if we have "publish" steps manual/workflow next to each other.
There was a problem hiding this comment.
Let me know @eladkal if that explains it :)
There was a problem hiding this comment.
I will make the release according to instructions in this PR, that way we can change it on the fly then merge the final version that actually works. It will be easier to write questions/comments on the fly when the PR is up and change it as we progress
There was a problem hiding this comment.
cool :) Sounds like a plan!
| # Move the artifacts to svn folder | ||
| mv ${AIRFLOW_REPO_ROOT}/dist/* . | ||
|
|
||
| # Add and commit | ||
| svn add * |
There was a problem hiding this comment.
This will not work.
We already moved from dist to release the packages to PyPi. When you get to this step dist is empty.
There was a problem hiding this comment.
Actually not. We moved the "SVN" packages to SVN, the "PyPI" packages are created again few lines before:
breeze release-management prepare-provider-packages \
--version-suffix-for-pypi rc1 --package-format both PACKAGE PACKAGE ....
Those will have rc1 extension unlike SVN packages which might become the final release.
BTW. We should likely change names of those "types" of packages - previously for release candidates we commited only "SVN" packages to "SVN" and "PyPI" packages for PyPI - but now both will be committed to SVN.
So we likely need new names. Maybe:
- "no-suffix" packages
- "suffix" packages
?
potiuk
force-pushed
the
modify-providers-release-process-for-trusted-publishing-transition
branch
from
fd14f8d to
adeb27d
Compare
potiuk
force-pushed
the
modify-providers-release-process-for-trusted-publishing-transition
branch
from
adeb27d to
44ed914
Compare
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.