Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release tool] Start go build and release tooling #8935

Merged
merged 56 commits into from
Aug 7, 2024

Conversation

radTuti
Copy link
Contributor

@radTuti radTuti commented Jun 22, 2024

Description

This is the core base of the go release tooling that will replace useage of Makefile and hack/release for a more cohesive process. It tries to adhere to standard go project layout

The borrows from the town in Bob the Builder: Ready Steady Build

Related issues/PRs

N/A

Todos

  • Tests
  • Documentation
  • Release note

Release Note

I'm going to go with not required atm until it gets intergrated

TBD

Reminder for the reviewer

Make sure that this PR has the correct labels and milestone set.

Every PR needs one docs-* label.

  • docs-pr-required: This change requires a change to the documentation that has not been completed yet.
  • docs-completed: This change has all necessary documentation completed.
  • docs-not-required: This change has no user-facing impact and requires no docs.

Every PR needs one release-note-* label.

  • release-note-required: This PR has user-facing changes. Most PRs should have this label.
  • release-note-not-required: This PR has no user-facing changes.

Other optional labels:

  • cherry-pick-candidate: This PR should be cherry-picked to an earlier release. For bug fixes only.
  • needs-operator-pr: This PR is related to install and requires a corresponding change to the operator.

- split DockerGoBuild to allow better reusability
- reorganize file structure to adhere to gostandards
- update reusable goyek section + rename
@radTuti radTuti added docs-not-required Docs not required for this change release-note-not-required Change has no user-facing impact labels Jun 22, 2024
@radTuti radTuti requested a review from a team as a code owner June 22, 2024 00:30
@marvin-tigera marvin-tigera added this to the Calico v3.29.0 milestone Jun 22, 2024
fixham/go.mod Outdated Show resolved Hide resolved
fixham/pkg/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
fixham/pkg/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
fixham/pkg/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
fixham/pkg/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
fixham/pkg/bootstrap/test.go Outdated Show resolved Hide resolved
fixham/pkg/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
fixham/build/main.go Outdated Show resolved Hide resolved
fixham/internal/client/client.go Outdated Show resolved Hide resolved
fixham/internal/client/client.go Outdated Show resolved Hide resolved
@radTuti radTuti force-pushed the go-release-pkg branch 4 times, most recently from 40970c2 to ea3550f Compare June 25, 2024 05:08
release/pkg/tasks/clean.go Show resolved Hide resolved
release/pkg/tasks/clean.go Outdated Show resolved Hide resolved
release/pkg/tasks/hashrelease.go Outdated Show resolved Hide resolved
release/pkg/tasks/hashrelease.go Outdated Show resolved Hide resolved
@radTuti radTuti mentioned this pull request Jul 30, 2024
3 tasks
- remove/address TODO comments
- use embed for template files
- update task names
- update sshn
- add release branch validations
- add steps for non-release branch
outstandings

- push operator manifest list
- publish hashrelease using rsync
@radTuti
Copy link
Contributor Author

radTuti commented Aug 1, 2024

Some outstanding issues that I am hoping to defer to a new PR

  • Publish
    • Creating Operator manifest list
    • Using command runner for rsync

Some additional optimization

  • Use types/reference for/ instead of the Image struct (learnt about it while trying to get manifest list working)

EDIT: crossed out items are in this PR

@radTuti radTuti force-pushed the go-release-pkg branch 2 times, most recently from d73d11f to 71538ee Compare August 1, 2024 20:10
release/internal/config/config.go Outdated Show resolved Hide resolved
for scanner.Scan() {
line := scanner.Text()
for _, name := range hashreleaseNames {
if !strings.Contains(line, name) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a little bit fragile. Instead of trying to make a diff of the existing file and the folders we just deleted, why can't we just build a new library file from the folders we know exist (i.e., all of the folders we didn't delete) and just overwrite what is there?

This would save us needing to fetch the current file, and would also save us needing to try and compare two sets of data (which is error prone).

Copy link
Contributor Author

@radTuti radTuti Aug 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered that but it will still result in making a diff still because the content is in the format <hash> - <note> which we do have ability to recreate

release/internal/outputs/releasenotes.go Outdated Show resolved Hide resolved
release/internal/utils/files.go Show resolved Hide resolved
release/pkg/tasks/clean.go Show resolved Hide resolved
@radTuti radTuti force-pushed the go-release-pkg branch 2 times, most recently from b4a389f to eb9f2cf Compare August 2, 2024 00:00
…try URL

+ simplify to use default docker config
- update DevTagSuffix & RepoReleaseBranchPrefix config to be more consistent with rest of repo
- Move RepoDefaultBranch from config to const in utils package
- Add comments to where missing
Copy link
Member

@caseydavenport caseydavenport left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Plenty more to do here but I think we should get this one merged so we can get incremental improvements on top of it.

I think the most important next steps are:

  • Actually switching hashreleases over to using this logic for OSS, disabling the old system.
  • Incorporating the hack/release logic into this tooling to build "real" releases as well.
  • Sharing as much code as possible between hashrelease / release.

In addition, some other things that we should track as tasks:

  • Fix that Calico clones operator, which then clones Calico again as part of gen-versions.
  • Consider hosting hashreleases elsewhere - e.g., S3

@radTuti
Copy link
Contributor Author

radTuti commented Aug 2, 2024

/merge-when-ready squash-commits

@caseydavenport caseydavenport merged commit 5c9ab1b into projectcalico:master Aug 7, 2024
1 of 2 checks passed
@radTuti radTuti deleted the go-release-pkg branch August 13, 2024 18:28
@radTuti radTuti changed the title Start go build and release tooling [release-tool] Start go build and release tooling Aug 22, 2024
@radTuti radTuti changed the title [release-tool] Start go build and release tooling [release tool] Start go build and release tooling Aug 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-not-required Docs not required for this change release-note-not-required Change has no user-facing impact
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants