Make different artifacts.go included in the release #2554

zeroalphat · 2024-05-01T06:41:24Z

Make different artifacts.go included in the release for different environments.

…ironments. Signed-off-by: zeroalphat <[email protected]>

Signed-off-by: zeroalphat <[email protected]>

masa213f

As I commented elsewhere, please consider whether we can stop using the build tag.

masa213f · 2024-05-07T03:03:58Z

.circleci/config.yml

+  # triggered weekdays at 15:00 UTC(00:00 UTC+9) on the main branch, 
+  # check if a new artefact has been released. 
+  daily-check-new-artifacts:
+    triggers:
+      - schedule:
+          cron: "0 15 * * 1-5"
+          filters:
+            branches:
+              only:
+                - main
+    jobs:
+      - check-new-artifacts


This workflow could be done with the daily workflow. How about merging them?
If we separate this workflow, we need to check the two links (CI result) in the daily confirmation.

morimoto-cybozu · 2024-05-14T08:30:52Z

.circleci/config.yml

+            diffs=$(git diff --name-only main)
+            if [ "$diffs" = "" ]; then printf "%s\n" "$diffs"; exit 0; fi


The comparison seems inverted.

Suggested change

diffs=$(git diff --name-only main)

if [ "$diffs" = "" ]; then printf "%s\n" "$diffs"; exit 0; fi

diffs=$(git diff --name-only main)

if [ "$diffs" != "" ]; then printf "%s\n" "$diffs"; exit 0; fi

And it could be simplified.

Suggested change

diffs=$(git diff --name-only main)

if [ "$diffs" = "" ]; then printf "%s\n" "$diffs"; exit 0; fi

git --no-pager diff --name-only main

I suppose that this run step is just for information.
We don't want to stop the sequence even if there were differences, right?

git diff without --exit-code option returns zero even if there were differences.
You can directly use this command.
In case you didn't know, if git diff returned non-zero, the assignment diffs=$(git diff ...) also returns non-zero and fails.

--no-pager may not be necessary.

morimoto-cybozu · 2024-05-14T12:51:38Z

.circleci/config.yml

      - deploy_github:
+          name: "build release with release prefix"


Suggested change

name: "build release with release prefix"

name: "deploy release with release prefix"

morimoto-cybozu · 2024-05-14T13:18:37Z

docs/generate-artifacts.md

 `generate-artifacts` is a command-line tool to generate `artifacts.go`
 which is a collection of latest components. This is used while


Suggested change

`generate-artifacts` is a command-line tool to generate `artifacts.go`

which is a collection of latest components. This is used while

`generate-artifacts` is a command-line tool to generate `artifacts.go`,

which is a collection of the latest components. This tool is used in development.

It is also used in CI to check differences between the latest release and the versions in use.

morimoto-cybozu · 2024-05-14T13:49:35Z

updater/release_checker.go

-		c.check = github.GetLatestPublishedTag
+		github.SetTagPrefix("test-")
+		c.check = github.GetLatestReleaseTag


~~Do we need to change this?~~

This change should require much more changes in CI workflows, tools, and documents.
Please let me investigate more.

At least the following should be updated:

docs/cicd.md

The steps of "deploy for the staging data center" should be updated to include the following.

Remove the "pre-release" label from the target test- release.

It is still necessary to create a release- tag. This may be done automatically for a released test- release.

It currently lacks a description for the dev data center, i.e., stage1.

download-neco-deb

This should prepare the same neco.deb that is used in the staging data center. We need to use the same algorithm with neco-updater.

I'm not sure whether we should update this just now or delay the update until we release this PR into stage0. Probably the former because this command is used in the reproduced staging environment.

find-installed-release

This should return the same "datetime-rev" release name that is used in the staging data center.

I'm not sure whether we should update this just now or delay the update. Probably the latter because this command is used to reproduce the staging environment.

The caller should be updated too.

I may have overlooked something.

morimoto-cybozu · 2024-05-16T10:02:48Z

generator/generate_test.go

+
+	type args struct {
+		repo            name.Repository
+		release         bool


This is not used because getLatestImage() no longer receives a release parameter.

morimoto-cybozu · 2024-05-16T10:04:46Z

generator/generate_test.go

+		{
+			name: "can set ignored versions that is no-release version",
+			args: args{
+				repo:            testRepository,
+				release:         false,
+				ignoredVersions: []string{"100.0.0"},
+			},
+			tagsList: []byte(`{"tags":["0.1.0","0.2.0", "1.0.0"]}`),
+			img: &neco.ContainerImage{
+				Name:       imageName(testRepository),
+				Repository: testRepository.String(),
+				Tag:        "1.0.0",
+				Private:    false,
+			},
+		},
+		{
+			name: "should get latest image with release",
+			args: args{
+				repo:    testRepository,
+				release: true,
+			},
+			tagsList: []byte(`{"tags":["0.1.0","0.2.0", "1.0.0"]}`),
+			img: &neco.ContainerImage{
+				Name:       imageName(testRepository),
+				Repository: testRepository.String(),
+				Tag:        "1.0.0",
+				Private:    false,
+			},
+		},


These two cases are the same.

morimoto-cybozu · 2024-05-16T10:05:46Z

docs/artifacts.md

@@ -4,24 +4,22 @@ Artifacts
 [Artifacts](../artifacts.go) is the collection of components tested in [dctest](../dctest/), which consists of container images, debian package, and Flatcar image.

 There are two artifacts files.  When generating neco binaries and the neco deb package, `artifacts.go` is used
-by default.  `artifacts_release.go` is used only when `release` build tag is given.
+by default.  `artifacts_prod.go` is used only when `release` build tag is given.

 - [artifacts.go](../artifacts.go)

    This file describes artifacts to be tested in development.


Suggested change

This file describes artifacts to be tested in development.

This file describes artifacts to be tested in development and staging.

morimoto-cybozu · 2024-05-16T10:06:41Z

docs/artifacts.md


-    This file is for CI.  The CI job updates this file by using
-    `generate-artifacts --release`.
+    This file describes artifacts to be tested in production.


Suggested change

This file describes artifacts to be tested in production.

This file describes artifacts to be used in production.

morimoto-cybozu · 2024-05-16T10:08:22Z

docs/artifacts.md

    The developers are PROHIBITED to modify this file.  This is to prevent
    merge conflicts in CI flow.


I don't think it is prohibited so strictly now.

zeroalphat self-assigned this May 1, 2024

zeroalphat force-pushed the improve-release-flow branch from 5d80f1c to 206a652 Compare May 2, 2024 00:24

Make different artifacts.go included in the release for different env…

3b9026e

…ironments. Signed-off-by: zeroalphat <[email protected]>

zeroalphat force-pushed the improve-release-flow branch from 206a652 to 3b9026e Compare May 2, 2024 00:40

zeroalphat marked this pull request as ready for review May 2, 2024 00:40

zeroalphat added 2 commits May 2, 2024 09:52

Fix circle ci config

2ac771d

Signed-off-by: zeroalphat <[email protected]>

Update CircleCI config to filter jobs based on the main branch

1f2c0b3

Signed-off-by: zeroalphat <[email protected]>

zeroalphat requested review from morimoto-cybozu and masa213f May 2, 2024 02:14

masa213f reviewed May 9, 2024

View reviewed changes

morimoto-cybozu reviewed May 14, 2024

View reviewed changes

morimoto-cybozu reviewed May 16, 2024

View reviewed changes

zeroalphat closed this Sep 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make different artifacts.go included in the release #2554

Make different artifacts.go included in the release #2554

zeroalphat commented May 1, 2024

masa213f left a comment

masa213f May 7, 2024

morimoto-cybozu May 14, 2024

morimoto-cybozu May 14, 2024

morimoto-cybozu May 14, 2024

morimoto-cybozu May 14, 2024 •

edited

Loading

morimoto-cybozu May 16, 2024

morimoto-cybozu May 16, 2024

morimoto-cybozu May 16, 2024

morimoto-cybozu May 16, 2024

morimoto-cybozu May 16, 2024

morimoto-cybozu May 16, 2024

		diffs=$(git diff --name-only main)
		if [ "$diffs" = "" ]; then printf "%s\n" "$diffs"; exit 0; fi

	diffs=$(git diff --name-only main)
	if [ "$diffs" = "" ]; then printf "%s\n" "$diffs"; exit 0; fi
	git --no-pager diff --name-only main

	name: "build release with release prefix"
	name: "deploy release with release prefix"

		`generate-artifacts` is a command-line tool to generate `artifacts.go`
		which is a collection of latest components. This is used while

-`generate-artifacts` is a command-line tool to generate `artifacts.go`
-which is a collection of latest components. This is used while
+`generate-artifacts` is a command-line tool to generate `artifacts.go`,
+which is a collection of the latest components. This tool is used in development.
+It is also used in CI to check differences between the latest release and the versions in use.

	This file describes artifacts to be tested in development.
	This file describes artifacts to be tested in development and staging.

	This file describes artifacts to be tested in production.
	This file describes artifacts to be used in production.

		The developers are PROHIBITED to modify this file. This is to prevent
		merge conflicts in CI flow.

Make different artifacts.go included in the release #2554

Make different artifacts.go included in the release #2554

Conversation

zeroalphat commented May 1, 2024

masa213f left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

morimoto-cybozu May 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

morimoto-cybozu May 14, 2024 •

edited

Loading