Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revamp build/CI #1097

Merged
merged 3 commits into from
Apr 5, 2023
Merged

Revamp build/CI #1097

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9cf6bee
Revamp build/CI
basil Apr 5, 2023
ef12c57
Merge branch 'master' into revamp-build-ci
timja Apr 5, 2023
f23c2bf
Fix merge
timja Apr 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,19 @@ if (needSplittingFromWorkspace) {

branches['CI'] = {
stage('CI') {
node('maven-11') {
checkout scm
sh "mvn verify --no-transfer-progress -DskipTests -P jenkins-release"
retry(count: 2, conditions: [kubernetesAgent(handleNonKubernetes: true), nonresumable()]) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consistent with core and BOM: even though in practice these are Kubernetes agents, we respect the contract provided by the infrastructure team by not relying on this implementation detail.

node('maven-11') {
checkout scm
def mavenOptions = [
'-Dset.changelist',
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needed for producing incrementals.

'-DskipTests',
'-U',
'clean',
'install',
Comment on lines +44 to +45
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needed for infra.prepareToPublishIncrementals() (as opposed to verify).

]
infra.runMaven(mavenOptions, 11)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Internally respects MAVEN_SETTINGS, preparing for the potential use of the artifact caching proxy.

infra.prepareToPublishIncrementals()
}
}
}
}
Expand Down Expand Up @@ -71,7 +81,7 @@ for (int i = 0; i < splits.size(); i++) {
set-java.sh $javaVersion
eval \$(vnc.sh)
java -version
run.sh firefox ${jenkinsUnderTest} -Dmaven.test.failure.ignore=true -DforkCount=1 -B
run.sh firefox ${jenkinsUnderTest} -Dmaven.repo.local=${WORKSPACE_TMP}/m2repo -Dmaven.test.failure.ignore=true -DforkCount=1 -B
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See jenkinsci/bom#1842.

"""
}
}
Expand All @@ -83,3 +93,4 @@ for (int i = 0; i < splits.size(); i++) {
}
}
parallel branches
infra.maybePublishIncrementals()
62 changes: 62 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
<trimStackTrace>false</trimStackTrace> <!-- surefire -->
<spotbugs.skip>true</spotbugs.skip>

<jenkins.version>2.399</jenkins.version>
<selenium.version>4.8.1</selenium.version>
<guava.version>31.1-jre</guava.version> <!-- aligned with selenium -->
<aether.version>1.1.0</aether.version>
Expand Down Expand Up @@ -362,6 +363,24 @@
<version>1.12.14</version>
<scope>test</scope>
</dependency>
<!--
This is just for dependency updates.
TODO: Delete this dependency and figure out how to automatically update
this dependency some other way.
-->
<dependency>
<groupId>org.jenkins-ci.main</groupId>
<artifactId>jenkins-core</artifactId>
<version>${jenkins.version}</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
Comment on lines +358 to +375
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copying the trick I did in plugin-compat-tester to define an unused test-scoped dependency to get Dependabot to automatically update jenkins.version (actually used further below by the copy mojo, which Dependabot can't update). I have confirmed this trick works. I am not interested in learning how to use any other dependency updating tools. I trust that those who are enthusiastic about other tools can find a better way to do this after this PR lands (or stick with this solution, which is "good enough", if they are not interested).

<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
Expand Down Expand Up @@ -788,5 +807,48 @@ and
</plugins>
</build>
</profile>
<profile>
<id>download-war</id>
<activation>
<file>
<missing>${env.JENKINS_WAR}</missing>
</file>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>jenkins-war</id>
<goals>
<goal>copy</goal>
</goals>
<phase>process-test-resources</phase>
<configuration>
<artifactItems>
<artifactItem>
<groupId>org.jenkins-ci.main</groupId>
<artifactId>jenkins-war</artifactId>
<version>${jenkins.version}</version>
<type>war</type>
</artifactItem>
</artifactItems>
<outputDirectory>${project.build.directory}</outputDirectory>
<stripVersion>true</stripVersion>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
Comment on lines +807 to +843
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are reading this PR from top to bottom, this hunk will only make sense after you have read the changes to run.sh.

<profile>
<id>lts</id>
<properties>
<jenkins.version>2.387.1</jenkins.version>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be updated by the release lead after shipping an LTS release, just like in BOM.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed #1098 for later improvement

</properties>
</profile>
</profiles>
</project>
71 changes: 31 additions & 40 deletions src/main/resources/ath-container/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,30 @@ if [[ $# -lt 2 ]]; then
It can use jenkins.war from local maven repository or download it when missing.

BROWSER: Value for BROWSER variable
JENKINS: Path to the jenkins.war, Jenkins version of one of "latest", "latest-rc", "lts" and "lts-rc"
JENKINS: Path to the Jenkins WAR, URL to the Jenkins WAR, version of the Jenkins WAR, "latest", or "lts"

Examples:

# Run full suite in FF against ./jenkins.war.
# Run full suite in Firefox against a URL:
$ ./run firefox https://updates.jenkins.io/download/war/2.394/jenkins.war

# Run full suite in Firefox against ./jenkins.war:
$ ./run firefox ./jenkins.war

# Run Ant plugin test in chrome against Jenkins 1.512.
$ ./run chrome 1.512 -Dtest=AntPluginTest
# Run Ant plugin test in chrome against Jenkins 2.399:
$ ./run chrome 2.399 -Dtest=AntPluginTest

# Run full suite in FF against LTS release candidate
$ ./run firefox lts-rc
# Run full suite in Firefox against LTS:
$ ./run firefox lts
USAGE
exit 2
fi

MVN='mvn -V -e -ntp'
if [[ -n ${MAVEN_SETTINGS-} ]]; then
MVN="${MVN} -s ${MAVEN_SETTINGS}"
fi
Comment on lines +29 to +32
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Observe MAVEN_SETTINGS if defined in preparation for #963.


function download() {
echo "Fetching $1 to $2"
status=$(curl --http1.1 -sSL --write-out '%{http_code}' --retry 3 --retry-delay 0 --retry-max-time 60 -o "$2" "$1")
Expand All @@ -39,50 +47,33 @@ fi

browser=$1
war=$2
extra_args=

if [[ ! -f $war ]]; then
case "$war" in
latest)
war=jenkins-latest.war
url=https://updates.jenkins.io/latest/jenkins.war
# Maven will download this in the process-test-resources phase
war=target/jenkins-war.war
;;
lts)
war=jenkins-lts.war
url=https://updates.jenkins.io/stable/latest/jenkins.war
# Maven will download this in the process-test-resources phase
war=target/jenkins-war.war
extra_args='-Plts'
;;
*.war)
download "$war" "jenkins.war" || exit 1
war=jenkins.war
;;
Comment on lines +63 to +66
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Due to the [[ ! -f $war ]]; then a few lines above, this is the case statement for URLs, not local file paths.

*)
# Maven will download this in the process-test-resources phase
war=target/jenkins-war.war
extra_args="-Djenkins.version=$2"
;;
esac

if [[ -n $url ]]; then
find $war -maxdepth 0 -mtime +1 -delete 2>/dev/null
if [[ ! -f $war ]]; then
download "$url" "$war" || exit 1
fi
fi
fi

if [[ ! -f $war ]] && [[ $war == *.war ]]; then
download "$war" "jenkins.war" || exit 1
war=jenkins.war
fi

if [[ ! -f $war ]]; then
wardir=~/.m2/repository/org/jenkins-ci/main/jenkins-war

war=$wardir/$2/jenkins-war-$2.war
if [[ ! -f $war ]]; then
mvn -B org.apache.maven.plugins:maven-dependency-plugin:2.7:get \
-DremoteRepositories=repo.jenkins-ci.org::::https://repo.jenkins-ci.org/public/ \
"-Dartifact=org.jenkins-ci.main:jenkins-war:$2:war"
fi

if [[ ! -f $war ]]; then
echo 'No such jenkins.war. Available local versions:'
ls $wardir/*/jenkins-war-*.war | sed -r -e 's/.*jenkins-war-(.+)\.war/\1/'
exit 1
fi
fi

shift 2

set -x

BROWSER=$browser JENKINS_WAR=$war mvn --show-version --no-transfer-progress test "$@"
BROWSER=$browser JENKINS_WAR=$war $MVN test $extra_args "$@"