Skip to content

Commit

Permalink
[JENKINS-65398] Terminology update (#414)
Browse files Browse the repository at this point in the history
* [JENKINS-65398] Terminology update
* [JENKINS-65398] Adapt for spotlessJava
* Update README.md
* [JENKINS-65398] Revert some update
* [JENKINS-65398] Revet some changes
* [JENKINS-65398] Running gradlew spotlessApply

Signed-off-by: Thierry Wasylczenko <[email protected]>

Co-authored-by: Vincent Latombe <[email protected]>
  • Loading branch information
twasyl and Vlatombe authored Aug 10, 2021
1 parent a4a7197 commit 7da7bc6
Show file tree
Hide file tree
Showing 10 changed files with 27 additions and 26 deletions.
27 changes: 13 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@
Jenkins on Mesos
----------------

The `jenkins-mesos` plugin allows Jenkins to dynamically launch Jenkins slaves on a
The `jenkins-mesos` plugin allows Jenkins to dynamically launch Jenkins agents on a
Mesos cluster depending on the workload!

Put simply, whenever the Jenkins `Build Queue` starts getting bigger, this plugin
automatically spins up additional Jenkins agent(s) on Mesos so that jobs can be
immediately scheduled! Similarly, when a Jenkins slave is idle for a long time it
immediately scheduled! Similarly, when a Jenkins agent is idle for a long time it
is automatically shut down.

## Table of Contents
Expand All @@ -25,7 +25,7 @@ is automatically shut down.
- __[Docker Containers](#docker-containers)__
- __[Docker Configuration](#docker-configuration)__
- __[Over provisioning flags](#over-provisioning-flags)__
- __[Single-Use Slave](#single-use-slave)__
- __[Single-Use Agent](#single-use-agent)__
- __[Freestyle jobs](#freestyle-jobs)__
- __[Pipeline jobs](#pipeline-jobs)__
- __[Plugin Development](#plugin-development)__
Expand Down Expand Up @@ -82,42 +82,42 @@ Simply run the environment variables `DCOS_SERVICE_ACCOUNT` containing the servi

### Configuring Jenkins Jobs ###

Finally, just add the label name you have configured in Mesos cloud configuration -> Advanced -> Slave Info -> Label String (default is `mesos`)
to the jobs (configure -> Restrict where this project can run checkbox) that you want to run on a specific slave type inside Mesos cluster.
Finally, just add the label name you have configured in Mesos cloud configuration -> Advanced -> Agent Info -> Label String (default is `mesos`)
to the jobs (configure -> Restrict where this project can run checkbox) that you want to run on a specific agent type inside Mesos cluster.

### Docker Containers ###

By default, the Jenkins slaves are run in the default Mesos container. To run the Jenkins agent inside a Docker container, there are two options.
By default, the Jenkins agents are run in the default Mesos container. To run the Jenkins agent inside a Docker container, there are two options.

1) "Use Native Docker Containerizer" : Select this option if Mesos slave(s) are configured with "--containerizers=docker" (recommended).
1) "Use Native Docker Containerizer" : Select this option if Mesos agent(s) are configured with "--containerizers=docker" (recommended).

2) "Use External Containerizer" : Select this option if Mesos slave(s) are configured with "--containerizers=external".
2) "Use External Containerizer" : Select this option if Mesos agent(s) are configured with "--containerizers=external".

### Docker Configuration ###

#### Volumes ####

At a minimum, a container path must be entered to mount the volume. A host path can also be specified to bind mount the container path to the host path. This will allow persistence of data between slaves on the same node. The default setting is read-write, but an option is provided for read-only use.
At a minimum, a container path must be entered to mount the volume. A host path can also be specified to bind mount the container path to the host path. This will allow persistence of data between agents on the same node. The default setting is read-write, but an option is provided for read-only use.

#### Parameters ####

Additional parameters are available for the `docker run` command, but there are too many and they change too often to list all separately. This section allows you to provide any parameter you want. Ensure that your Docker version on your Mesos slaves is compatible with the parameters you add and that the values are correctly formatted. Use the full-word parameter and not the shortcut version, as these may not work properly. Also, exclude the preceding double-dash on the parameter name. For example, enter `volumes-from` and `my_container_name` to recieve the volumes from `my_container_name`. Of course `my_container_name` must already be on the Mesos slave where the Jenkins slave will run. This shouldn't cause problems in a homogenous environment where Jenkins slaves only run on particular Mesos slaves.
Additional parameters are available for the `docker run` command, but there are too many and they change too often to list all separately. This section allows you to provide any parameter you want. Ensure that your Docker version on your Mesos agents is compatible with the parameters you add and that the values are correctly formatted. Use the full-word parameter and not the shortcut version, as these may not work properly. Also, exclude the preceding double-dash on the parameter name. For example, enter `volumes-from` and `my_container_name` to recieve the volumes from `my_container_name`. Of course `my_container_name` must already be on the Mesos agent where the Jenkins agent will run. This shouldn't cause problems in a homogenous environment where Jenkins agents only run on particular Mesos agents.

### Over provisioning flags ###

By default, Jenkins spawns slaves conservatively. Say, if there are 2 builds in queue, it won't spawn 2 executors immediately. It will spawn one executor and wait for sometime for the first executor to be freed before deciding to spawn the second executor. Jenkins makes sure every executor it spawns is utilized to the maximum.
If you want to override this behaviour and spawn an executor for each build in queue immediately without waiting, you can use these flags during Jenkins startup:
`-Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85`

## Single-Use Slave ##
## Single-Use Agent ##

### Freestyle jobs ###

In the Build Environment settings, you may select "Mesos Single-Use Slave" to schedule disposal of the slave after the build finishes.
In the Build Environment settings, you may select "Mesos Single-Use Agent" to schedule disposal of the agent after the build finishes.

### Pipeline jobs ###

To schedule slave disposal from a Pipeline job:
To schedule agent disposal from a Pipeline job:

node('mylabel') {
wrap([$class: 'MesosSingleUseSlave']) {
Expand Down Expand Up @@ -162,4 +162,3 @@ To release this plugin

1. Set the version in `build.gradle`.
2. Publish the plugin with `./gradlew publish`.

2 changes: 1 addition & 1 deletion dcos-testing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The `Dockerfile` defines a Docker image that supports DC/OS strict mode. It shou
```
dcos security org service-accounts create -p jenkins.pub.pem -d "Jenkins Service Account" jenkins
```
3. Store private key as secret so that the Jenkins master can access it
3. Store private key as secret so that the Jenkins controller can access it
```
dcos security secrets create -f ./jenkins.private.pem jenkins/private_key
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ public FormValidation doCheckCpus(@QueryParameter String cpus) {
/**
* Creates a LaunchPod command to to create a new Jenkins agent via USI
*
* @param jenkinsUrl the URL of the jenkins master.
* @param jenkinsUrl the URL of the Jenkins controller.
* @param name The name of the node to launch.
* @param role The Mesos role for the task.
* @return a LaunchPod command to be passed to USI.
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/org/jenkinsci/plugins/mesos/MesosCloud.java
Original file line number Diff line number Diff line change
Expand Up @@ -296,8 +296,8 @@ public Future<Node> startAgent(String name, MesosAgentSpecTemplate spec)
}

/**
* Checks whether the Jenkins master itself is running as a Mesos task and thus has the env var
* MESOS_SANDBOX defined.
* Checks whether the Jenkins controller itself is running as a Mesos task and thus has the env
* var MESOS_SANDBOX defined.
*
* @return
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ public synchronized boolean isPending() {
}

/**
* Updates the state of the slave and takes action on certain events.
* Updates the state of the agent and takes action on certain events.
*
* @param event The state event from USI which informs about the task status.
*/
Expand Down Expand Up @@ -194,7 +194,7 @@ protected void _terminate(TaskListener listener) {
}

public boolean getReusable() {
// TODO: implement reusable slaves DCOS_OSS-5048
// TODO: implement reusable agents DCOS_OSS-5048
return reusable;
}

Expand Down
4 changes: 2 additions & 2 deletions src/main/java/org/jenkinsci/plugins/mesos/MesosSlaveInfo.java
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ public class MesosSlaveInfo {
private transient boolean defaultSlave;

@SuppressFBWarnings("UUF_UNUSED_FIELD")
private transient String slaveAttributesString; // Slave attributes JSON representation.
private transient String slaveAttributesString; // Agent attributes JSON representation.

/**
* Resolves the old agent configuration after deserialization.
Expand Down Expand Up @@ -94,7 +94,7 @@ JSONObject parseSlaveAttributes(String slaveAttributes) {
return (JSONObject) JSONSerializer.toJSON(slaveAttributes);
} catch (JSONException e) {
logger.warn(
"Ignoring Mesos slave attributes JSON due to parsing error : " + slaveAttributes);
"Ignoring Mesos agent attributes JSON due to parsing error : " + slaveAttributes);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,9 @@ private Iterable<AgentFilter> buildAgentAttributeFilters() {
}
}

/** @return the Jnlp url for the agent: http://[master]/computer/[slaveName]/slave-agent.jnlp */
/**
* @return the Jnlp url for the agent: http://[controller]/computer/[agentName]/slave-agent.jnlp
*/
private URL buildJnlpUrl() throws MalformedURLException {
final String path = Paths.get("computer", this.id.value(), "slave-agent.jnlp").toString();
return new URL(this.jenkinsMaster, path);
Expand Down
2 changes: 1 addition & 1 deletion src/main/resources/index.jelly
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@
-->
<div>
This plugin can be used to connect Jenkins to a Mesos cluster
and dynamically launch Jenkins slaves based on the work load.
and dynamically launch Jenkins agents based on the work load.
</div>
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
<b>http://host:post</b> where Jenkins is actually running. For ex: http://myjenkinshost:8080.
Jenkins URL provided in Jenkins location configuration could be a reverse proxy URL.
In that case you can override it by specifying actual Jenkins URL here.
This URL would be used by Jenkins agents spawned by Mesos plugin to talk to Jenkins Master.
This URL would be used by Jenkins agents spawned by Mesos plugin to talk to Jenkins Controller.
</div>
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ public void testJenkinsAgentWithDockerImage(TestUtils.JenkinsRule j) throws Exce

MesosJenkinsAgent agent = (MesosJenkinsAgent) cloud.startAgent(name, spec).get();

// verify slave is running when the future completes;
// verify agent is running when the future completes;
await().atMost(5, TimeUnit.MINUTES).until(agent::isRunning);
assertThat(agent.isRunning(), is(true));
}
Expand Down

0 comments on commit 7da7bc6

Please sign in to comment.