DevPIandJenkins

Using "devpi" in Jenkins

Introduction

When using devpi in Jenkins jobs, especially in an environment where slaves are shared by many people and teams, some care has to be taken in order for things to run smoothly.

• You have to handle multiple concurrent jobs with different active indexes.
• You have to provide upload credentials to jobs that release packages into an index, the EnvInject plugin helps there.

Helper Definitions

Add the shell snippet you'll find below to your default Jenkins environment, e.g. in your slave startup script. This allows a job that needs packages specific to a team or product to call devpi_use my/repo early on in a shell build step, which enables the following features:

• devpi is aliased to use a client dir local to the workspace and containing the config for the given repo, and thus isolate any devpi call against other jobs on the same slave.
• The exported PIP_INDEX_URL makes pip works as intended, and can also be passed to related tools like a project's setup-py (via --index-url). Or you just use devpi install and devpi upload instead, so the index URL is taken from the config file that was created in devpi's client dir.

Additionally, you should have default configuration files in the user account that runs the Jenkins slave, ideally with read-only permissions, so that jobs don't accidentally change those defaults and create race conditions and spurious job failures. These default configurations should either refer to root/pypi, or else to a company-wide index with shared resources. They can be rolled out via your config management tool of choice.

# Jenkins environment definitions for "devpi"
export DEVPI_URL="http://devpi.example.com:31415/"

devpi_use() {
    local index="${1:?You MUST provide an index name to devpi_use}"
    local devpi_cfgdir="${WORKSPACE:?Envvar WORKSPACE is undefined, not in a Jenkins job?}/.devpi"

    test -d "$devpi_cfgdir" || mkdir "$devpi_cfgdir"
    alias devpi="$(LC_ALL=C command which devpi) --clientdir "\'"$devpi_cfgdir"\'
    export devpi
    export PIP_INDEX_URL=$(devpi use --urls "${DEVPI_URL%/}/$index" | grep simpleindex | tr -d ' ' | cut -f2- -d:)
    devpi use # dump status
}

test -z "$BASH_VERSION" && export devpi_use || export -f devpi_use