Skip to content

Latest commit

 

History

History
299 lines (227 loc) · 10 KB

5-custom-transformers.md

File metadata and controls

299 lines (227 loc) · 10 KB
Raw

Use custom transformers to customize GitHub Actions Importer's behavior

In this lab you will build upon the dry-run command to override GitHub Actions Importer's default behavior and customize the converted workflow using "custom transformers." Custom transformers can be used to:

  1. Convert items that are not automatically converted.
  2. Convert items that were automatically converted using different actions.
  3. Convert environment variable values differently.
  4. Convert references to runners to use a different runner name in Actions.

Prerequisites

  1. Followed the steps here to set up your GitHub Codespaces environment and start a Jenkins server.
  2. Completed the configure lab.
  3. Completed the dry-run lab.

Perform a dry run

You will be performing a dry-run command to inspect the workflow that is converted by default. Run the following command within the codespace terminal:

gh actions-importer dry-run jenkins --source-url http://localhost:8080/job/test_pipeline --output-dir tmp/dry-run

The converted workflow that is generated by the above command can be seen below:

Converted workflow 👇 
name: test_pipeline
on:
  push:
    paths: "*"
  schedule:
  - cron: 0-29/10 * * * *
env:
  DISABLE_AUTH: 'true'
  DB_ENGINE: sqlite
jobs:
  build:
    runs-on:
      - self-hosted
      - TeamARunner
    steps:
    - name: checkout
      uses: actions/checkout@v2
    - name: echo message
      run: echo "Database engine is ${{ env.DB_ENGINE }}"
#     # This item has no matching transformer
#     - sleep:
#       - key: time
#         value:
#           isLiteral: true
#           value: 80
    - name: echo message
      run: echo "DISABLE_AUTH is ${{ env.DISABLE_AUTH }}"
  test:
    runs-on:
      - self-hosted
      - TeamARunner
    needs: build
    steps:
    - name: checkout
      uses: actions/checkout@v2
    - name: Publish test results
      uses: EnricoMi/[email protected]
      if: always()
      with:
        files: "**/target/*.xml"

Note: You can refer to the previous lab to learn about the fundamentals of the dry-run command.

Custom transformers for an unknown step

The converted workflow above contains a sleep step that was not automatically converted. Answer the following questions before writing a custom transformer:

  1. What is the "identifier" of the step to customize?

    • sleep. The identifier will be the key of a key value pair within the step of a Jenkinsfile.

  2. What is the desired Actions syntax to use instead?

    • After some research, you have determined that the following bash script will provide similar functionality:

      - name: Sleep for 80 seconds
  run: sleep 80s
  shell: bash

Now you can begin to write the custom transformer. Custom transformers use a DSL built on top of Ruby and should be defined in a file with the .rb file extension. You can create this file by running the following command in your codespace terminal:

touch transformers.rb && code transformers.rb

Next, you will define a transform method for the sleep identifier by adding the following code to transformers.rb:

transform "sleep" do |item|
  wait_time = item["arguments"][0]["value"]["value"]

  {
    "name": "Sleep for #{wait_time} seconds",
    "run": "sleep #{wait_time}s",
    "shell": "bash"
  }
end

This method can use any valid Ruby syntax and should return a Hash that represents the YAML that should be generated for a given step. GitHub Actions Importer will use this method to convert a step with the provided identifier and will use the item parameter for the original values configured in Jenkins.

Now you can perform another dry-run command and use the --custom-transformers CLI option to provide this custom transformer. Run the following command within your codespace terminal:

gh actions-importer dry-run jenkins --source-url http://localhost:8080/job/test_pipeline --output-dir tmp/dry-run --custom-transformers transformers.rb

Open the workflow that is generated and inspect the contents. Now the sleep step is converted and uses the customized behavior.

- #     # This item has no matching transformer
- #     - sleep:
- #       - key: time
- #         value:
- #           isLiteral: true
- #           value: 80
+  - name: Sleep for 80 seconds
+    run: sleep 80s
+    shell: bash

Custom transformers for a known step

You can also override GitHub Actions Importer's default behavior. In this scenario, you may not want to use the third-party action for publishing junit test results that is used by default. Again, answer the following questions before writing a custom transformer:

  1. What is the "identifier" of the step to customize?

    • junit

  2. What is the desired Actions syntax to use instead?

    • After some research, you have determined that uploading test results as an artifact will be suitable:

      - uses: actions/upload-artifact@v3
  with:
    name: junit-artifact
    path: path/to/artifact/world.txt

You will build this custom transformer similar to the previous custom transformer, however, you first need to inspect the item keyword to programmatically use the file path to junit's test results in the actions/upload-artifact@v3 step.

To do this, you will print item to the console. You can achieve this by adding the following custom transformer to transformers.rb:

transform "junit" do |item|
  puts "This is the item: #{item}"
end

Now, you can perform another dry-run command with the --custom-transformers CLI option. The output of the dry-run command should look similar to this:

$ gh actions-importer dry-run jenkins --source-url http://localhost:8080/job/test_pipeline --output-dir tmp/dry-run --custom-transformers transformers.rb
[2022-08-20 22:08:20] Logs: 'tmp/dry-run/log/actions-importer-20220916-022628.log'
This is the item: {"name"=>"junit", "arguments"=>[{"key"=>"testResults", "value"=>{"isLiteral"=>true, "value"=>"**/target/*.xml"}}]}
[2022-08-20 22:08:20] Output file(s):
[2022-08-20 22:08:20]   tmp/dry-run/test_pipeline/.github/workflows/test_pipeline.yml

Now that you know the data structure of item, you can access the file path programmatically by editing the custom transformer to the following:

transform "junit" do |item|
  test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
  file_path = test_results.dig("value", "value")

  {
    "uses" => "actions/upload-artifact@v3",
    "with" => {
      "name" => "junit-artifact",
      "path" => file_path
    }
  }
end

Note: transformers.rb should contain a transform method for both sleep and junit.

Now you can perform another dry-run command with the --custom-transformers CLI option. When you open the converted workflow the EnricoMi/[email protected] action will have been replaced with the customized steps.

-    - name: Publish test results
-      uses: EnricoMi/[email protected]
-      if: always()
-      with:
-        files: "**/target/*.xml"
+    - uses: actions/upload-artifact@v3
+      with:
+        name: junit-artifact
+        path: "**/target/*.xml"

Custom transformers for environment variables

You can also use custom transformers to edit the values of environment variables in converted workflows. In this example, you will be updating the DB_ENGINE environment variable to be mongodb instead of sqlite.

To do this, add the following code at the top of the transformers.rb file.

env "DB_ENGINE", "mongodb"

In this example, the first parameter to the env method is the environment variable name and the second is the updated value.

Now you can perform another dry-run command with the --custom-transformers CLI option. When you open the converted workflow the DB_ENGINE environment variable will be set to mongodb:

env:
  DISABLE_AUTH: 'true'
-  DB_ENGINE: sqlite
+  DB_ENGINE: mongodb

Custom transformers for runners

Finally, you can use custom transformers to dictate which runners the converted workflows should use. To do this, answer the following questions:

  1. What is the label of the runner in Jenkins to update?

    • TeamARunner

  2. What is the label of the runner in Actions to use instead?

    • ubuntu-latest

With these questions answered, you can add the following code to the transformers.rb file:

runner "TeamARunner", "ubuntu-latest"

In this example, the first parameter to the runner method is the Jenkins label and the second is the Actions runner label.

Now you can perform another dry-run command with the --custom-transformers CLI option. When you open the converted workflow the runs-on statement will use the customized runner label:

runs-on:
-  - self-hosted
-  - TeamARunner
+  - ubuntu-latest

At this point the file contents of transformers.rb should match this:

Custom transformers 👇 
  runner "TeamARunner", "ubuntu-latest"

  env "DB_ENGINE", "mongodb"

  transform "sleep" do |item|
      wait_time = item["arguments"][0]["value"]["value"]

      {
        "name": "Sleep for #{wait_time} seconds",
        "run": "sleep #{wait_time}s",
        "shell": "bash"
      }
  end

  transform "junit" do |item|
    test_results = item["arguments"].find{ |a| a["key"] == "testResults" }
    file_path = test_results.dig("value", "value")

    [
      {
        "uses": "actions/upload-artifact@v3",
        "with": {
          "name": "my-artifact",
          "path": file_path
        }
      }
    ]
  end

That's it. Congratulations, you have overridden GitHub Actions Importer's default behavior by customizing the conversion of:

  • Unknown steps
  • Known steps
  • Environment variables
  • Runners

Next lab

Perform a production migration of a Jenkins pipeline