Integration with GitHub

This page describes how to configure Allure TestOps integration with GitHub in a project that uses GitHub to run tests. This integration supports both the public GitHub (github.com) as well as GitHub Enterprise Server.

Please read this whole page top to down first to understand how it works, then pass step by step without omitting any steps.

Every little thing we describe here is required for successful integration.

Entities relations

When working with GitHub it's important to understand the following relations between Allure TestOps entities and entities on GitHub side:

• One job in Allure TestOps corresponds to one GitHub workflow.
• One launch in Allure TestOps corresponds to one GitHub workflow run.

There is no relation towards a specific job in GutHub Workflow, so if a specific job needs to be triggered from Allure TestOps side, this needs to be managed by means of GH workflow.

A new workflow run can be triggered either by Allure TestOps or by GitHub itself, with both parties displaying its status in their web interfaces.

Limitations

1. For the time being, it's impossible to create a Job for a GutHub Workflow manually by clicking +Job button. The easiest way and the only way for the integration with GutHub is to let Allure TestOps create the job automatically by executing Workflow first time from the GH UI.
2. For GitHub Enterprise it's impossible to use GH Action to install and configure allurectl. This limitation will require a workaround described here.
3. Integration now supports only personal access tokens usage for the integration Allure TestOps → GitHub (triggering the workflows).

All the steps described below will lead you to the successful bi-directional integration with GutHub workflows as a CI/CD.

Uploading test results to Allure TestOps

Task

All following instructions are based on the assumption that our tasks are:

• To be able to Upload test results from a GH workflow with automated tests to Allure TestOps.

To upload test results to Allure TestOps from your GitHub workflow you need to pass the following steps.

Prerequisites

Enable the generation of the test results of the supported format in the scope of your pipeline.

We're encouraging the end users ti utilise the Allure Framework adaptors to generate allure results of allure2 format. This will allow you get the most from your tests and avoid manual work in Allure TestOps.

Alteration of the workflow file

1. Add allurectl installation action to your pipeline
   1. You cannot just download allurectl and then use it as the action also is preparing the integration data.
   2. We'll explain what are the parameters required.
2. Include allurectl to your workflow
   1. Preferably use watch mode.

Example workflow

Here is a very simple example of GitHub workflow for a Java tests project that is being built by Gradle. It's being started on push event to the main branch of the VCS repo.

name: GitHub integration testops

on:
  push:
    branches: [main]

jobs:
  all-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
      - name: Build with Gradle
        run: |
          ./gradlew clean test

What is does:

1. Checks-out the code to the pipeline (workflow execution)
2. Sets-up the Java environment required for the tests to run.
3. Executes the tests as ./gradlew clean test

Add allurectl action

This won't work for GitHub Enterprise. GitHub Enterprise requires a workaround described here.

First we need to add and configure allurectl

This needs to be added to the workflow:

      - name: Install and configure allurectl
        uses: allure-framework/[email protected]
        with:
          allure-endpoint: https://demo.testops.cloud
          allure-token: ${{ secrets.ALLURE_TOKEN }}
          allure-project-id: 111

So the resulting workflow will look like follows:

name: GitHub integration testops

on:
  push:
    branches: [main]

jobs:
  all-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
# <<<<
      - name: Install and configure allurectl
        uses: allure-framework/[email protected]
        with:
          allure-endpoint: https://demo.testops.cloud
          allure-project-id: 111
          allure-token: ${{ secrets.ALLURE_TOKEN }}
# >>>>
      - name: Build with Gradle
        run: |
          ./gradlew clean test

Action for the installation and configuration of allurectl has the following attributes:

• allure-endpoint
  • Full qualified domain name or IP address of Allure TestOps instance you are using. It must contain the protocol part (https:// or http://) as well
• allure-project-id
  • The identifier of the project you are going to upload the test results to.
• allure-token
  • API token generated on Allure TestOps side.
  • API token must be kept in secret.
  • Create a repo secret and store API token there.
  • We'll be referencing API token as ${{ secrets.ALLURE_TOKEN }} for it's added as ALLURE_TOKEN under the repository secrets.

Use allurectl watch in the workflow to run tests

Check the description of allurectl watch here.

Long story short, watch creates child process executing the automated tests and then sends created test results (in real time!) to Allure TestOps.

Now we need to wrap the execution of tests in allurectl watch:

      - name: Build with Gradle
        run: |
          allurectl watch -- ./gradlew clean test

Command must be executed exactly as described:

allurectl watch -- ./gradlew clean test

There must be spaces between watch and "--" and between "--" and the command you are using to trigger your tests execution.

name: GitHub integration with Allure TestOps

on:
  push:
    branches: [main]

jobs:
  all-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
# <<<<
      - name: Install and configure allurectl
        uses: allure-framework/[email protected]
        with:
          allure-endpoint: https://demo.testops.cloud
          allure-project-id: 111
          allure-token: ${{ secrets.ALLURE_TOKEN }}
# >>>>
# <<<<
      - name: Build with Gradle and upload to Allure TestOps
        run: |
          allurectl watch -- ./gradlew clean test
# >>>>

Adding workflow_dispatch event

The workflow we prepared in the previous will be triggered on push event to the main branch. For the purposes of the bi-directional integration we also need to add another GitHub event called workflow_dispatch, which can be used to trigger the workflow from GH UI. Workflow dispatch has attributes defined as inputs, we'll need these.

Add this to on: section:

on:
  workflow_dispatch:

name: GitHub integration with Allure TestOps

on:
  push:
    branches: [main]
  workflow_dispatch:

jobs:
  all-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
# <<<<
      - name: Install and configure allurectl
        uses: allure-framework/[email protected]
        with:
          allure-endpoint: https://demo.testops.cloud
          allure-project-id: 111
          allure-token: ${{ secrets.ALLURE_TOKEN }}
# >>>>
# <<<<
      - name: Build with Gradle and upload to Allure TestOps
        run: |
          allurectl watch -- ./gradlew clean test
# >>>>

Upload test results to Allure TestOps first time

Where are the results?

The Allure TestOps CLI tool - allurectl is supposed to upload the test results to Allure TestOps, but to do so, we need to instruct allurectl where to look for the results.

1. Define environment variable ALLURE_RESULTS in env section.
2. Provide the path where Allure results are expected.

The variable must be defined in the context of the workflow, so let's put it like this:

env:
  ALLURE_RESULTS: "build/allure-results"

name: GitHub integration with Allure TestOps

on:
  push:
    branches: [main]
  workflow_dispatch:

# <<<<
env:
  ALLURE_RESULTS: "build/allure-results"
# >>>>
jobs:
  all-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
      - name: Install and configure allurectl
        uses: allure-framework/[email protected]
        with:
          allure-endpoint: https://demo.testops.cloud
          allure-project-id: 111
          allure-token: ${{ secrets.ALLURE_TOKEN }}
      - name: Build with Gradle and upload to Allure TestOps
        run: |
          allurectl watch -- ./gradlew clean test

The path "build/allure-results" is specific to Gradle Java project, in your case the path described in ALLURE_RESULTS could be (or just will be) different.

Trigger the Workflow in GitHub UI

To proceed to next integration steps, we need execute our tests and upload test results to Allure TestOps.

1. Go to Actions section
2. Select the workflow we are changing.
3. Click on Run workflow.
4. Select the correct branch you are working with (we hope you don't use main/master branches for the experiments).
5. Hit Run workflow.

If everything has been done correctly, in couple of seconds you will see that a new launch has been created and test results arrived to the created launch.

Congratulations! This task has been completed:

• To be able to Upload test results from a GH workflow with automated tests to Allure TestOps.

Close the launch

Now close the created launch.

When launch is closed, Allure TestOps will create (and update) test cases for the automated tests.

Triggering GitHub workflows from Allure TestOps

This is the second direction of the bi-directional integration with GitHub.

Task

• We want to trigger GitHub workflows from Allure TestOps.

To do so, we need to have an integration with CI/CD server (in our case it's GitHub).

Generally, for each and any integration type on Allure TestOps side we have to perform 2 steps:

1. An instance administrator creates an integration globally:
   1. Provides type.
   2. Name.
   3. Endpoint(s) for the integration.
2. A project owner configures the integration on a project level
   1. Provides credentials for the authentication of API calls
   2. Provides additional configuration parameters if such action can be applied to the integration of certain type.

The integration created globally (on the instance level) is used in all projects that require such integration type, i.e. if we have created a global integration for github.com, then on the instance level we only do it once.

Add global GitHub integration

1. Log in to Allure TestOps using an administrator account.
2. Go to Administration → Integrations.
3. Click + Add integration in the top right corner of the page.
4. In the dialog that appears, select GitHub.
5. Fill in the fields:
   • Name — a name to help you recognise the GitHub instance (e.g. simply, "github.com").
   • Endpoint — the base URL of GitHub. For github.com, use https://github.com. For GitHub Enterprise Server, use the URL of your GitHub instance deployed in your infrastructure.
   • Endpoint for API calls — the URL of GitHub API. For github.com, use https://api.github.com. For GitHub Enterprise Server, use ⟨URL⟩/api/v3, where ⟨URL⟩ is the URL of your GitHub instance .
6. If you are using GitHub Enterprise Server with a self-signed SSL certificate, check the Disable certificate validation box.
7. Click Add integration.

Create a new personal access token in GitHub

We'll use the personal access token for the authentication to trigger workflows.

To be able to trigger workflows, Allure TestOps needs to have a personal access token generated on the GitHub side. You can choose the type of token that GitHub will generate: fine-grained or classic (see Managing your personal access tokens in the GitHub documentation).

Instructions for generating a token vary slightly depending on the token type.

Add the GitHub integration to the project

1. In Allure TestOps, open your project.
2. Go to Settings → Integrations.
3. Under Available integrations, find the GitHub integration and click Add integration next to it.
4. Under Secret, enter the GitHub access token you created in previous steps.
5. Click Test connection. If the token is correct, a "Connection established" message will appear within a few seconds.
6. Click Add integration to save the settings.

We're almost there.

Configure Job

Job is a Project entity that links your project with a single GitHub workflow.

Job also provides means to pass the parameters for a workflow execution but we'll describe that later on this page.

As soon as you've created your first launch and uploaded the test results, Allure TestOps creates a new job for the executed workflow. This is automatic process that does not require any actions from your side.

To trigger a workflow from Allure TestOps we need to configure created job and update the workflow file a bit.

Locate and configure job

1. Go to your project.
2. Jump to the jobs section.
3. Locate a new job and click on the edit menu item.

4. In the menu select item Configure

Setting Job's mandatory parameters

Application will render the following form.

In the form following mandatory attributes are required:

1. Build server is the name of the integration created for GitHub and added to the current project.
2. Parameter Job can be used to run tests must be checked.
3. Name of the parameter must be Branch (this is GitHub integration specific requirement).
4. Value must contain the default VCS branch name you want to be triggered in case no other information is specified.
5. Environment variable is to be set to the environment variable defined on the Instance level.

As soon as all parameters are set, hit Submit button.

Sync Job configuration with Workflow

This step is often omitted so we highlight it.

Please do not skip this step.

After you have provided mandatory parameters your job will look like this.

One little step is remaining.

Please Click Update job from the build server button (1).

Application will sync the information with GitHub and update the attributes of the configured job

Now the job configuration contains correct link to the GH Workflow and name defined in the said workflow.

Updating GitHub Workflow

Yes, we could apply all changes at once, but in this case if anything goes wrong, it's unclear who is the culprit, so we change Workflow gradually.

Add input parameters to workflow_dispatch

Add these parameters as required: false. This means that this parameter can be left blank when triggering workflow via GitHub UI, but they must be defined at all times.

Input parameters ALLURE_JOB_RUN_ID and ALLURE_USERNAME are mandatory as workflow_dispatch inputs when it comes to triggering the workflow from Allure TestOps, if these inputs are not present, then all attempts to trigger this workflow from Allure TestOps will fail with error 422 from GitHub side.

A bit on ALLURE_JOB_RUN_ID

ALLURE_JOB_RUN_ID is a crucial element for the correct upload of test results when a GitHub Workflow is triggered from Allure TestOps.

We need to add the variable itself to the env section and pass the value from the input ALLURE_JOB_RUN_ID we've created.

Without passing of the received value the correct work is not possible. More information on GitHub contexts can be found here.

    inputs:
      ALLURE_JOB_RUN_ID:
        description: "ALLURE_JOB_RUN_ID service parameter. Leave blank."
        required: false
      ALLURE_USERNAME:
        description: "ALLURE_USERNAME service parameter. Leave blank"
        required: false

When Allure TestOps triggers a workflow, it passes the information to the input parameters ALLURE_JOB_RUN_ID and ALLURE_USERNAME.

However, this information needs to be passed to the workflow's context to be available for allurectl.

To provide the values to the context we need to refer these inputs as per GitHub documentation:

env:
  ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }}
  ALLURE_USERNAME: ${{ github.event.inputs.ALLURE_USERNAME }}

Full example of the updated workflow.

name: GitHub integration with Allure TestOps

on:
 push:
   branches: [main]
 workflow_dispatch:
# <<<<
   inputs:
     ALLURE_JOB_RUN_ID:
       description: "ALLURE_JOB_RUN_ID service parameter. Leave blank."
       required: false
     ALLURE_USERNAME:
       description: "ALLURE_USERNAME service parameter. Leave blank"
       required: false
# >>>>
env:
 ALLURE_RESULTS: "build/allure-results"
# <<<<
 ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }}
 ALLURE_USERNAME: ${{ github.event.inputs.ALLURE_USERNAME }}
# >>>>
jobs:
 all-tests:
   runs-on: ubuntu-latest
   steps:
     - uses: actions/checkout@v3
     - name: Set up JDK 1.8
       uses: actions/setup-java@v1
       with:
         java-version: 1.8
     - name: Install and configure allurectl
       uses: allure-framework/[email protected]
       with:
         allure-endpoint: https://demo.testops.cloud
         allure-project-id: 111
         allure-token: ${{ secrets.ALLURE_TOKEN }}
     - name: Build with Gradle and upload to Allure TestOps
       run: |
         allurectl watch -- ./gradlew clean test

What happens when such workflow is triggered by Allure TestOps?

1. Allure TestOps creates a new launch and inside the created Launch Job run is created.
2. Job run entity contains information for allurectl and GitHub workflow.
3. Allurectl must upload the data to the created job run, hence we need its ID (ALLURE_JOB_RUN_ID) to be provided to allurectl's execution context.
4. When allurectl is started, it looks for ALLURE_JOB_RUN_ID, if ALLURE_JOB_RUN_ID is not present or empty, then it creates a new launch.
   • Basically, this happens when you are triggering the workflow from the GitHub UI.
5. If ALLURE_JOB_RUN_ID is present and is not empty, then information about Job run is used and allurectl uploads the data to the created launch.
6. That's it.

Trigger the Job

Triggering the created job is just one of the several ways to create a Launch and trigger CI/CD pipeline.

In the Jobs section of Allure TestOps Project, click on the Play button of Job we created and configured in the previous steps.

Application will render the form for a Launch creation.

For testing purposes only the following needs to be updated:

1. Uncheck all test cases if there is a checked list.
2. Click + Add to provide correct Branch name to trigger the workflow on.
3. Add the branch name you are using to update the workflow.
4. Submit the form.

Expected result

1. Application will created a launch.
2. Application created job run.
3. Application triggered GitHub workflow without errors.
4. allurectl uploaded the test results.

The task now has been completed.

• We want to trigger GitHub workflows from Allure TestOps.

Parameters in workflows and in Allure TestOps

Task

• We want to see what was the environment of executed workflow.
• We want to be able to provide parameters for tests execution when triggering a GitHub Workflow, including the cases when workflow is being triggered by Allure TestOps.

Parameters

• Parameters can be used by the workflow logic itself, e.g. we can select the VCS branch for execution, runners, or generally set parameters that change the resources used during the workflow execution.
• Parameters can be used by the automated tests, e.g. we can force tests using specific browser or specific browser version, or use different URLs that are defining the system under test.

Stages

1. Fist, we'll configure Allure TestOps so it'll be able to collect important environment information
2. Second, we'll configure job, so it'll allow us passing environment information to workflow.

Adding parameters for GitHub workflow

GitHub uses the inputs of workflow_dispatch event for passing parameters to workflow. To pass parameters to a workflow jobs/steps, you need to pass the values from the input parameters to an environment variable.

Declare inputs in the GitHub workflow

We've added workflow_dispatch with inputs in previous steps. Parameters we added before are required for Allure TestOps to trigger a workflow. Now we'll add inputs that will allow us passing parameters to workflows and automated tests.

Now our workflow looks like this:

name: GitHub integration with Allure TestOps

on:
 push:
   branches: [main]
 workflow_dispatch:
   inputs:
     ALLURE_JOB_RUN_ID:
       description: "ALLURE_JOB_RUN_ID service parameter. Leave blank."
       required: false
     ALLURE_USERNAME:
       description: "ALLURE_USERNAME service parameter. Leave blank"
       required: false
env:
 ALLURE_RESULTS: "build/allure-results"
 ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }}
 ALLURE_USERNAME: ${{ github.event.inputs.ALLURE_USERNAME }}
jobs:
 all-tests:
   runs-on: ubuntu-latest
   steps:
     - uses: actions/checkout@v3
     - name: Set up JDK 1.8
       uses: actions/setup-java@v1
       with:
         java-version: 1.8
     - name: Install and configure allurectl
       uses: allure-framework/[email protected]
       with:
         allure-endpoint: https://demo.testops.cloud
         allure-project-id: 111
         allure-token: ${{ secrets.ALLURE_TOKEN }}
     - name: Build with Gradle and upload to Allure TestOps
       run: |
         allurectl watch -- ./gradlew clean test

Defining new variable in workflow_dispatch

Let's add a new parameter TESTS_BROWSER, that will be defining the name of the browser application to be used in our tests.

We need to add it to the workflow_dispatch inputs, and then we need to pass the user's input to the workflow context.

Updated workflow_dispatch section will look like follows:

  workflow_dispatch:
    inputs:
      ALLURE_JOB_RUN_ID:
        description: "ALLURE_JOB_RUN_ID service parameter. Leave blank."
        required: false
      ALLURE_USERNAME:
        description: "ALLURE_USERNAME service parameter. Leave blank"
        required: false
# >>>>        
      TESTS_BROWSER:
        description: "Browser to be used in e2e tests"
        required: true
        default: chrome
# <<<<

Updated env section:

env:
  ALLURE_RESULTS: "build/allure-results"
  ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }}
  ALLURE_USERNAME: ${{ github.event.inputs.ALLURE_USERNAME }}
# >>>>
  TESTS_BROWSER: ${{ github.event.inputs.TESTS_BROWSER }}
# <<<<

As soon as we passed the value to the workflow context, this value can be used in our tests.

      - name: Build with Gradle and upload to Allure TestOps
        run: |
          allurectl watch -- ./gradlew clean test -Dbrowser=${TESTS_BROWSER}

Depending on the programming language, build tool, the way you pass the environment to your tests can vary of course.

Complete example workflow

name: GitHub integration with Allure TestOps

on:
 push:
   branches: [main]
 workflow_dispatch:
   inputs:
     ALLURE_JOB_RUN_ID:
       description: "ALLURE_JOB_RUN_ID service parameter. Leave blank."
       required: false
     ALLURE_USERNAME:
       description: "ALLURE_USERNAME service parameter. Leave blank"
       required: false
     TESTS_BROWSER:
       description: "Browser to be used in e2e tests"
       required: true
       default: chrome
env:
 ALLURE_RESULTS: "build/allure-results"
 ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }}
 ALLURE_USERNAME: ${{ github.event.inputs.ALLURE_USERNAME }}
 TESTS_BROWSER: ${{ github.event.inputs.TESTS_BROWSER }}
jobs:
 all-tests:
   runs-on: ubuntu-latest
   steps:
     - uses: actions/checkout@v3
     - name: Set up JDK 1.8
       uses: actions/setup-java@v1
       with:
         java-version: 1.8
     - name: Install and configure allurectl
       uses: allure-framework/[email protected]
       with:
         allure-endpoint: https://demo.testops.cloud
         allure-project-id: 111
         allure-token: ${{ secrets.ALLURE_TOKEN }}
     - name: Build with Gradle and upload to Allure TestOps
       run: |
         allurectl watch -- ./gradlew clean test -Dbrowser=${TESTS_BROWSER}

Why do we need this?

When allurectl executes tests and creates job-run on Allure TestOps side, it sends whole context of the tests execution to Allure TestOps. If not processed properly, then the context is just omitted and that's it, but if we want to see some important (for us, for the issues analysis) information, then we can have it in Allure TestOps as well.

Processing and viewing pipeline environment information

Allure TestOps has dedicated entities called Environment that are collecting the information on the environment used in the projects.

We need:

1. Define a global environment variable
2. Use this global environment variable in a project

Create global environment variable

1. Log in to Allure TestOps using an administrator account.
2. Go to Administration → Environment.
3. For each variable that you want to add:
   1. Click + Create.
   2. Enter a name for the variable, e.g. Browser (built-in), Branch, Product etc.
   3. Click Submit.

Map workflow environment variables to global environment variables

Once you've declared environment variables in your workflow and added global variables in Allure TestOps, you can configure your Allure TestOps project to start receiving this data:

1. In Allure TestOps, open your project.

2. Go to Settings → Environment.

3. For each variable that you want to use:

   1. Click + Create if the variable is not in the list. If the variable already exists, click the Edit icon next to its name.
   2. In the Mapping key field, specify the name of the environment variable as it's defined in env section of the workflow.
      1. in our example it's TESTS_BROWSER
   3. In the Environment variable field, select the global name from the previous step.
   4. Click Submit.

   

GitHub has default environment variables which are passed to Allure TestOps when allurectl is uploading test results. This means, we can get the information on the VCS branch name. The variable that contains the short name of the used branch is GITHUB_REF_NAME, so on the project level we also may want to create the mapping for this variable.

Use GITHUB_REF_NAME as mapping key and map this environment variable to global Branch environment created earlier.

Now, we are ready.

1. Go to GitHub → Actions
2. Select the required workflow
3. Press Run workflow
4. Select correct branch
5. Run workflow

Allure TestOps will process the information related to parameters and render this information in launches and test results.

Now we are ready to pass parameters from Allure TesOps.

Configure job to pass parameters to GitHub workflow

1. In Allure TestOps, open your project.

2. Go to Jobs.

3. Find the job you want to parametrize. Click ⋯ next to the job name and select Configure.

   • Job settings dialog will appear, which contains the Parameters section.

4. For each variable that you want to add, click Add and fill in the fields:

   • Name — name of the environment variable (same as Mapping key from step 3.3).
   • Value — default value you want to be used by default.
   • Environment Variable — Global environment variable we created earlier.

   

   Just to remind the stated before: Branch is GitHub specific, let's say – special, parameter. It is passed to the GitHub workflow context as a special attribute of the API call that triggers the workflow_dispatch. It is not supposed to set any variables in the workflow's content.

5. Click Submit.

One very important thing to keep in mind!

Whatever you put to the Parameters of a job on Allure TestOps side must be reflected in the inputs of the workflow_dispatch.

Situation like this (see image below) will result in the error 422 triggered by GitHub.

TESTS_OPERATING_SYSTEM is not present in workflow_dispatch inputs.

Run the configured job with parameter to be passed to tests

Now, we can trigger the pipeline and pass TESTS_BROWSER to our tests.

Run the configured job.

1. In the Environment section of the launch creation form click Add to set alternative parameters.
2. Set configured parameter (in the example it's Browser) to desired value.
3. Submit the form for the launch creation.

Parameter's value we've set will be then available in the Workflow context.

We've just completed both tasks

• We want to see what was the environment of executed workflow.
• We want to be able to provide parameters for tests execution when triggering a GitHub Workflow, including the cases when workflow is being triggered by Allure TestOps.

Examples

Below you can find two simple examples you can use as a basis for your GitHub workflow. It uses pytest as a testing framework and contains all the required settings to upload the test results to Allure TestOps and to trigger workflows from the Allure TestOps interface.

There are two ways you can add allurectl to your workflow:

• Using GitHub Action:

  1. Action will download allurectl.
  2. Action will add parameters based on the workflow context or the context provided by Allure TestOps.
  3. Action will set up the parameters required for uploading test results.

• Using a script inside your workflow (this option can be used if GitHub Action cannot be used like for example when using GitHub Enterprise:

  1. You need explicitly download allurectl to the workflow.
  2. Then use a script to get the parameters required for allurectl and Allure TestOps to work

Using GitHub action

This example uses the GitHub action that downloads allurectl and prepares all required parameters for bi-directional integration to work.

name: run-and-upload-to-testops

on:
  workflow_dispatch:
    inputs:
      TESTS_ENDPOINT:
        description: "System under test"
        required: true
        default: https://system.under.test
      TESTS_BROWSER:
        description: "Browser to be used in tests"
        required: true
        default: chrome
      ALLURE_JOB_RUN_ID:
        description: "Allure TestOps service parameter. Leave blank."
        required: false
      ALLURE_USERNAME:
        description: "Allure TestOps service parameter. Leave blank."
        required: false

env:
  ALLURE_RESULTS: "allure-results"
  ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }}

jobs:
  all-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - name: Install dependencies for pytest and allure framework integration
        run: |
          python -m pip install --upgrade pip
          pip install allure-pytest pytest
      - name: install and configure allurectl using GH Action
        uses: allure-framework/setup-allurectl@v1
        with:
          allure-endpoint: https://demo.testops.cloud
          allure-token: ${{ secrets.ALLURE_TOKEN }}
          allure-project-id: 9999
      - name: Run pytest tests
        run: |
          allurectl watch -- pytest ./test --alluredir=${ALLURE_RESULTS} --capture=no || true
        env:
          TESTS_ENDPOINT: ${{ github.event.inputs.TESTS_ENDPOINT }}
          TESTS_BROWSER: ${{ github.event.inputs.TESTS_BROWSER }}

GitHub Enterprise suitable example: script + download

This example can be used in github.com and when your team uses GitHub Enterprise and actions from the market cannot be installed.

name: pytest tests with allure framework

on:
  workflow_dispatch:
    inputs:
      TESTS_ENDPOINT:
        description: "System under test"
        required: true
        default: https://demo.testops.cloud
      TESTS_BROWSER:
        description: "Browser to be used in tests"
        required: true
        default: chrome
      ALLURE_JOB_RUN_ID:
        description: "Allure TestOps service parameter. Leave blank."
        required: false
      ALLURE_USERNAME:
        description: "Allure TestOps service parameter. Leave blank."
        required: false

env:
  ALLURE_ENDPOINT: ${{ secrets.ALLURE_ENDPOINT }}
  ALLURE_TOKEN: ${{ secrets.ALLURE_TOKEN }}
  ALLURE_PROJECT_ID: ${{ secrets.ALLURE_PROJECT_ID }}
  ALLURE_RESULTS: "allure-results"
  ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }}

jobs:
  tests:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'
    - uses: actions/github-script@v4
      id: allure-job-uid
      with:
        result-encoding: string
        script: |
          const result = await github.actions.getWorkflowRun({
            owner: context.repo.owner,
            repo: context.repo.repo,
            run_id: context.runId,
          });
          return `${context.repo.owner}/${context.repo.repo}/actions/workflows/${result.data.workflow_id}`
    - name: Download and make allurectl executable
      run: |
        wget https://github.com/allure-framework/allurectl/releases/latest/download/allurectl_linux_amd64 -O ./allurectl
        chmod +x ./allurectl
    - name: Install dependencies for pytest and allure framework integration
      run: |
        python -m pip install --upgrade pip
        pip install allure-pytest pytest
    - name: Test with pytest and gen allure results
      shell: bash
      working-directory: ${{ github.action_path }}
      env:
        ALLURE_JOB_UID: ${{steps.allure-job-uid.outputs.result}}
        TESTS_ENDPOINT: ${{ github.event.inputs.TESTS_ENDPOINT }}
        TESTS_BROWSER: ${{ github.event.inputs.TESTS_BROWSER }}
      run: |
        ./allurectl watch -- pytest --alluredir=${ALLURE_RESULTS}

Step id: allure-job-uid is mandatory top enable triggering of the workflow from Allure TestOps side.

ALLURE_JOB_UID uses the results of the execution of id: allure-job-uid to pass correct information to Allure TestOps.

Skipping these will break run and re-run actions from Allure TestOps side.

Other parameters or use-cases for allurectl can be found in the section dedicated to allurectl.

Troubleshooting

Error 422 when triggering GitHub workflow from Allure TestOps interface

General cause

Usually, error 422 occurs when you are trying to launch a workflow using workflow_dispatch and the parameters defined in inputs of workflow_dispatch do not match the parameters sent by Allure TestOps.

Expected configuration

As mentioned earlier, there are at least two parameters (with the required attribute set to false) that must be included as inputs in your workflow_dispatch section in order for Allure TestOps to have the ability to trigger the workflow:

• ALLURE_JOB_RUN_ID,
• ALLURE_USERNAME.

The values for these parameters will be set by Allure TestOps automatically.

All additional parameter values (e.g., browser, system under test, etc.) must be specified in the Job configuration dialog when launching the workflow using the Allure TestOps interface.

Example

name: Integration of Allure TestOps with GitHub 

on:
  workflow_dispatch:
    inputs:
      ALLURE_JOB_RUN_ID:
        description: "ALLURE_JOB_RUN_ID service parameter. Leave blank."
        required: false
      ALLURE_USERNAME:
        description: "ALLURE_USERNAME service parameter. Leave blank"
        required: false
      TEST_BROWSER:
        description: "Browser for tests"
        required: true
        default: chrome
      TEST_ENDPOINT:
        description: "System under test"
        required: false
        default: https://staging.system.under.test

For the above configuration, you need to set the following parameters in Allure TestOps:

1. TEST_BROWSER

   Must be set because of required: true.

2. TEST_ENDPOINT

   Can be set or omitted because of required: false.

3. Branch

   Must be set. Will be linked to the Branch environment variable (both starting with capital B).

   This is a special case variable that is not explicitly listed in inputs but must be set because it is required by the API. Its value will be used for the ref parameter.

Examples of GitHub behavior when trying to run a job

Will work correctly. All parameters are in sync with inputs
Will work correctly. TEST_BROWSER parameter is set, as required by inputs. TEST_ENDPOINT is not set, but it is not required.
Will not work and will cause error 422. RELEASE_ID parameter is set but does not exist in inputs.
Will not work and will cause error 422. TEST_BROWSER parameter is not set, but is required by inputs.

Solution

The only possible solution is to synchronize the configuration parameters of the job created in Allure TestOps with your inputs in workflow_dispatch.