Integration with GitHub

This integration is a specific case of Integrations with external systems.

Allure TestOps integration with GitHub is bidirectional, this means you can do the following.

CI: Upload test results from a GitHub workflow (GitHub > Allure TestOps)
- GitHub upload workflow uses allurectl to upload the test results.
CI: Trigger the dispatch of a GitHub workflow (Allure TestOps > GitHub)

Uploading the test results from GitHub

To upload test results from GitHub you need to use allurectl.

Authenticate GitHub workflows

The upload process requires authentication of allurectl in Allure TestOps, i.e. you need to generate an API token in a user's profile on Allure TestOps side.

Authentication token

For a user on Allure TestOps side you need to create API token. This API token will be used by GitHub to authenticate on Allure TestOps server.

Log in to Allure TestOps with account you are going to use to upload test results.
Go to user's profile.

In the section API tokens click a green button that says Create.
Name your API token and click Submit. Always give meaningful names to all the configuration parameters.

Allure go to profile

Allure will randomly generate the token and show you in modal window.

Copy the token and save it in a safe place as it cannot be retrieved by Allure TestOps means.

Add your Allure server instance information to GitHub configuration

The next you need to provide the Allure TestOps instance details to your GitHub repo, so it could be used by allurectl for the authentication and test results upload.

Navigate to the Settings of your repository.
Navigate to Secrets section.
Create 3 variables in Repository secrets as shown below:
1. ALLURE_ENDPOINT - Allure TestOps URL (for example, http://demo.testops.cloud).
2. ALLURE_TOKEN - Allure TestOps token you generated recently.
3. ALLURE_PROJECT_ID - Allure TestOps project ID to which you are going to upload the results of your tests.

These secrets will be used then in your workflows to define environment variables.

To upload the test results to a project in Allure TestOps the end user used to create API token must be explicitly added to the said project with at least Write permissions.

Add secrets to GitHub repository

Configure GitHub workflow

Here is an example of GitHub workflow you can use with Allure TestOps and allurectl.

For GitHub workflows we have prepared allurectl installation and configuration action. This action downloads allurectl into your GitHub workflow, and creates all necessary variables required for normal work of allurectl.

Defined inputs that can be changed by an end user before to running the workflow from GitHub UI. These inputs' values are used to provide the data to environment variables like browser name, or URL which tests will use.

All the variables starting with ALLURE are mandatory to run allurectl properly.

name: upload test results to demo.testops.cloud

on:
  push:
  workflow_dispatch:
    inputs:
      TEST_ENDPOINT:
        description: "Endpoint for tests"
        required: true
        default: https://dev.github.com
      TEST_BROWSER:
        description: "Browser for tests"
        required: true
        default: chrome
      ALLURE_JOB_RUN_ID:
        description: "ALLURE_JOB_RUN_ID service parameter. Leave blank."
        required: false
      ALLURE_USERNAME:
        description: "ALLURE_USERNAME service parameter. Leave blank"
        required: false

env:
  ALLURE_ENDPOINT: https://testing.allure.aws.qameta.in
  ALLURE_TOKEN: ${{ secrets.ALLURE_TOKEN }}
  ALLURE_PROJECT_ID: ${{ secrets.ALLURE_PROJECT_ID }}
  ALLURE_TESTPLAN_PATH: "./testplan.json"
  ALLURE_RESULTS: "build/allure-results"
  ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }}

jobs:
  all-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/[email protected]
      - name: Set up JDK 1.8
        uses: actions/[email protected]
        with:
          java-version: 1.8
      - name: install and configure allurectl
        uses: allure-framework/[email protected]
        with:
          allure-endpoint: https://testing.allure.aws.qameta.in
          allure-token: ${{ secrets.ALLURE_TOKEN }}
          allure-project-id: ${{ secrets.ALLURE_PROJECT_ID }}
      - name: Prepare TestPlan
        run: allurectl job-run plan --output-file ${ALLURE_TESTPLAN_PATH}
      - name: Build with Gradle
        run: | 
          allurectl watch -- ./gradlew clean test
          export $( allurectl job-run env)
          printenv | grep TEST_
          printenv | grep ALLURE_
        env:
          TEST_ENDPOINT: ${{ github.event.inputs.TEST_ENDPOINT }}
          TEST_BROWSER: ${{ github.event.inputs.TEST_BROWSER }}
      - name: Get ALLURE_ environments just to check
        run: | 
          printenv | grep ALLURE_
      - name: Get GITHUB_ environments just to check
        run: | 
          printenv | grep GITHUB_

Gathering environment information from GitHub workflow

When we say environment we consider the context of a pipeline (GitHub workflow) or context of a job inside a pipeline.

Context is usually stored in the environment variables like predefined GitHub variables or user-defined ones which can be used in Allure TestOps to distinguish results received from different jobs (here we can use predefined variables) or for different test environmens (here we use the information about a application under test, browser used and so on, i.e. data you pass to your tests).

Environment information can be used to

filter the test results generated with this environment information, e.g. you can filter out the test results executed using a certain browser name or executed in certain job of a pipeline
use the collected information to trigger CI pipelines with certain environment parameters

Preconditions

Environment variables need to be created in the Environment section, then they can be used in projects to map pipelines' env data to them.

To properly collect environment variables from GitLab, first you must understand what environment data you have in your pipeline.

Go to Workflow file in your GitHub repo (located at .github/workflows/worklow_file.yaml).
after allurectl execution add printenv command

The command printenv can be deleted later on.

Gathering the environment

Now, when you start the pipeline, the pipeline execution console will display all environment variables, so you can create mapping in TestOps.

For example we will add the mapping for Branch variable created on the global level (by an admin in Administration section).

GitHub workflow log

Go to Project > Settings > Environment and click Create.

Creating environment schemas

Input variable from GitHub to Mapping Key field.
In the drop-down list select the Environment variable that was created on the Global level for mapping.

Mapping branch from GitHub

Click Submit.

Now, if you start the pipeline, you will see the value of the added variable on the page with Launches in the Environments section.

Mapped branch from GitHub

Job and its configuration

Job is an Allure TestOps entity linking your Allure TestOps project with the pipeline where you send the test results from.

Job is created by Allure TestOps automatically after the first test results upload from a pipeline. Automatically created job requires configuration.

The configuration of the job for the triggering of the GitHub workflows is only possible when the integration with GitHub has been completed.

Setting up the integration with GitHub

Integration with GitHub allows you triggering GitHub workflows from Allure TestOps UI.

Preconditions

You have ROLE_ADMIN authority to configure Allure TestOps instance on the global level.
You have GitHub access token.
- either a classic access token
- or fine-grained token

How to generate GitHub token

Log into GitHub and in the upper-right corner click on your avatar, select Settings.
On the left hand side locate Developer settings section (on the very bottom of the page) and click it
In Developer settings click Personal access tokens
Select Generate new token
Select the token type
Assign required scope for the token being created

As soon as the Token has been generated, copy it and save until we use it in the setting of the integration with GitHub.

Classic token scope

For the triggering the workflows classic token need to have the scope workflow.

GitHub classic token scope

Fine-grained token scope

For the triggering the workflows fine-grained token need to have the following scope:

Correct resource owner of the repository where you are going to trigger workflows
Limited by the repositories where you are going to trigger workflows
With the permission to read and write Actions.

GitHub fine-grained token scope

Enabling GitHub integration in Allure TestOps

You need to have ROLE_ADMIN authority to perform this action

Go to User menu > Administration > Integrations.
In the upper-right corner click Add integration.
Search for GitHub.
Click on the GitHub banner.
Provide the name of integration.
- URL of your GitHub without http(s):// might be a good idea for the name.
Provide the GitHub API endpoint to the Endpoint field
- https://api.github.com for free/team/pro/enterprise cloud versions of GitHub
- http(s)://HOSTNAME/api/v3 for GitHub Enterprise server (in-house deployment)
Finalize by clicking Add integration.

Next you'll see the following screen:

Integration

Testing connection

Click Test connection.
Enter your Token for access (generated in GitHub).
Click Test connection.
After a successful connection test, click Save changes.

The token won't be saved for the integration at this stage, and it needs to be provided when adding the integration to your project.

Project level integration

As soon as the GitHub integration is enabled for the entire Allure TestOps server, you can use it in your projects.

Preconditions

GitHub integration enabled on Global level by Allure TestOps admin.
You are Project Owner, i.e. you have permissions to configure a project.

Adding GitHub integration to a project

Click on Settings > Integrations.
Click Add integration on integration you or admin created at Global level.

Add integration to a project

Enter your Token for access (generated in GitHub).
Click Test connection.
After a successful connection test, click Add integration.

Add project ID and token

Trigger GitHub pipelines from TestOps

Preconditions

You need to have integration with GitHub added on the level of your project.
Job must be created in your project
- for a job to be created in Allure TestOps just execute your workflow from GitHub and close the created launch
Integration needs to be added on the next stage of the Job's configuration.
Job must be allowed to trigger pipelines on next stages.
At least one (special) parameter Branch needs to be added to the Job configuration.

Job's initial state

Job final configuration

To trigger the pipelines execution from Allure TestOps a job must be properly configured.

Go to the Jobs page.
Open the Configure menu of the created job.

Go to job configuration

In Configure settings fill the following
1. Build server is the name of the integration with GitHub.
2. Allow the job to be used to run tests.
3. Add the job's parameters you want to pass to your pipeline

Go to job configuration