Integration with GitHub
The integration between Allure TestOps and GitHub allows you to set up the following relations:
- One job in Allure TestOps corresponds to one GitHub workflow.
- One launch in Allure TestOps corresponds to one GitHub workflow run.
A new workflow run can be triggered by either Allure TestOps or GitHub itself, with both parties displaying its status in their web interfaces.
The integration supports both the main GitHub server (github.com) and GitHub Enterprise Server installations.
For the integration to work properly, you will need to modify your workflows so that they use the setup-allurectl Action. This Action will set up the allurectl utility to communicate with Allure TestOps. Before running the tests, the utility receives the test plan, i.e. the selection of tests that need to run. Then, while the tests are running, the utility regularly scans for new files in the test results directory (e.g., “build/allure-results”, see Allure Report → How it works) and uploads them to the Allure TestOps server. This way, Allure TestOps receives the test results as soon as possible and is able to show a launch's partial results even before the job is finished.
To add Allure TestOps support in your GitHub project:
- Enable sending data from GitHub.
- Enable triggering GitHub workflows.
- Parametrize workflows (if necessary).
1. Enable sending data from GitHub
For GitHub to send workflow statuses and test results to Allure TestOps, you need to generate an authentication token in Allure TestOps, add the token to GitHub, and modify the workflow itself.
Then, run and check the workflow to make sure that everything works so far.
1.1. Create a token in Allure TestOps
In Allure TestOps, click your avatar and go to API Tokens.
Click + Token.
Enter a name for the token (e.g., "Token for GitHub") and click Create.
Allure TestOps will generate the token and display it in a modal window.
Click the Copy icon to copy the token into clipboard. You will need this token in the next step.
1.2. Specify the token in GitHub
In GitHub, open the project and go to its Settings.
In the menu on the left, click Secrets and variables → Actions.
Click New repository secret.
Fill in the fields:
- Key — “ALLURE_TOKEN”.
- Value — the API token that you got in step 1.1.
Click Add secret.
1.3. Modify the workflows
To make changes to a workflow, edit its YAML file in the .github/workflows directory of your GitHub repository. You can do it either locally or in the GitHub's web-based text editor.
For each workflow that runs tests, do the following:
Declare workflow inputs for the
workflow_dispatch
event:ALLURE_JOB_RUN_ID
ALLURE_USERNAME
It's mandatory to add these variables. If the variables won't be added, triggering of workflows from Allure TestOps side won't work and you will get 422 error.
You will also get 422 error if the job parameters specified in Allure TestOps (on the left side of the form) do not match the
inputs
forworkflow_dispatch
.Add or extend the workflow's
env
block. It must include:ALLURE_ENDPOINT
— the URL of the Allure TestOps server (e.g., “https://demo.testops.cloud/”).ALLURE_PROJECT_ID
— the ID of the Allure TestOps project, e.g., “1”. Id stays before the project name in Allure TestOps UI, and the id is present in the address field when you are working in Allure TestOps.ALLURE_TOKEN
—${{ secrets.ALLURE_TOKEN }}
.ALLURE_JOB_RUN_ID
—${{ github.event.inputs.ALLURE_JOB_RUN_ID }}
.ALLURE_RESULTS
— the path to the test results directory (e.g., “build/allure-results”). If there are multiple test result directories in your project, you can separate them with commas or use a wildcard pattern, e.g., “modules/*/build/allure-results”.
Before running tests, add two new steps:
- A step that downloads the allurectl utility. It is recommended to use the allure-framework/setup-allurectl@v1 Action for this.
- There is another option to download and setup allurectl without this Action. We'll provide both examples below.
Wrap the command that runs the tests into the
allurectl watch
command.If your Allure TestOps server uses a self-signed SSL certificate, use
allurectl --insecure watch
instead.
Example
Assume we have a Java project with a workflow file like this:
name: Run tests on: push: jobs: all-tests: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up JDK uses: actions/setup-java@v3 with: distribution: oracle java-version: 17 - name: Build with Gradle run: ./gradlew clean test
With Allure TestOps integration, the file will look something like this:
name: Run tests on: push: workflow_dispatch: inputs: ALLURE_JOB_RUN_ID: description: ALLURE_JOB_RUN_ID service parameter. Leave blank. ALLURE_USERNAME: description: ALLURE_USERNAME service parameter. Leave blank. env: ALLURE_TOKEN: ${{ secrets.ALLURE_TOKEN }} ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }} ALLURE_ENDPOINT: https://demo.testops.cloud/ ALLURE_PROJECT_ID: 1 ALLURE_RESULTS: build/allure-results jobs: all-tests: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Set up JDK uses: actions/setup-java@v3 with: distribution: oracle java-version: 17 - name: Install allurectl uses: allure-framework/setup-allurectl@v1 - name: Build with Gradle run: allurectl watch -- ./gradlew clean test
allurectl setup action takes the following parameters:
allure-endpoint
Mandatory parameter, setting the URL address of Allure TestOps instance to upload the test results to.allure-token
Mandatory parameter used for the authentication of allurectl at Allure TestOps server defined byallure-endpoint
parameter.allure-project-id
Mandatory parameter to define the project (numeric ID is used) where launch with test results has to be created.allurectl-version
Optional parameter allowing you to select specific version of allurectl to be used. Information on the available releases can be found here.github-token
Optional parameter to provide an alternative GitHub token.
1.4. Run and check the workflow
In GitHub, open the project and go to the workflow run triggered by the latest commit.
If the run is not yet finished, wait until it finishes.
In the run details, click the job that runs tests.
Close to the end of its log, there should be a link to the test report at Allure TestOps. Make sure that it is present and works.
In the test report at Allure TestOps, open a single test's results.
At the bottom of the page, there should be a link back to the GitHub workflow run. Make sure that it is present and works.
2. Enable triggering GitHub workflows
On the Allure TestOps side, the integration with GitHub needs to be configured on two levels.
First, the administrator needs to specify the URL of GitHub server.
Then, any project owner needs to create an authentication token in GitHub and add it to Allure TestOps.
2.1. Specify GitHub server in Allure TestOps
Log in to Allure TestOps using an administrator account.
Go to Administration → Integrations.
Click + Add integration in the top right corner of the page.
In the dialog that appears, select GitHub.
Fill in the fields:
- Name — a name to help you recognize the GitHub server (e.g., “GitHub production”).
- Endpoint — the base URL of GitHub. For github.com, use “https://github.com”. For GitHub Enterprise Server, use the URL of your GitHub instance.
- Endpoint for API calls — the URL of GitHub API. For github.com, use “https://api.github.com”. For GitHub Enterprise Server, use “⟨URL⟩/api/v3”, where ⟨URL⟩ is the URL of your GitHub instance.
If you are using GitHub Enterprise Server with a self-signed SSL certificate, check the Disable certificate validation box.
Click Add integration.
2.2. Create a token in GitHub
To be able to trigger workflows, Allure TestOps needs to have a personal access token generated by the GitHub server. You can choose the type of token that GitHub will generate: fine-grained or classic (see Managing your personal access tokens in the GitHub documentation).
Instructions for generating a token vary slightly depending on the type.
In GitHub, click your avatar and go to Settings.
In the menu on the left, click Developer settings.
Go to Personal access tokens → Fine-grained tokens.
Click Generate new token.
Fill in the fields:
- Token name — a name to help you recognize the token (e.g., “Token for Allure TestOps”).
- Expiration — specify how long the token should be valid. After this date, the integration will stop working, and you will need to create a new token to continue using the integration.
Under the Repository access section, click Only select repositories. In the drop-down list that appears, select the repositories you want to use.
Under the Permissions section, click Repository permissions. In the permission list that appears, find Actions and select Read and write next to it.
If you plan to add links to GitHub issues, additionally find Issues and select Read and write next to it.
Click Generate token.
The page will reload, and the new token will become temporarily visible. Click the Copy icon next to it.
You will need this token on the next step.
In GitHub, click your avatar and go to Settings.
In the menu on the left, click Developer settings.
Go to Personal access tokens → Tokens (classic).
Click Generate new token → Generate new token (classic).
Fill in the fields:
- Note — a name to help you recognize the token (e.g., “Token for Allure TestOps”).
- Expiration — specify how long the token should be valid. After this date, the integration will stop working, and you will need to create a new token to continue using the integration.
Under the Select scopes section, check workflow.
If you plan to add links to GitHub issues, additionally check repo.
Click Generate token.
The page will reload, and the new token will become temporarily visible. Click the Copy icon next to it.
You will need this token on the next step.
2.3. Add the token to an Allure TestOps project
- In Allure TestOps, open your project.
- Go to Settings → Integrations.
- Under Available integrations, find the GitHub integration and click Add integration next to it.
- Under Secret, enter the GitHub access token you created in step 2.2.
- Click Test connection. If the token is correct, a "Connection established" message will appear within a few seconds.
- Click Add integration to save the settings.
3. Parametrize workflows
GitHub uses inputs for passing parameters to workflows' context, i.e. you need to create an input variable which value then will be assigned to an environment variable in the context of your workflow execution and then this value can be used in the tests.
Allure TestOps integrates this feature with its Environment concept, which lets you both set parameters for new jobs, and see parameters set in workflows that were started from CI side.
Despite the tool (allurectl or a CI plugin) used to upload the test results to Allure TestOps, whole pipeline's context (in case of GitHub – the context of a workflow) is transferred to Allure TestOps, and can be used to link environment data to the uploaded test results.
3.1. Declare inputs in the GitHub workflow
To make changes to a workflow, edit its YAML file in the .github/workflows directory of your GitHub repository. You can do it either locally or in the GitHub's web-based text editor.
Add workflow dispatch inputs, their descriptions, and default values into the inputs
block. For example:
name: Run tests
on:
workflow_dispatch:
inputs:
ALLURE_JOB_RUN_ID:
description: ALLURE_JOB_RUN_ID service parameter. Leave blank.
ALLURE_USERNAME:
description: ALLURE_USERNAME service parameter. Leave blank.
TESTS_BROWSER:
description: Browser to use for tests
default: chrome
PRODUCT_VERSION:
description: Product version
default: '1.23'
...
You can use these inputs later in your workflow to pass the information to your tests via environment variables.
env:
PRODUCT_VERSION: ${{ github.event.inputs.PRODUCT_VERSION }}
TESTS_BROWSER: ${{ github.event.inputs.TESTS_BROWSER }}
These environment variables, along with others, will be sent by allurectl to Allure TestOps within the build context.
3.2. Add global environment variables names
Before using the environment variable data from a GitHub workflow, or generally from any kind of pipeline, you need to create a global environment variable which will store such a data.
Log into Allure TestOps using an administrator account.
Go to Administration → Environment.
For each variable that you want to add:
Click + Create.
Enter the variable name.
Click Submit.
3.3. Map workflow environment variables to global environment variables
After you have an environment variable declared in your workflow (pipeline), and created global environment variable in Allure TestOps, you can start configuring your project environment variables to extract the information you need from workflow's context.
In Allure TestOps, open your project.
Go to Settings → Environment.
For each variable that you want to use:
Click + Create if the variable is not in the list. If the variable already exists, click the Edit icon next to its name.
In the Mapping key field, specify the name of the environment variable.
In the Environment variable field, select the global name from step 3.2.
Click Submit.
3.4. Add parameters to the job
In Allure TestOps, open your project.
Go to Jobs.
Find the job which you want to parametrize. Click
⋯
next to the job name, then select Configure.The job settings dialog will appear, containing the Parameters section.
For each variable that you want to add, click Add and fill in the fields:
- Name — the name of the environment variable (same as the Mapping key from step 3.3).
- Value — the default value that should be used unless overridden for a specific launch.
- Environment Variable — the environment variable from step 3.2.
Click Submit.
Note that it is recommended to set the same default values both in Allure TestOps and GitHub. This means that a workflow run will get the same environment regardless of what triggered it.
3.5. Executing a workflow with certain environment data
In the previous steps we configured a job on Allure TestOps side, which now has all the parameters we want to pass to our GitHub workflow.
Configured job parameters are the ones, you will be able to pass to your pipeline when triggering CI pipelines from Allure TestOps UI.
Parameter rendered as the first one in the list, is the name of an environment variable that we are going to set on CI side (e.g. TEST_BROWSER). The parameter in the parenthesis in the name of a global environment variable which will provide the suggestions for the values (e.g. Browser). And the value (in our example it's "chrome") is the default value we are going to push to a CI system if no alternative value.
To execute tests on certain environment you need to invoke one of the following actions:
Please note, reruns of the tests are always executed on the same environment as it was set during the initial execution.
When you execute any of the said actions, you will see the following form for the creation of a launch.
You need to click the button saying Add in the Environment section.
Application will add the job parameters to the form and pre-fill these with the job parameters default values.
Then you can alter the values you want to pass to the CI pipeline.
If you need to execute your pipeline several times on different sets of the parameters, you need to create a new set of the environment variables and Allure TestOps will request CI to run several pipelines with different sets of parameters.
Examples
Below is a very simple workflow, you can use as a start for your GitHub workflow to try the execution of tests and uploading the data to Allure TestOps. It uses pytest as an example, and it contains all the elements required to upload the test results to Allure TestOps and allow the triggering of workflows from Allure TestOps side.
As we mentioned before, there are two ways to add allurectl to the workflow.
- using GitHub Action:
- Action will download allurectl.
- Action will generate all the required parameters based on workflow context or context provided from Allure TestOps.
- Action will provide configuration parameters required for the test results upload.
- using script inside your workflow (this option is used when GitHub cannot be used due to certain restrictions):
- Script will download allurectl.
- Script will generate the required parameters based on workflow context.
name: run-and-upload-to-allure-testops
on:
workflow_dispatch:
inputs:
GITHUB_TESTS_ENDPOINT:
description: "System under test"
required: true
default: https://system.under.test
GITHUB_TESTS_BROWSER:
description: "Browser to be used in tests"
required: true
default: chrome
ALLURE_JOB_RUN_ID:
description: "ALLURE_JOB_RUN_ID is service parameter required for triggering workflows from Allure TestOps. Leave blank."
required: false
ALLURE_USERNAME:
description: "ALLURE_USERNAME service parameter. Leave blank"
required: false
env:
ALLURE_RESULTS: "allure-results"
ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }}
jobs:
all-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies for pytest and allure framework integration
run: |
python -m pip install --upgrade pip
pip install allure-pytest pytest
- name: install and configure allurectl using GH Action
uses: allure-framework/setup-allurectl@v1
with:
allure-endpoint: https://demo.testops.cloud
allure-token: ${{ secrets.ALLURE_TOKEN }}
allure-project-id: 9999
- name: Run pytest tests
run: |
allurectl watch -- pytest ./test --alluredir=${ALLURE_RESULTS} --capture=no || true
env:
GITHUB_TESTS_ENDPOINT: ${{ github.event.inputs.GITHUB_TESTS_ENDPOINT }}
GITHUB_TESTS_BROWSER: ${{ github.event.inputs.GITHUB_TESTS_BROWSER }}
GITHUB_TESTS_BRANCH: ${{ github.ref_name }}
name: pytest tests with allure framework
on:
workflow_dispatch:
inputs:
GITHUB_TESTS_ENDPOINT:
description: "System under test"
required: true
default: https://system.under.test
GITHUB_TESTS_BROWSER:
description: "Browser to be used in tests"
required: true
default: chrome
ALLURE_JOB_RUN_ID:
description: "Inner parameter for Allure TestOps"
required: false
ALLURE_USERNAME:
description: "ALLURE_USERNAME service parameter. Leave blank"
required: false
env:
ALLURE_ENDPOINT: ${{ secrets.ALLURE_ENDPOINT }}
ALLURE_TOKEN: ${{ secrets.ALLURE_TOKEN }}
ALLURE_PROJECT_ID: ${{ secrets.ALLURE_PROJECT_ID }}
ALLURE_TESTPLAN_PATH: "./testplan.json"
ALLURE_RESULTS: "allure-results"
ALLURE_JOB_RUN_ID: ${{ github.event.inputs.ALLURE_JOB_RUN_ID }}
GH_BRANCH: ${{ github.ref_name }}
ALLURE_LAUNCH_TAGS: "github, demo, pytest, $GH_BRANCH"
jobs:
tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v4
with:
python-version: '3.11'
- uses: actions/github-script@v4
id: allure-job-uid
with:
result-encoding: string
script: |
const result = await github.actions.getWorkflowRun({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: context.runId,
});
return `${context.repo.owner}/${context.repo.repo}/actions/workflows/${result.data.workflow_id}`
- name: Download and make allurectl executable
run: |
wget https://github.com/allure-framework/allurectl/releases/download/2.15.1/allurectl_linux_amd64 -O ./allurectl
chmod +x ./allurectl
- name: Install dependencies for pytest and allure framework integration
run: |
python -m pip install --upgrade pip
pip install allure-pytest pytest
- name: Test with pytest and gen allure results
shell: bash
working-directory: ${{ github.action_path }}
env:
ALLURE_JOB_UID: ${{steps.allure-job-uid.outputs.result}}
GITHUB_TESTS_ENDPOINT: ${{ github.event.inputs.GITHUB_TESTS_ENDPOINT }}
GITHUB_TESTS_BROWSER: ${{ github.event.inputs.GITHUB_TESTS_BROWSER }}
run: |
./allurectl watch -- pytest --alluredir=${ALLURE_RESULTS}
Step with id: allure-job-uid
is a mandatory one if you are going to trigger the workflows from Allure TestOps.
Troubleshooting
Error 422 when triggering GitHub workflow via Allure TestOps interface
Root cause
Generally, the error 422 is returned by GitHub when you are trying to start workflow using workflow_dispatch
, and the parameters defined in the inputs
of workflow_dispatch
do not match those sent from Allure TestOps.
Expected configuration
As mentioned earlier in the instruction, there are at least two mandatory parameters (with the
required
attribute set tofalse
) that must be included asinputs
in theworkflow_dispatch
section for Allure TestOps to trigger a workflow:
ALLURE_JOB_RUN_ID
,ALLURE_USERNAME
.These parameters are set by Allure TestOps automatically without involving any user's actions. Both are necessary for successfully triggering workflows.
If your pipeline (workflow) logic requires additional mandatory data (e.g., browser name, system under test name), these parameters must be configured in Allure TestOps as job configuration parameters.
Example
name: Integration of Allure TestOps with GitHub
on:
workflow_dispatch:
inputs:
ALLURE_JOB_RUN_ID:
description: "ALLURE_JOB_RUN_ID service parameter. Leave blank."
required: false
ALLURE_USERNAME:
description: "ALLURE_USERNAME service parameter. Leave blank"
required: false
TEST_BROWSER:
description: "Browser for tests"
required: true
default: chrome
TEST_ENDPOINT:
description: "System under test"
required: false
default: https://staging.system.under.test
For the described configuration of inputs
for workflow_dispatch
on the Allure TestOps side, you must have the following configuration parameters for the created job:
Branch:
- Linked to the Branch environment variable (both starting with capital B).
- This is a special environment variable not explicitly listed in the
inputs
, but its value is passed toref
parameter in the API call. Therefore, this variable is mandatory (cannot be omitted).
TEST_BROWSER:
- Passes the browser name to
workflow_dispatch
. - Must be included in the job configuration parameters as it is defined in the
workflow_dispatch
asrequired: true
.
- Passes the browser name to
TEST_ENDPOINT:
- May be included as an optional job configuration parameter as it is defined in the
workflow_dispatch
asrequired: false
.
- May be included as an optional job configuration parameter as it is defined in the
Examples of GitHub behavior when you try executing the configured job
Will work correctly. All parameters are present in inputs |
![]() |
Will work correctly. The TEST_BROWSER parameter is present, which is mandatory for inputs in workflow_dispatch |
![]() |
Won't work, error 422. The RELEASE_ID parameter is not present in inputs of workflow_dispatch |
![]() |
Won't work, error 422. The TEST_BROWSER parameter is missing, which is mandatory for inputs in workflow_dispatch |
![]() |
Solution
The only possible solution is to synchronize the configuration parameters of the job created in Allure TestOps with the inputs
in workflow_dispatch
.