General information on test results upload

Test results upload

The aim of this chapter is to describe the general workflow of test results upload to Allure TestOps.

Options

Here are the options how can you upload the test results.

  1. CI plug-in or allurectl from a CI pipeline (check this very page).
    1. Allure TestOps plug-in for Jenkins
    2. allurectl for the rest of CI systems with scripted pipelines.
  2. Manual upload of test result via UI. Use upload button in Launches section or just drag these to Allure TestOps UI in launches section.
  3. Use Allure: Upload results menu item in JetBrains’ IDE. Check here for the details.

General process of the upload for the test results from CI

Important to understand – when it comes to the sending of the test results to Allure TestOps, the tests do not send the test results.

Tests are doing what they’re supposed to do – checking your application as you’ve programmed them. Full stop.

Here is the description on how your test results end up in Allure TestOps, please check the picture below and read the description.

Test Results upload

  1. CI’s pipeline triggers Allure TestOps’ agent which ultimately is responsible for the upload of the test results.
    • In case of GitLab, GitHub, CircleCI this will be allurectl.
    • In case of Jenkins this will be the Allure TestOps plugin for Jenkins.
  2. Allure TestOps agent gets the connection params from the settings, receives data from Allure TestOps (if the execution is initiated from Allure TestOps) and triggers the tests execution.
  3. Allure TestOps agent starts watching the allure-results folder.
  4. Your code executes the tests.
  5. Tests’ logic uses test framework.
  6. Test’s logic uses allure framework which listens to the events of test framework.
  7. Allure framework saves the events from a test framework to test results files.
  8. Allure TestOps agent sends the test results in async mode to Allure TestOps.

CI pipeline preparation

Despite the CI you are using, there are some steps you need to perform for all of them:

  • Authenticate and authorize your pipeline to upload the test results

Authenticate your pipeline

This step requires the generation of API key, which is to be provided to an Allure TestOps’ agent.

  1. Log in to Allure TestOps you are going to use to upload test results.
  2. Go to user’s profile.

Allure go to profile

  1. In the section API tokens click a button saying + Create.
  2. Name your API token and click Submit.

Allure go to profile

Allure will randomly generate the token and show you in modal window.

  1. Copy the token and save it in a safe place as it cannot be retrieved by Allure TestOps means.
  2. This string is to be provided to CI pipeline later on.

Authorization

To upload test results to a project the user whose API token is used in a pipeline, needs to be added to this project with Write access rights.

If you are trying to upload test results using API token of a user which is not added to the project you are working with, then no data will be uploaded.

Supported CI servers

The table below shows the information on the Allure TestOps agent type needed for different CI servers.

Basically, the test results upload is possible from any CI which supports external tool usage, i.e. download allurectl using wget/curl during the execution and then using allurectl to trigger the tests and upload the test results.

Here is a list of CI servers already tested and widely used with Allure TestOps.

Integration Agent to be used
Azure DevOps server allurectl
Bamboo Allure TestOps plug-in for Bamboo
CircleCI (WIP) allurectl
GitHub allurectl
GitLab allurectl
Jenkins Allure TestOps plug-in for Jenkins or allurectl
TeamCity Allure TestOps plug-in for TeamCity
Tekton allurectl