Test results upload
The aim of this chapter is to describe the general workflow of test results upload to Allure TestOps.
Here are the options how can you upload the test results.
- CI plug-in or allurectl from a CI pipeline (check this very page).
- Allure TestOps plug-in for Jenkins
- allurectl for the rest of CI systems with scripted pipelines.
- Manual upload of test result via UI. Use upload button in Launches section or just drag these to Allure TestOps UI in launches section.
- Use Allure: Upload results menu item in JetBrains’ IDE. Check here for the details.
General process of the upload for the test results from CI
Important to understand – when it comes to the sending of the test results to Allure TestOps, the tests do not send the test results.
Tests are doing what they’re supposed to do – checking your application as you’ve programmed them. Full stop.
Here is the description on how your test results end up in Allure TestOps, please check the picture below and read the description.
- CI’s pipeline triggers Allure TestOps’ agent which ultimately is responsible for the upload of the test results.
- In case of GitLab, GitHub, CircleCI this will be allurectl.
- In case of Jenkins this will be the Allure TestOps plugin for Jenkins.
- Allure TestOps agent gets the connection params from the settings, receives data from Allure TestOps (if the execution is initiated from Allure TestOps) and triggers the tests execution.
- Allure TestOps agent starts watching the allure-results folder.
- Your code executes the tests.
- Tests’ logic uses test framework.
- Test’s logic uses allure framework which listens to the events of test framework.
- Allure framework saves the events from a test framework to test results files.
- Allure TestOps agent sends the test results in async mode to Allure TestOps.
CI pipeline preparation
Despite the CI you are using, there are some steps you need to perform for all of them:
- Authenticate and authorize your pipeline to upload the test results
Authenticate your pipeline
This step requires the generation of API key, which is to be provided to an Allure TestOps’ agent.
- Log in to Allure TestOps you are going to use to upload test results.
- Go to user’s profile.
- In the section API tokens click a button saying + Create.
- Name your API token and click Submit.
Allure will randomly generate the token and show you in modal window.
- Copy the token and save it in a safe place as it cannot be retrieved by Allure TestOps means.
- This string is to be provided to CI pipeline later on.
To upload test results to a project the user whose API token is used in a pipeline, needs to be added to this project with Write access rights.
If you are trying to upload test results using API token of a user which is not added to the project you are working with, then no data will be uploaded.
Supported CI servers
The table below shows the information on the Allure TestOps agent type needed for different CI servers.
Basically, the test results upload is possible from any CI which supports external tool usage, i.e. download allurectl using wget/curl during the execution and then using allurectl to trigger the tests and upload the test results.
Here is a list of CI servers already tested and widely used with Allure TestOps.
|Integration||Agent to be used|
|Bamboo||Allure TestOps plug-in for Bamboo|
|Jenkins||Allure TestOps plug-in for Jenkins or allurectl|
|TeamCity||Allure TestOps plug-in for TeamCity|