This section covers the information about Launches section.
All the information about the test results is stored in Launches section.
Launches section UI overview
The main screen of Launches section provides the following information about the ongoing and finished launches as well as allows filtering the launches and manually upload the test results.
Allure Testops provides the following information about launches:
either generated by Allure Testops tools in CI side
or generated on Allure Testops side
or entered manually by a person that initiated a Launch
Current status of a Launch
Current status of the job run on CI side
including the hyperlink to build job
Number of tests with the distribution by execution status (Passed, Failed etc.)
browser used to run tests
host's address where tests were executed
Launches information filtering
Search (or filtering) across all the available launches is activated by clicking on the funnel icon in the upper right corner.
Filtering is supported by using all the meta information about the launches, e.g.:
Build job name
Marker of automated build
Link to an issue in issue tracker
Date and time of launch
Test results manual upload
First of all why could we need it?
Say, you are in the process of the automation of a test case and you need to check if your test is generating the results as you expect. You can upload the results, check the representation of the information and delete the launch created by the manual upload.
There was an interruption caused by the network and results weren't uploaded. So, you can upload the tests results manually as and emergency workaround.
In upper right corner click Upload button.
In the modal dialogue that will be opened fill the following data (any and all of this data can be omitted but you won't be able to filter/find these test results):
Name. Name your launch or leave blank and default naming will be applied.
Add launch's tags (smoke, manual, delete etc. - your choice).
Add links to an issue in issue tracker if this is related to some task you are checking.
Add links if any.
Add environment data if this information is important.
Add the test results either using drag and drop or select from the disk.
Result files - upload all files from your test results folder, i.e. select all and drag to the drop area.
Archives - archive your test results and then upload zip archive.
Test results format
Following test results format are supported at the moment:
xctest till xcode 11, newer versions are working via adaptor
Test results file size limitations
If a test results file exceeds 3 megabytes it will be processed by Allure Testops as an attachment.
Test documentation for manually uploaded test results
To create test documentation (test cases) you need to close the launch for manually uploaded test results. If you don't want to create the test cases for manually uploaded files, you need to delete manually created launch.
State of a launch
Each launch can have 2 states either open or closed.
Allure Testops will process test results and generate Test Cases (aka test documentation or live documentation) only when a launch is closed.
Launch can be in an open state in the following cases:
The transfer of test results to Allure Testops is in progress, i.e. build jobs is still running.
The team is still performs the triaging of the test results and resolving the errors in the test results.
The team either forgot to create automates closure rule or forgot to close a launch manually or a launch is being kept open intentionally for some reason.
An open launch state...
prevents creation of new test cases
prevents the update of existing test cases
prevents the calculation of the statistics
prevents the execution of the cleanup for the artefacts, thus can lead to the higher consumption of the artefacts storage and can lead to the growth of some tables in the database.
prevents the deletion of unlinked attachment candidates, thus can lead to the higher consumption of the artefacts storage.
Allure Testops will finally process the test results only for a closed launch, which means the following:
Test documentation (the content of Test cases section) will be either generated for new test cases or updated for the existing ones.
Test results will be used to calculate statistics
Test results will be used in analytics and can be viewed in dashboards.
Old artefacts will be deleted accordingly to the configured cleanup rules.
Unused files uploaded during the launch upload session will be deleted upon a launch's closure.
these are so-called attachment candidates files, which are supposed to be linked to test results.
Closing a launch
There are two ways to close a launch
Closing a launch manually
To close a launch you need to click Close launch button.
Closing a launch automatically
Setting for automatic launches closure are made on a project level.
Go to the project settings (you must be an owner of a project to see this section).