Launches

When you run a test case (or several at once), either manually or automatically, Allure TestOps creates a launch.

Each launch contains information about the run (date, test results, environment data, number of retries, etc.) and can have either an Open or Closed status:

• An open launch is usually a recently created launch that is still being worked on. For example, a QA engineer has launched a manual test case and is currently in the process of going through the test scenario. Or, a CI job has been run and you need to review the uploaded test results.

• A closed launch is a launch that has been completed. Only after a launch has been closed, Allure TestOps will process it: update the dashboards, create corresponding automated test cases, etc.

You can find all created launches in the Launches section. In addition to the launch name and status, the list displays optional launch attributes (metadata): tags, linked issues, environment variables, etc. If a launch is open, next to its status you can also find the time of recent activity that gets updated when you change the status of one of the tests inside the launch.

Test results, grouped by their status, are displayed on the right side of a launch. You can click on a status to see test results with that status only.

To open the actions menu, click ⋯ next to a launch. Clicking a launch status opens the status change menu, where you can see the date and the name of the user who opened or closed the launch.

Closing and reopening launches

You can close launches automatically or manually. To close launches automatically, you need to configure the Auto close policy for your project.

To close a launch manually:

1. Click ⋯ or the launch status.
2. Select Close launch.

To reopen a closed launch, select Reopen launch in the same menu.

Sorting and display options

To sort the list of launches:

1. In the Launches section, click Display.
2. From the Sort by menu, select the attribute by which you want to sort the launches.
3. From the Direction menu, select the direction of sorting.

Additionally, you can hide launch metadata to make the list more compact:

1. In the Launches section, click Display.
2. Click Hide metadata.

Filtering launches

You can filter launches by their name and by the values of their attributes: statuses, tags, environment variables, etc.

• To filter by name, you only need to enter the search text (a part of the launch name).
• To filter by attributes, you need to specify the attribute and its value.

You can specify a name and several attributes at the same time.

Filtering by name

To filter launches by name:

1. In the Launches section, click the search box above the launch list.
2. Enter a part of the name of the launch and press Enter.
3. To clear the search box, press Backspace or click Clear under the search box.

Filtering by attributes

To filter launches by their attributes:

1. In the Launches section, click the search box above the launch list.

2. Select an attribute to be searched.

3. Select one or more values for the attribute.

   To find launches that don't have a specified value for the attribute (e.g., find launches without tags), switch the No value toggle on.

4. To clear the search box, press Backspace or click Clear under the search box.

Saving a filter

To save a filter for future use:

1. After entering search terms, click Save under the search box.
2. Enter a name for the filter.
3. Choose who will have access to this filter: only you or all members of the project team.
4. Click Save.

The saved filter will appear below the search box. To modify or delete a saved filter, right-click it and select the action you need.

Manually uploading test results

Results of automated tests can be uploaded manually. This can be useful if you want to verify that the result files are generated correctly or if there was an error during the automatic upload.

You can upload either individual result files or a zipped folder. To learn more about the format of a result file, see this article in the Allure Report documentation.

To upload test results:

1. Go to Launches.

2. In the upper-right corner of the screen, click Import.

3. In the left side of the resulting window, enter a name for the new launch (or keep the suggested name), and optionally enter additional attributes: tags, links, and environment data.

   For example, you can add a Release tag to the launch to mark it as a set of tests that were performed before a public release.

4. In the right side of the window, select the upload type (individual JSON files or ZIP files) and select the files for the upload by clicking the upload area or by dragging the files into the area. Various test result formats are supported, see Supported file formats.

5. Click Upload files to start the upload.

If you need to add more test results to the created launch, click ⋯ next to it and select Add test cases → From test result files.

To update test cases and dashboards, close the launch after uploading the files.

Creating launches for automated tests

When you upload automated test results from a CI server or from an Allure TestOps IDE plugin, you don't need to create a launch for them manually. In this case, launch will be created automatically by Allure TestOps. The only thing you need to do is to specify all the launch attributes you need (launch name, tags, etc.) in the tool you are using (integration plugin, allurectl, or an IDE plugin).

Creating launches for tests started from Allure TestOps

If you start a test run from Allure TestOps, manual or automated, Allure TestOps will create a launch that will be used to store all relevant test results.

There are several ways you can initiate a test run from Allure TestOps:

• Using a list of test cases:

  1. In the Test cases section, select the test cases.
  2. Click Run at the bottom of the screen.
  3. Enter the parameters for the launch.

• Using a test plan:

  1. In the Test plans section, click Run test plan next to the test plan you need.
  2. Enter the parameters for the launch.

• Using a job:

  1. In the Jobs section, click Run job next to the job you need.
  2. Enter the parameters for the launch.

Closing launches automatically

Allure TestOps can close launches automatically after a certain amount of time configured in your project settings.

1. Go to Project settings → Launches.
2. Under Auto close policy, click Edit.
3. In the Close finished launches field, you can select which finished launches should be closed automatically using an AQL query. If you leave this field blank, all finished launches will be closed after the specified time interval.
4. In the Close finished launches after field, set the number of hours after which a finished launch should be closed. A finished launch is considered to be one which consists only of passed and/or failed test cases.
5. In the Close in progress launches after field, set the number of hours after which an in-progress launch should be closed. An in-progress launch is one that contains at least one test case with an incomplete status (in progress).
6. Click Submit to save the changes.

Launch status

A launch can have one of two statuses: Open or Closed.

Allure TestOps will create or update existing test cases and refresh the statistics you see in your dashboards only after the launch has been closed.

Open launch

Launch can be open in the following cases:

1. Uploading of test results to Allure TestOps is still in progress (i.e., build jobs are still running).
2. Your team is still performing the triaging of test results (sorting and resolving the errors).
3. Your team forgot to close the launch and there are no configured rules to close it automatically.
4. Launch is being kept open intentionally for some reason.

If a launch is open, Allure TestOps will not process it:

1. New test cases will not be created.
2. Existing test cases will not be updated.
3. Launch data will not be included in statistics.
4. Cleanup rules will not be executed against the launch. This prevents deletion of any artifacts uploaded with the test results and can increase the disk space used by the storage.

Closed launch

After a launch has been closed, Allure TestOps will perform the following:

1. Create new and update existing test cases in the Test cases section.
2. Refresh the statistics you see in the Dashboards section.
3. Delete unused artifacts that were uploaded with the test results.
4. Mark the remaining artifacts for future deletion according to your cleanup rules.

Exporting launch data

You can export the information about the test results in the launch. Two file formats are available: PDF and CSV.

1. Go to Launches.

2. Next to the launch you want to export, click ⋯ and select Export.

3. Choose the file format for export:

   • PDF — a human-readable PDF file that includes both the overall statistics about the launch and the details about individual test results, including the statuses of each step. The PDF file can be sent to a reader who does not have access to the Allure TestOps instance, and it is sufficient to provide them with the most important information about the launch.

   • CSV — a CSV-formatted table with the information about each of the test results, including the main meta information (author, tags, epics), the status, and the error message. CSV format allows to import this file into the analytics tools for further processing.

Changing launch processing rules

When you close a launch, Allure TestOps creates or updates the corresponding test cases. You can exclude certain launches from this process by specifying an AQL filter in the project settings. Excluded launches will still be shown in Launches and Dashboards, but no test cases will be updated when you close them.

To change launch processing rules:

1. Open your project.

2. Go to Settings → Launches.

3. Under Documentation (test cases) generation policy, click Edit.

4. Enter an AQL query to select only the launches that must be processed. If you leave the field empty, all launches will be processed.

   For example, you can enter tag != "debug" to process all launches except the ones with the debug tag.

   Note that this query uses launch attributes and not test case attributes to filter launches. Launch attributes are specified in your CI integration:

   • If you use allurectl, you can specify launch attributes using environment variables or command-line arguments (e.g., allurectl watch -- gradle --launch-tags debug).
   • If you use a CI plugin, check the instructions for your CI service.
   • When uploading test results manually, you can specify launch attributes in the same window where you select the files for the upload.

5. Click Submit.

You can find a complete list of available attributes for the filter in the AQL article.