Test cases

Test case is the main object in Allure TestOps. A test case defines the testing procedure, either manual or automated, and can include additional information such as tags to make the process of managing a large number of test cases easier.

Each test case in Allure TestOps has a name, which you provide when you create a test case, a unique identifier (AllureID), which is assigned automatically by Allure TestOps, and a workflow status — the current life cycle state of a test case (e.g., draft, active, outdated, etc.). You can learn more about workflows in a dedicated article.

Basic properties of a test case are shown in a line.

If a test case is manual, it usually includes a test scenario — a set of steps that must be followed to check whether the software is working as intended. It can also include test parameters, which allow you to run the test case with different data. To learn more, see Manual test scenarios and Manual test parameters.

Test cases can additionally include a number of attributes; each of them is described in more detail in a separate article:

Tags. Tags are custom labels you can apply to your test cases to help organize and filter them.
Test layers. Test cases can be organized into distinct testing layers, such as API tests, UI tests, unit tests, etc. Unlike tags, a test case can only belong to one testing layer at a time.
Custom fields. You can create your own set of attributes to use with your test cases.
Members. Team members can be assigned to test cases with specific roles, helping you track and manage the workload of your team.
External issue links and external test case links aka Test keys.
- Test cases can be linked to issues in third-party trackers (like Jira, Redmine, etc.)
- Test cases in third-party test management tools (like TestRail and Zephyr Scale) when synchronising test cases from Allure TestOps to third-party TMSs.
Relations. You can link a test case to other test cases and specify the type of relation between them.

All of these attributes can be added using the web interface or, with the exception of relations, by specifying them in the test results you upload to Allure TestOps.

Creating test cases

Allure TestOps allows you to work with both manual and automated test cases. Manual test cases are the test cases that you create using the web interface. Automated test cases are those that were created automatically by uploading the test results generated by your testing framework (using one of the Allure Report integrations).

Creating manual test cases

Go to the Test cases section.
At the bottom of the screen, click + Test Case, type a name for the new test case and press Enter.

If you need to create several test cases, press Shift + Enter instead of Enter. You can change this behavior using the Display → Creation settings menu.
Select the newly created test case.
On the right side of the page, you can add a description and a test scenario for the test case, and change the attributes of the test case.
To rename or delete the selected test case, click the ⋯ icon in the top right corner and select the required action.

Alternatively, you can hover over the test case, click ⋯ next to its name, and select the required action.

Creating automated test cases

Automated test cases are created by Allure TestOps when test results are uploaded to Allure TestOps after closing created launch.

Normally, this process is automated by using of allurectl or relevant CI plug-in and integrating your CI pipelines with Allure TestOps, but you can also upload test results manually using the web interface.

In general, the process looks like follows.

You integrate your test framework with Allure Framework by using appropriate adaptor.
You run your tests as usual (from CI pipeline, upload from IDE, or manual upload via UI). After running the tests, the adapter will generate test results — a set of files in JSON format.
These test results are then uploaded to Allure TestOps using allurectl or other means.
When Allure TestOps receives test results it creates a new launch, associates the test results from a single run with the created launch, then, after you close the created launch, it creates test cases for uploaded test results (if those don't exist already) or updates an existing ones.

To upload test results manually:

Go to the Launches section.
In the top right corner of the page, click the Upload new results button.
In the resulting window, select the type of upload (individual files or a ZIP archive) and click the upload area or drag and drop files to upload them.
Optionally, specify attributes that will be applied to the uploaded results.
Click Upload files.

You can achieve the same result by dragging a zip file with the test results to Launches page or dragging full directory content to the Launches page.

We also provide plugins for popular IDEs that make the process of creating and uploading test results easier.

Importing test cases

Test cases can be imported from a CSV file created by Microsoft Excel, Google Sheets, or a third-party test management tool.

To learn more about that, see Importing test cases from CSV files.

Sorting and display options

To sort the list of test cases:

In the Test cases section, click Display.
From the Sort by menu, select the field by which you want to sort the test cases.
From the Direction menu, select the direction of sorting.

Additionally, you can customize how a test case looks in the list of test cases:

In the Test cases section, click Display.
Under List options, select elements you want to display.

Filtering test cases

You can filter test cases by the values of their main and additional attributes:

To filter by main attributes, you only need to enter the search text.
To filter by additional attributes, you need to specify the attribute and its value.

One filter can contain several search terms for additional attributes and one search term for main attributes.

Main attributes

The main attributes of a test case are:

Name,
ID,
Description,
Scenario.

To filter test cases by their main attributes:

In the Test cases section, click on the search box above the test case list.
Type the text to search and press Enter. This text will be used to search through all the main test case attributes at the same time.
To clear the search box, press Backspace or click Clear under the search box.

Additional attributes

To filter test cases by their additional attributes:

In the Test cases section, click on the search box above the test case list.
Select an attribute to be searched.
Select or enter the required value for the attribute.
To clear the search box, press Backspace or click Clear under the search box.

Saving a filter

To save a filter for future use:

After entering search terms, click Save under the search box.
Enter a name for the filter.
Choose who will have access to this filter: only you or all members of the project team.
Click Save.

The saved filter will appear below the search box. To modify or delete a saved filter, right-click on it and select the action you need.

Bulk actions

You can delete, clone, and change attributes of several test cases at once using the bulk actions menu.

Go to the Test cases section.
Select one or more test cases using Ctrl or by checking the boxes next to them.
Click ⋯ in the bottom panel and select the action you need.

To clear the selection, click the cross icon in the bottom panel.

Deleting and restoring test cases

Deleted test cases are moved to the special "folder" (the Trash tree) from which they can be restored.

To open the Trash tree:

Go to the Test cases section.
In the top left corner, click All test cases (or the name of the tree if it has been changed).
Select Trash.

You will see a list of deleted test cases. For each test case in the list, you will have the option to restore or delete it permanently.

To restore or delete a single test case:

Select the test case.
In the top right part of the screen, click (Restore) to restore the test case, or ⋯ → Delete permanently to delete it.

Alternatively, you can hover over the test case, click ⋯ next to its name and select the required action.

To restore or delete several test cases:

Select the test cases using Ctrl or by checking the boxes next to them.
In the panel at the bottom, click Restore to restore the test cases, or ⋯ → Delete permanently to delete them.

Test cases are restored to their previous positions in the tree with all custom field values intact.

Muting test cases

Muting is an Allure TestOps feature that allows you to resolve a failing test result associated with particular test case by providing a reason why it can be ignored.

To mute a test case:

Go to Test cases and open a test case.
In the top right part of the screen, under the name of the test case, click on the Mutes tab.
Click Create mute.
Enter a name and a reason for the mute.
Optionally, add an issue link to the mute (see Issue links for more info).
Click Submit.

To mute several test cases at once, use the bulk actions menu.

Converting automated test cases to manual

Automated test cases can be converted to manual test cases. After this conversion, the automation parts of a test case will be detached from it and all future uploaded test results will no longer be linked to that test case. If an automated test case has test steps (the steps field in the JSON test result file), they can be automatically converted to a manual test scenario.

To convert an automated test case to a manual test case:

Go to Test cases.
Select an automated test case.
Click the ⋯ icon in the top right corner and select Convert into manual. In the resulting window, you will be able to select a workflow for the test case and choose the source for the test scenario.
Under Copy scenario from, select what do you want to use as the source for the test scenario.
- Test result — convert the test steps from test results to a test scenario.
- Test case — keep the current stored test scenario.
Under Workflow, select a workflow for the test case.
Under Status, select a workflow status for the test case.
Click Detach automation.