Test cases

Test case is the main object in Allure TestOps. A test case defines the testing procedure, either manual or automated, and can include additional information such as tags to make the process of managing a large number of test cases easier.

Each test case in Allure TestOps has a name, which you provide when you create a test case, a unique identifier (AllureID), which is assigned automatically by Allure TestOps, and a workflow status — the current life cycle state of a test case (e.g., draft, active, outdated, etc.). You can learn more about workflows in a dedicated article.

If a test case is manual, it usually includes a test scenario — a set of steps that must be followed to check whether the software is working as intended. It can also include test parameters, which allow you to run the test case with different data. To learn more, see Manual test scenarios and Manual test parameters.

Test cases can additionally include a number of attributes; each of them is described in more detail in a separate article:

• Tags. Tags are custom labels you can apply to your test cases to help organize and filter them.
• Test layers. Test cases can be organized into distinct testing layers, such as API tests, UI tests, unit tests, etc. Unlike tags, a test case can only belong to one testing layer at a time.
• Custom fields. You can create your own set of attributes to use with your test cases.
• Members. Team members can be assigned to test cases with specific roles, helping you track and manage the workload of your team.
• External issue links and external test case links aka Test keys.
  • Test cases can be linked to issues in third-party trackers (like Jira, Redmine, etc.)
  • Test cases in third-party test management tools (like TestRail and Zephyr Scale) when synchronising test cases from Allure TestOps to third-party TMSs.
• Relations. You can link a test case to other test cases and specify the type of relation between them.

All of these attributes can be added using the web interface or, with the exception of relations, by specifying them in the test results you upload to Allure TestOps.

Creating test cases

Allure TestOps allows you to work with both manual and automated test cases. Manual test cases are the test cases that you create using the web interface. Automated test cases are those that were created automatically by uploading the test results generated by your testing framework (using one of the Allure Report integrations).

Creating manual test cases

1. Go to the Test cases section.

2. In the Add a new test case input field, type a name for the new test case and press Enter.

   

3. Select the newly created test case.

4. On the right side of the page, you can add a description and a test scenario for the test case, and change the attributes of the test case.

5. To rename or delete the selected test case, click the ⋮ icon in the top right corner and select the required action.

Creating automated test cases

Automated test cases are created by Allure TestOps when test results are uploaded to Allure TestOps after closing created launch.

Normally, this process is automated by using of allurectl or relevant CI plug-in and integrating your CI pipelines with Allure TestOps, but you can also upload test results manually using the web interface.

In general, the process looks like follows.

1. You integrate your test framework with Allure Framework by using appropriate adaptor.
2. You run your tests as usual (from CI pipeline, upload from IDE, or manual upload via UI). After running the tests, the adapter will generate test results — a set of files in JSON format.
3. These test results are then uploaded to Allure TestOps using allurectl or other means.
4. When Allure TestOps receives test results it creates a new launch, associates the test results from a single run with the created launch, then, after you close the created launch, it creates test cases for uploaded test results (if those don't exist already) or updates an existing ones.

To upload test results manually:

1. Go to the Launches section.
2. In the top right corner of the page, click the Upload new results button.
3. In the resulting window, select the type of upload (individual files or a ZIP archive) and click the upload area or drag and drop files to upload them.
4. Optionally, specify attributes that will be applied to the uploaded results.
5. Click Upload files.

You can achieve the same result by dragging a zip file with the test results to Launches page or dragging full directory content to the Launches page.

We also provide plugins for popular IDEs that make the process of creating and uploading test results easier.

Importing test cases

Test cases can be imported from a CSV file created by Microsoft Excel, Google Sheets, or a third-party test management tool.

To learn more about that, see Importing test cases from CSV files.

Muting test cases

Muting is an Allure TestOps feature that allows you to resolve a failing test result associated with particular test case by providing a reason why it can be ignored.

To mute a test case:

1. Go to Test cases and select a test case.
2. On the top-right side of the page, under the name of the test case, click on the Mutes tab.
3. Click Create mute.
4. Enter a name and a reason for the mute.
5. Optionally, add an issue link to the mute (see Issue links for more info).
6. Click Submit.

Bulk actions

You can mute, delete, and change the attributes of multiple test cases at once using the Bulk actions menu.

1. Go to Test cases.
2. Select test cases by checking the boxes to the left of their names.
3. In the upper right of the test case list, click the Bulk actions button.
4. Choose the bulk action you need from the drop-down menu.

Filtering test cases

Test case attributes can be used to filter the test cases you see in the Test cases section.

To create a filter:

1. Go to Test cases.

2. At the top of the page, click the Open filters button.

3. The Filters panel will appear to the left of the test case list. Click the plus icon in that panel (New filter).

   In addition to the basic test cases attributes (name, ID, status) and additional attributes like tags, you can filter test cases by their mute status and their creation and mute date.

4. If you want to save the filter for future use, click ⋮ and select Save. In the resulting window, enter a name for the filter and check the Shared box if you want to make this filter available to other members of your team.

Converting automated test cases to manual

Automated test cases can be converted to manual test cases. After this conversion, the automation parts of a test case will be detached from it and all future uploaded test results will no longer be linked to that test case. If an automated test case has test steps (the steps field in the JSON test result file), they can be automatically converted to a manual test scenario.

To convert an automated test case to a manual test case:

1. Go to Test cases.

2. Select an automated test case.

3. Click the ⋮ icon in the top right corner and select Detach automation. In the resulting window, you will be able to select a workflow for the test case and choose the source for the test scenario.

4. Under Copy scenario from, select what do you want to use as the source for the test scenario.

   • Test result — convert the test steps from test results to a test scenario.
   • Test case — keep the current stored test scenario.

5. Under Workflow, select a workflow for the test case.

6. Under Status, select a workflow status for the test case.

7. Click Detach automation.