Running an automated test case

Unlike a manual test which contains human-readable instructions for a tester, an automated test is written as a code. The code not only performs all required actions and validations automatically, but also generates a test result: one or more files describing how long the execution took, what errors were discovered, and so on. Based on this data, Allure TestOps creates or updates the list of test case.

There are multiple ways to upload the test results to Allure TestOps:

• via a CI integration,
• via the allurectl utility,
• via the web interface.

The uploaded data will be reflected in Allure TestOps: first in the current test launch, and eventually (once the launch is closed) in the Test cases.

In this tutorial, we will focus on uploading test results manually, but you can choose other methods if they better suit your project's workflow.

Where do I get test results from?

The exact path to the test results depends on both the test framework in use and its configuration. When using one of the official Allure adapters and the default settings, test results will typically be generated to the allure-results or build/allure-results directory. The files in the directory are written in the Allure Report data format which is compatible with all the features of Allure TestOps. However, Allure TestOps recognizes other popular formats, too.

If you don't have a working project set up right now, use one from allurereport.org/start to generate a test results directory so that you can follow the tutorial.

Since a collection of test results is known as a launch in Allure TestOps, uploading the test results is done in the Launches section.

1. Go to the project page.

2. In the menu on the left, click Launches.

3. In the top right corner, click the Upload new results icon.

   

4. On the left pane of the dialog that appears, fill in the fields:

   • Name — the name that will be displayed in the list of launches.
   • Tags — arbitrary tags for the launch.
   • Issues — links to related issues (if you have configured an issue tracker integration).
   • Links — links to any web pages useful for the team members who will work with the test launch.
   • Environment — values to add to each result, e.g., the name of the browser which was used for testing.

   Except for Name, all the fields are optional.

   Feel free to establish your own conventions that will help you better navigate through the list of your launches. For example, you may want to include the word “release” in the name of each test launch that you perform before a public release, or set different tags for launches done by different team members. You will be able to search launches by these fields in the web interface or via AQL.

   

5. On the right pane of the dialog, choose the files to upload — for example, grab an allure-results directory produced by your project's tests and drag it to the upload area. Alternatively, you can compress it into a ZIP archive and then choose it as a single file.

6. Click Upload files and wait for a few moments while Allure Report processes the data.

   At this point, the uploaded data are isolated within the test launch. For example, if the data includes a test case that was never before added to Allure TestOps, it won't appear in the Test cases interface just yet.

   If you want to add some more test results to the same test launch (i.e., test results from another machine), go to the launch and select Upload files in its menu.

   

7. To close the launch, thus updating the Test cases list, go to the launch and click the Stop button:

   

What's next?

• View the test results in Launch Overview and Dashboards.
• Configure a CI integration to open new test launches and upload test results automatically.
• In the Allure Report documentation, learn about how to make sure tests are identified correctly.