Running build jobs from Allure TestOps

Prerequisites

1. You've successfully performed the integration to import test results.
2. You've successfully performed the integration for run build jobs from Allure.
3. You've successfully performed at least one import of test results.

If the item #3 in the list is done, then you've created at least 1 Job in Allure TestOps.

Run the build jobs from Allure TestOps

There are options!

1. Run tests from Allure TestOps' Jobs section.
2. Run selected test (or all) cases from Test cases section.
3. Run a Test plan from Test plans
4. Rerun tests or whole launch from Launches
5. Use alternative environment for tests run.

Run tests from a certain Job

1. In the Jobs section, select the job you want to run.

2. Optionally - click Configure and update Environment to desired values.

3. Click Run job and specify

   • Launch name - name of the current launch or skip.
   • Environments - fill these is you need the environment alternative to the one set in the Job config.
   • Job parameters - this section is deprecated and will disappear soon.
   • Any other parameters if applicable

4. Select the tests associated with current job you want to run.

5. Submit.

6. You can check if the job is running:

   • in the Launches tab of Allure TestOps.
   • in CI server's main UI.

Run tests from Test cases section

1. Jump to Test cases section of an Allure TestOps project.
2. Select test cases you want to run.
3. Click Bulk actions button and select Run.

Run modal dialogue will appear.

1. Check Jobs tab.
   • Jobs tab will show if all the test cases have a Job where these test can be started.
   • If some test have a mark they won't run, you need to add a Job for these tests.
2. Add Launch parameters
3. Hit Submit.

Go to the Launches section and enjoy the view:

As you can see, launch started three build jobs on different CI servers with different environments.

Run a test plan

Before you run tests from a test plan, you need to create it.

1. Jump to Test plans section of Allure TestOps project.
2. Decide which one you want to run
3. Update test plan parameters if needed.
4. Hit Run button.

Now, add any needed information to the Launch...

... and hit Submit.

In Launches section you will see the Launch for the test plan you selected.

Rerun of tests

Why do I need it?

Say, you have a launch and test results and some of the tests in this launch are either failed or broken. Before you close the launch and send the test results for creation of live documentation and to analytics you might want to rerun failed tests to check if there is real problem or the problem was just random.

What do we do

1. Go to the Launch with failed test.
2. Go to the Tree tab of the Launch.
3. Locate and select failed test.
4. Go to the Bulk actions.
5. Select Rerun.

If you aren't going to check the steps of the test manually, just hit Submit, otherwise check Force manual and assign a QA Engineer for manual test rerun.

In the Launches you will see that the Launch with failed test has active job(s) running to rerun failed test and completed jobs.

Using alternative environment for tests run

Generally, for all CI servers the selsetting of an alternative environment is to be performed the same way with some deviations in case when we select alternative branches of VCS to run tests with.

Common case - alternative host

This will apply to any environment variable you're using, that could be host, browser, OS and so on.

1. Jump to Jobs section and update the configuration of your job from the build server.

2. Then click Job's menu to update the configuration to desired values.

3. ... and set desired value for the environment

4. Start the job and check a) in Allure's Launches and b) in CIs build logs

Alternative VCS branch

Here, we have some difference between CI servers.

Bamboo	Gitlab	Jenkins	TeamCity
Specific for Bamboo	Specific for Gitlab	Same as other ENV	Specific for TeamCity