Running build jobs from Allure TestOps

Running build jobs from Allure TestOps

Prerequisites

  1. You’ve successfully performed the integration to import test results.
  2. You’ve successfully performed the integration for run build jobs from Allure.
  3. You’ve successfully performed at least one import of test results.

If the item #3 in the list is done, then you’ve created at least 1 Job in Allure TestOps.

Run the build jobs from Allure TestOps

There are options!

  1. Run tests from Allure TestOps’ Jobs section.
  2. Run selected test (or all) cases from Test cases section.
  3. Run a Test plan from Test plans
  4. Rerun tests or whole launch from Launches
  5. Use alternative environment for tests run.

Run tests from a certain Job

  1. In the Jobs section, select the job you want to run.

  2. Optionally - click Configure and update Environment to desired values.

  3. Click Run job and specify

    • Launch name - name of the current launch or skip.
    • Environments - fill these is you need the environment alternative to the one set in the Job config.
    • Job parameters - this section is deprecated and will disappear soon.
    • Any other parameters if applicable
  4. Select the tests associated with current job you want to run.

  5. Submit. Run a build job

  6. You can check if the job is running:

    • in the Launches tab of Allure TestOps.
    • in CI server’s main UI.

Run tests from Test cases section

  1. Jump to Test cases section of an Allure TestOps project.
  2. Select test cases you want to run.
  3. Click Bulk actions button and select Run.

ALT TEXT

Run modal dialogue will appear.

  1. Check Jobs tab.
    • Jobs tab will show if all the test cases have a Job where these test can be started.
    • If some test have a mark they won’t run, you need to add a Job for these tests. ALT TEXT
  2. Add Launch parameters ALT TEXT
  3. Hit Submit.

Go to the Launches section and enjoy the view:

ALT TEXT

As you can see, launch started three build jobs on different CI servers with different environments.

Run a test plan

Before you run tests from a test plan, you need to create it.
  1. Jump to Test plans section of Allure TestOps project.
  2. Decide which one you want to run
  3. Update test plan parameters if needed.
  4. Hit Run button.

ALT TEXT

Now, add any needed information to the Launch…

ALT TEXT

… and hit Submit.

In Launches section you will see the Launch for the test plan you selected.

ALT TEXT

Rerun of tests

Why do I need it?

Say, you have a launch and test results and some of the tests in this launch are either failed or broken. Before you close the launch and send the test results for creation of live documentation and to analytics you might want to rerun failed tests to check if there is real problem or the problem was just random.

ALT TEXT

What do we do

  1. Go to the Launch with failed test.
  2. Go to the Tree tab of the Launch.
  3. Locate and select failed test.
  4. Go to the Bulk actions.
  5. Select Rerun.

ALT TEXT

If you aren’t going to check the steps of the test manually, just hit Submit, otherwise check Force manual and assign a QA Engineer for manual test rerun.

ALT TEXT

In the Launches you will see that the Launch with failed test has active job(s) running to rerun failed test and completed jobs.

ALT TEXT

Using alternative environment for tests run

Generally, for all CI servers the selsetting of an alternative environment is to be performed the same way with some deviations in case when we select alternative branches of VCS to run tests with.

Common case - alternative host

This will apply to any environment variable you’re using, that could be host, browser, OS and so on.

  1. Jump to Jobs section and update the configuration of your job from the build server.

ALT TEXT

  1. Then click Job’s menu to update the configuration to desired values.

ALT TEXT

  1. … and set desired value for the environment

ALT TEXT

  1. Start the job and check a) in Allure’s Launches and b) in CIs build logs

ALT TEXT

ALT TEXT

Alternative VCS branch

Here, we have some difference between CI servers.

Bamboo Gitlab Jenkins TeamCity
Specific for Bamboo Specific for Gitlab Same as other ENV Specific for TeamCity