Defects

Main concept. What's a defect?

Say, you are running a lot of automated tests.

At some point in time you could have many failed tests that are failing with the same cause: there could be certain bug or whatever event that causes tests to fail. And you are pretty certain what causes these failed tests as you have already analysed these, and some actions are already taken (we hope).

In the same time there are other failed tests which have different root cause and you need quickly distinguish between the tests failed with a known reason - defect from the other failed tests which root cause(s) you still need to understand and investigate by analysing the test results.

So, to allow you to easily detect if a test has failed with a known cause, you can link tests to a defect.

Linking failed tests to a defect

To do all the manipulations described in this section, you need to have a Launch in open state. If a launch is already closed, you need to reopen it by clicking Play button.

You can find the results of completed test runs in Launches area of a project.

1. Jump to a test launch with tests that are failed. In its Tree section click the Filter icon.

2. Check filter Without resolution

3. Inevitably you will get the list of tests which have no resolution and contain a description of the things went wrong

   

4. Now, let's say that the error message we get is well known and caused by the 'Main page header change'. Let's link this test to this known defect which causes the failed test - click Link defect.

5. Name the defect (remember, it's 'Main page header change'?), then click +Create blah-blah-blah....

6. You will be transferred to more detailed defect description form.

7. Now, let's create automation rule that will allow us referencing similar failed tests to this very defect caused by the same bug/issue/event.

   • Click Create automation rule.

8. Then give the name for this automation rule in Rule name field - (1) in the picture bellow.

9. Then enter the pattern for the error message. You can use a regular expression in this field - (2) in the picture below.

   

10. Then to check that your message pattern is actually working, press View matched results button.

    

    Then go back to the form and finalise the linking of the defect by finally clicking Link the defect button.

Defect is now visible in the launch

1. Go to the Overview section of a launch and check Defects area of the dashboard.
2. Go to the Launches area of the project and check the launch where you introduced new link to the defect.

Now, the magic

When new failed tests arrive (we hope that won't happen of course but still), Allure TestOps will automatically link failed tests to a defect if their error messages match the criteria you created to link your test to a defect.

The information about the failed or broken tests that are linked to some defect could be obtained from:

1. Defects section of a project.
2. Launches section of a project.
3. Overview section of a launch.

All these options will be described below.

Defects section of a project

1. Jump to a project
2. Jump to defects section
3. Select registered defect to check in which Launches failed tests with this very defect appear.
4. In the right side of the screen you will see the list of Launches (test runs) that have tests which failed due to certain defect.

Launches section of a project

For all the launches that contain failed tests caused by a defect Allure will now report information that the launch has defects and will show the number of tests that are linked to existing defects.

1. Jump to a project
2. Jump to the launches section
3. For a launch which has newly introduced defect you will see Defects: +1
4. For the launches with known defects you will see the number of known defects present in the launch

Launch overview

1. Jump to a project
2. Jump to the launches section
3. Go to a certain launch you want to analyse.
4. Refer to defects section for known problems in the launch
5. Refer to Unresolved test results section for new issues.

Recap

So, having such a wonderful possibility to assign the tests results to defects in automated way will separate the problems in your tests into two big groups:

1. Known issues - defects, which are already processed, analysed, checked and considered.
2. New issues which still need to be analysed.

This will give your team the clear marks to perform certain actions for new items and control the old ones, which ultimately will save you a considerable amount of time.