Triage failed test results

After a run, failed and broken results are unresolved: nobody has said yet why they failed. Triage means going through these results and leaving a decision on each one that the rest of the team can see: the failure was flaky and passed on rerun, it has a known cause tracked in a defect, or the test case is muted for now. Until a result gets such a decision, it stays in the unresolved queue of its launch.

Triage happens in an open launch. Once the launch is closed, you can no longer rerun, mute, or link results from it.

Choose a decision

Find the failures that need a decision

1. Go to Launches and open the launch.
2. On the Overview tab, look at the Unresolved test results area. On the Tree tab, you can filter results by Without resolution to see the same queue in the tree.
3. Open a result and read its error message and stack trace.
4. Check the History tab to see whether the test has been failing in earlier launches, and the Retries tab to see earlier attempts in this launch. See History vs retries.

If your project uses error categories, failures are already grouped by error type, which helps you handle similar failures together.

Rerun to separate flaky failures

Rerun a failure when it may not be reproducible. Select one or more results and rerun them from the result page or with the bulk actions in the bottom panel.

A rerun creates a new job run inside the same launch. The latest attempt becomes the current result, and earlier attempts stay available on the Retries tab. If the rerun passes, the failure no longer needs a decision. If it fails again, link it to a defect or mute the test case.

Reruns need a launch that came from a connected CI job, a test framework integration that supports selective execution, and a launch that is still open. See Rerun prerequisites.

Link the failure to a defect

A defect records a known problem, such as a product bug or a broken test environment, and the results it affects. When you link a result to a defect, the result becomes resolved and appears in that defect.

1. Open the failed result and click Link defect.
2. Select an existing defect, or enter a name for a new one and click Create.
3. If the same failure can happen in other tests or in later launches, add an automation rule with an error message pattern and, if needed, a stack trace pattern. Matching failures in new launches are then linked to the defect automatically. To check another open launch against a new rule, use Apply defect matchers in that launch.

If the fix is tracked in an issue tracker, link the defect to the issue. When the problem is fixed, close the defect, so its automation rules stop resolving new failures.

Mute a test case that should be ignored for now

Mute a test case when its failure is understood, no fix is expected soon, and the failure should not distract the team, for example when the feature under test is being reworked. A mute belongs to the test case, not to a single result.

You can mute a test case from a failed result, with the bulk actions in a launch, or on the Mutes tab of the test case, where you enter a name, a reason, and optionally an issue link. See Muting test cases.

Until you remove the mute, failures of this test case in later launches are resolved, hidden from the launch overview, and not counted in statistics. Remove the mute as soon as the test is expected to pass again, otherwise new failures of this test case go unnoticed.

Next steps

• Defects for automation rules, issue links, and the defect life cycle.
• Test results for how execution status and resolution relate.
• Build quality gates to fail a CI pipeline while a launch still has unresolved results.