Flaky test detection

This feature is available starting from release 26.2.1.2

Allure TestOps automatically detects unstable (flaky) tests without any configuration. A test is marked as flaky when at least 3 status transitions occur within its 10 most recent executions.

A status transition happens when consecutive results differ — PASSED to FAILED or BROKEN, or FAILED or BROKEN back to PASSED.

The table below shows when Allure TestOps starts marking results as unstable. The flaky mark appears from the sixth result onward, once the third transition is confirmed.

Result 1	Result 2	Result 3	Result 4	Result 5	Result 6	Result 7	Result 8	Result 9	Result 10
✅	✅	❌	✅	✅	❌	✅	✅	✅	✅
		1st	2nd		3rd
-	-	-	-	-

Flaky results are highlighted in the test results tree inside a launch.

Flaky marker in the test results tree

Retries and filtering

Retries are included in flakiness detection alongside regular executions.

To view only flaky tests in a launch, apply the Flaky attribute filter in the test results tree.

Flaky filter