Integration XRay test management system
This section describes the integrations available/configurable in Allure TestOps before Release 4.0,0
Allure TestOps supports the integration with XRay. Using this integration you can create and update your XRay tests data directly from Allure TestOps on the fly.
General concept
Integration works this way:
- Create a new test management system in Allure TestOps. You’ll need to specify your XRay host’s URL and credentials to use XRay’s
- Every Test Case in Allure TestOps should have XRay id. You can provide XRay id from your tests code, or add it directly to Test Case via Allure TestOps UI.
- Then you need to create Custom Fields mapping between Allure TestOps and XRay.
After the completion of all the steps above, every update of a test case in Allure TestOps will be automatically pushed to XRay.
Integration with TMS
Prerequisites
- You need to have an active user with the following access rights in XRay:
- Tests creation
- Execution creation
- You need to have the administrator’s privileges in Allure TestOps to add credentials and add TMSs.
Adding credentials
- Go to the Administration.
- Jump to Credentials section.
- Press +Create
- Provide a name for new credentials, e.g. XRay credentials.
- Credentials type should be basic.
- Username will be the username from XRay.
- Password will be the password from XRay.
- Hit Submit.
Adding TMS - XRay
- Go to Administration.
- Jump to the Test Management System section.
- Click +Create.
- Provide a name for new TMS, e.g. XRay TMS.
- Provide XRay’s URL.
- Type will be the xray.
- Credentials will be the the ones created on
Adding credentials
step.
- Press Test connection to check you have provided correct data and TMS is accessible from Allure TestOps.
- Hit Submit.
TMS information in your tests
Information about XRay IDs is to be provided from your test using labels .
There are few different ways to add labels: using Allure LabelAnnotation
Meta Annotation or by using Allure Runtime API (by calling Allure.label()
method in test runtime).
As we want to mark tests with XRay id, and there is no need to change such id in test runtime, the correct solution would be to use Allure LabelAnnotation
meta annotation:
- First, we need to create an custom annotation, lets call it
XRayId
:
import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Inherited;
import java.lang.annotation.Repeatable;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import io.qameta.allure.LabelAnnotation;
@Documented
@Inherited
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
@LabelAnnotation(name = "xray_id")
@interface XRayId {
String value();
}
The important part is @LabelAnnotation(name = "xray_id")
. This marker annotation will tell Allure to add xray_id
label with annotation’s value()
to every test, marked with this annotation.
- Then we need to place new
XRayId
annotation on top of each test method:
@XRayId("AS-23")
@Test
void testEveryting() {
...
}
- That’s it, now every test provides a label, and you can follow steps from Allure TestOps configuration section in order to configure integration on Allure TestOps side.
Working with existing TmsLink annotation
Probably you already marked your tests with TmsLink
annotation, to add links to XRay test cases to your Allure Report:
@TmsLink("AS-23")
@Test
void testEveryting() {
...
}
Replacing @TmsLink("AS-23")
with @XRayId("AS-23")
will do the trick, but you’ll lose links in your Allure Report. In case you want to keep links, there is a solution: Allure’s LinkAnnotation
Meta Annotation!
Lets modify the XRayId
class, by adding an link annotation on top of it:
import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Inherited;
import java.lang.annotation.Repeatable;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import io.qameta.allure.LabelAnnotation;
import io.qameta.allure.LinkAnnotation;
import static io.qameta.allure.util.ResultsUtils.TMS_LINK_TYPE;
@Documented
@Inherited
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
@LabelAnnotation(name = "xray_id")
@LinkAnnotation(type = TMS_LINK_TYPE)
@interface XRayId {
String value();
}
What we just did: we marked XRayId
annotation with additional LinkAnnotation
. Now, when processing XRayId
annotation Allure will also add a link to every id with tms
type (exactly what TmsLink
annotation did). Now you can safely replace TmsLink
with XRayId
, and you’ll have both an label and link added to Allure Results.
Custom fields mapping
Allure TestOps will map your test results and test cases based on the labels values it receives from the test results after tests run (on CI, on your local test run, it really does not matter).
In order for Allure TestOps to be able to process this information you need to set up the matching of the labels to test keys in TMS.
This operation is done on a project level.
In your Project
- Go to the project settings.
- Jump to Test Key section.
- Click +Create.
- Enter the key for your label (
xray_id
from our examples above ). - And map this key to the TMS you’ve integrated with Allure TestOps .
After you’ve completed the steps, information for the label xray_id
will be used to send the test cases information to XRay.
<snip>
"labels": [
{
"name": "tag",
"value": "api"
},
{
"name": "xray_id",
"value": "AE-T1"
},
<snip>
],