Integrations with XRay TMS

Integration XRay test management system

Allure TestOps supports the integration with XRay. Using this integration you can create and update your XRay tests data directly from Allure TestOps on the fly.

General concept

Integration works this way:

  1. Create a new test management system in Allure TestOps. You’ll need to specify your XRay host’s URL and credentials to use XRay’s
  2. Every Test Case in Allure TestOps should have XRay id. You can provide XRay id from your tests code, or add it directly to Test Case via Allure TestOps UI.
  3. Then you need to create Custom Fields mapping between Allure TestOps and XRay.

After the completion of all the steps above, every update of a test case in Allure TestOps will be automatically pushed to XRay.

Integration with TMS

Prerequisites

  1. You need to have an active user with the following access rights in XRay:
    • Tests creation
    • Execution creation
  2. You need to have the administrator’s privileges in Allure TestOps to add credentials and add TMSs.

Adding credentials

  1. Go to the Administration.
  2. Jump to Credentials section.
  3. Press +Create
  4. Provide a name for new credentials, e.g. XRay credentials.
  5. Credentials type should be basic.
  6. Username will be the username from XRay.
  7. Password will be the password from XRay.
  8. Hit Submit.

if you see this write to support.qameta.io

Adding TMS - XRay

  1. Go to Administration.
  2. Jump to the Test Management System section.
  3. Click +Create.
  4. Provide a name for new TMS, e.g. XRay TMS.
  5. Provide XRay’s URL.
  6. Type will be the xray.
  7. Credentials will be the the ones created on Adding credentials step.
    • Press Test connection to check you have provided correct data and TMS is accessible from Allure TestOps.
  8. Hit Submit.

if you see this write to support.qameta.io

Information about XRay IDs is to be provided from your test using labels .

There are few different ways to add labels: using Allure LabelAnnotation Meta Annotation or by using Allure Runtime API (by calling Allure.label() method in test runtime).

As we want to mark tests with XRay id, and there is no need to change such id in test runtime, the correct solution would be to use Allure LabelAnnotation meta annotation:

  1. First, we need to create an custom annotation, lets call it XRayId :
import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Inherited;
import java.lang.annotation.Repeatable;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import io.qameta.allure.LabelAnnotation;

@Documented
@Inherited
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
@LabelAnnotation(name = "xray_id")
@interface XRayId {
    
    String value();

}

The important part is @LabelAnnotation(name = "xray_id"). This marker annotation will tell Allure to add xray_id label with annotation’s value() to every test, marked with this annotation.

  1. Then we need to place new XRayId annotation on top of each test method:
@XRayId("AS-23")
@Test
void testEveryting() {
    ...
}
  1. That’s it, now every test provides a label, and you can follow steps from Allure TestOps configuration section in order to configure integration on Allure TestOps side.

Probably you already marked your tests with TmsLink annotation, to add links to XRay test cases to your Allure Report:

@TmsLink("AS-23")
@Test
void testEveryting() {
   ...
}

Replacing @TmsLink("AS-23") with @XRayId("AS-23") will do the trick, but you’ll lose links in your Allure Report. In case you want to keep links, there is a solution: Allure’s LinkAnnotation Meta Annotation!

Lets modify the XRayId class, by adding an link annotation on top of it:

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Inherited;
import java.lang.annotation.Repeatable;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import io.qameta.allure.LabelAnnotation;
import io.qameta.allure.LinkAnnotation;

import static io.qameta.allure.util.ResultsUtils.TMS_LINK_TYPE;

@Documented
@Inherited
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD, ElementType.TYPE})
@LabelAnnotation(name = "xray_id")
@LinkAnnotation(type = TMS_LINK_TYPE)
@interface XRayId {
    
    String value();

}

What we just did: we marked XRayId annotation with additional LinkAnnotation. Now, when processing XRayId annotation Allure will also add a link to every id with tms type (exactly what TmsLink annotation did). Now you can safely replace TmsLink with XRayId, and you’ll have both an label and link added to Allure Results.

Custom fields mapping

Allure TestOps will map your test results and test cases based on the labels values it receives from the test results after tests run (on CI, on your local test run, it really does not matter).

In order for Allure TestOps to be able to process this information you need to set up the matching of the labels to test keys in TMS.

This operation is done on a project level.

In your Project

  1. Go to the project settings.
  2. Jump to Test Key section.
  3. Click +Create.
  4. Enter the key for your label (xray_id from our examples above ).
  5. And map this key to the TMS you’ve integrated with Allure TestOps .

After you’ve completed the steps, information for the label xray_id will be used to send the test cases information to XRay.

<snip>
"labels": [
        {
            "name": "tag",
            "value": "api"
        },
        {
            "name": "xray_id",
            "value": "AE-T1"
        },

     <snip>
    ],