Metadata / labels / custom fields

We evangelize the idea that an automated test is the best source for the test's documentation, i.e. an automated test actually contains all the information about itself, and it should be the only the source of this information.

Test documentation is being generated based on the test results of your tests and their metadata which automation QA engineers can add to the tests.

Tests metadata is also used to make self-documented tests.

We suggest to use Allure's labels feature to provide metadata to your tests.

Allure Labels is a set of key-value pairs used to provide metadata to Allure results. Allure TestOps uses this metadata to add auxiliary attributes to test cases and test results and shows these in Allure TestOps' UI. These fields can be used to filter the results, group test cases and many more.

Allure TestOps supports different types of attributes to construct different flows.

Attributes

Supported types of the attributes are shown in the table below.

Name	Description	Used for
Custom Fields	Contains information about Test Cases' Epic, Feature, Story, Component, Microservice etc.	Test case filtering Test results filtering Analytics filtering Grouping (build trees)
Test layer	Contains information about tests' layers (UI Tests, Selenium Test, API Tests, etc)	Test case filtering Test results filtering Analytics filtering
Members	Contains information about members (Author, Owner, Team Lead, etc)	Test case filtering Test results filtering Analytics filtering Notifications (WIP)
Issues	Contains information about issues (bugs, features, epics, etc)	Test case filtering Test results filtering Analytics filtering Export data to issue
Tags	Simple container for tags	Test case filtering Test results filtering Analytics filtering
Environment	Contains information about test execution environment (browser, host, OS, etc.)	Test cases history filtering Test result filtering Running tests on different environments

How it works?

Let's take a look at a simple code example. The example below uses Java, but Labels can be implemented in a variety of programming languages. You can find examples for other languages in the Allure Report documentation.

@Manual
@Feature("Regions")
@DisplayName("Goods checking")
@Story("Main page in a region")
@Tags({@Tag("web"), @Tag("regress"), @Tag("smoke")})
@ParameterizedTest(name = "{displayName} in region `{0}`")
@MethodSource("regions")
public void addToFavorites(final String city) {
    step("Open site for a region: " + city);
    step("Check that region is: " + city);
    step("Check that main page contains region specific goods", () -> {
        step("Check that ads are specific for the region: " + city);
        step("Check that best goods area is specific for the region: " + city);
    });
}

Regardless of the used test framework and programming language, Allure Framework will generate the test results files in JSON format and files will look like <uid>-result.json.

{
  "name": "Goods checking",
    <snipped>
  
  "labels": [
    {
      "name": "tag",
      "value": "web"
    },
    {
      "name": "tag",
      "value": "regress"
    },
    {
      "name": "tag",
      "value": "smoke"
    },
    {
      "name": "story",
      "value": "Main page in a region"
    },
    {
      "name": "feature",
      "value": "Regions"
    },
    {
      "name": "ALLURE_MANUAL",
      "value": "true"
    },
    {
      "name": "framework",
      "value": "junit-platform"
    },
    {
      "name": "language",
      "value": "java"
    },
    {
      "name": "package",
      "value": "io.qameta.allure.ParameterizedManualTest"
    },
    {
      "name": "testClass",
      "value": "io.qameta.allure.ParameterizedManualTest"
    },
    {
      "name": "testMethod",
      "value": "addToFavorites"
    },
    {
      "name": "suite",
      "value": "io.qameta.allure.ParameterizedManualTest"
    }
  ],
  "links": []
}

Basically, your task is to relay the meta information from your tests to the section Labels. This is done by the Allure Report adaptor; for each programming language the way to provide these labels will be different but the approach will be the same - you need to provide allure_label["some_key"] = "some_value" to the test results.

1. Implement allure_label["some_key"] = "some_value" in your code.

2. In Allure TestOps Administration create global attribute to which you will link your some_key key(s).

3. In a project create a schema for mapping of labels from test results to your global attribute.

   Mapping works as One-To-Many, i.e. to one global attribute you will map many project specific ones.

Examples

Custom Filed

Examples: Epic, Feature, Story, Component, Microservice, etc.

Let's add a new custom field to your test. For example, Microservice: Report

Steps

Here, we describe the generalized concept; programming languages' specific examples can be checked here.

1. Go to Administration → Custom fields.

2. Create the new field Microservice. This field is a global custom field which we'll use for the mapping of the information about the microservices.

3. In your code implement mapping allure_label["microservice"] = "Report".

   This label contains three parts:

   • allure_label is the pointer for allure adapter (it will look different in different programming languages).
   • microservice is the label's key.
   • Report is the value.

4. Go to project Settings → Custom fields.

5. Click Add field and select the field (Microservice).

6. On the right, switch to the Mapping tab and click New mapping.

7. Enter the mapping key (microservice) and click Create.

8. Run your test locally

9. Go to Launches and click Upload new results in the upper-right corner.

10. Navigate to the test result. The custom field value should appear in the Fields section

TestLayer

Examples: Selenium, React E2E, API, WEB, Web-UI, UI, Integration, etc.

Let's add new test layer to your test. For example Selenium.

Steps

1. Go to Administration → Test Layer.

2. Create a layer named Selenium.

3. Implement allure_label["layer"] ="selenium" in your test.

   This label contains three parts:

   • allure_label is pointer for allure adapter.
   • layer is label's key.
   • selenium is the value for this label.

4. Go to project Settings → Test layers.

5. Create new mapping:

   • Key: selenium (value selenium for the key layer from step 3);
   • Test Layer: Selenium (from step 2).

6. Run your test locally

7. Go to Launches and click Upload new results in the upper-right corner.

8. Open the report. The layer name should be displayed next to the test name.

Members

Examples: Author, Owner, Lead, etc.

Let's add a new member to your test. For example, Author: johndoe

Steps

1. Go to Administration → Roles.

2. Add a role Author

3. Add new tag allure_label["author"] = "johndoe" to your test

   This label contains three parts:

   • allure_label is the pointer for allure adapter.
   • author is the label's key.
   • johndoe is the value.

4. Go to project Settings → Roles.

5. Create new mapping:

   • Key: author (like label's key from step 3);
   • Role: Author (the value of the label in the step 2).

6. Run your test locally.

7. Go to Launches and click Upload new results in the upper-right corner.

8. Navigate to the test result. The custom field value should appear in the Fields section

9. Navigate to the test result and take a look on Members section. The author's name should be displayed under the appropriate label.

Issues

Examples: AE-1, AE-2, AE-3, etc. Check how are they defined in your Issue tracker.

Let's link and issue to your test. For example, AE-2 for Jira issue tracker

Steps

1. Implement allure_label["jira"] = "AE-2" in your test.

   This label contains three parts:

   • allure_label is pointer for allure adapter.
   • jira is label's key.
   • AE-2 is the value for this label.

2. Go to project Settings → Integrations.

3. Add a new integration with an issue tracker.

   This action also requires enabling of a certain integration on the global level.

4. Go to project Settings → Issues and create a new mapping with the following fields:

   • Key — the label's key that we used in the code on step 1 (e.g., jira);
   • Issue tracker — the integration that we enabled for the project on step 3.

5. Run your test.

6. Go to Launches and click Upload new results in the upper-right corner.

7. Navigate to the test result and take a look on Issues links section. The “AE-2” link should appear there.

List of standard labels used in Allure Framework

Label	Occurrence*	Allure Report v.2	Allure TestOps	Comment
ALLURE_ID	first	❌	✅
AS_ID	first	❌	✅	Deprecated, use ALLURE_ID instead
ALLURE_MANUAL	first	❌	✅
package	first	✅	❌
testClass	first	✅	✅
testMethod	first	✅	✅
parentSuite	all	✅	✅
suite	all	✅	✅
subSuite	all	✅	✅
epic	all	✅	✅
feature	all	✅	✅
story	all	✅	✅
framework	first	✅	❌	No default processing for Allure TestOps, but often used in user-defined mappings
language	first	✅	❌	No default processing for Allure TestOps, but often used in user-defined mappings
layer	first	✅	✅
thread	first	✅	✅
host	first	✅	✅
severity	first	✅	❌
tag	all	✅	✅
owner	all	❌	✅
lead	all	❌	✅
custom labels	all	❌	✅

Occurrence column states which of the occurrences of labels are considered by Allure Framework for the test results generation.

If the column Occurrence states first, then allure framework takes the first occurrence of such label, and adds this piece of the metadata to the test result (*-result.json) file from your code and ignores the next occurrences of the same label.

If the column Occurrence states all, then allure framework takes all occurrences of such label, and adds this piece of the metadata to the test result (*-result.json) file from your code for all instances of such label.