TestOps logo
Try Testops

Metadata / labels / custom fields

We evangelize the idea that an automated test is the best source for the test's documentation, i.e. an automated test actually contains all the information about itself, and it should be the only the source of this information.

Test documentation is being generated based on the test results of your tests and their metadata which automation QA engineers can add to the tests.

Tests metadata is also used to make self-documented tests.

We suggest to use Allure's labels feature to provide metadata to your tests.

Allure Labels is a set of key-value pairs used to provide metadata to Allure results. Allure TestOps uses this metadata to add auxiliary attributes to test cases and test results and shows these in Allure TestOps' UI. These fields can be used to filter the results, group test cases and many more.

Allure TestOps supports different types of attributes to construct different flows.

Attributes

Supported types of the attributes are shown in the table below.

Name Description Used for
Custom Fields Contains information about
Test Cases' Epic, Feature, Story,
Component, Microservice etc.		 1. Test case filtering
2. Test results filtering
3. Analytics filtering
4. Grouping (build trees)
Test layer Contains information about tests' layers
like, UI Tests, Selenium Test, API Tests, etc)		 1. Test case filtering
2. Test results filtering
3. Analytics filtering
Members Contains information about members
like Author, Owner, Team Lead, etc)		 1. Test case filtering
2. Test results filtering
3. Analytics filtering
4. Notifications (WIP)
Issues Contains information about issues
like bugs, features, epics, etc		 1. Test case filtering
2. Test results filtering
3. Analytics filtering
4. Export data to issue
Tags Simple container for tags 1. Test case filtering
2. Test results filtering
3. Analytics filtering
Environment Contains information about test
execution environment like browser, host, OS, etc.		 1. Test cases history filtering
2. Test result filtering
3. Running tests on different environments

How it works?

Let's have a look to a simple code below (in the example it is Java, but the language doesn't matter as Labels can be implemented in any programming language):

    @Manual
    @Feature("Regions")
    @DisplayName("Goods checking")
    @Story("Main page in a region")
    @Tags({@Tag("web"), @Tag("regress"), @Tag("smoke")})
    @ParameterizedTest(name = "{displayName} in region `{0}`")
    @MethodSource("regions")
    public void addToFavorites(final String city) {
        step("Open site for a region: " + city);
        step("Check that region is: " + city);
        step("Check that main page contains region specific goods", () -> {
            step("Check that ads are specific for the region: " + city);
            step("Check that best goods area is specific for the region: " + city);
        });
    }

Regardless of the used test framework and programming language, Allure Framework will generate the test results files in JSON format and files will look like <uid>-result.json.

{
  "name": "Goods checking",
    <snipped>
  
  "labels": [
    {
      "name": "tag",
      "value": "web"
    },
    {
      "name": "tag",
      "value": "regress"
    },
    {
      "name": "tag",
      "value": "smoke"
    },
    {
      "name": "story",
      "value": "Main page in a region"
    },
    {
      "name": "feature",
      "value": "Regions"
    },
    {
      "name": "ALLURE_MANUAL",
      "value": "true"
    },
    {
      "name": "framework",
      "value": "junit-platform"
    },
    {
      "name": "language",
      "value": "java"
    },
    {
      "name": "package",
      "value": "io.qameta.allure.ParameterizedManualTest"
    },
    {
      "name": "testClass",
      "value": "io.qameta.allure.ParameterizedManualTest"
    },
    {
      "name": "testMethod",
      "value": "addToFavorites"
    },
    {
      "name": "suite",
      "value": "io.qameta.allure.ParameterizedManualTest"
    }
  ],
  "links": []
}

Basically, your task is to relay the meta information from your tests to the section Labels. This is done by the Allure Report adaptor; for each programming language the way to provide these labels will be different but the approach will be the same - you need to provide allure_label["some_key"] = "some_value" to the test results.

  1. Implement allure_label["some_key"] = "some_value" in your code.
  2. In Allure TestOps Administration create global attribute to which you will link your some_key key(s).
  3. In a project create a schema for mapping of labels from test results to your global attribute.
  • Mapping works as One-To-Many, i.e. to one global attribute you will map many project specific ones.

Examples

Custom Filed

Examples: Epic, Feature, Story, Component, Microservice, etc.

Let's add a new custom field to your test. For example, Microservice: Report

Steps

Here, we describe the generalized concept; programming languages' specific examples can be checked here.

  1. Go to Administration settings page.

    ALT TEXT

  2. Navigate to Custom Fields section and create new field Microservice. This field is a global Custom Field which we'll use for the mapping of the information about the microservices.

ALT TEXT

  1. In your code implement mapping allure_label["microservice"] = "Report".

    This label contains three parts:

    1. allure_label is the pointer for allure adapter (it will look different in different programming languages).
    2. microservice is the label's key
    3. Report is the value.

  2. Go to Project Settings page > Custom field section

  3. Create new mapping

    1. Key: microservice (like label key from step 3)
    2. Custom Field: Microservice (from step 2)

    ALT TEXT

  4. Run your test locally

  5. Go to Launches section and click Upload button in the right top corner.

  6. Navigate to the test result and take a look on Fields section

ALT TEXT

TestLayer

Examples: Selenium, React E2E, API, WEB, Web-UI, UI, Integration, etc.

Let's add new test layer to your test. For example Selenium.

Steps

  1. Go to Administration settings page.

  2. Navigate to Test Layers section and create layer named Selenium.

  3. Implement allure_label["layer"] ="selenium" in your test.

    This label contains three parts:

    1. allure_label is pointer for allure adapter.
    2. layer is label's key.
    3. selenium is the value for this label.

  4. Go to Project Settings section Test Layers

  5. Create new mapping:

    1. Key: selenium (value selenium for the key layer from step 3)
    2. Test Layer: Selenium (from step 2)

    ALT TEXT

  6. Run your test locally

  7. Go to Launch page and click Upload button

  8. Open report and take a look on test name

    ALT TEXT

Members

Examples: Author, Owner, Lead, etc...

Let's add a new member to your test. For example, Author: eroshenkoam

Steps

  1. Go to Administration settings page.
  2. Navigate to Roles section and add role Author

Global settings. Add a new role.

  1. Add new tag allure_label["author"] = "eroshenkoam" to your test

    This label contains three parts:

    1. allure_label is the pointer for allure adapter.
    2. author is the label's key.
    3. eroshenkoam is the value.

  2. Go to Project Settings page > Roles section

  3. Create new mapping:

    1. Key: author (like label's key from step 3)
    2. Role: Author (the value of the label in the step 2)

Project settings. Add mapping to process `author` label

  1. Run your test locally.

  2. Go to Launches page and click Upload button.

  3. Navigate to the test result and take a look on Members section

    ALT TEXT

Issues

Examples: AE-1, AE-2, AE-3, etc. Check how are they defined in your Issue tracker.

Let's link and issue to your test. For example, AE-2 for Jira issue tracker

Steps

  1. Implement allure_label["jira"] = "AE-2" in your tests.
  2. Go to Administration.
  3. Navigate to Integrations section of your project and add a new integration with an issue tracker.
  1. Navigate to Issues section of your project and create a new mapping (Issue schema)
    1. jira is the label's key. It has to be provided Key section.
    2. AE-2 is the value. It'll be added to your test result (and finally to a test case) by Allure TestOps
    3. Link the Key to the appropriate issue tracker.

Create issue schema

  1. Run your test.
  2. Go to Launches page and click Upload button
  3. Navigate to the test result and take a look on Issues links section

Issue is mapped to the test result

List of standard labels used in Allure Framework

Label Occurrence Allure
Report		 Allure
TestOps		 Comment
ALLURE_ID first ❌ ✅
AS_ID first ❌ ✅ Deprecated,
use ALLURE_ID
instead
ALLURE_MANUAL first ❌ ✅
package first ✅ ❌
testClass first ✅ ✅
testMethod first ✅ ✅
parentSuite all ✅ ✅
suite all ✅ ✅
subSuite all ✅ ✅
epic all ✅ ✅
feature all ✅ ✅
story all ✅ ✅
framework first ✅ ❌ No default processing for Allure TestOps, but often used in user-defined mappings
language first ✅ ❌ No default processing for Allure TestOps, but often used in user-defined mappings
layer first ✅ ✅
thread first ✅ ✅
host first ✅ ✅
severity first ✅ ❌
tag all ✅ ✅
owner all ❌ ✅
lead all ❌ ✅

Occurrence column states which of the occurrences of labels are considered by Allure framework for the test results generation. There are two options – first or all.

© Qameta Software Inc. All rights reserved.