Working with labels

Overview

We evangelize the idea that an automated test should be the source for the test’s documentation, i.e. a test must contain all the actual information about itself and should be the only the source of this information. If you don’t think the same way, you still can use Allure TestOps, but you’ll need some manual work to do to organize things we organize automatically.

Test documentation is being generated based on the metadata that an automation QA engineer adds to the tests.

Tests metadata is used to make self-documented tests.

We suggest to use Allure’s labels mechanism to provide metadata to your tests

Allure Labels is a set of key-value pairs that are used to store metadata in Allure results. Allure TestOps uses this metadata to add auxillary attributes to test cases and test results and shows these in Allure TestOps’ UI. These fields can be used to filter the results, group test cases and many more.

Allure TestOps supports different types of attributes to construct different flows.

Attributes

Supported types of the attributes are shown in the table below.

Name Description Functions
Custom Fields Contains information about
Test Cases Structure like Epic, Feature, Story,
Component, Microservice etc.
1. Test case filtering
2. Test results filtering
3. Analytics filtering
4. Tree grouping
Test layer Contains information about tests’ layers
like, UI Tests, Selenium Test, API Tests, etc)
1. Test case filtering
2. Test results filtering
3. Analytics filtering
Members Contains information about members
like Author, Owner, Team Lead, etc)
1. Test case filtering
2. Test results filtering
3. Analytics filtering
4. Notifications (WIP)
Issues Contains information about issues
like bugs, features, epics, etc
1. Test case filtering
2. Test results filtering
3. Analytics filtering
4. Export data to issue
Tags Simple container for tags 1. Test case filtering
2. Test results filtering
3. Analytics filtering
Environment Contains information about test
execution environment like browser, host, OS, etc.
1. Test cases history filtering
2. Test result filtering
3. Running tests on different environments

How it works?

Let’s have a look to a simple code below (in the example it is Java, but the language doesn’t matter as Labels can be implemented in any programming language):

    @Manual
    @Feature("Regions")
    @DisplayName("Goods checking")
    @Story("Main page in a region")
    @Tags({@Tag("web"), @Tag("regress"), @Tag("smoke")})
    @ParameterizedTest(name = "{displayName} in region `{0}`")
    @MethodSource("regions")
    public void addToFavorites(final String city) {
        step("Open site for a region: " + city);
        step("Check that region is: " + city);
        step("Check that main page contains region specific goods", () -> {
            step("Check that ads are specific for the region: " + city);
            step("Check that best goods area is specific for the region: " + city);
        });
    }

Regardless the used test framework and programming language, allure framework will generate the test results files in JSON format and files will look like something-result.json.

{
  "name": "Goods checking in region `Easttown`",
    <snipped>
  
  "labels": [
    {
      "name": "tag",
      "value": "web"
    },
    {
      "name": "tag",
      "value": "regress"
    },
    {
      "name": "tag",
      "value": "smoke"
    },
    {
      "name": "story",
      "value": "Main page in a region"
    },
    {
      "name": "feature",
      "value": "Regions"
    },
    {
      "name": "ALLURE_MANUAL",
      "value": "true"
    },
    {
      "name": "host",
      "value": "WHOLE-BLACK"
    },
    {
      "name": "thread",
      "value": "14396@WHOLE-BLACK.ForkJoinPool-1-worker-7(19)"
    },
    {
      "name": "framework",
      "value": "junit-platform"
    },
    {
      "name": "language",
      "value": "java"
    },
    {
      "name": "package",
      "value": "io.qameta.allure.ParameterizedManualTest"
    },
    {
      "name": "testClass",
      "value": "io.qameta.allure.ParameterizedManualTest"
    },
    {
      "name": "testMethod",
      "value": "addToFavorites"
    },
    {
      "name": "suite",
      "value": "io.qameta.allure.ParameterizedManualTest"
    }
  ],
  "links": []
}

Basically, your task is to relay the meta information from your tests to the section Labels. For each programming language the way to provide these labels will be different but the approach will be the same - you need to provide allure_label[“some_key”] = “some_value” to the test results.

  1. Implement allure_label["some_key"] = "some_value" in your code.
  2. In Allure TestOps Administration create global attribute to which you will link your some_key key(s).
  3. In a project create a schema for mapping of labels from test results to your global attribute.
    • Mapping works as One-To-Many, i.e. to one global attribute you will map many project specific ones.

Examples

Custom Filed

Examples: Epic, Feature, Story, Component, Microservice, etc.

Let’s add a new custom field to your test. For example, Microservice: Report

Steps

Here, we describe the generalized concept; programming languages’ specific examples will be shown later.
  1. Go to Administration settings page.

    ALT TEXT

  2. Navigate to Custom Fields section and create new field Microservice. This field is a global Custom Field which we’ll use for the mapping of the information about the microservices.

ALT TEXT

  1. In your code implement mapping allure_label["microservice"] = "Report".

    This label contains three parts:

    1. allure_label is the pointer for allure adapter (it will look different in different programming languages).
    2. microservice is the label’s key
    3. Report is the value.
  2. Go to Project Settings page > Custom field section

  3. Create new mapping

    1. Key: microservice (like label key from step 3)
    2. Custom Field: Microservice (from step 2)

    ALT TEXT

  4. Run your test locally

  5. Go to Launches section and click Upload button in the right top corner.

  6. Navigate to the test result and take a look on Fields section

ALT TEXT

TestLayer

Examples: Selenium, React E2E, API, WEB, Web-UI, UI, Integration, etc.

Let’s add new test layer to your test. For example Selenium.

Steps

  1. Go to Administration settings page.

  2. Navigate to Test Layers section and create layer named Selenium.

  3. Implement allure_label["layer"] ="selenium" in your test.

    This label contains three parts:

    1. allure_label is pointer for allure adapter.
    2. layer is label’s key.
    3. selenium is the value for this label.
  4. Go to Project Settings section Test Layers

  5. Create new mapping:

    1. Key: selenium (value selenium for the key layer from step 3)
    2. Test Layer: Selenium (from step 2)

    ALT TEXT

  6. Run your test locally

  7. Go to Launch page and click Upload button

  8. Open report and take a look on test name

    ALT TEXT

Members

Examples: Author, Owner, Lead, etc…

Let’s add a new member to your test. For example, Author: eroshenkoam

Steps

  1. Go to Administration settings page.
  2. Navigate to Roles section and add role Author

ALT TEXT

  1. Add new tag allure_label["author"] = "eroshenkoam" to your test

    This label contains three parts:

    1. allure_label is the pointer for allure adapter.
    2. author is the label’s key.
    3. eroshenkoam is the value.
  2. Go to Project Settings page > Roles section

  3. Create new mapping:

    1. Key: author (like label’s key from step 3)
    2. Role: Author (the value of the label in the step 2)

ALT TEXT

  1. Run your test locally.

  2. Go to Launches page and click Upload button.

  3. Navigate to the test result and take a look on Members section

    ALT TEXT

Issues

Examples: AE-1, AE-2, AE-3, etc. Check how are they defined in your Issue tracker.

Let’s link and issue to your test. For example, AE-2 for Jira issue tracker

Steps

  1. Go to Administration.

  2. Navigate to Credentials section and create a new one .

  3. Navigate to Issue Tracker section and create Jira Integration

  4. Implement allure_label["jira"] = "AE-2" in your test

    This label contains three parts:

    1. allure_label is the pointer for allure adapter.
    2. jira is the label’s key.
    3. AE-2 is the value.
  5. Go to Project Settings page > Issues section

  6. Create new mapping

    1. Key: jira (like label key from step #4-2)
    2. Issue Tracker: Jira Server (or as name is defined it the step #3)

    ALT TEXT

  7. Run your test locally.

  8. Go to Launches page and click Upload button

  9. Navigate to the test result and take a look on Issues links section

ALT TEXT

Language specific examples

Java

Python

Kotlin

PHP

JavaScript - Mocha