Metadata / labels / custom fields
We evangelize the idea that an automated test is the best source for the test's documentation, i.e. an automated test actually contains all the information about itself, and it should be the only the source of this information.
Test documentation is being generated based on the test results of your tests and their metadata which automation QA engineers can add to the tests.
Tests metadata is also used to make self-documented tests.
We suggest to use Allure's labels feature to provide metadata to your tests.
Allure Labels is a set of key-value pairs used to provide metadata to Allure results. Allure TestOps uses this metadata to add auxiliary attributes to test cases and test results and shows these in Allure TestOps' UI. These fields can be used to filter the results, group test cases and many more.
Allure TestOps supports different types of attributes to construct different flows.
Attributes
Supported types of the attributes are shown in the table below.
Name | Description | Used for |
---|---|---|
Custom Fields | Contains information about Test Cases' Epic, Feature, Story, Component, Microservice etc. |
1. Test case filtering 2. Test results filtering 3. Analytics filtering 4. Grouping (build trees) |
Test layer | Contains information about tests' layers like, UI Tests, Selenium Test, API Tests, etc) |
1. Test case filtering 2. Test results filtering 3. Analytics filtering |
Members | Contains information about members like Author, Owner, Team Lead, etc) |
1. Test case filtering 2. Test results filtering 3. Analytics filtering 4. Notifications (WIP) |
Issues | Contains information about issues like bugs, features, epics, etc |
1. Test case filtering 2. Test results filtering 3. Analytics filtering 4. Export data to issue |
Tags | Simple container for tags | 1. Test case filtering 2. Test results filtering 3. Analytics filtering |
Environment | Contains information about test execution environment like browser, host, OS, etc. |
1. Test cases history filtering 2. Test result filtering 3. Running tests on different environments |
How it works?
Let's have a look to a simple code below (in the example it is Java, but the language doesn't matter as Labels can be implemented in any programming language):
@Manual
@Feature("Regions")
@DisplayName("Goods checking")
@Story("Main page in a region")
@Tags({@Tag("web"), @Tag("regress"), @Tag("smoke")})
@ParameterizedTest(name = "{displayName} in region `{0}`")
@MethodSource("regions")
public void addToFavorites(final String city) {
step("Open site for a region: " + city);
step("Check that region is: " + city);
step("Check that main page contains region specific goods", () -> {
step("Check that ads are specific for the region: " + city);
step("Check that best goods area is specific for the region: " + city);
});
}
Regardless of the used test framework and programming language, Allure Framework will generate the test results files in JSON format and files will look like <uid>-result.json
.
{
"name": "Goods checking",
<snipped>
"labels": [
{
"name": "tag",
"value": "web"
},
{
"name": "tag",
"value": "regress"
},
{
"name": "tag",
"value": "smoke"
},
{
"name": "story",
"value": "Main page in a region"
},
{
"name": "feature",
"value": "Regions"
},
{
"name": "ALLURE_MANUAL",
"value": "true"
},
{
"name": "framework",
"value": "junit-platform"
},
{
"name": "language",
"value": "java"
},
{
"name": "package",
"value": "io.qameta.allure.ParameterizedManualTest"
},
{
"name": "testClass",
"value": "io.qameta.allure.ParameterizedManualTest"
},
{
"name": "testMethod",
"value": "addToFavorites"
},
{
"name": "suite",
"value": "io.qameta.allure.ParameterizedManualTest"
}
],
"links": []
}
Basically, your task is to relay the meta information from your tests to the section Labels. This is done by the Allure Report adaptor; for each programming language the way to provide these labels will be different but the approach will be the same - you need to provide allure_label["some_key"] = "some_value" to the test results.
- Implement
allure_label["some_key"] = "some_value"
in your code. - In Allure TestOps Administration create global attribute to which you will link your some_key key(s).
- In a project create a schema for mapping of labels from test results to your global attribute.
- Mapping works as One-To-Many, i.e. to one global attribute you will map many project specific ones.
Examples
Custom Filed
Examples: Epic, Feature, Story, Component, Microservice, etc.
Let's add a new custom field to your test. For example, Microservice: Report
Steps
Here, we describe the generalized concept; programming languages' specific examples can be checked here.
Go to Administration settings page.
Navigate to Custom Fields section and create new field Microservice. This field is a global Custom Field which we'll use for the mapping of the information about the microservices.
In your code implement mapping
allure_label["microservice"] = "Report"
.This label contains three parts:
allure_label
is the pointer for allure adapter (it will look different in different programming languages).microservice
is the label's keyReport
is the value.
Go to
Project Settings
page >Custom field
sectionCreate new mapping
- Key:
microservice
(like label key from step 3) - Custom Field:
Microservice
(from step 2)
- Key:
Run your test locally
Go to Launches section and click Upload button in the right top corner.
Navigate to the test result and take a look on
Fields
section
TestLayer
Examples: Selenium, React E2E, API, WEB, Web-UI, UI, Integration, etc.
Let's add new test layer to your test. For example Selenium.
Steps
Go to Administration settings page.
Navigate to Test Layers section and create layer named Selenium.
Implement
allure_label["layer"] ="selenium"
in your test.This label contains three parts:
allure_label
is pointer for allure adapter.layer
is label's key.- selenium is the value for this label.
Go to Project Settings section Test Layers
Create new mapping:
- Key: selenium (value selenium for the key layer from step 3)
- Test Layer: Selenium (from step 2)
Run your test locally
Go to
Launch
page and click Upload buttonOpen report and take a look on test name
Members
Examples: Author, Owner, Lead, etc...
Let's add a new member to your test. For example, Author: eroshenkoam
Steps
- Go to
Administration
settings page. - Navigate to
Roles
section and add role Author
Add new tag
allure_label["author"] = "eroshenkoam"
to your testThis label contains three parts:
allure_label
is the pointer for allure adapter.author
is the label's key.eroshenkoam
is the value.
Go to
Project Settings
page >Roles
sectionCreate new mapping:
- Key:
author
(like label's key from step 3) - Role:
Author
(the value of the label in the step 2)
- Key:
Run your test locally.
Go to Launches page and click Upload button.
Navigate to the test result and take a look on
Members
section
Issues
Examples: AE-1, AE-2, AE-3, etc. Check how are they defined in your Issue tracker.
Let's link and issue to your test. For example, AE-2 for Jira issue tracker
Steps
- Implement
allure_label["jira"] = "AE-2"
in your tests. - Go to Administration.
- Navigate to Integrations section of your project and add a new integration with an issue tracker.
- this action also requires enabling of a certain integration on the global level.
- Navigate to
Issues
section of your project and create a new mapping (Issue schema)jira
is the label's key. It has to be provided Key section.AE-2
is the value. It'll be added to your test result (and finally to a test case) by Allure TestOps- Link the Key to the appropriate issue tracker.
- Run your test.
- Go to Launches page and click Upload button
- Navigate to the test result and take a look on Issues links section
List of standard labels used in Allure Framework
Label | Occurrence | Allure Report |
Allure TestOps |
Comment |
---|---|---|---|---|
ALLURE_ID | first | ❌ | ✅ | |
AS_ID | first | ❌ | ✅ | Deprecated, use ALLURE_ID instead |
ALLURE_MANUAL | first | ❌ | ✅ | |
package | first | ✅ | ❌ | |
testClass | first | ✅ | ✅ | |
testMethod | first | ✅ | ✅ | |
parentSuite | all | ✅ | ✅ | |
suite | all | ✅ | ✅ | |
subSuite | all | ✅ | ✅ | |
epic | all | ✅ | ✅ | |
feature | all | ✅ | ✅ | |
story | all | ✅ | ✅ | |
framework | first | ✅ | ❌ | No default processing for Allure TestOps, but often used in user-defined mappings |
language | first | ✅ | ❌ | No default processing for Allure TestOps, but often used in user-defined mappings |
layer | first | ✅ | ✅ | |
thread | first | ✅ | ✅ | |
host | first | ✅ | ✅ | |
severity | first | ✅ | ❌ | |
tag | all | ✅ | ✅ | |
owner | all | ❌ | ✅ | |
lead | all | ❌ | ✅ |
Occurrence column states which of the occurrences of labels are considered by Allure framework for the test results generation. There are two options – first or all.