There are 2 types of dashboards - standard ones, that by default are present in the Dashboard section and custom dashboards you can create in addition to the standard ones.
Standard dashboard (see the picture below) provides instant insight on status of tests you’re having in your project and resides in the Dashboards section of a project.
You will get following insights with the default dashboard:
- Number of test cases and distribution per state (active, in review, outdated etc.)
- Number of test cases and distribution between the manual tests and automated tests.
- Automation trend with the insight regarding the number of test during last 14 days
- the number tests on a day and their status (failed, passed).
- The duration tests runs for a day.
- Mutes trend, i.e. the number of tests excluded from statistics due to permanent failed state or flaky state.
Aim of custom dashboards is to provide additional customized analytical data for the decision makers.
End user creates custom dashboard in the dashboard section of a project (+New Dashboard button in upper-right corner).
Additional analytical information is provided via widgets.
There is following set of widgets available for adding to a custom dashboard:
- Markdown widget
- Launch analytics widget
- Launch statistic trend widget
- Test cases pie chart widget
- Automation trend widget
- Top test cases widget
- Test case tree map chart widget
- Launches widget
This is simple text widget that takes textual information and shows it on a dashboard.
Launch analytics widget
Launch analytics widget provides analytical information about the tests in Launches for the following metrics:
- Average Test Cases success rate
- Average Test Cases duration
- Summary Test Cases duration
- Test Cases count
- Test Results count
- Retries count
- Test Results count exclude retries
- Summary retries duration
- Summary duration exclude retries
You can adjust the output of Launch analytics widget by filtering either test cases or launches with help of built-in query language - RQL.
RQL example for test cases
The example below will help to calculate the analytical data only for the test cases that have user name
username in any of the configured roles (owner, admin, author etc.)
member is "userName"
RQL example for launches
The example below will help to calculate the analytical data only for the launches that have tag
master (VSC branch).
tag is "master"
Launch statistic trend widget
This widget is similar to the one you can see in the default dashboard but you can define additional filtering by using RQL for Test cases and/or launches. After all the configuration it will show filtered statistics that will show you the number of executed tests in connection with the status (Passed, Failed)
Test cases pie chart widget
This widget is similar to the one you can see in the default dashboard but you can define additional filtering by using RQL for Test cases and/or launches. After all the configuration it will show filtered statistics for the test cases in the project with following alternative groupings:
- status (Active, In Review, outdated etc)
- automation (Manual, Automated)
- last execution status (Passed, failed)
Automation trend widget
This widget will show the split between manual and automated tests that were executed during a day over last 14 days. The results can be filtered with RQL on test cases and/or on launches.
Tops test cases widget
This widget will provide you the information about the worst test cases in terms of
- success rate of a test case
- duration of a test case
Worst test cases will be on top of the list, i.e. if there are 2 test cases, one with success rate of 40% and the other one with the success rate of 98%, then the test case with the success rate of 40% will be on the top of the list. Same for the duration - the one that is being executed longer than the others will be on the top.
Test case tree map chart widget
This widget will show you the distribution of the test cases between existing views (e.g. per suite, feature, story, microservice etc.).
Additionally, to have only specific test cases in the view, you can filter them using RQL on test cases.
This widget will show you the last 5 launches with the links to the launches and tests statuses per launch (Passed/Failed etc.).