What's Allure TestOps?

Allure TestOps is a refined Test Operations Swiss army knife which allows you to manage all test related stuff in one place.

We support the entire life cycle for testing for both - manual and automated tests:

The system can be easily adapted to your processes, and it will help you to deliver great products.

What can we do out-of-the-box

Tests

• Design both manual and automated tests.
• Create test plans to test your product with both manual and automated tests in one test plan.
• Manage the tests runs for all your tests from one place.
• Real time tests documentation creation for automated tests based on your rules.
• Compare historical tests data vs. currently created documentation.
• Import tests from other TMS either manually or in automated way.
• Group and regroup the tests into different tree views based on custom fields.

Test frameworks

• Integrate test frameworks for various programming languages with the open-source testing tool Allure Framework:

  • C# (e.g. Allure SpecFlow, Allure xUnit.net);
  • Java (e.g. Allure Cucumber-JVM, Allure JBehave, Allure JUnit 5);
  • JavaScript, TypeScript (e.g. Allure Cucumber.js, Allure Cypress, Allure Playwright);
  • PHP (e.g. Allure PHPUnit);
  • Python (e.g. Allure Behave, Allure Pytest, Allure Robot Framework);
  • Ruby (e.g. Allure Cucumber.rb, Allure RSpec).

  A list of all supported frameworks is available in the official Allure Report documentation.

• Transfer metadata of automated test cases, which can be used to:

  • Diagnose test crashes step by step (steps and statusDetails).
  • Analyze test execution time and identify slow steps (start and stop).
  • Customize filters for tests (labels and parameters).
  • Automate reporting (attachments and descriptionHtml).
  Metadata

  • name — title of the test or the name of the scenario step.
  • fullName — unique identifier based on the filename and the test title. Different Allure plugins can use different notations to generate this identifier.
  • description — description of the test or step in Markdown format.
  • descriptionHtml — description of the test or step in HTML format.
  • links — array of links added to the test or step.
  • labels — array of various labels added to the test or step.
  • parameters — array of parameters added to the test or step.
  • attachments — array of attachments added to the test or step.
  • status — status with which the test or step finished (passed, failed).
  • statusDetails — object with the detailed information about the test or step.
  • stage — stage in the lifecycle of the test or step.
  • start/stop — time when the execution of the test or step started and stopped, in the UNIX timestamp format.
  • steps — array of test steps.

The list of transmitted metadata depends on the test framework. Detailed information is available here.

Tests runs

• Start, stop, rerun build jobs from Allure TestOps on build servers.
• Set different environment parameters from Allure TestOps to run the tests with.
• Select tests (filter out the tests you don't need to be started) in a build job for execution.
• Select several build jobs (even on different build servers) for test plan execution.
• Watch the build progress and the status in real time.

Test results

• Gather tests results from build server in real time during the build job execution.
• Link failed tests to defects to save time on the analysis.
• Link failures to issues in your issue tracking system.
• Export tests results to issue tracking systems.
• Import tests results manually if needed.
• Import tests results right from your IntelliJ IDEA project.

Analytics

• Analyse the trends using built-in dashboards.
• Analyse the tests results using built-in query language to build your custom dashboards.
• Create the test reports as PDF or CSV.

Integrations

• Ready-to-use integrations with popular build servers.
  • Jenkins, Bamboo, GitLab, GitHub, TeamCity, etc.
• Ready-to-use integrations with popular issue tracking systems.
  • Jira, YouTrack, GitLab, Bitbucket, etc.
• Ready-to-use integrations with popular TMSs for tests results exports.
  • Zephyr, TestRail, Xray, Azure DevOps, etc.

Next steps

Following sections will provide the information about ways to deploy Allure TestOps in your infrastructure, the architecture of Allure TestOps, and main terms and concepts of Allure TestOps.