Integrations for Allure TestOps ver. 3

This section describes the integrations available/configurable in Allure TestOps ver. 3

Ver. 3 is outdated.

Ver. 3 is not supported.

You need to upgrade to ver. 4 to benefit from the most recent features added to Allure TestOps ver. 4.

1. CI servers (you can also call them build servers).
2. Issue/bug trackers.
3. Third party TMS systems.
4. Test frameworks

Our approach in terms of integrations is quite simple - our users won't need to write any code to integrate Allure TestOps with external systems, so we'll provide all the integrations and tools for this task.

What do existing integrations allow you to do

Currently supported integrations allow you to do the following:

1. Import test results from build jobs from CI servers.
   • Create test documentation for automated tests based on the test results.
   • Create custom fields for groupings of test cases and test results based on meta information from your tests.
   • Create test results and their groupings in Allure TestOps.
   • Create reports and diagrams in Allure TestOps based on the test results.
2. Collect Environment information from the build jobs.
   • Use the environment information to start build jobs from Allure TestOps on CI servers.
3. Create Jobs on Allure TestOps side to run jobs on CI side
4. Initiate selected test cases run on a CI server side.
   • Based on test plan created on Allure TestOps side.
   • Based on your selection.
   • Based on the need to rerun broken or failed tests on CI servers.
5. Export to the to issue/bug tracking systems
   • Test cases.
   • Test results.
   • Link test cases to dev issues.
6. Export the test cases to other TMSs.

CI servers integration

Practically, we can integrate Allure TestOps with any CI server.

To learn how to integrate the most popular CI servers and enhance the experience with them using Allure TestOps, proceed to the dedicated pages

Issue/bug trackers integrations

For sure, if you've found a bug or performed some tests relates to a DEV issue, you might want to provide this information to your development team and preferably right to the issue/bug tracker your team is using.

Allure TestOps can handle that as well. Let's see how...

Proceed to the page dedicated to the issue/bug tracking systems integrations.

Test management systems

Why?

Good question! Why do we need to integrate one test management system with another?

There are some companies that highly invested in other test management system that do not allow a good integration with test automation frameworks,, although those system are have very tight integration with internal processes of these companies like audits. So these companies require to have all the test documentation to be present in third party test management system but are willing to manage all the automated tests in Allure TestOps.

So we've built this integration that allows you running tests, collecting the test results, creating the test documentation in Allure TestOps and then you can feed third party TMS with the docs created in Allure TestOps and update 3rd party TMS test documentation when your tests have been updated. So, you will have single source of truth in terms of test documentation in the third party TMS.

Test frameworks

Generally, the integration with test frameworks is covered by Allure Framework - the open source project.

How does this integration work:

1. You have some test framework that executes your tests.
2. To collect the events from your test framework you need to add allure framework adaptor to your project.
3. When the tests are being executed by your test framework the following happens:
   1. Framework runs the tests
   2. Allure framework adaptor intercepts the events related to
      • a start of a test
      • an execution of a step in a test
      • a generation of an attachment
      • a stop event of a test
4. Allure adaptor stores all the events for a test as <uid>-result.json to path/to/allure-results directory
5. As a result of the tests execution you get a folder path/to/allure-results where all the files related to tests execution are stored

• <uid>-container.json for each test
• <uid>-result.json for each test
• <uid>-attachment.[ext] attachment files related to a test

How can you use the content of allure-results after the content is generated?

1. You can build allure report
   • report will show you the information for a separate launch only
2. You can send the test results to Allure TestOps server using either allurectl or CI plug-in or upload these via Allure TestOps UI.
   • Allure TestOps will generate or update the test documentation for your automated tests
   • Allure TestOps will calculate the statistics and analytics for your tests
   • Allure TestOps will store the information about your tests launches
   • Allure TestOps will store the information about environment used to run the tests
     • this information will be used to build analytics
     • this information will be used to execute the tests on CI

So, only the test results generated by allure framework can be processed?

As a matter of fact no!

Allure TestOps can process the following results:

• allure1 - it's an old Allure Framework's test results format
• allure2 - actual Allure Framework's test results format
• cucumberjson test results
• jgiven test results
• junit test results
• trx test results
• xctest till xcode 11, newer versions are working via adaptor
• xunit test results

Still, you need to keep in mind, Allure Framework's test results format will allow you having the most comprehensive information about the test results and more detailed scenario information.

Supported test frameworks

This page has the extensive information on currently supported test frameworks.