IDE plug-ins

IDE plug-ins

Overview

IntelliJ plug-in is developed to help you working with the automation of your test cases. It allows you doing the following:

  1. Upload the test results of a local test run to Allure TestOps server.
    • use case: local test run for newly automated tests
    • use case: the automation of a manual test which exists in Allure TestOps
  2. Creation of a new automated test case in Allure TestOps from IDE.
    • use case: Creation of a brand new test case, AllureID will be assigned to the method created in the code
  3. Bulk operation for AllureID assignment
    • use case: to prevent creation of new test cases and abandoning the old ones when code changes you need to assign Allure ID to a test case in your code.
  4. Import of all the meta-data of a manual test case for its automation in IDE.
    • use case: you’ll save a lot of time on the meta-data creation in your code if you import test case data from Allure TestOps to IDE.

Supported IDEs, programming languages and test frameworks

Programing language Test Framework IDE
Java, Kotlin JUnit4 JetBrains IntelliJ IDEA
Java, Kotlin JUnit5 JetBrains IntelliJ IDEA
Java, Kotlin JUnit5 JetBrains IntelliJ IDEA
Java, Kotlin TestNG JetBrains IntelliJ IDEA
Java Cucumber JetBrains IntelliJ IDEA
Python pytest JetBrains PyCharm
Python Robot Framework JetBrains PyCharm
PHP Robot Framework JetBrains PHP Storm
JS, TS Work in progress JetBrains - Work in progress
JS, TS Work in progress VS Code - Work in progress

Installation

  • Go to plug-ins section of your IDE settings.

    • go File > Settings > Plugins
  • In search field start typing Allure TestOps Support

  • Then in the results field find the plug-in and hit Install

    Install Allure TestOps plug-in

  • Restart IDE.

Next step is to authorize the plug-in to perform the operations on Allure TestOps side.

To authorize the plug-in in Allure TestOps you need to generate personal access token in your profile section on Allure TestOps side.

Plug-in connection settings (for release 3.18+)

Plug-in settings reside in the Tools section of Jetbrains’ IDEs.

Proceed to IDE > Preferences > Tools > Allure TestOps

Install Allure TestOps plug-in

Configuring the plug-in

Preconditions

  • Generate personal access token
  • To work with Allure TestOps projects using the IDE plugin for IDE, you need to have at least ROLE_USER authority and at least project:write permission.

Plugin configuration

  1. Open plug-in settings.
  2. Enter the URL of your Allure TestOps instance.
  3. Provide the personal access token.
  4. Click Check connection
  5. If all the data above (items 2 and 3) are correct, system will provide your username as it registered in Allure TestOps.

Install Allure TestOps plug-in

Project settings

Plug-in allows you to select the default Allure TestOps project for the current IDE’s project, i.e. all the operations will be for a certain project in ATO Allure TestOps, and you won’t need to select the project each time you need to upload the results.

Proceed to IDE > Preferences > Tools > Allure TestOps > Project settings and select the project you want to be used by default by the plug-in. Install Allure TestOps plug-in

Upload test results to Allure TestOps from IntelliJ IDEA

Why

Use case for the test results upload is pretty simple: surely, you want to see how your freshly baked automated tests look like in the Allure TestOps launches. So, you’ll be able to check. Plug-in will create a new launch and upload your test results to this new launch on Allure TestOps side.

How

  1. Run tests on your development environment, i.e. on your local machine.
  2. Select folder with allure results in the project tree and then right click the folder.
  3. In the context menu select Allure: Upload results IDEA export results
  4. Update the name for a new launch plug-in is going to create (if needed) IDEA export settings
  5. The results will be available in the Allure TestOps in section Launches. IDEA allure launch

What

Allure TestOps uses certain rules to link your automated test to Allure TestOps test case.

Linking rules depend on a test framework, but generally it works similar to the next example:

AllureID will be linked to the full path of your test method like <package>.<class name>.<method name>.

Allure TestOps also uses the parameters of a test to link a test case to certain AllureID, i.e. test <package>.<class name>.<method name> without parameters and with one parameter aren’t the same, these will be two different test cases with different AllureIDs.

Why

Simply, AllureID is the ID of a test case in the database of Allure TestOps. AllureID is a unique number, meanwhile the full path and parameters of a test could change over time due to refactoring of your code for example.

If you modify any part of this mapping, Allure TestOps will consider such test as a new test case and will create one for you with new AllureID. So, you will have to test cases – one with old path (set of parameters) and the new one with new path and set of parameters.

This will lead to the following:

  • old test case will stop receiving updates from your code and will become abandoned one, statistics will be frozen on the latest result referring to the old test case
  • new test case will receive updates and statistics will start with the first test result.
  • Ultimately, your statistics and analytics will become irrelevant for these two test cases.

How

To avoid abandoning test cases, you must mark your test with AllureID annotation.

  1. If the tests were never reported to Allure TestOps, you need to upload your test results first time your tests into Allure TestOps as described:
    • right click the allure results’ folder
    • select Allure: Upload results
    • select the project to upload to
    • name the launch

IDEA upload results

  1. Close the Launch to process the results, new test cases will be created
    • test cases are created and updated in Allure TestOps only when a launch is closed.

IDEA upload test cases

  1. Right click a package or a class to be processed and select Allure: Assign ID

IDEA assign the ID

  1. In the modal window select the project in Allure TestOps to which you’ve sent the test results
  2. Tests will be annotated with @AllureId.

IDEA allure ID

Now, if you modify either package name or the class name or the test name it will be correctly mapped to Allure TestOps test case.

Manual test cases automation

What

Following workflow allows you easy-peasy automation of a manual test case you have in Allure TestOps.

How

  1. Say, in a project you have a manual test case you want to automate.

IDEA create manual test case

  1. In IDE create new class or use an existing one and add a new method (test) marked with @Test annotation

IDEA create test pattern

  1. Right click the method and select Allure: Import test case

IDEA create test pattern

  1. In the modal dialogue provide Allure ID from Allure TestOps to Test Case ID field and select the items to import from Allure TestOps manual test case.

IDEA allure automate

  1. Your test in IntelliJ IDEA will be updated with the information you selected and will have @AllureId assigned

IDEA test automate result

  1. Update the scenario of your test

IDEA implement test logic

  1. Execute your updated test and upload the results
  2. Close the Launch on Allure TestOps side to process the results
  3. You Allure TestOps manual test case will become an automated one

IDEA upload automated test

  1. Now, to complete the automation process you need to compare manual and automated scenarios by clicking compare button near Scenario item

IDEA compare scenarios

  1. If you’re satisfied with the results, remove the manual scenario, and the scenario from test results will be used

IDEA compare scenarios

  1. Now, your test case’s scenario is being taken from the latest test result

IDEA use auto scenario

  1. Now, your test is fully automated

Creation of automated tests form IDE and and reservation of Allure ID

Why

If you are developing an automated test from scratch, you don’t need importing the data from a manual test case, so you can create an empty automated test case from IDE and assign Allure ID to it form the very beginning, so all the changes in names and parameters will be reflected in the same Test case on Allure TestOps side.

What

Workflow is the following:

  1. You create a test method in your IDE
  2. You request the Allure ID for your test
  3. Allure TestOps reserves the AllureID for your test and makes link between AllureID and full path of your test (<packageName>.<ClassName>.<methodName>).
  4. The test case in Allure TestOps is marked as deleted until you sent test results to Allure TestOps.

How

  1. Create a new test method and right click on it. Select Allure: Create Test Case

Click Allure: Create Test Case

  1. Select project name where you want your TC to be created in.

Select a project

  1. Allure TestOps plug-in will create a test case in deleted state and will provide AllureID to your code.

ID is added to your code

  1. Now, you can open test case in Allure TestOps interface by clicking on the context menu item Allure: Open Test Case

Open created test case in Allure TestOps

  1. Created test case is added in deleted state as the creation only does the reservation of AllureID.

Test is marked as deleted, this is correct

  1. Run your test to create test results and upload the test results the test results to Allure TestOps. Remember you need to close the created launch for magic to happen. Click the close button.

Launch must be closed.

  1. As soon as test results arrived and launch was closed, your test case will appear in the list with the other test cases and Deleted state will be removed.

Launch must be closed.

Change log

3.18.0

  • Fully reworked the settings and authentication
  • Added per IDE project settings – the link between IDE project and a project on Allure TestOps side
  • Added notifications.