IntelliJ plug-in is developed to help you working with the automation of your test cases. It allows you doing the following:
- Upload the test results of a local test run to Allure TestOps server.
- use case: local test run for newly automated tests
- use case: the automation of a manual test which exists in Allure TestOps
- Creation of a new automated test case in Allure TestOps from IDE.
- use case: Creation of a brand new test case, AllureID will be assigned to the method created in the code
- Bulk operation for AllureID assignment
- use case: to prevent creation of new test cases and abandoning the old ones when code changes you need to assign Allure ID to a test case in your code.
- Import of all the meta-data of a manual test case for its automation in IDE.
- use case: you'll save a lot of time on the meta-data creation in your code if you import test case data from Allure TestOps to IDE.
Supported IDEs, programming languages and test frameworks
|Programing language||Test Framework||IDE|
|Java, Kotlin||JUnit4||JetBrains IntelliJ IDEA|
|Java, Kotlin||JUnit5||JetBrains IntelliJ IDEA|
|Java, Kotlin||JUnit5||JetBrains IntelliJ IDEA|
|Java, Kotlin||TestNG||JetBrains IntelliJ IDEA|
|Java||Cucumber||JetBrains IntelliJ IDEA|
|Python||Robot Framework||JetBrains PyCharm|
|PHP||Robot Framework||JetBrains PHP Storm|
|JS, TS||Work in progress||JetBrains - Work in progress|
|JS, TS||Work in progress||VS Code - Work in progress|
Go to plug-ins section of your IDE settings.
- go File > Settings > Plugins
In search field start typing Allure TestOps Support
Then in the results field find the plug-in and hit
Next step is to authorize the plug-in to perform the operations on Allure TestOps side.
To authorize the plug-in in Allure TestOps you need to generate personal access token in your profile section on Allure TestOps side.
Plug-in connection settings (for release 3.18+)
Plug-in settings reside in the Tools section of Jetbrains' IDEs.
Proceed to IDE > Preferences > Tools > Allure TestOps
Configuring the plug-in
- Generate personal access token
- To work with Allure TestOps projects using the IDE plugin for IDE, you need to have at least ROLE_USER authority and at least
- Open plug-in settings.
- Enter the URL of your Allure TestOps instance.
- Provide the personal access token.
- Click Check connection
- If all the data above (items 2 and 3) are correct, system will provide your username as it registered in Allure TestOps.
Plug-in allows you to select the default Allure TestOps project for the current IDE's project, i.e. all the operations will be for a certain project in ATO Allure TestOps, and you won't need to select the project each time you need to upload the results.
Proceed to IDE > Preferences > Tools > Allure TestOps > Project settings and select the project you want to be used by default by the plug-in.
Upload test results to Allure TestOps from IntelliJ IDEA
Use case for the test results upload is pretty simple: surely, you want to see how your freshly baked automated tests look like in the Allure TestOps launches. So, you'll be able to check. Plug-in will create a new launch and upload your test results to this new launch on Allure TestOps side.
- Run tests on your development environment, i.e. on your local machine.
- Select folder with allure results in the project tree and then right click the folder.
- In the context menu select Allure: Upload results
- Update the name for a new launch plug-in is going to create (if needed)
- The results will be available in the Allure TestOps in section Launches.
Link tests in IDE to Tests cases in Allure TestOps via AllureID
Allure TestOps uses certain rules to link your automated test to Allure TestOps test case.
Linking rules depend on a test framework, but generally it works similar to the next example:
AllureID will be linked to the full path of your test method like
<package>.<class name>.<method name>.
Allure TestOps also uses the parameters of a test to link a test case to certain AllureID, i.e. test
<package>.<class name>.<method name> without parameters and with one parameter aren't the same, these will be two different test cases with different AllureIDs.
Simply, AllureID is the ID of a test case in the database of Allure TestOps. AllureID is a unique number, meanwhile the full path and parameters of a test could change over time due to refactoring of your code for example.
If you modify any part of this mapping, Allure TestOps will consider such test as a new test case and will create one for you with new AllureID. So, you will have to test cases – one with old path (set of parameters) and the new one with new path and set of parameters.
This will lead to the following:
- old test case will stop receiving updates from your code and will become abandoned one, statistics will be frozen on the latest result referring to the old test case
- new test case will receive updates and statistics will start with the first test result.
- Ultimately, your statistics and analytics will become irrelevant for these two test cases.
To avoid abandoning test cases, you must mark your test with AllureID annotation.
- If the tests were never reported to Allure TestOps, you need to upload your test results first time your tests into Allure TestOps as described:
- right click the allure results' folder
- select Allure: Upload results
- select the project to upload to
- name the launch
- Close the Launch to process the results, new test cases will be created
- test cases are created and updated in Allure TestOps only when a launch is closed.
- Right click a package or a class to be processed and select Allure: Assign ID
- In the modal window select the project in Allure TestOps to which you've sent the test results
- Tests will be annotated with @AllureId.
Now, if you modify either package name or the class name or the test name it will be correctly mapped to Allure TestOps test case.
Manual test cases automation
Following workflow allows you easy-peasy automation of a manual test case you have in Allure TestOps.
- Say, in a project you have a manual test case you want to automate.
- In IDE create new class or use an existing one and add a new method (test) marked with @Test annotation
- Right click the method and select Allure: Import test case
- In the modal dialogue provide Allure ID from Allure TestOps to Test Case ID field and select the items to import from Allure TestOps manual test case.
- Your test in IntelliJ IDEA will be updated with the information you selected and will have
- Update the scenario of your test
- Execute your updated test and upload the results
- Close the Launch on Allure TestOps side to process the results
- You Allure TestOps manual test case will become an automated one
- Now, to complete the automation process you need to compare manual and automated scenarios by clicking compare button near Scenario item
- If you're satisfied with the results, remove the manual scenario, and the scenario from test results will be used
- Now, your test case's scenario is being taken from the latest test result
- Now, your test is fully automated
Creation of automated tests form IDE and and reservation of Allure ID
If you are developing an automated test from scratch, you don't need importing the data from a manual test case, so you can create an empty automated test case from IDE and assign Allure ID to it form the very beginning, so all the changes in names and parameters will be reflected in the same Test case on Allure TestOps side.
Workflow is the following:
- You create a test method in your IDE
- You request the Allure ID for your test
- Allure TestOps reserves the AllureID for your test and makes link between AllureID and full path of your test (
- The test case in Allure TestOps is marked as deleted until you sent test results to Allure TestOps.
- Create a new test method and right click on it. Select Allure: Create Test Case
- Select project name where you want your TC to be created in.
- Allure TestOps plug-in will create a test case in deleted state and will provide AllureID to your code.
- Now, you can open test case in Allure TestOps interface by clicking on the context menu item Allure: Open Test Case
- Created test case is added in deleted state as the creation only does the reservation of AllureID.
- Run your test to create test results and upload the test results the test results to Allure TestOps. Remember you need to close the created launch for magic to happen. Click the close button.
- As soon as test results arrived and launch was closed, your test case will appear in the list with the other test cases and Deleted state will be removed.
- Fully reworked the settings and authentication
- Added per IDE project settings – the link between IDE project and a project on Allure TestOps side
- Added notifications.