Release Notes

We strongly advise upgrading your Allure TestOps instance only to the releases listed in this very release notes document unless different release is advised by our support team.

For the server version, migration from Allure TestOps ver. 4 to Allure TestOps ver. 5 requires new configuration files and databases merging.

The source of the configuration files and description of the parameters is described in the installation section for each type of the installation. The databases merging process is described in this section.

If you have any doubts or questions about the merging process, please do not hesitate to consult the support team.

Migration from version 4 is only possible from release 4.26.5. Using older releases will brick your system and lead to data loss. In this case, the support team won't be able to help with data recovery.

Starting with Allure TestOps version 5.11, migration of test case trees to the new data structure is enabled. For the server version of Allure TestOps installation, this may lead to increased database load as the background migration takes several days to a week.

The risk of data structure corruption is minimal. If you encounter problems with the migration, please contact support team.

To continue working with the previous data structure, disable the New test cases tree checkbox in the user profile. Please note that the data migration will remain active and may increase CPU load by up to 10%.

20 December 2024 — Allure TestOps 24.4.1 (previously 5.12.0)

Starting with Allure TestOps version 24.4.1 (previously 5.12.0), we are implementing the new version numbers to simplify keeping up with releases. Now each version of Allure TestOps will be in the YY.Q.X format.

New Features

New Navigation

Updated the design and functionality of the navigation in Allure TestOps:

• Optimized the size of the sidebar: it takes up less space now and can be hidden and expanded when needed.
• Added the breadcrumb navigation at the top of the screen: now there is quick access to favorite projects and Administration section, users can navigate Allure TestOps sections via drop-down menu.
• Moved the user menu to the top right corner of the screen.

Updated dashboards

After upgrading to Allure TestOps version 24.4.1 (previously 5.12.0), the Automation Trend widget on dashboards will initially appear empty. Data for it will start being collected from the date of installation of this release once a day.

Removed the Analytics section: all features of this section will be available in the Dashboards section.

Updated the Dashboards section:

• Changed the approach to working with the Overview tab that was added to the project by default: it is now an editable dashboard.
• Updated the dashboard creation process: dashboard can now be created from a template with a specific set of widgets or as an empty dashboard to be configured from scratch.
• Added the ability to drag and drop tabs with dashboards: to do this, left-click a dashboard tab and drag it. The order of dashboard tabs will be updated for all project collaborators.
• Added the ability to copy dashboards: an already created and customized dashboard can be copied to any project to which the user has access.
• Updated interaction with dashboards and widgets via menu: now right-click on a dashboard tab opens menu with settings of the whole dashboard and right-click on a widget opens widget settings menu.
• Updated design of widgets and empty dashboards to optimize data display and dashboard customization.

Test case trees and filters

Added a feature flag to display new test case trees and filters in the Test cases section. This public feature flag is enabled by default for all users, you can disable it in the user profile.

The updated test case trees and filters allow users to:

• drag and drop multiple test cases and categories at once within the tree;
• create empty folders in the test case tree;
• apply additional options for sorting test cases;
• customize the test case tree in more detail for better performance;
• search for test cases by text in their scenario.

Added a section with deleted test cases: it contains test cases after deletion, they can be restored if necessary or deleted permanently. To access it, select Trash from the test case tree drop-down list.

Feature flag “New launches list”

Added new feature flag that activates the updated Launches section with the following improvements:

• Display launch duration time — the launch status shows how much time has passed since it was started.
• Environment variables — both names and values of environment variables are displayed in the launches list.
• Filtering by tags and environment variables — clicking on the tag or environment variable automatically filters the launches list by the selected parameter.
• Updated progress bar — the launch progress bar is now interactive. Clicking on the specific status color opens the test results filtered by the corresponding status.
• Defect viewing — for each launch, a list of related defects is displayed with active links to view them.
• Updated filters — the filter interface and functionality were improved.

The feature flag is available to all Allure TestOps users. To activate it, follow the instructions in Manage feature flags.

If you have any suggestions or feedback regarding the new feature flag, please submit a request to our support team at https://help.qameta.io/.

Improvements

• Optimized the process of deleting and restoring test cases. Now, when deleting test cases, custom field values are not deleted: after restoring a deleted test case, the custom fields in it will be filled in and you will not need to specify the values of custom fields again.
• Optimized adding related issues from an issue tracker to a test case. Now, if Allure TestOps is integrated with only one issue tracker, its name is automatically filled into the Issue tracker field when editing the Issues links.
• Added minor updates and fixes to the Allure TestOps interface to optimize it and improve user experience.

Bug Fixes

• Fixed an issue in the Launches section when users with the Guest role could not see the Compare button. Now users with view-only permissions can compare test launches.
• Fixed an issue that caused custom fields from other test cases in the project to be deleted when deleting test cases using bulk operations. The error occurred when a user selected all test cases in the tree, deselected some test cases and deleted the selected test cases using bulk operations.
• Fixed the error display in test results. Now, when opening the result of an automated test, detailed text about the errors encountered during testing is shown in its scenario.
• Fixed the issue where step progress was reset when reopening a test result if the user had previously selected the Pass previous option in the Step run actions menu in the test scenario.
• Fixed the issue that caused Firefox browser to display outdated user profile image after uploading a new photo.
• Fixed the error that occurred when renaming a custom field value from the test case list (test case tree) if that value already existed in the project.
• Fixed the issue with text color that made text in test case status message blend with the background in Dark mode.
• Fixed the issue with browser cache not being properly utilised that caused longer loading times for Allure TestOps sections.

30 October 2024 — Allure TestOps 5.11.5

15 October 2024 — Allure TestOps 5.11.4 (server version)

27 September 2024 — Allure TestOps 5.11.3

Starting from release 5.11.3, Allure TestOps will not start if the PostgreSQL database server version is below 15. This release requires an update to the configuration files.

Please refer to the configuration update page for details on the changes required before upgrading Allure TestOps to version 5.11.3.

Bug Fixes

• Fixed the issue where opening a saved filter with values for the Member parameter reset those values and resulted in a 400 error.
• Fixed the API request handling when an invalid token is used. Now it returns a 401 error instead of a 200 response.
• Fixed the issue where test results did not save user-specified details about the execution of steps with expected results.
• Fixed the incorrect display of test case statuses in the Jira plugin iFrame.
• Fixed the distribution of database connections between the main pool and the pool used for data migrations within the database.
• [5.11.4] Fixed the error “HTTP Status 500 — Internal Server Error” that occurred when attempting to authenticate users in Allure TestOps using Okta as the OpenID identity provider.
• [5.11.5] Optimized API query /api/evv/suggest to improve database performance under high load conditions.
• [5.11.5] Updated the versions of the plugins io.spring.dependency-management and org.springframework.boot used in Allure TestOps.

11 September 2024 — Allure TestOps 5.10.1

Improvements

• Added minor updates and fixes to the Allure TestOps interface to optimize it and improve user experience.

Bug Fixes

• Fixed the issue when users running test cases with both empty and filled scenarios could see steps from other test cases in the results of test cases with empty scenarios within the same launch.
• Fixed the text alignment in markdown fields from justified to left-aligned.
• Restored the functionality of the Ctrl+C (for Windows) and Command+C (for macOS) keyboard shortcuts in test result scenarios.
• Optimized the rendering of test results, thereby fixing the issue of their slow loading when processing large volumes of data.
• Fixed the error "HTTP Status 500 — Internal Server Error" during failed user authentication in Allure TestOps using OpenID. Now, if users have errors in their OpenID configuration or encounter sign-in issues on the Allure TestOps side, users are redirected to the sign-in form.

27 August 2024 — Allure TestOps 5.9.0

New Features

Checking for complete user profile information: first name, last name, email

To help us integrate notifications into Allure TestOps, it's essential to learn more about our users. After you sign in, we will ask you to update your personal information in user profiles if any part of it is missing. Please fill out the Complete your profile form with your work email, first name, and last name just once and then continue using Allure TestOps features.

Improvements

• Added the ability to manage the duration of inactive user sessions via configuration parameters of the instance.
• Added the ability for all Allure TestOps users to edit their profile information including name, last name, and email. Additionally, users can no longer reset their passwords through the user profile, but they can still do so via the sign-in form as before.
• Updated the capabilities of the Jira plugin iFrame. If a defect is linked to a Jira issue, the related test cases and launches will now be displayed within the issue.

Bug Fixes

• Restored the automatic assignment of test results to the user who started executing the steps.
• Fixed the incorrect display of column headers in the launches comparison table when these columns could only be viewed through horizontal scrolling.
• Fixed the issue when an error in a nested step was duplicated in the parent steps in the uploaded test results. Now an error message is visible only in the step where the error occurred.
• Fixed the problem when attempting to edit an issue link in a test case or test result would create a new issue link instead of updating the existing one.
• Fixed the issue when automated test cases with scenarios generated from test results displayed unnecessary text about attachments in the test results. Now the scenarios in such test cases no longer include steps in the format Attachment [ID] from TestResult.
• Fixed the display of the button for editing links in automated test cases when the permission for this editing is enabled in the meta data update policy.
• Fixed the issue when changes to the full name of an automated test case were not reflected in its change log.
• Fixed the incorrect handling of actions when pressing the Enter key on the numeric keypad. Now the Allure TestOps response is identical when pressing any Enter key, whether on the numeric keypad or the main alphanumeric section of the keyboard.
• Fixed the issue that caused SVG attachments in scenarios of test cases or test results to download when attempting to expand the attachments.
• Fixed the behavior of the Environment section in the Filters panel of test results. Now users can filter test results only by the names of environment variables used in the launch.
• Fixed the unstable behavior of radio button filters caused by the loss of their default values in test cases, launches, and test results.
• Fixed the excessive creation of records in the Redis database when using an authorization token instead of a bearer token for authorization API calls.

16 August 2024 — Allure TestOps 5.8.1

13 August 2024 — Allure TestOps 5.8.0

Improvements

• Added minor updates and fixes to the Allure TestOps interface to optimize it and improve user experience.

Bug Fixes

• Fixed the display of scenarios in test results for automated tests. Now scenarios are collapsed by default.
• Fixed the issue when users couldn't send API requests using Swagger UI with Nginx configured.
• Fixed the display of the test result options bar in closed launches. Now the buttons to rerun test results manually, mute them, and link defects are not available in closed launches.
• Fixed the behavior of the environment variable value field during launch creation. Now when the user changes the name of an environment variable, the system resets its previous value.
• Fixed the issue when scenarios generated from test results in automated tests could not be collapsed. Now users can manually collapse/expand individual steps of such scenarios.
• Fixed the issue of duplicate test results in the response of the API request /testresult/__search.
• Fixed the playback of MOV format videos in Google Chrome.
• Fixed the launch name when creating a job run. Now the name field uses a default value that includes the launch date and time.
• Fixed the issue when, during user authentication via OpenID, the restriction on roles synchronization was ignored, granting the user more privileges than allowed.
• Fixed the issue of the empty test case window in Jira when integrated with Allure TestOps.
• Fixed the issue with the first admin invitation email being resent every time the Allure TestOps instance is restarted.
• [5.8.1] Fixed the issue when users couldn’t sign in via OpenID with Azure AD as the identity provider.

30 July 2024 — Allure TestOps 5.7.0

New Features

• Added ‘No assignee’ option to filters in the Launches/Tree section. Previously it was possible to filter only by known assignee — only tests with assignee were shown.

Improvements

• Improved the OpenAPI documentation for the Allure TestOps API. Now API documentation provides information about the most relevant path for API methods used in Allure TestOps.
• Added minor updates and fixes to the Allure TestOps interface to optimize it and improve user experience.

Bug Fixes

• Fixed the validation error issue with manual test result uploads. Now when the user selects different file type in the upload form, error message disappears.
• Stabilized the behavior of the input fields for entering environment variables in the launch creation form.
• Fixed the issue with resetting custom field name values when renaming custom fields, which occured when the user switched between the tabs before saving changes.
• Fixed the incorrect behavior of the defect creation form resulting in the deletion of information entered by the end user in automation rule name and automation rule regular expression when attemping to alter the name of the defect.
• Fixed the incorrect behavior of the test scenario comparison form when attempting to copy steps from test result scenario to the stored scenario.
• Fixed the backend issue which resulted in 5xx errors for the end user when processing attachments with '%' at the end of the file name.
• Fixed the issue with parameters of the test case scenario steps not being rendered in the test results.

23 July 2024 — Allure TestOps 5.6.4

New Features

Bulk operations with custom field values

We’ve redesigned the bulk operations for custom field values. Now you can merge multiple custom field values into one and delete multiple values from custom fields using bulk operations in the Project settings/Custom Fields section.

To merge multiple custom field values into one, in the Project settings/Custom Fields section open the custom field card, select the required custom field values, click Merge on the toolbar that appears when selecting multiple values, and set the name for the new value. All merged custom field values will be replaced with the new value in test cases. You can also assign the created value as default for new test cases with this custom field.

To delete multiple values from custom fields, in the Project settings/Custom Fields section open the custom field card, select two or more values, and click Delete on the toolbar that appears.

OpenID in Allure TestOps server versions

We’ve returned user authentication via OpenID for server versions of Allure TestOps.

Improvements

• We’ve redesigned the empty state screens in the Test cases, Launches, and Dashboards sections. Updated empty states provide more clarity on how to interact with Allure TestOps.

Bug Fixes

• We’ve fixed the ability to navigate within multi-line scenario steps using navigation arrow keys on the keyboard.
• We’ve updated the error message for cases when users launched automated tests that do not have a job assigned to them. Now the error text describes the problem in more detail and users are being navigated to the Jobs tab within the launch window.
• We’ve fixed the issue with AQL tooltip disappearing when users hover over it and try to click the link in it’s description.
• We’ve fixed the issue with applying previously used names to custom field values. Now you can rename custom field values and later return their previous names.

08 July 2024 — Allure TestOps 5.5.0 (cloud)

New Features

Notifications in trial period

We’ve added notifications about trial period to help our users navigate licensing. Now you can see how many days are left before your Allure TestOps trial period expires, how to upgrade to a paid version, and who to contact to learn more about available pricing options.

Formatted scenarios in Test results

We’ve refactored the scenario structure to be able to display scenario formatting in the Launches section: steps in test result scenarios are displayed in Launches the same way they are displayed in the Test cases section.

Improvements

• We’ve updated the settings for users with administrator role:
  • Now the administrator's (the one created with the instance and usually having the username admin) password is not reset after Allure TestOps instance was restarted.
  • All administrators can see when the license expires and add new license key.
  • Administrators without name and email in user profile will need to provide this information after sign-in.
  • [Server version] Administrator’s email should be specified in the configuration file when starting a new instance, SMTP configuration is mandatory for all new instances. In existing instances, the administrator will be able to sign in using the previously created login/password combination.
• We’ve improved the process of moving tests to another project, now only target projects where user has write permissions will be suggested.
• We’ve improved the process of deleting attachments in test results. Now only attachments added to a test result as the Launch progressed can be edited and deleted. If the user deletes such an attachment in the attachment tab, it is also deleted from the test result's scenario. Attachments, that came from test cases, can’t be edited or deleted from test results.
• We’ve optimized the performance when rendering test case scenarios with large number of steps.
• We’ve enhanced filters in the Launches section, so they will display the suggested lists for tags only used in the scope of the current launch.
• We introduced new feature for highly loaded systems deployed 'on-premises' allowing to split the datasources for upload process and getting the statistical information for dashboards, so now you can point the stats requests to a read-only replica, thus unloading the production database.
• We’ve returned and updated bulk operations for custom fields. Now users can add and remove custom field values from the test case tree using bulk operations, which enhances the efficiency and ease of management.

Bug Fixes

• We’ve fixed the problem with searching for issues in Azure integration.
• We’ve fixed the deletion process for stored scenarios (the ones you have from manual test cases when performing the automation) in automated test cases.
• We’ve fixed the behavior in test scenarios resulting in scenario steps being deleted after users select all text in scenario step and press Backspace key. Now the scenario step stays intact and only selected text is deleted.
• We’ve fixed the permissions selector in the Project settings/Access section, now all previously created access permissions are displayed.
• We’ve fixed the issue with project name not being displayed when editing the TMS Export configuration settings. For previously configured TMS Export settings, users need to re-select project and save settings.
• We’ve fixed processing of tags with spaces and special characters in Jira integrations. Now tags that aren't matching Jira's labels format requirements will be omitted and won't create unnecessary workload.
• We’ve fixed the authentication exception behavior when processing authentication tokens.
• We’ve fixed the issue with restoring previously deleted test cases.
• We’ve fixed the issue where test results displayed custom fields deleted from the Administration/Custom fields section before creating the launch.
• We’ve fixed errors caused by external system authorization in Allure TestOps.
• We’ve fixed the issue related to displaying project participants and the inability to create projects for the first admin.

04 June 2024 — Allure TestOps 5.3.2 (cloud)

At the moment, version 5.3.x is available for SaaS users only. We are preparing migration guides for existing users, and we will issue new documentation and configuration files in a couple of days. Hold on a bit.

Bug Fixes

• We’ve fixed the issue with displaying test result trees in the Launches section.

03 June 2024 — Allure TestOps 5.3.1 (cloud)

31 May 2024 — Allure TestOps 5.3.0 (cloud)

New Features

New profile page and UI themes

We’ve redesigned the user profile page and added the ability to select the UI appearance theme: now you can switch between the light, dark, ands system themes.

Updated deletion process for custom fields

Now custom fields are deleted in two steps:

1. Remove custom field from all projects it has been added to previously, then delete this custom field from the list of active custom fields by using the Delete option in the custom field menu. It will be moved to the Deleted tab and disabled in all test cases.
2. In the Deleted tab, click Delete permanently. Custom field will be deleted from the system.

After the custom fields are deleted, they disappear from test cases and test case trees. Test case trees that use the deleted custom fields can’t be selected.

The two-step deletion process allows to make sure that deleted custom fields are not used in any projects and recover them if necessary.

Required custom fields

Custom fields can now be assigned ‘required’ status independently for each project.

To assign the ‘required’ status to custom field from the Administration section:

1. In the Administration section, open the Custom fields subsection and click on the custom field.
2. Open the project menu next to the selected project and switch on the Required field toggle. Custom field will be marked as required in this project.

To assign the ‘required’ status to custom field from the project settings:

1. In the project settings, open the Custom fields subsection and click on the custom field.
2. Open the custom field menu and switch on the Required field toggle. Custom field will be marked as required in the current project.

In test cases, required custom fields are highlighted in red when empty.

In filters, required custom fields are marked as required with a tooltip.

Filtering by empty values in custom fields

We’ve added the ability to filter test cases by empty custom fields.

Advanced options in cloning test cases

We’ve expanded the list of actions for missing custom fields when cloning or moving test cases between projects. Now you can:

• add missing custom fields to the target project,
• map them to existing custom fields in the target project,
• exclude missing custom fields from the resulting copy of the test case.

We’ve also improved the UI to simplify the process of cloning test cases.

Improvements

• We’ve made some UI improvements: updated icons, buttons, and colours across the app.
• We’ve added the confirmation for closing in progress launches. Now the users are warned about the consequences and can make an informed decision when closing in progress launches.
• We’ve updated the UI in the Launches/Test results section. Now it follows the UI in the Test cases section.

• We’ve improved the processing for large amounts of data and now Allure TestOps supports:
  • data caching in the Dashboards and Analytics sections,
  • several data sources separately for loading/processing operations of test results and widgets,
  • separate configuration of the MQ listeners related to test results processing and post-processing.
• We’ve changed the behaviour of CSV import. Now allure_id field is skipped by default in CSV import mappings.

Bug Fixes

• We’ve fixed the issue with Edit and Delete buttons shifting in automation rules with long Trace pattern.
• We’ve fixed the issue when attempting to run automated tests without selected a job. Now job selection is mandatory.
• We’ve fixed the UI bug with checkboxes being visible through search input.
• We’ve added mandatory validation for assignees when running tests.
• We’ve fixed the issue with Xray integration sync.
• We’ve fixed errors with linking issues to Yandex Tracker.
• We’ve fixed the issue with users being unable to fully clear search input in select component.
• We’ve fixed the issue with test scenarios displaying duplicate steps with expected results.
• We’ve fixed the issue with environment variables in the Launches section containing values that are not used in the current project.
• We’ve fixed initial loading of options for the integration project select component.
• [5.3.1] We’ve fixed the issue with deleting custom fields that were previously used in deleted test cases. Now custom fields are automatically removed from deleted test cases and can be deleted from the project.

Allure TestOps 4.x

For the release notes of Allure TestOps version 4.x, please refer to this page.

Allure TestOps 3.x

For the release notes of Allure TestOps version 3.x, please refer to this page.