Migration from Allure TestOps 4.x to 5.x

The architecture of an Allure TestOps deployment was changed in the 5.x release. Migrating from 4.x to 5.x requires a few manual steps and will involve a downtime for your users.

The purpose of the migration described below is to merge data from the UAA service database into the report service database.

After successful merging:

• the resulting report service database will be used in Allure TestOps version 5.x;
• the UAA service database will no longer be used and can be deleted.

0. Upgrade to the last release of version 4 → 4.26.5

Before proceeding, you need to upgrade your current deployment to 4.26.5. Attempting to upgrade any other release directly to 5.x will brick your Allure TestOps deployment, and our support team will not be able to help you recover your data.

Follow the minor upgrade instructions for 4.x first if you are starting with an older release:

• Kubernetes,
• Docker Compose,
• DEB,
• RPM.

After the upgrade to 4.26.5, please make sure that Allure TestOps works fine without errors.

1. Prepare new configuration file

With the migration to version 5.x, the architecture of Allure TestOps has changed — we have merged the gateway, UAA, and report services into a single service called testops. As a result, the configuration files to deploy Allure TestOps have been updated.

Helm chart deployment

Allure TestOps 5.x uses a new Helm chart, hence a new values.yaml file which is incompatible with the one used for version 4.x. You need to create a new configuration from scratch using the new template and set all parameters you need in it. Most parameter have the same or similar effect as they did in 4.x, but some of them have different names and enclosing structure than in 4.x.

Detailed description of the deployment in Kubernetes using Helm chart can be found in the appropriate instruction.

Docker Compose deployment

Allure TestOps 5.x uses new configuration files for docker compose deployment which is incompatible with the one used for version 4.x. You need to create a new configuration from scratch using the new template and set all parameters you need in it. Most parameter have the same or similar effect as they did in 4.x, but some of them have different names.

Detailed description of the Docker compose deployment can be found in the appropriate instruction.

DEB and RPM packages

1. Allure TestOps 5.x uses new paths for storing the configuration file (one configuration file for version 5.x instead of three files in version 4.x). Please refer to instructions for installing Allure TestOps using DEB and RPM packages.

   The parameter names in version 5.x remain the same, but the set of parameters in the single service of Allure TestOps is different. You need to fill the values of the new configuration file based on the settings done for the services for version 4.x.

2. Allure TestOps version 5.x no longer requires the usage of Consul service. It can be safely uninstalled alongside with the version 4.x of Allure TestOps.

2. Shut down 4.26.5 instance

Before proceeding, shut down the Allure TestOps 4.x instance to avoid any data corruption while working with the databases on the next step.

We recommend closing all launches and informing all your users about the upcoming maintenance period.

helm delete allure-testops

docker compose down

sudo systemctl stop allure-gateway allure-uaa allure-report

3. Merge databases

All actions described in this section are mandatory.

Allure TestOps 4.x used two PostgreSQL databases. Allure TestOps 5.x uses a single PostgreSQL database for both user management and test report data.

Before starting Allure TestOps 5.x, you need to migrate the existing data from the UAA service database to the report service database.

The commands below use the pg_dump and pg_restore command-line utilities, along with the following variables:

• password_of_uaa_db_user_here — your password for the UAA service database;
• password_of_report_db_user_here — your password for the report service database;
• --host="db.example.com" — your actual host address.

Back up databases

The way of creating of the database backup depends on your preference and internal procedures. If you are creating regular backups/snapshots of your databases, please create unplanned backup before continuing.

Create full dumps of report and UAA services databases

1. Create a full dump of the report service database:

   # export the password for the database
   export PGPASSWORD='password_of_report_db_user_here'
   # create the database dump
   pg_dump --file=report-dump.sql \
     --host="db.example.com" --port=5432 --username=report --dbname=report
   

2. Create a full dump of the UAA service database:

   # export the password for the database
   export PGPASSWORD='password_of_uaa_db_user_here'
   # create the database dump
   pg_dump --file=uaa-dump.sql \
     --host="db.example.com" --port=5432 --username=uaa --dbname=uaa
   

The resulting files report-dump.sql and uaa-dump.sql will not be used in the merging process. However, we recommend keeping them as a backup to simplify the recovery if something goes wrong.

Create partial dump of UAA service database

This dump will be used in the merging process.

1. Export the password for the UAA service database so pg_dump tool could use it:

   export PGPASSWORD='password_of_uaa_db_user_here'
   

2. Create a partial dump of the UAA service database:

   @hint[warning] In the command below, you need to update database schema in switch --schema if you are using database schema different from public. @end_hint

   Please also make sure that you modified the arguments related to authentication. Removing or modifying other arguments may lead to a failed migration.

   pg_dump --file=uaa_for_merge.dump \
     --host="db.example.com" --username=uaa --role=uaa --dbname="uaa" \
     --schema="public" \
     --exclude-table-data="databasechangelog" \
     --exclude-table-data="databasechangeloglock" \
     --exclude-table-data="shedlock" \
     --format=c --blobs --encoding=UTF8 --compress=9 \
     --no-owner --no-privileges --no-comments  --verbose
   

   Note that the command will produce an extensive log output. Make sure that there are no error messages in the output.

Import partial dump of UAA service database to the report service database

In this step, we are going to load the partial dump created in the previous step into the report service database, as follows:

1. Export the password for the report service database so pg_restore tool could use it:

   export PGPASSWORD='password_of_report_db_user_here'
   

2. Restore the UAA service database partial dump created in the previous step to report service database:

   @hint[warning] In the example below, you need to update database schema in switch --schema if you are using database schema different from public. @end_hint

   pg_restore --host="db.example.com" \
       --username="report" --role="report" --dbname="report" \
       --no-owner --no-privileges --verbose \
       --schema "public" \
       "uaa_for_merge.dump"
   

   Note that the command will produce an extensive log output. Make sure that there are no error messages in the output, except for minor ones:

   • function "now_epoch_mills" already exists;
   • relation "databasechangelog" already exists;
   • relation "databasechangeloglock" already exists;
   • multiple primary keys for table "databasechangeloglock" are not allowed;
   • errors related to the availability of shedlock table.

   If any other error occur on this stage, please consult support team before proceeding any further (select the “Urgent” priority when creating a ticket).

Insert UAA database service changelog to resulting report service database

This step cannot be omitted.

Download the file insert_to_changelog.sql (the changelog for the UAA service database) and run it for the resulting report service database as follows:

1. Export the password for the report service database so psql tool could use it:

   export PGPASSWORD='password_of_report_db_user_here'
   

2. Import the UAA service database changelog into the resulting report service database:

   psql --file="insert_to_changelog.sql" \
     --host="db.example.com" --port=5432 --username=report --dbname=report
   

4. Start 5.x instance

Configure database connection

The database for Allure TestOps version 5.x will be the resulting report service database we got in the previous steps.

docker compose up -d

sudo systemctl start testops