TestOps logo
Try Testops

Deployment of Allure TestOps using docker compose

Important notice

The docker compose deploy described in this article is rewritten in August 2022 and fully replaces previous configuration and becomes the only supported as of 1st of October 2022.

New configuration templates have "lego-like" approach, i.e. there is basic configuration provided in a template and there are additional "brick" elements allowing you to add certain feature to the template.

Applicability

Docker compose deployment is not recommended for the environments with a high workload. If the number of test results in a single launch exceeds 3000, or you have several runs over 24 hours with more than 30000 test results, then you need to deploy Allure TestOps in a Kubernetes cluster.

Docker compose is a good, reliable, and fast option to deploy Allure TestOps for your proof of concept, trial, projects of a small team, and as a system with relatively low workload (see the warning above).

Production deployment hints

For a production deployment using docker compose, we strongly recommend having

  • dedicated PostgreSQL database server version is to be no less than 12.
    • uaa and report databases could reside on the same server or on different ones.
  • dedicated RabbitMQ deployment.
  • external cloud object storage services (AWS, Google) or locally deployed S3 services like e.g. min.io with SSDs under hood.

Stuff described above generally means you need to disable said services in the docker compose configuration file and use dedicated deployments of the services on separated machines in your environment.

Production deployment diagram

Production system deployed via docker compose need to be deployed as displayed in the diagram below.

if you see this text, report to help.qameta.io

  • Allure TestOps services are stateless.
  • Database and S3 storage are stateful.
  • Database server must not reside on the same machine / file system as Allure TestOps deployment.
  • Redis component is only used to store end users' sessions data and is stateless as well.

Target OS

  • We recommend using dedicated bare metal of VM with Linux family OS (Ubuntu, CentOS, RedHat) for the deployment.
  • We do not recommend using servers with Windows OS for the deployment.
  • And we strongly recommend avoiding installation on a laptop or PC as Allure TestOps is a server grade software requiring considerable resources when processing automated tests results.

HW requirements

For the docker compose deployment on a single server you need to have at least the following HW resources:

CPU RAM Storage size Storage type
4vCPU 8GB at least 50 GB SSD

We're talking about modern HW here. If your server is 5 year old and even older then requirements will be different.

The resources described above are enough to start Allure TestOps without hiccups; however, final resources requirement are defined by your workload profile.

Using HDD instead of SSD will dramatically degrade the performance.

Prerequisites

The following items need to be closed before you will be able to deploy Allure TestOps using docker compose:

  1. You need to have docker compose version 2+ installed on your server.
  2. You need to have the username and the password (could be referenced as token as well) to pull the docker images from our private docker hub registry for the deployment. These are usually provided with the trial or commercial licence (see the next item).
  3. You need to have a trial or commercial licence to be able to start Allure TestOps.

Images registry

Here, we assume you have a licence and username and password (token) for pulling images (see items #2 and #3 in prerequisites).

Images for the docker compose deployment need to be downloaded from docker hub registry as follows:

Please read carefully 2 items below, and use a terminal application for all the actions described.

  • Login via docker hub web UI is not possible.
  • Login via docker desktop UI is not possible.

The docker registry to download images from is a private one, so you must use login and password provided with your licence (see item #2 in the prerequisites).

Log in to docker hub

This can be done using the terminal application only.

In the terminal application execute the the following command:

sudo docker login --username qametaaccess

depending on the configuration of your server the sudo may not be required

qametaaccess is the standard login for our private registry at the moment. All the users need to use this username and personal tokens as password provided by the sales team.

next step, docker will prompt you to enter the password, you need to provide the password received from our sales with the trial licence or with a commercial licence. This password could be also referenced as a token.

If you've used correct command and provided correct credentials, you will see something like this:

[email protected]:~$ sudo docker login --username qametaaccess
[sudo] password for user:
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

This is it. Now, we can start the configuration.

Docker compose configuration files

Docker compose configuration consists of 2 files:

  1. .env file where you keep the configuration parameters specific for your deployment. It is prepared by you (don't worry we have a good template prepared already).
  2. docker-compose.yml which contains all the settings needed to run the Allure TestOps services and some auxiliary stuff needed for Allure TestOps to start and run. This file is managed by Qameta Software and you should not apply any changes to it unless specifically recommended by our support team.

GitHub repository with the docker compose configuration files

Here is the address of our official repository with the configuration templates for docker compose deployment:


https://github.com/qameta/allure-docker-compose

Getting configuration templates

Production deployment

Here is set of command which would help you deploying Allure TestOps using docker compose as production system.

In our terms the Production deployment is the one, where only stateless components exist in the docker compose configuration and all the stateful components like databases, artifacts storage and RabbitMQ are deployed as stand-alone solutions.

On the target machine where you are going to deploy Allure TestOps open the terminal application and execute the following commands sequence:


git clone https://github.com/qameta/allure-docker-compose.git
cd allure-docker-compose
mkdir -p /opt/allure-testops
cp ${PWD}/testops/docker-compose.yml /opt/allure-testops/
cp ${PWD}/testops/env-example /opt/allure-testops/.env
cd /opt/allure-testops/

The content of .env file must be altered prior to the deployment. All the following actions can only be performed after you've deployed the database server, S3 storage and RabboitMQ solution.

Edit /opt/allure-testops/.env with your favourite text editor.

What to configure

The following variables need to be set to the real values before starting the deployment:

  • ALLURE_VERSION - Allure TestOps release. Check the release notes for the most recent release
  • ALLURE_HOST - real host's address (FQDN or IP address) as it is accessible for the end users.
  • ALLURE_INSTANCE_PORT - Set your LoadBalancer to route Allure TestOps traffic to port configured in .env file in this parameter
  • ALLURE_PROTO - set accordingly to your preferences, keep in mind that integration with cloud solutions like Jira will require https usage.
  • TZ - set correct time zone of the server where Allure TestOps is deployed.
  • ALLURE_ADMIN_PASS - set strong password for the admin (username is admin) user.
  • ALLURE_JWT_SECRET - secret used by Allure TestOps to cipher the traffic. You can generate the parameter's value using a command openssl rand -base64 16. Copy the result to ALLURE_JWT_SECRET
  • ALLURE_CRYPTO_PASSWORD is used to cipher the secrets in the database, so after adding this to the config, please keep this parameter safe as in case of loss, it won't be possible to decipher the data stored in the DB and you will need to recreate all the secrets.
  • ALLURE_SECURE_COOKIE is to be set to true only if ALLURE_PROTO is set to https.
  • All parameters having ALLURE_REPORT_DB_ prefix
  • All parameters having ALLURE_UAA_DB_ prefix
  • All the parameters with _PASS postfix. Set your data or leave the example passwords as they are set in the template.

Allure TestOps demo config deployment

Demo deployment is only suitable for PoC phase. In case of any configuration changes this deployment type will require tedious migrations which will complicate your life. We strongly recommend avoiding this type of deploy as a production system.

  1. Install Docker by running
curl -fsSL https://get.docker.com -o get-docker.sh
chmod +x get-docker.sh
./get-docker.sh
  1. Download the repo with the config template and copy the template to deployment dir:
git clone https://github.com/qameta/allure-docker-compose.git
cd allure-docker-compose
mkdir -p /opt/allure-testops
cp ${PWD}/testops-demo/docker-compose.yml /opt/allure-testops/
cp ${PWD}/testops-demo/env-example /opt/allure-testops/.env
cd /opt/allure-testops/

Edit /opt/allure-testops/.env with your favourite text editor.

What to configure

The following variables need to be set to the real values before starting the deployment:

  • ALLURE_VERSION - Allure TestOps release. Check the release notes for the most recent release
  • ALLURE_HOST - real host's address (FQDN or IP address) as it is accessible for the end users.
  • ALLURE_INSTANCE_PORT - Set your LoadBalancer to route Allure TestOps traffic to port configured in .env file in this parameter
  • ALLURE_PROTO - set accordingly to your preferences, keep in mind that integration with cloud solutions like Jira will require https usage.
  • TZ - set correct time zone of the server where Allure TestOps is deployed.
  • ALLURE_ADMIN_PASS - set strong password for the admin (username is admin) user.
  • ALLURE_JWT_SECRET - secret used by Allure TestOps to cipher the traffic. You can generate the parameter's value using a command openssl rand -base64 16. Copy the result to ALLURE_JWT_SECRET
  • ALLURE_CRYPTO_PASSWORD is used to cipher the secrets in the database, so after adding this to the config, please keep this parameter safe as in case of loss, it won't be possible to decipher the data stored in the DB and you will need to recreate all the secrets.
  • ALLURE_SECURE_COOKIE is to be set to true only if ALLURE_PROTO is set to https.
  • All the parameters with _PASS postfix. Set your data or leave the example passwords as they are set in the template.

Set your LoadBalancer to proxy traffic to ALLURE_INSTANCE_PORT port (Gateway)

Start Allure TestOps

Once you have all of your configuration options completed in .env and the file is saved (save, yes please!!!), you can start the deployment.

docker compose up -d

The very first start usually takes up to 5 minutes. Further starts should take less than 1 minute.

Accessing Web interface

Allure TestOps web UI is available at http://ALLURE_HOST:ALLURE_INSTANCE_PORT

Please remember, host, port and protocol are defined by you in .env configuration file.

Initial login

Log in to Allure TestOps using username admin and password from your configuration file.

admin log-in information

Allure TestOps requires an admin user account to be created and kept in the system.

This user's name and password are defined in the configuration file, and it will be restored to the state described in the configuration to ensure you won't lose the access to your Allure TestOps instance.

You cannot delete/disable this user and you cannot remove the admin's rights for this user – during the next start of Allure TestOps, it will recreate the user again with all set of available rights, with the password defined in the configuration file.

If you omit provisioning of initial admin's password, then default user admin will be created and admin's strong password will be generated by the system, and then the generated password will be sent to the logs of uaa service.

Each time Allure TestOps is restarted, the admin user account is restored to its initial state defined in the config file .env

Provide the license for your Allure TestOps instance

Next thing you see will be the modal window to which you will need to provide the license you acquired.

if you see this write to help.qameta.io

After the correct license is provided, you'll be able to start your work.

If you aren't able to log-in more than 10 minutes after the up -d command, please check the troubleshooting section for logs generation and create a support ticket with the logs. Please always use your corporate email address (email address of the company for which the licence is issued).

Release upgrades

If your release of Allure TestOps is lower than 3.177.2, please refer to special update procedure to 3.177.2 and next ones.

Once your Allure TestOps started, configuration changes and images updates should be done using the following set of commands:

Update release version in the file .env for the variable ALLURE_VERSION, then run the commands:

docker-compose pull
docker-compose down

Wait until all the services have stopped and run Allure TestOps via docker-compose:

docker-compose up -d

Troubleshooting

In case of any troubles, do not immediately bring Allure TestOps down and up. We need the log files of each component to understand what has happened.

What is not needed for tech support

Please do not

  • provide the screenshots of logs in your terminal window. These are completely useless for the analysis.
  • provide copy paste of full log to a support ticket. These are impossible to analyse in the WEB UI of help desk.
  • provide a log for all elements together

What is needed for support

Please always provide the following information:

  • Allure TestOps release (see URL/status page of your Allure TestOps instance)
  • Releases of all the tools/integrations you have problems with (Allure TestOps plugins, allurectl, IDE plugins)
  • logs for each Allure TestOps' service as a separate text file

Getting the logs

You can get all the logs for your Allure TestOps instance by executing following command:

docker compose logs -f
  • this will continuously update the logs from all the services
  • this log is good for a quick analysis on your side, please do not provide this log to the technical support.

Logs of specific component

To get logs for a specific component you need to specify a service name for which you want to see the logs.

To get the name of services of your docker compose deployment you need either check your docker-compose.yml file or execute the following command in your terminal (your current directory must contain docker compose config files).

docker compose ps

this command will result into a similar output:

[email protected] folder-name % docker compose ps
NAME                 COMMAND                  SERVICE                    STATUS               PORTS
allure-gateway       "/bin/sh -c /entrypo…"   allure-gateway             running (healthy)   0.0.0.0:10777->8080/tcp
allure-report        "/bin/sh -c /entrypo…"   allure-report              running (healthy)
allure-uaa           "/bin/sh -c /entrypo…"   allure-uaa                 running (healthy)
autoheal             "/docker-entrypoint …"   autoheal                   running (healthy)
minio-local          "/opt/bitnami/script…"   minio-local                running              0.0.0.0:9000->9000/tcp
minio-provisioning   "/bin/sh -c 'mc conf…"   minio-local-provisioning   exited (0)
rabbit               "/opt/bitnami/script…"   rabbitmq                   running              25672/tcp
redis                "/opt/bitnami/script…"   redis                      running (healthy)    6379/tcp
report-db            "docker-entrypoint.s…"   report-db                  running              5432/tcp
uaa-db               "docker-entrypoint.s…"   uaa-db                     running              5432/tcp

The column we need is SERVICE.

to get all logs of Allure TestOps services saved to separate files you need to execute the following command:

docker compose logs allure-report > report-logs.txt && \
docker compose logs allure-uaa > uaa-logs.txt && \
docker compose logs allure-gateway > gateway-logs.txt

What else is worth checking

Also check the following:

  1. Available memory for the artifacts storing df -h --total on your storage
  2. Available inodes for the artifacts storage df -i
  3. Available RAM free -h, top
  4. Other services logs.

Alternatively you can use this script to get the needed logs.

Uninstall

This will remove all the volumes with their data; no data will remain on your system, and this data cannot be restored.

To uninstall the Allure TestOps, run the following:

docker compose down -v --rmi local

Database for production deployment

Dedicated PostgreSQL database server is a must in a production environment.

Connecting the external database contains following steps:

Minimal supported release of PostgreSQL is 14 as of the 1st January 2023.

  1. Creating the databases for services uaa and report using recommended Postgres DB version (see the line above).
  2. Migration of data from existing database to the standalone one. This step is to be skipped if you're deploying Allure TestOps from scratch.
  3. Updating the settings to connect uaa and report services to stand alone database.

Creating the databases

Using your Postgres DBA you need to create 2 databases - for the services 1. uaa and 2. report.

You cannot store the data for these 2 services in a single common database, this will brick the system.

  1. The uaa database stores all the data related to the users' profiles and licensing.
  2. The report database stores tests' data.
Creating uaa database

Following script will create empty database and during the first start uaa service will execute a series of SQL update scripts to create actual database structure.

CREATE DATABASE uaa TEMPLATE template0 ENCODING 'utf8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8';
CREATE USER uaa with encrypted password '[email protected]';
GRANT ALL PRIVILEGES ON database uaa to uaa;
Creating report database

Following script will create empty database and during the first start report service will execute a series of SQL update scripts to create actual database structure.

CREATE DATABASE report TEMPLATE template0 ENCODING 'utf8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8';
CREATE USER report with encrypted password '[email protected]';
GRANT ALL PRIVILEGES ON database report to report;

Migration from the DB in a container to standalone DB

The migration of the existing data to the standalone database is described in FaQ section.

You need to create a dump from uaa and report databases and then restore the database dumps to appropriate database. The date restoration to a wrong database will corrupt the data as the databases of uaa and report services have different scripts to get these to the actual database structure.

Updating the settings of services

UAA service

For uaa service to work with an external database you need to update the following in .env file, only the items to be updated are shown:

ALLURE_UAA_DB_HOST=uaa-db.host.address # <<< your hostname here. e.g. 192.168.0.100
ALLURE_UAA_DB_NAME=uaa # <<< your db name for UAA service, uaa is used by default, it's better to keep it the same
ALLURE_UAA_DB_USERNAME=uaa # <<< uaa DB username (see above the creation of the DB for UAA)
ALLURE_UAA_DB_PASS=<password> # <<< uaa DB password (see above the creation of the DB for UAA)
ALLURE_UAA_DB_PORT=5432 # <<< uaa DB port, 5432 is the default one

report service

For report service to work with an external database you need to update the following in .env file only the items to be updated are shown:

Report

ALLURE_REPORT_DB_HOST=report-db.host.address # <<< your hostname here. e.g. 192.168.0.100
ALLURE_REPORT_DB_NAME=report # <<< your db name for report service, `report` is used by default, it's better to keep it the same.
ALLURE_REPORT_DB_USERNAME=report # <<< uaa DB username (see above the creation of the DB for report).
ALLURE_REPORT_DB_PASS=pasword # <<< report DB password (see above the creation of the DB for UAA)
ALLURE_REPORT_DB_PORT=5432 # <<<report DB port, 5432 is the default one

Configuring minio external S3 storage for artifacts

S3 storage type is more reliable in terms of processing of the collisions when saving and deleting the data during the batch operations with files, so it is recommended at least using min.io instead of the direct usage of a file system to store the artifacts.

If you are going to connect S3 storage for the test cases and test results artifacts (which is recommended for the production) you need to add the following parameters to report service (only the parameters you need to update in .env file are sh):

ALLURE_S3_PROVIDER=s3
ALLURE_S3_URL=https://<your_S3_provider_API_endpoint_here>
ALLURE_S3_BUCKET=allure-testops-bucket-name>
ALLURE_S3_REGION=us-east-1 # S3 region is to be set
ALLURE_S3_ACCESS_KEY=<your_access_key>
ALLURE_S3_SECRET_KEY=<your_secret_key>
# Do not change ALLURE_S3_PATHSTYLE, it's Minio's parameter, not external S3's
ALLURE_S3_PATHSTYLE=true

Connecting S3 storage

The integration with external storage is being performed using the environment variables.

If you're using the deployment via container the connection to S3 storage is being performed by adding of environment variables to report container as follows:

For AWS' S3

      ALLURE_S3_PROVIDER: "S3"
      ALLURE_S3_URL: "s3.amazonaws.com"
      ALLURE_S3_REGION: "us-east-1" # region, update to real region you are using
      ALLURE_S3_BUCKET: "<your-bucket-name>" # replace string in <> with your bucket's name
      ALLURE_S3_ACCESS_KEY: "<ACCESS-KEY>" # replace string in <> with your Access key ID
      ALLURE_S3_SECRET_KEY: "<SecretKey>" # replace string in <> with your Secret access key
      ALLURE_S3_PATHSTYLE: "true"

If your AWS region is different from us-east-1, it's recommended to use the S3 endpoint like described here: https://docs.aws.amazon.com/general/latest/gr/s3.html

Access rights settings for S3 bucket

For the S3 integration to work smoothly with Allure TestOps following access rights to be configured for S3 bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3assets",
            "Effect": "Allow",
            "Action": [
                "s3:PutObjectAcl",
                "s3:PutObject",
                "s3:ListMultipartUploadParts",
                "s3:ListBucketMultipartUploads",
                "s3:ListBucket",
                "s3:GetObjectAcl",
                "s3:GetObject",
                "s3:GetBucketLocation",
                "s3:GetBucketAcl",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::<ALLURE_S3_BUCKET_NAME>/*",
                "arn:aws:s3:::<ALLURE_S3_BUCKET_NAME>"]
        }
    ]
}

For Google's S3


      ALLURE_S3_PROVIDER: "S3"
      ALLURE_S3_URL: "https://storage.googleapis.com" # leave as it is here
      ALLURE_S3_REGION: "europe-west3" # region, update accordingly your real region
      ALLURE_S3_BUCKET: "<your-bucket-name>" # replace string in <> with your bucket's name
      ALLURE_S3_ACCESS_KEY: "<ACCESS-KEY>" # replace string in <> with your Access key ID
      ALLURE_S3_SECRET_KEY: "<SecretKey>" # replace string in <> with your Secret access key
      ALLURE_S3_PATHSTYLE: "true"

For Google S3 you need to use fine-granted access control and settings for public access Subject to object ACLs, this is needed for interoperability with AWS SDK, otherwise files in S3 bucket won't be fully accessible by Allure TestOps and report service will fail to start.

If you have any doubts, please consult our Support

Using self-signed certificates

If you have requirements to use the self-signed certificates to cipher the traffic in your network, you need to set the following parameters;


TLS_ENDPOINTS=ldaps.replace.with.real.data.com:636,rabbit.replace.with.real.data.com:5672
TLS_DB_ENDPOINTS=db.replace.with.real.data.com:5432

If you aren't using self-signed certificates, then these parameters need to be set to empty value or commented.

Preparation for the production deployment

Before putting your system to the production, please refer to the recommendations here.

If you have any doubts, please consult our Support before getting into troubles.

Back to installation

© 2023 Qameta Software Inc. All rights reserved.