Deploy Allure TestOps with Kubernetes

Deploying Allure TestOps using Kubernetes

The easiest method to deploy Allure TestOps on Kubernetes is to take advantage of Allure’s Helm charts. Helm is a package management tool for Kubernetes allowing applications to be easily managed via their Charts.

The allure-ee chart includes all required dependencies, and it takes a few minutes to deploy.

For production deployments, we strongly recommend using the detailed installation instructions utilizing external PostgreSQL, RabbitMQ, and object storage services.

Prerequisites

Hardware requirements

Check out this page and then get back to this one.

Prerequisites

  1. You need to obtain the license key and image pull token.
  2. You need to have helm and kubectl installed on your machine.

Configuration

  1. You don’t need to download whole chart.
  2. You need to create values.yaml (name is up to you actually) containing the settings specific for your deployment.
  3. Helm will take your settings and apply these to the chart.

Prepare configuration file

Starting from Allure TestOps release 3.181.5 licensing information is stored in the uaa database, you don’t need to provide license into configuration files.

Create configuration file config.yaml with the minimal content:

Refer to this article for the mandatory params.

version: 3.190.4

username: admin
password: admin

imagePullSecret: docker-hub


jwtSecret: AAAAAAAAAAAAAAAA

ingress:
  host: allure.local
  
uaa:
  env:
    open:
      ALLURE_ENDPOINT: http://allure.local
    secret: 
      ALLURE_CRYPTO_PASSWORD: "AAAAAAAAAAAAAAAA"
      

report:
  env:
    open:
      ALLURE_ENDPOINT: http://allure.local
    secret:
      ALLURE_CRYPTO_PASSWORD: "AAAAAAAAAAAAAAAA"
  probes:
    enabled: false
# it makes sense to disable probes for the initial deploy and enable after the system is deployed and functional

Update the version of Allure TestOps with the latest recommended one.

jwtSecret is a random string, you can generate it by using this command:

openssl rand -base64 16

Install

By default Allure is started in a default namespace.

If you are using a namespace alternative to the default one, do not forget adding -n <yournamespace> to all your commands when applying configuration for Allure TestOps.

Create docker registry secret

Run the following command with hub.docker.com credentials:

kubectl create secret docker-registry docker-hub \
  --docker-server=https://index.docker.io/v1/ \
  --docker-username=qametaaccess \
  --docker-password=<image pull token>

Get Allure TestOps up and running

Once you have all of your configuration options collected, you can get all dependencies and run helm. The helm release name is allure-ee.

You need to use upgrade command exactly as it is stated below (allure-ee qameta/allure-ee cannot be altered) with no deviations except the namespace if you are going to use an alternative to the default one:

helm repo add qameta https://qameta.github.io/helm-charts
helm repo update
helm upgrade --install allure-ee qameta/allure-ee -f config.yaml 

This will output the list of the resources installed once the deployment finished. The deployment can take up to 10 minutes depending on your infrastructure.

Accessing the web interface of Allure TestOps

First system start could take some time, please wait up to 10 minutes before accessing the web-interface.

Prerequisites

To be able to access the web UI you need to have ingress controller installed. See an example here .

Accessing Allure TestOps UI

Allure is available at http://allure.local (URL is defined in your configuration - ingress).

Initial login

If you created the secret for initial admin password manually, you can use that to sign in as admin user.

If you haven’t specified the password, Chart will automatically create a random password for admin user.

This password can be extracted by using the following command:

kubectl get secret allure-ee -ojsonpath={.data.password} | base64 --decode ; echo

Super user (admin) log-in information

Allure TestOps requires a super user account to be created and kept in the system. This super user’s name and password are defined in the configuration file (see admin above) and it will be restored to the state described in the configuration to ensure you won’t lose the access to your Allure TestOps instance, this means if you disable this user in the UI, remove their roles and change the password, during the next start of Allure TestOps, it will recreate the user again with all set of available rights, with the password defined in the configuration file.

Please consider changing the admin’s password in the configuration before the deployment or remove it to have a new password each time system restarts.

If you omit provisioning of initial admin’s password, then default user admin will be created and admin’s strong password will be generated by the system, and then the generated password will be sent to the logs of uaa service.

Each time Allure TestOps is restarted, the super user account is restored to its initial state.

Provide the license for your Allure TestOps instance

Next thing you see will be the modal window to which you will need to provide the license you acquired.

if you see this write to support.qameta.io

After the correct license is provided, you’ll be able to start your work.

Updating Allure TestOps

Once your Allure TestOps Chart is installed, all the configuration changes and chart updates should be done using helm upgrade.

In the example below the changes to be applied are in a file named config.yaml.

helm repo update
helm upgrade --reuse-values allure-ee qameta/allure-ee -f config.yaml

Troubleshooting Allure TestOps

To get proper debug information you need to perform the following set of the commands:

Get the list of all the installed components

kubectl get all

Show logs for each Allure TestOps component

kubectl logs -f --tail=20 -l app=allure-ee-uaa
kubectl logs -f --tail=20 -l app=allure-ee-report
kubectl logs -f --tail=20 -l app=allure-ee-gateway

Continuous logging

kubectl logs -f -l app=allure-ee-uaa
kubectl logs -f -l app=allure-ee-report
kubectl logs -f -l app=allure-ee-gateway

Show logs for RabbitMQ and PostgreSQL

kubectl logs -f --tail=20 -l app=rabbitmq
kubectl logs -f --tail=20 -l app=postgresql

Uninstalling Allure TestOps

To uninstall Allure TestOps Chart, you need to execute the following command:

helm delete allure-ee

Altering the configuration

This section is intended to provide the recommendations for the integration of Allure TestOps with the existing end user’s infrastructure.

  • Standalone database (outside of a container)
  • Consul service
  • RabbitMQ service
  • Redis

General workflow with Helm chart

We use Helm chart to provide services configuration to k8s. Generally this works as follows:

  1. There is a big helm template containing all the information to configure an application to run in the k8s.
  2. There is values.yaml containing the settings specific for the deployment in end user’s infrastructure.
  3. During the deployment Helm takes the settings from values.yaml specified by the end user and applies these to the template replacing the default settings.
  4. When service’s template gets updated, i.e. any new parameter added to a service, this change will be populated to end user’s installation by Helm when updating the product .

Effect and limitations

  1. Qameta software is managing the Helm chart to deploy Allure TetstOps, and end users are advised to use standard Helm chart.
  2. In case an end user prefers other way to deploy Allure TestOps, this implies end user is capable of configuring such installation has enough knowledge and experience.
  3. It is the end user’s responsibility to track all the changes in the original Helm chart and populate the updates to their configuration.
  4. In case an end user prefers other way to deploy Allure TestOps, then the support of such solution will be limited.

Alteration of values.yaml

The file with user specific setting used in the following examples is named values.yaml. The name could be different. All the services which are used as external ones should be explicitly disabled in the values.yaml file (see an example below).

redis:
  enabled: false

rabbitmq:
  enabled: false

postgresql:
  enabled: false

The structure of values.yaml and variables

values.yaml is a YAML file. All the settings are added as environment variables to env: section.

There are two types of environment variables:

  1. open variables, these will be transferred to a POD in open format
  2. secret variables, these will be transferred to a POD as secrets.
 report:
  env:
    open:
      # database
      OPEN_VAR1: "value" # e.g. URL will go here
    secret:
      # database
      OPEN_VAR1: "value" #e.g. username and password for DB connection

Using standalone database

Standalone database is highly recommended for production environment as the performance of database running inside a container is very limited and will become a bottle neck with the growth of your test cases quantity.

Connecting the external database contains following steps:

  1. Creating the databases for services uaa and report using recommended Postgres DB.
  2. Migration of data from existing database to the standalone one. This step is to be skipped if you’re deploying Allure TestOps from scratch.
  3. Updating the settings to connect uaa and report services to standalone database.

Creating uaa and report databases

Below are the scripts you need to run to get uaa and report databases:

CREATE DATABASE uaa TEMPLATE template0 ENCODING 'utf8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8';
CREATE USER uaa with encrypted password 'uaa';
GRANT ALL PRIVILEGES ON database uaa to uaa;

CREATE DATABASE report TEMPLATE template0 ENCODING 'utf8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8';
CREATE USER report with encrypted password 'report';
GRANT ALL PRIVILEGES ON database report to report;

Data migration from existing database to the standalone database

This step is to be omitted if you’re deploying Allure TestOps from scratch.

The data migration to standalone database is described in FaQ section.

Updating values.yaml

If you have migration step, update of the values.yaml needs to be done in 2 steps:

  1. Update env variables for datasource, username and password to connect to the standalone database (see below). Update the config and check if the migration is successful. You must see all your tests and rets results and launches when connected to standalone database.
  2. Disable postgresql in values.yaml.

As said before, values.yaml needs to be updated — postgresql service is to be disabled. You need to provide following variables to configure the connection of each – uaa and report services tho their respective databases:

  • SPRING_DATASOURCE_URL - URL of the DBE serving your database. This is an open env variable.
  • SPRING_DATASOURCE_USERNAME - username to connect to a database. This is a secret env variable.
  • SPRING_DATASOURCE_PASSWORD - password to connect to a database

SPRING_DATASOURCE_URL has the following format:

#for uaa service
jdbc:postgresql://<database_url>:5432/uaa
#for report service
jdbc:postgresql://<database_url>:5432/report

where

  • <database_url> is the URL address of your external database engine to which your report or uaa service are to connect
  • uaa is the name of the database to which your uaa service needs to be connected
  • report is the name of the database to which your report service needs to be connected
  • 5432 is the port used by the application (DBE), it can be different, but the one used in the example is the default one. Consult with your DBA.

So the final values.yaml file for the standalone database connection will look like follows:

version: 3.181.5 # check release notes for most recent recommended version

imagePullSecret: docker-hub #label of your secret to pull the images

postgresql:
  enabled: false # service will be disabled and k8s won't start it

uaa:
  env:
    open:
      # database
      SPRING_DATASOURCE_URL: jdbc:postgresql://<database_url>:5432/uaa
    secret:
      SPRING_DATASOURCE_USERNAME: uaa 
      SPRING_DATASOURCE_PASSWORD: "password for uaa db"
<snip>
report:
  env:
    open:
      # database
      SPRING_DATASOURCE_URL: jdbc:postgresql://<database_url>:5432/report
    secret:
      SPRING_DATASOURCE_USERNAME: report
      SPRING_DATASOURCE_PASSWORD: "password for report db"

When changes have been made, you need to update the installaiton

Using S3 storage for report service

S3 storage type is more reliable in terms of processing of the collisions when saving and deleting the data during the batch operations with files so it is recommended at least using min.io instead of the direct usage of a file system storages.

If you’re using the deployment via Kubernetes the connection to S3 storage is being performed by adding of environment variables to report container as follows (all strings must be in double quotes):

List of parameters to be added to the report service

  • ALLURE_BLOBSTORAGE_TYPE - should be S3 for S3 bucket or for min.io
  • ALLURE_BLOBSTORAGE_S3_ENDPOINT - the URL of your S3 management endpoint
  • ALLURE_BLOBSTORAGE_S3_BUCKET - name of your S3 bucket
  • ALLURE_BLOBSTORAGE_S3_REGION - the region of S3 bucket (see the S3 details during the creation)
  • ALLURE_BLOBSTORAGE_S3_ACCESSKEY - the access key ID created to access the S3 bucket
  • ALLURE_BLOBSTORAGE_S3_SECRETKEY - the secret key linked to the access key
  • ALLURE_BLOBSTORAGE_S3_PATHSTYLEACCESS - parameter setting the link styles, should be true
Example for AWS’ S3
version: 3.181.5 # check release notes for most recent recommended version
imagePullSecret: docker-hub #reference to the secret

report:
#<snip>
  env:
    open:
      SPRING_PROFILES_ACTIVE: kubernetes
      #storage
      ALLURE_BLOBSTORAGE_TYPE: "S3"
      ALLURE_BLOBSTORAGE_S3_ENDPOINT: "s3.amazonaws.com" #leave as it is
      ALLURE_BLOBSTORAGE_S3_REGION: "us-east-1" # region, update accordingly your real region
      ALLURE_BLOBSTORAGE_S3_BUCKET: "<your-bucket-name>" # replace string in <> with your bucket's name 
    secret:
      ALLURE_BLOBSTORAGE_S3_ACCESSKEY: "<ACCESS-KEY>" # replace string in <> with your Access key ID 
      ALLURE_BLOBSTORAGE_S3_SECRETKEY: "<SecretKey>" # replace string in <> with your Secret access key 
  #<snip>    
Access rights settings for AWS S3 bucket

For the S3 integration to work smoothly with Allure TestOps following access rights to be configured for S3 bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "S3assets",
            "Effect": "Allow",
            "Action": [
                "s3:PutObjectAcl",
                "s3:PutObject",
                "s3:ListMultipartUploadParts",
                "s3:ListBucketMultipartUploads",
                "s3:ListBucket",
                "s3:GetObjectAcl",
                "s3:GetObject",
                "s3:GetBucketLocation",
                "s3:GetBucketAcl",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::<ALLURE_S3_BUCKET_NAME>/*",
                "arn:aws:s3:::<ALLURE_S3_BUCKET_NAME>"]
        }
    ]
}
Example for Google’s S3
version: 3.181.5 # check release notes for most recent recommended version
imagePullSecret: docker-hub #reference to the secret

report:
#<snip>
  env:
    open:
      SPRING_PROFILES_ACTIVE: kubernetes
      #storage
      ALLURE_BLOBSTORAGE_TYPE: "S3"
      ALLURE_BLOBSTORAGE_S3_ENDPOINT: "https://storage.googleapis.com" #leave as it is
      ALLURE_BLOBSTORAGE_S3_REGION: "europe-west3" # region, update accordingly your real region
      ALLURE_BLOBSTORAGE_S3_BUCKET: "<your-bucket-name>" # replace string in <> with your bucket's name 
    secret:
      ALLURE_BLOBSTORAGE_S3_ACCESSKEY: "<ACCESS-KEY>" # replace string in <> with your Access key ID 
      ALLURE_BLOBSTORAGE_S3_SECRETKEY: "<SecretKey>" # replace string in <> with your Secret access key 

  #<snip>    
For Google S3 you need to use fine-granted access control and settings for public access Subject to object ACLs, this is needed for interoperability with AWS SDK, otherwise files in S3 bucket won’t be fully accessible by Allure TestOps and report service will fail to start.

Connecting external Redis service

Connecting the external redis installed in your infrastructure require the setting of the following variables for Redis service in the env section of the gateway service.

  • SPRING_SESSION_STORE_TYPE: REDIS should be always included in the configuration as Redis is to be used in any case.

  • SPRING_REDIS_HOST - is the URL for Redis host

  • SPRING_REDIS_PORT - is the port of Redis application

  • SPRING_REDIS_SSL - the mark if SSL to be used.

  • SPRING_REDIS_PASSWORD - password to use when connecting to Redis.

  • ALLURE_REDIS_NAMESPACE - reference to the Redis’ namespace designated to Allure TestOps.

Updating values.yaml for external Redis service

version: 3.181.5 # check release notes for most recent recommended version
imagePullSecret: docker-hub #reference to the secret

gateway:
#<snip>
  env:
    open:
      SPRING_SESSION_STORE_TYPE: REDIS
      SPRING_REDIS_HOST: "https://<redis-host>"
      SPRING_REDIS_PORT: "<port-id>" # only if differs form the default
      SPRING_REDIS_SSL: "true/false"

    secret:
      SPRING_REDIS_PASSWORD: "<password for Allure TestOps>"
      ALLURE_REDIS_NAMESPACE: "<namespace for Allure TestOps>"
  #<snip>    

Connecting external RabbitMQ service

Connecting the external RabbitMQ installed in your infrastructure require the setting of the following variables for RabbitMQ service in the env section of the report service.

  • SPRING_RABBITMQ_HOST - the URL of the host where RabbitMQ application is running
  • SPRING_RABBITMQ_PORT - port of RabbitMQ application
  • SPRING_RABBITMQ_USERNAME - the username set to use with Allure TestOps
  • SPRING_RABBITMQ_PASSWORD - the password set to use with Allure TestOps
  • SPRING_RABBITMQ_SSL_ENABLED - mark if SSL to be used
  • SPRING_RABBITMQ_VIRTUAL_HOST - the virtual host reference created for the usage with Allure TestOps

Updating values.yaml for external RabbitMQ service

version: 3.181.5 # check release notes for most recent recommended version
imagePullSecret: docker-hub #reference to the secret

report:
#<snip>
  env:
    open:
      #RabbitMQ
      SPRING_RABBITMQ_HOST: "https://<rabbitmq-host>"
      SPRING_RABBITMQ_PORT: "<port-id>"
      SPRING_RABBITMQ_SSL_ENABLED: "true/false"

    secret:
      #RabbitMQ
      SPRING_RABBITMQ_USERNAME: "<username for Allure TestOps>"
      SPRING_RABBITMQ_PASSWORD: "<password for Allure TestOps>"
      SPRING_RABBITMQ_VIRTUAL_HOST: "<virtual host for Allure TestOps>"
  #<snip>    

Preparation for the production deployment

Before putting your system to the production, please refer to the recommendations here.

If you have any doubts, please consult our Support

Back to installation