Deploying Allure TestOps using Kubernetes

This page describes the deployment of Allure TestOps in Kubernetes for 4+ release.

Kubernetes is the best and the only recommended way to deploy your Allure TestOps instance in the production.

Officially supported deployment in the Kubernetes is the deployment using our official Helm chart.

Official Helm chart includes all needed dependencies and configuration for the integration with the existing infrastructure.

Prerequisites

You need to have a Kubernetes cluster.
You need to have helm deployed.
You need to have the username and the password (could be referenced as token as well) to pull the images from our private registry. These are usually provided with the trial or commercial licence (see the next item).
You need to have a trial or commercial licence to be able to start Allure TestOps.
- Trial licence could be obtained by filling a free trial request form on our web site.
- Commercial licence could be purchased with help of our sales team

Resources requirements

To ensure the deployed system is able to perform under a production workload, we recommend using following resources for Allure TestOps related services:

You need to obtain the license key and image pull token.
- Please contact our sales team to get these.
You need to have helm and kubectl installed on your machine.

gateway:
 <snip>
  resources:
    requests:
      memory: 1Gi
      cpu: 1
    limits:
      memory: 2Gi
      cpu: 2
<snip>
uaa:
<snip>
resources:
    requests:
      memory: 1Gi
      cpu: 500m
    limits:
      memory: 2Gi
      cpu: 1
<snip>
report:
<snip>
resources:
    requests:
      memory: 3Gi
      cpu: 1
    limits:
      memory: 4Gi
      cpu: 4
<snip>

However, final resources settings are the subject for the adjustment based on the production workload. We recommend using the resources described above by default and never going lower than defined requests.

Configuration

The configuration of your deployment using our Helm chart is done using values.yaml file which should contain the parameters specific for your deployment.

When working with the deployment using our Helm chart please consider the following:

You don't need to download the whole chart.
You must not alter chart's files, this part is managed by Qameta Software only.
You need to create values.yaml containing the settings specific for your deployment.
Helm will take your settings and apply these to the chart before deploying Allure TestOps.

Prepare configuration file

Please take template value.yaml file prepared by our DevOps as a base for your deployment.

This file contains all the settings you need to configure the deployment.
This file contains the comments on the settings and end user might need to change.

In the following sections we'll describe the main configuration parameters you must updating before the deployment.

version

Sample:

version: 4.1.0

Meaning: the release of Allure TestOps to be deployed.

Recommendation: Update the version of Allure TestOps with the latest recommended one. Do not deploy other releases except the cases when it is explicitly recommended by our support team.

admin credentials

Sample:

username: admin
password: admin

Meaning: name of admin user, password of admin user

Allure TestOps has an admin account which is to be created and kept in the system.

This user's name (admin by default) and password are defined in the configuration file and it will be restored to the state described in the configuration file to ensure you won't lose the access to your Allure TestOps instance no matter what. You cannot you disable this user in the UI, you cannot remove their roles and change the password via UI. During the next start of Allure TestOps, it will recreate the admin user again with all set of all available rights, and with the password defined in the configuration file.

If you omit provisioning of initial admin's password, then default user admin will be created and admin's strong password will be generated by the system, and then the generated password will be sent to the logs of uaa service.

Each time Allure TestOps is restarted, the super user account is restored to its initial state.

Recommendation: Change the admin's password in the configuration before the deployment.

host

Sample:

host: alure.host.name

Meaning: FQDN of Allure TestOps instance which will be used by the end users to access the UI.

Recommendation: Must be updated.

network

Meaning: this is actually a whole section you need to alter accordingly to your network settings. Check the sample file's comments.

service postgresql

Sample:

enabled: false
  connectionTimeout: 60000
  postgresqlUsername: allure
  postgresqlPassword: allure
  external:
    endpoint: db.example.com
    port: 5432
    sslMode: prefer
    uaaDbName: uaa
    uaaUsername: uaa
    uaaPassword: secret
    reportDbName: report
    reportUsername: report
    reportPassword: secret

Meaning:

enabled: true This will deploy PostgreSQL database in scope of Allure TestOps deployment. However, this is not recommended for the production system, as the performance of PostgreSQL is limited when running in a container and is not recommended. Set it to false, and use dedicated PostgreSQL server (release should be no less than 14).

enabled: false will prevent deploying PostgreSQL database in scope of Allure TestOps deployment.

Recommendation:

On the dedicated PostgreSQL server you need to create 2 databases: 1) for UAA service 2) for report service. These services cannot live in the same database.

creating uaa database

CREATE DATABASE uaa TEMPLATE template0 ENCODING 'utf8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8';
CREATE USER uaa with encrypted password '[email protected]$w0rd';
GRANT ALL PRIVILEGES ON database uaa to uaa;

Please consider updating the password

creating report database

CREATE DATABASE report TEMPLATE template0 ENCODING 'utf8' LC_COLLATE = 'en_US.UTF-8' LC_CTYPE = 'en_US.UTF-8';
CREATE USER report with encrypted password '[email protected]$w0rd';
GRANT ALL PRIVILEGES ON database report to report;

Please consider updating the password

settings for the files storage

Allure TestOps processes and stores a lot of files related to the test results. There are several options to store these files:

Not recommended for the production:

file system (LOCAL) as PVC of report service

Recommended for production:

S3 - the best option is the AWS' solution, min.io as part of Allure TestOps deployment is a good option as well
NFS - is better than PVC in report's pod but still S3 is a better option

Sample:

fs:
  # Supported: S3, LOCAL, NFS
  type: "S3"
  external: false
  pathstyle: true
  migration:
    enabled: false
    directory: /opt/allure/report/storage
  s3:
    endpoint: https://s3.amazonaws.com
    bucket: allure-testops
    region: us-east-1
    accessKey: foo
    secretKey: bar
  mountPoint: /storage
  pvc:
    claimName: ""

Meaning:

type could be S3, LOCAL or NFS
external - set to true is you aren't using "built-in" S3 like minio or LOCAL file system
migration is used if you are going to migrate the artifacts from report's pod PVS to an S3 solution, otherwise should be always false
s3 - settings for the external S3 solution. You need to provide the settings for API endpoint, region, bucket name and secrets used to access your S3

Recommendation: consider using S3 of AWS or similar cloud services provider. Acceptable alternative is dedicated min.io with SSD.

'built-in' min.io settings

This section is required if you are using min.io deployed in a pod alongside with Allure TestOps deployment. This will work better than the FS used as PVC in the report's pod.

Meaning:

enabled: true will deploy min.io in a pod
gateway if enabled: true will deploy min.io in the gateway mode
- this mode will allow to perform some operations like test results processing on min.io side and then storing the results in underlying S3 solution,so you'll save traffic on processing operation.

Recommendation:

Use minio as gateway if your payment depends on the traffic, as the caching will allow you processing of the results avoiding sending these to S3, and finally storing only the final data on S3.

allure section

This section describes the general settings of Allure TestOps you need to adjust before deploying the solution.

Meaning:

timeZone: "Europe/London" - Make this the same time zone, where your server resides.
sessionLifespan: 57600 - sets the max duration for a session. E.g., if your test run is being executed longer than 16 hours (57600 seconds, yes, there are such test runs!), you need to increase this parameter.
secureCookies: "true" - sets the cookies as secure for HTTPS usage. Should be set to "false" if you use http to access the UI, otherwise you won't be able to log in to the system
- If set to false the integrations with Jira (Allure TestOps plug-in) won't work.
- If you are going to access UI via https, i.e. you will hide Allure TestOps behind a reverse proxy with https, then this should be set to true
auth > primary defined which auth way (in case you are using an external IAM provider) should be used by default.
- options are system, ldap, oauth2, oidc (openID)
- if you are not using certain provider, just leave the parameter enabled set to false

Deployment

Get Allure TestOps up and running

Once you have done all the configuration in your values.yaml file, you can start the deployment of Allure TestOps.

Add Qameta's Helm charts repo

helm repo add qameta https://dl.qameta.io/artifactory/helm

Update Helm repositories information

helm repo update

Deploy Allure TestOps

helm upgrade --install allure-testops qameta/allure-testops -f values.yaml

Accessing the web interface of Allure TestOps

First system start could take some time, please wait up to 10 minutes before accessing the web-interface.

Prerequisites

To be able to access the web UI you need to have ingress controller like nginx or istio deployed.

Accessing Allure TestOps UI

Allure will be available at the URL defined in host parameter of your values.yaml file, e.g. for the instance to be accessible at http://allure.local your values file should contain the following:

## your-domain.tld
## the real DNS name (FQDN) of the allure testops instance
host: alure.local

If you created the secret for initial admin password manually, you can use that to sign in as admin user.

If you haven't specified the password, Chart will automatically create a random password for admin user.

Provide the license for your Allure TestOps instance

Next thing you see will be the modal window to which you will need to provide the license you acquired.

if you see this write to help.qameta.io

After the correct license is provided, you'll be able to start your work.

Updating Allure TestOps deployment

Once your Allure TestOps Chart is deployed, all the configuration changes and chart updates should be done using helm upgrade.

In the example below the changes to be applied are in a file named values.yaml.

helm repo update
helm upgrade --install allure-testops qameta/allure-testops -f values.yaml

Release upgrade

To upgrade the release of Allure TestOps, you need to update one line in your values.yaml; the ID of the most recent recommended release can be found on the release notes page.

version: 4.1.0

then same as for the updates:

helm repo update
helm upgrade --install allure-testops qameta/allure-testops -f values.yaml

Troubleshooting Allure TestOps

To get proper debug information you need to perform the following set of the commands:

Get the list of all the installed components

kubectl get all

Save logs for each Allure TestOps component

kubectl logs -l app=allure-testops-uaa > uaa-log.txt
kubectl logs -l app=allure-testops-report > report-log.txt
kubectl logs -l app=allure-testops-gateway > gateway-log.txt

Continuous logging

kubectl logs -f -l app=allure-testops-uaa
kubectl logs -f -l app=allure-testops-report
kubectl logs -f -l app=allure-testops-gateway

Uninstalling Allure TestOps

To uninstall Allure TestOps Chart, you need to execute the following command:

helm delete allure-testops

Altering the configuration

This section is intended to provide the recommendations for the integration of Allure TestOps with the existing end user's infrastructure.

Standalone database (outside of a container)
Consul service
RabbitMQ service
Redis

General workflow with Helm chart

We use Helm chart to provide services configuration to k8s. Generally this works as follows:

There is a big helm template containing all the information to configure an application to run in the k8s.
There is values.yaml containing the settings specific for the deployment in end user's infrastructure.
During the deployment Helm takes the settings from values.yaml specified by the end user and applies these to the template replacing the default settings.
When service's template gets updated, i.e. any new parameter added to a service, this change will be populated to end user's installation by Helm when updating the product the deployed instance.

Effect and limitations

Qameta software is managing the Helm chart to deploy Allure TestOps, and end users are advised to use standard Helm chart.
In case an end user prefers other way to deploy Allure TestOps, this implies end user is capable of configuring such installation has enough knowledge and experience.
It is the end user's responsibility to track all the changes in the original Helm chart and populate the updates to their configuration.
In case an end user prefers other way to deploy Allure TestOps, then the support of such solution will be limited.

Alteration of values.yaml

The file with user specific setting used in the following examples is named values.yaml. All the services which are used as external ones should be explicitly disabled in the values.yaml file (see an example below).

redis:
  enabled: false

rabbitmq:
  enabled: false

postgresql:
  enabled: false

If you set a service in values.yaml to enabled: false then you need to specify the integration parameters for the service/application in your existing infrastructure.

Example:

In the example below, we are going to use RabbitMQ service running as a part of existing infrastructure and won't use it in scope of Allure TestOps deployment:

rabbitmq:
  enabled: false
  host: mq.rabbit.internal
  auth:
    erlangCookie: <takeFromDeployedRabbitMQ>
    username: allure
    password: allure
  resources: {}

The rest of the services will have same approach for the integration.

Preparation for the production deployment

Before putting your system to the production, please refer to the recommendations here.

If you have any doubts, please consult our Support

Back to installation

Deploying Allure TestOps using Kubernetes

Prerequisites

Resources requirements

Configuration

Prepare configuration file

version

admin credentials

host

network

service postgresql

creating uaa database

creating report database

settings for the files storage

'built-in' min.io settings

allure section

Deployment

Get Allure TestOps up and running

Add Qameta's Helm charts repo

Update Helm repositories information

Deploy Allure TestOps

Accessing the web interface of Allure TestOps

Prerequisites

Accessing Allure TestOps UI

Initial login

Provide the license for your Allure TestOps instance

Updating Allure TestOps deployment

Release upgrade

Troubleshooting Allure TestOps

Get the list of all the installed components

Save logs for each Allure TestOps component

Continuous logging

Uninstalling Allure TestOps

Altering the configuration

General workflow with Helm chart

Effect and limitations

Alteration of values.yaml

Preparation for the production deployment