Install Allure TestOps in Kubernetes

This section describes how to install and run Allure TestOps version 5 in a Kubernetes cluster using Qameta Software managed Helm Chart.

For reference, we kept the outdated installation manual for version 4.

Because of its scalability, a Kubernetes cluster is the recommended way to run Allure TestOps in a production environment with higher workload.

The support of the authentication using OpenID is available starting from Allure TestOps 5.6.4. OpenID, SAML2, LDAP are available for on-prem deployment for the time being. The support of external IdP providers in SaaS is planned in Q4 2024.

Migration from version 4

Allure TestOps version 5 cannot be deployed above the version 4. The upgrade process requires the data migration and switching to the new Helm Chart.

The upgrade is possible only from the release 4.26.5. Using older releases for the upgrade process will brick the system and will lead to the data loss.

The upgrade to version 5 requires the following steps:

1. Create the new values.yaml file based on the template for the newly developed Helm Chart.
2. Stop the existing instance.
3. Merge the information from the UAA service database to the report service database.
4. Start the deployment using the new Helm Chart.

For more detailed information on upgrading your version of Allure TestOps to 5.x, follow the migration guide.

Prior to the merging of the databases, we highly recommend moving the databases to the standalone server and performing all migrations on the standalone server if your current deployment uses the databases in pods with PVC.

Please also consider creating backups for all databases prior to merging process.

Preparation steps

1. Install kubectl.

   See the installation guide on the Kubernetes website.

2. Install Helm.

   See the installation guide on the Helm website.

3. Deploy additional services required for Allure TestOps to start:

   • PostgreSQL server version 15 or higher (see the database creation instructions),
   • Amazon S3, Google Cloud Storage, MinIO or another S3-compatible storage solution,
   • RabbitMQ message queue service,
   • Redis service.

   See Storage requirements for more details.

   Installation without dedicated external services is also possible.

   The demo option is intended for evaluation purposes only and is not supported for production deployment. Maintaining and upgrading such a configuration can involve considerable downtimes or data loss if performed by non-skilled personnel. We won't be able to assist with the troubleshooting and service restoration.

4. Get a license for Allure TestOps.

   To receive the license key, fill out the form for a trial version or contact us at [email protected] for a commercial license.

   When you receive the license key, you will also get a username and a password for accessing Qameta's Docker repository.

5. Get detailed information about configuring authentication within your organization.

   The data required for configuration depends on the authentication method you choose. Possible options:

   • Standard authentication by a username and a password, implemented by Allure TestOps itself.

     This option uses an SMTP server to send invitations to new users and does not require additional services.

   • Authentication via an external identity provider.

     The details required for this may vary depending on what type of identity provider you use. Usually, you will need to specify an external URL and a secret token generated by the identity provider.

   See Configuration → Authentication for more details.

Downloading and editing configuration file

1. Check the latest TGZ archive for the Helm Chart release.

2. Download the latest TGZ archive for the Helm Chart release:

   • When working on a Unix-like machine, use wget command to download the archive:

     wget https://dl.qameta.io/artifactory/helm/testops/testops-<latest-release-number>.tgz
     

     where instead of <latest-release-number> use the version number of the latest TGZ archive, for example, 5.10.2.

   • Alternatively, open this page and download the archive manually.

3. Extract the values.yaml file from the downloaded archive manually or using the command:

   tar -xzvf testops-<latest-release-number>.tgz  testops/values.yaml
   

   where instead of <latest-release-number> use the version number of the downloaded archive.

   The file will be saved to the testops folder.

4. Open the values.yaml file in a text editor.

   The file defines parameters that will affect behavior of your Allure TestOps instance. For the most typical setup, just do the following:

   1. Choose the Allure TestOps release.
   2. Provide the correct registry and repository information.
   3. Specify Helm credentials.
   4. Set the Allure TestOps URL.
   5. Configure load balancing.
   6. Set the first admin email.
   7. Configure authentication.
   8. Set up storage services.

Choose Allure TestOps version

In the version parameter, specify the version of Allure TestOps you want to use. For example:

version: 5.10.2

The release of the application is not the same as the release of the Helm Chart. Generally, the Helm Chart release defines the minimum version of the application you can use with the chart. For example, if the chart is version 5.7.0 and there is no newer chart available, you can use it with Allure TestOps 5.7.0 and later. However, if a newer chart release, such as 5.8.1, is available, you cannot use Helm Chart 5.7.0 with Allure TestOps releases 5.8.1 and beyond.

These installation instructions are relevant for any Allure TestOps version starting with version 5.4.0, but we recommend to always use the latest version available. Check out the Release notes section to see available versions.

Images registry, repository, image name

To pull the Allure TestOps image for the deployment, you need to specify the registry and repository to pull the image from. If you are using an internal registry for the images, you need to supply your registry's details. Otherwise, please use the Qameta Software registry in Docker Hub. The settings in values.yaml need to be filled as follows in this case:

image:
  registry: docker.io
  repository: allure
  imageName: testops
  pullPolicy: IfNotPresent
  authRequired:
    enabled: true
    username: username-you-received-from-sales
    password: password-you-received-from-sales
    credsKubeSecretName: secret-name-here

Specify Helm credentials

Specify the credentials Helm will use to download container images for Allure TestOps:

• image.authRequired.username — the username received from the sales team,
• image.authRequired.password — the password received from the sales team.

Set TestOps URL

The URL of the installation is determined by three parameters:

• instanceFqdn — the hostname that users will use to access Allure TestOps;
• port — the port that users will use to access Allure TestOps;
• network.tls.enabled — if true, the Kubernetes Ingress controller will use HTTPS protocol instead of HTTP.

In the example below, the parameters are set so that the Allure TestOps instance would be available at “http://testops.example.com/” (using port 80, which is the default port for HTTP).

instanceFqdn: testops.example.com
port: 8080
# ...
network:
  tls:
    enabled: false
    # ...
  # ...
# ...

See Network configuration for more details.

Configure load balancing

There are multiple ways to manage traffic in and out from a Kubernetes cluster. Selecting the best one depends on your specific needs. However, Allure TestOps comes with two options for common use cases:

• To use an Ingress service, set the network.ingress.enabled parameter to true.
• To use an Istio service, set the network.istio.enabled parameter to true.

Set first admin email

Starting with the version 5.4.0, valid SMTP credentials are required in any configuration. Allure TestOps will use them to send an invitation email to the administrator, regardless of the main authentication method. The administrator's email address must be specified in the email parameter.

Below is an example of configuration parameters for a Gmail account. Note that Google requires to generate dedicated app passwords for SMTP. Other mail services may have similar requirements.

email: [email protected]

# ...

smtp:
  enabled: true
  host: smtp.gmail.com
  port: 465
  authEnabled: true
  from: [email protected]
  username: [email protected]
  password: Ohku6Zo9gee5aen0
  startTLSEnabled: true
  startTLSRequired: true
  sslEnabled: true
  sslTrust: smtp.gmail.com

Configure authentication

A necessary step when deploying Allure TestOps is to configure how users (other than the very first administrator) will be registered and logged in. See Configuration → Authentication for more details.

Below are some configuration examples for various identity providers.

auth:
  primary: system
  defaultRole: ROLE_GUEST

# this is just an example which won't work if applied without any changes.
auth:
  primary: ldap
  defaultRole: ROLE_GUEST
  ldap:
    enabled: false
    auth:
      user: user
      pass: SecretPaSSw0rd
    referral: follow
    url: ldap://ldap.example.com:389
    usernamesToLowercase: false
    passwordAttribute: userPassword
    user:
      dnPatterns: ""
      searchBase: dc=example,dc=com
      searchFilter: (&((objectClass=Person))(uid={0}))
    syncRoles: false
    uidAttribute: uid
    group:
      searchBase: ou=qa,ou=Security Groups,dc=example,dc=com
      searchFilter: (&(objectClass=Person)(uid={1}))
      roleAttribute: cn
    userGroupName: allure_users
    adminGroupName: allure_admins

# this is just an example which won't work if applied without any changes.
auth:
  primary: saml2
  saml:
    enabled: false
    id: "testsaml"
    entityId: https://<testops>/api/login/saml2/authenticate/{registrationId}
    acsUrl: https://<your-domain>/api/login/saml2/sso/{registrationId}
    identityProviderMetadataUri: https:///path/to/metadata.xml
    syncRoles: false
    firstNameAttribute: firstNameAttribute
    lastNameAttribute: lastNameAttribute
    emailAttribute: emailAttribute
    groups:
      groupRoleAttribute: groupRoleAttribute
      roleUserGroups: roleUserGroups
      roleAdminGroups: roleAdminGroups

auth:
  primary: openid
  defaultRole: ROLE_GUEST
  openid:
    enabled: true
    clientName: testops
    clientId: testops
    providerName: keycloak
    clientSecret: your_client_secret
    redirectUri: https://<your-domain>/login/oauth2/code/<add_providerName_here>
    scope: openid,email,profile
    authorizationGrantType: authorization_code
    authorizationUri: http://keycloak/realms/<you_realm>/protocol/openid-connect/auth
    userinfoUri: http://keycloak/realms/<you_realm>/protocol/openid-connect/userinfo
    issuerUri: http://keycloak/realms/<you_realm>
    usernameAttribute: preferred_username
    defaultRole: ROLE_GUEST
    syncRoles: false
    groupRoleAttribute: groupRoleAttribute
    roleUserGroups: roleUserGroups
    roleAdminGroups: roleAdminGroups

Configure storage services

Specify the details for connecting to the storage services required for Allure TestOps.

• Connection to the Redis storage server

  • redis.host — server hostname.
  • redis.port — server port.
  • redis.auth.password — password for connecting to the storage.

• Connection to the PostgreSQL databases

  • postgresql.enabled — if true, Kubernetes will create a new database inside the cluster instead of using the connection parameters. Not recommended for production environment.
  • datasources.mainDatasource.dbHost — database server hostname.
  • datasources.mainDatasource.dbPort — database server port.
  • datasources.mainDatasource.dbName — database name.
  • datasources.mainDatasource.username — username for connecting to the database.
  • datasources.mainDatasource.password — password for connecting to the database.

  See also Separating connection pools for how to configure database connection pools for different types of operations.

• Connection to the RabbitMQ server

  • rabbitmq.deployByChart — if true, Kubernetes will create a new RabbitMQ server inside the cluster instead of using the connection parameters. Not recommended for production environment.
  • rabbitmq.hosts — comma-separated list of server locations, each one in the AMQP format.
  • rabbitmq.auth.username — username for connecting to the message queue.
  • rabbitmq.auth.password — password for connecting to the message queue.

• Connection to the S3-compatible storage server

  • minio.enabled — if true, Kubernetes will create a new S3 server inside the cluster instead of using the connection parameters. Not recommended for production environment.
  • storage.s3.endpoint — server URL.
  • storage.s3.bucket — S3 bucket name.
  • storage.s3.region — S3 region name.
  • storage.s3.accessKey — access key for connecting to the bucket.
  • storage.s3.secretKey — secret key for connecting to the bucket.

Some additional settings can be modified to fine-tune the performance or support non-standard configurations of the storage services. Check the comments in the values.yaml file and consult the Qameta support team if necessary.

Deploying Allure TestOps

Installing the chart

Once you have prepared the values.yaml file, you can install the Helm Chart and run Allure TestOps.

1. Open a terminal application.

2. Connect Kubernetes to the Qameta's Helm repository.

   helm repo add qameta https://dl.qameta.io/artifactory/helm
   

3. Refresh the repository data.

   helm repo update
   

4. Navigate to the directory containing the values.yaml file, for example:

   cd ~/allure-testops
   

Deploying the application

Examples do not contain the commands for the creation of a separate name space for Allure TestOps, please consider creating dedicated name space for Allure TestOps and use the reference to the created name space in all commands.

1. Deploy Allure TestOps with the Helm Chart and configured values.yaml file.

   helm install testops qameta/testops \ 
         --namespace testops \
         --create-namespace \
         --values values.yaml \
         --set probes.enabled=false
   

   At this point, you will need to wait for two time-consuming operation to finish:

   1. Before any services can start, Kubernetes needs to download the necessary images from the Qameta Software's images registry.
   2. During the startup, the Allure TestOps services will need to initialize its database.

   The --set argument in the command above disables the readiness and liveness probes for the services, which prevents Kubernetes from killing them due to timeouts. This measure will also be needed whenever you upgrade Allure TestOps.

2. Wait for the testops-testops deployment to become ready.

   Use kubectl get all to view the status of the deployments and pods. When each pod becomes Running, the initialization of Allure TestOps is completed.

3. Once the initialization is completed, enable the readiness and liveness probes that were disabled during installation. To do so, upgrade the chart without the --set argument.

   helm upgrade testops qameta/testops --namespace testops --values values.yaml
   

First start

When you start your Allure TestOps installation for the first time, you need to activate it by providing the license key. You should have received the license key in an email from the Qameta sales team.

1. Soon after startup, Allure TestOps will send an invitation email to the address that you specified in email. Check the inbox and click the Accept invite link in the email.

   If you get an “Unable to connect” or “This site can't be reached” error in the browser, it may indicate that some services are still initializing. Normally, the initialization takes no more than a couple of minutes.

   If the web browser still cannot open the page after five minutes, see the Troubleshooting section.

2. In the invitation accept form, fill in the administrator's full name and the credentials that will be used in the login form: Username and Password. The Username must be unique within the Allure TestOps instance.

   Click Continue.

3. In the activation form, enter the license key that you got from Qameta sales team. Make sure to paste the key without any extra spaces or other characters.

   Click Activate license.

Upgrading

Before you begin, make sure to read the release notes for the Allure TestOps version you are upgrading to. The release notes may contain crucial information on how to adapt your configuration files or migrate your data to the new version of Allure TestOps. In case you are skipping one or more versions, read the release notes for the skipped versions, too.

1. Open a terminal application.

2. Refresh the repository data.

   helm repo update
   

3. Navigate to the directory containing the values.yaml file, for example:

   cd ~/allure-testops
   

4. Perform the upgrade:

   helm upgrade testops qameta/testops --namespace testops --values values.yaml --set probes.enabled=false
   

   At this point, you will need to wait for two time-consuming operations to finish:

   1. Before any services can start, Kubernetes needs to download the necessary images from the internet.
   2. During the startup, the Allure TestOps service will need to re-initialize its databases.

   The --set argument in the command above disables the readiness and liveness probes for the services, which prevents Kubernetes from killing them due to timeouts.

5. Wait for the testops-testops deployment to become ready.

   Use kubectl get all to view the status of the deployments and pods. When each pod becomes Running, the re-initialization of Allure TestOps is completed.

6. Once the re-initialization is completed, enable the readiness and liveness probes that were disabled during installation. To do so, upgrade the chart once again, this time without the --set argument.

   helm upgrade testops qameta/testops --namespace testops --values values.yaml
   

Uninstalling

To uninstall the Allure TestOps instance, delete the Kubernetes namespace that you used for installing it:

kubectl delete namespace testops

Troubleshooting

If you run into any issues, please check our FAQ page or Troubleshooting page or consult our support team.

Also, in case of any troubles, do not immediately bring Allure TestOps down and up. We need the log files of each component to understand what has happened.

If you encounter an issue not covered in the FAQ, feel free to create a support ticket using your corporate email address. Please include the following information in the ticket:

1. Description of the problem you are experiencing.
2. Versions of Allure TestOps and all extra components you are having problems with (allurectl, integration plugins, etc.).
3. Logs from your system as attachments to the ticket (see the details below). Please only send us the logs described below and only as an attachment.

To get proper debug information you need to perform the following set of the commands:

Get the list of all the installed components

kubectl get all --namespace testops

Save logs

kubectl logs --namespace testops deployments/testops-testops > testops-log.txt

Continuous logging

kubectl logs --namespace testops -f deployments/testops-testops