TestOps logo
Try Testops

Frequently asked questions

This document contains the collection of frequently asked questions regarding the day-to-day usage of Allure TestOps

Release upgrades

  • Q: How do I upgrade my Allure TestOps?
  • A: Here is what you do
    1. Check what is the latest release of Allure TestOps in Release notes page. Upgrade only to a version mentioned there. Use a different version only if explicitly advised by our support.
    2. Check if there is any special upgrade procedure available for your release. All special upgrade procedures/migrations are described here.
    3. Either follow the instructions for your deployment type's update section or follow the special upgrade procedure.

Tests metadata, labels, custom fields, environment

Error 503

This error says that there is some service that cannot be accessed. Reasons could vary and this error requires some troubleshooting and looking into logs.

RabbitMQ deprecated environments

  • Q: Allure TestOps UI shows 503 report.

    • the log of report service contains following string: org.springframework.amqp.AmqpConnectException: org.springframework.amqp.AmqpIOException: java.net.UnknownHostException: report-mq1

    • The log of rabbitmq service (report-mq) contains strings like follows and it does not start:

      error: RABBITMQ_DEFAULT_PASS is set but deprecated
error: RABBITMQ_DEFAULT_USER is set but deprecated
error: RABBITMQ_DEFAULT_VHOST is set but deprecated
error: deprecated environment variables detected

  • A:: Check your docker-compose configuration file - docker-compose.yaml. There were some changes with the naming of environment variables and you need to update the part relates to the environment:

    # As is
RABBITMQ_DEFAULT_VHOST: ${REPORT_RABBITMQ_VHOST}
RABBITMQ_DEFAULT_USER: ${REPORT_RABBITMQ_USERNAME}
RABBITMQ_DEFAULT_PASS: ${REPORT_RABBITMQ_PASSWORD}

# To be
DEFAULT_VHOST: ${REPORT_RABBITMQ_VHOST}
DEFAULT_USER: ${REPORT_RABBITMQ_USERNAME}
DEFAULT_PASS: ${REPORT_RABBITMQ_PASSWORD}

Test results

Missing test results, duplicated test results

Missing test results in a launch of Allure TestOps vs. your expectations, inconsistency between the Overview of a launch and launch's content, etc. Unexpected number of the test results when making rerun and retries, tests permanently in progress.

There is dedicated page for missing test results related questions.

Resolution

When a test is considered as resolved?

  • Q: when a test is considered as resolved?
  • A: Test is marked by Allure TestOps as resolved when a user links a defect to the test or marks a test as muted. Both actions are the deliberate ones and imply that failure of a test is analysed, understood and processed, and as a result of the processing either defect is created/linked or test is muted.

Database

External DB

  • Q: How to set up external database for Allure TestOps?
  • A: Excellent question! External database is highly recommended for production environments. The migration process via backup and restore is described here.

Backups

Errors linked to databases

Removing the database locks

  • Q: How to fix inability of report or uaa service to start with the following error message in logs:
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'liquibase' defined in class path resource org.
springframework.beans.factory.BeanCreationException: **Error creating bean with name 'liquibase'** defined in class path resource [org/springframework/boot/autoconfigure/liquibase/LiquibaseAutoConfiguration$LiquibaseConfiguration.class]: 
Invocation of init method failed; 
nested exception is liquibase.exception.LockException: 
Could not acquire change log lock.

or a lot of these

liquibase.lockservice: Waiting for changelog lock...
liquibase.lockservice: Waiting for changelog lock...
liquibase.lockservice: Waiting for changelog lock...
liquibase.lockservice: Waiting for changelog lock...
  • A: This is related to a database lock. Service cannot get the access to the database.
    • Cause: this issue can be caused by unexpected service restart due to infrastructure or severe network malfunction.
    • Solution: Follow the steps described below:

  1. Stop all services that are not DB related, e.g. docker-compose stop gateway uaa report

  2. Log in to the Postgres database: psql -h <host> -U <user> <database>

    • Example for docker:
        docker exec -it <containder id> psql -h 127.0.0.1 -U report report

  3. Get the content of the table containing locks data

    select * from databasechangeloglock;

  4. Delete the lock preventing a service to start:

    update databasechangeloglock set locked=false, lockgranted=null, lockedby=null where id=1;

  5. Start the services stopped on the step #1.

Test cases

Deleted test cases

  • Q: Is it possible to restore a deleted test case.
  • A: Good news: test cases aren't deleted in Allure TestOps. Reality is they are marked as archived, so you can easily restore a test case you've deleted:
    • Go to URL {allure-url}/project/{project_id}/test-cases-list
    • Tick the check-box Only deleted
    • Click search button (looks like magnifier)
    • Click a test case by its ID
    • In the opened window select Restore
    • Confirm the action

Cloning, copying

  • Q: How to copy test cases within one project or between projects?
  • A: This action is called cloning. You can clone test cases within one project or you can clone test cases to other project.
    • Select the test cases you need (tick the check-boxes).
    • Click Bulk actions button (it'll become active as you select at least 1 test case)
    • Select Clone and follow your intuition, it's pretty straightforward.
    • To move the tests to other project, select Move to other project

Storage

Out of the box storage

  • Q: Where does Allure TestOps store the files?
  • A: If you haven't integrated S3 storage (which is highly recommended as FS in a container is quite slow and only suitable for trial period), then all the files related to your tests (scenarios, attachments) are stored in report service container under /storage if you see something more than /storage/v2, then you have mix of blob storage v1 and blob storage v2. If you need to migrate all the files to the blob storage v2, please create a support ticket to our Service Desk.
    • After 3.177.X release we added min.io into the installation to avoid collisions related to file system. So all the I/O operations are handled now by min.io service.

S3 as storage

Docker compose and S3

  • Q: How do I migrate to S3 bucket with docker-compose installation?
  • A: Please refer to this document.

Kubernetes and S3

  • Q: How do I migrate to S3 bucket with Kubernetes installation?
  • A: Please refer to this document.

License status

  • Q: How do I check the license of the system?
  • A: This action is available for Allure TestOps users with ROLE_ADMIN assigned to them. To see the current status of the license you need to go Administration and then to the License section of the settings.

License expiration

  • Q: What will happen when my license has expired?
  • A: When the license has expired, the system falls into Read Only state. In terms of back end this means all the users will be considered as the ones having ROLE_AUDITOR, i.e. all the integrations will stop working and system won't receive any test results from the CIs. There will be only one active account – the super user of the system, which will have only one action available – the entering of a new license.

Information about system version

  • Q: How do I check Allure TestOps' current running version?
  • A: You can access a system's page with the version and services' status information: ALLURE_URL/status

Allure TestOps API

  • Q: Could we use Allure TestOps API to integrate with our tools?
  • A: Indeed, you could. API is open for our clients and is accessible via following URL: ALLURE_URL/swagger-ui.html

Allure TestOps performance

  • Q: Is there a limit on test cases number? Is there an expected amount that might degrade the performance of Allure TestOps?
  • A: There are several factors affecting the performance:
    • First, please check if your environment complies with the minimal and recommended HW requirements, this is important and very often (too often, to be honest) DevOps or Ops or whoever is responsible for the allocation of the resources, do not allocate enough resources to Allure TestOps and this leads to general degradation of the performance.
    • Second, these are the requirements for the production system
      • the database needs to be a standalone one and must not run in a container
      • the storage should be S3 not the storage in a container
      • both, the DB in a container and FS storage in a container severely degrade the performance.
    • Third, if you have several thousands of tests daily, you need to scale up the report service. Three instances of report should be sufficient.
    • Some of our clients are running 800K+ tests daily and Allure TestOps processes this number of tests without any problems.

Users management

Disable the self-registration

  • Q: How do I disable the capability of the self registration in Allure TestOps (Sign Up! link in the log-in page)
  • A: You need to add an environment variable ALLURE_REGISTRATION_ENABLED: "false" to uaa service configuration's environment section.

Registration approval

  • Q: How do I disable activation of self-registered users?
  • A: You need to set the environment variable ALLURE_REGISTRATION_AUTOAPPROVE to "false" in uaa service configuration's environment section.

Kubernetes installation

Persistent volume claim

  • Q: Do we need to set up persistent volume claims for k8s installation with S3?
  • A: If you are using S3 to store artifacts, then only the database and S3 should be persistent.
© 2023 Qameta Software Inc. All rights reserved.