TestOps logo
Try TestOps

Integration with CircleCI

This page describes how to set up both Allure TestOps and CircleCI in a project that uses CircleCI to run tests.

Like all other CI integrations for Allure TestOps, this involves communication in two directions.

  • A job in a CircleCI pipeline will use allurectl to send its status and test results to Allure TestOps.
  • Allure TestOps will use CircleCI API to trigger new pipelines, optionally passing a test plan.

Once a pipeline run stops, the corresponding launch in Allure TestOps may be closed either manually or automatically, according to the Auto close policy.

To enable Allure TestOps support in your CircleCI pipeline:

  1. enable sending data from CircleCI,
  2. enable triggering CircleCI pipelines,
  3. parametrize jobs (if necessary).

1. Enable sending data from CircleCI

For CircleCI to send pipeline statuses and test results to Allure TestOps, you need to generate an authentication token in Allure TestOps, add the token to CircleCI, and modify the job itself.

Then, run and check the pipeline to make sure that everything works so far.

1.1. Create a token in Allure TestOps

  1. In Allure TestOps, click on your avatar and go to Your profile.

  2. Under the API tokens section, click + Token.

  3. Enter a Token name (e.g., “Token for CircleCI”), then click Submit.

  4. In the dialog that appears, click the Copy icon to copy the token into clipboard.

    You will need this token on the next step.

1.2. Specify the token in CircleCI

  1. In CircleCI, open the project's page and click Project Settings in the top right corner.

  2. In the menu on the left, click Environment Variables.

  3. Click Add Environment Variable.

  4. In the dialog that appears, fill in the fields:

    • Name — “ALLURE_TOKEN”.
    • Value — the API token that you got on step 1.1.

  5. Click Add Environment Variable to close the dialog and save the settings.

1.3. Modify the job

To make changes to the pipeline, make the following changes in the .circleci/config.yml file in your repository.

  1. Make sure the version is set to 2.1.

  2. Add or extend the parameters block. It must include two optional string parameters: ALLURE_JOB_RUN_ID and ALLURE_USERNAME.

  3. For each job that runs tests:

    1. In steps, add the first step that downloads the allurectl tool and makes it executable.

      In the example below, we use curl for downloading the file. If curl is not included in the Docker image that you use for the job, use wget or a similar tool.

      Alternatively, consider building and using your own Docker image with allurectl included in it.

    2. Add or extend the job's environment block. It must include the following variables:

      • ALLURE_ENDPOINT — the URL of the Allure TestOps server.
      • ALLURE_PROJECT_ID — the ID of the Allure TestOps project.
      • ALLURE_RESULTS — the path to the test results directory, e.g., “build/allure-results”. If there are multiple test result directories in your project, you can separate them with commas or use a wildcard pattern, e.g., “modules/*/build/allure-results”.
      • ALLURE_JOB_RUN_ID — << pipeline.parameters.ALLURE_JOB_RUN_ID >>.
      • ALLURE_USERNAME — << pipeline.parameters.ALLURE_USERNAME >>.

    3. In the step that runs results, wrap the command into the ./allurectl watch command.

Example

Assume we have a Java project with a .circleci/config.yml file like this:

version: 2.1
workflows:
  test:
    jobs:
      - test
jobs:
  test:
    docker:
      - image: cimg/openjdk:17.0
    working_directory: ~/repo
    steps:
      - checkout
      - run:
          name: Run tests
          command: gradle clean test

With Allure TestOps integration, the file will look something like this:

version: 2.1
workflows:
  test:
    jobs:
      - test
parameters:
  ALLURE_JOB_RUN_ID:
    type: string
    default: ''
  ALLURE_USERNAME:
    type: string
    default: ''
jobs:
  test:
    docker:
      - image: cimg/openjdk:17.0
    working_directory: ~/repo
    environment:
      ALLURE_ENDPOINT: https://demo.testops.cloud
      ALLURE_PROJECT_ID: 1
      ALLURE_RESULTS: build/allure-results
      ALLURE_JOB_RUN_ID: << pipeline.parameters.ALLURE_JOB_RUN_ID >>
      ALLURE_USERNAME: << pipeline.parameters.ALLURE_USERNAME >>
    steps:
      - checkout
      - run:
          name: Download allurectl
          command: curl -fsSL https://github.com/allure-framework/allurectl/releases/latest/download/allurectl_linux_amd64 -o allurectl && chmod +x allurectl
      - run:
          name: Run tests
          command: ./allurectl --insecure watch -- gradle clean test

1.4. Run and check the pipeline

  1. In CircleCI, open the project and go to the pipeline triggered by the latest commit.

  2. If the pipeline is not yet finished, wait until it finishes.

  3. In the pipeline details, click on the step that runs tests.

    Close to the end of its log, there should be a link to the test report at Allure TestOps. Make sure that it is present and works.

    The link to the launch is in the pipeline run log.

  4. In the test report at Allure TestOps, open a single test's results.

    At the bottom of the page, there should be a link back to the CircleCI job. Make sure that it is present and works.

    The link to the CircleCI pipeline is at the bottom of the test results pane.

2. Enable triggering CircleCI pipelines

Integration with CircleCI needs to be configured on two levels.

First, the administrator specifies the URL of CircleCI server.

Then, any project's owner creates an authentication token in CircleCI, adds it to Allure TestOps, and configures the job in Allure TestOps.

2.1. Specify CircleCI server in Allure TestOps

  1. Log into Allure TestOps using an administrator account.

  2. Go to Administration → Integrations.

  3. Click + Add integration in the upper-right corner of the screen.

  4. In the dialog that appears, select CircleCI.

  5. Fill in the fields:

    • Name — a name to help you recognize the CircleCI server, e.g., “CircleCI production”.
    • Endpoint — the URL of the CircleCI server, e.g., “https://circleci.com/”.

  6. If your CircleCI server uses a self-signed SSL certificate, check the Disable certificate validation checkbox.

  7. Click Add integration.

2.2. Create a token in CircleCI

  1. In CircleCI, click on your avatar in the bottom left corner.

  2. In the menu on the left, click Personal API Tokens.

  3. Click Create New Token.

  4. In the dialog that appears, enter a Token Name to help you recognize the token, e.g., “Token for Allure TestOps”.

  5. Click Add API Token.

    The generated token will appear in the dialog. Click the Copy button to copy it.

    You will need this token on the next step.

2.3. Add the token to an Allure TestOps project

  1. In Allure TestOps, open your project.

  2. Go to Settings → Integrations.

  3. Under the Available integrations, find the CircleCI integration and click Add integration next to it.

  4. In the dialog that appears, specify the Token that you got on step 1.2.

    Check that the credentials are correct

    Click Test connection. After a few moments, a message should appear saying “Connection established”.

  5. Click Add integration to close the dialog and save the settings.

2.4. Configure the job in Allure TestOps

  1. In Allure TestOps, open your project.

  2. Go to Jobs.

    The page should contain a new job automatically added and configured during the launch on step 1.4.

  3. Click ⋯ next to the job name, then select Configure.

  4. In the dialog that appears, edit the fields:

    • Name — a name to help you recognize the job.
    • Build server — the name of the integration that you added on step 2.1.
    • Job can be used to run tests — if checked, users will be able to trigger this job from Allure TestOps.
    • Parameters — parameters that should be passed to CircleCI via environment variables, see Environment.

    The dialog allows to set job settings.

  5. Click Submit.

3. Parametrize jobs

CircleCI pipelines can accept parameters that are being passed to the commands. Allure TestOps integrates this feature with its own Environment concept, which lets you both set parameters for new jobs and see parameters set for jobs that were started outside Allure TestOps.

In CircleCI, the project's owner adds variables and their default values.

In Allure TestOps, the administrator declares the required display names for parameters globally, then the project's owner configures the parameters mapping both in the project's settings and in the job's settings.

If your project's repository has multiple branches, make sure to create a “Branch” environment variable in Allure TestOps and pass it to your job. This special name will instruct CircleCI which of the branches it should use.

3.1. Set the default values in CircleCI

To make changes to the pipeline, edit the .circleci/config.yml in your repository.

  1. In the global parameters block, declare the parameters and their default values.

  2. In the job's environment block, wrap the parameters into environment variables so that the allurectl tool can access them.

For example:

version: 2.1
workflows:
  test:
    jobs:
      - test
parameters:
  ALLURE_JOB_RUN_ID:
    type: string
    default: ''
  ALLURE_USERNAME:
    type: string
    default: ''
  PRODUCT_VERSION:
    type: string
    default: '1.23'
  TESTS_BROWSER:
    type: string
    default: chrome
jobs:
  test:
    docker:
      - image: cimg/openjdk:17.0
    working_directory: ~/repo
    environment:
      ALLURE_ENDPOINT: https://demo.testops.cloud
      ALLURE_PROJECT_ID: 1
      ALLURE_RESULTS: build/allure-results
      ALLURE_JOB_RUN_ID: << pipeline.parameters.ALLURE_JOB_RUN_ID >>
      ALLURE_USERNAME: << pipeline.parameters.ALLURE_USERNAME >>
      PRODUCT_VERSION: << pipeline.parameters.PRODUCT_VERSION >>
      TESTS_BROWSER: << pipeline.parameters.TESTS_BROWSER >>
    steps:
      - checkout
      - run:
          name: Download allurectl
          command: curl -fsSL https://github.com/allure-framework/allurectl/releases/latest/download/allurectl_linux_amd64 -o allurectl && chmod +x allurectl
      - run:
          name: Run tests
          command: ./allurectl --insecure watch -- gradle clean test

3.2. Add global parameter names in Allure TestOps

  1. Log into Allure TestOps using an administrator account.

  2. Go to Administration → Environment.

  3. For each parameter name that you want to add, click + Create, enter the name of the new parameter and click Submit.

    Note that this name can be a human-readable label and does not have to be equal to the name from step 3.1.

    Environment variable names are added in the Administration settings.

3.3. Map parameters to environment variables in Allure TestOps

  1. In Allure TestOps, open your project.

  2. Go to Settings → Environment.

  3. For each parameter name that you want to use:

    1. Click + Create if the parameter is not in the list. Otherwise, click the Edit icon next to its name.

    2. In the Mapping key field, specify the name of the environment variable name from step 3.1.

    3. In the Environment variable field, select the global parameter name from step 3.2.

    4. Click Submit.

    Mapping environment variables is configured in the project settings.

3.4. Add the parameters into the Allure TestOps job

  1. In Allure TestOps, open your project.

  2. Go to Jobs.

  3. Find the job which you want to parametrize. Click ⋯ next to the job name, then select Configure.

    The job settings dialog will appear, containing the Parameters section.

  4. For each parameter that you want to use, click Add and fill in the fields:

    • Name — the name of the environment variable name from step 3.1.
    • Value — the default value, same as the default from step 3.1.
    • Environment Variable — the global parameter name from step 3.2.

    Multiple environment variables can be specified in the dialog.

  5. Click Submit.

© Qameta Software Inc. All rights reserved.