Commandline tool - allurectl

allurectl is a command-line tool that simplifies integration with Allure TestOps. It handles uploading test results and enables selective test execution when a pipeline is triggered from Allure TestOps. So, your team can focus on working with the tests.

What to do?

You need to download allurectl during CI pipeline execution or include it into the image you are using to run tests, then wrap the tests execution in allurectl commands. Simplified it'll look like below.

allurectl watch -- /run/my/tests

Binary files location

allurectl iw written using Golang, meaning there are executable files, and these are built for certain OS and CPU type.

Download the binary that matches your operating system and processor architecture from the latest allurectl release.

Supported platforms

Example download for Linux x64:

wget https://github.com/allure-framework/allurectl/releases/latest/download/allurectl_linux_amd64 -O ./allurectl
chmod +x ./allurectl

Before you use allurectl

Create a Allure TestOps API token first:

1. In Allure TestOps, click your avatar and open API Tokens.
2. Click + Token.
3. Create the token and copy it.

For the detailed token workflow, see User menu and API tokens.

Execution modes

allurectl operates in two modes:

• CI mode when allurectl is executed on a CI server and has the access to CI pipeline execution context.
  • this mode requires the usage of watch workflow (see below) in the vast majority of the cases.
• local mode when you run it on your PC or remote VM/server and execution context is not linked to a CI/CD pipeline.
  • this mode can use either workflow — watch or upload.
  • this mode creates a single launch without job-runs.

When allurectl is executed in the CI mode, Allure TestOps links uploaded results to the correct CI pipeline execution by creating internal entities such as job, launch, job-run, and allows requesting re-run of tests on CI side or create a new launch with new pipeline execution triggered from Allure TestOps UI.

For the local mode upload there are no involvement of entities such as Job or job-run, thus there is no connection to CI/CD pipeline execution.

watch and upload workflows

There are two workflows for uploading test results to Allure TestOps:

• watch — run as allurectl watch
• upload — run as allurectl upload

Which workflow you should use

Use allurectl in a pipeline

1. Add allurectl to your pipeline — either download it at runtime or include it in your test image.
2. Provide the required configuration parameters.
3. Upload results using the watch or upload.

Download allurectl

In the scope of a CI pipeline or when preparing an image for tests, you can use wget or curl to download the required tool version.

wget https://github.com/allure-framework/allurectl/releases/latest/download/allurectl_linux_amd64 -O ./allurectl
chmod +x ./allurectl

curl -L https://github.com/allure-framework/allurectl/releases/latest/download/allurectl_linux_amd64 -o ./allurectl
chmod +x ./allurectl

iwr "https://github.com/allure-framework/allurectl/releases/latest/download/allurectl_windows_amd64.exe" -OutFile "./allurectl.exe"

curl -L "https://github.com/allure-framework/allurectl/releases/latest/download/allurectl_windows_amd64.exe" -o allurectl.exe

Provide configuration parameters

Mandatory configuration parameters

For allurectl to run and successfully upload the test results the following configuration parameters are mandatory:

• Allure TestOps endpoint.
• Allure TestOps API token.
• Allure TestOps Project ID.
• Path to the directory with test results.

Ways to provide configuration parameters

Configuration parameters of allurectl can be provided using three methods described below:

• via environment variables (best for pipelines)
• via command line parameters and switches (cold look a bit bulky and confusing)
• via allurectl configuration file (for pipelines)

# defined somewhere in the pipeline or on the CI project level
export ALLURE_ENDPOINT=https://testops.example.com
export ALLURE_TOKEN=55555555-5555-5555-5555-555555555555
export ALLURE_PROJECT_ID=100
export ALLURE_RESULTS=build/allure-results
export ALLURE_LAUNCH_NAME="Nightly API tests"

#  executing tests and uploading results
./allurectl watch -- ./gradlew clean test

# defined somewhere in the pipeline
export ALLURE_ENDPOINT=https://testops.example.com
export ALLURE_TOKEN=55555555-5555-5555-5555-555555555555
export ALLURE_PROJECT_ID=100
export ALLURE_LAUNCH_NAME="Nightly API tests"

# tests executed somewhere
# results are copied from somewhere to /path/to/allure-results

# uploading results
./allurectl upload /path/to/allure-results

./allurectl watch \
  --endpoint https://testops.example.com \
  --token 55555555-5555-5555-5555-555555555555 \
  --project-id 100 \
  --launch-name "Local verification run" \
  --launch-tags "regress,nightly" \
  --results path/to/allure-results \
  -- /path/to/tests/execution/command

./allurectl upload \
  --endpoint https://testops.example.com \
  --token 55555555-5555-5555-5555-555555555555 \
  --project-id 100 \
  --launch-name "Local verification run" \
  --launch-tags "regress, nightly" \
  /path/to/allure-results

endpoint: https://your-testops.example.com
# avoid token in the file by all means
# use secret/environment variable ALLURE_TOKEN of your CI to pass token info
# token: avoid-using-your-api-token-here

project-id: 999
launch-name: "Static launch name"
launch-tags:
  - "regress"
  - "progress"
  - "egress"
# other parameters on the level 1

exclude-files:
  - "[0-9a-f-]+-container\.json"
skip-too-big: true
ignore-passed-test-attachments: false

allurectl watch --config ./ctl-conf.yaml

Pass pipeline context to remote/isolated execution

Tests can run outside of the context of the main CI job, e.g. in a Docker container, on a remote runner, or in a downstream pipeline, but their results still need to go into the same launch, and we need to be able to request re-run of the tests. This section explains how to do that.

When tests are executed in a container or on remote VM/bare metal server, the execution context is different from the one existing in the pipeline and for allurectl it won't be possible to create job-run and link the results to the pipeline. Such execution of tests and upload will be treated as local/manual mode.

To link a pipeline execution with a launch in Allure TestOps, allurectl uses set of CI environment variables (different variables for each of the supported pipelines) to establish such link. For natively supported CI systems, this works out of the box with minimal configuration.

Our task is to deliver minimal required pipeline context (variables and their values) to the environment where the tests are executed.

In the "parent" pipeline execute:

ALLURE_CI_ENV=$(allurectl ci env --encode)

Store the content of ALLURE_CI_ENV and pass it to the environment where the tests will actually run, then start allurectl there as usual:

Here is an example for tests executed in a docker container:

ALLURE_CI_ENV=$(allurectl ci env --encode)


docker run --rm \
  -e ALLURE_CI_ENV=${ALLURE_CI_ENV} \
  -e OTHER_VARIABLES="stuff"
  registry.int/image-name:latest

In this case, allurectl will

• detect ALLURE_CI_ENV,
• take the required context from ALLURE_CI_ENV
• continue the upload of test results as if it was executed in the pipeline run context, and it'll link test results to the pipeline where docker run command was executed.

Tests rerun and selective run

Selective execution if two-fold.

1. The allure adaptor for the test framework must be an official one, i.e. listed here.
2. Selective execution depends on a testplan.json file that identifies which tests the runtime should execute. The file testplan.json when we are talking about Allure TestOps is created by allurectl or by a CI plugin like Allure TestOps for Jenkins based on the information available in the job-run.

In the current TestOps-driven flow:

1. TestOps starts a job-run and provides the job-run context.
2. Based on the information from the created job-run allurectl creates the testplan.json file and exposes the related test plan path to the runtime via ALLURE_TESTPLAN_PATH variable that will be available in the execution context of tests.
3. The Allure adapter for the test framework detects ALLURE_TESTPLAN_PATH, then reads that testplan.json and runs only the selected tests based on testplan.json content.

Selective runs triggered by Allure TestOps do not require you to manually generate testplan.json in the pipeline.

testplan.json structure

If you need to validate adapter behavior locally, you can still create a plan file manually and point the runtime at it.

Example:

{
  "version": "1.0",
  "tests": [
    {
      "id": "10",
      "selector": "io.qameta.allure.examples.junit4.AllureSimpleTest.allureSimpleTest"
    },
    {
      "id": "11",
      "selector": "io.qameta.allure.examples.junit4.AllureParameterizedTest.allureParameterizedTest[First Name]"
    }
  ]
}

Where:

• id is the Allure ID from TestOps;
• selector is the test-framework-specific selector used by the adapter.

Troubleshooting the selective run

If your CI starts every test when Allure TestOps triggers the pipeline the most likely causes are:

• the test framework process cannot find testplan.json, so the adapter has nothing to filter by. Child processes, including shell scripts, inherit ALLURE_TESTPLAN_PATH from allurectl watch, but the variable or the file is often lost on the way:

  • the tests run in a Docker container or on a remote runner, while allurectl watch runs outside of it. Run allurectl where the tests run and pass the pipeline context with ALLURE_CI_ENV, see Pass pipeline context to remote execution;

  • the script starts the tests through sudo or env -i, which reset the environment;

  • ALLURE_TESTPLAN_PATH contains a relative path, and the tests start from another working directory, for example after cd tests, or in a Maven or Gradle submodule.

    To check, print the variable and the file right before the test command:

      echo "$ALLURE_TESTPLAN_PATH" && cat "$ALLURE_TESTPLAN_PATH"
    

• the test framework adapter does not consume the testplan.json;
  • old version
  • not official integration, i.e. not listed here.
• the runtime's context does not have ALLURE_TESTPLAN_PATH and Allure Framework adaptor is not reading the testplan.json;
• your tests execution command has test framework filters provided via command line parameters
  • if filtering options are provided via CLI, they prevail and any other options stop working;
  • so, if selective test run is required, then the start of tests needs to be without any additional filtering options in the command line.

mvn test -Dtest='LoginTest'          # or -Dgroups=smoke etc.
./gradlew test --tests '*LoginTest'

pytest -m smoke
pytest -k login
pytest tests/test_login.py
pytest --lf

npx playwright test --grep @smoke
npx playwright test tests/login.spec.ts --project=chromium
npx playwright test --last-failed

Validate selective execution locally

To validate the selective tests run locally follow the steps.

1. In Allure TestOps Test Cases section select one test case and copy its full path in the UI.
2. Create testplan.json with the structure described here and just one record in the root of your tests project folder.
   1. id attribute can be anything, it won't be considered in case you have no AllureID assigned in the code, otherwise use the AllureID as per test code.
3. Point ALLURE_TESTPLAN_PATH to it.
   1. That'll be export ALLURE_TESTPLAN_PATH=./testplan.json
4. Run your tests

If only one test has been executed, then integration with framework works fine and you can proceed.

Aggregate test results from several pipelines to a single launch

This is useful when tests don't run in the pipeline that starts first, but in a downstream pipeline that it triggers. In that case, run allurectl in the first (upstream) pipeline, before any tests start. That pipeline creates the launch and then passes its launch ID to the downstream pipeline, so all test results are uploaded to that same launch.

When running in CI mode, allurectl can expose the created job-run is and launch id values to later pipeline steps.

After watch or upload finishes:

export $(allurectl job-run env)

This creates a new launch and a job-run, ten returns environment variables and fills their values which can be used by allurectl in downstream pipeline or in following stages of current pipeline, use ALLURE_LAUNCH_URL to point the upload of test results to the created launch.

Full output of the command will be

ALLURE_LAUNCH_ID=646248
ALLURE_LAUNCH_URL=https://demo.testops.cloud/launch/646248
ALLURE_JOB_RUN_ID=600760
ALLURE_TESTOPS_JOB_RUN_URL=https://demo.testops.cloud/jobrun/600760

Pass ALLURE_LAUNCH_ID to the next stage (export the env variable ALLURE_LAUNCH_ID with the launch id value) and next stage will report test results to the same launch.

Cookbook

Run tests even when TestOps is unreachable

Available from 2.19.0.

To process Allure TestOps outage, you can use the following flags when starting allurectl.

• --fail-fast flag to stop watched command on unrecoverable TestOps errors.
• --continue-on-error flag to preserve command exit code despite upload errors.
  • --silent is deprecated in favour of --continue-on-error and now works alias for --continue-on-error for backward compatibility.

By default, allurectl fails the pipeline if it cannot connect to TestOps. Use --continue-on-error to let the tests run anyway without uploading results.

./allurectl watch --continue-on-error -- <your test command>

You can also use ALLURE_CONTINUE_ON_ERROR=true defined in the pipeline context.

Skip result files that exceed the size limit

Available starting from allurectl release 2.15.0.

Allure TestOps does not process *-result.json files larger than 2,000,000 bytes. Meaning, after such file is uploaded it will be skipped by Allure TestOps and deleted.

Use --skip-too-big to skip bug results files at upload time and avoid silent failures.

./allurectl watch --skip-too-big -- <your test command>

Files that exceed the limit are logged as warnings in allurectl's stdout output log and not uploaded.

Skip attachments for passed tests and generally skip some files

This reduces storage usage and speeds up upload.

Use --ignore-passed-test-attachments to avoid uploading attachments from passed tests.

./allurectl watch --ignore-passed-test-attachments -- <your test command>
# or
./allurectl upload --ignore-passed-test-attachments ${ALLURE_RESULTS}

Attachments in fixture setup and teardown blocks are uploaded regardless of this flag.

Available starting from allurectl release 2.9.0.

To skip files by name pattern, use --exclude-files with a Go-flavored regular expression:

./allurectl watch --exclude-files '/*-container.json' -- <your test command>

Add issue links to a launch

In CI mode, after watch completes add the following command:

allurectl launch add-issue --current-launch \
  --integration-id <integration_id> \
  --issue-keys AE-1,AE-2

In non-CI mode, provide the launch ID explicitly:

allurectl launch add-issue --launch-id <launch_id> \
  --integration-id <integration_id> \
  --issue-keys AE-1,AE-2

The integration_id is available in the Allure TestOps integration settings.

Available starting from allurectl release 2.2.0.

Run a Allure TestOps test plan from the pipeline

To execute only the test cases from a specific Allure TestOps test plan, generate testplan.json before starting the tests:

export ALLURE_TESTPLAN_PATH="testplan.json"
./allurectl test-case plan -q "testPlan=<plan_id>" --output-file ${ALLURE_TESTPLAN_PATH}
./allurectl watch -- <your test command>

Replace <plan_id> with the numeric ID of the test plan in Allure TestOps.

Reading allurectl output

After upload completes, allurectl prints a full summary. Example:

Allure TestOps Client v2.23.1
Project [5] with name [my project]
Creating new launch with name [nightly 2024-01-15 113340]
Launch [34] with name [nightly 2024-01-15 113340]
Session [35] started
Report link: https://testops.example.com/launch/34
...
Total files indexed: 29796 || Finished files: 28968 || Orphan Files: 828

+-----------+-------------------------------------------------------------+-----------------+--------------+----------------+
| INDEXING: |                         Total=29796                         | Completed=28968 |              |                |
+-----------+-------------------------------------------------------------+-----------------+--------------+----------------+
|   STATS:  |          type           |      count      |              |                |
+-----------+-------------------------------------------------------------+-----------------+--------------+----------------+
|           |        allure.attach    |      19123      |              |                |
|           |        allure.result    |      6074       |              |                |
|           |       allure.container  |      3610       |              |                |
|           |           unknown       |       989       |              |                |
+-----------+-------------------------------------------------------------+-----------------+--------------+----------------+
|  TIMINGS: |          Duration=6m26s |  Min=325ms      | Avg=2m50s    | Max=6m26s      |
+-----------+-------------------------------------------------------------+-----------------+--------------+----------------+
|   SKIPS:  |                            Reason                           |      Count      |              |                |
+-----------+-------------------------------------------------------------+-----------------+--------------+----------------+
|           | can't find *-result.json or *-container.json for attachment |       828       |              |                |
+-----------+-------------------------------------------------------------+-----------------+--------------+----------------+

+----------+--------------------+------------------+---------------------+--------------------+---------------+
|  FILES:  | TotalBatches=3645  | TotalFiles=28968 | UploadedFiles=28968 | TotalSize=7.170 GB | ErrorsCount=0 |
+----------+--------------------+------------------+---------------------+--------------------+---------------+

Watcher finished in [6m28s]
You can find report here: https://testops.example.com/launch/34

Key fields:

• Completed — files successfully processed by TestOps. If this is zero and ErrorsCount is non-zero, the results directory was already populated before allurectl watch started — use allurectl upload instead.
• Orphan Files / SKIPS — attachments with no matching *-result.json or *-container.json. These are not stored and are deleted when the launch closes. A non-zero count at the end of the run may indicate the wrong results path.
• unknown — files in the results directory that do not follow the Allure naming convention; likely orphans or garbage.
• ErrorsCount — upload errors. Any non-zero value warrants investigation with --log debug.

Troubleshooting

Enable debug logging

Use the --log switch to collect detailed upload logs. The switch must be placed before the watch or upload command — if added after, it will be ignored.

./allurectl --log debug watch -- <your test command>
# or 
./allurectl --log debug upload ${ALLURE_RESULTS}

Available log levels from least to most verbose:

• fatal,
• error,
• warn,
• info,
• debug,
• trace.

For each uploaded batch, allurectl outputs details on the upload process. Debug level includes the full HTTP exchange with the TestOps API, which helps identify authentication failures, incorrect endpoints, and upload errors.

Detailed debug logging is available starting from allurectl release 2.11.0.

No results arrive in Allure TestOps

Check these points first:

• ALLURE_ENDPOINT, ALLURE_TOKEN, and ALLURE_PROJECT_ID point to the expected TestOps project;
• ALLURE_RESULTS points to the directory where the test framework actually writes results;
  • if this one is incorrect you will see in the logs:
• the command is using watch or upload in the intended place in the pipeline.

A TestOps-triggered run creates a new launch instead of using the job run

This usually means the TestOps service context did not reach allurectl. Confirm that the CI integration passes the job-run variables through to the environment where allurectl runs.

Selective execution is ignored

If the runtime executes every test instead of following the plan, verify the adapter integration for the test framework and confirm that the plan path is available to the test process.

Tests produce too much stdout output

When tests write a large amount of data to stdout (for example, trace-level logging), allurectl watch can fail with:

Error: failed to scan err pipe: bufio.Scanner: token too long

This is a pipe buffer limitation. Two options:

1. Reduce stdout output in the tests — keep only what is needed for troubleshooting rather than tracing everything.
2. Increase the buffer size with --stdout-max-buff-size. The default is 1048576 bytes; increase gradually rather than setting the maximum.

./allurectl watch --stdout-max-buff-size 2097152 -- <your test command>

For the full CLI surface, use ./allurectl watch --help or ./allurectl upload --help.

Related pages

• Run an automated test for the first evaluation workflow.
• GitHub Actions and GitLab CI for CI-specific wrappers around allurectl.
• Run automated tests for the TestOps-side execution flow.
• allurectl releases on GitHub for changelogs and binary downloads.