How to Use the Command 'tmt run' (with Examples)

tmt run is a versatile command-line tool provided by the Test Management Tool (TMT) designed to automate and simplify the execution of tests as part of a continuous integration process. By using TMT, developers and testers can run test plans and various steps associated with testing, such as discover, provision, execute, report, and finish. The tool allows for detailed customization and configuration of test execution, catering to a wide array of testing scenarios.

Use case 1: Run all test steps for each plan

Code:

tmt run

Motivation:

Running all test steps for each plan provides a comprehensive testing approach, ensuring that every aspect of the application is validated. This is essential for developers and testers who want to ensure that any changes or new features have not introduced regressions.

Explanation:

• tmt run: The base command that triggers the execution of all test plan steps. By default, it will perform a full run, executing each step from discovering the tests to generating a report.

Example Output:

/var/tmp/tmt/run-003

/plans/test-plan

Summary
-----------------
1 test passed
0 errors
0 failures

Use case 2: Run only the discover step to show what tests would be run

Code:

tmt run discover -v

Motivation:

Sometimes, it is essential to verify which tests are selected without actually running them. This is helpful when planning test coverage or when configuring test environments where insight into potential load is needed.

Explanation:

• discover: Specifies that only the discover step should be executed.
• -v: Enables verbose output, providing detailed information about discovered tests.

Example Output:

/var/tmp/tmt/run-004

Discover

tests:
- /tests/test-example

Use case 3: Run all steps and adjust the provision step options

Code:

tmt run --all provision --how container --image fedora:rawhide

Motivation:

Customizing the environment in which tests run can be crucial, especially when tests need to be run in isolated environments such as containers. This flexibility allows for testing in different distributions or under specified conditions.

Explanation:

• --all: Ensures that all steps are executed as part of the run process.
• provision: Specifies execution of the provisioning step.
• --how container: Indicates that the provisioning should happen in a containerized environment.
• --image fedora:rawhide: Specifies the particular Docker image to use, in this case, Fedora’s Rawhide version.

Example Output:

/var/tmp/tmt/run-005

Provision
provision: Fedora container created

Summary
-----------------
2 tests passed
0 errors
0 failures

Use case 4: Run only selected plans and tests

Code:

tmt run plan --name /plan/name test --name /test/name

Motivation:

Focusing on specific plans and tests is particularly useful during development or debugging when retesting specific scenarios or functionality. This optimizes testing time and resources by targeting relevant test plans and cases only.

Explanation:

• plan --name /plan/name: Filters to execute a specific plan by its name.
• test --name /test/name: Further narrows down execution to a particular test by its name within the specified plan.

Example Output:

/var/tmp/tmt/run-006

/plans/selected-plan

Execute

/test/selected-test
test: Selected test executed

Use case 5: Show results from the last run in a web browser

Code:

tmt run --last report --how html --open

Motivation:

Viewing the test run results in an easily digestible format like an HTML report can help stakeholders quickly understand test outcomes. This accessibility facilitates better communication across team members who benefit from visual elements over CLI outputs.

Explanation:

• --last: Refers to the most recent test run.
• report: Instructs TMT to generate a report from the last run.
• --how html: Specifies HTML as the format of the report.
• --open: Automatically opens the generated report in a default web browser.

Example Output:

Automatically opens the HTML report in your preferred web browser, displaying organized test results.

Use case 6: Run tests with the provided context

Code:

tmt run --context key=value -c distro=fedora

Motivation:

Contextual operations allow users to define specific variables or parameters necessary for test execution. This can affect how tests are run and set up, enabling flexible testing scenarios depending on the required configurations or platforms.

Explanation:

• --context key=value: Provides extra context parameters for the test run.
• -c distro=fedora: Short form to include context, setting the distribution to Fedora as part of the test context.

Example Output:

/var/tmp/tmt/run-007

Context
distro: fedora

Summary
-----------------
3 tests passed
0 errors
0 failures

Use case 7: Run tests interactively

Code:

tmt run --all execute --how tmt --interactive

Motivation:

Interactive test runs are essential for debugging when you need to pause and examine the state of the test at specific points. This is crucial for developers troubleshooting complex issues that arise during automated testing.

Explanation:

• --all: Executes all steps in the run.
• execute: Focuses on the execution step of the plan.
• --how tmt: Specifies the method used for executing tests.
• --interactive: Allows the user to interact directly with the test execution process.

Example Output:

/var/tmp/tmt/run-008

Execute
Starting interactive session for test debugging

Use case 8: Use dry mode to see what actions would happen and use the highest verbosity

Code:

tmt run --dry -vvv

Motivation:

Simulating a test run without executing any commands is beneficial in planning and preparing test plans. It provides insights into what the test execution would entail and helps identify potential configuration issues before an actual run.

Explanation:

• --dry: Initiates a dry run, simulating the actions without making changes or executing tests.
• -vvv: Sets the verbosity to the highest level, providing detailed output about every action and decision TMT would make during a real test execution.

Example Output:

/var/tmp/tmt/run-009

Dry Run
Discovering tests
Provisioning environment
Executing tests

Conclusion:

The tmt run command offers a range of features tailored to meet various testing requirements, from running comprehensive test plans, focusing on specific tests, to customizing environments and viewing results. Its flexibility allows developers and testers to integrate testing smoothly into their workflows, ensuring robust software delivery with meaningful feedback at every stage of development.