EN 
DE High test coverage achieved quickly
Author: Thomas Schütz, PROTOS Software GmbH
Contribution – Embedded Software Engineering Congress 2018
State-transition tests are recommended in many standards for safety-critical systems. However, they are an excellent method for all embedded systems to achieve high test coverage quickly and systematically.
Methods for deriving test cases
The most well-known methods for deriving test cases are: Equivalence classes and Limit value analysis. They are typically used in unit tests in conjunction with code coverage measurements. (C0, C1, …) They are used to check the completeness of the tests. However, asynchronous concurrent components, such as those used in embedded systems, are very difficult to test with synchronous sequential unit tests. These test methods are also hardly suitable for integration or system tests based on hardware or software in the loop.
State Transition Tests are less widely used. However, they offer excellent possibilities for testing complex, concurrent software with high coverage. This makes it possible to define path coverage for the state space of the application under test (n-switchThe required test cases can be derived or generated manually. By appropriately applying the method, even tests for data combinatorics (n-wise) carry out.
See Fig. 1 (PDF): Test Coverage Metrics for State Transition Tests
The methodology of the state-transition test with path coverage
State-transition tests, as black-box tests, first define all externally accessible states of the system under test (SUT) in a state-transition diagram. Transitions describe all state changes between system states. Every path from the initial state (initial) to the final state (end) is a possible test path and thus a possible test case. To achieve complete transition and state coverage (0-switch), the paths of all generated test cases must collectively traverse each transition at least once.
See Fig. 2 (PDFExecution paths for test cases in the state-transition diagram
Higher coverages define the combination of consecutive transitions:
- 0-switch: Each transition was completed at least once.
This coverage already includes full coverage of the states. - 1-switch: All combinations of two consecutive transitions
- n-switch: All combinations of n+1 consecutive transitions
The high path coverage allows the tests to uncover many problems. One example of this is sporadically occurring issues. Race Conditions. These are difficult to detect using most other methods.
Tests for data combinatorics
State-transition tests are also well-suited for testing data combinations. In this example, all combinations of two parameters, each with three values, are tested against each other. This corresponds to the pairwise Test. Combinations of more parameters are also possible (n-wise) can be easily generated. The necessary values for the individual parameters can be easily determined using Equivalence classes and Limit value analysis can be derived.
See Fig. 3 (PDF): Description of data combinatorics using state-transition diagrams
Combination and generation of data and path coverage
Data and path coverage can also be effectively combined. For example, for all combinations of 3 parameters (n-wise) all flow paths (n-switch) tested.
Since the large number of necessary test cases can hardly be developed manually anymore, the use of suitable tools with test case generators is recommended for larger coverage areas.
See Fig. 4 (PDF): Combination of data and path coverage
Analysis and documentation of test procedures
Different perspectives can be used to analyze, understand, and document the resulting test cases.
- Execution trees provide an overview of all generated test paths and their different processes.
- Sequence diagrams They depict individual test sequences and their interaction with the system under test in great detail. A target/actual comparison of the sequences enables the rapid analysis and narrowing down of error causes.
See Fig. 5 (PDF): Different views on the generated test cases
Conclusion
- State-transition tests are a structured method for developing tests for concurrent embedded systems.
- State-transition tests enable the development of combinatorial test cases for high path and data coverage.
- Small covers can be developed manually. For larger covers, the use of tools is recommended.
author
Thomas Schütz studied aerospace engineering in Munich and founded PROTOS Software GmbH in 1997. As a software project manager and architect, he has contributed his experience in combining model-based approaches with the requirements of embedded systems to numerous projects. Thomas Schütz advises companies on the development of domain-specific toolchains and test systems for embedded systems and is the project manager of the Eclipse project eTrice.
Testing, Quality & Debugging – Our Training & Coaching
Do you want to bring yourself up to date with the latest technology?
Then find out more here MircoConsult offers training courses/seminars/workshops and individual coaching on the topics of testing, quality & debugging.
Training & coaching on the other topics in our portfolio can be found here. here.
Testing, Quality & Debug – Expertise
Valuable expertise on the topics of testing, quality & debugging is available. here Available for you to download free of charge.
You can find expertise on other topics in our portfolio here. here.
