EN 
DE Read on!
Embedded Test: Table of Contents
- Trend Spots
- Create a solid foundation for the test:
From requirements analysis to test specification - Good planning is half the battle: Test planning as a management task
- Specific requirements for testers: The test specification in detail
- The underappreciated testing method: Static analysis as a cost killer
- Ensuring robustness and functionality: Dynamic testing checks the code
- Moment of truth: Meaningful test results
- On the brakes when it comes to costs: Design for Test
- Faster to market: Automated testing
- Information pool: Book and web tips
Create a solid foundation for the test
1. From requirements analysis to test specification
The ability of a product to reach market maturity quickly through testing is determined as early as the requirements analysis stage of a project. The more precisely the system's intended and undesired capabilities are defined, the more clearly the testers' subsequent tasks will be outlined.
Requirements analysis describes a system with its inputs and outputs, as well as its environment and the constraints under which it operates. This analysis results in the requirements specification, which defines the system's context—that is, what it needs to do (functional) and what qualities it must possess (non-functional). Functional requirements should ideally be represented as sequences. Generally, requirements are still mostly discussed verbally with the client, although a written, ideally model-based, documentation could prevent future problems. In the case of verbal agreements, a detailed test specification is always recommended. Furthermore, the tester should be present during the formulation of the requirements specification to ensure the final system is testable.
Another safeguard is the description of functions that the system should not perform, which is usually frowned upon by marketing and sales departments for competitive reasons. However, the main purpose of the requirements specification is to define the framework for development and testing as narrowly as possible.
Quality feature |
Partial features |
| Functionality | Appropriateness, correctness, interoperability, regularity, safety |
| reliability | Maturity, fault tolerance, recoverability |
| Usability | Comprehensibility, learnability, usability |
| Efficiency | Time behavior, consumption behavior |
| Changeability | Analyzability, modifiability, stability, testability |
| Transferability | Adaptability, installability, conformity, interchangeability |
The quality characteristics of the ISO/IEC 9126 standard should be weighted differently depending on the project, for reasons of cost and effort. For embedded systems, the focus should be on timing behavior, as this contributes most to product quality.
A professional requirements specification also names the so-called stakeholders, i.e., the people who have a legitimate interest in the system, including developers, testers, management, and, on the client side, the purchaser and users. The latter are differentiated into primary users, who have to work with the product daily, and secondary users, such as an administrator, who only accesses it occasionally.
Furthermore, the specification describes the purpose of the system, which should be closely aligned with its economic objectives. In a further step, the boundaries of the system are defined; at this stage, it is considered a black box: Through which interfaces does the data enter, who feeds it in, and where and by whom is it retrieved?
The specification can be effectively extended with UML, primarily to ensure the plausibility of the requirements. UML diagrams serve to refine the specification and help the creator to delve deeper into the system. An example of this is the sequence diagram, which is ideally suited to representing processes between the system and its external environment.
The sequence diagrams of the requirements specification treat the system as a black box that communicates with actors, i.e., sensors and actuators, via interfaces. These represent the external world, such as devices or the environment.
Image 1: Embedded test – The focus is on the context of the product and not its internal workings.
On the trail of the error
Following the functional requirements specification, a fault analysis is conducted, which addresses potential problems that actors within the system could cause. From this, tests or test sequences are later derived to verify the system's robustness against external influences. For example, the fault analysis might reveal that inputs and outputs must be designed redundantly because the system is safety-critical. This ensures that, in the event of a fault, the system continues to operate with minimal disruption or without any loss of functionality.
Error analysis is therefore of great importance because it establishes the robustness, and thus the reliability and safety, of the final product right from the requirements specification stage. This is achieved through the detection, identification, and classification of errors, as well as the written documentation of tolerances. Unfortunately, this detailed type of description is not yet widespread, even though it significantly simplifies the tester's work. Ideally, the analysis would also be linked to error databases: This would make the error history of a system traceable at any time, facilitating the derivation of test cases.
Software error |
Hardware error |
| Processing errors – Floating-point numbers – Exceeding the limits – Algorithm |
failure – Total hardware – Periphery |
| Memory error – Overwriting memory – Pointer – Memory overflow – Inadequate initialization of the variables |
Faulty initialization of the peripheral |
| execution – Parallelism, e.g., deadlocks – Disregard for time constraints – Recursions |
Sporadic errors – Intermittents – Time-outs |
Error analysis serves to ensure the robustness of the system.
Among the various sources of error, hardware-related errors are of primary interest during the system specification stage, as software bugs typically only appear later in development. However, the requirements specification can already include measures to enable the detection of processing, storage, and execution errors. Errors should always be listed, categorized by error type, source, description, type (system-critical or non-critical), and response. Lists offer the advantage of significantly faster and more efficient work compared to a sequential representation.
Mistake |
source |
Description |
Art |
reaction |
| Object detector failure | Object detector | Ongoing diagnostics reveal the failure of the object detector. | system-critical | Opening the gate; shutting down the system |
| Engine failure | Motor | system-critical | Turn on the warning light |
Example of a clear error list
Derive test cases
From the completed requirements specification, concrete tests and inspections can be derived in the next step. The tester looks for test cases with which to examine the behavior of the system under test in specific situations: Does it behave according to the specified requirements and target values? Test cases consist of parameters and processes. Parameters can be set to valid or invalid, so that multiple test cases can be generated through clever parameterization. The tester considers the system's boundaries, i.e., which input and output signals the actuators and sensors provide. Interfaces and protocols are also taken into consideration. For the test procedures, the tester considers use cases, processes, state machines, and again, the protocols.
Figure 2: The parameters and procedures of the test cases are derived from the requirements specification.
A distinction must be made regarding non-functional requirements: External quality characteristics such as reliability can be examined, for example, through endurance tests in which the test subject must meet the required failure rate. For time requirements, it must complete a specific task within a specific time, which the tester derives as a target value from the sequence diagrams of the requirements analysis.
Testing internal quality characteristics such as portability, however, proves problematic. Here, the tester would have to port the system to different hardware platforms, which is practically impossible due to time and cost constraints. Therefore, the tester limits themselves to verification, examining, for example, what changes to the system architecture are necessary to port a system from processor A to processor B. Verification is always the first choice when a system cannot be executed.
„You can’t test what you can’t execute!“
(Bruce Powell Douglass in his book "Doing Hard Time")
Typical verification methods include reviews and static analyses, such as those using compilers or LINT. Verifications tend to be more general and comprehensive, while tests are precise and deliver more concrete results. In this sense, testing can be considered a type of verification – albeit the most meaningful and important one.
Test name |
Train Entrance 1 |
| Test objective | Testing the system behavior when a train with one carriage enters the station. |
| Test case description | A train with one carriage enters the level crossing from the right. |
| Expected results | 1. Traffic light flashes red (approx. 1 second) 2. Traffic light turns red 3. The barrier closes until it is in a horizontal position. |
| Specific procedure | Prerequisites: – Barrier open – Traffic light green Proceed: 1. Press the right retraction switch (1x) … |
Example of a tabular representation of a test case
Complete Trend Guide „Embedded test“ Download as PDF.
Training & coaching on the topics of our portfolio can be found here. here.
Furthermore, there is the possibility to explore the topic area Embedded software testing also in tailor-made workshops to address. They are tailored to the specific needs of tasks, projects, teams, and roles.
Get in touch with us! Contact form
More Trend Guides
MicroConsult supports you with training and coaching on all aspects of embedded testing:
Embedded software testing for C: Best practices for unit/module/component testing
Object-oriented embedded software testing for C++: Best practices for class and component testing
Quality, testing and software development in the medical field
ISTQB® Certified Tester Foundation Level – Testing of embedded and IT systems
Agile testing and test-driven development of embedded systems (Agile TDD)
Embedded Linux for testers, support and service
Download the complete trend guide
