EN 
DE Increased trust in the code through automated feedback
Author: Daniel Penning
Contribution – Embedded Software Engineering Congress 2015
Test-driven development can significantly improve software quality. For embedded systems, automated tests are often only possible with specialized test hardware. A software-based dual-target approach allows tests to be executed without additional hardware. This article argues that relying solely on traditional unit tests is insufficient. It demonstrates how to build a more comprehensive test environment.
Introduction
Agile methodologies have led to remarkable advances in software development. Despite their proven advantages, Scrum and similar methods for developing embedded systems are only slowly gaining traction. A key reason for this is the specific challenges of embedded systems, such as hardware dependencies and the software's execution environment.
The core of agile methodology lies in welcoming changes in requirements as the project progresses (embrace changeChanges to a codebase can only be implemented quickly if it is ensured that the modification does not impair existing functionality. Therefore, particular emphasis is placed on automated software testing. Fast and reliable feedback allows developers to quickly assess the impact of major changes. Implementing this feedback element represents the central hurdle in the successful application of agile methods for embedded systems development. Without this element, or with insufficient test coverage, the full potential of such approaches cannot be realized.
Problem statement and analysis
Figure 1 (see PDFFigure 1 shows the power electronics control system discussed below. It consists of a combination of a DSP and an FPGA. A real-time operating system (RTOS) and the application developed in C++ run on the DSP. The FPGA enables precise timing of signal values and switching via PWM with a well-defined timing response. For the purposes of this analysis, a buck converter is considered the power electronic component.
For this exemplary system, a test-driven development approach will be presented here, in line with agile development principles. As described at the beginning, this requires the provision of fast and high-quality feedback. This feedback should reliably provide information about the stability of the current software version and thus answer questions such as:
- Do the individual methods behave as desired?
- How do different classes interact?
- Are the necessary tasks being completed in the correct order?
- Does a rule implemented in the code meet the requirements?
- Does a software-controlled overload protection system trigger correctly?
(1) and (2) represent the classic application area of unit tests. Methods are tested independently against specific expected values. Stubs and mocks are used to isolate dependencies. These tests are increasingly used in embedded systems. Since all hardware and system dependencies are isolated, the code to be tested, including the tests, can be compiled for the architecture of the development computer (PC) and executed there, thus allowing for verification. Excellent literature [1, 2] deals with the practical implementation, especially the software architecture required for unit tests. Test frameworks such as googletest [3] support developers in formulating meaningful unit tests in C++.
(3) focuses on the interaction between the software and the RTOS. Task priority, the scheduling method used, and the availability of resources (mutex, queue, etc.) significantly determine the execution order of tasks. Errors in designing the software for a multitasking environment can easily lead to deadlocks, interrupting the entire execution. The dependency on the operating system and the time dimension clearly distinguish these integration tests from traditional unit tests. The necessary test configurations must be compiled for the target system and executed there. Furthermore, the result must be readable. Therefore, such tests are typically difficult to implement.
(4) and (5) incorporate the external environment into the tests. For certain output signals of the external power electronics, the software is expected to respond with defined control signals. In practice, this is usually demonstrated using system tests. For this purpose, either a hardware simulator or – if possible – the actual external hardware is connected to the system and the process is monitored.
The concept of classic unit tests can be directly applied to embedded systems. By definition, however, these tests cannot and should not test the complex temporal interaction with RTOS and external hardware. In traditional software development, these dependencies often play only a minor role. The goal of embedded systems, however, is precisely to test this interaction. embedding Environment. Optimal feedback can only be achieved here if the integration tests (3, 4, 5) are also included.
In practice, manual testing on the target system is very time-consuming and expensive, and therefore cannot be performed with the desired frequency and within the short timeframe. Therefore, a way to automate the process must be found.
Solution
Unit tests are easy to implement because they can be executed on a PC. This is achieved using a so-called dual-target approach. The source code, which is actually intended for the target system (here, the DSP), is additionally compiled for the PC's architecture (e.g., x86). This approach cannot be directly applied to integration tests because the code has dependencies that do not exist on the PC. Figure 2 (see PDF) shows these dependencies for the example system.
Nearly every major embedded system already uses the abstraction from the actual hardware referred to here as HAL (Hardware Abstraction Layer). If this HAL is also made available to the PC in a sufficient form, the entire code can be compiled and executed directly on the development machine. The elimination of hardware dependency accelerates the development process and enables optimal feedback through easily automated integration tests.
The RTOS used must first be simulated for execution on a PC (e.g., under Microsoft Windows®). This can be achieved by emulating the tasks using the native threads available on every desktop operating system. Particular attention must be paid to the implementation of the scheduler. The RTOS uses preemptive multitasking, meaning it can interrupt running tasks at any time. This behavior can be simulated under Windows if the scheduler operates in its own high-priority task. All actual program tasks are encapsulated in low-priority threads. Another high-priority task can simulate interrupt generation. This simulation can prove complex and time-consuming, depending on the operating system. For the popular FreeRTOS...™ An official port for Windows already exists [4].
Additionally, necessary drivers for the peripherals must be provided with sensible implementations.
The example system interacts with the power electronics to be controlled exclusively via the FPGA. Integration tests should be able to verify the behavior of the control system. Therefore, in addition to simply replicating the FPGA blocks, a simulation of the connected power electronics must also be performed.
Figure 3 (see PDFFigure 4 shows the behavior of the PWM block. The simulation must retain its time-deterministic character. For this purpose, the behavior shown in Figure 4 (see Figure 4) is used. PDFThe structure shown is used. An Environment Controller (EC) is implemented using a time-discrete simulated system time. The EC controls the triggering of time-dependent events, such as the On event (setting the output to a high level). The PWM block itself is simultaneously connected to a model of the connected system.
Based on the previously completed controller design, PLECS® models [5] for the power electronics – in this case, the buck converter – are available. PLECS allows these models to be exported in an ANSI C-compatible format. The generated discrete-time models can thus be easily connected to the simulated PWM block. The FPGA interface used by the application can now access a realistic model. Control interventions via the PWM duty cycle result in a dynamic response in the model. System parameters can be read from the code via an additional analog-to-digital converter.
This establishes the basic prerequisites for integration tests. For automated execution, the signal waveforms of relevant parameters must be recorded and compared with predefined reference signals. A signal framework, not discussed in detail here, was developed for this purpose. This framework reads the necessary information from XML files, records signals during the test run, and performs an automatic evaluation.
Example
Figure 5 (see PDFFigure 6 shows a test for verifying the shutdown logic. At time t=4ms, the applied model is manipulated by software to simulate a load short circuit. The shutdown logic is implemented as part of the application within the DSP in a control task. Therefore, all simulated components are involved in executing this test. Furthermore, a graphical export is generated, which is shown in Figure 6 (see Figure 6). PDF) is shown. From this, the success/failure of the test can be clearly understood.
Summary
The developed approach demonstrates that an agile and test-driven methodology is possible for complex systems with hardware dependencies. It is essential to ensure that the developer receives optimal feedback on the program status at all times. A completely software-based testing approach is automatable, minimizes feedback time, and reduces costs. Independence from physical hardware allows for early development and reduces time-to-market.
Hardware-specific details can only be represented with considerable effort, and time-dependent processes within the DSP cannot be represented at all. Systems with a high logic component, on the other hand, will benefit from increased test coverage. The complexity and scope of programs for embedded systems are constantly increasing. At the same time, increasing computing power allows for higher levels of abstraction at the software level. Therefore, it can be assumed that more and more application areas will emerge in the future that will benefit from the application of such a test-driven methodology.
References
[1] Martin, Robert C.: Agile software development: principles, patterns, and practices. Prentice Hall PTR, 2003.
[2] Grenning, James W.: Test-driven development for embedded C. Pragmatic Bookshelf, 2011
[3] https://github.com/google/googletest
[5] https://www.plexim.com/de/plecs
Testing, Quality & Debugging – Our Training & Coaching
Do you want to bring yourself up to date with the latest technology?
Then find out more here MircoConsult offers training courses/seminars/workshops and individual coaching on the topics of testing, quality & debugging.
Training & coaching on the other topics in our portfolio can be found here. here.
Testing, Quality & Debug – Expertise
Valuable expertise on the topics of testing, quality & debugging is available. here Available for you to download free of charge.
You can find expertise on other topics in our portfolio here. here.
