EN 
DE The calibrated hardware-in-the-loop test system
Author: Kristian Trenkel, iSyst Intelligente Systeme GmbH
Contribution – Embedded Software Engineering Congress 2017
This article addresses the requirements for test systems from a functional safety perspective, using hardware-in-the-loop (HIL) test systems as an example. The ISO 26262 standard is considered. These considerations lead to requirements for the accuracy and reliability of the test systems. Analog inputs and outputs of the test systems are examined as examples. On the one hand, the results of investigations into the calibrability of various test systems, as well as the possibilities of automated calibration, automated adjustment, and automated self-testing, are presented. On the other hand, this article aims to stimulate discussion about further requirements for test systems for testing safety-critical systems.
In the automotive industry, an increasing number of safety-critical functions, such as emergency braking assist, lane keeping assist, and autonomous driving, are being developed and integrated into vehicles. The requirements of ISO 26262 [1] must be observed for the development and testing of these systems. This standard sets requirements not only for the development process and its tools, but also for testing, including test systems and test tools.
There are detailed requirements for test tools, as there are for software tools in general, which must be evaluated as part of tool qualification. For test systems (meaning hardware systems), such as a HIL system, the standard provides very little information (section 8.4.1.5 of Part 4). Therefore, in practice, there are differing opinions regarding the necessary measures for securing test systems. This ranges from simple commissioning tests to defined maintenance procedures and intervals. The accuracy of the (mostly analog) interfaces of the test systems plays only a minor role in this context.
In practice, however, it becomes clear that not only the basic functionality of the test system must be ensured. The system's accuracy, as is standard for other measurement systems, must also be guaranteed. For example, if analog inputs and outputs are used for simulating sensor values or measuring currents, precise specifications are absolutely essential, since safety-critical functions usually have exact requirements for fault detection thresholds and fault detection times. Therefore, calibrating and subsequently adjusting the interfaces of a test system is both sensible and necessary. However, this procedure is not yet common practice.
State of the art
Hardware-in-the-Loop (HIL) test systems are used as test environments in various areas of development. In this approach, the embedded system under development is operated in a simulated environment, and tests are performed.
A typical setup of a HIL test system, as is frequently used in the automotive sector, is shown in Figure 1 (see below). PDFThe central component is the real-time computer, which calculates the simulation model for the embedded system's environment in real time. The models used are usually created in Matlab/Simulink and translated into executable programs using the integrated code generation, which are then run on the real-time computer.
Connecting the real-time computer to its I/O interfaces (e.g., analog outputs, analog inputs, digital inputs and outputs, and CAN interfaces) requires signal conditioning components (e.g., signal levels or signal power), which are summarized under the heading "Signal Adaptation." This section is of great interest for the calibration problem presented here, as these components influence the analog signal paths.
Following signal adaptation, the embedded system, often referred to as the control unit, is connected. The interfaces intended for normal operation are typically used for this purpose. The loads are then connected to the control unit. Fault injection modules are inserted between the embedded system and the loads. These modules allow for the switching of various short circuits (e.g., short circuit to ground or short circuit to the supply voltage) as well as the disconnection of the load. This enables the testing of fault responses that are difficult to replicate in field trials or test drives in the automotive sector.
The final component is the control PC. This takes over the control of the real-time system during test execution. The test cases are stored on the control PC in the form of scripts or models that can be executed. Furthermore, the control PC usually has interfaces to the bus systems (e.g., CAN, FlexRay, or Ethernet) to allow access to the embedded system during testing.
In Fig. 2 (see. PDF) is a typical HIL test system in 19″ construction.
Within the various test levels, such as those described in the V-model, HIL testing is classified as either software testing or system testing. This means that the software's behavior is tested in relation to the requirements. For safety-critical systems, this also includes all safety functions implemented in software. In practice, hardware-based safety functions are also included, as the actual hardware is used for testing.
This implies that the test system must be suitable for testing safety-relevant systems. Unfortunately, the relevant standards, such as ISO 26262, provide little information on what this means and how it can be demonstrated.
ISO 26262 states in section 8.4.1.5 of part 4:
„The test equipment shall be subject to the control of a monitoring quality system.“ [2]
In practice, implementations regarding this point vary widely. On the one hand, systems are considered permanently suitable after a one-time test during commissioning. On the other hand, cyclical maintenance or even tests before each test run are prescribed.
Another aspect to consider is the accuracy with which safety-relevant functions must be tested. For example, threshold values from analog signals (e.g., from sensors) are often used to detect a fault and trigger a fault response. Most HIL systems currently available on the market do not offer an official method for calibrating or adjusting the analog channels.
Problem statement
From the previous chapter State of the art The points outlined above present problems for the design and use of HIL test systems for testing safety-relevant systems and functions.
This work will focus on the accuracy and thus the reliability of the analog inputs and outputs of such a system. This is particularly relevant with regard to testing the fault detection of safety functions.
A typical signal path of an analog input of the test system, as shown in Fig. 3 (see Fig. 3). PDFThe diagram shown is intended to serve as an example of the problem. Analog inputs are usually used to measure control signals of the embedded system under test.
The analog signal is first routed through signal conditioning, which uses operational amplifiers to adjust the signal levels between the embedded system and the real-time system. Filters can also be inserted into the signal path to minimize interference from the system. The signal is then applied to the analog input of the real-time system, which consists of an input filter and an analog-to-digital converter (ADC). All parts of the signal chain are subject to errors. For example, the operational amplifiers have offset and slew errors. This also applies to the ADC.
Depending on the real-time system used, more or less information is available regarding the accuracy of the analog channels. Factory calibration or adjustment is only available or provided for a few I/O cards.
This poses a major problem for testing safety-relevant systems.
Investigation into the calibration and adjustment of HIL systems
Based on the information in chapter Problem statement The signal path shown was used in the work [3] to investigate the calibratability and adjustability of HIL test systems based on various real-time systems.
A real-time system from dSPACE with the analog input card DS2004 [4] and a real-time system based on the µTCA standard from NAT with the analog input card TPMC554 [5] from TEWS were used.
The first step involved identifying the available accuracy data from the datasheets. This revealed that the specifications for the DS2004 are extensive. In addition to information on rise and offset errors, data on temperature drift and aging are also provided. A disadvantage is that the card offers no hardware-based calibration option and is not calibrated at the factory.
Only a maximum total measurement error is specified for the TPMC554. The card is factory calibrated and can also be calibrated by the manufacturer. Furthermore, calibration factors for rise and offset errors can be passed to the card's hardware at runtime.
As part of this study, the actual error of the two maps and their temporal stability were determined using a voltage source and a calibrated multimeter (Fluke 8845A). The results showed that the measured maps were well within the specified error limits, and no significant deviation due to aging could be detected within the six-month study period.
In parallel, signal conditioning using the analog signal conditioning card [6] from iSyst was investigated. The signal path of this card consists of an input operational amplifier, a filter IC, and an output operational amplifier. Measurement series were performed with defined input voltages, measured with a Fluke 8845A multimeter, and the output voltage was determined in each case. The total error was found to be ±1 %. For example, an input voltage of 9.9858 V resulted in an output voltage of 10.064 V, which corresponds to an error of -0.785 %.
Further investigation revealed that a major part of the error was caused by an offset error in the operational amplifiers (op-amps), which could be largely eliminated by a variable resistor in the op-amp's circuitry. In later development, the op-amp was replaced with a type exhibiting a significantly lower offset.
Furthermore, the stability of the total error with respect to temperature and time was determined. According to the operational amplifier's datasheet, the temperature influence was < 1 % in the range of 0 °C – 40 °C. Aging effects could also be neglected over the measurement period of 6 months.
Finally, the system behavior of an iSyst HIL system [7] using a dSPACE system as a real-time system was investigated. For this purpose, an input voltage was applied directly to the pin of the ECU connector, and the measurements of the real-time system were evaluated. This allowed the influence of the system's wiring to be determined. As it turned out, the system's wiring had a significant influence on the measurements. Upon closer examination, however, the influences could be traced back to interference from adjacent signals, which were subsequently eliminated.
In summary, calibrating and adjusting the individual components already yields a significant improvement. However, to ensure reliability at the interface of the embedded system, calibration and adjustment starting at the control unit connector was deemed necessary. This can be done manually, but is very time-consuming. For this reason, extensive automation of the process was pursued.
Development of a concept for the semi-automated calibration of a HIL test system
The starting point for the analysis was the one described in chapter Problem statement The signal path described above. Since the desired concept should function in the same way for all real-time systems and I/O cards, it was decided to perform the calibration in the software, i.e., in the simulation model (Matlab/Simulink). This necessitates implementing the calibration and adjustment process in Matlab or Simulink as well. A Matlab implementation was chosen. The resulting system setup is shown in Fig. 4 (see below). PDF) to see.
An adjustable voltage source was required as the signal source. For easy integration with Matlab, control via a serial interface (RS232) was chosen. This interface is also used to control the Fluke 8845A multimeter. A custom-designed module was used, capable of outputting a voltage between -15 V and +15 V.
This voltage is then measured using the Fluke 8845A multimeter and read out via a serial interface using Matlab. This provides the value of the reference voltage. The measured value is then read from the commissioning model of the real-time system using Matlab. This provides the input and output values, allowing the deviation to be determined.
For the entire calibration and adjustment process, the offset error at 0V is determined first. Next, the slope error is determined within an adjustable range using a selectable number of data points (up to 16 data points in the range of -15V to +15V), and the correction factors are calculated. These results are saved in a report and a MAT file. The report serves as documentation.
Using the MAT file, the correction factors can now be automatically imported into the Simulink environment model with the help of the specially developed blockset. Only the selection of the MAT file within Simulink is necessary.
In Fig. 5 (see. PDF) shows the interface of the Matlab tooling for performing the calibration and determining the adjustment values.
In Fig. 6 (see. PDFThe blockset used for aligning the analog channels is shown. This blockset allows for the correction of offset and slew error.
Furthermore, the process can also be used for calibrating and adjusting analog outputs. In this case, the real-time system serves as the signal source, and the output voltage is measured at the ECU connector pin using the Fluke 8845A multimeter. The determined correction factors can be incorporated into the simulation model using the described Simulink blockset.
Evaluation of the prototype system
For the evaluation of the calibration and adjustment process, a HIL system with a dSAPCE real-time system and a DS2004 I/O card was used. First, the deviation of the analog channels was determined using the calibration process. The result for one channel is shown in Figure 7 (see Figure 7). PDF) can be seen. Subsequently, the calculated correction factors were incorporated into the Simulink model and the deviation was recalculated. The result is shown as an example for the same channel in Fig. 8 (see below). PDF) to see.
As can be seen from the values in the figures, a reduction in deviation and thus in measurement error from approximately +/- 0.8 % to approximately +/- 0.15 % was achieved. Further measurements after several hours of system operation have shown that the achieved accuracy is maintained. Further investigations regarding temporal stability and environmental influences (especially temperature) are necessary.
Further development towards automated self-testing
Based on the insights gained, further development is underway towards fully automated adjustment and fully automated self-testing. Unlike semi-automated calibration, where measurements are taken at the control unit connector, this requires measuring the signals within the HIL system itself. This is achieved using a dedicated, high-precision measurement card, which is connected to all measurement channels of the HIL system via the existing fault injection unit.
This makes it possible to automatically check the system before every quality-relevant test and thus ensure the reliability of the test system.
Summary
This work extensively investigated the feasibility of calibrating and adjusting HIL test systems. It demonstrated that calibration and adjustment are both possible and beneficial. Particularly when testing safety-critical systems and their failure thresholds, reliable analog measurement is essential for a test system.
Using the developed process and its associated components and tools, calibration and adjustment can be largely automated. Implementing the adjustment in the Simulink model ensures independence from specific real-time systems.
The automated creation of a calibration protocol makes it easier to document the properties of the test system in accordance with the process.
The further development to fully automated self-testing enables an additional increase in the reliability of the test systems. Furthermore, it allows for comprehensive verification of the test system's accuracy for every quality-relevant test, including automated logging.
List of sources
| [1] | ISO, „ISO 26262:2011 Road vehicles – functional safety“, International Standard ISO, Geneva, 2011. |
| [2] | ISO, „ISO26262:2011-4 Road vehicles: Functional safety- : Part 4: product development at the system level“, International Standard ISO, Geneva, 2011. |
| [3] | MA Mushtaq, „THE REQUIREMENTS OF CALIBRATED COMPONENTS FOR HIL (HARDWARE-IN-THE-LOOP) TEST SYSTEMS AND ITS IMPLEMENTATION“, Master's thesis – Ernst-Abbe-Hochschule, Jena, 2015. |
| [4] | dSPACE, „DS2004 High-Speed A/D Board,“ dSPACE, 2017. [Online]. Available: https://www.dspace.com/de/gmb/home/products/hw/modular_hardware_introduction/i_o_boards/ds2004_high_speed_a_d_board.cfm. [Accessed 10/10/2017]. |
| [5] | TEWS, „TPMC554 32 /16 Channels of 16 bit D/A with FIFOs,“ TEWS, 2015. [Online]. Available: https://www.tews.com/products/ArticleGroup/TPMC/TPMC554.html. [Accessed 10/10/2017]. |
| [6] | iSyst Intelligente Systeme GmbH, „Test Components“, iSyst Intelligente Systeme GmbH, 2017. [Online]. Available: https://isyst.de/produkte/testkomponenten/. [Accessed 10 October 2017]. |
| [7] | iSyst Intelligente Systeme GmbH, „Hardware In The Loop Test Systems“, iSyst Intelligente Systeme GmbH, 2017. [Online]. Available: https://www.isyst.de/produkte/hil-testsysteme/. [Accessed 10 October 2017]. |
author
Dr.-Ing. Kristian Trenkel Mr. Trenkel studied Electrical Engineering/Computer Science at the Jena University of Applied Sciences from 2001. After successfully completing his studies with a degree in Engineering (Dipl.-Ing. (FH)) in 2005, he worked as a development engineer in the field of industrial automation. From 2008, he worked as a test engineer at iSyst GmbH while simultaneously pursuing a cooperative doctorate with the Chemnitz University of Technology, Chair of Systems Engineering and Automation (SSE). Since 2013, Mr. Trenkel has been supervising research and funding projects as well as student theses at iSyst GmbH. He successfully completed his doctorate at the end of 2015.
Automotive – our training & coaching
Do you want to bring yourself up to date with the latest technology?
Then find out more here MircoConsult offers training courses/seminars/workshops and individual coaching on the topic of automotive/embedded and real-time software development.
Training & coaching on the other topics in our portfolio can be found here. here.
Automotive – Expertise
Valuable expertise in automotive/embedded and real-time software development is available. here Available for you to download free of charge.
You can find expertise on other topics in our portfolio here. here.
