Select Page

Automatic multi-core real-time validation

A process for model-based software development

Authors: Felix Martin, Andreas Sailer, Michael Deubzer, Timing-Architects Embedded Systems GmbH, Jürgen Mottok, Ostbayerische Technische Hochschule

Contribution – Embedded Software Engineering Congress 2015

The trend toward using multi-core processors to meet the increasing performance demands of embedded real-time systems is clearly evident, as are the associated challenges regarding timing behavior, data dependencies, performance, and reliability. This paper presents a model-based approach, using the open-source real-time operating system Erika Enterprise as an example, to monitor compliance with safety-critical real-time requirements through tracing across the entire development process.

Introduction

A model-based development approach allows compliance with system-critical requirements and constraints to be ensured as early as the planning phase of software projects [1]. For this purpose, a model is created that describes the temporal behavior of the system.

Such a model consists of a hardware component, an operating system component, and an application component. The hardware component abstractly describes the behavior of the processor on which the final application is to be executed. The operating system component models the behavior of service routines, such as the number of instructions required for inter-process activation. The application model represents the actual functionality of the application. Real-time critical software systems are composed of various tasks, which in turn call application-specific functions called runnables.

Based on such a model, requirements regarding the desired timing behavior can then be defined. Requirements can be specified for all objects defined in the application part of the model. A classic example of this is the maximum response time of a task, i.e., the time from its activation until its termination. Requirements for data dependencies can be defined in the form of data age constraints [2]. Furthermore, so-called event chains allow time requirements for the execution of any sequence of system events to be defined.

Using discrete event simulation, it is now possible to calculate the runtime behavior of the final application based on such a system description. The result is a so-called trace – a sequence of events that reflects the real-time behavior of the system. This trace can then be evaluated and compared with the defined requirements, as shown in Figure 1 (see PDF) is shown. This allows errors in the system design to be identified and corrected early.

Even when an application enters the development phase, the defined requirements can still be validated. This requires recording the software's behavior on the target platform – a process known as tracing or trace recording. The first part of this article demonstrates the various methods for recording traces from the hardware. Recorded tasks are typically not in a format suitable for validating real-time requirements. Therefore, the second part explains the necessary steps for converting the data into a suitable format. Finally, it describes how these steps can be automated using the model.

Tracing

Tracing allows the behavior of embedded systems to be recorded by the hardware. As with simulation, the result is a trace, i.e., a series of events representing different occurrences in the system. According to Ferrari [3], there are three principal approaches: software-based, hybrid, and hardware-based tracing.

Software-based tracing involves adding special code to the application. This so-called instrumentation ensures that events relevant to runtime behavior analysis are logged in memory with their corresponding timestamps. After execution, the trace can be read via a debugger or serial interface. However, this approach alters the application's runtime behavior [4]. Furthermore, the maximum possible trace duration is limited by the available memory on the hardware.

Hybrid tracing also relies on the application's instrumentation, thus altering its runtime behavior. Unlike the software-based approach, the recorded events are sent directly via a dedicated interface for further analysis. This means that, with sufficient bandwidth, the maximum possible trace duration is not limited by available storage space.

Hardware-based tracing, also known as "on-chip tracing," does not require software instrumentation. Instead, a dedicated area on the processor is used to record the CPU's instruction execution and data transfers. These events can be filtered according to user-defined rules and then sent from the chip via a special hardware trace interface. Using a suitable high-performance interface such as the AURORA interface, recordings of arbitrary length are possible [5].

Hardware-based tracing is the only method that allows for meaningful validation of real-time requirements on the hardware. As mentioned, instrumentation-based approaches alter runtime behavior. Therefore, especially in heavily loaded applications, requirements that are normally met may no longer be fulfilled due to the additional computational overhead caused by instrumentation. Furthermore, adding and maintaining instrumentation to the application represents an additional development effort.

Hardware-based tracing can generate three types of traces: instruction, data, and signal traces [6]. Instruction traces consist of instructions executed by the CPU. Together with debug information, the complete program path of the application can be reconstructed from these. Data traces contain memory accesses by the CPUs, such as read and write accesses to application variables. Signal traces contain information about the behavior of peripherals, such as the Generic Timer Module (GTM) of the Infineon AURIX.-Processor family.

The following listing shows the textual representation of a data trace event, exported to the tool TRACE32 Lauterbach GmbH [7].

0.0013952200,EE_th_status[0],"wr-data",00000003,0

Each event consists of five fields. The first corresponds to the timestamp at which the event was recorded. The time unit is seconds. The second field represents the variable affected by the data access. The third field indicates whether a read or write access occurred. In this case, it is the latter ("write-data"). The fourth field indicates the value that was written to the variable. The last field shows the ID of the processor core on which the event took place.

The following listing shows the textual representation of an instruction trace event, also exported from TRACE32.

+1408860; R_2_1; fstart; 0

The first field again corresponds to the time at which the event was recorded. This time, the time is specified in nanoseconds. The second field indicates the name of the software object affected by the event. The third field specifies which action is performed for the software object; in this case, the function is started. R_2_1. The last field again represents the processor core on which the event took place.

Assignment

The introduction described how requirements for system objects, such as tasks and signals, can be defined. In contrast, events recorded during hardware-based tracing refer to software objects, i.e., entities defined at the source code level, such as variables and functions.

To use hardware-based traces for validating real-time requirements, it is therefore necessary to transform the traces from software level to system level, as shown in Figure 2 (see PDF) is illustrated. For this, the various system events must be mapped to the corresponding events at the software level. Furthermore, a suitable format, such as the Best Trace Format (BTF) [8], is necessary to represent the trace at the system level.

To illustrate the transformation, the data trace event from above will be used. This event indicates that after 1395220 Nanoseconds, the value 3 is entered into the first field of the array. EE_th_status on the core with the ID 0 This statement cannot be used to evaluate real-time requirements, however, because additional information is needed to interpret the event at the system level. The Erika Enterprise operating system stores the state of each task in a field of the EE_th_status Arrays. The array with the index 0 In this example, refers to the task T_IMS_0. In conclusion, the event's message can be reformulated as follows: 1395220 The task was completed in nanoseconds T_IMS_0 on the core 0 terminated.

The OSEK Run Time Interface (ORTI) was specified to facilitate the mapping of software to system objects for OSE-compliant operating systems. For example, it allows the automatic detection of how the operating system stores task states. This means the user doesn't necessarily need to know that EE_th_status is used for this purpose. Instead, this information can be read from the ORTI file.

However, ORTI does not necessarily provide information for all system objects that are relevant for analyzing the real-time behavior of systems. For example, runnables and signals are not covered by ORTI. In this case, it is necessary to provide this information via other means. Otherwise, mapping and thus transformation for certain objects is not possible. The instruction trace event from the example above represents the start of the function. R_2_1. Only if it is known at the time of the transformation that this function corresponds to a Runnable at the system level can the correct system event be created.

Even if an object type – such as Tasks – is supported by ORTI, not all events can necessarily be reconstructed at the system level. For example, BTF defines a mtalimitexceeded An event that occurs when a task is activated for which no further activation is permitted. (About ORTI) lasterror The attribute allows us to detect these events, but it's not possible to determine which task triggered the event. In this case, additional information about how the operating system works is needed. Erika Enterprise reads the variable EE_th_rnact, to determine whether further task activation is permitted. If not, the corresponding error code is displayed directly after the read event. lasterror written. If this behavior is known, the event can be reconstructed despite the lack of ORTI support.

Based on these examples, in principle all events specified by BTF can be transformed from software to system level. Depending on which requirements are to be validated, more or less detailed knowledge of the operating system's functionalities is necessary.

automation

As the previous sections have shown, requirements validation via tracing is not trivial. First, a suitable tracing technology must be selected, and then the generated traces must be correctly interpreted to enable real-time analysis. A model-based development approach can support both tasks.

To record a correct trace, the trace hardware must be configured correctly. This depends on both the processor and the operating system. The system model mentioned above contains information about these two aspects. This means that the correct trace configuration can be generated automatically based on the model. For example, the correct processor type can be determined from the hardware component, and the correct data trace configuration can be derived from the operating system component.

Furthermore, the application code itself can be automatically generated from the model. This means that as soon as a model is created, an initial prototype of the application can be tested on real hardware. Functions that are not yet implemented are replaced by function stubs, which consume runtime according to the configuration in the model. As soon as new functional components are implemented, the corresponding stubs are replaced. Based on the timing behavior of the real functions, the model can be iteratively improved. This approach allows for continuous comparison between the model and the real application, and consequently, continuous monitoring of the requirements.

With an automatically generated application prototype and the corresponding trace configuration, which was also automatically generated, hardware traces can be recorded fully automatically. The trace then needs to be transformed at the system level to perform a timing analysis. Some of the necessary information can be extracted from the model. Based on the model, it can be determined which functions are runnables and which variables are signals. Accordingly, events are created for all objects relevant to the real-time analysis.

Finally, the generated trace is evaluated and compared with the specified requirements. Based on the events, a report can be created that reflects the current status of the application. If the timing behavior does not meet the specified requirements, this can be determined from the report, and the system design can be improved. After the improvement, the process is repeated until the system behavior meets the requirements. Figure 3 (see PDF) visualizes this process.

Summary and Outlook

A model-based development approach allows embedded multi-core applications to be analyzed for their temporal behavior as early as the design phase. This is possible without committing to a specific processor or operating system. This paper demonstrates how the requirements defined and validated via simulation during the design phase can be further monitored during the implementation phase. This requires recording system behavior using hardware-based tracing and translating the resulting trace into a suitable format. Based on a model, these steps can be automated.

Modern multi-core processors support three types of tracing. Currently, only two types are used for real-time validation: data tracing and instruction tracing. The ability to trace peripheral hardware, such as timer units or CAN modules, via signal tracing could be used in the future to monitor the collective behavior of multiple processors. This means that individual processors are not considered in isolation, but rather the communication between the distinct units is also included in the analysis of real-time behavior.

List of sources

[1] M. Deubzer, A. Stingl, E. Simsek, S. Krämer, J. Mottok, „High-performance and reliable embedded multi-core systems“, ESE Kongress, Sindelfingen, 2014.
[2] M.-A. Peraldi-Frati, A. Goknil, J. DeAntoni, J. Nordlander, „A timing model for specifying multi clock automotive systems: The timing augmented description language v2,“ in Engineering of Complex Computer Systems (ICECCS), 2012 17th International Conference on Engineering of Complex Computer Systems, Paris, IEEE, 2012, pp. 230-239.
[3] D. Ferrari, Computer systems performance evaluation, Prentice Hall, 1987.
[4] J. Kraft, A. Wall, H. Kienle, „Trace recording for embedded systems: Lessons learned from five industrial projects,“ in Runtime Verification, Springer, 2015, pp. 315-329.
[5] A. Martin, „Streaming trace data to the hard drive“, ESE Kongress, Sindelfingen, 2013.
[6] J. Braunes, „The new trace world beyond code and data trace“, ESE Kongress, Sindelfingen, 2013.
[7] Lauterbach GmbH, „TRACE32 – General Commands Reference Guide T (Accessed: 2015-10-19)“, Höhenkirchen-Siegertsbrunn, 2015.
[8] Timing Architects Embedded Systems GmbH, „BTF Specification„, AMALTHEA ITEA2 Project.
[9] Timing Architects Embedded Systems GmbH, „TA Tool Suite – TA Inspector„", (Accessed: 2015-10-20), Regensburg, 2015.

Download the article as a PDF


Multicore – our training & coaching

Do you want to bring yourself up to date with the latest technology?

Then find out more here MircoConsult offers training courses/seminars/workshops and individual coaching on the topic of multicore/microcontrollers.

Training & coaching on the other topics in our portfolio can be found here. here.


Multicore – Expertise

Valuable expertise in modeling/embedded and real-time software development is available. here Available for you to download free of charge.

To the specialist information

You can find expertise on other topics in our portfolio here. here.

MicroConsult Newsletter

With the MicroConsult newsletter, you'll stay on the pulse of the embedded world. Look forward to proven practical knowledge, real professional tips, and current events – directly from our experts for your project success.

Subscribe now!

Published by

weissblau media

weissblau media