EN 
DE Observations and experiences at Bosch Engineering GmbH
Authors: Fred Härtelt and Dr. Klaus Herz, Bosch Engineering
Contribution – Embedded Software Engineering Congress 2015
Testing and test management in ECU development are becoming increasingly important due to the growing complexity and system variance, in order to, on the one hand, The goal is to enable cost-optimized product development without technical debt and, conversely, to identify software quality risks early on. Both objectives are addressed by implementing a holistic test management system that complies with common norms and standards (e.g., ISO/IEC/IEEE 29119, ISO 26262, ASPICE).
Initial situation
As a systems development partner to the automotive industry, Bosch Engineering GmbH offers development services for powertrains, safety, comfort, and E/E systems, from concept to series production. The company's diverse fields of application range from passenger cars and commercial vehicles to off-highway applications, rail and recreational vehicles, ships, and industrial applications. The Powertrain Systems division's project landscape is characterized by a wide variety of systems exhibiting high complexity in system and software development and data application. This, in turn, necessitates a tailored approach for each project to adequately cover testing and test management. This is achieved through the application of a "three-pillar model," which is implemented as a holistic approach across the organization. This model is designed to consider universal framework conditions for the various relevant aspects (including frontloading, legal and internal requirements, organizational structure, available test methods and systems, etc.) and thus provides clear guidance for product-based tailoring.
The „three-pillar model“
A holistic test management approach is characterized by several aspects in order to cover all relevant tests within a project cycle. The first pillar is the Test strategy, which specifies how the project will proceed in concrete terms. The second pillar forms a suitable Test methodology, which are based on the Test strategy builds up. The Test methodology This primarily includes a selection of different test procedures and test systems. The third pillar is test management within the projects. The central role here is played by the Test Manager one. He is responsible for creating a suitable Test strategy and Test methodology for the respective project.
Pillar I – the testing strategy
The project-specific selection of a suitable Test strategy This is crucial for deriving cost-effective tests – tailored to the specific context. Therefore, a corresponding generic [approach/method] was developed. Test strategy Designed to be adaptable to a wide variety of ECU applications and project configurations, it is based on the "Organizational Test Strategy" according to ISO/IEC/IEEE 29119-1 and defines, for example, test methods and procedures. This forms an important basis for selecting the Test strategy The next step is defining the test objectives. These were defined generically as follows:
a) Test objective 1: Ensuring compliance with the contractually guaranteed functional requirements
b) Test objective 2: Proof of compliance with the agreed safety objectives
c) Test objective 3: Proof of compliance with the defined boundary conditions (e.g., implementation of the quality requirements)
From these test objectives, specific test criteria are derived with regard to the respective test level (e.g., system, integration, software, etc.). These test criteria, in turn, lead to the following: Test methodology selected and a test plan according to the Test strategy is carried out.
Pillar II – the selection of a suitable testing methodology
Selecting a suitable Test methodology This includes specific test procedures assigned to the respective test level. Suitable test procedures can be, for example, white-box or black-box testing, which are selected on a project-specific basis and influence the choice of a suitable test system [1]. While PC-based test systems are more commonly used at the software level, hardware-in-the-loop (HIL) simulators or vehicles are primarily used at the system level. However, low-cost test systems [2] can also be a viable alternative for conducting comprehensive tests in an early development phase.
In Figure 1 (see PDFTest objective 2 ("Verification of compliance with the agreed safety objectives") is assigned to a test level ("System"). This is intended to verify the safety objectives of a component. Relevant test procedures in this example include the application of a requirements-based test and fault injection testing. Specifically, the safety objectives are broken down into safety requirements and tested using various black-box test cases on the vehicle or in a hardware-in-the-loop (HIL) simulator. The requirements of the functional safety standard (ISO 26262-4) are thus addressed at the system level with the respective safety integrity level (ASIL).
Each test objective can therefore be assigned at least one test level or several test levels, test areas, test procedures, and test systems. The selection of Test methodology On the one hand, this ensures that all test objectives are covered – on the other hand, the project context is crucial at this point in order to be able to select the appropriate test procedures and test systems in a targeted manner.
Pillar III – the test manager
The use of a Test Manager In the project, it is crucial to have a tailored approach. Test strategy and Test methodology to be able to derive. He deliberately chooses from a toolbox of generic Test strategy the elements needed for the project. Test Manager is also responsible for test planning, monitoring all test activities and reviews, analysis and reporting, as well as coaching project members to raise their awareness of test procedures and systems. He ensures the execution and evaluation of all relevant tests and ultimately issues a release recommendation.
In addition to existing project experience and specific knowledge of the vehicle domains, the technical depth of the Test Manager A key success factor. A suitable training concept ensures that the necessary competencies regarding testing and test management are present – for example, through various training courses covering the role itself, including testing procedures and test systems.
This role has already been implemented in various projects at Bosch Engineering GmbH. The early clarification of testing requirements (keyword "frontloading") and the continuous monitoring of testing activities within the project were particularly well-received. Customer feedback—specifically, the importance of having a designated contact person for testing and test management—was also rated very positively.
Summary
Testing and test management are becoming increasingly important due to the growing complexity of project landscapes and diverse customer requirements. The holistic test management approach presented here, based on a three-pillar model, is a targeted method for achieving the desired test objectives.
literature
[1] Fred Härtelt: „Development of a practical course for white-box tests using a test tool“ (Technical University of Dresden, Faculty of Computer Science, July 2003)
[2] Dr. Petra Langjahr, Dr. Stefan Müller: „Does it always have to be a HIL system?“ (Embedded Software Engineering Congress, December 2011)
Testing, Quality & Debugging – Our Training & Coaching
Do you want to bring yourself up to date with the latest technology?
Then find out more here MircoConsult offers training courses/seminars/workshops and individual coaching on the topics of testing, quality & debugging.
Training & coaching on the other topics in our portfolio can be found here. here.
Testing, Quality & Debug – Expertise
Valuable expertise on the topics of testing, quality & debugging is available. here Available for you to download free of charge.
You can find expertise on other topics in our portfolio here. here.
