EN 
DE A question of safety or security – or both?
Authors: Stefan Kriso, Jürgen Klarmann, Claudia Loderhose, Franziska Wiemer, Carsten Gebauer, Simon Burton, Robert Bosch GmbH; Markus Ihle, ETAS GmbH
Contribution – Embedded Software Engineering Congress 2018
For safe automated movement in road traffic, environmental monitoring by vehicle sensors is essential. A specific class of hazards is currently not adequately addressed – "environmental hacks," meaning manipulation of the environment. For example, a stop sign must be reliably recognized. A relatively minor manipulation of a traffic sign, not necessarily perceptible to the human eye, can lead to misclassification, which in turn can endanger people. Therefore, the system must be robust against such manipulations. Similar effects can also occur due to dirt or grime on the traffic sign. This must also be considered when assessing the safety of the nominal function. The question is how to address the issue of "environmental hacks," as it affects both security (manipulation) and safety (impact). This article uses examples to demonstrate how this question can be answered.
Motivation and examples
Safety, in the sense of security, has various „enemies“ that are addressed through different methods and measures.
See image 1 (PDF): „Enemy images“ of automotive safety (selection)
Systematic and random hardware failures that can lead to faulty system behavior are addressed by "functional safety," and the corresponding methods and measures are described in ISO 26262. A standard for the safety of the intended functionality (SOTIF – Safety of the Intended Functionality) is currently under development: ISO 21448. This standard addresses the question of how, for example, a safe system can be designed while considering physical limitations of the sensors. Deliberate manipulation of the system can also affect safety. This is addressed through cybersecurity measures; a standard for this is also in preparation (ISO/SAE 21434).
Furthermore, there is another „enemy image“, the Environmental Hacks, which we will introduce using three examples:
While the manipulation of traffic signs may still be recognizable as such to the human observer, and the manipulated sign may still be recognizable as such to the driver in its original intention, this is not necessarily the case with automated driving. Even a small but targeted manipulation can lead to misclassifications in image recognition algorithms ([3]), resulting in the traffic sign being "overlooked" by the system.
In [4] a manipulation was even demonstrated in which a system misinterpreted a manipulated stop sign as a speed limit sign.
See image 2 (PDF): Example of a manipulated traffic sign [1]
Another example of environmental hacks are laser attacks, which are occasionally mentioned in the press ([5]). While these reports usually focus on attacks against human drivers, such attacks also affect sensors and can ultimately lead to failure or malfunction.
This attack thus leads to a "Denial of Service" and can therefore be classified as a DoS attack, even if the source of the disruption is outside the system.
Jamming and spoofing GNSS (Global Navigation Satellite System) signals represent further forms of environmental manipulation. Jamming involves preventing the reception of GNSS signals using a jamming device. GNSS spoofing, on the other hand, refers to the transmission of falsified GNSS signals. These are not detectable as forgeries by sensors alone, as they are inherently valid.
GNSS jamming and spoofing can be used deliberately to influence vehicle behavior. However, there are also scenarios in which GNSS jamming or spoofing is directed against other targets, and the vehicle only accidentally enters the area of influence of jammed or falsified signals.
In [13], a case is reported in which a truck driver used a GPS jammer to disrupt the vehicle's built-in GPS tracker and thus conceal his location from his supervisor. This inadvertently disrupted a GPS-based aircraft guidance system at a nearby airport. These effects can be extrapolated to vehicles with GPS-based functions.
[14] describes a GPS spoofing attack that can unintentionally affect vehicles with GPS-based functions. Using a software-defined radio, fake GPS signals are generated, which trick the app „Pokemon Go“ into believing the player has a false location. This allows the player to play from home without having to go for a walk.
The use cases for GNSS spoofing are diverse. The same approach can be used, for example, to illicitly activate location-based services or to circumvent GPS-based toll systems. [12] has recorded an increase in GNSS jamming and spoofing. Therefore, it is important to consider and analyze it in the context of GNSS-based vehicle functionalities.
Relevance of Environmental Hacks for Safety
As the examples clearly show, environmental hacks can affect the safety, availability, and performance of the product. For safety, an environmental hack would be another "fault pattern" to consider (Figure 1, (PDF).
The availability and performance aspects of Environmental Hacks no longer focus on product safety, but rather on user acceptance: In our example, the relevant questions would be whether a user would accept a self-driving vehicle frequently stopping unexpectedly or making wrong turns, or whether they would still consider it acceptable if an automated vehicle could be disabled by a laser pointer costing 10 euros.
Environmental Hacks: Safety or Security?
The main threats in functional safety are systematic errors and random hardware failures. To avoid systematic errors, requirements are placed on the development process ("safety lifecycle"), while to manage random hardware failures and remaining systematic errors (whose complete avoidance cannot always be guaranteed), requirements are placed on the technical implementation in the product (e.g., redundancies, monitoring, diagnostics).
Let us now leave the failure modes of functional safety, namely systematic and random hardware failures, and consider another threat: the deliberate attack on the vehicle or its E/E systems. First, it should be noted that this threat lies outside the scope of functional safety and therefore outside the application of ISO 26262. Nevertheless, this threat can also lead to safety-critical behavior and is therefore, in principle, safety-relevant [8]. This threat is addressed by cybersecurity measures, which are currently being standardized in ISO/SAE 21434 "Cybersecurity Engineering".
See image 3 (PDF): Basic procedures for safety and security
Cybersecurity often employs cryptography as a typical countermeasure, but this appears to be ineffective in the case of environmental hacks: Cryptographic measures encrypt or authenticate the path from the input to the output of a digital function. However, the manipulation in environmental hacks occurs beforehand (before the input), rendering this security measure useless. A single input channel cannot be used to assess whether the input is valid or not.
It can therefore be concluded that environmental hacks are neither captured by the safety analysis, because they do not correspond to the typical error pattern, nor by the security analysis, since the perspective there is usually only the digital system itself and not its environment.
Typical safety measures such as redundancy, diversity and plausibility checks may well be promising, but they are not used if they are not derived as such in the safety analysis.
Approach to the development process
Upon closer examination of the requirements for the development process, many similarities between the safety and security activities can be found [8]:
See image 4 (PDF): Similarities in safety and security processes
However, when it comes to environmental hacks, neither the safety nor the security processes alone are sufficient, as neither adequately addresses this threat. This leads us to a more comprehensive, process-oriented approach:
See image 5 (PDF): Procedural approach to dealing with environmental hacks
- Expansion of the scope (TOE = Target of Evaluation [9]) of the risk and threat analysis to include the potential impacts of environmental hacks. This usually falls within the remit of the security domain.
- The subsequent analysis leads to further top-level requirements (security goals).
- Transfer of requirements to the appropriate safety target domain. Environmental manipulations often only become relevant to vehicle behavior when they are detected by the vehicle's sensors and incorporated into its algorithms. Therefore, a transfer to the SOTIF (Safety of the Intended Functionality) domain is typically expected, but other domains such as Functional Safety may also be affected.
- The implementation then takes place there, using the measures that are available in each case.
- Testing, verification and validation are carried out at the overall system level, generally in accordance with the specifications of the security domain.
When transferring requirements to the functional safety domain, it is important to note that the ASIL according to ISO 26262 cannot be used to assess the safety risk, as it implicitly assumes statistical independence between fault occurrence and driving situation [11], which is not necessarily the case with an environmental hack. Therefore, a different criterion is needed to assess the safety risk of external manipulation, which is currently the subject of discussion in the safety and security communities [10].
Exemplary implementation
a) Traffic signs
The original form of the question regarding the recognition of traffic signs, even with modifications, is already addressed in product development. Even without deliberate manipulation, it cannot be assumed today that, for example, a stop sign only appears in its purest form and is found in the real environment without any deviation from a reference sign. Instead, it must be assumed that, for example, the sign might be covered by snowflakes in winter or only partially visible due to dirt. Furthermore, it can be expected that a system in assisted or automated driving, even with minor modifications, will still be able to recognize the sign as such and classify it correctly.
Solutions to this problem are being developed within the SOTIF domain. Expanding the focus from snowflakes and dirt particles to include adhesive strips or covers broadens the scope of this topic to include deliberate environmental manipulation, thus identifying SOTIF as the target domain for environmental hacks.
Weather-related deviations are taken into account in image recognition and are already part of the training phase of a "deep neural network". One approach to extending the problem-solving capabilities to include environmental hacks is to deliberately introduce such manipulated objects into the deep learning process in order to sensitize the algorithms to these variations and improve their robustness.
However, countermeasures already used in the SOTIF environment can also be adapted for this class of attack: For example, additional validation measures based on context are a promising countermeasure. It would not be plausible, for instance, if a stop sign were placed on a crossroads-free country lane in the forest, or if a 60 km/h speed limit sign were to appear directly at a traffic intersection.
Another effective countermeasure, which is also applicable in the SOTIF context, would be, for example, the establishment of additional infrastructure (traffic signs emit diverse sensor signals) or a comparison of the recorded traffic signs with map information.
b) Laser pointer
The safety relevance of glare from laser pointers is well known, for example in aviation, and is also penalized accordingly ([5]). So far, the focus has been primarily on the glare affecting people. However, it is also conceivable that sensors could be glared, thus impairing their ability to detect the vehicle's surroundings.
In principle, this error pattern should already be considered in a similar manner within the SOTIF framework, as natural sources of glare are also conceivable, for example, a low sun. This, too, must not lead to undesirable, safety-critical vehicle behavior. Therefore, it makes sense to refer the question of glare from laser pointers to the SOTIF domain, as measures for the general prevention or control of this error pattern can be expected there if the considerations made there are extended to the frequency ranges of laser pointers.
Possible solutions include redundant sensors, maintaining a minimum level of functionality, or handing over the driving task to the driver.
c) GPS spoofing
At a technical level, measures are available to counter the effects of GNSS jamming and spoofing. At the vehicle level, GNSS jamming has a similar effect to a GNSS sensor failure. Scenarios of this kind are part of the safety considerations and can be mitigated by safety measures, such as switching to a redundant channel for position determination or degrading the vehicle to a state that does not require GNSS data.
The effects of GNSS spoofing are more difficult to prevent because the GNSS sensor cannot distinguish fake GNSS data from genuine data. However, there are measures the system can take to detect an irregularity. A sudden, significant offset in the received position data can be detected through continuous plausibility checks of the received data. A gradual offset becomes apparent when compared with redundant position data, such as map data.
At the process level, however, classifying GNSS jamming and spoofing is difficult. It falls outside the scope of functional safety, as it is not caused by systematic errors or random hardware failures. Nor are such attacks relevant to SOTIF (Safety of Technology in the Event of Infection), because they are not based on inherent deficiencies in the sensors themselves. Currently, the handling of GNSS jamming and spoofing is not fully covered by security, as risk and threat analysis only considers (intentional) attacks on the vehicle or individual components. The collateral damage that can result from GNSS jamming or spoofing with other objectives is overlooked. Therefore, a target domain must be identified, which may need to expand its scope and potentially its methodologies to analyze such attacks and derive appropriate countermeasures.
Summary
Environmental hacks can have a significant impact on vehicle behavior and its safety (both in terms of safety and security). This is demonstrated by the disciplines discussed so far (Figure 1).PDFThis is not fully addressed, as it is a cross-domain issue. The identification of the affected error patterns can be carried out within the framework of a security analysis, followed by the transfer to the respective relevant safety target domains. For this purpose, the existing security risk and threat analysis should be expanded to include environmental hacks. The Safety of the Intended Functionality (SOTIF) will often be a suitable target domain for the safety analysis, as this is where the greatest synergies are expected; however, other safety target domains are also conceivable.
References
[1] https://upload.wikimedia.org/wikipedia/commons/4/41/Stop_eating_animals_sign.jpg
[2] Autonomous cars: Incorrect traffic signs completely misinterpreted, www.winfuture.de/news,99034.html (accessed on 24.07.2018)
[3] Robust Physical-World Attacks on Deep Learning Models,
https://arxiv.org/abs/1707.08945 (accessed on 30.07.2018)
[4] https://www.youtube.com/watch?time_continue=2&v=1mJMPqi2bSQ
(accessed on 30.07.2018)
[5] Prison sentence for laser blinding,
https://www.aerokurier.de/general-aviation/haftstrafe-fuer-laser-blendung-in-berlin/738030 (accessed on 27.09.2018)
[6] Laser pointer attack: Bus driver injured in the eye
https://www.ln-online.de/Lokales/Luebeck/Anschlag-mit-Laserpointer-Busfahrer-am-Auge-verletzt (accessed on 24.07.2018)
[7] Black Hat Europe: It's easy and costs only $60 to hack self-driving car sensors,
https://www.computerworld.com/article/3005436/cybercrime-hacking/black-hat-europe-it-s-easy-and-costs-only-60-to-hack-self-driving-car-sensors.html (accessed on 24.07.2018)
[8] Security in the context of functional safety: How safety and security are related, Embedded Software Engineering Congress 2015, Sindelfingen
[9] ISO/IEC 15408: “The Common Criteria for Information Technology Security Evaluation“
[10] Benjamin Glas, Carsten Gebauer, Jochen Hänger, Andreas Heyl, Jürgen Klarmann, Stefan Kriso, Priyamvadha Vembar, Philipp Wörz: „Automotive Safety and Security Integration Challenges“, Automotive Safety and Security 2015, Stuttgart-Feuerbach
[11] Stefan Kriso, Markus Ihle: “Automotive Security in the Context of Functional Safety“, Forum Safety & Security, Munich, July 6–7, 2016
[12] GPS under attack: Jamming and spoofing are on the rise
https://www.heise.de/newsticker/meldung/GPS-unter-Beschuss-Jamming-und-Spoofing-nehmen-zu-4038137.html (accessed on 27.09.2018)
[13] Truck driver has GPS jammer, accidentally jams Newark airport,
https://www.cnet.com/news/truck-driver-has-gps-jammer-accidentally-jams-newark-airport/ (accessed on 27.09.2018)
[14] We Declare The Grandmaster Of Pokemon Go GPS Cheats
https://hackaday.com/tag/gps-spoofing/ (accessed on 27.09.2018)
authors
Dipl.-Phys. Stefan Kriso After studying physics, he joined Robert Bosch GmbH in 1995. In various positions, he dealt with issues related to hardware and software development. From 2005 to 2011, he led a project in central research and advanced development that involved representing Bosch's interests in national and international standards bodies for ISO 26262 and coordinating the company-wide implementation of ISO 26262. Since 2011, he has headed the "Center of Competence Functional Safety" at Bosch. Since 2005, he has also been a part-time lecturer in physics at the Baden-Württemberg Cooperative State University in Stuttgart.
Contact: stefan.kriso@de.bosch.com
Dr. Jürgen Klarmann He studied computer science at the University of Stuttgart, graduating with a diploma in 1995. From 1995 to 2002, he worked as a research assistant at the University of Stuttgart, where he also earned his doctorate (Dr. rer. nat.). Since 2002, he has worked at Robert Bosch GmbH in various roles within the Mobility Solutions division. Since 2008, he has been increasingly involved in functional safety and ISO 26262. From 2013 to 2015, he worked at ETAS GmbH as a senior consultant for safety, security, and embedded systems. Since 2016, he has worked in the Chassis Systems Controls division at its "CC Center of Competence Security.".
Contact: juergen.klarmann@de.bosch.com
Dr. Claudia Loderhose She holds a degree in mathematics and a doctorate in computer science. Since 2017, she has worked at Robert Bosch GmbH in the Chassis Systems Controls division. Her areas of expertise include methods for risk and threat analysis, the integration of security aspects into the product development process, and the interplay between safety and security.
Contact: claudia.loderhose@de.bosch.com
Franziska Wiemer She studied IT Security and Information Security (B.Sc. and M.Sc.) at Ruhr University Bochum and joined Robert Bosch GmbH in 2017. Since then, she has focused on IT security within vehicles. Her work includes developing processes for integrating security into electronic control units (ECUs) – the interplay between safety and security currently plays a significant role in this work. Another focus of her work is data protection within chassis control units, both during development and during data feedback in later series production.
Contact: franziska.wiemer@de.bosch.com
Dipl.-Phys. Carsten Gebauer He joined the Bosch Group in 2000 after completing his studies. Since 2004, his focus has been on safety. He is a member of the national and international working groups tasked with developing ISO 26262 and ISO (PAS) 21448 and has worked at the Bosch Center of Competence Functional Safety since 2014.
Contact: carsten.gebauer@de.bosch.com
Dr. Simon Burton Dr. Burton studied computer science at the University of York, where he also earned his doctorate in the field of verification and validation of safety-critical systems. He has over 20 years of experience in various safety-critical industries. His experience includes leading research and development projects at major OEMs as well as consulting, service, and product organizations. Currently, he serves as Chief Expert at Robert Bosch GmbH, where he coordinates the research strategy in the areas of functional safety, security, reliability, and availability of software-intensive systems.
Contact: simon.burton@de.bosch.com
Markus Ihle He studied electrical engineering at the University of Karlsruhe and graduated with a diploma in 1999. Since 2003, he has worked in various departments and functions within the Bosch Group. His responsibilities have included research and development of semiconductor components and software for embedded products, such as in-vehicle communication (CAN, FlexRay) and security modules for enhanced tamper protection of microcontrollers (HSMs). Since 2013, he has headed the Bosch Center of Competence Security at ETAS.
Contact: markus.ihle@etas.com
Automotive – our training & coaching
Do you want to bring yourself up to date with the latest technology?
Then find out more here MircoConsult offers training courses/seminars/workshops and individual coaching on the topic of automotive/embedded and real-time software development.
Training & coaching on the other topics in our portfolio can be found here. here.
Automotive – Expertise
Valuable expertise in automotive/embedded and real-time software development is available. here Available for you to download free of charge.
You can find expertise on other topics in our portfolio here. here.
