EN 
DE Fundamentals and practical experience
Author: Axel Wintsche, Philotech
Contribution – Embedded Software Engineering Congress 2017
Software security is a criterion that has gained immense importance, but it is difficult to adequately define and test as a requirement. However, a test strategy like fuzzing offers a way to automatically test software robustness and thus increase security. Here we describe what fuzzing is, the hurdles involved in testing embedded software, and possible solutions.
Secure software development: Goal and perspective of software testing
The V-model is a classic example of how requirements from the various phases of software development can be verified using appropriate testing methods. And even with the incremental agile methods often used today, there is usually a close integration between development and testing, with the goal of increasing quality.
To increase software security, appropriate methods can be applied during development and testing. However, the key difference is that security is difficult to define and test as a requirement. Even if comprehensive threat modeling leads to the derivation of relevant requirements that are met by the software, new, previously unknown threats can create unmet requirements. These cases can be addressed through appropriate incident response management. New threats, i.e., situations not specified by requirements, can also be proactively identified through appropriate testing methods, such as fuzzing.
Secure software should therefore be understood as a process that must be actively implemented through various methods and procedures, from design and implementation to delivery and maintenance. Examples include Microsoft's Secure Development Lifecycle (SDL) and the resources of the Open Web Application Security Project (OWASP) – both of which recommend the use of fuzzing. The consequences of vulnerabilities in software embedded systems have been demonstrated by various studies from security researchers – from successful hacks for remotely controlling cars [1,2] and the manipulation of pacemakers and other medical devices [3] to discovered security gaps and attacks on infrastructure [4,5]. The need to establish a security testing process for these systems is therefore significant.
Fuzzing tests
Fuzzing is a method used to test software by introducing invalid or random inputs to provoke malfunctions. In its simplest form, fuzzing generates a random sequence of bits or randomly modifies any number of bits from an existing bit sequence and uses this as new input for the software under test. Since the vast majority of inputs generated in this way do not conform to any formal requirements, fuzzing is particularly useful for testing how robustly the software handles faulty inputs. Fuzzing tests, therefore, aim to assess the robustness of software against unexpected inputs. One criterion for robustness is, for example, that the software remains in a defined state at all times and does not crash. In principle, any data processing software or software component can be tested using fuzzing.
Fuzzing essentially consists of three consecutive steps: 1) generating data, 2) providing the data, and 3) monitoring the software. These steps are independent of each other but are often executed in a loop. Frequently, the same bugs are found multiple times, so at the end of a fuzzing test, it makes sense to group the found bugs together (bug triage).
Generate data
This is the actual fuzzing process, in which the test data is generated. A distinction is made here between two methods: "generating" and "mutating." Generating involves creating test data based on a specification or model and making random changes to its content and structure. Mutating, on the other hand, involves making random changes to existing data. Both methods are used in different scenarios.
Provide data
The fuzzed data is then passed to the software under test, depending on the type of input interface, e.g. as a parameter when calling the program, as a file or in the form of a network message.
Monitoring
Errors in software execution are detected through monitoring. This can range from analyzing return values to using debuggers that can detect the internal state. The type of monitoring determines which errors can be detected.
Fuzzing can be used as both black-box and white-box testing. Black-box fuzzing programs are often easier to configure and automate, and are usually universally applicable. However, they tend to find simpler errors (dumb fuzzing). If specifications or even source code are available, it's worthwhile investing more effort in data generation and monitoring, as this can significantly increase efficiency (smart fuzzing). A wide range of fuzzing tools is now available (https://github.com/secfigo/Awesome-Fuzzing#tools). In addition to many specialized tools, such as fuzzers for specific file formats or network protocols, there are also frameworks that can be used in a variety of ways through appropriate configuration.
Errors found through fuzzing are often serious in nature, such as buffer overflows, integer overflows, denial-of-service (DoS) attacks, or code injection vulnerabilities (XSS, SQLi) [6]. Furthermore, fuzzing is a readily automatable, cost-effective testing method with the potential to detect errors missed during code review, unit testing, and similar methods (extreme or nonsensical inputs). It should be noted, however, that executing millions of test cases can be very time-consuming and therefore requires careful planning and integration into the development process. As with all testing methods, fuzzing cannot provide a complete picture of existing vulnerabilities, and software that is fuzz-free is not automatically secure.
Fuzzing of embedded systems
Software on electronic control units (ECUs) often operates under different conditions, which presents new challenges for fuzzing, primarily because the fuzzing tool and the software under test run on separate systems. Test data formats typically require adherence to specific message protocols such as CAN, LIN, or ARINC, and transmission to the ECU often necessitates additional hardware and software. Breakout boxes can also be used to simulate a system environment (sensors and message bus). Monitoring is difficult when detecting errors during execution on the ECU. JTAG debuggers are often the only way to observe the internal state. In a black-box test scenario, timeout criteria can be a practical solution, although any existing watchdog should be disabled.
Example of CAN message fuzzing
The test aims to evaluate a software component that processes CAN messages on an electronic control unit (ECU) at the application layer. A CAN message, in simplified terms, consists of an ID, the message length, and the message content. Non-protocol-compliant messages are blocked during transmission via the CAN hardware, and messages with invalid IDs are not passed to the corresponding software component. Specific login and logoff messages must be sent at the beginning and end of a connection, and each sent message is followed by a response message from the ECU.
To fuzz such a system, the prerequisites for sending and receiving data on the CAN bus must be established. The necessary hardware and drivers must be installed, and the fuzzing program must be connected to the driver API. The generation of the test data should take login and logoff into account and primarily fuzz the data range and length. This increases the efficiency of the fuzzing and thus the chance of detecting errors. Despite the custom requirements, it is often unnecessary to develop a specialized fuzzing program, as there are fuzzing frameworks that can be configured accordingly. Several monitoring options are available. For example, the absence of a response message from the control unit can be used for error detection (timeout criterion). By using a JTAG debugger, the stack trace can be determined when errors occur.
Summary
Fuzzing is a serious testing method and is already successfully used in desktop and web applications. However, fuzzing tests are rarely used in the context of embedded systems, even though there are certain similarities to conventional computer networks. Individual control units are interconnected via various network types, such as CAN, LIN, or ARINC; therefore, fuzzing as part of the security testing process is quite conceivable.
As more and more embedded systems have components with "online" functionality, the experience gained from fuzzing tests of classic protocols (TCP/IP) and modern applications (e.g., web and smartphone apps) can also be used. Security testing methods such as fuzzing could also be incorporated into safety standards such as ISO 26262 in the future to meet the current state of the art in security requirements [7].
literature
[1] C. Miller and C. Valasek, „Adventures in Automotive Networks and Control Units,“ DEFCON 21 Hacking Conference, 2013.
[2] „Car Hacking Research: Remote Attack Tesla Motors,“ Keen Security Lab
[3] „"Go ahead, hackers, break my heart!: Security of networked medical technology put to the test"“
[4] „"Malware at the Gundremmingen nuclear power plant," heise.de News
[5] „92 Percent of Internet-Available ICS Hosts Have Vulnerabilities“
[6] Prasanna Padmarajulu, „Discovering vulnerabilities with fuzzing“, PenTest Magazine 04/2017
[7] Bayer S., Enderle T., Oka DK., Wolf M. (2016) Automotive Security Testing – The Digital Crash Test. In: Langheim J. (eds) Energy Consumption and Autonomous Driving. Lecture Notes in Mobility. Springer
Our training courses & coaching sessions
Do you want to bring yourself up to date with the latest technology?
Then find out more here Regarding training courses/seminars/workshops and individual coaching sessions offered by MircoConsult on the topic Quality, Safety & Security.
Training & coaching on the other topics in our portfolio can be found here. here.
Quality, Safety & Security – Expertise
Valuable expertise on the topics of quality, safety & security is available. here Available for you to download free of charge.
You can find expertise on other topics in our portfolio here. here.
