EN 
DE How STPA and BDD can help
Author: Prof. Dr. Stefan Wagner, Institute for Software Technology, University of Stuttgart
Contribution – Embedded Software Engineering Congress 2018
While agile software engineering has significantly changed how software is developed in practice, it still plays a subordinate role in safety-critical systems. However, integrating security analyses into a Scrum-based approach is possible without sacrificing agility. Below, we examine an example of this and link security analyses with Behavior-Driven Development for improved quality assurance.
Introduction
Software is increasingly becoming a central component in safety-critical systems, as it is often responsible for controlling these systems. In all safety-critical domains, be it automotive engineering, aircraft manufacturing, or machine tools, digitalization and thus the massive use of software has now arrived.
Agile software engineering, with its self-organizing teams, short cycles, and flexible requirements management, is increasingly becoming the standard for developing software outside of embedded and safety-critical systems. However, this approach often results in the absence of comprehensive and detailed requirements documents and architectural designs at the project's outset, which contradicts many safety analysis methods and standards. These standards expect a detailed system architecture to enable analysis and documentation of the results for certification.
How can this challenge be met? How can the advantages of agile software development be used for safety-critical systems?
Security analysis in Scrum with STPA
The most widely used agile development process is Scrum [1]. It defines the roles of the Scrum Master, the Product Owner, and the team. Scrum emphasizes that the team is self-organizing, while the Scrum Master ensures adherence to the Scrum rules, and the Product Owner acts as the interface to the customer. Requirements are stored in a backlog and prioritized. Requirements from the backlog are then addressed in two- to four-week sprints. In addition to the actual development work, the sprints consist of a planning meeting, a sprint review, and a retrospective. The planning meeting determines what will be accomplished in the sprint. At the end of the sprint, a potentially shippable product increment is presented to the customer in the sprint review. In the retrospective, the team gathers process improvements.
There are a few suggestions for how Scrum can be used to develop safety-critical systems. Safe Scrum [2] and R-Scrum [3] are the two best-known examples that already manage to integrate many documentation requirements into Scrum, ensuring compliance with standards such as IEC 61508. However, one difficulty remains: the aspects relating to functional safety stay outside the sprint. This somewhat limits agility.
Therefore, we decided to use STPA as our security analysis method. (System-Theoretic Process Analysis) [4] to be used. The method developed by Nancy Leveson at MIT offers us two crucial advantages here: (1) It is based on an iterative development of system and security analysis in alternation and (2) unlike other methods, it does not focus on individual components but on their interaction, which makes it easy to integrate software as well as users.
We have a corresponding Scrum extension S-Scrum [5] proposed, which in many aspects is strongly oriented towards Safe Scrum. The basic process is shown in Figure 1. Here, there is also a Software Safety Specification (SSRS) before the sprints, but the execution of STPA is within the sprint and even in the daily workflow. Communication between security experts and the team is ensured by an explicit Regular Safety Meeting, which does not necessarily have to be held daily. The direct integration of the results is achieved by mapping them to tests in continuous integration (TDD/BDD/CI). After the sprints, there is also a Final STPA Validation, which explicitly validates all security requirements so that this can be included in the documentation.
Figure 1 (see. PDFSimplified S-Scrum process
Despite some additional activities and documents that exist in S-Scrum compared to Scrum, we were able to show in a study [5] that agility is hardly affected from the developers' perspective. We have therefore come a step closer to the agile development of safety-critical systems.
Behavior-Driven Development and Security Requirements
In addition to integrating a suitable security analysis method, a simple link to agile quality assurance is also necessary to be agile in the development of safety-critical software. Agile software development offers a promising approach for combining it with security requirements: Behavior-Driven Development (BDD) [6].
The basic idea here is that the behavior is simulated in advance with the help of examples and scenarios in a so-called Three Amigos Meeting, The specification is defined by a team consisting of a developer, a tester, and a customer representative. The specification is usually in the form of simple, natural language text. For example, the language used might be... Gherkin These Gherkin specifications are then automatically read in by testers and checked against the system as test cases. This specification and the test are written before the specified scenario is implemented, so that the developers have a clear criterion for when they have fulfilled a requirement: when all tests can be executed successfully.
What's exciting about safety analyses is that, in functional safety, we also consider the system's behavior: What behaviors can lead the system into an unsafe state and trigger a hazard? This allows us to easily transform the safety analyses we develop in S-Scrum using the STPA into Gherkin specifications, which in turn can serve as test specifications. An example of such a specification is shown in Figure 2.
Figure 2 (see. PDF): An example of an unsafe scenario and associated scenario in BDD
We investigated this link between STPA safety analysis and BDD in experiments [7]. We were able to show that, compared to the conventional creation of acceptance tests, BDD significantly improved communication between the stakeholders. The Three Amigos Meeting, together with the natural language scenario specification, appears to make it considerably easier to discuss requirements and tests. Therefore, BDD is a very promising method for the context of safety-critical systems.
Summary and Outlook
Our results certainly haven't yet provided a comprehensive solution to the problem. However, S-Scrum with embedded BDD now offers an empirically proven development process that allows for both iterative security analysis and close integration with agile quality assurance. We believe this is just the beginning. To substantiate the empirical results, we are seeking partners for further studies. BDD and its associated methods for specifying scenarios could also support many other areas in the development of embedded software systems. The focus on ease of understanding and communication combined with fully automated test execution represents an interesting combination for many types of requirements. We plan to further explore these possibilities.
Bibliography
2. T. Stålhane, T. Myklebust, G. Hanssen. The application of Safe Scrum to IEC 61508 certifiable software. In: Proceedings of the 11th International Probabilistic Safety Assessment and Management Conference and the Annual European Safety and Reliability Conference. 2012.
3. B. Fitzgerald, K.-J. Stol, R. O'Sullivan, D. O'Brien. Scaling agile methods to regulated environments: An industry case study. In: Proceedings of the 35th International Conference on Software Engineering. IEEE, 2013.
4. N. Leveson. Engineering a Safer World: Systems Thinking Applied to Safety. MIT Press, 2011.
5. Y. Wang, J. Ramadani, S. Wagner. An exploratory study of applying a Scrum development process for safety-critical systems. In: Proceedings of the 2017 International Conference on Product-Focused Software Improvement. Springer, 2017.
6. M. Wynne, A. Hellesoy. The Cucumber Book: Behavior-Driven Development for Testers and Developers. Pragmatic Bookshelf, 2012.
7. Y. Wang, S. Wagner. Combining STPA and BDD for safety analysis and verification in agile development: A controlled experiment. In: Proceedings of the 2018 International Conference on Agile Software Development. Springer, 2018.
author
Stefan Wagner is Professor of Software Engineering and Managing Director of the Institute for Software Technology at the University of Stuttgart. He studied computer science in Augsburg and Edinburgh and received his doctorate from the Technical University of Munich. His research focuses on requirements engineering, software quality, functional safety, and agile software engineering. He enjoys exploring these topics using empirical and psychological methods in close collaboration with industry. He also works as a freelance consultant and trainer on these subjects. He is a member of the German Informatics Society (GI), the American College of Management (ACM), and the IEEE.
Our training courses & coaching sessions
Do you want to bring yourself up to date with the latest technology?
Then find out more here Regarding training courses/seminars/workshops and individual coaching sessions offered by MircoConsult on the topic Quality, Safety & Security.
Training & coaching on the other topics in our portfolio can be found here. here.
Quality, Safety & Security – Expertise
Valuable expertise on the topics of quality, safety & security is available. here Available for you to download free of charge.
You can find expertise on other topics in our portfolio here. here.
