How models make testing more efficient

Model-based testing with the OMG's UML Testing Profile V2

Author: Markus Schacher, KnowGravity Inc.

Contribution – Embedded Software Engineering Congress 2016

Testing costs still represent a substantial portion of the development costs of complex systems. Model-based testing (MBT) promises a significant reduction in these costs, especially in the design and maintenance of test specifications. On the one hand, the comprehensive reuse of test-relevant concepts simplifies their maintenance and further development. On the other hand, the increased degree of formalization in MBT enables both test design and execution to be largely automated, leading to substantial cost savings, particularly in regression testing. The UML Testing Profile V2 (UTP 2) is a new specification from the Object Management Group (OMG) that is entirely focused on model-based testing.

Models

Before we delve into model-based testing, it's important to clarify exactly what a "model" is. Essentially, a model is something that can be used for a specific purpose as a (highly) simplified description of an original (the object being modeled). Two things are essential when creating or using a model:

• It must be clear at all times what the Original is, i.e., what exactly is being "modeled" or described.
• It must be clear at all times what the Purpose of the model This is because it determines, on the one hand, which aspects of the original are omitted (the simplification) and, on the other hand, which form of representation is used or should be used for the model.

The benefits arising from the creation and use of models can be grouped into the following topics:

• Models enable the communication both between people (from the initial exchange of ideas to the formulation of the order) and between people and machines (for example, for automated order execution by machines).
• Models promote this Understanding about the object of modeling, by examining it and recording the resulting insights in the form of a model (for example, models about human behavior or about the processes in a company).
• Models of a certain precision enable formal analysis and thus the prediction of certain properties or behavior of the modeled original (for example, weather models or models from structural engineering and strength of materials).

Models come in many different forms. They differ, for example, in their mode of expression (physical objects like a model ship, textual descriptions like formula collections, or graphical representations like electrical diagrams) and their level of precision (informal models versus mathematically analyzable or simulable models). The appropriate mode of expression and level of precision for a model depends on its purpose and, consequently, on the target audience (see also Figure 1)., PDF).

For models that describe constantly changing originals, or for models based on an incomplete and continuously evolving understanding of the original, it is essential that the models can be efficiently adapted to the latest state of knowledge. An important characteristic of easily maintainable models is that they work with references: such a model is built from many model elements that reference each other.

In contrast to a description in the form of a sequential prose text, the description of the entire original is constructed from a large number of highly reused partial descriptions (the model elements) through referencing. This allows, on the one hand, the production of different (target group-specific) views of a single model and, on the other hand, the centralization of changes to individual model elements and the reproduction of a consistent set of views from them (see Figure 2)., PDF).

Model-based testing

When technical or socio-technical systems need to be tested, test specifications are often created. Such test specifications are complex descriptions of various Modeling objects, which are relevant in connection with testing:

1. The object to be tested, as well as its properties, behavior, and configurations to be tested.
2. The intention behind the testing, i.e., what, why, and with what intensity is to be tested.
3. The strategies, procedures and techniques to be used for testing
4. The resources and activities planned for testing, as well as the resulting time schedule.
5. The specific instructions for carrying out the individual tests
6. The tools required for testing, such as external systems, test components, and test data.
7. The expected measured values or reactions of the object being tested, as well as the evaluation rules used to decide whether a test is passed or failed.
8. Test coverage with respect to the properties, behavior, and configurations of the object under test.

All of the elements listed above can now be considered as modeling objects that can be described in a test specification. If this description is model-based, the following results. Advantages:

1. By deliberately omitting detailed information, a higher level of abstraction The test specification is achieved, making it more compact and easier to understand. (1 = Advantage A)
2. Through referencing often reused elements (Up to and including standard libraries), test specifications become much more adaptable if the object under test changes or evolves (e.g., in regression tests). (2 = Advantage B)
3. Will the models be in a corresponding precision Once created, they can be automatically analyzed (e.g., impact analysis), automatically transformed into further models, or the tests can even be executed automatically. (3 = Advantage C)

Using a standardized modeling language to describe these aspects offers further advantages:

1. The standardized modeling language provides a certain Structuring the relevant information a point of reference for the tester. (1 = Advantage D)
2. Communication regarding the various aspects of the test (e.g., test objectives, test strategies, test procedures, etc.) is simplified and made more efficient, as all participants have a common language speak. (2 = Advantage E)
3. A standard language for formulating test specifications promotes ecosystem various tools and tools that can be used in the creation and management, but also for the automatic execution of test specifications. (3 = Advantage F)

The UML Testing Profile V2 (UTP 2)

The UML Testing Profile is a specification from the Object Management Group (OMG), the first version of which was published in 2005. Even then, its goal was to create a standard for model-based testing. Two minor updates (1.1 and 1.2) were released in 2012 and 2013, and work on the new major version, UTP 2, was initiated concurrently. UTP 2 is a standardized modeling language for testing, pursuing the following objectives:

• UTP 2 should Tests of any kind Support for all types of systems (not just software testing)
• UTP 2 should the entire testing process to support you, from test conception to test execution and evaluation
• UTP 2 is intended to Advantages of model-based approaches to be used in the area of testing
• UTP 2 should a standard for model-based testing (MBT)
• UTP 2 should (if needed) be like this formally It is possible that further artifacts can be generated from test specifications.
• UTP 2 is intended to be based on the Unified Modeling Language (UML), so that UTP can be expressed using common UML tools and can be used with standard UML knowledge
• UTP 2 is intended to replace other UML-based Modeling languages complement (for example, SysML, MARTE, UPDM, or BMM)
• UTP 2 should compatible with other standards in the area of testing, such as ISO/IES/IEEE 29119, ISTQB, ETSI TTCN-3 or TDL

The development of UTP 2 is currently nearing completion, and the specification is scheduled to be submitted to the Object Management Group for finalization and publication by December 2016.

UTP 2 possesses various properties and mechanisms that achieve the aforementioned advantages:

• UTP 2 offers various levels of abstraction and detail UTP 2 offers features that allow the user to focus on the essentials. With various abstraction techniques (specialization, reuse, parameterization, instantiation, etc.), it provides the ability to specify important relationships in a very compact yet precise manner. (Advantage A)
• UTP 2 defines a metamodel from standardized model element types, which can be referenced in different views and thus reused consistently. (Advantage B)
• UTP 2 offers precisely defined options when needed. Standard test actions, data operations, and test evaluation rules, This allows, for example, test cases to be generated from other models or executed automatically. (Advantage C)
• UTP 2 offers a Comprehensive coverage of topics in the field of testingTest analysis, test design, test architecture, test behavior, test data, and test evaluation. UTP 2 offers the user concrete, structured assistance on how to develop these topics and how they are interrelated. (Advantage D)
• UTP 2 is a domain-specific, diagram-oriented language which Standardized wording of test-relevant aspects and thus represents a key basis for unambiguous communication. (Advantage E)
• Since UTP 2 is an expectation of the Unified Modeling Language (UML), UTP 2 can be use in common UML tools, and it can build on existing know-how. (Advantage F)

An example

To illustrate how the topics covered by UTP 2 can be applied in practice, some examples from the development of a passenger elevator are compiled below.

One of the first and most important considerations in testing is defining what the test object actually is (called "Test Item" in UTP 2), what tools are required for this (called "Test Component" in UTP), and what configuration of these elements is necessary for conducting tests. Figure 3 (see PDFFigure 1 shows four different such "Test Configurations," which describe various elements of a passenger elevator with different aids and for different purposes (modeling objects 1 and 6).

In contrast, Figure 4 (see PDF) completely different relationships, namely, which goals («Objective») and requirements («Requirement») for the «Test Item» are to be validated («validate» relationships) or verified («verify» relationships) with which test cases («Test Case») (modeling objects 2, 3 and 8).

It also shows which test cases fulfill which test requirements (satisfy relationships). These relationships enable detailed traceability, if required. Furthermore, this diagram illustrates that certain test cases are specializations of other test cases or extend other test cases by specific aspects.

Finally, Figure 5 (see PDF) an example of a specific test procedure (that of test case „TC01: Elevator Request“ in Figure 4, see PDF), which is built from standardized test actions („create stimulus“, „expect response“, „check property“, etc.) (modeling objects 5 and 7).

This diagram shows the same tester from Figure 3 (see PDFThe test component is shown, and the "Test Item" is the one from the "Test Configuration" "Acceptance Tests". These are examples of how diagrams simply show references to model elements that have only been defined once. If changes need to be made, these changes can be made in one place, and all views based on it will be updated automatically.

Summary

Model-based testing promises increased testing efficiency, resulting from leveraging the advantages of modeling in general. The use of reusable model elements enhances the consistency of a test specification and simplifies its maintenance when changes occur. Furthermore, the precision of a modeling language like UTP 2 allows for the automation of various testing-related tasks.

Bibliography

[ETSI] European Telecommunications Standards Institute (ETSI): ES 202 951: Requirements for Modeling Notations. ETSI standard,
Methods for Testing and Specification (MTS); Model-Based Testing (MBT), V1.1.1 (2011-07)

[ISO] International Organization for Standardization: Software and systems engineering – Software testing – Parts 1-5,
ISO 29119-1:2013(E), 2013-09-01, September 2013

[TTCN3] ETSI: Methods for Testing and Specification (MTS) – The Testing and Test Control Notation Version 3 – Part 1:
TTCN-3 Core Language (Version 4.7.1), ETSI ES 201 873-1, June 2015,
www.ttcn-3.org/index.php/downloads/standards/

[UTP] Object Management Group: UML Testing Profile (UTP) – version 1.2, ptc/2012-09-13, September 2012

Download the article as a PDF

---

Modeling – MicroConsult Training & Coaching

Do you want to bring yourself up to date with the latest technology?

Then find out more here MircoConsult offers training courses/seminars/workshops and individual coaching on the topic of modeling/embedded and real-time software development.

Training & coaching on the other topics in our portfolio can be found here. here.

---

Modeling – Expertise

Valuable expertise in modeling/embedded and real-time software development is available. here Available for you to download free of charge.

To the specialist information

You can find expertise on other topics in our portfolio here. here.