EN 
DE Clearly defined test environment for reproducible results
Author: Michael Burkard, bbv Software Services AG
Contribution – Embedded Software Engineering Congress 2015
When you run your application tests on the development machine, it's difficult to assess the impact of certain factors on your test results. After a software update, libraries may be missing or replaced, rendering the test results unreproducible. Furthermore, how well do the resources of your development machine match those of the target system? With embedded systems, the differences are often so significant that running the tests on the target system is essential. Using Linux containers, a test environment can be created that runs in isolation from the host system, meets the requirements of the target system, and is still performant.
Good reasons not to run the tests on the development computer
You can keep your development environment up to date without affecting your test results. You can select a test environment that has the same resources as the target system, such as CPU architecture, number of CPU cores, or available RAM. Special software, users, groups, or permissions that you only need to run the tests are explicitly present in the test environment and not on your development machine, where they would interfere and cause unwanted side effects.
Requirements for a test environment
On the one hand, a test environment should always have the same parameters to ensure reproducible results. On the other hand, the test environment should mirror the target system; that is, it should be as close to the target system's conditions as possible. This implies that application tests should ideally be run in an isolated environment. Linux containers are perfectly suited for precisely these kinds of tasks.
Why Linux containers are the ideal candidates
You might be wondering why virtual machines aren't being used. Virtual machines offer an isolated environment, and resources can be tailored to your needs. Linux containers have a clear advantage in the following areas:
Boot process Linux containers start up and shut down in a fraction of a second. This allows you to seamlessly integrate Linux containers into your build process for running tests.
performance – Normally, no hardware simulation is necessary; for this reason, programs are run at the native speed of the host system.
Easy setup Additional tools can be easily installed via a shell script. Adding users and groups can be done in the same way.
Easy data exchange Whether it's the test application itself or the generated test results: Data between container and host can be easily exchanged.
Step-by-step instructions for creating a container
The following conditions apply to this example: 32-bit Ubuntu, 512MB RAM, single core and no swap partition.
Choose the appropriate Linux distribution with release and CPU architecture that best suits your target system (see PDF (for code).
The container configuration file allows you to limit resources. The following entries let you restrict the number of CPU cores and the amount of RAM (see...). PDF).
To disable the swap partition, the following arguments must first be passed to the kernel (see PDF).
You can then disable the swap partition in the container configuration file by setting the memory area for RAM and the swap partition to the same value as the RAM (see PDF).
To make data available within the container, directories can be mounted via the container configuration file. This allows the test program to be made available in the container, and the test results are then available to the host (see PDF).
The resources for the container are now set. What's still missing is a list of software, test-specific users/groups, and setting permissions. All these tasks can be defined in a shell script, which is called after the container is created (see PDF).
The same command sequence can later be used to execute the test script. The process of creating the container and executing the tests can be automated using a suitable shell script (see...). PDF).
Summary
Linux containers perfectly meet the requirements of a test environment without sacrificing performance. As you've seen, configuring and running the test environment requires minimal effort. Whether on your development machine or in the build environment, use Linux containers to test your application more closely to the real-world conditions of the target system.
List of sources
- Matt Fischer's blog, „Limiting LXC Memory Usage“, January 25, 2013
- Safematix IT Security, „Ubuntu LXC Memory & Swap Limit“, May 8, 2015
Open Source – our training & coaching
Do you want to bring yourself up to date with the latest technology?
Then find out more here MircoConsult offers training courses/seminars/workshops and individual coaching on the topic of Open Source / Embedded Software Engineering.
Training & coaching on the other topics in our portfolio can be found here. here.
Open Source – Expertise
Valuable expertise in modeling/embedded and real-time software development is available. here Available for you to download free of charge.
You can find expertise on other topics in our portfolio here. here.
