EN 
DE The question of a modern update mechanism and Android's answer
Authors: Matthias Schaff and Dominik Helleberg, inovex GmbH
Contribution – Embedded Software Engineering Congress 2015
In the embedded systems sector, hardly any requirements specification exists without the need to update the firmware of a Linux-based, embedded system in the field. The implementation of this is varied: from on-site visits by a technician to sending CF cards to providing self-flashing instructions for the customer – everything is possible. In the age of IoT, Industry 4.0, and the resulting widespread networking of components, these approaches seem positively antiquated. A modern update mechanism operates fully automatically, without compromising the security or stability of the process. In the following sections, we will briefly discuss the requirements for such a process, as well as various update architectures, before concluding with a detailed examination of the Android FOTA process.
Requirements
The dominant requirement for a FOTA mechanism is stability. This includes aspects such as surviving power outages during the update process and handling faulty update packages (caused by transmission or storage errors). Besides stability, security is of paramount importance: only genuine FOTA updates, properly signed by the manufacturer, should be accepted, and the process should not create any vulnerabilities for attackers.
When evaluating different update architectures, it's important to consider that the components of an embedded system are associated with varying update risks. Similarly, the expected update frequency differs from component to component. The closer a component is to the hardware, the less frequently it is updated, but the higher the risk. Each update architecture strikes a different balance between frequency and risk. The appropriate focus cannot be determined universally but must be decided on a case-by-case basis.
Common update architecture
The following section will describe three update architectures in more detail and briefly explain their advantages and disadvantages. In all cases, the process is similar: The system to be updated initiates the update process after successfully downloading and, if necessary, verifying an update package. A boot flag is then set in persistent memory accessible by the bootloader of the currently running system, and a reboot is subsequently initiated. The bootloader checks the boot flag and, if necessary, starts the update process. The boot flag is only reset after a successful boot of the updated system.
The bootloader as an updater
The bootloader acts as a standalone updater. Modern bootloaders can access block- or flash-based file systems and are therefore suitable as updaters. However, such a limited operating environment typically lacks the necessary tools to perform a stable update process. While this functionality can theoretically be fully integrated into the bootloader, it's important to remember that the bootloader's minimalist design is deliberate: it's a fundamental part of the system, and every feature is a potential source of errors. Furthermore, bootloader updates are inherently risky. Therefore, it's advisable to avoid adding further complexity to the bootloader and instead delegate the actual update process to a more appropriately suited system component.
OS-Switching Architecture
The embedded system contains two fully functional operating systems, including all device drivers and a kernel for each. Only one version of the operating system is booted at a time, and this version can then update the currently inactive system. The boot flag indicates which of the two systems should be booted. The boot flag is toggled only after a successful flashing process and successful checksum verification. This ensures that a functional system version is always available. In case of an error, the system can revert to the previous version at any time. The system is always in normal operating mode, downtime due to updates is minimal, and the required storage space doubles.
Recovery OS Architecture
This architecture is similar to the "OS-switching" architecture, with the difference that only one system has the full range of functions; the second is used solely for the update process. It includes its own kernel and all device drivers, but only a small number of userland tools covering functionalities such as flashing and verification. The recovery OS is usually updated by the main OS, which also includes the necessary tools.
The Android update process
The Android update process is based on the recently introduced Recovery OS architecture. The entire process can be divided into four closely intertwined and interconnected stages. The result is a very robust, secure, and flexible update process that has already proven itself millions of times in practice.
The Android update process is resistant to transmission errors and interruptions, ensuring the integrity and authenticity of update packages. Furthermore, the update packages are optimized for minimal size to keep the amount of data transferred as small as possible (delta updates). We will examine the precise implementation below.
The Android update package
The package format for an Android update is a standard, digitally signed ZIP archive. The partitions and directory structures of the Android system are also reflected in the update package: The "recovery" directory contains the updates for the recovery OS, and the "system" directory contains everything for the Android OS.
The kernel and root file system of the Android OS are delivered as "boot.img". Since the recovery OS and the root file system of the Android OS differ only minimally, the recovery OS is generated as a binary diff from the boot.img.
The "META-INF" directory contains, in addition to the signature information and the public key for verification, a file named "updater-script". This script describes each step of the update process in Android's "edify" scripting language. "edify" was specifically created for update tasks, enabling, among other things, the easy mounting, formatting, copying, and patching of partitions and files.
The last relevant part of the update package is the binary "update-binary". This is the script interpreter that executes the edify script. It is started by the recovery OS and performs the actual update. This structure ensures that the update package is not tied to any specific recovery OS version.
The update package is a build artifact from the Android AOSP build system and is generated fully automatically. The build system provides interfaces for plugins to allow for device-specific customizations. When creating an update package, a starting version can be specified. This allows the build system to generate an incremental update containing only differences from the starting version. In this case, only a binary diff of the corresponding previous version is stored as a patch in the update package for each file, which drastically reduces file sizes, especially for small bug-fix firmware updates.
The Android update process in steps
Part 1 – Android OS: Download, Verify, Restart
Android doesn't specify how the update package gets onto the system. Normally, an app checks if a new update package is available on a previously configured update server, provided an internet connection is available. If so, it is downloaded after explicit user confirmation. Since it's just a ZIP archive, it can be installed via USB or an SD card.
The digital signature of the update package is then applied using RecoverySystem.verifyPackage() The signature is checked before the installation starts. It is verified against the OTA public keys [1] stored in the Android OS. Consequently, a valid update package can only be created if the corresponding private keys are available.
With a suitable signature, the following is used: RecoverySystem.installPackage() The installation process has started. This call first creates a command file [2] containing the actions to be performed by the recovery OS. In our case, the content would be, for example, “-update-package=/data/update.zip“.
In the last step of the first part, with PowerManager.reboot(PowerManager.REBOOT_RECOVERY) The boot flag was set and a reboot into the recovery OS was initiated. The boot flag itself is only set during the kernel call for the reboot.
Part 2 – Recovery OS: Verify, Unpack, Run
During the boot process, the bootloader evaluates the boot flag and boots the recovery OS accordingly, which then starts the "recovery" binary. This binary then verifies the signature of the update package again, this time against the public keys stored in the recovery OS [3]. If successful, the update binary is then extracted from the update package and executed.
Part 3 – Update Binary: Applying the Update
The update binary now begins interpreting the update script [4] and thus executes the actual update of the Android OS. In the case of an incremental update, the file to be patched is first copied, then patched, and a checksum is calculated. Only if this checksum matches the one specified in the update script is the patched file written back. Since the recovery OS cannot flash itself while running, the patch that generates the recovery OS from the boot.img, as well as a shell script [5] that applies this patch, are written to the system partition of the Android OS.
After a successful update, the control flow returns to the recovery binary, which resets the boot flag and initiates a reboot.
Part 4 – Android OS: Updating and cleaning up recovery
The bootloader now starts the updated Android OS for the first time. During the init process, the shell script copied from the recovery OS is executed to install the recovery OS. This script calculates the checksum of the recovery OS partition. If this checksum does not match the new recovery OS, it is updated. Finally, the update package itself is deleted, and the system can resume normal operation.
Sources
[1] see /system/etc/security/otacerts.zip
[2] see /cache/recovery/command
[3] see /res/keys
[4] See update package under META-INF/com/google/android/update-binary
[5] see /system/etc/install-recovery.sh
Open Source – our training & coaching
Do you want to bring yourself up to date with the latest technology?
Then find out more here MircoConsult offers training courses/seminars/workshops and individual coaching on the topic of Open Source / Embedded Software Engineering.
Training & coaching on the other topics in our portfolio can be found here. here.
Open Source – Expertise
Valuable expertise in modeling/embedded and real-time software development is available. here Available for you to download free of charge.
You can find expertise on other topics in our portfolio here. here.
