Getting somewhere safely depends on more than just good brakes, working taillights, and someone with excellent reflexes behind the wheel.
Getting somewhere safely depends on more than just good brakes, working taillights, and someone with excellent reflexes behind the wheel. Increasingly, the components that keep your car on the road and your plane in the air are not only human, or even just mechanical. They are sophisticated pieces of embedded software running on complex heterogeneous multicore processors, controlling everything from flight management system to power steering, and executing to strict timing deadlines measured in microseconds.
Herein lies the challenge. The timing behavior of software in a multicore system is affected not only by the software running on it and its inputs, but also by the other software running on other cores.
Critical embedded systems require an immense effort and investment (millions of euros/dollars and years of engineering effort) to be developed. Safety has to be at the heart of the architecture and design, right from the earliest stages of the software development process. In particular, systems designers must understand the timing behavior of their software, to ensure it can execute within safe timeframes.
Solving the multicore timing analysis (MTA) puzzle
Although the awesome computing capacity of a multicore processor should (in theory) make embedded systems more powerful and efficient, software executing on one core can slow down execution of software running on the other cores. In this situation, software can take longer to execute due to interference caused by contention for shared resources such as buses, memory, caches, devices, FPGAs and GPUs that are shared with tasks running on other cores.
How do you quantify the effects of this interference? How do you analyze, test, and provide concrete evidence that your safety-critical software, when running on a multicore platform, can always execute within its timing deadlines?
Experts at the Barcelona Supercomputing Center (BSC), Rapita Systems Ltd (RPT), Raytheon Technologies (RTRC), and Marelli Europe (MAR) have been investigating answers to these questions for many years. BSC and Rapita have been developing a solution that will soon be rolled out across the aerospace and automotive industries. Specialized tooling and automation, combined with a requirements-based, safety-focused methodology were the keys to solving the puzzle.
This work has formed the basis of the MASTECS project, a multi-disciplinary research and development project funded by the European Commission and launched in December 2019. The MASTECS project will mature the technologies and support their use for certification of avionics and automotive systems. A key part of the MASTECS project is to provide a demonstration of the approach in two industries through case studies deployed by RTRC and MAR.
State of the art tools
Commercially available tools to support timing analysis are effective for simple (single-core) electronics, but do not scale to meet novel multicore-specific certification requirements and recommendations.
To our knowledge, no commercial tool is available in the market, other than the one being matured in MASTECS, that is capable of analyzing the timing of software on multicore platforms, with strong focus on applicable safety standards and emerging certification requirements.
Interference analysis and control in action
The key to understanding interference is a structured test methodology, using hardware and software experts to produce evidence about multicore timing behavior. A specialized technology from BSC (known as multicore micro-benchmark technology or MμBT, commercialized by Rapita as RapiDaemons) lets system designers analyze and quantify the effects of interference in a multicore-based application by creating additional interference scenarios to stress-test different parts of the multicore processor.
Micro-benchmarks, at the heart of MuBT, are well-crafted pieces of code that operate at the lowest interface between hardware and software to stress a specific shared resource. Micro-benchmarks expose the impact of interference channels on software timing. To do so, micro-benchmarks can be deployed to cause a configurable and quantifiable pressure on a specific application. Micro-benchmarks are specifically designed to exhibit a single, clearly defined behavior with anticipated effect on a specific hardware resource, while preventing as much as possible to generate contention on other interference channels. Micro-benchmark key features include the following:
A wide range of micro-benchmarks have been developed to have specific roles, including matching a desired level of interference, maximizing interference on the resource, or simply being very sensitive to contention (‘victims’).
In analyzing the effects of interference, the use of MμBT is supported with a task contention model (TCM) that provides early estimates of the contention delay tasks can suffer. Software automation and testing tools RapiTest and RapiTime developed by Rapita are used to write tests and run them on the embedded target.
By following a seven-step test design process along the standard software ‘V’ development process (Figure 2), engineers can more fully understand the impact of interference.
Hardware expertise and the timing analysis process
Injecting hardware (multicore) expertise is a key trait in the proposed MTA approach for its success on modern complex multicores. During early software development stages:
During late design stages:
The bigger picture
The 7-step test design process is only one part of a wider multicore verification methodology shown earlier in Figure 2. This methodology, which will continue to be matured as part of the MASTECS project, is designed to achieve full traceability, from comprehensive evidence and results back to the corresponding requirements and designs. The methodology is designed to meet the objectives defined in CAST-32A, the key guidance document issued by aerospace certification authorities. It is also specifically aligned with ISO 26262, the safety standard for the automotive sector, which advocates freedom from interference.
CAST-32A was published by the Certification Authorities Software Team (CAST) in 2016, and identifies factors that impact the safety, performance and integrity of airborne software systems executing on multicore processors. If you want to use multicore hardware in an avionics system, this is the go-to document. It provides objectives intended to guide the production of safe multicore avionics systems including objectives related to identifying and bounding the impact of interference channels. View the CAST-32A position paper here. EASA and FAA are working on an adaptation of the multicore generic CRI into a common AMC/AC material (AMC 20-193). It is expected to be published “later this year”.
Expertise cannot be automated
Interference effects are complex. To unravel their mysteries, you need experts who understand both the components of the multicore architecture, and the scheduling and resource allocation systems in the software. Collaboration between hardware and software experts will be a central feature of the MASTECS project as it continues into the future. But while collaboration leads to great strides in software tooling and automation, it’s important to remember that you can’t automate every step of a validation process – especially not when multicore timing analysis is involved.
You need experienced engineers who know the systems in detail. For example, during the early stages, multicore experts can identify the processor configurations (also known as hardware critical configuration settings) that determine the software’s functional and timing behavior, as well as the potential interference channels. When it comes to analyzing test results, nothing beats the input of an experienced human expert to revisit and evaluate the original assumptions made about the platform, and use their knowledge to feed into a new testing cycle.
— Dr Francisco J. Cazorla (BSC) is the leader of the Computer Architecture / CAOS group in the BSC and the technical coordinator of MASTECS.
— Dr. Enrico Mezzetti (BSC) is a senior researcher in the Computer Architecture / CAOS group.
— Dr. Ian Broster (RPT) is a founder and the General Manager of Rapita Systems Ltd.
— Dr. Juan Valverde (RTRC) is a Staff Research Scientist at United Technologies Research Centre Ireland Ltd.
— Stefania Botta (MAR) has a degree in Computer Science, inside Marelli – Powertrain B.U. she is part of “Software Tools and Methodologies” team.
— Dr. Jaume Abella (BSC) is a senior researcher at BSC.
— Dr. Christos Evripidou (RPT) is the Technical Lead of Rapita Systems’ UK Multicore Timing Analysis team.
— Dr. Javier Mora de Sambricio (RTRC) is a Senior Research Scientist at United Technologies Research Centre Ireland Ltd.