Automation tools are a boon to the development of automotive solutions. These solutions are quite complex as they weave together intricate architecture, diverse input sources, and the paramount importance of safety. When ISO 26262 compliance is thrown into the mix, complexity goes a few notches higher.
Software Tools, to a large extent, automate the tasks needed to be carried out during the ISO 26262 mandated safety lifecycle. However, they may also introduce errors in the form of fault-injection or non-detection of faults in the software.
Consider a scenario where an automotive software development team relies on a seemingly reliable static analysis tool to identify potential errors in their code. Despite the tool's efficiency, it begins generating false positives—erroneously flagging sections of code as problematic. This not only slows down the development process but also raises questions about the accuracy of the tool's error detection capabilities. Such incidents underscore the importance of ISO 26262 tool qualification and evaluation.
What does ISO 26262 Standard Say About Software Tool Qualification
Clause 11 of Part-8 of the ISO 26262 standard comprises the ISO 26262 tool qualification methods. As per the standard, ISO 26262 tool qualification is a two-phased process that begins with the tool evaluation where Tool Confidence Level is determined and is then followed by Tool validation.
A natural question that pops in the mind is, ‘Do all the tools require qualification for every project?’
Well, a simplistic answer to that question is yes! A more nuanced answer would be that it depends on the particular use cases, software requirements and project specific workflow. It might be possible that a tool requires qualification in one project and does not in another. The question to be asked here is, “Will the tool introduce an error in the software?” If there is a possibility of error introduction, the tool must pass through the scrutiny.
Why ISO 26262 Tool Qualification Matters?
Understanding the necessity of software tool qualification under ISO 26262 involves delving deeper into the intricacies of how software tools can impact the safety and reliability of automotive systems.
This process is not just about compliance; it's about ensuring that every step in the development of automotive software is aligned with the highest safety standards.
Let's expand on the previous example involving a software development tool used for programming the electronic control units (ECUs) of a vehicle's braking system. Imagine this tool contains a bug that causes it to wrongly compile certain critical sections of the code. This could be due to a specific combination of syntax and commands that's not used frequently, but critical for the braking system's emergency response.
During testing, the compiled software might perform as expected under normal conditions, passing all routine checks. However, in a real-world scenario where some conditions trigger the bug, the braking system might fail to activate in time, leading to a failure in the vehicle's ability to stop promptly.
The Process and Impact of ISO 26262 Tool Qualification
The ISO 26262 tool qualification process is designed to uncover such potential risks by requiring a thorough evaluation of the software tools used in the development of safety-related systems. This evaluation is multifaceted, assessing not just the tool's functionality but its potential to introduce errors into the product.
- Tool Evaluation: Initially, the tool's functions and their relevance to the safety-related aspects of the system are scrutinized. The evaluation considers the tool's operational environment, the complexity of its functions, and the potential consequences of its failure.
- Determining the Tool Confidence Level (TCL): Based on this evaluation, a TCL is assigned, which reflects the level of confidence that can be placed in the tool to perform its intended functions without introducing unacceptable risks. The TCL informs the extent of qualification activities needed.
- Tool Validation and Qualification: For tools with a higher potential risk (lower TCL), a rigorous validation process is necessary. This may involve additional testing, the creation of workarounds for known issues, or even the development of custom testing frameworks to ensure that the tool does not introduce errors.
- Documentation and Rationale: Throughout this process, detailed documentation is maintained, providing a rationale for the tool's use, describing the evaluation process, and detailing any mitigations put in place for identified risks. This documentation is crucial for demonstrating compliance with ISO 26262 and for providing transparency in the tool qualification process.
The goal of tool qualification under ISO 26262 is to ensure that tools do not become the weak link in the safety of automotive systems. By identifying, evaluating, and mitigating the risks associated with software tool usage, manufacturers and developers can prevent errors that might not be evident until they lead to failure in real-world operating conditions.
Approach to ISO 26262 Tool Qualification
As already mentioned, Part-8 of the ISO 26262 standard contains detailed guidelines on software tool qualification. The objective is to get evidence suggesting that the tool used for the project is suitable for safety-related software development. The different use-cases analysed during the tool evaluation process are documented as evidence.
A Snapshot of one such document is shown below:
Certain terms such as Tool Impact, Tool Error Detection and TCL can be seen in this snapshot. We will explain them in detail as we proceed further.
These use-cases help in the evaluation of the following aspects:
- Does a malfunctioning software tool and its wrong output violate any safety-goal derived during HARA?
- What is the probability of a software tool detecting errors in the output derived from the tool?
The analysis of these use-cases helps derive the Tool Confidence Level (TCL). The TCL along with ASIL assigned to the project is used to determine the right method for tool evaluation. We will discuss the methods while explaining the work products required for ISO 26262 tool qualification.
The first step in the process of ISO 26262 tool qualification is the planning of the usage of the tool. From the configuration of the tool to its use cases and execution environment, a plethora of information is required at this step.
A gist of information that must be available to ensure proper tool evaluation:
- Description of features, function, and properties of the tool
- Environment required for the tool operation
- Known malfunctions of the tool and associated safeguards
- Known expected behaviour of the tool under anomalous working environment
Now comes the analysis part. This process is also called Software Tool Classification Analysis
This analysis is required to determine the Tool Confidence Level (TCL). In order to derive TCL, tool impact and tool error detection need to be understood.
Tool Impact: If a tool can introduce faults in the software or fail in detecting the errors, the tool is considered to impact the final software. In such a case, the Tool Impact is denoted by TI 2. If the tool is not impacting the final software output, its falls in the category of TI 1.
Let us understand with an example. LabView tool is a test automation tool used to validate the functionality of a controller such as Torque-speed characteristic. As it is an important task, this tool impacts the final software. Hence, it is classified as TI 1.
Tool Error Detection: The use-case scenarios of the tool decide whether there are enough safeguards in the tool to prevent and detect malfunctioning and erroneous output. This capability of the tool decides the error detection confidence level.
| Degree of confidence | Tool error detection level |
| High | TD1 |
| Medium | TD2 |
| Low | TD4 |
The ISO 26262 Functional Safety experts take multiple factors into consideration while assigning Tool Detection levels. For instance, an IDE called CS+ may have a tool error detection level of TD 2 (medium) due to its incapability of detecting all the errors in the code. Another code generator for which the output code is tested according to ISO 26262 standard, can be given a TD1 level (High confidence).
There is another scenario where a different software tool is used to verify the output from a software tool. In such cases, the interdependence between the tools is also considered.
Tool Confidence Level:
When TI and TD are established, Tool Confidence Level can be determined using the following schematics.
TCL 1 is the lowest level of confidence. Tools assigned this level do not have high impact on the quality of the final product. Therefore, tool qualification is not required.
TCL 2 and 3 are medium and high level of confidence, respectively. A tool qualification is required to demonstrate tool reliability.
And when we have the TCL, we refer to the ISO 26262 standard Part-8 Table 4 and 5 to understand which Tool Qualification Method applies.
ISO 26262 Tool Qualification Methods Mandated by Standard
The tool qualification methods prescribed in the standard for TCL 2 and TCL 3 are almost similar, with a slight difference based on ASIL value.
The tables will make it clearer.
Let’s understand these methods and how they are applied for ISO 26262 tool qualification.
- Increased Confidence from use: The confidence is derived from previous use of the tool under similar development environment and use cases. There is an increased evidence only when the similar version of the tool has been used for the same purpose with similar functional constraints. Also, the data collected from the tool’s previous usage must be adequate. If these conditions are not met, this method is not applied. As the software tool versions upgrade very frequently, this method is not among the most used ones.
- Evaluation of the tool development process: This method relates to the appropriate standard applied during the development of the tool itself. In most cases, the tool evaluation team does not get in touch with the tool developers and assess their development process. However, the tool developers do get their tools certified by organizations such as TUV.
- Validation of software tool: Validation of the software tool is a method that is required for higher ASIL compliance (ASIL C and ASIL D). Also, the test suites must be designed to evaluate both functional and non-functional aspects of the software. The standard states that the tool validation can be performed by either the tool vendor or the tool’s end-users.
- Development in accordance with ISO 26262 standard: This method is rarely applied as a majority of software tools are PC based and are not developed as per ISO 26262 standard.
*For ASIL B, C, and D, an additional step of confirmation review as per Part2 of the ISO 26262 standard must be carried out. Different levels of confirmation review are applied as per the ASIL.
Conclusion
ISO 26262 tool qualification is a joint task for both the tool vendors and the users. Although tool qualification must be performed by the user with respect to specific projects and use-cases, the tool vendors can do their bit to make the process easier and less time-consuming.
