‘Reliability’ and ‘Quality’ in the automotive software ecosystem are moving targets. How do you predict the behaviour of a system that interfaces with varied components in incalculable scenarios? The obvious answer is to test the system with as many variables as is possible. Compliance to standards like ISO 26262 and ASPICE makes this task both simpler and tougher. Simpler because these standards provide test methodologies and best practices to be followed and tougher because the process entails a whole lot of tests to be performed.
And this where test automation tools enter the fray to make the life of test engineers a bit easier. In this blog, we talk about one such tool, Tessy and its role in automation of integration testing of automotive software. But first, lets refresh our understanding of integration test.
Integration testing is the next level of test performed after unit testing. As per the V-Model, integration testing is executed against software architecture design. In this process, several software modules (units) are integrated as one module and tested together. And hence, the name integration testing. The interdependence of different software functions, the interfaces between them and time dependencies are some of the specifications that are verified during the process of integration testing.
Generally speaking, integration testing of automotive software is no different than testing of any regular software. The difference, however, lies in the tools that are deployed for integration testing of automotive software. This brings to the main highlight of this blog- Tessy tool.
Now, most of us identify Tessy tool as the quintessential unit testing tool mostly deployed for ISO 26262 compliant unit testing. However, there is a different side to Tessy as well. And we are here to explore that side and examine how Tessy tool helps the automotive engineers automate and speed up automotive software integration testing. We will also touch upon its role in achieving the test methodologies and best practices laid down by ISO 26262 and ASPICE.
Tessy, an Industry Wide Recognized Tool for ISO 26262 and ASPICE compliance
Tessy is a qualified tool for both ISO 26262 standard and Automotive SPICE
The makers of Tessy provide a tool qualification package that can be used within certification activities according to standards like ASPICE. Integration testing finds detailed emphasis in ASPICE process flow.
ASPICE defines the purpose of software integration test as “integrating the software units into larger software items up to a complete integrated software consistent with the software architectural design and to ensure that the software items are tested to provide evidence for compliance of the integrated software items with the software architectural design, including the interfaces between the software units and between the software items.
Tessy is designed to establish bi-directional traceability between all derived requirements and all related tests for each requirement. One of the key purposes of ASPICE is to establish bi-directional traceability and Tessy does it with ease.
With respect to ISO 26262 standard, Tessy has a clearer advantage. It is among the few qualified test tool for ISO 26262 compliant verification. Features that make it one of the most widely used testing tool for ISO 26262 compliant projects are:
- Mutation test feature that checks the error detection capability of the test cases
- Coverage selection of Tessy tool is based on various standards including ISO 26262
Since several automotive projects generally follow both ISO 26262 and ASPICE, Tessy comes across as a preferred tool for software integration testing.
Integration Testing, the Tessy Way!
Tessy tool comes equipped with the capability to perform two types of testing- Unit Testing and Component Testing. In the Tessy environment, integration test is called as component testing as it treats a group of units to be tested, as one component. Tessy tool is designed to automate various processes of integration testing starting from requirements import to a comprehensive report generation.
Essentially, the component test feature of Tessy supports the engineers in creating scenarios where they can check the internal values of the component variables as well as the interfaces between different modules.
For example, a Battery evaluation module of an EV requires battery voltage data from a separate module to evaluate the remaining power in the battery. The engineers can create certain test scenarios to check whether the relevant functions are correctly called, and accurate data is transferred. Let’s learn about the steps in integration testing to understand the process better.
- Requirement Import from Requirement Management Tools: A component under test must provide the expected output for the test, to be marked successful. It is a way to ensure that the software component is exhibiting the functionalities as expected in the requirement document. And for Tessy to understand these requirements, it must first import them from the requirement management tools such as DOORS or Polarion.
Tessy tool provides a one-click import to make the process easier and less time-consuming. The tool lets you check the requirement and edit it before it is committed to a test case. After the test cases are prepared, these requirements are linked to the respective test cases in order to ensure traceability. - Creating Test Scenarios: We mentioned earlier in the blog that for verifying the interfaces between different modules, scenarios with test inputs are created. Tessy provides a scenario editor (SCE) function for this purpose. A scenario can be seen as a group of test objects and test cases that applies for a component under test. Instead of having individual test cases for each function, scenario provides a more consolidated and efficient way to perform integration testing. By creating one scenario for one component, multiple tests can be performed viz, checking global variables, calling component functions, setting input variables and so on. A scenario fulfills two main requirements:
- A scenario simulates the external application for testing the component. Both normal and abnormal behavior are simulated to check the error handling capabilities of the component.
- It also records the reaction of the component under the simulation.
- Test Interface Creation: The primary goal of integration testing using Tessy is to verify the interfaces between different modules. Hence, it is imperative that all these interfaces under test are enlisted in the tool. In this step, the interfaces of all non-static functions are summarized. As clear from the image, the functions are categorized under external and component functions and few other variables.
External functions list the interfaces belonging to the underlying software that will be called by the component under test. Component function lists those interfaces that are callable from outside of the component. A sequence of all such function calls is what integration tests are mostly about. - Work Task Configuration: Applications running under an RTOS (Real time operating system) usually have a timeframe for completing a function call. Hence, it becomes important to check whether a resulting call is executed after the stimulating call within the stipulated time frame. And for this sort of verification, a special handler function called the heartbeat function is called. This heartbeat function is called Work Task in Tessy testing environment. Simply stated, a work task tests the timely behavior of a component.
In the work task configuration tab, the test engineers can drop the component functions and configure them as work tasks. All they need to do is to specify the start time and cycle time. Rest of the job is taken care of by Tessy tool. - Test Case Design: Test case design is nothing but creating scenarios for testing. As multiple scenarios are required for stimulating a component and checking its behavior, they are created as and when required. Post the creation of these scenarios, they need to be configured too. Let’s understand what the required configurations are:
- Addition of Function calls: External and component functions are added to the time step.
- Test data configuration: Input value can be set, or output value can be checked for any time step of the scenario.
- Return value of functions: Parameter value is set for the functions added to the scenario. When Tessy calls a function, it always does so with the parameter value specified here. If there is no parameter value, an error will be displayed.
- Calling sequence of functions: Call sequence can be checked by specifying certain information for the expected function call within the scenario. This is a proven mechanism to check whether the behavior of a component is as expected or not.
Once the scenarios are designed, they are executed. All one must do is to select the scenario and execute them with one click. After the execution is completed, a detailed report can be retrieved from the tool.
Value-Adds of Tessy Tool for Integration Testing of Automotive Software
- ISO 26262 Compliant Integration Testing: Tessy is one of the most sought-after ISO 26262 qualified unit and integration testing tool. It has a built-in template where you select the targeted ASIL value and Tessy takes care of the test methods as mentioned in the ISO 26262 standard document.
- Automation of Test Execution: Tessy is designed to be a test automation tool. Once, you import the requirements and create the scenarios, Tessy handles the execution of the test – traversing the code and providing complete coverage.
- Comprehensive Test Report: Test reports created by Tessy tool are quite comprehensive and detailed showing the linked requirements, pass and failed test cases, etc. There is also an option to edit the test report setting on the basis your preferred information. These test reports provide all the information (evidence) required for ISO 26262 certification or qualification.
- Requirement Traceability and Accuracy of Test: With Tessy, it is easy to achieve requirement traceability as it commits the requirements to respective test cases. It makes change management easier. Since it is an ISO 26262 qualified tool, the test results have been found to be very accurate and reliable.
Final Thoughts
Tessy has proven to be a very helpful tool in integration test automation for automotive software development. Being a ISO 26262 qualified tool makes it even more desirable for the engineers. Its reliability can also be gauged by the fact that many OEMs, Tier-1 suppliers and product engineering companies are deploying Tessy for integration testing, in addition to unit testing.
