Automotive suppliers incorporate more and more microprocessors in vehicles – from powertrain to Driver Assistance (ADAS) to Infotainment to Electronic Control Units (ECU). As a result, software complexity increases with each additional component or subsystem. Managing the implementation, operation, and quality of that additional complexity presents a challenge. Automotive suppliers want to deliver increased functionality while preserving operational safety standards.
InterWorking Labs provides communications testing solutions for the connected car. Our protocol tests address the Transport Layer (TCP) and the Internet Layer (IP) of the Internet Protocol Suite to verify the correctness and robustness of the implementations.
In addition, our communication performance tests reveal how well the automotive components and subsystems perform under adverse network conditions, so that the component provider can anticipate, plan and address component behavior under these conditions.
The figure below illustrates the OSI model of communications networking. InterWorking Labs tests for the correct implementation of the protocols at Layer 3 the Network Layer and Layer 4 the Transport Layer.
Layer 2 the Data Link Layer and Layer 1 the Physical Layer represent the lower level electronic signaling and physical interfaces supporting the network communication. The chip and card suppliers for these products provide tests for the protocol implementation.
Typically communications performance testing for connected cars means a mobile network.
Mobile networks perform more poorly than their wired network counterparts on four performance metrics: Throughput, Round Trip Time (RTT), Jitter and Packet Loss.
The component or subsystem designer must tune and improve the network communication algorithms to yield optimum performance. However, in some conditions, the network will prove inadequate and the user must be notified to “try again later”.
Lear More about Mobile Network Performance Testing here.
Components and Subsystems that Require Testing
Components exchanging data with another component, another vehicle, a satellite or other external communication nodes (vehicle-to-infrastructure communication) require network communications testing:
- safety systems
- driver assistance (ADAS)
Functional Tests include:
- Negative Testing (Fuzz Testing)
- Inopportune Testing
- Conformance/Compliance Testing
- Interoperability Testing
- Deep-path Testing
Stress and Reliability Tests include:
- Load Testing
- Stress Testing
- Performance Testing
- Line Speed Testing
- Robustness (Security) Testing
- Endurance Testing
Read our White Paper Network Protocol Testing Overview to learn more.
Read a White Paper
The AUTomotive Open System Architecture is used for most ECUs (electronic control units). (See diagram). For infotainment and driver assistance (NAV), the original AUTOSAR standard did not scale well, so developers used embedded operating systems, such as – Elektrobit, Euros, Freescale, Green Hills, Linux, Mentor Graphics, QNX and many others.
AUTOSAR classic did not scale well for Infotainment and Driver Assistance, but the future version does.
InterWorking Labs has an AUTOSAR Testing Solution.
The automotive supplier may choose to:
- Read the standards specifications and do the implementation.
- License an AUTOSAR implementation from an embedded system supplier
- Implement an open source solutions
Why is this important?
TCP, IPv4, IPv6 related protocols are used in AUTOSAR and open source embedded software.
Unfortunately, many of the existing TCP/IP stacks or engines are not sufficiently robust for automotive quality standards.
The testing results for a popular desktop operating system are published here and reveal the poor quality of implementations that were “certified” at the highest levels.
How does one effectively test?
The TCP family is used in AUTOSAR.
How do we test for robustness, security, interoperability and conformance/compliance?
Use Maxwell Pro TCP/IP Test Suite to send pathological TCP segments to the device under test.
- The device will have to respond appropriately.
- It should not hang or crash.
- It should exhibit the proper behavior as defined in the standards document
Example 1 - TCP “Negative” Test:
Negative testing verifies that the device under test responds correctly to error conditions or unacceptable input conditions. Datagrams are fragmented into MTU sized chunks, starting at the front of the datagram. The fragments overlap with fragment N+1 overlapping the end of fragment N by 8 bytes (one Network Fragmentation Block). E.g. Here is a sequence of NFBs in the fragments each holding 6 NFBs:
pkt 0: ABCDEF
pkt 1: FGHIJK
pkt 2: KLMNOP
The MTU is 512. A final short fragment may occur at the end. Fragments are transmitted in orderof increasing segment offset.
The fragments must be reassembled into the original packet by the receiving IP layer and delivered to the next higher layer.
This test we just observed is also an example of a Conformance/Compliance test, because the requirements are defined in the following specifications:
RFC 791, sections 2.3, 3.2.
RFC 1122, section 3.3.2.
Thus, a device that does not pass this test, like Windows 7, is not in conformance with the RFCs.
Example 2 - TCP “Robustness - Security Test":
Robustness testing is the process of subjecting a device under test to particular input streams in an attempt to cause the device to fail. The input streams may be one of three types:Random input streams Valid input streams, and Invalid input streams.
Our aim is a precise, controlled, repeatable input stream.
- Enough fragments are sent to create 65536 bytes of data in any destination buffer (one more that is allowed.)
- The MTU is 512 + header length. A final fragment of 8 bytes occurs at the end.
- Fragments are transmitted in order of increasing segment offset.
The fragments must eventually be discarded before anything is delivered to upper protocol layers. The device under test must not crash! This test is also an example of a Conformance/Compliance test, because the requirements are defined in the following specifications:
RFC 791, pages 8, 13, 24, 25
RFC 1122, section 3.3.2.
To learn more about Infotainment and Driver Assistance Testing click here.