Traceable Test Data Management for Validation and Homologation of Autonomous Driving Functions

Currently we hear almost every day of new strategies, standards and tools for the validation and homologation of autonomous driving functions. For that, in all stages of the validation process huge amount of data are required, like e.g. recorded or generated driving scenarios, test parameter sets as well as sensor and bus data coming from manifold simulations, test benches and real road tests. They are used to evaluate the behavior and proof of safety of the respective function under test.

But so far, OEM´s and suppliers do the validation of autonomous driving functions mainly within fragmented tool chains and closed data loops. Even most of the working groups currently busy with the standardization of data formats or the development of open source tools still think within their domain specific boundaries, focusing merely on the interoperability of single tools.

Recently, the ASAM Test Specification Study Group emphasized for the first time that a comprehensive test data management approach is key to an efficient and traceable validation process. It became clear, that building up shared file directories or proprietary data silos based on vendor-specific tools cannot be a solution in the long-term, because this makes it difficult to find, share and reuse the necessary data among team members and across teams.

Experts agree: A standardized data model for storing different types of data including the associated descriptive information is required. This has to be supplemented by an open data access layer allowing different tools to retrieve test parameters and store test results during the ADAS development and homologation. The solution approach has to ensure that all the verification and validation activities performed are traceable and audit-proof across departmental boundaries. In this way, developers, technical services and authorities can compile and assess the data relevant for  them with manageable effort.

Due to its versatility, it is obvious to use ASAM ODS for implementing test data management solutions in the ADAS domain as well. The standard has proven itself for many years as a reliable and powerful foundation for the persistent storage and retrieval of all types of test data in a variety of test domains.

ASAM ODS provides a so-called “base model” from which an “application model” for describing, storing and managing all relevant ADAS-specific artifacts can be derived. This includes, for example, requirement definitions and test cases with reference to driving scenarios and environmental descriptions. In addition, test specifications can be described with regard to the functions to be tested and the devices and test setups used for the respective tests. And of course, the data model of ASAM ODS also allows to manage all the data captured, measured or calculated during a test run and the results determined from it. A relation to other ADAS-relevant standards or original data files can be established for each data object at any time.

A well-defined, HTTP-based Application Programming Interface (API) allows all the data objects stored in the ASAM ODS application model to be accessed. Despite the flexibility of the data model mentioned above, each ASAM ODS-compliant client can correctly interpret the meaning of the data while querying them from the data base.

In practice, well-known companies have taken advantage of this and already been successful in implementing ASAM ODS-based solutions for describing, storing and managing test data coming from the ADAS domain. It has been shown that even very large amounts of data (e.g. recorded driving scenarios as well as log and trace data) can be managed reliably with ASAM ODS.

Given that the overall conceptual approach of the test data management solution is correct, companies can also take advantage of the BigODS Associate Standard. It specifies the storage of ASAM ODS data in big data ecosystems. Via the Peak ODS Server and the Peak ODS Spark Adapter, test engineers and data scientists can then easily access the ASAM ODS data stored in different IT environments and process it using a variety of commercial analysis products from different vendors or free available programming tools such as Python.

About ASAM ODS

ASAM ODS was one of the first standards developed by ASAM, reaching back as far as the early 1990th. Today, the standard is part of many comprehensive solutions for the enterprise-wide storage and retrieval of test-related data. ASAM ODS is widely used by German OEMs, including BMW, Daimler and Porsche, who initiated the standard. Audi started in 1999 the “Measurement Data Management” project based upon ASAM ODS and moved major parts of the core software to an open source project by 2008. Well-known OEMs and suppliers participated in this project.

Over the years, vendors like Peak Solution have developed powerful ASAM ODS servers and complementary tools. They help with the rapid implementation of comprehensive test data management solutions. Furthermore, many vendors of measurement and analysis systems have expanded their products to include the specifications of the standard and thus made them ASAM ODS-compliant.

Some of the tool and service providers have specialized in certain areas of application for ASAM ODS, such as NVH. Others, like Peak Solution, offer cross-departmental enterprise solutions for different test domains that seamlessly integrate with the company’s existing or planned measurement, analysis and management systems. These solutions are provided either as an on-site application in the company’s IT-infrastructure (on-premise), as a cloud service (software as a service) or as part of a hybrid approach.

Against this background, the use of ASAM ODS in Europe and North America has been increasing steadily for years. Since 2016, ASAM has supported a regional ASAM ODS interest group in Japan. This further strengthened the position of ASAM ODS in Japan. Well-known companies in China have recently started to show an interest in the use of ASAM ODS as well.

All together the ASAM ODS standard and supplementing tools can be considered as mature and stable. They enable test fields manager a resilient and at the same time sustainable digital transformation.