Peak Spark ODS Adapter: Gain additional insights from the existing measurement data in ASAM ODS

A widely used standard for the persistent storage of test data is ASAM ODS. Automotive manufacturers and suppliers use it primarily to set up overarching test data management solutions for different measurement systems and data formats. As a result, their test data is prepared for long-term availability, regardless of the tools of individual vendors.
To gain more insight from their existing measurement data, Peak Solution has developed the Peak Spark ODS Adapter.

The Peak Spark ODS Adapter allows you to access instance and mass data of ASAM ODS using Apache Spark SQL and DataFrames. You can compare it to the functionality of the interactive Peak Test Data Workplace web client but as programming experience to execute arbitrarily complex queries on it and use the returned DataFrames to apply further analytics and statistics of the data.

Apache Spark provides several high-level APIs such as Python and other which can for instance perfectly be used in Apache Zeppelin or Jupyter Notebooks for the interactive analysis of the measurement data. The great strength of notebooks is explorative evaluations and data mining. This allows you to gain an overview of the data first to then incrementally reach your analysis goals. Additionally, sharing your ideas is as easy as sending your notebook around or provide it in the notebook development environment of your organization where authorized persons can then view and continue using it.

An important advantage of the Peak Spark ODS Adapter is its scalability. The implementation of the Peak Spark ODS Adapter makes it possible to perform complex data queries and data analysis on large amounts of data with high speed. Depending on the number of licensed executors and your Apache Spark cluster you can scale from local (Standalone) to high-performance clusters.

Furthermore, the Peak Spark ODS Adapter implements the ASAM ODS Big Data Connector – a standardized way to export contents of your Peak ODS Server in form of AVRO or Parquet files. These exports integrate with an existing big data eco systems such as Hadoop.

As a result, companies can leverage Peak ODS Spark Adapter within their existing ODS infrastructure without extensive IT investments before gradually building it into a powerful big data cluster for managing and analyzing huge amounts of test data.
 According to the motto: Start small, think tall.