You work as a machine learning specialist for an electric bicycle company. The electric bicycles your company produces have IoT sensors on them that transmit usage and maintenance information to your company data lake. You are using Kinesis Data Streams to gather the bicycle IoT data and store it into an S3 data store that you can use for your machine learning models. You are on the team that has the assignment of using the IoT data to predict when your customer’s electric bicycles need maintenance.
The IoT data that the electric bicycles produce is unstructured, and sometimes, depending on the manufacturer of the IoT part, the data has a different schema structure. You need to clean and classify the IoT data before using it in your machine learning model. How can you build an ETL script to perform the necessary cleaning and classification knowing that you have message data with differing schema structures?

Answer options:

A.Use AWS Glue to build a series of transforms that use Apache Spark SparkSQL DataRecord to pass the data from transform to transform. Each transform performs a different cleaning and/or transforming task.
B.Use AWS Glue to build a series of transforms that use Apache Spark SparkSQL DataFrames to pass the data from transform to transform. Each transform performs a different cleaning and/or transforming task.
C.Use AWS Glue to build a series of transforms that uses DynamicFrames to pass the data from transform to transform. Each transform performs a different cleaning and/or transforming task.
D.Use AWS Glue to build a series of transforms that uses DynamicRecord to pass the data from transform to transform. Each transform performs a different cleaning and/or transforming task.

Answer: C
Option A is incorrect. There is no DataRecord construct in Apache Spark SparkSQL.
Option B is incorrect. The Apache Spark SparkSQL DataFrame does not efficiently handle data with unknown schema structure. This option would produce suboptimal results.
Option C is correct. The AWS Glue DynamicFrame allows each record to be self-describing to handle unknown or changing schemas.
Option D is incorrect. DynamicRecord represents a logical record within a DynamicFrame. It is a row in a DynamicFrame. So you wouldn’t pass individual DynamicRecords from transform to transform. You pass a DynamicFrame.
Reference:
Please see the AWS Glue developer guide titled Machine Learning Transforms in AWS Glue, and the AWS Glue developer guide titled DynamicFrame Class.