You work for a machine learning team at a global retail auto parts chain. Your team ingests purchasing data from its 100,000 global auto parts stores to S3 using Kinesis Data Firehose. You are now ready to start training an improved machine learning model that will be used to predict purchasing patterns by global region. The training data requires additional simple transformations. Also, you will need to combine some data attributes. Finally, your team expects to train the model on a daily basis.
Based on a large number of stores plus changing data ingestion, which of the following options will require the least amount of administration and development effort?

Answer options:

A.Have the stores capture their purchasing data locally on Storage Gateway and then load the data into S3. Transform the data using Glue.
B.Create an EMR cluster with Apache Spark installed to perform the transformation logic. Run the cluster each day on the incremental records in S3, and write the transformed records to S3.
C.Create a fleet of EC2 instances that run the transformation logic which transforms the incremental data records on S3, and write the transformed records to S3.
D.Create a Kinesis Data Analytics stream and use it as the destination of the Kinesis Data Firehose stream. Use Kinesis Data Analytics to transform the raw purchasing data attributes into transformed values using SQL and write the transformed data to S3.

Answer: D
Option A is incorrect. Having 100,000 stores use Storage Gateway to move the data to S3 would require a tremendous administrative effort.
Option B is incorrect. Using EMR for this solution would require administrative cost to build and maintain EMR. Also, your development team would have to write the Apache Spark code to perform the transformations.
Option C is incorrect. Using a fleet of EC2 instances would require the administrative cost of creating and maintaining the EC2 instances. Also, your development team would have to write the transformation logic that runs on the EC2 instances.
Option D is CORRECT. Kinesis Data Analytics can receive your data from Kinesis Data Firehose, transform it, and then write it to S3. The code needed to perform the transformations in Kinesis Data Analytics would be much simpler than the coding suggested in the other options. Your machine learning model can then use the transformed data in S3 for training.
Reference:
Please see the Amazon Kinesis Data Analytics developer guide titled Example: Writing to an Amazon S3 Bucket.
Please refer to the Amazon Kinesis Data Firehose developer guide titled Using Amazon Kinesis Data Analytics.