You work for a financial services company where you have a large Hadoop cluster hosting a data lake in your on-premises data center. Your department has loaded your data lake with financial services operational data from your corporate actions, order management, cash management, reconciliations, and trade management systems. Your investment management operations team now wants to use data from the data lake to build financial prediction models. You want to use data from the Hadoop cluster in your machine learning training jobs. Your Hadoop cluster has Hive, Spark, Sqoop, and Flume installed.
How can you most effectively load data from your Hadoop cluster into your SageMaker model for training?

Answer options:

A.Use the distcp utility to copy your dataset from your hadoop platform to the S3 bucket where your SageMaker training job can use it.
B.Use the HadoopActivity command with AWS Data Pipeline to move your dataset from your hadoop platform to the S3 bucket where your SageMaker training job can use it.
C.Use the SageMaker Spark library using the data frames in your Spark clusters to train your model.
D.Use the Sqoop export command to export your dataset from your Hadoop cluster to the S3 bucket where your SageMaker training job can use it.

Answer: C
Option A is incorrect. The Hadoop distcp utility is used for inter/intra cluster data movement. It is not an efficient method to get data into your SageMaker training instance. (See the Apache Hadoop distcp guide)
Option B is incorrect. The HadoopActivity command is used to run a job on a cluster. You would have to write the job to extract and load the data onto S3. This would not be the most efficient method of the options listed. (See AWS Data Pipeline developer guide titled HadoopActivity)
Option C is correct. The SageMaker Spark library makes it; so you can easily train models using data frames in your Spark clusters. This is the most efficient method of the options listed. (See the Amazon SageMaker developer guide titled Use Apache Spark with Amazon SageMaker)
Option D is incorrect. The Sqoop export command is used for exporting files from HDFS to an RDBMS. This would not help you load your data into your SageMaker training instance. (See the Sqoop User Guide)
Reference:
Please see the Amazon SageMaker developer guide titled Use Machine Learning Frameworks with Amazon SageMaker.