You work for a global consulting company as a machine learning specialist. You work with a team of data scientists that continually create datasets for your consultancy’s analysis and trend prediction work using machine learning. You have been assigned the job of creating a data repository to store the large amount of training data generated by your data scientists for use in your machine learning models. You do not know how many new datasets your data scientists will create each day, so your solution must scale automatically, and your management team wants the storage solution to be cost-effective. Also, the data scientists and machine learning specialists must be able to query the data using SQL. Which option is the best solution to meet your requirements?

Answer options:

A.Have your data scientists store their new datasets in DynamoDB using global tables.
B.Have your data scientists store their new datasets as tables in a Redshift cluster using RA3 nodes with managed storage and Redshift Spectrum.
C.Have your data scientists store their new datasets as files in an EFS attached to EC2 instances instance.
D.Have your data scientists store their new datasets as files in S3.

Correct Answer: D
Option A is incorrect. DynamoDB will not be the most cost-effective option when compared to S3. Also, there is no requirement suggesting the need for global data distribution. Also, when you need to use the data in your machine learning models, you will have to extract the data from DynamoDB and put it to S3.
Option B is incorrect. Redshift using RA3 node types with managed storage will give you very fast query access to your data, but it is not cost-effective when compared to S3. Also, when you need to use the data in your machine learning models, you can leverage Redshift Spectrum to extract the data from Redshift and put it to S3. But this adds another level of complexity and cost.
Option C is incorrect. Using EC2 instances and EFS volumes to store your data is very cost-ineffective when compared to using S3. Also, when you need to use the data in your machine learning models, you will have to move the data from the EC2 EFS volumes to S3.
Option D is correct. S3 is cost-effective, scales infinitely to any dataset volume and size, and it is where you need to have your data when you use it in your machine learning models.
References:
Please see the Amazon Machine Learning blog titled Building secure machine learning environments with Amazon SageMaker (https://aws.amazon.com/blogs/machine-learning/building-secure-machine-learning-environments-with-amazon-sagemaker/),
The Amazon Redshift products page titled Amazon Redshift Pricing (https://aws.amazon.com/redshift/pricing/),
The Amazon DynamoDB products page titled Amazon DynamoDB pricing (https://aws.amazon.com/dynamodb/pricing/)