You are a machine learning specialist for a manufacturing company that has ingested structured and semi-structured manufacturing process data into their S3 buckets in their corporate data lake. Your data scientists now want to use SQL to run queries on this data to build manufacturing process KPI dashboards using a business intelligence tool.
Which option gives your data scientists the analysis and visualization capabilities they need most efficiently?

Answer options:

A.Transform the structured and semi-structured manufacturing process data into the parquet format using AWS Data Pipeline and then load the data into RDS from which your data scientists can run queries. Provide Kibana to your data scientists as the data visualization tool.
B.Catalog the structured and semi-structured manufacturing process data using a Glue crawler to populate your Glue data catalog. Then have your data scientists use Athena to run queries on their manufacturing data. Finally, the data scientists can build their KPI dashboards using the QuickSight Athena dataset feature.
C.Transform the structured and semi-structured manufacturing process data then load the data into Aurora using an AWS Batch ETL job. Have your data scientists use a SQL tool to query the manufacturing data stored in Aurora and visualize the results by building their KPI dashboards using the QuickSight Aurora dataset feature
D.Transform the structured and semi-structured manufacturing process data into the parquet format using a Lambda function and use Kinesis Data Analytics to run queries and build the KPI dashboard visualizations.

Answer: B
Option A is incorrect. Using AWS Data Pipeline to transform and load the data into RDS is not the most efficient option listed. Also, Kibana is best used as a visualization tool with AWS Elasticsearch, not RDS.
Option B is CORRECT. The AWS Glue crawler is the best option listed for making your manufacturing data available to a query tool like Athena by cataloging the data in your Glue data catalog. Athena is built to leverage the Glue data catalog to enable simple, efficient query capabilities for data stored in S3. Finally, QuickSight integrates directly with Athena through its Athena dataset connector. QuickSight has KPI dashboard capabilities built into it, making it the best BI visualization tool for your data scientists.
Option C is incorrect. Using AWS Aurora as the data store for your data scientist visualization work is far too complex. You would have to create the Aurora schema and database implementation. The Glue data catalog and Athena option are much more efficient.
Option D is incorrect. Using Lambda and Kinesis Data Analytics as the data source provider solution for your data scientist visualization work is far too complex. You would have to write the Lambda function code to process your manufacturing data. The AGlue data catalog and Athena option are much more efficient.
Reference:
Please see the Towards Data Science article titled Getting Started with Data Analysis on AWS.
Please refer to the AWS Big Data blog titled Analyzing Data in S3 using Amazon Athena.
Please review the AWS Big Data blog titled Build a Data Lake Foundation with AWS Glue and Amazon S3.