ProcessFin, A financial services company uses a multi-shard Kinesis streams is used to ingest database streams of applications running on Aurora databases, process and load data into Redshift which provides the DWH layer. The size of different data records from different data elements varies between 5 KB to 5 MB (de-normalized records, complex business transaction) of data. Based on the amount of the data ingested, submission of multiple vs single records programmatically in a single HTTP request is needed to maintain the throughput. The processing of data and loading into Redshift needs to be done using a pull model where the data is directly accessed from the shards of the stream.
Which data ingestion and consumer mechanism mentioned below could fulfil the above requirement? Select 2 options

Answer options:

A.KPL Library with data aggregation, for data ingestion
B.Kinesis Data Streams API for data ingestion
C.Kinesis Agent for data ingestion
D.Consumers based on KCL Library
E.Consumers based on Kinesis Data Streams API
F.KPL Library with data collection, for data ingestion

Answer : B, E
Option A is incorrect - The limits of the data ingested by KPL library is capped at 1000 records/sec and 1 MB throughput per shard.The maximum size of a user record is capped at 1 MB. Though KPL supports both aggregation and collection, there is a limitation with the amount of data that is being processed.
https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-kpl.html
Option B is correct - Streams API, using PutRecords operation sends multiple records to Kinesis Data Streams in a single request. By using PutRecords, producers can achieve higher throughput when sending data to their Kinesis data stream. Each PutRecords request can support up to 500 records. Each record in the request can be as large as 1 MB, up to a limit of 5 MB for the entire request, including partition keys. Also the platform programmatically supports changing between submissions of single records versus multiple records in a single HTTP request.
https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-sdk.html
Option C is incorrect - Kinesis Agent is a stand-alone Java software application that offers an easy way to collect and send data to Kinesis Data Streams. The agent continuously monitors a set of files and sends new data to your stream. The agent handles file rotation, checkpointing, and retry upon failures. It delivers all of your data in a reliable, timely, and simple manner.
https://docs.aws.amazon.com/streams/latest/dev/writing-with-agents.html
Option D is incorrect -Consumers based on KCL library use push mechanism. use the record processor support provided by the Kinesis Client Library (KCL) to retrieve stream data in consumer applications. This is a push model, where you implement the code that processes the data. The KCL retrieves data records from the stream and delivers them to your application code.
https://docs.aws.amazon.com/streams/latest/dev/developing-consumers-with-sdk.html
Option E is correct - The Kinesis Data Streams API provides the getShardIterator and getRecords methods to retrieve data from a stream. This is a pull model, where your code draws data directly from the shards of the stream.
https://docs.aws.amazon.com/streams/latest/dev/developing-consumers-with-sdk.html
Option F is incorrect - The limits of the data ingested by KPL library is capped at 1000 records/sec and 1 MB throughput per shard.The maximum size of a user record is capped at 1 MB. Though KPL supports both aggregation and collection, there is a limitation with the amount of data that is being processed.
https://docs.aws.amazon.com/streams/latest/dev/developing-producers-with-kpl.html