A company has two batch processing applications that consume financial data about the day`s stock transactions. Each transaction needs to be stored durably and the order of transactions needs to be guaranteed so that the billing and audit batch processing applications can process the data. The billing application firstly processes the transaction information and after several hours, the audit application access to the same data. After the transaction information for the day is processed, the information no longer needs to be stored. What is the best way to architect this application so that the above requirements are achieved? Choose the correct answer from the options below.

Answer options:

A.Use SQS for storing the transaction messages. When the billing batch process consumes each message, have the application create an identical message and place it in a different SQS for the audit application to use several hours later.
B.Use SQS for storing the transaction messages; when the billing batch process performs first and consumes the message, write the code in a way that does not remove the message after consumed, so it is available for the audit application several hours later. The audit application can consume the SQS message and remove it from the queue when completed.
C.Store the transaction information in a DynamoDB table. The billing application can read the rows while the audit application will read the rows then remove the data.
D.Use Kinesis to store the transaction information. The billing application consumes data from the stream and then the audit application consumes the same data several hours later.

Answer – D
The main architectural considerations in this scenario are: (1) each transaction needs to be stored durably (no loss of any transaction), (2)they should be processed in serial order, (3) guaranteed delivery of each record to the audit and billing batch processing, and (4) the processing of the record by two application is done with a time gap of several hours.
Based on the considerations above, it seems that we must use Amazon Kinesis Data Streams which enables real-time processing of streaming big data. It provides ordering of records and the ability to read and/or replay records in the same order to multiple Amazon Kinesis Applications. The Amazon Kinesis Client Library (KCL) delivers all records for a given partition key to the same record processor, making it easier to build multiple applications reading from the same Amazon Kinesis data stream (for example, to perform counting, aggregation, and filtering).
Option A is incorrect because (a) the onus of ensuring that each message is copied to the audit queue is on the application, and (b) this option is not fail-proof. i.e. If the application fails to copy the message between the queue, there is no logic to put the message back to the billing queue.
Option B is incorrect because even though it uses SQS, there is an overhead of ensuring that the message is processed properly by the billing and audit processes. When the billing process is processing the message, the message is unavailable for the audit process. Also, there is a possibility of processing the same record (message) multiple times by the instances processing it (there is no way to know if the record has been already processed).
Option C is incorrect because it adds the overhead of delivery guarantee on the application itself. Also, the use of DynamoDB in this scenario is not a cost-effective solution. To store the transaction data in real-time data, Kinesis is a better choice.
Option D is CORRECT because Amazon Kinesis is best suited for applications that need to process large real-time transactional records and have the ability to consume records in the same order a few hours later. For example, you have a billing application and an audit application that runs a few hours behind the billing application. Because Amazon Kinesis Data Streams stores data for up to 7 days, you can run the audit application up to 7 days behind the billing application.
More information on when Amazon Kinesis Data Streams and Amazon SQS should be used:
AWS recommends Amazon Kinesis Data Streams for use cases with requirements that are similar to the following:
Routing related records to the same record processor (as in streaming MapReduce). For example, counting and aggregation are simpler when all records for a given key are routed to the same record processor.
Ordering of records. For example, you want to transfer log data from the application host to the processing/archival host while maintaining the order of log statements.
Ability for multiple applications to consume the same stream concurrently. For example, you have one application that updates a real-time dashboard and another archives data to Amazon Redshift. You want both applications to consume data from the same stream concurrently and independently.
Ability to consume records in the same order a few hours later. For example, you have a billing application and an audit application that runs a few hours behind the billing application. Because Amazon Kinesis Data Streams stores data for up to 7 days, you can run the audit application up to 7 days behind the billing application.
AWS recommends Amazon SQS for use cases with requirements that are similar to the following:
Messaging semantics (such as message-level ack/fail) and visibility timeout. For example, you have a queue of work items and want to track each item`s successful completion independently. Amazon SQS tracks the ack/fail, so the application does not have to maintain a persistent checkpoint/cursor. Amazon SQS will delete asked messages and redeliver failed messages after a configured visibility timeout.
Individual message delay. For example, you have a job queue and need to schedule individual jobs with a delay. With Amazon SQS, you can configure individual messages to have a delay of up to 15 minutes.
Leveraging Amazon SQS’s ability to scale transparently. For example, you buffer requests and the load changes as a result of occasional load spikes or the natural growth of your business. Because each buffered request can be processed independently, Amazon SQS can scale transparently to handle the load without any provisioning instructions from you.
https://aws.amazon.com/kinesis/data-streams/faqs/