You work for a marketing firm that wants to analyze Twitter user stream data to find popular subjects among users who buy products produced by the firm’s clients. You need to analyze the streamed text to find important or relevant repeated common words and phrases and correlate this data to client products. You’ll then include these topics in your client product marketing material.
Which of the following text feature engineering techniques is the best solution for this task?

Answer options:

A.Orthogonal Sparse Bigram (OSB)
B.Term Frequency-Inverse Document Frequency (tf-idf)
C.Bag-of-Words
D.N-Gram

Answer: B
Option A is incorrect. The Orthogonal Sparse Bigram natural language processing algorithm creates groups of words and outputs the pairs of words that include the first word. You are trying to determine how important a word is in a document by finding relevant repeated common words.
Option B is correct. Term Frequency-Inverse Document Frequency determines how important a word is in a document by giving weights to words that are common and less common in the document. You can use this information to select the most important repeated phrases in the user’s tweets in your client marketing material.
Option C is incorrect. The Bag-of-Words natural language processing algorithm creates tokens of the input document text and outputs a statistical depiction of the text. The statistical depiction, such as a histogram, shows the count of each word in the document. You are looking for relevant common repeated phrases, not individual words.
Option D is incorrect. The N-Gram natural language processing algorithm is used to find multi-word phrases in the text of a document. However, it does not weigh common words or phrases. You need the weighting aspect of the tf-idf algorithm to find the relevant, important repeated phrases used in the tweets.
Reference:
Please see the article titled Introduction to Natural Language Processing for Text.