You are a machine learning specialist at a company that is exploring conversational user interface application development. As an experiment, your team is building a natural language processing application. Your application needs to process the transcribed conversation data from your conversational user interface. For training, you are starting with a dataset comprising 5 million sentences. You plan to run a model based on the Word2Vec algorithm to generate embeddings of the sentences. This will allow your team to make different types of predictions.
Based on this example sentence: “My funy LARGE MEME went over the audiences head.”
Which operations should your team perform to sanitize and prepare the data in a repeatable manner? (CHOOSE THREE)

Answer options:

A.Correct the spelling of "funy" to "funny" and “audiences” to “audience’s.”
B.Perform normalization by making the sentence lowercase.
C.Using an English stopword dictionary, remove all stop words.
D.Use One-hot encoding on the sentence.
E.Use part-of-speech tagging to keep the action verbs and the nouns only.
F.Perform tokenization of the sentence, creating a word vector.

Answers: B, C and F
Option A is incorrect. In natural language processing, the spelling of words has a relatively lower bearing on the importance of the word.
Option B is CORRECT. Using normalization, you change all text so that it is on the same level. For example, converting all characters to lowercase. This allows algorithms like Bag Of Words and Word2Vec to perform more accurately.
Option C is CORRECT. Removing stop words like not, nor, never, etc., which are the most common words in a given language. Removing these words allows your algorithm to focus on differentiating words.
Option D is incorrect. One-hot-encoding is a technique used to encode categorical data.
Option E is incorrect. Using part-of-speech tagging to keep only the action verbs and nouns, you would strip the conversation data of much of its meaning. The example sentence would become: Meme went head.
Option F is CORRECT. NLP algorithms like Word2Vec work best with tokenized data as their input.
Reference:
Please see the article titled Natural Language Processing: Text Data Vectorization.
Please refer to the Towards Data Science article titled NLP: Extracting the main topics from your dataset using LDA in minutes.
Please see the Towards Data Science article titled NLP Text Preprocessing: A Practical Guide and Template.
Please see the Towards Data Science article titled 3 basic approaches in Bag of Words which are better than Word Embeddings.
Please see the Towards Data Science article titled Treat Negation Stopwords Differently According to Your NLP Task.
Please see the Machine Learning Mastery article titled Why One-Hot Encode Data in Machine Learning?.
Please see the article titled How to get started with Word2Vec — and then how to make it work.