You are a data scientist working for a cancer screening center. The center has gathered data on many patients that have been screened over the years. The data is obviously skewed toward true negative results, as most screened patients don’t have cancer. You evaluate several machine learning models to decide which model best predicts true positives when using your cancer screening data. You have split your data into a 70/30 ratio of the training set to the test set. You now need to decide which metric to use to evaluate your models.
Which metric will most accurately determine the model best suited to solve your classification problem?

Answer options:

A.ROC Curve
B.Precision
C.Recall
D.PR Curve

Answer: D
Option A is incorrect because it is best used when both outcomes have equal importance. Due to the importance of true negative in this equation, it will not differentiate models well for the cancer screening problem, since this data set is skewed to true negatives. The true negative cases are heavily weighted in the equation, thus amplifying the impact of the imbalance.
Option B is incorrect because it only takes into account the percentage of positive cases out of the total predicted positive.
Option C is incorrect because it only takes into account the percentage of positive cases out of the total actual positive.
Option D is correct because the PR Curve is best used to evaluate models on data sets where most of the cases are negative, as in the cancer screening data set. The true negative cases are not weighted heavily in the equation, thus reducing the impact of the imbalance.
Reference:
Please see the article Various ways to evaluate a machine learning model’s performance.