You work as a machine learning specialist for a social media software company. Your company produces social media game apps. Your machine learning team has been asked to produce a machine learning model to predict user purchase of apps similar to apps they have already purchased. You have created a model based on the SageMaker built-in XGBoost algorithm. You are now using hyperparameter tuning to get the best performing model for your problem. Which evaluation metrics and corresponding optimization direction should you choose for your automatic model tuning (a.k.a. hyperparameter tuning)? (Select TWO)

Answer options:

A.f1, minimize
B.map, minimize
C.ndcg, maximize
D.rmse, maximize
E.mae, minimize

Correct Answers: C and E
Option A is incorrect. XGBoost uses the f1 metric for model validation. However, you will want to maximize this metric.
Option B is incorrect. XGBoost uses the map (mean average precision) metric for model validation. However, you will want to maximize this metric.
Option C is correct. XGBoost uses the ndgc (Normalized Discounted Cumulative Gain) metric for model validation, and you will want to maximize this metric.
Option D is incorrect. XGBoost uses the rmse (Root mean square error) metric for model validation, and you will want to minimize this metric.
Option E is correct. XGBoost uses the mae (Mean Absolute Error) metric for model validation, and you will want to minimize this metric.
References:
Please see the Amazon SageMaker developer guide titled Define Metrics (https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-define-metrics.html),
The Amazon SageMaker developer guide titled Tune an XGBoost Model (https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost-tuning.html)