You work as a machine learning specialist for an analytics firm that produces machine learning models for clients that want to purchase data analysis on things like estimates for efficacy of advertising campaigns. You are currently working on an estimator for the effectiveness of a proposed direct mailing campaign. You have gathered your data, performed feature engineering and chosen the XGBoost algorithm for your model. Now you are ready to tune your hyperparameters for your model training. Which configuration strategy will give you the best model performance?

Answer options:

A.Large learning rate, small number of estimators, without early stopping.
B.Large learning rate, large number of estimators, with early stopping.
C.Small learning rate, large number of estimators, with early stopping.
D.Small learning rate, large number of estimators, without early stopping.

Correct Answer: C
Option A is incorrect. A large learning rate with a small number of estimators without using early stopping will cause your model to oscillate.
Option B is incorrect. A large number of estimators with early stopping is a good set of configurations, but a large rate will cause your model to oscillate.
Option C is correct. Using a small learning rate, a large number of estimators, with early stopping will allow your model to find the correct number of estimators.
Option D is incorrect. Using a small learning rate, a large number of estimators without early stopping will make your training run very long and probably overfit your data.
Reference:
Please see the Kaggle article titled XGBoost (https://www.kaggle.com/alexisbcook/xgboost), and the Amazon SageMaker developer guide titled XGBoost Hyperparameters (https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost_hyperparameters.html)