Hyperparameter Tuning

Topics

This week’s assignments will guide you through the following topics:

• Model hyperparameters
• Hyperparameter tuning using Bayesian Optimization

Reading

Please read the following:

• Read Sections Sections 2.4, 2.5, and 3.2 of An optimized XGBoost based diagnostic system for effective prediction of heart disease
• Supplemental Reading Bayesian Optimization of xgBoost

Replication task

• Starting with your trained XGBoost model, tune the nine hyperparameters the authors mention using a Bayesian Optimization approach
• Document your approach and reasoning in a clear and consise way

Tasks

Complete the following tasks:

• Read the specified sections in the Optimized XGBoost paper
• Incorporate a hyperparameter tuning step to your model construction workflow
• Evaluate your tuned model and compare to the original to gauge improvement

Weekly Questions

Answer the following questions

• What are the differences between model parameters and hyperparameters?
• In your own words, discuss why we might be interested in modifying (tuning) hyperparameters?
• What is Bayesian Optimization and how does it differ from manual or grid search approaches to hyperparameter tuning?