Validation
Overview
1. Introduction
2. Application
3. EDA
4. Learning Process
5. Bias-Variance Tradeoff
6. Regression (review)
7. Classification
8. Validation
9. Regularisation
10. Clustering
11. Evaluation
12. Deployment
13. Ethics
Lecture outline
- Validation set approach
- Overfitting
- Cross-validation
- Data leakage
- Nested cross-validation
- Bootstrapping
Validation set approach
Validation set approach
- Randomly split the original data into two, a training set and a test set.
- Fit the OLS model on the training set and predict the responses in the
validation set
- Calculate the test MSE (MSE from using the model on the test set)