Regularization is done to deal with overfitting and underfitting.
Underfitting:
- Model isn’t complex enough to describe the data
- The model won’t do well in predictions
Overfitting:
- Data consist of signal and noise. While signal is the underlying relationship between the predictors and the target variables, noise is randomness in the data generation process.
- A model is overfitting, when it describes the noise instead of the signal. Prediction performance on the test set is poor then, on the train dataset it will be good.