Machine Learning · Chapter 29 of 40

Regularization (L1 / L2)

REGULARIZATION penalises large coefficients to reduce overfitting. L2 (Ridge) shrinks all coefficients. L1 (Lasso) drives some to zero, giving automatic feature selection.

Elastic Net combines both.

Example 1 (python)

from sklearn.linear_model import Ridge, Lasso
ridge = Ridge(alpha=1.0).fit(X_train, y_train)
lasso = Lasso(alpha=0.1).fit(X_train, y_train)

alpha controls strength.

Example 2 (python)

print((lasso.coef_ == 0).sum(), 'zeroed coefficients')

Output

12 zeroed coefficients

L1 removes features.

Key points

Reduces overfitting.
L1 zeroes coefficients (feature selection).
L2 shrinks all coefficients.
Alpha controls strength.

💡 Note: Always scale features before applying regularization — otherwise the penalty hits large-scale features harder.

Regularization (L1 / L2)

Key points

📝 Quick Quiz