02 L2 regularization

L2 regularization technically refers to any regularization technique that uses the Euclidean distance (-norm) to apply a penalty in a loss function. It is typically used in reference to parametric models, where the penalty is proportional to the sum of squares of the weights.

L2 regularization can help to prevent overfitting, and can help to address collinearity of features by spreading weight across all of the correlated features. However, it is unlikely to reduce any particular feature’s weight to zero, and therefore does not contribute to model sparsity; for this, L1 regularization is needed. L1 and L2 regularization can be used together; this is often a good approach for linear models.