<jupyter_start><jupyter_text>03 - Beware Overfitting  **Overfitting** When your model captures patterns in your training data too well - meaning it doesn't generalize well to unseen data.  **Preventing Overfitting** **Regularization:** Introducing a penalty for overly complex features that reduces - or eliminates - their weight in our model.Two common types of regularization include + **Lasso or L1 regularization**+ **Ridge or L2 regularization**.Each of these will shrink the weights of coefficients in the model. But L1 can reduce the weight for some features to zero, thereby removing them entirely from the model. ------  Data<jupyter_code>import pandas as pd
from sklearn.datasets import load_breast_cancer
cancer = load_breast_cancer()
cancer.keys()
df = pd.DataFrame(cancer['data'], columns=cancer['feature_names'])
df['target'] = pd.Series(cancer['target'])
df.head(2)<jupyter_output><empty_output><jupyter_text>--------  Modelling **Example: L1 regularization in a Logistic Regression model.**  Train/Test Split<jupyter_code>from sklearn.model_selection import train_test_split
X = df.drop('target', axis=1)
y = df['target']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
#startify=y<jupyter_output><empty_output><jupyter_text>Scale features<jupyter_code>from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaled_X_train = scaler.fit_transform(X_train)
scaled_X_test = scaler.transform(X_test)<jupyter_output><empty_output><jupyter_text>Training Model<jupyter_code>from sklearn.linear_model import LogisticRegression
lr = LogisticRegression(penalty='l1',
                       C=0.1,
                       solver='liblinear',
                       multi_class='ovr')
lr.fit(scaled_X_train, y_train)
print('Training accuracy:', round(lr.score(scaled_X_train, y_train),3))
print('Test accuracy:', round(lr.score(scaled_X_test, y_test),3))<jupyter_output>Training accuracy: 0.977
Test accuracy: 0.959
StarcoderdataJupyterStructuredCleanDedup