<jupyter_start><jupyter_text>Pandas Interoperability<jupyter_code># Install dependencies for google colab
import sys
if 'google.colab' in sys.modules:
    %pip install -r https://raw.githubusercontent.com/thomasjpfan/ml-workshop-intermediate-1-of-2/master/requirements.txt
import sklearn
assert sklearn.__version__.startswith("1.0"), "Plese install scikit-learn 1.0"
import numpy as np
sklearn.set_config(display='diagram')<jupyter_output><empty_output><jupyter_text>Categorical Data<jupyter_code>import pandas as pd

df_train = pd.DataFrame({
    "pet": ["snake", "dog", "cat", "cow"],
})<jupyter_output><empty_output><jupyter_text>OridinalEncoder<jupyter_code>from sklearn.preprocessing import OrdinalEncoder
ord_encoder = OrdinalEncoder()
ord_encoder.fit_transform(df_train)
ord_encoder.categories_
df_test = pd.DataFrame({
    "pet": ["cow", "cat"]
})
df_test
ord_encoder.transform(df_test)<jupyter_output><empty_output><jupyter_text>Categories that are unknown during `fit`<jupyter_code>df_test_unknown = pd.DataFrame({
    "pet": ["bear"]
})
try:
    ord_encoder.transform(df_test_unknown)
except ValueError as err:
    print(err)<jupyter_output><empty_output><jupyter_text>How to handle unknown categories in OridinalEncoder?  Provide all the categories in the constructor<jupyter_code>df_train
ord_encoder = OrdinalEncoder(
    categories=[['snake', 'dog', 'cat', 'cow', 'bear']])
ord_encoder.fit_transform(df_train)
df_test_unknown
ord_encoder.transform(df_test_unknown)<jupyter_output><empty_output><jupyter_text>Setting a value for unknown values directly<jupyter_code>ord_encoder = OrdinalEncoder(handle_unknown='use_encoded_value', unknown_value=-1)
ord_encoder.fit_transform(df_train)
df_test_unknown
ord_encoder.transform(df_test_unknown)<jupyter_output><empty_output><jupyter_text>OneHotEncoder<jupyter_code>from sklearn.preprocessing import OneHotEncoder

ohe = OneHotEncoder()
X_trans = ohe.fit_transform(df_train)
X_trans<jupyter_output><empty_output><jupyter_text>By default it is sparse!<jupyter_code>X_trans.toarray()<jupyter_output><empty_output><jupyter_text>Switch to dense<jupyter_code>ohe = OneHotEncoder(sparse=False)
ohe.fit_transform(df_train)<jupyter_output><empty_output><jupyter_text>Unknown categories during transform?<jupyter_code>df_test_unknown
# this will fail
try:
    ohe.transform(df_test_unknown)
except ValueError as exc:
    print(exc)<jupyter_output><empty_output><jupyter_text>OHE can handle unknowns<jupyter_code>ohe = OneHotEncoder(sparse=False, handle_unknown='ignore')
ohe.fit(df_train)
ohe.transform(df_test_unknown)
ohe.categories_<jupyter_output><empty_output><jupyter_text>Two categorical features<jupyter_code>df_train = pd.DataFrame({
    "pet": ["cat", "dog", "snake"],
    "city": ["New York", "London", "London"]
})
ohe.fit(df_train)
ohe.categories_
ohe.transform(df_train)<jupyter_output><empty_output><jupyter_text>Column Transformer!<jupyter_code>import pandas as pd
X_df = pd.DataFrame({
    'age': [10, 20, 15, 5, 20, 14],
    'height': [5, 7, 6.5, 4.1, 5.4, 5.4],
    'pet': ['dog', 'snake', 'cat', 'dog', 'cat', 'cat']
})
X_df<jupyter_output><empty_output><jupyter_text>With OridinalEncoder<jupyter_code>from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler
ct = ColumnTransformer([
    ('numerical', StandardScaler(), ['age', 'height']),
    ('categorical', OrdinalEncoder(), ['pet'])
])

ct.fit_transform(X_df)<jupyter_output><empty_output><jupyter_text>With OneHotEncoder<jupyter_code>ct = ColumnTransformer([
    ('numerical', StandardScaler(), ['age', 'height']),
    ('categorical', OneHotEncoder(), ['pet'])
])
ct.fit_transform(X_df)<jupyter_output><empty_output><jupyter_text>Titanic dataset<jupyter_code>from sklearn.datasets import fetch_openml
titanic = fetch_openml(data_id=40945, as_frame=True)
X, y = titanic.data, titanic.target
y
X.head()<jupyter_output><empty_output><jupyter_text>Are three categories already encoded in the dataset?<jupyter_code>X.dtypes<jupyter_output><empty_output><jupyter_text>Are there missing values in the dataset?<jupyter_code>missing_values = pd.concat({"na_cnt": X.isna().sum(), "dtypes": X.dtypes}, axis='columns')
missing_values<jupyter_output><empty_output><jupyter_text>Split data into training and test set<jupyter_code>from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, stratify=y, random_state=42)<jupyter_output><empty_output><jupyter_text>ColumnTransformer<jupyter_code>missing_values<jupyter_output><empty_output><jupyter_text>Numerical preprocessing<jupyter_code>numerical_features = ['age', 'sibsp', 'parch', 'fare', 'body']
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer

num_prep = Pipeline([
    ('imputer', SimpleImputer()),
    ('scaler', StandardScaler())
])
num_prep<jupyter_output><empty_output><jupyter_text>Running only on numerical features<jupyter_code>num_trans = num_prep.fit_transform(X_train[numerical_features])
num_trans
num_trans.shape<jupyter_output><empty_output><jupyter_text>Categorical preprocessing<jupyter_code>categorical_features = ['sex', 'embarked']
cat_prep = Pipeline([
    ('imputer', SimpleImputer(strategy='constant', fill_value='sk_missing')),
    ('ohe', OneHotEncoder(handle_unknown='ignore', sparse=False))
])
cat_prep<jupyter_output><empty_output><jupyter_text>Running only on the categorical features<jupyter_code>cat_trans = cat_prep.fit_transform(X_train[categorical_features])
cat_trans
cat_trans.shape<jupyter_output><empty_output><jupyter_text>ColumnTransformer!<jupyter_code>ct = ColumnTransformer([
   ('numerical', num_prep, numerical_features),
   ('categorical', cat_prep, categorical_features)
])
ct
X_trans = ct.fit_transform(X_train)
X_trans[:, :5]
X_trans[:, 5:]
X_trans.shape<jupyter_output><empty_output><jupyter_text>Linear model<jupyter_code>from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
log_reg = Pipeline([
    ('preprocess', ct),
    ('log_reg', LogisticRegression(random_state=42))
])
log_reg
log_reg.fit(X_train, y_train)
log_reg.score(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Random Forest<jupyter_code>from sklearn.ensemble import RandomForestClassifier
rf = Pipeline([
    ('preprocess', ct),
    ('log_reg', RandomForestClassifier(random_state=42))
])
rf
rf.fit(X_train, y_train)
rf.score(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Exercise 11. Load the ames housing dataset using `sklearn.datasets.fetch_openml` with `data_id=41211` and `as_frame=True`. -     - **Hint**: You may ignore the version warning1. How many samples and features are there?1. Find and save the categorical and numerical feature names.    - **Hint**: You can use `X.select_dtypes(include='category').columns` and `X.select_dtypes(include='number').columns`1. What are the categorical feature names? What are the numerical feature names?1. Split the data into training and test dataset.1. Build pipeline using a `ColumnTransformer`, `OrdinalEncoder`, and `sklearn.ensemble.HistGradientBoostingRegressor` and fit on the train dataset.    - **Hint**: Use `'passthrough'` option for numerical columns.    - **Hint**: Use `OrdinalEncoder` with `handle_unknown='use_encoded_value'` and `unknown_value=-1`.1. Evaluate the model on the test set.1. **Extra**: Use `sklearn.compose.make_column_selector` instead of passing the feature names directly.<jupyter_code>from sklearn.datasets import fetch_openml
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.compose import make_column_selector<jupyter_output><empty_output><jupyter_text>**If you are running locally**, you can uncomment the following cell to load the solution into the cell. On **Google Colab**, [see solution here](https://github.com/thomasjpfan/ml-workshop-intermediate-1-of-2/blob/master/notebooks/solutions/04-ex01-solutions.py).<jupyter_code># %load solutions/04-ex01-solutions.py<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup