# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3 (ipykernel)
#     language: python
#     name: python3
# ---

# # Bagging Exercises
#
# ![Bagging.png](Assets/Bagging.png)

# # Learning Objectives
#
# - Associate concepts of bootstrapping and aggregating (commonly known as “bagging”) to reduce variance
# - Apply Random Forest algorithm that further reduces the correlation seen in bagging models
# - Apply Intel® Extension for Scikit-learn* to leverage underlying compute capabilities of hardware
#
# # scikit-learn* 
#
# Frameworks provide structure that Data Scientists use to build code. Frameworks are more than just libraries, because in addition to callable code, frameworks influence how code is written. 
#
# A main virtue of using an optimized framework is that code runs faster. Code that runs faster is just generally more convenient but when we begin looking at applied data science and AI models, we can see more material benefits. Here you will see how optimization, particularly hyperparameter optimization can benefit more than just speed. 
#
# These exercises will demonstrate how to apply **the Intel® Extension for Scikit-learn*,** a seamless way to speed up your Scikit-learn application. The acceleration is achieved through the use of the Intel® oneAPI Data Analytics Library (oneDAL). Patching is the term used to extend scikit-learn with Intel optimizations and makes it a well-suited machine learning framework for dealing with real-life problems. 
#
# To get optimized versions of many Scikit-learn algorithms using a patch() approach consisting of adding these lines of code PRIOR to importing sklearn: 
#
# - **from sklearnex import patch_sklearn**
# - **patch_sklearn()**
#
# ## This exercise relies on installation of  Intel® Extension for Scikit-learn*
#
# If you have not already done so, follow the instructions from Week 1 for instructions

# ## Introduction
#
# We will be using the customer churn data from the telecom industry that we used in week 1 for this week's exercises. The data file is called `Orange_Telecom_Churn_Data.csv`. 

# +
from __future__ import print_function
import os
data_path = [ 'data']

from sklearnex import patch_sklearn
patch_sklearn()

from sklearn.model_selection import StratifiedShuffleSplit
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.metrics import classification_report, accuracy_score, precision_score, recall_score
from sklearn.metrics import f1_score, roc_auc_score
from sklearn.metrics import roc_curve, precision_recall_curve, confusion_matrix

# -

# ## Question 1
#
# * Import the customer churn data, which is found in the file `Orange_Telecom_Churn_Data.csv`. 
# * Remove any columns that are likely not to be used for prediction.
# * Encode data types as appropriate.



# The prediction should definitely not include phone number. And it is unlikely that area code or state would be desired, unless there is some reason to assume the model has a very specific geographic factor.



# Examine the data types. Everything looks OK other than `int_plan` and `voice_mail_plan` need to be boolean encoded.



# ## Question 2
#
# * Examine distribution of the predicted variable (`churned`).
# * Split the data into train and test sets. Decide if a stratified split should be used or not based on the distribution.
# * Examine the distribution of the predictor variable in the train and test data.



# Given the skew in the predictor variable, let's split the data with the *churned* values being stratified.



# ## Question 3
#
# * Fit random forest models with a range of tree numbers and evaluate the out-of-bag error for each of these models.
# * Plot the resulting oob errors as a function of the number of trees.
#
# *Hint:* since the only thing changing is the number of trees, the `warm_start` flag can be used so that the model just adds more trees to the existing model each time. Use the `set_params` method to update the number of trees.

# +
# Suppress warnings about too few trees from the early models

import warnings
warnings.filterwarnings("ignore", category=UserWarning)
warnings.filterwarnings("ignore", category=RuntimeWarning)
# -



# The error looks like it has stabilized around 100-150 trees.



# ## Question 4
#
# * Repeat question 3 using extra randomized trees (`ExtraTreesClassifier`). Note that the `bootstrap` parameter will have to be set to `True` for this model.
# * Compare the out-of-bag errors for the two different types of models.



# Combine the two dataframes into a single one for easier plotting.



# The random forest model performs consistently better than the extra randomized trees.



# ## Question 5
#
# * Select one of the models that performs well and calculate error metrics and a confusion matrix on the test data set. 
# * Given the distribution of the predicted class, which metric is most important? Which could be deceiving?



# Unsurprisingly, recall is rather poor for the customers who churned (True) class since they are quite small. We are doing better than random guessing, though, as the accuracy is 0.932 (vs 0.85 for random guessing).



# ## Question 6
#
# * Print or visualize the confusion matrix.
# * Plot the ROC-AUC and precision-recall curves.
# * Plot the feature importances.



# The ROC-AUC and precision-recall curves.



# The feature importances. Total daily cost is the biggest predictor of customer churn.



StarcoderdataJupyterScriptsDedupFiltered