# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     name: python3
# ---

# + [markdown] id="oPhviFdXoo-R"
# **DATASET LINK** https://drive.google.com/drive/folders/1C8kZeA0_kwlQG2Oxd3pPqI4I34Fikogi?usp=sharing

# + [markdown] id="_5t7roxvBuxc"
# #  Data engineering with Titanic Dataset

# + [markdown] _uuid="20c66c96b3cf0abd94f514e362a80a084b9ac562" id="iVMtPCvR7G_i"
#
# ## Machine Learning Workflow
#
# 1. Define Problem
# 1. Specify Inputs & Outputs
# 1. Exploratory Data Analysis
# 1. Data Collection
# 1. Data Preprocessing
# 1. Data Cleaning
# 1. Visualization
# 1. Model Design, Training, and Offline Evaluation
# 1. Model Deployment, Online Evaluation, and Monitoring
# 1. Model Maintenance, Diagnosis, and Retraining
#

# + [markdown] _uuid="1bef8401fc30b062fd63af4dc5a8cb6a0e1e7cad" id="kPHUUBan7G_q"
# ## Các đặc trưng của tập dữ liệu
# Tàu Titanic bị đắm vào **15 tháng 4 năm 1912**, làm chết **1502 / 2224** hành khách và thủy thủ
#
# Trong tập dữ liệu về tàu Titanic này, chúng ta sẽ có **12 đặc trưng như sau**:
#
# Variable | Name	Description 
# --- | --- 
# Survived |	Survived (1) or died (0)
# Pclass |	Passenger's class
# Name	| Passenger's name
# Sex	| Passenger's sex
# Age	| Passenger's age
# SibSp	| Number of siblings/spouses aboard
# Parch	| Number of parents/children aboard
# Ticket	| Ticket number
# Fare	| Fare
# Cabin	| Cabin
# Embarked	| Port of embarkation
#
# Chi tiết: https://www.kaggle.com/c/titanic/data
#
#

# + _cell_guid="b1076dfc-b9ad-4769-8c92-a6c4dae69d19" _kg_hide-input=true _uuid="8f2839f25d086af736a60e9eeb907d3b93b6e0e5" id="9j_RmvVt7HAS"
import warnings
import json
import sys
import csv
import os


import pandas as pd

import matplotlib as mpl
import matplotlib.pyplot as plt
import seaborn as sns

import scipy
import numpy as np
np.random.seed(1612)

# + _kg_hide-input=true _uuid="cb7e4af0977f267f0055ef6c7b9d7081cbaeb889" id="FVIYJfS37HBJ"
sns.set(style='white', context='notebook', palette='deep')
warnings.filterwarnings('ignore')
mpl.style.use('ggplot')
sns.set_style('white')
# %matplotlib inline

# + id="Y4_EMOuTOA5i" colab={"base_uri": "https://localhost:8080/", "height": 54} executionInfo={"status": "ok", "timestamp": 1596029171765, "user_tz": -420, "elapsed": 2322, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="e4303e4c-d471-44ac-9b93-936f00aa0c8a"
from google.colab import drive
drive.mount('/content/drive')

# + [markdown] _uuid="04ff1a533119d589baee777c21194a951168b0c7" id="a4JKE33_7HBP"
# ## Exploratory Data Analysis(EDA)
# *   Data Collection
# *   Visualization
# *   Data Preprocessing
# *   Data Cleaning
#
# <img src="http://s9.picofile.com/file/8338476134/EDA.png">

# + [markdown] _uuid="cedecea930b278f86292367cc28d2996a235a169" id="63zD-4ME7HBQ"
# ## Data Collection
# Đây là quá trình thu thập bất kì dữ liệu nào đáng chú ý, có tiềm năng giải đáp vấn đầ của chúng ta.

# + _kg_hide-input=true _uuid="9269ae851b744856bce56840637030a16a5877e1" id="-kVIMEM27HBT" colab={"base_uri": "https://localhost:8080/", "height": 255} executionInfo={"status": "ok", "timestamp": 1596029171772, "user_tz": -420, "elapsed": 2307, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="64a1a253-16ce-478c-afe3-0579d0d08e2b"
df_train = pd.read_csv('/content/drive/My Drive/Colab Notebooks/AI Practitioner - 2020/Files/titanic/train.csv')
df_train.head()

# + id="bGPuf5KTS049" colab={"base_uri": "https://localhost:8080/", "height": 204} executionInfo={"status": "ok", "timestamp": 1596029171773, "user_tz": -420, "elapsed": 2290, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="2f6925d9-4b85-4f70-a326-b161353521ec"
df_test= pd.read_csv('/content/drive/My Drive/Colab Notebooks/AI Practitioner - 2020/Files/titanic/test.csv')
df_test.head()

# + [markdown] _uuid="58ed9c838069f54de5cf90b20a774c3e236149b3" id="dyTQR5Q47HBY"
# ><font color="red"><b>Note: </b></font>
#
# * Cách gọi khác của 1 dòng **dòng**: observation, sample, example, instance, record
# * Cách gọi khác của 1 **cột**: feature, attribute

# + [markdown] id="6XaUbZMOPpsi"
# Cách xem ngẫu nhiên một vài dòng trong tập dữ liệu

# + id="W2J_MXVAPwZ7" colab={"base_uri": "https://localhost:8080/", "height": 255} executionInfo={"status": "ok", "timestamp": 1596029172257, "user_tz": -420, "elapsed": 2753, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="770a6f99-229a-4eb4-bf6f-b128a28d984b"
df_train.sample(5) 

# + [markdown] id="QnYfm4lRP6E8"
# Xem qua thống kê sơ bộ của các cột bằng describe()

# + id="QEj3o-9cP6_S" colab={"base_uri": "https://localhost:8080/", "height": 297} executionInfo={"status": "ok", "timestamp": 1596029172258, "user_tz": -420, "elapsed": 2730, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="bb97397a-ee82-47d2-fb5e-63944d1f84f6"
df_train.describe() 

# + [markdown] _uuid="055772bd170aa8018aabd85106b76675802c33b3" id="w1FXIedg7HBp"
#
# ## Visualization
#

# + [markdown] id="p84M9RT_PC6Y"
# **Lọc ra những giá trị có thể xuất hiện trong từng cột**
#
# TÌm những cột theo giá trị categorical (PClass) thay vì continuous (price).

# + id="-ftHgWGETz8j" colab={"base_uri": "https://localhost:8080/", "height": 323} executionInfo={"status": "ok", "timestamp": 1596029172259, "user_tz": -420, "elapsed": 2706, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="db0b7dfb-4e2a-4e65-b241-8d8536211ee6"
# 1. Print out all unique values

print('Pclass unique values: ', df_train.Pclass.unique())
print('SibSp unique values: ', df_train.SibSp.unique())
print('Parch unique values: ', df_train.Parch.unique())
print('Sex unique values: ', df_train.Sex.unique())
print('Cabin unique values: ', df_train.Cabin.unique())

# + [markdown] id="Zgek1BflPVvf"
# **Đếm số lượng unique value**

# + id="z8pM-kcXPZM9" colab={"base_uri": "https://localhost:8080/", "height": 357} executionInfo={"status": "ok", "timestamp": 1596029172260, "user_tz": -420, "elapsed": 2685, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="518fec39-a9d6-4596-f953-459bbe6764d7"
print(df_train['Pclass'].value_counts())
print(df_train['SibSp'].value_counts())
print(df_train['Parch'].value_counts())
# Ko in Cabin vì quá nhiều

# + id="UDfj8u_zbH5k" colab={"base_uri": "https://localhost:8080/", "height": 68} executionInfo={"status": "ok", "timestamp": 1596029172263, "user_tz": -420, "elapsed": 2612, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="9b2ac1dd-8396-4abe-b9ef-3548d2d09eea"
print(df_train['Sex'].value_counts())

# + [markdown] id="5CkEbmyq2aF9"
# ### Count Plot

# + [markdown] id="lCAWG2hSz-07"
# **Survived vs Sex**

# + id="e6VaVNI-TTWs" colab={"base_uri": "https://localhost:8080/", "height": 373} executionInfo={"status": "ok", "timestamp": 1596029172264, "user_tz": -420, "elapsed": 2558, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="8054f2d2-cdfe-40ae-ff00-b1a9ddedf2fa"
plt.figure(figsize=(15,5))

plt.subplot(1,3,1)
sns.countplot(df_train.Sex)
plt.title('Count plot for Sex')

plt.subplot(1,3,2)
sns.countplot(df_train.Survived) # truyền DataFrame.Column Name
plt.title('Number of passengers Survived')

plt.subplot(1,3,3)
sns.countplot(x="Survived", hue="Sex", data=df_train)
plt.title('Number of passengers Survived by Sex')

# + [markdown] id="nnMQBPtC0HmK"
# **Survived vs Pclass**

# + id="L7FH1vawzy4O" colab={"base_uri": "https://localhost:8080/", "height": 536} executionInfo={"status": "ok", "timestamp": 1596029172595, "user_tz": -420, "elapsed": 2810, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="3c58dc79-42f4-4222-839d-5e2bcce0b3a4"
plt.figure(figsize=(15,8))
plt.subplot(1,3,1)
sns.countplot(df_train.Pclass)
plt.title('Count plot for Pclass')

plt.subplot(1,3,2)
sns.countplot(df_train.Survived) # truyền DataFrame.Column Name
plt.title('Number of passengers Survived')

plt.subplot(1,3,3)
sns.countplot(x='Survived', hue='Pclass', data=df_train)
plt.title('Number of passengers Survived by Pclass')

# + [markdown] _uuid="b0014a7a52e714996bc443981c853095926d20e5" id="yn1lgNPq7HBq"
# ### Scatter Plot
#
# [Scatter plot](https://en.wikipedia.org/wiki/Scatter_plot) Xác định quan hệ giữa 2 đặc trưng.
#
#

# + [markdown] id="uzxhoFx3mE3a"
# **Survived vs Pclass vs Fare vs Age**

# + _kg_hide-input=true _uuid="af099546eed64ebc796403d4139cb4c977c27b03" id="8PopSalL7HB0" colab={"base_uri": "https://localhost:8080/", "height": 225} executionInfo={"status": "ok", "timestamp": 1596029173758, "user_tz": -420, "elapsed": 3940, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="41fe03e5-f2dd-4074-f504-c0df45b793cf"
#https://seaborn.pydata.org/generated/seaborn.FacetGrid.html

g = sns.FacetGrid(df_train, hue="Survived", col="Pclass", margin_titles=True,
                  palette={1:"seagreen", 0:"gray"}) 
g=g.map(plt.scatter, "Fare", "Age", edgecolor="w").add_legend();

# + [markdown] id="GTwlWR5VmUb-"
# **Survived vs Pclass vs SibSp vs Parch**

# + id="CqXN4eoW31ZH" colab={"base_uri": "https://localhost:8080/", "height": 225} executionInfo={"status": "ok", "timestamp": 1596029174966, "user_tz": -420, "elapsed": 5115, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="97488101-5338-4170-cb88-5e060b5975e6"
f = sns.FacetGrid(df_train, hue='Survived', col='Pclass', margin_titles=True,
                  palette={1:'blue', 0:'red'})
f = f.map(plt.scatter, 'SibSp', 'Parch', edgecolor='w').add_legend();

# + [markdown] _uuid="d1c7b62b5f8cba427bca13049256365141655372" id="st7R7FYW7HCJ"
#
# ### Box
# Box là phương pháp mô ta dữ liệu số thông qua những phần tư của chúng.
# Chi tiết: 
# - https://www.khanacademy.org/math/statistics-probability/summarizing-quantitative-data/box-whisker-plots/a/box-plot-review
# - https://www.khanacademy.org/math/statistics-probability/summarizing-quantitative-data/box-whisker-plots/a/identifying-outliers-iqr-rule
# <img src="https://miro.medium.com/max/13500/1*2c21SkzJMf3frPXPAR_gZA.png">
#

# + [markdown] id="Rcv8QRneI1pf"
# **Example**
#
# ![boxplot](https://i.imgur.com/Mcw6vXv.png)

# + _kg_hide-input=true _uuid="7b193e4aa7e6fb337d3f65c334849094addd097a" id="mbP7WkE17HCL" colab={"base_uri": "https://localhost:8080/", "height": 285} executionInfo={"status": "ok", "timestamp": 1596029174967, "user_tz": -420, "elapsed": 5088, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="dd28bc5e-7e81-49f5-fa70-e48f3734d608"
ax= sns.boxplot(x="Pclass", y="Age", data=df_train)
plt.show()
# Nhiễu thì xoá dòng or thay bằng min or max của data đó

# + [markdown] _uuid="743a92c3c2fff1a1f99845518247f7971ad18b7c" id="pzLreWJY7HCQ"
#
# ### Histogram

# + _kg_hide-input=true _uuid="5da0520ed3e738ee8814b2d91843ed4acec2b6e6" id="GdR2nt4c7HCR" colab={"base_uri": "https://localhost:8080/", "height": 645} executionInfo={"status": "ok", "timestamp": 1596029177282, "user_tz": -420, "elapsed": 7377, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="9cfe470d-d755-4ab7-873c-c168b5a63145"
# histograms
df_train.hist(figsize=(15,10),bins=40)
plt.figure()

# + [markdown] id="NiQ7RSPQNOrF"
# ### Correlation Heatmap

# + id="4Pgg-RdCNQxO" colab={"base_uri": "https://localhost:8080/", "height": 559} executionInfo={"status": "ok", "timestamp": 1596029178148, "user_tz": -420, "elapsed": 8168, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="d833aea9-c018-49e7-f231-0013b3bb384c"
# 1. Find correlation matrix using df.corr()
# 2. Plot heatmap
plt.figure(figsize=(8,8))
sns.heatmap(df_train.corr(), annot=True) # corr() only used for DataFrame
plt.title('Corelation Matrix');

# + [markdown] _uuid="91dda1f631cf4ed362162501aaaac6d19cfd6cc7" id="DGYJCxIk7HFY"
# ## Data Preprocessing
# **Data preprocessing** là kĩ thuật chuẩn hóa, làm sạch dữ liệu trước khi đưa vào thuật toán.
# Một vài bước tiêu biểu của Data preprocessing:
# * Xử lý dữ liệu không cân bằng
# * Xử lý dữ liệu bị thiếu Nan
# * Xử lý nhiễu (https://towardsdatascience.com/ways-to-detect-and-remove-the-outliers-404d16608dba)
# * Chuẩn hóa dữ liệu (https://medium.com/@urvashilluniya/why-data-normalization-is-necessary-for-machine-learning-models-681b65a05029) Scaling
# * Chọn lọc đặc trưng (https://medium.com/analytics-vidhya/feature-selection-using-scikit-learn-5b4362e0c19b)

# + [markdown] _uuid="581b90e6a869c3793472c7edd59091d6d6342fb2" id="Jy-9EZ6y7HFZ"
#  ## Types of features
# Features:
# * số 
# * theo từng mục (category)
# * ngày tháng (datetime)
#
# ### Hãy xác định dạng đặc trưng trong tập dữ liệu:
#

# + id="wo0-9NzAMezn" colab={"base_uri": "https://localhost:8080/", "height": 340} executionInfo={"status": "ok", "timestamp": 1596029178148, "user_tz": -420, "elapsed": 8102, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="ab3fbf7b-63c1-4815-f29a-9f73fa5b5048"
# 1. check data type of each columns
df_train.info()

# + [markdown] _uuid="73ab30f86273b590a51fc363d9bf78c2709558fa" id="-y4rapQG7HFa"
# ## Data exploration

# + _kg_hide-input=true _uuid="4b45251be7be77333051fe738639104ae1005fa5" id="BIfxVarS7HFd" colab={"base_uri": "https://localhost:8080/", "height": 34} executionInfo={"status": "ok", "timestamp": 1596029178149, "user_tz": -420, "elapsed": 8069, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="24f6eba2-d6da-4a52-977e-bd1f61244b1e"
# shape
print(df_train.shape)

# + [markdown] _uuid="a6315bf510cecb907b2d23aad25faf6ccad32ac4" id="B43NqHRJ7HF1"
# >  <font color="red"><b>Note:</b></font>
# how many NA elements in every column
#

# + id="mjlAG0SfB4Ke" colab={"base_uri": "https://localhost:8080/", "height": 419} executionInfo={"status": "ok", "timestamp": 1596029178150, "user_tz": -420, "elapsed": 8043, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="a2c18db5-562b-4473-b2c2-a4e54787d09f"
df_train.isnull()

# + id="QmpnEPYlqTRU" colab={"base_uri": "https://localhost:8080/", "height": 238} executionInfo={"status": "ok", "timestamp": 1596029178151, "user_tz": -420, "elapsed": 8008, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="6364fc37-82eb-47d8-bcdc-481aec03ef9d"
# Check how many Null elements per column
# Do hạng thấp ko đc đánh số Cabin nên Nul nhiều. Chỉ hạng cao có số Cabin
# Cabin vị trí thì có thể quan trọng. Nhưng xử lý nó dài nên bài này xoá Cabin
df_train.isnull().sum()

# + id="GJaC8zNrOWTF" colab={"base_uri": "https://localhost:8080/", "height": 221} executionInfo={"status": "ok", "timestamp": 1596029178152, "user_tz": -420, "elapsed": 7981, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="4f073113-f383-40c0-e35d-4d5a62bf6f92"
# 1. Check null on df_train and df_test
df_test.isnull()
df_test.isnull().sum()

# + [markdown] id="tZ2xcb2QQtu5"
# **Xóa cột**

# + id="633VeukQQvvN"
# 1. Drop 4 columns Embarked, PassengerId, Cabin, Ticket
df_train = df_train.drop(columns = ['Embarked', 'PassengerId', 'Cabin', 'Ticket'])
df_test = df_test.drop(columns = ['Embarked', 'PassengerId', 'Cabin', 'Ticket'])

# + id="WGaDvHx9rWfW" colab={"base_uri": "https://localhost:8080/", "height": 204} executionInfo={"status": "ok", "timestamp": 1596029178153, "user_tz": -420, "elapsed": 7943, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="02270f01-1326-460f-ebc8-f48f42aff05e"
df_train.head(5)

# + id="H6xnlt9Ircf_" colab={"base_uri": "https://localhost:8080/", "height": 204} executionInfo={"status": "ok", "timestamp": 1596029178154, "user_tz": -420, "elapsed": 7925, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="afebc064-5501-4c51-e43b-1992db470afd"
df_test.head(5)

# + [markdown] _uuid="414a457068cc1e0390f7be45ad85cab0cab3cb72" id="hiUbTh1q7HGF"
# **Xóa những dòng có giá trị NA**

# + _kg_hide-input=true _uuid="e8e124ca20643ad307d9bfdc34328d548c6ddcbc" id="83GuMdZ-7HGH"
# remove rows that have NA's # do chỉ có 1 dòng # drop những dòng bị null
# df_test['Fare'] = df_test['Fare'].dropna() # incorrect
df_test.dropna(subset=['Fare'],inplace=True) # !!! phải là subset rồi truyền vào column name

# + id="qdde-sRxvtyz"
# Remove all rows having Nan
# df_test.dropna(inplace=True)

# + [markdown] id="EmpYbhRdNIHB"
# **Replace giá trị NA bằng giá trị khác**
#
# Replace bằng trung vị or trung bình của cái Class đó. Dựa theo heatmap để đánh giá.

# + id="DlkJaRCnNHz6"
# Left: return location of Nulls in Age column. Right: groupby theo nhóm Pclass, transform Nulls trong Age bằng median Age theo / của Pclass (do Pclass có corr lớn nhất vs Age)
df_train.loc[df_train.Age.isnull(), 'Age'] = df_train.groupby("Pclass").Age.transform('median')

df_test.loc[df_test.Age.isnull(), 'Age'] = df_test.groupby("Pclass").Age.transform('median')

# + id="GGPE-fZReMgg"


# + id="NpI8g4JbuDZB" colab={"base_uri": "https://localhost:8080/", "height": 419} executionInfo={"status": "ok", "timestamp": 1596029178536, "user_tz": -420, "elapsed": 798, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="7844a26e-539e-4292-9bee-06226319c1be"
df_train.isnull()

# + id="mcZwUuWwuGFW" colab={"base_uri": "https://localhost:8080/", "height": 170} executionInfo={"status": "ok", "timestamp": 1596029179285, "user_tz": -420, "elapsed": 850, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="85b44e30-2a53-46cb-a636-5b6e315b416e"
df_train.isnull().sum()

# + id="7GvKJP5NuJr_" colab={"base_uri": "https://localhost:8080/", "height": 419} executionInfo={"status": "ok", "timestamp": 1596029179605, "user_tz": -420, "elapsed": 470, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="169817a5-8709-43d9-e87b-931cf26d2f7e"
df_test.isnull()

# + id="Ej9SGzLEuJvK" colab={"base_uri": "https://localhost:8080/", "height": 153} executionInfo={"status": "ok", "timestamp": 1596029180350, "user_tz": -420, "elapsed": 619, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="91dfd457-8f28-4295-cd0d-37d85e5a6572"
df_test.isnull().sum()

# + [markdown] _uuid="166e019ad0c531d7b82447eadbd61fb09a062047" id="9UHRnUf77HIQ"
# ## Feature transformations
# Chúng ta sẽ chuyển đổi dữ liệu sang một dạng mới với kì vọng sẽ đem lại giá trị tốt hơn cho việc phân tích, đưa ra quyết định
# Ở đây chúng ta sẽ biến đổi:
# 1. Name
# 1. Age
# 1. SibSp & Parch
#
#

# + [markdown] id="vgHMEfy-TwbL"
# **Transform Name**

# + id="ZClkqLlEWBic" colab={"base_uri": "https://localhost:8080/", "height": 204} executionInfo={"status": "ok", "timestamp": 1596029308565, "user_tz": -420, "elapsed": 877, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="8bd8334c-a7cc-445d-f00c-45629a7e37e7"
df_train['Name'].head(10)

# + id="YmJYODRxUAtZ"
df_train['Name'] = df_train.Name.str.extract(' ([A-Za-z]+)\.', expand = False)
df_test['Name'] = df_test.Name.str.extract(' ([A-Za-z]+)\.', expand = False)

# + id="j2nlIWxaULin" colab={"base_uri": "https://localhost:8080/", "height": 306} executionInfo={"status": "ok", "timestamp": 1596029391848, "user_tz": -420, "elapsed": 713, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="d4d11e7a-9595-46b0-e4f8-f6ee9a077e88"
df_train['Name'].unique().tolist()

# + id="aF93M8CXTxI9"
df_train.rename(columns={'Name' : 'Title'}, inplace=True)
df_train['Title'] = df_train['Title'].replace(['Rev', 'Dr', 'Col', 'Ms', 'Mlle', 'Major', 'Countess', 
                                       'Capt', 'Dona', 'Jonkheer', 'Lady', 'Sir', 'Mme', 'Don'], 'Other')
# Còn lại là Other                                      
df_test.rename(columns={'Name' : 'Title'}, inplace=True)
df_test['Title'] = df_test['Title'].replace(['Rev', 'Dr', 'Col', 'Ms', 'Mlle', 'Major', 'Countess', 
                                       'Capt', 'Dona', 'Jonkheer', 'Lady', 'Sir', 'Mme', 'Don'], 'Other')

# + id="A7UL8ANWUbgC" colab={"base_uri": "https://localhost:8080/", "height": 119} executionInfo={"status": "ok", "timestamp": 1596029557001, "user_tz": -420, "elapsed": 754, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="c3163ec0-e3e7-45de-8235-f0beebe1f710"
df_train.Title.value_counts()

# + [markdown] id="GxQrn6DfU5q3"
# **Transform Age**

# + id="EB6OOZFUU8zi"
# Gom nhóm 
bins = (0, 5, 12, 18, 25, 35, 60, 120)
group_names = [ 'Baby', 'Child', 'Teenager', 'Student', 'Young Adult', 'Adult', 'Senior']
categories = pd.cut(df_train.Age, bins, labels=group_names)
df_train.Age = categories
df_test.Age = categories

# + id="r2cZTlnLVLqT" colab={"base_uri": "https://localhost:8080/", "height": 153} executionInfo={"status": "ok", "timestamp": 1596029766925, "user_tz": -420, "elapsed": 801, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="1e043ae2-8bae-45ee-ab50-4197291c4981"
df_train.Age.value_counts()

# + [markdown] id="gPbwpTRAVS3s"
# **Transform SibSp & Parch**

# + id="odyA9LfRVeny"
# # +1 là người đc count
df_train['FamilySize'] = df_train['SibSp'] + df_train['Parch'] + 1
df_test['FamilySize'] = df_test['SibSp'] + df_test['Parch'] + 1

# + id="WeqihBypVrji"
df_train = df_train.drop(columns=['SibSp','Parch'])
df_test = df_test.drop(columns=['SibSp','Parch'])

# + id="1OJxkQiIW6eN" colab={"base_uri": "https://localhost:8080/", "height": 221} executionInfo={"status": "ok", "timestamp": 1596029974816, "user_tz": -420, "elapsed": 822, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="61c722ad-c8af-46db-f4fb-c979c6846cd9"
df_train.FamilySize

# + [markdown] _uuid="460e83e7cdb3db098cff0a82432a729619aac918" id="eFjB0daq7HIW"
# ## Feature Encoding
#
# Apply One Hot Encoding for all categorical columns

# + id="NzkFJU_bZOhd" colab={"base_uri": "https://localhost:8080/", "height": 204} executionInfo={"status": "ok", "timestamp": 1596029980221, "user_tz": -420, "elapsed": 929, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="fed1cc28-223b-406b-91bb-df67f2772933"
df_train.head()

# + [markdown] id="tx-kzcG2ZS8A"
# **Encode Sex column**

# + _kg_hide-input=true _uuid="b06cb1191a0f52a904c52a918d1f999536e79bda" id="TDVyCqlz7HIi"
# One hot encoding for 2 values
df_train['Sex'] = df_train['Sex'].map({'male':1,'female':0})
df_test['Sex'] = df_test['Sex'].map({'male':1,'female':0})

# + id="Kdh4oE16aofe" colab={"base_uri": "https://localhost:8080/", "height": 204} executionInfo={"status": "ok", "timestamp": 1596030115287, "user_tz": -420, "elapsed": 755, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="6b15c70d-7757-4aec-89fa-d3fcaeba6c3d"
df_train.head()

# + [markdown] id="bFI5hMaqZ3wl"
# **Encode other columns**

# + id="-JwlHXAjaC5P" colab={"base_uri": "https://localhost:8080/", "height": 241} executionInfo={"status": "ok", "timestamp": 1596030124256, "user_tz": -420, "elapsed": 949, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="6d71214a-e925-4e57-977b-99297492862f"
# One hot encoding for more than 2 values: use get_dummies
df_train = pd.get_dummies(df_train)
df_test = pd.get_dummies(df_test)
df_train.head()

# + [markdown] id="PZ_qcnyya_qu"
# ## Prepare dataset

# + [markdown] id="6mwzxe5S8qt6"
# **Lấy ra label y**

# + id="_ksHLGd68sX0"
# 1. get y from column Suvived
# 2. drop column Survived
y = df_train.Survived
df_train = df_train.drop(columns=['Survived'])

# + id="JXhysNKN0L_M" colab={"base_uri": "https://localhost:8080/", "height": 241} executionInfo={"status": "ok", "timestamp": 1596030253012, "user_tz": -420, "elapsed": 673, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="e5d8008f-92bf-4c3d-efb3-1b706f1bd011"
df_train.head(5)

# + [markdown] _uuid="7b6aa3e1e00043b658ba2111f4296c598a3ccfd2" id="hVaThlQI7HIr"
# **Train / Validation Split**

# + _uuid="0921e1c0ee59db81f42300294bc37428659a2ce9" id="M4CTrsomIflM" colab={"base_uri": "https://localhost:8080/", "height": 85} executionInfo={"status": "ok", "timestamp": 1596030436069, "user_tz": -420, "elapsed": 783, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="6d3984e3-be28-4ebd-db5f-3143b8d304b1"
# test_size=0.3 and stratify=y
from sklearn.model_selection import train_test_split
X_train, X_val, y_train, y_val = train_test_split(df_train, y, test_size=0.3, stratify=y, shuffle=True, random_state=1612)   
print('Shape of X train', X_train.shape)
print('Shape of y train', y_train.shape)
print('Shape of X val', X_val.shape)
print('Shape of y val', y_val.shape)

# + id="cv4pafaF6yix"
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, shuffle=True, random_state=1612)   
print('Shape of X train', X_train.shape)
print('Shape of y train', y_train.shape)
print('Shape of X_test', X_test.shape)
print('Shape of y_test', y_test.shape)


# + [markdown] id="6xA6Y7NnbBrH"
# **Feature Scaling (MinMaxScaler)**

# + id="LnpeuG31E98C"
from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()
# Muốn dùng MinMaxScaler thì phải lấy cột đó ra reshape, transform lại
fares_train = np.array(X_train['Fare']).reshape(-1, 1)
fares_val = np.array(X_val['Fare']).reshape(-1, 1)
fares_test = np.array(df_test['Fare']).reshape(-1, 1)

X_train['Fare'] = scaler.fit_transform(fares_train)
X_val['Fare']= scaler.transform(fares_val)
df_test['Fare'] = scaler.transform(fares_test)

# + [markdown] id="Aw2I0eWT2OLF"
# ## 2D Visualization

# + [markdown] id="QO1a6sMR2TBO"
# ### PCA giảm chiều dữ liệu cách 1
#
# Hơi cùi bắp

# + id="_4MNKcCc2asZ"
# 1. import PCA
# 2. Initializa pca with 2 components
# 3. Apply pca on X_train
# 3. Name the new variable df_pca
from sklearn.decomposition import PCA
import numpy as np
import matplotlib.pyplot as plt

pca = PCA(n_components = 2) # 2D nên number of components = 2
df_pca = pca.fit_transform(X_train)

# + id="AcLL56SI20Hh" colab={"base_uri": "https://localhost:8080/", "height": 502} executionInfo={"status": "ok", "timestamp": 1596030900535, "user_tz": -420, "elapsed": 1668, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="c6ec8c45-447b-46d6-baf0-5d4b2e7d92a8"
plt.figure(figsize =(8, 8))
sns.scatterplot(df_pca[:,0], df_pca[:,1], hue=y_train, legend='full') # seaborn hiện đại hơn matplotlib

# + id="K9fMhMV63Fql" colab={"base_uri": "https://localhost:8080/", "height": 34} executionInfo={"status": "ok", "timestamp": 1596031003983, "user_tz": -420, "elapsed": 1322, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="0ff14533-6dbd-4e1a-9268-9ebd3d452234"
print(pca.explained_variance_ratio_)
# Lượng thông tin đc giữ lại chỉ còn 0.56 + 0.16 = 72%

# + [markdown] id="c4_K34QX2U_q"
# ### T-SNE giảm xuống 2 chiều cách 2
#
# Visualize vector nhúng Z 

# + id="V3OWZBwP3DY_" colab={"base_uri": "https://localhost:8080/", "height": 502} executionInfo={"status": "ok", "timestamp": 1596031098152, "user_tz": -420, "elapsed": 6365, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="4ec3f6ad-6204-475f-a780-aaa3d874ef3e"
from sklearn.manifold import TSNE

tsne = TSNE()
df_tsne = tsne.fit_transform(X_train)

plt.figure(figsize =(8, 8))
sns.scatterplot(df_tsne[:,0], df_tsne[:,1], hue=y_train, legend='full')

# + [markdown] _uuid="3d240a0df576395a9c08df161df732bba89fda1e" id="XaiI_8LS7HKU"
# ## Ensemble Model
#
# Tổng hợp nhiều models với nhau

# + [markdown] id="iaOoz_-Yc5ft"
# ### Logistic Regression

# + _kg_hide-input=true _uuid="70478a52a315460748cce36779e25dbb4d109258" id="QAIcGOkD7HKV" colab={"base_uri": "https://localhost:8080/", "height": 102} executionInfo={"status": "ok", "timestamp": 1596031537413, "user_tz": -420, "elapsed": 1447, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="6ca8e5b2-f864-4905-e660-29baf5ed7850"
# 1. Define LogisticRegression model
# 2. Fit on Train
from sklearn.linear_model import LogisticRegression
logreg = LogisticRegression()
logreg.fit(X_train, y_train) # fit trên cả X, y train

# + _kg_hide-input=true _uuid="a0fb319adb2974342f5825b77cedfb7969d06fda" id="GQesSMkk7HKZ" colab={"base_uri": "https://localhost:8080/", "height": 51} executionInfo={"status": "ok", "timestamp": 1596031684620, "user_tz": -420, "elapsed": 1195, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="e7dac109-2b85-4344-f94a-22a49d5fce8a"
# Classification thì score có default accuracy
# 1. Use score() function on 2 Set
print('Accuracy on Train Set: ', logreg.score(X_train, y_train))
print('Accuracy on Validation Set: ', logreg.score(X_val, y_val))
# Ko cao lắm. Chỉ đúng trên người sống, còn ng chết thì ko đúng. vẽ confusion matrix ra coi accuracy trêng gì

# + [markdown] id="z14KfBBFd6m_"
# ### Support Vector Machine

# + id="fE550aYMAB-4"
C_values = [0.01, 0.1, 1] # 0.01 tới 10
gamma_values = [0.01, 0.1, 1]
kernel_values = ['linear', 'poly', 'rbf']

param_grid = {
    'kernel': kernel_values,
    'C': C_values,
    'gamma': gamma_values
}

# + id="TXnLoc3BeK96"
# 1. import SVC, GridSearchCV
# 3. khởi tạo GridSearchCV với số k-fold (cv) = 3
# 4. fit trên Train Set 
# 5. lấy ra best model
# 6. 
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV

model = SVC(random_state=1612)
grid = GridSearchCV(model, param_grid, cv=2)
grid.fit(X_train, y_train)

svc = grid.best_estimator_

# + id="vum6yBpwf-y_"
# 1. Use score() function on 2 Sets
print('Accuracy on Train Set: ', model.score(X_train, y_train))
print('Accuracy on Validation Set: ', model.score(X_val, y_val))

# + [markdown] id="UmPSQ27Teuyg"
# ### Decision Tree

# + id="f-dkPG-oeuIy" colab={"base_uri": "https://localhost:8080/", "height": 119} executionInfo={"status": "ok", "timestamp": 1596032341984, "user_tz": -420, "elapsed": 868, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="7c6848a4-60f2-4103-aa3f-4a08f710773a"
params = {
    'criterion': ['entropy','gini'],
    'max_depth': [3,5,7],
    'min_samples_split': np.linspace(0.1, 1.0, 10), 
    'max_features':  ['auto', 'log2']
}

# 1. import Decision Tree
# 3. khởi tạo GridSearchCV với số k-fold (cv) = 2
# 4. fit trên Train Set 
# 5. lấy ra best model
from sklearn.tree import DecisionTreeClassifier

# grid = GridSearchCV(DecisionTreeClassifier(random_state=1612), params, cv=2, n_jobs=1)
# grid.fit(X_train, y_train)

decision_tree = DecisionTreeClassifier(random_state=1612)
decision_tree.fit(X_train,y_train)

# + id="y3nWXzG5gO1J" colab={"base_uri": "https://localhost:8080/", "height": 51} executionInfo={"status": "ok", "timestamp": 1596032361530, "user_tz": -420, "elapsed": 724, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="4650f3e1-30a2-4fb5-c254-9e4ea6c2b3b2"
# 1. Use score() function on 2 Sets
print('Accuracy on Train Set: ', decision_tree.score(X_train, y_train))
print('Accuracy on Validation Set: ', decision_tree.score(X_val, y_val))

# + [markdown] id="s7uObNWXgWqC"
# ### Random Forest

# + id="6PR7wqEGgYEV"
 param_grid_random={'criterion': ['gini', 'entropy'],
            'max_depth': [3, 5, 7],
            'max_features': ['auto', 'log2'],
            'n_estimators': [100, 300, 500]})

# 1. Import RandomForestClassifier
# 2. khởi tạo GridSearchCV với số k-fold (cv) = 2
# 3. fit trên Train Set 
# 4. lấy ra best model



# + id="w50f2SAPgrta"
# 1. Use score() function on 2 Sets


# + [markdown] id="3_BtWp2OhTm1"
# ### Voting Classifier

# + [markdown] id="8JjuA6Ugh6Q4"
# voting = {'hard', 'soft'}
#
# If 'hard', uses predicted class labels for majority rule voting. Else if 'soft', predicts the class label based on the argmax of the sums of the predicted probabilities, which is recommended for an ensemble of well-calibrated classifiers.
#
# Model tune kĩ bằng prob rồi thì dùng soft. Bỏ qua SVM do SVM có output ko phải prob.

# + id="FFbtTTSihVEj" colab={"base_uri": "https://localhost:8080/", "height": 510} executionInfo={"status": "ok", "timestamp": 1596032510457, "user_tz": -420, "elapsed": 2074, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="617abf59-019f-4c8f-bbbf-17cdfc6db788"
from sklearn.ensemble import VotingClassifier

ensemble = VotingClassifier(estimators=[
                              ('Logistic Regression', logreg), 
                              ('SVM', svc), 
                              ('Decision Tree', decision_tree)],
                            voting='hard',
                            n_jobs=-1) # hard: label nào nhiều thì đc lấy ra # ('Random Forest', random_forest)



ensemble.fit(X_train, y_train)

# + id="UcYT_7tfifSO" colab={"base_uri": "https://localhost:8080/", "height": 51} executionInfo={"status": "ok", "timestamp": 1596032672546, "user_tz": -420, "elapsed": 711, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="c029b2c5-35c3-4bb4-cf52-986ec24e1c89"
# 1. Use score() function on 2 Sets
print('Accuracy on Train Set: ', ensemble.score(X_train, y_train))
print('Accuracy on Validation Set: ', ensemble.score(X_val, y_val))

# + [markdown] id="zEHaCLywi51k"
# **Confusion Matrix**

# + id="K15ddp5FioH4" colab={"base_uri": "https://localhost:8080/", "height": 628} executionInfo={"status": "ok", "timestamp": 1596032904158, "user_tz": -420, "elapsed": 927, "user": {"displayName": "<NAME>", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14GhDuFEkZpoMboQyDt6AbRtjoe6e1C2ZFpBJNeRrd_4=s64", "userId": "10380833966901257703"}} outputId="a81b9a21-1831-4558-aa26-22605785b090"
# 1. Import confusion matrix from sklearn
# 2. Use confusion_matrix to draw a heatmap
# 3. if your heatmap show 8e+2 numbers. Insise heatmap() function, put a parameter fmt='.1f'  
from sklearn.metrics import confusion_matrix
y_pred = ensemble.predict(X_val)
cm = confusion_matrix(y_val, y_pred)
plt.figure(figsize=(10,10))
sns.heatmap(cm, annot=True, fmt='.1f')
plt.title('Confusion Matrix')

# + id="6p7yB1l--Weg"


StarcoderdataJupyterScriptsDedupFiltered