<jupyter_start><jupyter_text>Machine Learning Laboratory (410302)BE Sem I Honors in AI/MLAcademic Year: 2021-22Lab Assignment No. 7  Name: <NAME>Roll Number: 41301Branch: Department of Computer Engineering Problem Statement:Write a program to solve a problem using Decision tree or Random forest algorithm Lab Exercise 1Use famous iris flower dataset from sklearn.datasets to predict flower species using random forest classifier.1. Measure prediction score using default n_estimators (10)2. Now fine tune your model by changing number of trees in your classifier and tell me what best score you can get using how many trees<jupyter_code>from sklearn import datasets  

iris = datasets.load_iris() 
print(iris.target_names)

print(iris.feature_names)
X, y = datasets.load_iris( return_X_y = True)
  

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.70)
from sklearn.ensemble import RandomForestClassifier
import pandas as pd
data = pd.DataFrame({'sepallength': iris.data[:, 0], 'sepalwidth': iris.data[:, 1],
                     'petallength': iris.data[:, 2], 'petalwidth': iris.data[:, 3],
                     'species': iris.target})
print(data.head())
clf = RandomForestClassifier(n_estimators = 10)  
clf.fit(X_train, y_train)
y_pred = clf.predict(X_test)
from sklearn import metrics  
print("ACCURACY OF THE MODEL: ", metrics.accuracy_score(y_test, y_pred)*100,"%")
from sklearn.ensemble import RandomForestClassifier
clf = RandomForestClassifier(n_estimators = 100)
clf.fit(X_train, y_train)
y_pred = clf.predict(X_test)
print("ACCURACY OF THE MODEL: ", metrics.accuracy_score(y_test, y_pred)*100, "%")
from sklearn.ensemble import RandomForestClassifier
clf = RandomForestClassifier(n_estimators = 1000)
clf.fit(X_train, y_train)
y_pred = clf.predict(X_test)
print("ACCURACY OF THE MODEL: ", metrics.accuracy_score(y_test, y_pred)*100, "%")
from sklearn.ensemble import RandomForestClassifier
clf = RandomForestClassifier(n_estimators = 5)
clf.fit(X_train, y_train)
y_pred = clf.predict(X_test)
print("ACCURACY OF THE MODEL: ", metrics.accuracy_score(y_test, y_pred)*100, "%")<jupyter_output>ACCURACY OF THE MODEL:  97.14285714285714 %<jupyter_text>ConclusionComparing the classifiers by their result, I found the best model with 5 trees giving an accuracy of  97.14285714285714 %. Lab Exercise 2 Build decision tree model to predict survival of titanic based on certain parameters. .csv file is available to download at Kaggle or is also with lab assignment. In this file using following columns build a model to predict if person would survive or not,1. Pclass 2. Sex 3. Age 4. FareCalculate score of your model<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
df = pd.read_csv('titanic.csv')
df.head()
df.isnull()<jupyter_output><empty_output><jupyter_text>Exploratory Data Analysis Missing DataUse seaborn to create a simple heatmap to see where we are missing data!<jupyter_code>sns.heatmap(df.isnull(), yticklabels=False, cbar=False, cmap='viridis')<jupyter_output><empty_output><jupyter_text>Roughly 20 percent of the Age data is missing. The proportion of Age missing is likely small enough for reasonable replacement with some form of imputation. Looking at the Cabin column, it looks like we are just missing too much of that data to do something useful with at a basic level.<jupyter_code>sns.set_style('whitegrid')
sns.countplot(x='Survived', data=df, palette='RdBu_r')
sns.set_style('whitegrid')
sns.countplot(x='Survived', hue='Sex', data=df, palette='RdBu_r')
sns.set_style('whitegrid')
sns.countplot(x='Survived', hue='Pclass', data=df, palette='rainbow')
sns.distplot(df['Age'].dropna(), kde=False, color='darkred', bins=30)
df['Age'].hist(bins=30, color='darkred', alpha=0.7)
sns.countplot(x='SibSp', data=df)
df['Fare'].hist(color='green', bins=40, figsize=(8,4))<jupyter_output><empty_output><jupyter_text>Data CleaningFill in missing age data instead of dropping the missing age data rows. One way to do this is by filling in the mean age of all the passengers (imputation).However we can check the average age by passenger class. For example:<jupyter_code>plt.figure(figsize=(12, 7))
sns.boxplot(x='Pclass', y='Age', data=df, palette='winter')<jupyter_output><empty_output><jupyter_text>Wealthier passengers in the higher classes tend to be older, which makes sense. We'll use these average age values to impute based on Pclass for Age.<jupyter_code>def impute_age(cols):
    Age = cols[0]
    Pclass = cols[1]
    if pd.isnull(Age):
        if Pclass == 1:
            return 37
        elif Pclass == 2:
            return 29
        else:
            return 24
    else:
        return Age
df['Age'] = df[['Age', 'Pclass']].apply(impute_age, axis=1)
sns.heatmap(df.isnull(), yticklabels=False, cbar=False, cmap='viridis')<jupyter_output><empty_output><jupyter_text>Drop the Cabin column and the row in Embarked that is NaN.<jupyter_code>df.drop('Cabin', axis=1, inplace=True)
df.head()
df.dropna(inplace=True)<jupyter_output><empty_output><jupyter_text>Converting Categorical Features Convert categorical features to dummy variables using pandas.<jupyter_code>df.info()
sex = pd.get_dummies(df['Sex'], drop_first=True)
embark = pd.get_dummies(df['Embarked'], drop_first=True)
df.drop(['Sex', 'Embarked', 'Name', 'Ticket'], axis=1, inplace=True)
df = pd.concat([df, sex, embark], axis=1)
df.head()
df.drop('PassengerId', axis=1, inplace=True)
df.head()
y = df['Survived']
X = df.iloc[:,1:]
X.head()
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
from sklearn.tree import DecisionTreeClassifier

classifier = DecisionTreeClassifier()
classifier.fit(X_train, y_train)
y_pred = classifier.predict(X_test)
from sklearn.metrics import confusion_matrix

confusion_matrix(y_test, y_pred)
from sklearn.metrics import accuracy_score

accuracy_score(y_test, y_pred)<jupyter_output><empty_output>