<jupyter_start><jupyter_text>Phase 1 Progress Preprocess of datasetThe dataset of this assignments is from the file 'data/fake_or_real_news.csv', which contains thousands of news within. The very first opeartion I did is to trans the title_vectors from string into float list, because the although the string is made up of figures, it can not be processed before transfer into an parameter that can be use. And after that, I use 1 and 0 instead of Fake and Real in lebal column. And then I call the describe function... not for much use, I think.<jupyter_code>import pandas as pd
df = pd.read_csv('data/fake_or_real_news.csv')
df.head()
list_str = lambda i: i.strip('[').strip(']').split()
list_float = lambda i: [float(j) for j in i]
str_to_list_float = lambda i: list_float(list_str(i))
df.title_vectors = df.title_vectors.apply(str_to_list_float)
df.head()
label_trans = lambda i: 0 if i == 'FAKE' else 1
df.label = df.label.apply(label_trans)
df.head()
import numpy as np
df.describe(include=[np.number])
x = np.array(df.title_vectors.apply(lambda x: pd.Series(x)))
y = np.array(df['label'])<jupyter_output><empty_output><jupyter_text>Split the datasetJust like the operation we did in lecture 1, I also use the 0.33 as the ratio of training set and testing set. I choose 2 as the random seed of spliting operation.<jupyter_code>from sklearn.model_selection import train_test_split
seed = 2
test_size = 0.33
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=test_size, random_state=seed)
print (x_train.shape, x_test.shape, y_train.shape, y_test.shape)<jupyter_output>(4244, 300) (2091, 300) (4244,) (2091,)<jupyter_text>Logistic regressionThe first method I use is logistic regression. It is pretty fast than the other method and give us an 0.79 accuracy by calculating that modal out. Consider the time spend it turns out the logistic regression model is pretty acceptable.<jupyter_code>import time
from sklearn.linear_model import LogisticRegression
LR_model = LogisticRegression()
start1 = time.time()
LR_model = LR_model.fit(x_train, y_train)
end1 = time.time()
LR_model.score(x_test, y_test)
LR_time = end1-start1
LR_time
from sklearn.metrics import classification_report
target_names = ['FAKE', 'REAL']
y_pred = LR_model.predict(x_test)
print(classification_report(y_test, y_pred, target_names=target_names))
from sklearn.metrics import * 
LR_result = [accuracy_score(y_test, y_pred), precision_score(y_test, y_pred), recall_score(y_test, y_pred),
             f1_score(y_test, y_pred), LR_time]
LR_result<jupyter_output><empty_output><jupyter_text>Random forestThere are so many parameter that we can change in the method of random forest, and I find that the time of model spend is highly correlate to the number of estimator we used, and that make perfect sense to the concept of random forest. The accuracy is a little bit lower than the accuracy of logistic regression and the time is also over 1 sec.<jupyter_code>from sklearn.ensemble import RandomForestClassifier
RF_model = RandomForestClassifier(n_estimators = 20, max_features=20, random_state=seed)
start2 = time.time()
RF_model = RF_model.fit(x_train, y_train)
end2=time.time()
RF_model.score(x_test, y_test)
RF_time = end2 - start2
RF_time
y_pred = RF_model.predict(x_test)
print(classification_report(y_test, y_pred, target_names=target_names))
RF_result = [accuracy_score(y_test, y_pred), precision_score(y_test, y_pred), recall_score(y_test, y_pred),
             f1_score(y_test, y_pred), RF_time]
RF_result<jupyter_output><empty_output><jupyter_text>XGboostLast but not least, I use the XGboost model to evaluate the perform of these data. XGboost require a lots of approach attempts, and that use a lot of time so the time spend of XGboost is much higher than the other two models, however, in the other site, that make sure the accuracy of result.<jupyter_code>from xgboost import XGBClassifier
start3 = time.time()
XG_model = XGBClassifier(max_depth=7, learning_rate=0.2, 
                        n_estimators=20, silent=True, 
                        objective='binary:logistic', nthread=-1, 
                        gamma=0, min_child_weight=1, max_delta_step=0, 
                        subsample=1, colsample_bytree=1, 
                        colsample_bylevel=1, reg_alpha=0, 
                        reg_lambda=1, scale_pos_weight=1, 
                        base_score=0.5, seed=0, missing=None)
XG_model = XG_model.fit(x_train, y_train)
end3 = time.time()
XG_model.score(x_test, y_test)
XG_time = end3 - start3
XG_time
y_pred = XG_model.predict(x_test)
print(classification_report(y_test, y_pred, target_names=target_names))
XG_result = [accuracy_score(y_test, y_pred), precision_score(y_test, y_pred), recall_score(y_test, y_pred),
             f1_score(y_test, y_pred), XG_time]
XG_result<jupyter_output><empty_output><jupyter_text>PlottingI use three list to store the data of the calculation result, and use pyplot to present three different models. Surprisingly it didn't distribute that much comparing in a large scale, maybe that is because the dataset is not choosing wisely or the data is really connect to each other that does not have much difference. Also, I plot a seperate graph to present time spent in each model.<jupyter_code>Y1 = np.array(LR_result[:4])
Y2 = np.array(RF_result[:4])
Y3 = np.array(XG_result[:4])
from matplotlib import pyplot as plt
plt.figure(figsize=(11,6))
n = 4
X = np.arange(n)+1
plt.xticks([1.3,2.3,3.3,4.3],[r'$Accuracy$', r'$Precision$', r'$Recall$',r'$F1-score$'])
plt.bar(X,Y1,width = 0.3,facecolor = 'lightskyblue',edgecolor = 'white',label='LR')
plt.bar(X+0.3,Y2,width = 0.3,facecolor = 'yellowgreen',edgecolor = 'white',label='RF')
plt.bar(X+0.6, Y3, width = 0.3,facecolor = 'coral',edgecolor = 'white',label='XG')
for x,y in zip(X,Y1):
    plt.text(x, y, '%.2f' % y, ha='center', va= 'bottom')
for x,y in zip(X,Y2):
    plt.text(x+0.3, y, '%.2f' % y, ha='center', va= 'bottom')
for x,y in zip(X,Y3):
    plt.text(x+0.6, y, '%.2f' % y, ha='center', va= 'bottom')
plt.ylabel('Percentage')
plt.ylim(0,+1)
plt.legend()
plt.show()
data = [LR_result[4], RF_result[4],XG_result[4]]
plt.barh(range(len(data)), data)
plt.yticks([0,1,2],[r'$LR$', r'$RF$', r'$XG$'])
plt.xlabel('Time')
plt.ylabel('Models')
plt.show()<jupyter_output><empty_output>