<jupyter_start><jupyter_text>练习 3-1 为MNIST训练一个分类器，并在测试集上达到97%的精度。- KNeighborsClassifier对这个任务非常有效，你只需要找到合适的超参数值即可<jupyter_code>%matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import fetch_openml
# 下载MNIST数据集
mnist = fetch_openml('mnist_784')
mnist
mnist_data = mnist['data']
mnist_label = mnist['target'].astype(np.uint8)
len(mnist_data), len(mnist_label)
X_train, y_train, X_test, y_test = mnist_data[:60000], mnist_label[:60000], mnist_data[60000:], mnist_label[60000:]
y_train[:10]
# 无论合适都要养成对数据进行清洗的好习惯
shuffled_index = np.random.permutation(len(X_train))
X_train, y_train = X_train[shuffled_index], y_train[shuffled_index]

some_digit = X_train[10]
some_digit_label = y_train[10]

plt.imshow(some_digit.reshape(28, 28), cmap=matplotlib.cm.binary)
some_digit_label
from sklearn.model_selection import cross_val_predict, cross_val_score
from sklearn.metrics import accuracy_score
from sklearn.neighbors import KNeighborsClassifier
knn_clf = KNeighborsClassifier()
knn_clf
#cross_val_score(knn_clf, X_train, y_train, cv=3, scoring='accuracy', n_jobs=-1)
knn_clf.fit(X_train, y_train)
y_pred = knn_clf.predict(X_test)
accuracy_score(y_test, y_pred)<jupyter_output><empty_output><jupyter_text>使用网络搜索找到最佳的参数<jupyter_code>from sklearn.model_selection import GridSearchCV
param_grid=[{
    'weights': ['uniform', 'distance'],
    'n_neighbors': [2, 3, 4, 5, 6, 7, 8, 9],
    'algorithm': ['ball_tree', 'kd_tree', 'brute']
}]
grid_search = GridSearchCV(knn_clf, param_grid, cv=3, verbose=3, n_jobs=-1)
grid_search.fit(X_train, y_train)
grid_search.best_score_, grid_search.best_params_
y_pred = grid_search.predict(X_test)
accuracy_score(y_test, y_pred)<jupyter_output><empty_output><jupyter_text>对模型进行更多探索<jupyter_code># 说先把网格搜索出来的比较好的模型实例化出来
knn_clf = KNeighborsClassifier(**grid_search.best_params_)<jupyter_output><empty_output><jupyter_text>混淆矩阵 1. 计算混淆矩阵<jupyter_code>y_train_pred = cross_val_predict(knn_clf, X_train, y_train, cv=3)
from sklearn.metrics import confusion_matrix
conf_mx = confusion_matrix(y_train, y_train_pred)
conf_mx<jupyter_output><empty_output><jupyter_text>2. 可视化混淆矩阵<jupyter_code>row_sums = conf_mx.sum(axis=1, keepdims=True)
norm_conf_mx = conf_mx/row_sums
norm_conf_mx
np.fill_diagonal(norm_conf_mx, 0) #对角线填充为0
plt.figure(figsize=(8, 8))
plt.matshow(norm_conf_mx, cmap= plt.cm.gray)
plt.show()<jupyter_output><empty_output><jupyter_text>计算Precision/Recall 1. 计算Precision和Recall<jupyter_code>from sklearn.metrics import precision_score, recall_score
y_pred = knn_clf.predict(X_test)


precision_score(y_test, y_pred, average='macro'), recall_score(y_test, y_pred, average='macro')<jupyter_output><empty_output>