<jupyter_start><jupyter_text>多元线性回归-正规方程 [![LR9.png](https://i.postimg.cc/NMwQDTGT/LR9.png)](https://postimg.cc/LqNd6J16)[![LR10.png](https://i.postimg.cc/jjfsyvTt/LR10.png)](https://postimg.cc/XZVSbfXz)[![LR11.png](https://i.postimg.cc/sD0y8bD3/LR11.png)](https://postimg.cc/YhgJLnWP)- **即是多元线性回归的正规方程解（Normal Equation）**- 问题：时间复杂度高：O(n^3)- 优点：不需要对数据做归一化处理  2. 动手实现<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
boston = datasets.load_boston()

X = boston.data
y = boston.target

X[1, :]
X = X[y < 50.0]
y = y[y < 50.0]
X.shape
from LR.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, seed=666)
from LR.LinearRegression import LinearRegression
reg = LinearRegression()
reg.fit_normal(X_train, y_train)
reg.coef_
reg.intercept_
reg.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>3. scikit-learn中的线性回归<jupyter_code>from sklearn.linear_model import LinearRegression
lin_reg = LinearRegression()
lin_reg.fit(X_train, y_train)
lin_reg.coef_
lin_reg.intercept_
lin_reg.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>4. 使用scikit-learn中的kNN Regression 解决回归问题<jupyter_code>from sklearn.neighbors import KNeighborsRegressor
knn_reg = KNeighborsRegressor()
knn_reg.fit(X_train, y_train)
knn_reg.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>- **显然，kNN预测分数低于线性回归模型的分数**- 下面对 kNN 使用网格搜索找到最佳超参数<jupyter_code>from sklearn.model_selection import GridSearchCV

param_grid = [
    {
        'weights': ['uniform'],
        'n_neighbors': [i for i in range(1, 11)]
    },
    {
        'weights': ['distance'],
        'n_neighbors': [i for i in range(1, 11)],
        'p': [i for i in range(1, 6)]
    }
]
knn_reg = KNeighborsRegressor()
grid_search = GridSearchCV(knn_reg, param_grid, n_jobs=-1, verbose=1)
grid_search.fit(X_train, y_train)
grid_search.best_params_
grid_search.best_score_ # 此时是网格搜索自己的评估算法
# 此时使用和线性回归相同的评估算法，可以看出，得分有很大的提高，但仍然不如线性回归模型
grid_search.best_estimator_.score(X_test, y_test)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup