<jupyter_start><jupyter_text>Linear Regression with closed form solution Import necessary libraries<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import scipy.io as sio
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Define the function to compute cost function<jupyter_code>def computeTheta(X, Y):
    """
    Calculates theta using the closed form solution.
    This method uses the following formula to
    calculate the value of regression parameters:
    theta = ((X.T*X)^(-1))*(X.T*Y)
    where:
    X is a matrix storing the input data points,
    Y is the observed outputs,
    X.T represents the transpose of a matrix X and
    theta is a matrix storing the regression params
    
    Parameters
    ----------
    X : Nxd matrix
        N is the number of input samples and d is 
        the number of features
    Y : Nx1 matrix
        The matrix storing the actual outputs
    
    Returns
    -------
    dx1 matrix
        The calculated regression parameters
    """
    inversePart = np.power(X.T*X, -1)  #
    rest = X.T*Y
    return inversePart*rest<jupyter_output><empty_output><jupyter_text>Import data<jupyter_code>data = sio.loadmat("dataset1.mat")
data = pd.DataFrame(np.hstack((data['X_trn'], data['Y_trn'])))
data.columns = ['X_trn', 'Y_trn']
data.describe()
data.plot(kind="scatter", x="X_trn", y="Y_trn", figsize=(12, 8))<jupyter_output><empty_output><jupyter_text>Add a column of 1s to X<jupyter_code>data.insert(0, "ones", 1)
X = data.iloc[:, 0:2]
X.head()
X = np.matrix(X.values)
Y = data.iloc[:, 2:]
Y.head()
Y = np.matrix(Y.values)
theta = computeTheta(X, Y)
print(theta)<jupyter_output>[[ 196.5580325 ]
 [ 117.95493298]]