<jupyter_start><jupyter_text>Lambda School Data Science*Unit 2, Sprint 1, Module 1*---  Regression 1 AssignmentYou'll use another **New York City** real estate dataset. But now you'll **predict how much it costs to rent an apartment**, instead of how much it costs to buy a condo.The data comes from renthop.com, an apartment listing website.- [ ] Look at the data. Choose a feature, and plot its relationship with the target.- [ ] Use scikit-learn for linear regression with one feature. You can follow the [5-step process from <NAME>](https://jakevdp.github.io/PythonDataScienceHandbook/05.02-introducing-scikit-learn.htmlBasics-of-the-API).- [ ] Define a function to make new predictions and explain the model coefficient.- [ ] Organize and comment your code.> [Do Not Copy-Paste.](https://docs.google.com/document/d/1ubOw9B3Hfip27hF2ZFnW3a3z9xAgrUDRReOEo-FHCVs/edit) You must type each of these exercises in, manually. If you copy and paste, you might as well not even do them. The point of these exercises is to train your hands, your brain, and your mind in how to read, write, and see code. If you copy-paste, you are cheating yourself out of the effectiveness of the lessons.If your **Plotly** visualizations aren't working:- You must have JavaScript enabled in your browser- You probably want to use Chrome or Firefox- You may need to turn off ad blockers- [If you're using Jupyter Lab locally, you need to install some "extensions"](https://plot.ly/python/getting-started/jupyterlab-support-python-35) Stretch Goals- [ ] Do linear regression with two or more features.- [ ] Read [The Discovery of Statistical Regression](https://priceonomics.com/the-discovery-of-statistical-regression/)- [ ] Read [_An Introduction to Statistical Learning_](http://faculty.marshall.usc.edu/gareth-james/ISL/ISLR%20Seventh%20Printing.pdf), Chapter 2.1: What Is Statistical Learning?<jupyter_code>import sys

# If you're on Colab:
if 'google.colab' in sys.modules:
    DATA_PATH = 'https://raw.githubusercontent.com/LambdaSchool/DS-Unit-2-Applied-Modeling/master/data/'

# If you're working locally:
else:
    DATA_PATH = '../data/'
    
# Ignore this Numpy warning when using Plotly Express:
# FutureWarning: Method .ptp is deprecated and will be removed in a future version. Use numpy.ptp instead.
import warnings
warnings.filterwarnings(action='ignore', category=FutureWarning, module='numpy')
# Read New York City apartment rental listing data
import pandas as pd
df = pd.read_csv(DATA_PATH+'apartments/renthop-nyc.csv', 
                 parse_dates=['created'], 
                 index_col='created')

# assert df.shape == (49352, 34)


# dtype_dict = {'ZIP_CODE': 'object',
#               'YEAR_BUILT': int}

# df = pd.read_csv(DATA_PATH+'condos/tribeca.csv', 
#                  dtype=dtype_dict, 
#                  parse_dates=['SALE_DATE'],
#                  index_col='SALE_DATE')
# Remove outliers: 
# the most extreme 1% prices,
# the most extreme .1% latitudes, &
# the most extreme .1% longitudes
df = df[(df['price'] >= 1375) & (df['price'] <= 15500) & 
        (df['latitude'] >=40.57) & (df['latitude'] < 40.99) &
        (df['longitude'] >= -74.1) & (df['longitude'] <= -73.38)]
df.head()
df.info()
# plotting histogram of the target variable 
df['price'].plot(kind='hist')
# the data is positively skewed 
df['elevator'].value_counts()
df['bedrooms'].value_counts()
import matplotlib.pyplot as plt

# style
plt.style.use('seaborn')

plt.scatter(df['bedrooms'], df['price']);

plt.xlabel('Number of Bedrooms')
plt.ylabel('Price')

plt.show()
# style
plt.style.use('seaborn')

plt.scatter(df['latitude'], df['price']);

plt.xlabel('Latitude')
plt.ylabel('Price')

plt.show()
plt.style.use('seaborn')

plt.scatter(df['longitude'], df['price']);

plt.xlabel('longitude')
plt.ylabel('Price')

plt.show()
# working with total bedrooms and price
X = df[['bedrooms']]
y = df['price']
# # convert the created from an object to datetime so we can split our dataset using created 
# df["created"] = pd.to_datetime(df["created"])
# df["date_created"] = df["created"].dt.date
# df["date_created"]
# # make the new column date_created as the index
# df = df.set_index('created')
# df.head()
# date created ranges from April to June. We will make month of June as our test set 
# now we split the dataset in train and test 

cutoff = '2016-06-01'

# applying the filter 
filt = X.index < cutoff
X_train, y_train = X.loc[filt], y.loc[filt]
X_test, y_test = X.loc[~filt], y.loc[~filt]
# baseline guess 

plt.hist(y_train);
baseline_guess = y_train.mean()

moe = abs(baseline_guess - y_train).mean()

print(f'prediction of a baseline model: ${round(baseline_guess,2)}, with a margin of error: ${round(moe,2)}')
# need to make a model that is more accurate than the baseline model above 
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
import numpy as np

lin_reg = LinearRegression()
lin_reg.fit(X,y);
lin_reg.coef_[0]
lin_reg.intercept_<jupyter_output><empty_output><jupyter_text>we could write the line equation to solve for the rent as follows:rent = 853.25 * (total_bedrooms) + 2267.97<jupyter_code>X_model = np.linspace(0, X_train['bedrooms'].max(), 50).reshape(-1,1)
# Note how we use the .predict() method with our model
rent_pred = lin_reg.predict(X_model)

# Plot our data
plt.scatter(X_train, y_train)

# Plot the regression line
plt.plot(X_model, rent_pred , color='red', label='Our Model')

plt.xlabel('Bedroom')
plt.ylabel('Rent Price')
plt.title('linear Regression')
plt.legend()
# Calculating RMSE score 
rent_predictions = lin_reg.predict(X)
rmse_scores = np.sqrt(mean_squared_error(rent_predictions, y))
rmse_scores
# validating the score using cross_val 
from sklearn.model_selection import cross_val_score

score = cross_val_score(lin_reg, X, y, scoring= 'neg_mean_squared_error', cv=10)
rmse = np.sqrt(-score)
std_rmse = np.std(rmse)

print(rmse)
print(rmse.mean())
std_rmse
# Scores on the test and validation set are almost the same. this model is low on bias and low on variance. Hence, a perfect generalized model<jupyter_output><empty_output><jupyter_text>we should be able to make the score better by adding polynomial regression as the reation is not exactly linear.<jupyter_code>from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures

poly_reg  = make_pipeline(PolynomialFeatures(degree=9), LinearRegression())
poly_reg.fit(X,y);

X_model = np.linspace(0, X_train['bedrooms'].max(), 50).reshape(-1,1)
# Note how we use the .predict() method with our model
rent_pred = poly_reg.predict(X_model)

# Plot our data
plt.scatter(X_train, y_train)

# Plot the regression line
plt.plot(X_model, rent_pred , color='red', label='Our Model')

plt.xlabel('Bedroom')
plt.ylabel('Rent Price')
plt.title('using Polynomial Regression')
plt.legend()
plt.show()
# Calculating RMSE score 

rent_pred1 = poly_reg.predict(X)
rmse_scores = np.sqrt(mean_squared_error(rent_pred1, y))
rmse_scores
# validating the score using cross_val 

score = cross_val_score(poly_reg, X, y, scoring= 'neg_mean_squared_error', cv=10)
rmse = np.sqrt(-score)
std_rmse = np.std(rmse)

print(rmse)
print(rmse.mean())
std_rmse
# we got the best model possible with polynomial features degrees set to 9<jupyter_output><empty_output><jupyter_text>Lets do it using Ridge Regression and see what we get. Don't accept to better resulst because model was not overfitting.<jupyter_code>from sklearn.linear_model import Ridge
# use Random Search to find the best value of alpha for ridge regression 
from sklearn.model_selection import RandomizedSearchCV

ridge = Ridge(normalize=True, random_state=42)
parameters = {'alpha':[1e-15, 1e-10, 1e-8, 1e-5, 1e-2, 1, 5,10, 20,30,40, 50, 60, 100, 110]}

ridge_reg = RandomizedSearchCV(ridge, parameters, scoring= 'neg_mean_squared_error', cv=10, random_state=42)
ridge_reg.fit(X,y)

print(ridge_reg.best_params_)
print(ridge_reg.best_estimator_)
# using the best value of alpha for ridge lets calculate the rmse and see if we get any better results 

ridge = ridge_reg.best_estimator_

# Calculating RMSE score 

rent_pred2 = ridge.predict(X)
rmse_scores = np.sqrt(mean_squared_error(rent_pred2, y))
rmse_scores
X_model = np.linspace(0, X_train['bedrooms'].max(), 50).reshape(-1,1)
# Note how we use the .predict() method with our model
rent_pred = ridge.predict(X_model)

# Plot our data
plt.scatter(X_train, y_train)

# Plot the regression line
plt.plot(X_model, rent_pred , color='red', label='Our Model')

plt.xlabel('Bedroom')
plt.ylabel('Rent Price')
plt.title('Using Ridge Regression')
plt.legend()
plt.show()
# validating the score using cross_val 

score = cross_val_score(ridge, X, y, scoring= 'neg_mean_squared_error', cv=10)
rmse = np.sqrt(-score)
std_rmse = np.std(rmse)

print(rmse)
print(rmse.mean())
std_rmse<jupyter_output>[1513.61744289 1442.79760998 1499.81967906 1467.77577537 1472.22972372
 1532.00247255 1516.93963049 1505.28687873 1454.30489411 1477.66023195]
1488.243433883667<jupyter_text>Lets try usin Stochastic Gradient Descent with ridge<jupyter_code>from sklearn.linear_model import SGDRegressor

sgd_reg = SGDRegressor(random_state=42)
# lets start by finding out the best value of the learing rate 

parameters1 = {'eta0': [0.001, 0.005, 0.01, 0.03, 0.06, 0.09, 1, 1.05]}

sgd_regressor = RandomizedSearchCV(sgd_reg, parameters1, cv=10, scoring = 'neg_mean_squared_error', random_state=42)
sgd_regressor.fit(X,y)

print(sgd_regressor.best_params_)
print(sgd_regressor.best_estimator_)
sgd_reg = sgd_regressor.best_estimator_

rent_pred4 = sgd_reg.predict(X)
rmse = np.sqrt(mean_squared_error(rent_pred4, y))
rmse
X_model = np.linspace(0, X_train['bedrooms'].max(), 50).reshape(-1,1)
# Note how we use the .predict() method with our model
rent_pred5 = sgd_reg.predict(X_model)

# Plot our data
plt.scatter(X_train, y_train)

# Plot the regression line
plt.plot(X_model, rent_pred5 , color='red', label='Our Model')

plt.xlabel('Bedroom')
plt.ylabel('Rent Price')
plt.title('Using SGD regression with Ridge')
plt.legend()
plt.show()
# validating the score using cross_val 
# shouldn't be any different than ridge regression

score = cross_val_score(sgd_reg, X, y, scoring= 'neg_mean_squared_error', cv=10)
rmse = np.sqrt(-score)
std_rmse = np.std(rmse)

print(rmse)
print(rmse.mean())
std_rmse<jupyter_output>[1513.57503353 1443.298727   1499.67209299 1466.98882474 1472.31476895
 1532.83228657 1516.84217954 1504.98170739 1455.4029412  1477.19573594]
1488.310429785467
StarcoderdataJupyterStructuredCleanDedup