# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     name: python3
# ---

# + [markdown] id="qOBFhUxcYB_7"
# ## Recommendation Systems in Python
#
# We will work on the MovieLens dataset and build a model to recommend movies to the end users. This data has been collected by the GroupLens Research Project at the University of Minnesota. 
#
# This dataset consists of:
# *   100,000 ratings (1–5) from 943 users on 1682 movies
# *   Demographic information of the users (age, gender, occupation, etc.)
#
# First, we’ll import our standard libraries and read the dataset in Python.

# + id="5E7yKqZCX7Da"
import pandas as pd
# %matplotlib inline
import matplotlib
import matplotlib.pyplot as plt
import numpy as np

# + id="iVSqiMCjYg55"
# pass in column names for each CSV as the column name is not given in the file and read them using pandas.
# You can check the column names from the readme file
#Reading users file:
u_cols = ['user_id', 'age', 'sex', 'occupation', 'zip_code']
users = pd.read_csv('/content/drive/MyDrive/ml-100k/u.user', sep='|', names=u_cols,encoding='latin-1')

#Reading ratings file:
r_cols = ['user_id', 'movie_id', 'rating', 'unix_timestamp']
ratings = pd.read_csv('/content/drive/MyDrive/ml-100k/u.data', sep='\t', names=r_cols,encoding='latin-1')

#Reading items file:
i_cols = ['movie id', 'movie title' ,'release date','video release date', 'IMDb URL', 'unknown', 'Action', 'Adventure',
'Animation', 'Children\'s', 'Comedy', 'Crime', 'Documentary', 'Drama', 'Fantasy', 'Film-Noir', 'Horror', 'Musical', 'Mystery', 'Romance', 'Sci-Fi', 'Thriller', 'War', 'Western']

items = pd.read_csv('/content/drive/MyDrive/ml-100k/u.item', sep='|', names=i_cols,encoding='latin-1')

# + [markdown] id="GD6-26-fZILv"
# **After loading the dataset, we should look at the content of each file (users, ratings, items).**
#
# ## - Users

# + colab={"base_uri": "https://localhost:8080/", "height": 221} id="9hnGkwCSZApC" outputId="f56d1813-5f54-4736-b637-7eb2ad1d1513"
print(users.shape)
users.head()

# + [markdown] id="EMLmTGgTZmJM"
# So, we have 943 users in the dataset and each user has 5 features, i.e. user_ID, age, sex, occupation and zip_code. Now let’s look at the ratings file.
#
# ## - Ratings

# + colab={"base_uri": "https://localhost:8080/", "height": 221} id="PsY78af8Zghq" outputId="58cf2ca6-3622-49fd-a34c-55211e5a2545"
print(ratings.shape)
ratings.head()

# + [markdown] id="5sLoWAFpZzia"
# We have 100k ratings for different user and movie combinations. Now finally examine the items file.
#
# ## - Items

# + colab={"base_uri": "https://localhost:8080/", "height": 394} id="fDwt0JQGZw6p" outputId="d01c06e0-721b-42ed-e6b9-b4c5bca23fd8"
print(items.shape)
items.head()

# + [markdown] id="yZbir-sMaEgh"
# This dataset contains attributes of 1682 movies. There are 24 columns out of which last 19 columns specify the genre of a particular movie. These are binary columns, i.e., a value of 1 denotes that the movie belongs to that genre, and 0 otherwise.
#

# + colab={"base_uri": "https://localhost:8080/"} id="KRk4UsA7Z5mG" outputId="b43e1991-0e9c-4706-8a97-c0fdd4bbb5f6"
r_cols = ['user_id', 'movie_id', 'rating', 'unix_timestamp']

ratings_train = pd.read_csv('/content/drive/MyDrive/ml-100k/ua.base', sep='\t', names=r_cols, encoding='latin-1')

ratings_test = pd.read_csv('/content/drive/MyDrive/ml-100k/ua.test', sep='\t', names=r_cols, encoding='latin-1')

ratings_train.shape, ratings_test.shape

# + [markdown] id="kfSPvIGebOIK"
# The dataset has already been divided into train and test by GroupLens where the test data has 10 ratings for each user, i.e. 9,430 rows in total. We will read both these files into our Python environment.

# + [markdown] id="8E3rdTCvbRZQ"
# It’s finally time to build our recommend engine!
#
# ## Building collaborative filtering model
#
# We will recommend movies based on user-user similarity and item-item similarity. For that, first we need to calculate the number of unique users and movies.

# + colab={"base_uri": "https://localhost:8080/"} id="-bXQCl4AbDLz" outputId="5b5ab103-6d2c-4d59-f868-dd39c1e535c7"
n_users = ratings.user_id.unique().shape[0]
n_items = ratings.movie_id.unique().shape[0]

n_users,n_items

# + [markdown] id="52rEX2xZb2nW"
# Now, we will create a user-item matrix which can be used to calculate the similarity between users and items. we will first initialize it with zeros array of shape 943 x 1643 having 943 users and 1643 movies
# then we will loop in the ratings data frame row by row

# + colab={"base_uri": "https://localhost:8080/"} id="Emi7yvePbu6h" outputId="299a7964-af53-4626-dabb-f25e6faf5db2"
data_matrix = np.zeros((n_users, n_items))
for line in ratings.itertuples():
    data_matrix[line[1]-1, line[2]-1] = line[3]

# + [markdown] id="JJr0Fd_Qb7bi"
# here,\
# line[1] is the userId and we are subtracting 1 from it since array indexing starts from 0 = row\
# line[2]-1 is the movie id = column\
# now at that specifec row and column i.e, user and movie we will add line[3] which is the movie rating
#
# Now, when we have rating os all the movies given by each user in a matrix we will calculate the similarity. We can use the pairwise_distance function from sklearn to calculate the cosine similarity.
#

# + colab={"base_uri": "https://localhost:8080/"} id="-XYn4bqUcuPR" outputId="d2307c3a-0d95-4805-8676-021437f75940"
data_matrix[:5]

# + colab={"base_uri": "https://localhost:8080/"} id="jz26uGMVclr4" outputId="8c6bbcee-3e66-4f9d-a8e6-a6e92c4e3b92"
from sklearn.metrics.pairwise import pairwise_distances 

user_similarity = pairwise_distances(data_matrix, metric='cosine')
item_similarity = pairwise_distances(data_matrix.T, metric='cosine')

print(user_similarity.shape, item_similarity.shape)


# + [markdown] id="lemlNTI5eazh"
# This gives us the item-item and user-user similarity in an array form. The next step is to make predictions based on these similarities. Let’s define a function to do just that.
#

# + id="9Ndn5gEXePMM"
def predict(ratings, similarity, type='user'):
    
    if type == 'user':
        mean_user_rating = ratings.mean(axis=1).reshape(-1,1)
        #We use np.newaxis so that mean_user_rating has same format as ratings
        
        ratings_diff = (ratings - mean_user_rating)
        pred = mean_user_rating + similarity.dot(ratings_diff) / np.array([np.abs(similarity).sum(axis=1)]).T
    
    elif type == 'item':
        pred = ratings.dot(similarity) / np.array([np.abs(similarity).sum(axis=1)])
    
    return pred


# + [markdown] id="kpEsvk0he_-i"
# Finally, we will make predictions based on user similarity and item similarity.

# + id="0w9PxVwXe_PV"
user_prediction = predict(data_matrix, user_similarity, type='user')
item_prediction = predict(data_matrix, item_similarity, type='item')

# + colab={"base_uri": "https://localhost:8080/"} id="4AKbLIwkfE6P" outputId="8d2d6423-e65a-4eac-e135-61e4e147699a"
user_prediction[0]

# + id="<KEY>"


StarcoderdataJupyterScriptsDedupFiltered