# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# + [markdown] slideshow={"slide_type": "slide"}
#
#
# # Project: Analyzing TMDB movies dataset- Exploratory data analysis
#
# ## Table of Contents
# <ul>
# <li><a href="#intro">Introduction</a></li>
# <li><a href="#wrangling">Data Wrangling</a></li>
# <li><a href="#eda">Exploratory Data Analysis</a></li>
# <li><a href="#conclusions">Conclusions</a></li>
# </ul>

# + [markdown] slideshow={"slide_type": "subslide"}
# <a id='intro'></a>
# ## Introduction
# > **Questions**: In this project, we find top 10 movies based on profit and rating and, secondly, what are the properties associated with high revenue movies?  
#
# > **Information about the dataset**: Here we investigate the movies data based on 10000 movies using NumPy and Pandas. For each movie, the dataset includes information on aspects such as popularity, budget, revenue, cast, directors, production house, date of release, runtime, and its rating.

# + slideshow={"slide_type": "subslide"}
# Use this cell to set up import statements for all of the packages that you
#   plan to use.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
data=pd.read_csv('tmdb-movies.csv')

# %matplotlib inline
# -

data.shape

# **Data**: There are 10866 instances with 21 columns. 

# + [markdown] slideshow={"slide_type": "subslide"}
# <a id='wrangling'></a>
# ## Data Wrangling
#
# ### General Properties
# -

# Here, we are looking at the general properties of the dataset. 

data.head()

# + slideshow={"slide_type": "slide"}
# Looking at the types of data and missing values
data.info()
# -

# **Missing values**
#
# Looks like there are some missing values.
# I decided to drop 5 columns as they contained a lot of missing values and were not relevant to the questions. 
#

# +
# Drop colums with a lot of missing variables
data.drop(columns=['homepage','tagline','keywords','imdb_id','overview'],inplace=True)

# Check number of missing rows after removing those columns
data.isna().sum()
# -

# Eliminate missing rows
data.dropna(axis=0,inplace=True)
data.isna().sum()

data.info()

# We have 9773 samples of movies after removing rows containing missing values. 

# Dealing with zero values: For the variables such as 'budget' and 'revenue', there are some cells with zero values that are due to errors. So, I replaced 
# them with NaN. 

0 in data['budget'].values

0 in data['revenue'].values

data['budget'].eq(0).sum()

# For more than half of the movies, zero values were recorded for the budget column. 

data['revenue'].eq(0).sum()

# For more than half of the movies, zero values were recorded for the revenue column. 

# Removing these zero values are equivalent to missing a lot of information. So, I decided to replace them with NaNs and keep them. 

data['budget']=data['budget'].replace(0,np.NaN)
data['revenue']=data['revenue'].replace(0,np.NaN)

data.info()

# Changing the format of release_date to day_time format

data['release_date']=pd.to_datetime(data['release_date'])
data.head()

# Check for duplicated values and drop duplicates

# Check duplicated values
data.duplicated().sum()
# This shows only 1 duplicate and we now delete the duplicate 
data.drop_duplicates(inplace=True)
data.duplicated().sum()

# Dealing with zero values under budget and revenue--- let's see how many cells have zero values?
print(data[data['budget']==0].shape[0],data[data['revenue']==0].shape[0])

# Final shape of data
data.shape

# After the data cleaning step, there are 9772 samples of data. 

# <a id='eda'></a>
# ## Exploratory Data Analysis
#
#
# ### Research Question 1 

#
# **Question 1** What are the top 10 movies of all time based on the highest profit achieved and average rating? Are there any common movies in top 10 lists based on both profit and average rating?

# Here we calculate the profit by subtracting budget from revenue. 
data['profit']=data['revenue']-data['budget']
data['profit']

print('The movie with minimum profit is',data.loc[data['profit'].idxmin()]['original_title'],'.The movie with maximum profit is',data.loc[data['profit'].idxmax()]['original_title'])

# Findings: The movie with maximum profit is Avator and the movie with minimum profit is Warrier's way. 
# Below is the plot of top 10 movies based on highest profit. We use the seaborn library which provides an appropriate visualization for our need. 

ax = sns.barplot(x=data.sort_values(by=['profit'],ascending=False)['profit'][0:9],y=data.sort_values(by=['profit'],ascending=False)['original_title'][0:9])
sns.set(rc={'figure.figsize':(10,5)})
ax.set_title("Top 10 Movies based on profit",fontsize = 15)
ax.set_xlabel("Profit",fontsize = 13)
ax.set_ylabel("Original Title",fontsize = 13)
#setup the stylesheet
sns.set_style("darkgrid")

# highest and lowest vote counts
min_vote=data.loc[data['vote_average'].idxmin()]['original_title']
max_vote=data.loc[data['vote_average'].idxmax()]['original_title']
print('The movie with minimum average rating is '+min_vote+'. The movie with maximum average rating is '+max_vote)

# Top 10 highest rated movies
ax = sns.barplot(x=data.sort_values(by=['vote_average'],ascending=False)['vote_average'][0:9],y=data.sort_values(by=['vote_average'],ascending=False)['original_title'][0:9])
sns.set(rc={'figure.figsize':(10,5)})
ax.set_title("Top 10 Highest Rated Movies",fontsize = 15)
ax.set_xlabel("Vote Average",fontsize = 13)
ax.set_ylabel("Original Title",fontsize = 13)
#setup the stylesheet
sns.set_style("darkgrid")

# Findings: There is no movie in common in top 10 lists based on rating and profit.  

# ### Reseach Question 2

#
# **Question 2** What kinds of properties are associated with movies that have high revenues?

# Print top 10 directors based on highest revenue
director_by_revenue=data.groupby('director').mean()['revenue'].sort_values(ascending=False)
director_by_revenue[:19]

# Does revenue relate to budget?
fig=plt.figure()
plt.scatter(data['budget'],data['revenue'])
plt.xlabel('budget')
plt.ylabel('revenue')

# It looks like there is a positive relationship between budget and revenue. More money to invest can lead to a higher revenue. 

# Does revenue relate to average rating?
fig=plt.figure()
plt.scatter(data['revenue'],data['vote_average'])
plt.xlabel('vote_average')
plt.ylabel('revenue')

# There appears to be a positive relationship between revenue and average rating despite being a non-linear one. 

# How does revenue change with the release year since 2000?
data.groupby('release_year').mean()['revenue'][-16:].plot()

# There appears to be almost a steady increase in the amount of revenue since 2000 except a downfall in 2006. 

# ### Limitations:
# The main limitation I encountered while doing this project was a large number of samples with zero revenue and budget. Excluding them leads to losing more than half of our data. So I kept these samples and replaced them with NaNs in order to not account them in my calculations. In addition, there were some attributes with a large number of missing data and excluding associated rows leads to losing many data samples. Instead, I excluded the columns with a lot of missing data that were not relevant to the questions I asked. 

# <a id='conclusions'></a>
# ## Conclusions
#

# I computed the top 10 movies based on profit and average rating and found no movies in common in both top 10 lists. Among the top 10 movies listed, Avator has the highest profit and Pink Floyed has the highest average rating. Secondly, I explored how movie revenues related to various factors. Director <NAME>|<NAME> has achieved the highest revenue among other directors. Without statistical testing, I cannot confirm the presence or absence of relationships. However, revenue seems to be linked to budget and vote_average and has undergone a steady increase since 2000. 

StarcoderdataJupyterScriptsDedupFiltered