# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# # Data analysis with Pandas
# ***
# Importing, cleaning and analyzing the "hotel booking demand" dataset from Kaggle, which is available [here](https://www.kaggle.com/jessemostipak/hotel-booking-demand).

# +
# Importing all required packages

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# +
# Downloading the dataset

# !kaggle datasets download -d jessemostipak/hotel-booking-demand
# !unzip hotel-booking-demand.zip
# !rm hotel-booking-demand.zip

# +
# Loading the dataset and checking some of its features

dataset = pd.read_csv("hotel_bookings.csv")

dataset.head()
#dataset.describe()

# +
# As a safety precaution, and following best practices, the analysis will not be conducted on the original dataset
# Instead, we'll create a deep copy of the original dataset and use it for our analysis

hotel_bookings = dataset.copy(deep=True)

# +
# We've noticed that there are null values in this dataset. However, how many are there?

null_data = hotel_bookings.isna()
null_total = null_data.sum().sum()
null_relative = null_total/hotel_bookings.size

print(f"Total number of null features: {null_total}")
print(f"% of null features relative to the overall data: {np.round(null_relative * 100, 3)}%")

# +
# It seems that roughly 3.4% of our data is null
# Although it may seem insignificant, we need to know which columns are being affected by this

null_data.columns[null_data.any()]

# +
# With this information, what is the ratio of each column's null to filled values?

null_children = null_data.children.mean()
null_country = null_data.country.mean()
null_agent = null_data.agent.mean()
null_company = null_data.company.mean()

print(f"% of null values in the children column: {np.round(null_children * 100, 3)}%")
print(f"% of null values in the country column: {np.round(null_country * 100, 3)}%")
print(f"% of null values in the agent column: {np.round(null_agent * 100, 3)}%")
print(f"% of null values in the company column: {np.round(null_company * 100, 3)}%")
# -

# ## Dealing with the null columns
# ***
# Now, the question is: should these columns be cleaned? It depends. However, in this scenario, having things such as a null amount of children and null country could mean that the information is unknown. Alternatively, it could mean that the information is truly missing, i.e. someone forgot to add it in or it was corrupted somehow.
#
# Still, leaving ```NaN``` values doesn't help us either. Thus, we must change this if we are to extract meaningful data from this dataset. We can think about it the following way:
#
# * Having a ```NaN``` amount of children could mean that the person has no children, or has 1 or more. However, given we don't know that, we could assign the number 0 to those null values as, either way, they won't affect our data analysis, yet having a numerical value is preferable.
#
# * Being from a ```NaN``` country could mean, as stated previously, that the information was lost or is unknown. However, rather than having a null value, we could replace all of those ```NaN``` with ```UNK```, thus preserving the original data structure and getting rid of null values at the same time.
#
# * Having a ```NaN``` agent ID could, once again, mean that the information was lost or is unknown. As it's a numerical value, and as it represents a travel agency ID, we could use the number 0 to denote this lack of information, as no agency has 0 as its ID.
#
# * A ```NaN``` company ID could mean that the person truly doesn't work at a company, or that the information was lost (missing or corrupted). Once again, assigning 0 to a null company ID will have the same effect, given no company has such an ID.

# +
# Replacing all NaNs with appropriate values

hotel_bookings.children.fillna(0, inplace=True)
hotel_bookings.country.fillna("UNK", inplace=True)
hotel_bookings.agent.fillna(0, inplace=True)
hotel_bookings.company.fillna(0, inplace=True)

# +
# Checking some samples from our cleaned dataframe

hotel_bookings.sample(10)
# -

# ## Additional cleaning
# ***
# We can do some more cleaning on this dataframe. For instance, the ```children```, ```agent``` and ```company``` columns are of type float while they'd be better suited for the integer data type. Additionally, ```arrival_date_year```, ```arrival_date_month``` and ```arrival_date_day_of_month``` could be combined into a single column called ```arrival_date```
#

# +
# Changing the data type of children, agent and company

change_columns = ["children", "agent", "company"]

hotel_bookings[change_columns] = hotel_bookings[change_columns].astype(int)

# +
# Creating a datetime column from the day, month and year columns

columns = ["arrival_date_day_of_month", "arrival_date_month", "arrival_date_year"]
new_columns = ["day", "month", "year"]

arrival_date = hotel_bookings[columns]
arrival_date.columns = new_columns
arrival_date.month = pd.to_datetime(arrival_date.month, format='%B').dt.month
arrival_date.insert(0, "arrival_date", pd.to_datetime(arrival_date[new_columns].copy(), format="%mm/%dd/%YYYY").dt.date)

arrival_date.drop(["day", "month", "year"], axis=1, inplace=True)

arrival_date.sample(10)

# +
# Now, all we need to do is insert the new column on our dataframe and drop the old ones

columns_to_drop = ["arrival_date_day_of_month", "arrival_date_month", "arrival_date_year"]

hotel_bookings.insert(3, "arrival_date", arrival_date)
hotel_bookings.drop(hotel_bookings[columns_to_drop], axis=1, inplace=True)

hotel_bookings.head()

# +
# Saving the cleaned dataset to a .csv file

hotel_bookings.to_csv("../datasets/cleaned_hotel_bookings.csv", index=False)
# -

# ## Exploratory data analysis
# ***
# Now that the dataframe has been properly cleaned and saved, it's ready for analysis. As such, now we can conduct statistical measurements on our data and plot information that is relevant to us.

# +
# 1st statistical measurement: what's the number of people that have visited the hotels in our dataset? And 
# what is the proportion of adults relative to the overall population in our data?

columns = ["adults", "children", "babies", "hotel"]

total_people = hotel_bookings[columns].groupby("hotel").sum()
hotel_population = hotel_bookings[columns[0:3]].sum().sum()
adults_ratio = hotel_bookings.adults.sum().sum()/hotel_population

print(f"Average number of adults per hotel type: \n{np.round(total_people, 2)}")
print(f"% of adults in population: {np.round(adults_ratio * 100, 2)}")

# +
# As expected, adults constitute a majority of our population
# We can plot our results to get a clearer picture of our data

sns.set_theme()

ticks = np.arange(0, 175000, 25000)

ax = total_people.plot(kind="bar",
                  figsize=(10,10),
                  legend=True,
                  fontsize=14,
                  rot=0
                 )

ax.set_title("Population per hotel", fontsize=14)
ax.set_yticks(ticks)
ax.set_xlabel("Hotel", fontsize=14)
ax.legend(loc=1, prop={"size": 14})
ax.set_ylabel("Population", fontsize=14)

for p in ax.patches:
    ax.annotate("%d" % p.get_height(), (p.get_x() + p.get_width()/2., p.get_height()), ha="center", va="center", xytext=(0, 10), textcoords='offset points')

# +
# 2nd statistical measurement: what is the arrival rate by date? 
# Which date had the most arrivals?

arrival_by_date = pd.Series(hotel_bookings["arrival_date"])
frequency = arrival_by_date.value_counts() 

print(f"Arrival by date: \n{frequency}")

# +
# It seems that December 5th, 2015 had the most arrivals
# Let's use a histogram to plot our results

x = np.arange(0, 500, 50)

ax = frequency.plot(kind="hist",
                    figsize=(10,10),
                    bins=45,
                    alpha=0.7,
                    legend=False,
                    fontsize=14,
                    rot=0
                   )

ax.set_title("Arrival rate by number of hotels", fontsize=14)
ax.set_xlabel("People", fontsize=14)
ax.set_ylabel("Number of hotels", fontsize=14)

# -



StarcoderdataJupyterScriptsDedupFiltered