<jupyter_start><jupyter_text>Data Science Unit 1 Sprint Challenge 1 Loading, cleaning, visualizing, and analyzing dataIn this sprint challenge you will look at a dataset of the survival of patients who underwent surgery for breast cancer.http://archive.ics.uci.edu/ml/datasets/Haberman%27s+SurvivalData Set Information:The dataset contains cases from a study that was conducted between 1958 and 1970 at the University of Chicago's Billings Hospital on the survival of patients who had undergone surgery for breast cancer.Attribute Information:1. Age of patient at time of operation (numerical)2. Patient's year of operation (year - 1900, numerical)3. Number of positive axillary nodes detected (numerical)4. Survival status (class attribute)-- 1 = the patient survived 5 years or longer-- 2 = the patient died within 5 yearSprint challenges are evaluated based on satisfactory completion of each part. It is suggested you work through it in order, getting each aspect reasonably working, before trying to deeply explore, iterate, or refine any given step. Once you get to the end, if you want to go back and improve things, go for it!  Part 1 - Load and validate the data- Load the data as a `pandas` data frame.- Validate that it has the appropriate number of observations (you can check the raw file, and also read the dataset description from UCI).- Validate that you have no missing values.- Add informative names to the features.- The survival variable is encoded as 1 for surviving >5 years and 2 for not - change this to be 0 for not surviving and 1 for surviving >5 years (0/1 is a more traditional encoding of binary variables)At the end, print the first five rows of the dataset to demonstrate the above.<jupyter_code>#load data, validating shape, adding informative names
import pandas as pd
df= pd.read_csv('http://archive.ics.uci.edu/ml/machine-learning-databases/haberman/haberman.data', header=None, names=['age', 'op_year', 'nodes_pos', 'survival'])
print(df.shape)
df.head()
#missing values check
df.isna().sum()
#no missing values
#value count pre-edit
print(df['survival'].value_counts())
#survival feature engineering
df['survival'] = [0 if d==2 else 1 for d in df['survival']]
#evaluating success
df['survival'].value_counts()
#success
#making operation year into datetime
df['op_year'] = [(d-70)*365 for d in df['op_year']]
df['op_year'] = pd.to_datetime(df['op_year'], unit='D')
df['op_year'] = df['op_year'].dt.year<jupyter_output><empty_output><jupyter_text>Part 2 - Examine the distribution and relationships of the featuresExplore the data - create at least *2* tables (can be summary statistics or crosstabulations) and *2* plots illustrating the nature of the data.This is open-ended, so to remind - first *complete* this task as a baseline, then go on to the remaining sections, and *then* as time allows revisit and explore further.Hint - you may need to bin some variables depending on your chosen tables/plots.<jupyter_code>#table 1: basic stats
df.describe()
#table 2: operation year, survival crosstab
pd.crosstab(df['survival'],df['op_year'], normalize='columns')
#table 3: age, survival crosstab
age_bins = pd.cut(df['age'], 5)
pd.crosstab(age_bins, df['survival'], normalize='index')
#plot 1: survival vs op year
import matplotlib.pyplot as plt
df.groupby('survival')['op_year'].value_counts().unstack(0).plot.bar()
plt.xlabel('Year of Operation')
plt.ylabel('Outcome Frequency (Success = 1)')
plt.title('Breast Cancer Removal Success 1958-1969');
#plot 2: survival vs age
df.groupby('survival')['age'].value_counts().unstack(0).plot.bar(figsize=(30,6))
plt.xlabel('Age of Patient')
plt.ylabel('Outcome Frequency (Success = 1)')
plt.title('Breast Cancer Removal Success by Age of Patient');
#plot 3: nodes vs age
df.groupby('survival')['nodes_pos'].value_counts().unstack(0).plot.bar(figsize=(30,6))
plt.xlabel('Number of Axillary Cancerous Nodes')
plt.ylabel('Outcome Frequency (Success = 1)')
plt.title('Breast Cancer Removal Success by Number of Axillary Cancerous Nodes');<jupyter_output><empty_output><jupyter_text>Part 3 - Analysis and InterpretationNow that you've looked at the data, answer the following questions:- What is at least one feature that looks to have a positive relationship with survival?- What is at least one feature that looks to have a negative relationship with survival?- How are those two features related with each other, and what might that mean?Answer with text, but feel free to intersperse example code/results or refer to it from earlier. *   Having no or few axillary (additional) cancerous nodes makes survival much more likely, regradless of age or year of operation.*   Being very young or very old will decrease the chance of survival.*   Unfortunately, cancer can grow much quicker than people age, so regardless of age, it is more important to get screened for Breast Cancer frequently to minimize the chance of cancer growth and maximize prompt treatment if necessary.<jupyter_code><jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup