<jupyter_start><jupyter_text>Plotting with Pandas  Program so far ***- Python Basics- Python Programming Constructs- Data Structures- Functions- Object Oriented Programming in Python- NumPy- Pandas  What are we going to learn today?***- Python Data Visualization Libraries    - Matplotlib        - Matplotlib basics        - Barplot        - Histograms        - Box Plots        - Scatter Plots    - Seaborn- Plot Types - Plot Customization   Python Data Visualization LibrariesPython is vastly popular among data scientists, one key reason being its strength with data analysis. Data Visualization is a key skill for aspiring data scientists. It helps us to interpret data easily and convey information in an effective way. Why is it important?1. Explore data to get a better grasp of the data.2. Report insights to convey their results to somebody else.*** Image source : https://matplotlib.org/*** Introduction:Arguably, the most popular data visualization library with Python is Matplotlib. It produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. Matplotlib can be used in Python scripts, the Python and IPython shell, Jupyter notebook, web application servers, and four graphical user interface toolkits.Matplotlib makes hard things easy to interpret. You can generate plots, histograms, power spectra, bar charts, errorcharts, scatterplots, etc., with just a few lines of code.It is used along with NumPy and Pandas to provide an environment that is an effective open source alternative for MatLab. It can also be used with graphics toolkits like PyQt and wxPython.Here, you will learn to build various types of plots and to customize them to make them more visually appealing and interpretable.*** -->   Matplotlib Basics***At the core of it, we need to remember some important things1. The canvas to "draw" data(stored in the computer's memory)2. The axes3. "Show" that dataEverything else is just details.   Technical Details***<jupyter_code># Import the required Libraries
from matplotlib import pyplot as plt
import numpy as np

# Set it up to work with Jupyter Notebook
# (this is a Jupyter command, not Python code)
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Sample***<jupyter_code># Create data

def create_sample_chart():
    x_series = np.arange(10)
    y_series = x_series ** 2

    #Plotting to our canvas in memory
    plt.plot(x_series, y_series)
    #Title of our canvas
    plt.title('String Title Here')
    #X-axis label
    plt.xlabel('X Axis Title Here')
    #Y_axis label
    plt.ylabel('Y Axis Title Here')

    #Showing what we plotted
    plt.show();

create_sample_chart()<jupyter_output><empty_output><jupyter_text>Mini-Challenge - 1***Create a graph for `y = sin(x)`. X values should be from 0 to 10, incrementing in steps of 0.1Remember to label the axes and the chart title appropriately.<jupyter_code>def sine_wave():
    '''Write your code here'''
sine_wave()<jupyter_output><empty_output><jupyter_text>Let's do this with the Weather data!***Quick Refresher:The Weather Dataset is a time-series data set with per-hour information about the weather conditions at a particular location. It records Temperature, Dew Point Temperature, Relative Humidity, Wind Speed, Visibility, Pressure, and Conditions.  Read the data set<jupyter_code># Data Introduction
import pandas as pd
weather_df = pd.read_csv('../data/weather_2012.csv', parse_dates=True, index_col='Date/Time')
weather_df.head(5)<jupyter_output><empty_output><jupyter_text>How did the temperature vary amongst the months of the year?***Since we want to compare amongst months, we need one value for each month. Let's pick the mean for doing this.Generate a line chart that visualizes the temperature readings in the month of January:* x-values should be the values should be the months.* y-values should be the mean values in the Temp (C) column per month.Display the plot.  Creating the data in the required format<jupyter_code>monthly_data = weather_df.groupby(weather_df.index.month).mean()
monthly_data.head(3)
x_series = monthly_data.index
y_series = monthly_data['Temp (C)']<jupyter_output><empty_output><jupyter_text>Plotting: Step 1<jupyter_code>plt.plot(x_series, y_series)

plt.title('Temperature Trend, 2012')
plt.xlabel('Month')
plt.ylabel('Temp (C)')
plt.show()<jupyter_output><empty_output><jupyter_text>Plotting: Step 2The chart we plotted is decent, but we aren't done yet. The X axis is labelled correctly ("Month"), but the values themselves are 2, 4, etc, instead of January, February, etc.    Mini-Challenge - 2***Generate the same plot as above, but this time the values of the "Month" should be January, February etc instead of 1,2,3,etc.<jupyter_code># First, get calendar month names
import calendar
calendar_months = calendar.month_name[1:]


print(calendar_months)
# now we have the names, let's use them instead
''' Your code goes here'''


# which x_series values to replace by which names
'''Your code goes here'''<jupyter_output><empty_output><jupyter_text>Plotting: Step 3Almost done, but not quite. We need the X-axis labels to be a bit more legible. Let's rotate them a bit.    Mini-Challenge - 3***Generate the same graph as above, but with the X-axis labels rotated, so that they are legible.<jupyter_code>'''Your code goes here'''<jupyter_output><empty_output><jupyter_text>Other Plot Types***The chart we have so far worked with is called a Line Chart. There are other types of plots. Some important examples are:- Bar Plots- Histograms- Box Plots- Scatter Plots   Barplot***Some of the important characteristics of a barplot are:1. It is a chart or graph that presents grouped data with rectangular bars with lengths proportional to the values that they represent.2. They can be both vertical and horizontal.3. It shows the relationship between a numerical variable and a categorical variable.4. Bar charts can also show big changes in data over time.5. Line graphs are useful for displaying smaller changes in a trend over time. Bar graphs are better for comparing larger changes or differences in data among groups.6. Bar graphs are an effective way to compare items between different groups.Types of bar chart- 1. Vertical 2. Horizontal3. StackedThe only difference in plotting a bar chart from plotting a line chart is you call **`plt.bar()`** instead of **`plt.plot()`**   Let's see how barplot looks !***Let's find the variation in mean visibility, aggregated by day of the week<jupyter_code>import matplotlib.pyplot as plt
import calendar

def bar_plot():
    weekly_data = weather_df.groupby(weather_df.index.dayofweek).mean()
    
    plt.bar(weekly_data.index, weekly_data['Visibility (km)'])

    plt.title('Visibility by week, 2012')
    plt.xlabel('Day of week')
    plt.ylabel('Visibility (km)')

    plt.xticks(weekly_data.index, calendar.day_abbr, rotation=45)

    plt.show()
bar_plot()<jupyter_output><empty_output><jupyter_text>Histogram***Some important characteristics of histogram are:1. It is a display of statistical information that uses bars to show the frequency of data items in successive numerical intervals of equal size.2. They can be both vertical and horizontal.3. It has only a numerical variable as input and shows its distribution.4. It divides up the range of possible values in a data set into classes or groups.5. Generally, a histogram will have bars of equal width, although this is not the case when class intervals vary in size.6. It is generally used when dealing with large data sets (greater than 100 observations).7. A histogram can also help detect any unusual observations (outliers) or any gaps in the data.8. In a histogram, it is the area of the bar that indicates the frequency of occurrences for each bin. Since a histogram is a plot of frequency(X) vs X, it only needs one series.<jupyter_code># Sample histogram 

x = np.arange(0, 10, 0.1)
y1 = (((x - 3) ** 3 ) - 100) + np.random.randint(-20, 20, size=len(x))

plt.hist(y1)
plt.show()<jupyter_output><empty_output><jupyter_text>Mini-Challenge - 4***Find the distribution of values in the Wind Speed column<jupyter_code>def hist_plot():
    '''Your code goes here'''
hist_plot()<jupyter_output><empty_output><jupyter_text>Box Plot***In descriptive statistics, a box plot is a method for graphically depicting groups of numerical data through their quartiles.Some important characteristics of a box plot are:1. A boxplot is a standardized way of displaying the distribution of data based on the five number summary: minimum, first quartile, median, third quartile, and maximum.2. Simplest possible box plot displays the full range of variation (from min to max), the likely range of variation (the IQR), and a typical value (the median). Not uncommonly real datasets will display surprisingly high maximums or surprisingly low minimums called  outliers. <NAME> has provided a precise definition for two types of outliers: (a) Outliers are either 3×IQR or more above the third quartile or 3×IQR or more below the first quartile. (b) Suspected outliers are are slightly more central versions of outliers: either 1.5×IQR or more above the third quartile or 1.5×IQR or lesser below the first quartile. Now, lets discuss some important definitions that we commonly encounter while generating a boxplot.**Median**-The median (middle quartile) marks the mid-point of the data and is shown by the line that divides the box into two parts. Half the scores are greater than or equal to this value and half are less.**Inter-quartile range**-The middle “box” represents the middle 50% of scores for the group. The range of scores from lower to upper quartile is referred to as the inter-quartile range. The middle 50% of scores fall within the inter-quartile range.**Upper quartile**-Seventy-five percent of the scores fall below the upper quartile.**Lower quartile**-Twenty-five percent of scores fall below the lower quartile.**Whiskers**-The upper and lower whiskers represent scores outside the middle 50%. Whiskers often (but not always) stretch over a wider range of scores than the middle quartile groups.<jupyter_code># Sample boxplot 

x= np.arange(0, 10, 0.1)
y = np.exp(x)

plt.boxplot(y)
plt.show()<jupyter_output><empty_output><jupyter_text>Mini-Challenge - 5***Draw a boxplot for wind speed<jupyter_code>def box_plot():
    '''Your code goes here'''
box_plot()<jupyter_output><empty_output><jupyter_text>Scatter Plot***Scatter plots are similar to line graphs in that they use horizontal and vertical axes to plot data points. However, they have a very specific purpose. Scatter plots show how much one variable is affected by another. The relationship between two variables is called their correlation .Some important characteristics of a scatter plot are:1. A scatter plot can indicate the presence or absence of an association or relationship between two variables.    * If some association or relationship exists, the data will tend to cluster on or around some line or      curve that cuts through the plotted points.    * If no association or relationship exists, the data will appear spread out.2. When some association or relationship exists between two variables, a scatter plot can indicate the direction of the relationship.    * When high values of one variable correspond to high values of another variable, and low values of one variable correspond to low values of another variable, we say the relationship is positive because there is a direct relationship between the two variables.    * When high values of one variable correspond to low values of another variable, and low values of one variable correspond to high values of another variable, we say the relationship is negative because there is an inverse relationship between the two variables.3. Calculating the correlation coefficient will give us a precise number and a scatter plot helps us find outliers, gain a more intuitive sense of how spread out the data is. 4. A scatter plot can be used to help determine if an association or relationship is linear or nonlinear.5. A scatter plot provides a visual revelation of outliers on dubiously measured or erroneously plotted data.  Spotting correlation in scatter plots![](../images/KssJXmts69TqMUTHuuFVDQFu.bmp)  Example![](../images/sample_scatterplot2.png)<jupyter_code># Sample scatter plot

x= np.arange(0, 10, 0.1)
y1 = (((x - 3) ** 3 ) - 100) + np.random.randint(-20, 20, size=len(x))
y2 = (((3 - x) ** 3 ) + 50) + np.random.randint(-20, 20, size=len(x))

plt.scatter(x, y1, c='r')
plt.scatter(x, y2, c='b')
plt.show()<jupyter_output><empty_output><jupyter_text>Mini-Challenge - 6***Are the temperature and pressure correlated, according to the data in the month of January? Look at it by generating a scatter plot<jupyter_code>jan_df = weather_df['2012-01']

def scatter_plot():
    '''Your code goes here'''
scatter_plot()<jupyter_output><empty_output><jupyter_text>Plot Customizations***Matplotlib being a low level library, provides a lot of ways to customize your plots. Some examples include:- Configuring the line (color, width, ticks, etc)- Configuring the axes (setting limits, layout, etc.)- Figure and Sub Plots: Multiple plots on one canvas- Configuring the figure size- Adding legends, etc.  Configuring the plotted line***It's easy to set colors, line types, etc. An example is below.<jupyter_code>fig, ax = plt.subplots()
ax.plot(x, x**2, 'b.-') # blue line with dots
ax.plot(x, x**2.5, 'g--') # green dashed line
ax.plot(x, x**3, c='r') # red line color
fig.show()<jupyter_output>/home/pranav/anaconda3/lib/python3.6/site-packages/matplotlib/figure.py:459: UserWarning: matplotlib is currently using a non-GUI backend, so cannot show the figure
  "matplotlib is currently using a non-GUI backend, "<jupyter_text>`.figure()` and `.axes()`*****Figure:** This object essentially returns a new figure to work with. Axes need to be added to the figure to plot on it. The Figure is the overall window or page that everything is drawn on. You can create multiple independent Figures. A Figure can have several other things in it(subtitle, legend, color bar etc).**Axes:** To the figure you add Axes. The Axes is the area on which the data is plotted with functions such as plot() and scatter() and that can have ticks, labels, etc. associated with it. This explains why Figures can contain multiple Axes.The methods on axes work the same way as **`.plot()`**<jupyter_code>def two_plots():
    x = np.array([0, 1, 2, 3, 4, 5])
    y = x ** 2

    # Create Figure (empty canvas)
    fig = plt.figure()

    # Add set of axes to figure
    axes1 = fig.add_axes([0.1, 0.1, 0.8, 0.8]) # main axes
    axes2 = fig.add_axes([0.2, 0.5, 0.4, 0.3]) # inset axes 
    # (0.2x left, 0.5x bottom) to (0.4x width, 0.3x height)

    # Larger Figure Axes 1
    axes1.plot(x, y, 'b')
    axes1.set_xlabel('X_label_axes1')
    axes1.set_ylabel('Y_label_axes1')
    axes1.set_title('Axes 1 Title')

    # Insert Figure Axes 2
    axes2.plot(y, x, 'r')
    axes2.set_xlabel('X_label_axes2')
    axes2.set_ylabel('Y_label_axes2')
    axes2.set_title('Axes 2 Title');    
two_plots()<jupyter_output><empty_output><jupyter_text>`.subplots()`***The plt.subplots() object will act as a more automatic axis manager and is used to creating multiple charts on the same figure. It is similar to `plt.figure()`, except use tuple unpacking to grab fig and axes.<jupyter_code># Canvas of 2 by 2 subplots
fig, axes = plt.subplots(nrows=2, ncols=2)

# axes is an array of shape (2, 2)<jupyter_output><empty_output><jupyter_text>Mini-Challenge - 7*** Create a figure with four subplots, and plot `y = x ** 2` on the first and last plotsMake the color red in the first subplot, and a green dashed line in the last<jupyter_code>def sub_plots_example():
    '''Your code goes here'''
sub_plots_example()<jupyter_output><empty_output><jupyter_text>`.xlim` & `.ylim`We can configure the ranges of the axes using the set_ylim and set_xlim methods in the axis object, or axis('tight') for automatically getting "tightly fitted" axes ranges.  `.tight_layout()`A common issue with matplotlib is overlapping subplots or figures. We can use **`fig.tight_layout()`** or **`plt.tight_layout()`** method, which automatically adjusts the positions of the axes on the figure canvas so that there is no overlapping content.  `.figsize`figsize is a tuple of the width and height of the figure in inches  `.legend()`You can use the label="label text" keyword argument when plots or other objects are added to the figure, and then using the legend method without arguments to add the legend to the figure.  `.savefig()`***Matplotlib can generate high-quality output in a number formats, including PNG, JPG, EPS, SVG, PGF and PDF.    fig.savefig("filename.png", dpi=200)   Seaborn***Seaborn is a Python visualization library based on matplotlib. It provides a high-level interface for drawing attractive statistical graphics.Understanding matplotlib was essential for understanding how to create plots, but we'll be using Seaborn for general data analysis.  Why? Because it's beautiful and easy!***Let's revisit a question we solved today: **What is the distribution of values in the Wind Speed column?**<jupyter_code>import seaborn as sns
sns.distplot(weather_df['Wind Spd (km/h)'], bins=25)<jupyter_output>/home/pranav/anaconda3/lib/python3.6/site-packages/matplotlib/axes/_axes.py:6462: UserWarning: The 'normed' kwarg is deprecated, and has been replaced by the 'density' kwarg.
  warnings.warn("The 'normed' kwarg is deprecated, and has been "