<jupyter_start><jupyter_text>Introduction to Pandas<NAME> (<EMAIL>)The latest version of this notebook is available at [https://github.com/Asterics2020-Obelics](https://github.com/Asterics2020-Obelics/School2019/tree/master/pandas)<jupyter_code>%matplotlib inline
import pandas as pd
import matplotlib as ml
import sys
plt = ml.pyplot
ml.rcParams['figure.figsize'] = (10.0, 5.0)

print("Python version: {0}\n"
      "Pandas version: {1}\n"
      "Matplotlib version: {2}"
      .format(sys.version, pd.__version__, ml.__version__))
from IPython.core.magic import register_line_magic

@register_line_magic
def shorterr(line):
    """Show only the exception message if one is raised."""
    try:
        output = eval(line)
    except Exception as e:
        print("\x1b[31m\x1b[1m{e.__class__.__name__}: {e}\x1b[0m".format(e=e))
    else:
        return output
    
del shorterr<jupyter_output><empty_output><jupyter_text>The basic data structures in Pandas  `DataFrame`<jupyter_code>data = {'a': [1, 2, 3],
        'b': [4.1, 5.2, 6.3],
        'c': ['foo', 'bar', 'baz'],
        'd': 42}
df = pd.DataFrame(data)
df
type(df)<jupyter_output><empty_output><jupyter_text>`Series`<jupyter_code>df['a']
type(df['a'])  
df['a'] * 23
np.cos(df['a'])
s = pd.Series(np.random.randint(0, 10, 5))
s
s.sort_values()  # indices are kept!
s * s.sort_values()  # and are used to match elements
s * s.sort_values().reset_index(drop=True)<jupyter_output><empty_output><jupyter_text>Examining a `DataFrame`<jupyter_code>df
df.dtypes
df.columns
df.shape<jupyter_output><empty_output><jupyter_text>Looking into the data<jupyter_code>df.head(2)
df.tail(2)
df.describe()<jupyter_output><empty_output><jupyter_text>Indexing and SlicingThere are different ways to index/slice data in pandas, which is a bit confusing at first.<jupyter_code>df.loc
df.iloc<jupyter_output><empty_output><jupyter_text>Using `.loc[]`This one treats the input as label or "row-name".<jupyter_code>df.loc[2]
df['b'].loc[2]
%shorterr df.loc[-1]<jupyter_output>[31m[1mKeyError: -1[0m<jupyter_text>Accessing multiple rows/columns<jupyter_code>df.loc[[1, 2], ['b', 'd']]
df.loc[1:3, ['a']]<jupyter_output><empty_output><jupyter_text>Using `.iloc[]`<jupyter_code>df.iloc[2]
df.iloc[-1]<jupyter_output><empty_output><jupyter_text>Grouped operations<jupyter_code>df = pd.DataFrame({'location' : ['Italy', 'France', 'Italy', 'Italy',
                                 'France', 'Italy', 'France', 'France'],
                   'detector' : ['ARCA_DU1', 'ORCA_DU2', 'ARCA_DU3', 'ARCA_DU4',
                                 'ANTARES', 'NEMO', 'ORCA_DU1', 'ORCA_DU5'],
                   'x' : np.random.randn(8),
                   'y' : np.random.randn(8)})
df
df.groupby(['location']).groups
df.groupby(['location']).get_group('France')
df = pd.DataFrame({
    'event_id': [1, 1, 1, 2, 2, 3, 3, 3, 3],
    'n_hits': [23, 13, 14, 44, 23, 45, 1, 2, 4]
})
df
df.groupby('event_id')
for name, group_data in df.groupby('event_id'):
    print("Group name: {0}".format(name))
    print(group_data, "\n")
df.groupby('event_id').aggregate([np.mean, np.median, sum, min, max])<jupyter_output><empty_output><jupyter_text>Plotting<jupyter_code>df = pd.DataFrame({'foo': np.cumsum(np.random.random(1000) - 0.5),
                   'bar': np.cumsum(np.random.random(1000) - 0.5),
                   'narf': np.cumsum(np.random.random(1000) - 0.5)})
df.plot();
df.hist();
plt.scatter(df['foo'], df['bar']);<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup