# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     name: python3
# ---

# + [markdown] id="view-in-github" colab_type="text"
# <a href="https://colab.research.google.com/github/DanRHowarth/Artificial-Intelligence-Cloud-and-Edge-Implementations/blob/master/Oxford_Numpy.ipynb" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/></a>

# + [markdown] id="3KIiXVqiex-8" colab_type="text"
# # Numpy Tutorial

# + [markdown] id="5QVOlDnXe7hK" colab_type="text"
# ### This tutorial covers the following:
#  1. Overview of Numpy 
#  2. Introduction to Numpy Arrays
#    *  Creating arrays 
#    *  Locating elements of an array 
#    *  Altering properties of an array, including it's `shape` and `datatype`
#  3. Performing operations on Arrays
#   * Arithmetic operations
#   * Boolean Operations 
#   * Universal functions
#  
# #### Exercises
#  * Each section will have an exercise to help reinforce your learning. We suggest you:
#    * Write out each line of code by hand (rather than copy and paste it from the relevant example) - this will improve your understanding of code syntax
#    * Write out, above each line of code, an explanation as to what the code, using a comment `#` - this will improve your understanding of how the code works

# + [markdown] id="jxFeWxklf9Du" colab_type="text"
# ### 1. Overview of Numpy
#
#
# * [Python for Data Analysis](https://www.amazon.co.uk/Python-Data-Analysis-Wrangling-IPython-ebook/dp/B075X4LT6K/ref=sr_1_1?s=digital-text&ie=UTF8&qid=1540755909&sr=1-1&keywords=python+for+data+analysis ) sets out the following benefits of the Numpy library: 
#
#   * the `ndarray`, an efficient multidimensional array providing fast array-oriented arithmetic operations and flexible broadcasting capabilities.
#
#   * Mathematical functions for fast operations on entire arrays of data without having to write loops.
#
#   * Tools for reading/writing array data to disk and working with memory-mapped files.
#
#   * Linear algebra, random number generation, and Fourier transform capabilities.
#
#
# * It is designed for efficiency on large arrays of data because it internally stores data in a contiguous block of memory, independent of other built-in Python objects. Numpy operations perform complex computations on entire arrays without the need for Python for loops.
#
# * See *Python for Data Analysis*, or *[Python Data Science Handbook](https://jakevdp.github.io/PythonDataScienceHandbook/)* (free) for more background on the library and its benefits.
#

# + id="qDS8J9qdhmAJ" colab_type="code" colab={}
## this simple example will demonstrate the difference in efficiency between numpy and python 

# it is convention to import numpy this way
import numpy as np

# we create our first numpy array - we will come back to this type of operation later 
my_arr = np.arange(1000000)

# create a python list with the same values, but this is obviously not stored as a numpy array
my_list = list(range(1000000))

# + id="ijRQnOpgh_df" colab_type="code" colab={}
## let's time how long an operation takes in numpy
# %time my_arr2 = my_arr * 2

# + id="xJRUSo4FPcmC" colab_type="code" colab={}
# and compare it to the same operation in python
# %time my_list2 = [x * 2 for x in my_list]

# + [markdown] id="2zqt3F1GiEi6" colab_type="text"
# ### 2. Introduction to Numpy Arrays
#
# * The *N-dimensional array object*, or `ndarray`, is a fast, flexible container for large datasets in Python. Arrays enable you to perform mathematical operations on whole blocks of data using similar syntax to the equivalent operations between scalar elements.
# * In this section, we will cover:
#   * How to create arrays
#   * How to access different parts of the array 
#   * The properties of arrays, and how to modify them.

# + [markdown] id="LVNOHN7hifnc" colab_type="text"
# #### 2.1 Creating arrays
# * There are a number of different ways to create an array. We cover some of the main ways here, including:
#   * passing in values to the parameters of `np.array`
#   * `zeroes`: an array of zero values 
#   * `ones`: an array of all ones 
#   * `full`: an array of a specified constant value 
#   * `random`: an array of random values
#   * `eye`: an array with ones on the diagonal and zeroes otherwise
#   * converting from a python list
#   * `arange`: an array with values within the range specified 
#   * `linspace`: an array of equally spaced values within the upper and lower bounds specified
#   * `from pandas`
#  
#

# + id="5onuLDx6ZLLe" colab_type="code" colab={}
# Create an array by passing in values  
a = np.array([0, 1, 2]) 
a

# + id="5UYT_-vyh6Mn" colab_type="code" colab={}
# Create a 2D array
b = np.array([[0,1,2],[3,4,5]])  
b

# + id="3vb8RDjrwClW" colab_type="code" colab={}
# Create an array of all zeros - with three rows of three columns each 
c = np.zeros((3,3)) 
c

# + id="PzuRFsCAwYUx" colab_type="code" colab={}
# Create a 2x2 array of all ones
d = np.ones((2,2)) 
d

# + id="XxYqqjMkxJYI" colab_type="code" colab={}
# Create a 3x3 constant array
e = np.full((3,3), 7) 
e

# + id="REEwBbVBXx1X" colab_type="code" colab={}
# Create a 3x3 array filled with random values
f = np.random.random((3,3)) 
f

# + id="zNisQgDpYMPD" colab_type="code" colab={}
# Create a 3x3 matrix with ones on the diagonal 
# np.identity also returns the same result 
g = np.eye(3)    
g

# + id="GRs_J9kzYTuq" colab_type="code" colab={}
# convert list to array
h = list((2, 3, 1, 0))
h = np.array(h) 
h

# + id="Eka1O6M6Yotd" colab_type="code" colab={}
# arange() will create arrays with regularly incrementing values
i = np.arange(20)
i

# + id="PNZkdxria_7v" colab_type="code" colab={}
# linspace() will create arrays with a specified number of items which are 
# spaced equally between the specified beginning and end values
j = np.linspace(2., 4., 5)
j

# + [markdown] id="fTAItFP0XvDi" colab_type="text"
# #### EXERCISE 2.1: Creating an array
# * Create an array using the `np.arange()` function. 
# * Use the [documentation](https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.arange.html) to add start, stop and step parameters 

# + id="uKy7AZ_XYuP5" colab_type="code" colab={}
## EXERCISE CODE HERE



# + [markdown] id="z0YAQvcSSV5M" colab_type="text"
# #### 2.2 Accessing arrays
#
# * The elements of numpy arrays can be accessed using their index. Numpy (and Pandas) refers to elements *along* the rows as being on `axis 1`, and elements *down* the rows as being on `axis 0`. This is set out below:
#
# ![alt text](https://www.safaribooksonline.com/library/view/python-for-data/9781491957653/assets/pyda_0401.png)
#
# * Indexing a 1D array is relatively straightforward as the index values are all on the same axis.
# * Indexing a 2D array (or higher dimensions) is slightly trickier, as we need to access `axis 0` first, and then `axis 1`.
# * The examples below should make this a little clearer. 
#

# + id="LzmzbGQCUaTt" colab_type="code" colab={}
# lets go back to on the first array we created
print("1D array:\n", a)

# we can access the shape of the array like this
print("\nDimensions of array: ", a.shape)

# and call the individual elements of the array using the element's index
print("\nFirst element of array: ", a[0])
print("\nSecond element of array: ", a[1])
print("\nThird element of array: ", a[2])

# + id="HCSWw_djU_VN" colab_type="code" colab={}
# lets compare this to a 2D array
print("2D array:\n", b)

# and contrast the shape of this array with the shape of a
print("\nDimensions of 2D array: ", b.shape)

# pause and make sure you understand how the values are being accessed
print("\nArray element [0,0]:", b[0, 0])
print("\nArray elements [1,2] [0,1] [1,0]: ", b[1, 2], b[0, 1], b[1, 0])

# + id="XAS6j6-omJ58" colab_type="code" colab={}
# We can also slice arrays, using the following approach:

# a[start:end] # items start through end-1
# a[start:]    # items start through the rest of the array
# a[:end]      # items from the beginning through end-1
# a[:]         # a copy of the whole array


# The key point to remember is that the :end value represents the first value that is not in the selected slice. So, the difference 
# beween end and start is the number of elements selected (if step is 1, the default).
# The other feature is that start or end may be a negative number, which means it counts from the end of the array 
# instead of the beginning. So:

# a[-1]    # last item in the array
# a[-2:]   # last two items in the array
# a[:-2]   # everything except the last two items

# create a new array
k = np.array([1,2,3,4])

# access the first two elements
k[0:2]

# + id="PIW3JAOwjsjA" colab_type="code" colab={}
# higher dimension slicing 
j = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])

# access one row of the array
j[2]

# + id="aq15Uy1RnmCY" colab_type="code" colab={}
# access values within one row - note that we can also express this as j[2][:2]
j[2,:2]

# + id="fuvVu0SxoLNX" colab_type="code" colab={}
# access all the rows (axis 0) and two of the columns (axis 1)
j[:,:2]

# + [markdown] id="uRbSFcl7r6ot" colab_type="text"
# #### EXERCISE 2.2: Accessing Arrays 
# * Create a 2D array using a different using one of the methods from section 2.1
# * Create a new variable containing a slice of the original array
# * Replace one of the values of the sliced array using the following template:
#   * `array_slice[index_value] = new_value`
# * Call the original 2D array and notice if the new_value has been updated  

# + id="z-vUF1Fus2kC" colab_type="code" colab={}
## EXERCISE CODE HERE



# + [markdown] id="AUuFjyldSYsV" colab_type="text"
# #### 2.3 Other properties of arrays
# * An ndarray is a generic multidimensional container for homogeneous data. Every array has a shape (which we discoverd above), a tuple indicating the size of each dimension, and a dtype, an object describing the data type of the array
# * We can access information on these properties, and alter them, as we will run through below. Operations to reshape the data are common in Deep Learning, and some of the methods used are covered here. 
#

# + id="DjYaanYkYM2v" colab_type="code" colab={}
# One important thing to note that when we can change an element of the array *inplace*
print ("a before changing an element value in place:\n", a)

# change the value at the first elements 
a[0] = 5                 

# and show the result 
print ("\na after changing an element value in place:\n", a)

# + id="Bz7WHxLEUDPV" colab_type="code" colab={}
# we can access the data type of an array using '.dtype'
print("Data type of a: ", a.dtype)

# + id="9NgJDlen4KA2" colab_type="code" colab={}
# note that numpy automatically assigned us the data type - but we can specify it
a_float = np.array(a, dtype=np.float64)

# print the datatype
print(a_float.dtype)

# and we can see that the array has a decimal point after the number 
a_float

# + id="tLyzikOU4TuD" colab_type="code" colab={}
# we can explicitly convert or cast an array from one dtype to another using ndarray’s astype method
a_int32 = a_float.astype(np.int32)

# print the dtype
print(a_int32.dtype)

# and we can see that the numbers have been converted back to integers 
a_int32

# + id="DyYiJpyE4jcK" colab_type="code" colab={}
# Converting from strings to numbers can also be useful
numeric_strings = np.array(['1.25', '-9.6', '42'], dtype=np.string_)

# here we use 'float', we could use np.float
numeric_strings.astype(float)

# + id="eGmL0LMV36MX" colab_type="code" colab={}
# earlier we looked at the .shape attribute. Lets now look at how to reshape numpy arrays 
print(j.shape)

# reshape - noting that the new shape must 'fit' the same number of values 
k = j.reshape(6,2)
k

# + id="eC1hb5c4wwY0" colab_type="code" colab={}
# passing -1 as the second parameter means numpy will infer the size of the second value
# this is useful if you have large array 

l = np.arange(1500).reshape(2,-1)
l.shape

# + id="O4kMKr1xxjv-" colab_type="code" colab={}
# we can join two different arrays either vertically (axis 0) or horizontally (axis 1)
m = np.array([np.arange(0,3),np.arange(3,6), np.arange(6,9)])
n = np.random.random((3,3))

# + id="NhvMJjNBzD7Y" colab_type="code" colab={}
# join - 'stack' vertically - note it takes a tuple as an argument 
o = np.vstack((m,n))
o

# + id="q3VUX0oazxBm" colab_type="code" colab={}
# stack horizontally
p = np.hstack((m,n))
p

# + id="3DTa6PXmz9H0" colab_type="code" colab={}
# we can then use .ravel() or .flatten() to return 1D arrays
q = o.ravel()
q

# + id="12zseK9m0HMz" colab_type="code" colab={}
# flatten returns a copy of the array, ravel does not - otherwise they perform the same
r = p.flatten()
r

# + [markdown] id="38D2S3Lmsg8R" colab_type="text"
# #### EXERCISE 2.3: Properties of Arrays
# * Create two 2D arrays
# * Reshape one or both of them 
# * Stack them together
# * Convert this array to a 1D array 

# + id="9D5cbpgTshIf" colab_type="code" colab={}
## EXERCISE CODE HERE 



# + [markdown] id="YeOZr6HjsUCm" colab_type="text"
# ### 3. Performing Operations on Arrays
#
# * We can perform a range of operations on arrays, including: 
#   * Arithmetic operations between arrays
#   * Boolean operations 
#   * Universal functions
#   * Mathematical and Statistical operations 
# * We will cover examples of these below; these are not the only operations that can be performed but will provide you with a good toolkit for a range of operations.

# + id="7jXNSx_X0s3i" colab_type="code" colab={}
# Any arithmetic operations between equal-size arrays applies the operation element-wise:
r + q

# + id="Wg28jq-t43jS" colab_type="code" colab={}
# we can also use universal functions (ufuncs), a function that performs element-wise operations on data in ndarrays
np.add(m,n)

# + id="QFAHuQBV0uEW" colab_type="code" colab={}
# the same applies to other arithmetic functions
m * n 

# + id="MpSgqcv96nu6" colab_type="code" colab={}
# with the ufunc equivalent 
np.multiply(r,q)

# + id="vJaNh7Bq41SO" colab_type="code" colab={}
# the same principles apply to subtraction and division 
# Arithmetic operations with scalars apply the scalar argument to each element in the array
1 / a

# + id="z6ibKEdU7Js4" colab_type="code" colab={}
# another example, raising a to the power 0.5
a ** 0.5

# + id="2TqrDJc97WmY" colab_type="code" colab={}
# here are a couple more examples 
np.exp(a)

# + id="0YyoDhWEIo8i" colab_type="code" colab={}
# Elementwise square root
np.sqrt(r)

# + id="v_ubq_FM99rD" colab_type="code" colab={}
# create two arrays
s = np.random.randn(8)
t = np.random.randn(8)

# compares the arrays 
np.maximum(s, t)

# + id="VAcLTILiAP7y" colab_type="code" colab={}
# statistical methods are available, such as mean(), sum() and cumsum() - cumulative sum()
a.mean()

# + id="WsmOzJMf-2yN" colab_type="code" colab={}
# this format also works for all the statistical methods
np.sum(a)

# + id="iW9gty247Gbb" colab_type="code" colab={}
# we can use boolean operations to filter valies
u = np.array([[1,2], [3, 4], [5, 6]])

# first find the elements of a that are bigger than 2
print("Find elements of array bigger than 2")
print((u > 2))

# + id="mpd_wcr07kpS" colab_type="code" colab={}
# then use the mask to get the actual value
print("Values of array elements bigger than 2")
print(u[u > 2])

# + [markdown] id="kLV1BiNmuyRD" colab_type="text"
# #### EXERCISE: Performing Operations on Arrays
# * Create two arrays
# * Perform an operation between the two arrays
# * Apply a scalar operation to one of the arrays
# * Use boolean indexing to select a subset of one of the arrays

# + id="XH8LnNiHu42u" colab_type="code" colab={}
## EXERCISE CODE GOES HERE 




# + [markdown] id="rX3PrnWBs411" colab_type="text"
# ### Review and Further Reading
# * We now have a good background in numpy, in particular what the library is and the benefit of vectorized operations over the approach taken by python on its own; how to create and locate values in arrays; how to perform operations on arrays; and how to broadcast between different numpy arrays.
# * There is of course much more to learn. We would suggest looking next at:
#   * `fancy indexing`
#   * `Broadcasting` 
#   * Exploring `ufuncs`, mathematical and statistical operations in greater detail
#   * Array-Oriented Programming with Arrays
#   * Expressing Conditional Logic as Array Operations
#   * Linear Algebra
#
#
#
# * Here are some follow-up resources:
#   * http://cs231n.github.io/python-numpy-tutorial/
#   * the books linked to at the start of the tutorial
#
#

# + id="IdNPePWB_fcl" colab_type="code" colab={}


StarcoderdataJupyterScriptsDedupFiltered