<jupyter_start><jupyter_text>Fixed-Type Arrays in PythonPython offers several different options for storing data in efficient, fixed-type data buffers.The built-in ``array`` module (available since Python 3.3) can be used to create dense arrays of a uniform type:<jupyter_code>import array
L = list(range(10))
A = array.array('i', L)
A<jupyter_output><empty_output><jupyter_text>Here ``'i'`` is a type code indicating the contents are integers.Much more useful, however, is the ``ndarray`` object of the NumPy package.While Python's ``array`` object provides efficient storage of array-based data, NumPy adds to this efficient *operations* on that data.We will explore these operations in later sections; here we'll demonstrate several ways of creating a NumPy array.We'll start with the standard NumPy import, under the alias ``np``:<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>Creating Arrays from Python ListsFirst, we can use ``np.array`` to create arrays from Python lists:<jupyter_code># integer array:
np.array([1, 4, 2, 5, 3])<jupyter_output><empty_output><jupyter_text>Remember that unlike Python lists, NumPy is constrained to arrays that all contain the same type.If types do not match, NumPy will upcast if possible (here, integers are up-cast to floating point):<jupyter_code>np.array([3.14, 4, 2, 3])<jupyter_output><empty_output><jupyter_text>If we want to explicitly set the data type of the resulting array, we can use the ``dtype`` keyword:<jupyter_code>np.array([1, 2, 3, 4], dtype='float32')<jupyter_output><empty_output><jupyter_text>Finally, unlike Python lists, NumPy arrays can explicitly be multi-dimensional; here's one way of initializing a multidimensional array using a list of lists:<jupyter_code># nested lists result in multi-dimensional arrays
np.array([range(i, i + 3) for i in [2, 4, 6]])<jupyter_output><empty_output><jupyter_text>The inner lists are treated as rows of the resulting two-dimensional array.  Creating Arrays from ScratchEspecially for larger arrays, it is more efficient to create arrays from scratch using routines built into NumPy.Here are several examples:<jupyter_code># Create a length-10 integer array filled with zeros
np.zeros(10, dtype=int)
# Create a 3x5 floating-point array filled with ones
np.ones((3, 5), dtype=float)
# Create a 3x5 array filled with 3.14
np.full((3, 5), 3.14)
# Create an array filled with a linear sequence
# Starting at 0, ending at 20, stepping by 2
# (this is similar to the built-in range() function)
np.arange(0, 20, 2)
# Create an array of five values evenly spaced between 0 and 1
np.linspace(0, 1, 5)
# Create a 3x3 array of uniformly distributed
# random values between 0 and 1
np.random.random((3, 3))
# Create a 3x3 array of normally distributed random values
# with mean 0 and standard deviation 1
np.random.normal(0, 1, (3, 3))
# Create a 3x3 array of random integers in the interval [0, 10)
np.random.randint(0, 10, (3, 3))
# Create a 3x3 identity matrix
np.eye(3)
# Create an uninitialized array of three integers
# The values will be whatever happens to already exist at that memory location
np.empty(3)<jupyter_output><empty_output><jupyter_text>NumPy Standard Data TypesNumPy arrays contain values of a single type, so it is important to have detailed knowledge of those types and their limitations.Because NumPy is built in C, the types will be familiar to users of C, Fortran, and other related languages.The standard NumPy data types are listed in the following table.Note that when constructing an array, they can be specified using a string:```pythonnp.zeros(10, dtype='int16')```Or using the associated NumPy object:```pythonnp.zeros(10, dtype=np.int16)``` | Data type	    | Description ||---------------|-------------|| ``bool_``     | Boolean (True or False) stored as a byte || ``int_``      | Default integer type (same as C ``long``; normally either ``int64`` or ``int32``)| | ``intc``      | Identical to C ``int`` (normally ``int32`` or ``int64``)| | ``intp``      | Integer used for indexing (same as C ``ssize_t``; normally either ``int32`` or ``int64``)| | ``int8``      | Byte (-128 to 127)| | ``int16``     | Integer (-32768 to 32767)|| ``int32``     | Integer (-2147483648 to 2147483647)|| ``int64``     | Integer (-9223372036854775808 to 9223372036854775807)| | ``uint8``     | Unsigned integer (0 to 255)| | ``uint16``    | Unsigned integer (0 to 65535)| | ``uint32``    | Unsigned integer (0 to 4294967295)| | ``uint64``    | Unsigned integer (0 to 18446744073709551615)| | ``float_``    | Shorthand for ``float64``.| | ``float16``   | Half precision float: sign bit, 5 bits exponent, 10 bits mantissa| | ``float32``   | Single precision float: sign bit, 8 bits exponent, 23 bits mantissa| | ``float64``   | Double precision float: sign bit, 11 bits exponent, 52 bits mantissa| | ``complex_``  | Shorthand for ``complex128``.| | ``complex64`` | Complex number, represented by two 32-bit floats| | ``complex128``| Complex number, represented by two 64-bit floats|  More advanced type specification is possible, such as specifying big or little endian numbers; for more information, refer to the [NumPy documentation](http://numpy.org/).NumPy also supports compound data types, which will be covered in [Structured Data: NumPy's Structured Arrays](02.09-Structured-Data-NumPy.ipynb).  The Basics of NumPy Arrays Data manipulation in Python is nearly synonymous with NumPy array manipulation: even newer tools like Pandas are built around the NumPy array.This section will present several examples of using NumPy array manipulation to access data and subarrays, and to split, reshape, and join the arrays.While the types of operations shown here may seem a bit dry and pedantic, they comprise the building blocks of many other examples.Get to know them well!We'll cover a few categories of basic array manipulations here:- *Attributes of arrays*: Determining the size, shape, memory consumption, and data types of arrays- *Indexing of arrays*: Getting and setting the value of individual array elements- *Slicing of arrays*: Getting and setting smaller subarrays within a larger array- *Reshaping of arrays*: Changing the shape of a given array- *Joining and splitting of arrays*: Combining multiple arrays into one, and splitting one array into many  NumPy Array Attributes First let's discuss some useful array attributes.We'll start by defining three random arrays, a one-dimensional, two-dimensional, and three-dimensional array.We'll use NumPy's random number generator, which we will *seed* with a set value in order to ensure that the same random arrays are generated each time this code is run:<jupyter_code>import numpy as np
np.random.seed(0)  # seed for reproducibility

x1 = np.random.randint(10, size=6)  # One-dimensional array
x2 = np.random.randint(10, size=(3, 4))  # Two-dimensional array
x3 = np.random.randint(10, size=(3, 4, 5))  # Three-dimensional array<jupyter_output><empty_output><jupyter_text>Each array has attributes ``ndim`` (the number of dimensions), ``shape`` (the size of each dimension), and ``size`` (the total size of the array):<jupyter_code>print("x3 ndim: ", x3.ndim)
print("x3 shape:", x3.shape)
print("x3 size: ", x3.size)<jupyter_output>x3 ndim:  3
x3 shape: (3, 4, 5)
x3 size:  60<jupyter_text>Another useful attribute is the ``dtype``, the data type of the array:<jupyter_code>print("dtype:", x3.dtype)<jupyter_output>dtype: int64<jupyter_text>Other attributes include ``itemsize``, which lists the size (in bytes) of each array element, and ``nbytes``, which lists the total size (in bytes) of the array:<jupyter_code>print("itemsize:", x3.itemsize, "bytes")
print("nbytes:", x3.nbytes, "bytes")<jupyter_output>itemsize: 8 bytes
nbytes: 480 bytes<jupyter_text>In general, we expect that ``nbytes`` is equal to ``itemsize`` times ``size``.  Array Indexing: Accessing Single Elements If you are familiar with Python's standard list indexing, indexing in NumPy will feel quite familiar.In a one-dimensional array, the $i^{th}$ value (counting from zero) can be accessed by specifying the desired index in square brackets, just as with Python lists:<jupyter_code>x1
x1[0]
x1[4]<jupyter_output><empty_output><jupyter_text>To index from the end of the array, you can use negative indices:<jupyter_code>x1[-1]
x1[-2]<jupyter_output><empty_output><jupyter_text>In a multi-dimensional array, items can be accessed using a comma-separated tuple of indices:<jupyter_code>x2
x2[0, 0]
x2[2, 0]
x2[2, -1]<jupyter_output><empty_output><jupyter_text>Values can also be modified using any of the above index notation:<jupyter_code>x2[0, 0] = 12
x2<jupyter_output><empty_output><jupyter_text>Keep in mind that, unlike Python lists, NumPy arrays have a fixed type.This means, for example, that if you attempt to insert a floating-point value to an integer array, the value will be silently truncated. Don't be caught unaware by this behavior!<jupyter_code>x1[0] = 3.14159  # this will be truncated!
x1<jupyter_output><empty_output><jupyter_text>Array Slicing: Accessing Subarrays Just as we can use square brackets to access individual array elements, we can also use them to access subarrays with the *slice* notation, marked by the colon (``:``) character.The NumPy slicing syntax follows that of the standard Python list; to access a slice of an array ``x``, use this:``` pythonx[start:stop:step]```If any of these are unspecified, they default to the values ``start=0``, ``stop=``*``size of dimension``*, ``step=1``.We'll take a look at accessing sub-arrays in one dimension and in multiple dimensions.  One-dimensional subarrays<jupyter_code>x = np.arange(10)
x
x[:5]  # first five elements
x[5:]  # elements after index 5
x[4:7]  # middle sub-array
x[::2]  # every other element
x[1::2]  # every other element, starting at index 1<jupyter_output><empty_output><jupyter_text>A potentially confusing case is when the ``step`` value is negative.In this case, the defaults for ``start`` and ``stop`` are swapped.This becomes a convenient way to reverse an array:<jupyter_code>x[::-1]  # all elements, reversed
x[5::-2]  # reversed every other from index 5<jupyter_output><empty_output><jupyter_text>Multi-dimensional subarraysMulti-dimensional slices work in the same way, with multiple slices separated by commas.For example:<jupyter_code>x2
x2[:2, :3]  # two rows, three columns
x2[:3, ::2]  # all rows, every other column<jupyter_output><empty_output><jupyter_text>Finally, subarray dimensions can even be reversed together:<jupyter_code>x2[::-1, ::-1]<jupyter_output><empty_output><jupyter_text>Accessing array rows and columnsOne commonly needed routine is accessing of single rows or columns of an array.This can be done by combining indexing and slicing, using an empty slice marked by a single colon (``:``):<jupyter_code>print(x2[:, 0])  # first column of x2
print(x2[0, :])  # first row of x2<jupyter_output>[12  5  2  4]<jupyter_text>In the case of row access, the empty slice can be omitted for a more compact syntax:<jupyter_code>print(x2[0])  # equivalent to x2[0, :]<jupyter_output>[12  5  2  4]<jupyter_text>Subarrays as no-copy viewsOne important–and extremely useful–thing to know about array slices is that they return *views* rather than *copies* of the array data.This is one area in which NumPy array slicing differs from Python list slicing: in lists, slices will be copies.Consider our two-dimensional array from before:<jupyter_code>print(x2)<jupyter_output>[[12  5  2  4]
 [ 7  6  8  8]
 [ 1  6  7  7]]<jupyter_text>Let's extract a $2 \times 2$ subarray from this:<jupyter_code>x2_sub = x2[:2, :2]
print(x2_sub)<jupyter_output>[[12  5]
 [ 7  6]]<jupyter_text>Now if we modify this subarray, we'll see that the original array is changed! Observe:<jupyter_code>x2_sub[0, 0] = 99
print(x2_sub)
print(x2)<jupyter_output>[[99  5  2  4]
 [ 7  6  8  8]
 [ 1  6  7  7]]<jupyter_text>This default behavior is actually quite useful: it means that when we work with large datasets, we can access and process pieces of these datasets without the need to copy the underlying data buffer.  Creating copies of arraysDespite the nice features of array views, it is sometimes useful to instead explicitly copy the data within an array or a subarray. This can be most easily done with the ``copy()`` method:<jupyter_code>x2_sub_copy = x2[:2, :2].copy()
print(x2_sub_copy)<jupyter_output>[[99  5]
 [ 7  6]]<jupyter_text>If we now modify this subarray, the original array is not touched:<jupyter_code>x2_sub_copy[0, 0] = 42
print(x2_sub_copy)
print(x2)<jupyter_output>[[99  5  2  4]
 [ 7  6  8  8]
 [ 1  6  7  7]]<jupyter_text>Reshaping of ArraysAnother useful type of operation is reshaping of arrays.The most flexible way of doing this is with the ``reshape`` method.For example, if you want to put the numbers 1 through 9 in a $3 \times 3$ grid, you can do the following:<jupyter_code>grid = np.arange(1, 10).reshape((3, 3))
print(grid)<jupyter_output>[[1 2 3]
 [4 5 6]
 [7 8 9]]<jupyter_text>Note that for this to work, the size of the initial array must match the size of the reshaped array. Where possible, the ``reshape`` method will use a no-copy view of the initial array, but with non-contiguous memory buffers this is not always the case.Another common reshaping pattern is the conversion of a one-dimensional array into a two-dimensional row or column matrix.This can be done with the ``reshape`` method, or more easily done by making use of the ``newaxis`` keyword within a slice operation:<jupyter_code>x = np.array([1, 2, 3])

# row vector via reshape
x.reshape((1, 3))
# row vector via newaxis
x[np.newaxis, :]
# column vector via reshape
x.reshape((3, 1))
# column vector via newaxis
x[:, np.newaxis]<jupyter_output><empty_output><jupyter_text>We will see this type of transformation often.  Array Concatenation and SplittingAll of the preceding routines worked on single arrays. It's also possible to combine multiple arrays into one, and to conversely split a single array into multiple arrays. We'll take a look at those operations here.  Concatenation of arraysConcatenation, or joining of two arrays in NumPy, is primarily accomplished using the routines ``np.concatenate``, ``np.vstack``, and ``np.hstack``.``np.concatenate`` takes a tuple or list of arrays as its first argument, as we can see here:<jupyter_code>x = np.array([1, 2, 3])
y = np.array([3, 2, 1])
np.concatenate([x, y])<jupyter_output><empty_output><jupyter_text>You can also concatenate more than two arrays at once:<jupyter_code>z = [99, 99, 99]
print(np.concatenate([x, y, z]))<jupyter_output>[ 1  2  3  3  2  1 99 99 99]<jupyter_text>It can also be used for two-dimensional arrays:<jupyter_code>grid = np.array([[1, 2, 3],
                 [4, 5, 6]])
# concatenate along the first axis
np.concatenate([grid, grid])
# concatenate along the second axis (zero-indexed)
np.concatenate([grid, grid], axis=1)<jupyter_output><empty_output><jupyter_text>For working with arrays of mixed dimensions, it can be clearer to use the ``np.vstack`` (vertical stack) and ``np.hstack`` (horizontal stack) functions:<jupyter_code>x = np.array([1, 2, 3])
grid = np.array([[9, 8, 7],
                 [6, 5, 4]])

# vertically stack the arrays
np.vstack([x, grid])
# horizontally stack the arrays
y = np.array([[99],
              [99]])
np.hstack([grid, y])<jupyter_output><empty_output><jupyter_text>Similary, ``np.dstack`` will stack arrays along the third axis.  Splitting of arraysThe opposite of concatenation is splitting, which is implemented by the functions ``np.split``, ``np.hsplit``, and ``np.vsplit``.  For each of these, we can pass a list of indices giving the split points:<jupyter_code>x = [1, 2, 3, 99, 99, 3, 2, 1]
x1, x2, x3 = np.split(x, [3, 5])
print(x1, x2, x3)<jupyter_output>[1 2 3] [99 99] [3 2 1]<jupyter_text>Notice that _N_ split-points, leads to *N + 1* subarrays.The related functions ``np.hsplit`` and ``np.vsplit`` are similar:<jupyter_code>grid = np.arange(16).reshape((4, 4))
grid
upper, lower = np.vsplit(grid, [2])
print(upper)
print(lower)
left, right = np.hsplit(grid, [2])
print(left)
print(right)<jupyter_output>[[ 0  1]
 [ 4  5]
 [ 8  9]
 [12 13]]
[[ 2  3]
 [ 6  7]
 [10 11]
 [14 15]]<jupyter_text>Similarly, ``np.dsplit`` will split arrays along the third axis.  Computation on NumPy Arrays: Universal Functions Up until now, we have been discussing some of the basic nuts and bolts of NumPy; in the next few sections, we will dive into the reasons that NumPy is so important in the Python data science world.Namely, it provides an easy and flexible interface to optimized computation with arrays of data.Computation on NumPy arrays can be very fast, or it can be very slow.The key to making it fast is to use *vectorized* operations, generally implemented through NumPy's *universal functions* (ufuncs).This section motivates the need for NumPy's ufuncs, which can be used to make repeated calculations on array elements much more efficient.It then introduces many of the most common and useful arithmetic ufuncs available in the NumPy package.  The Slowness of LoopsPython's default implementation (known as CPython) does some operations very slowly.This is in part due to the dynamic, interpreted nature of the language: the fact that types are flexible, so that sequences of operations cannot be compiled down to efficient machine code as in languages like C and Fortran.Recently there have been various attempts to address this weakness: well-known examples are the [PyPy](http://pypy.org/) project, a just-in-time compiled implementation of Python; the [Cython](http://cython.org) project, which converts Python code to compilable C code; and the [Numba](http://numba.pydata.org/) project, which converts snippets of Python code to fast LLVM bytecode.Each of these has its strengths and weaknesses, but it is safe to say that none of the three approaches has yet surpassed the reach and popularity of the standard CPython engine.The relative sluggishness of Python generally manifests itself in situations where many small operations are being repeated – for instance looping over arrays to operate on each element.For example, imagine we have an array of values and we'd like to compute the reciprocal of each.A straightforward approach might look like this:<jupyter_code>import numpy as np
np.random.seed(0)

def compute_reciprocals(values):
    output = np.empty(len(values))
    for i in range(len(values)):
        output[i] = 1.0 / values[i]
    return output
        
values = np.random.randint(1, 10, size=5)
compute_reciprocals(values)<jupyter_output><empty_output><jupyter_text>This implementation probably feels fairly natural to someone from, say, a C or Java background.But if we measure the execution time of this code for a large input, we see that this operation is very slow, perhaps surprisingly so!We'll benchmark this with IPython's ``%timeit`` magic:<jupyter_code>big_array = np.random.randint(1, 100, size=1000000)
%timeit compute_reciprocals(big_array)<jupyter_output>1.56 s ± 36.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)<jupyter_text>It takes several seconds to compute these million operations and to store the result!When even cell phones have processing speeds measured in Giga-FLOPS (i.e., billions of numerical operations per second), this seems almost absurdly slow.It turns out that the bottleneck here is not the operations themselves, but the type-checking and function dispatches that CPython must do at each cycle of the loop.Each time the reciprocal is computed, Python first examines the object's type and does a dynamic lookup of the correct function to use for that type.If we were working in compiled code instead, this type specification would be known before the code executes and the result could be computed much more efficiently.  Introducing UFuncsFor many types of operations, NumPy provides a convenient interface into just this kind of statically typed, compiled routine. This is known as a *vectorized* operation.This can be accomplished by simply performing an operation on the array, which will then be applied to each element.This vectorized approach is designed to push the loop into the compiled layer that underlies NumPy, leading to much faster execution.Compare the results of the following two:<jupyter_code>print(compute_reciprocals(values))
print(1.0 / values)<jupyter_output>[0.16666667 1.         0.25       0.25       0.125     ]
[0.16666667 1.         0.25       0.25       0.125     ]<jupyter_text>Looking at the execution time for our big array, we see that it completes orders of magnitude faster than the Python loop:<jupyter_code>%timeit (1.0 / big_array)<jupyter_output>958 µs ± 15.6 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)<jupyter_text>Vectorized operations in NumPy are implemented via *ufuncs*, whose main purpose is to quickly execute repeated operations on values in NumPy arrays.Ufuncs are extremely flexible – before we saw an operation between a scalar and an array, but we can also operate between two arrays:<jupyter_code>np.arange(5) / np.arange(1, 6)<jupyter_output><empty_output><jupyter_text>And ufunc operations are not limited to one-dimensional arrays–they can also act on multi-dimensional arrays as well:<jupyter_code>x = np.arange(9).reshape((3, 3))
2 ** x<jupyter_output><empty_output><jupyter_text>Computations using vectorization through ufuncs are nearly always more efficient than their counterpart implemented using Python loops, especially as the arrays grow in size.Any time you see such a loop in a Python script, you should consider whether it can be replaced with a vectorized expression.  Exploring NumPy's UFuncsUfuncs exist in two flavors: *unary ufuncs*, which operate on a single input, and *binary ufuncs*, which operate on two inputs.We'll see examples of both these types of functions here.  Array arithmeticNumPy's ufuncs feel very natural to use because they make use of Python's native arithmetic operators.The standard addition, subtraction, multiplication, and division can all be used:<jupyter_code>x = np.arange(4)
print("x     =", x)
print("x + 5 =", x + 5)
print("x - 5 =", x - 5)
print("x * 2 =", x * 2)
print("x / 2 =", x / 2)
print("x // 2 =", x // 2)  # floor division<jupyter_output>x     = [0 1 2 3]
x + 5 = [5 6 7 8]
x - 5 = [-5 -4 -3 -2]
x * 2 = [0 2 4 6]
x / 2 = [0.  0.5 1.  1.5]
x // 2 = [0 0 1 1]<jupyter_text>There is also a unary ufunc for negation, and a ``**`` operator for exponentiation, and a ``%`` operator for modulus:<jupyter_code>print("-x     = ", -x)
print("x ** 2 = ", x ** 2)
print("x % 2  = ", x % 2)<jupyter_output>-x     =  [ 0 -1 -2 -3]
x ** 2 =  [0 1 4 9]
x % 2  =  [0 1 0 1]<jupyter_text>In addition, these can be strung together however you wish, and the standard order of operations is respected:<jupyter_code>-(0.5*x + 1) ** 2<jupyter_output><empty_output><jupyter_text>Each of these arithmetic operations are simply convenient wrappers around specific functions built into NumPy; for example, the ``+`` operator is a wrapper for the ``add`` function:<jupyter_code>np.add(x, 2)<jupyter_output><empty_output><jupyter_text>The following table lists the arithmetic operators implemented in NumPy:| Operator	    | Equivalent ufunc    | Description                           ||---------------|---------------------|---------------------------------------||``+``          |``np.add``           |Addition (e.g., ``1 + 1 = 2``)         ||``-``          |``np.subtract``      |Subtraction (e.g., ``3 - 2 = 1``)      ||``-``          |``np.negative``      |Unary negation (e.g., ``-2``)          ||``*``          |``np.multiply``      |Multiplication (e.g., ``2 * 3 = 6``)   ||``/``          |``np.divide``        |Division (e.g., ``3 / 2 = 1.5``)       ||``//``         |``np.floor_divide``  |Floor division (e.g., ``3 // 2 = 1``)  ||``**``         |``np.power``         |Exponentiation (e.g., ``2 ** 3 = 8``)  ||``%``          |``np.mod``           |Modulus/remainder (e.g., ``9 % 4 = 1``)|Additionally there are Boolean/bitwise operators; we will explore these in [Comparisons, Masks, and Boolean Logic](02.06-Boolean-Arrays-and-Masks.ipynb).  Absolute valueJust as NumPy understands Python's built-in arithmetic operators, it also understands Python's built-in absolute value function:<jupyter_code>x = np.array([-2, -1, 0, 1, 2])
abs(x)<jupyter_output><empty_output><jupyter_text>The corresponding NumPy ufunc is ``np.absolute``, which is also available under the alias ``np.abs``:<jupyter_code>np.absolute(x)
np.abs(x)<jupyter_output><empty_output><jupyter_text>This ufunc can also handle complex data, in which the absolute value returns the magnitude:<jupyter_code>x = np.array([3 - 4j, 4 - 3j, 2 + 0j, 0 + 1j])
np.abs(x)<jupyter_output><empty_output><jupyter_text>Trigonometric functionsNumPy provides a large number of useful ufuncs, and some of the most useful for the data scientist are the trigonometric functions.We'll start by defining an array of angles:<jupyter_code>theta = np.linspace(0, np.pi, 3)<jupyter_output><empty_output><jupyter_text>Now we can compute some trigonometric functions on these values:<jupyter_code>print("theta      = ", theta)
print("sin(theta) = ", np.sin(theta))
print("cos(theta) = ", np.cos(theta))
print("tan(theta) = ", np.tan(theta))<jupyter_output>theta      =  [0.         1.57079633 3.14159265]
sin(theta) =  [0.0000000e+00 1.0000000e+00 1.2246468e-16]
cos(theta) =  [ 1.000000e+00  6.123234e-17 -1.000000e+00]
tan(theta) =  [ 0.00000000e+00  1.63312394e+16 -1.22464680e-16]<jupyter_text>The values are computed to within machine precision, which is why values that should be zero do not always hit exactly zero.Inverse trigonometric functions are also available:<jupyter_code>x = [-1, 0, 1]
print("x         = ", x)
print("arcsin(x) = ", np.arcsin(x))
print("arccos(x) = ", np.arccos(x))
print("arctan(x) = ", np.arctan(x))<jupyter_output>x         =  [-1, 0, 1]
arcsin(x) =  [-1.57079633  0.          1.57079633]
arccos(x) =  [3.14159265 1.57079633 0.        ]
arctan(x) =  [-0.78539816  0.          0.78539816]<jupyter_text>Exponents and logarithmsAnother common type of operation available in a NumPy ufunc are the exponentials:<jupyter_code>x = [1, 2, 3]
print("x     =", x)
print("e^x   =", np.exp(x))
print("2^x   =", np.exp2(x))
print("3^x   =", np.power(3, x))<jupyter_output>x     = [1, 2, 3]
e^x   = [ 2.71828183  7.3890561  20.08553692]
2^x   = [2. 4. 8.]
3^x   = [ 3  9 27]<jupyter_text>The inverse of the exponentials, the logarithms, are also available.The basic ``np.log`` gives the natural logarithm; if you prefer to compute the base-2 logarithm or the base-10 logarithm, these are available as well:<jupyter_code>x = [1, 2, 4, 10]
print("x        =", x)
print("ln(x)    =", np.log(x))
print("log2(x)  =", np.log2(x))
print("log10(x) =", np.log10(x))<jupyter_output>x        = [1, 2, 4, 10]
ln(x)    = [0.         0.69314718 1.38629436 2.30258509]
log2(x)  = [0.         1.         2.         3.32192809]
log10(x) = [0.         0.30103    0.60205999 1.        ]<jupyter_text>Aggregations: Min, Max, and Everything In Between Often when faced with a large amount of data, a first step is to compute summary statistics for the data in question.Perhaps the most common summary statistics are the mean and standard deviation, which allow you to summarize the "typical" values in a dataset, but other aggregates are useful as well (the sum, product, median, minimum and maximum, quantiles, etc.).NumPy has fast built-in aggregation functions for working on arrays; we'll discuss and demonstrate some of them here.  Summing the Values in an ArrayAs a quick example, consider computing the sum of all values in an array.Python itself can do this using the built-in ``sum`` function:<jupyter_code>import numpy as np
L = np.random.random(100)
sum(L)<jupyter_output><empty_output><jupyter_text>The syntax is quite similar to that of NumPy's ``sum`` function, and the result is the same in the simplest case:<jupyter_code>np.sum(L)<jupyter_output><empty_output><jupyter_text>However, because it executes the operation in compiled code, NumPy's version of the operation is computed much more quickly:<jupyter_code>big_array = np.random.rand(1000000)
%timeit sum(big_array)
%timeit np.sum(big_array)<jupyter_output>69 ms ± 1.07 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
317 µs ± 6.59 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)<jupyter_text>Be careful, though: the ``sum`` function and the ``np.sum`` function are not identical, which can sometimes lead to confusion!In particular, their optional arguments have different meanings, and ``np.sum`` is aware of multiple array dimensions, as we will see in the following section.  Minimum and MaximumSimilarly, Python has built-in ``min`` and ``max`` functions, used to find the minimum value and maximum value of any given array:<jupyter_code>min(big_array), max(big_array)<jupyter_output><empty_output><jupyter_text>NumPy's corresponding functions have similar syntax, and again operate much more quickly:<jupyter_code>np.min(big_array), np.max(big_array)
%timeit min(big_array)
%timeit np.min(big_array)<jupyter_output>46 ms ± 1.15 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
339 µs ± 5.24 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)<jupyter_text>For ``min``, ``max``, ``sum``, and several other NumPy aggregates, a shorter syntax is to use methods of the array object itself:<jupyter_code>print(big_array.min(), big_array.max(), big_array.sum())<jupyter_output>7.071203171893359e-07 0.9999997207656334 500216.8034810001<jupyter_text>Whenever possible, make sure that you are using the NumPy version of these aggregates when operating on NumPy arrays!  Multi dimensional aggregatesOne common type of aggregation operation is an aggregate along a row or column.Say you have some data stored in a two-dimensional array:<jupyter_code>M = np.random.random((3, 4))
print(M)<jupyter_output>[[0.79832448 0.44923861 0.95274259 0.03193135]
 [0.18441813 0.71417358 0.76371195 0.11957117]
 [0.37578601 0.11936151 0.37497044 0.22944653]]<jupyter_text>By default, each NumPy aggregation function will return the aggregate over the entire array:<jupyter_code>M.sum()<jupyter_output><empty_output><jupyter_text>Aggregation functions take an additional argument specifying the *axis* along which the aggregate is computed. For example, we can find the minimum value within each column by specifying ``axis=0``:<jupyter_code>M.min(axis=0)<jupyter_output><empty_output><jupyter_text>The function returns four values, corresponding to the four columns of numbers.Similarly, we can find the maximum value within each row:<jupyter_code>M.max(axis=1)<jupyter_output><empty_output><jupyter_text>The way the axis is specified here can be confusing to users coming from other languages.The ``axis`` keyword specifies the *dimension of the array that will be collapsed*, rather than the dimension that will be returned.So specifying ``axis=0`` means that the first axis will be collapsed: for two-dimensional arrays, this means that values within each column will be aggregated.  Other aggregation functionsNumPy provides many other aggregation functions, but we won't discuss them in detail here.Additionally, most aggregates have a ``NaN``-safe counterpart that computes the result while ignoring missing values, which are marked by the special IEEE floating-point ``NaN`` value (for a fuller discussion of missing data, see [Handling Missing Data](03.04-Missing-Values.ipynb)).Some of these ``NaN``-safe functions were not added until NumPy 1.8, so they will not be available in older NumPy versions.The following table provides a list of useful aggregation functions available in NumPy:|Function Name      |   NaN-safe Version  | Description                                   ||-------------------|---------------------|-----------------------------------------------|| ``np.sum``        | ``np.nansum``       | Compute sum of elements                       || ``np.prod``       | ``np.nanprod``      | Compute product of elements                   || ``np.mean``       | ``np.nanmean``      | Compute mean of elements                      || ``np.std``        | ``np.nanstd``       | Compute standard deviation                    || ``np.var``        | ``np.nanvar``       | Compute variance                              || ``np.min``        | ``np.nanmin``       | Find minimum value                            || ``np.max``        | ``np.nanmax``       | Find maximum value                            || ``np.argmin``     | ``np.nanargmin``    | Find index of minimum value                   || ``np.argmax``     | ``np.nanargmax``    | Find index of maximum value                   || ``np.median``     | ``np.nanmedian``    | Compute median of elements                    || ``np.percentile`` | ``np.nanpercentile``| Compute rank-based statistics of elements     || ``np.any``        | N/A                 | Evaluate whether any elements are true        || ``np.all``        | N/A                 | Evaluate whether all elements are true        |We will see these aggregates often throughout the rest of the book.  Comparison Operators as ufuncsWe introduced ufuncs, and focused in particular on arithmetic operators. We saw that using ``+``, ``-``, ``*``, ``/``, and others on arrays leads to element-wise operations.NumPy also implements comparison operators such as ```` (greater than) as element-wise ufuncs.The result of these comparison operators is always an array with a Boolean data type.All six of the standard comparison operations are available:<jupyter_code>x = np.array([1, 2, 3, 4, 5])
x < 3  # less than
x > 3  # greater than
x <= 3  # less than or equal
x >= 3  # greater than or equal
x != 3  # not equal
x == 3  # equal<jupyter_output><empty_output><jupyter_text>It is also possible to do an element-wise comparison of two arrays, and to include compound expressions:<jupyter_code>(2 * x) == (x ** 2)<jupyter_output><empty_output><jupyter_text>As in the case of arithmetic operators, the comparison operators are implemented as ufuncs in NumPy; for example, when you write ``x < 3``, internally NumPy uses ``np.less(x, 3)``.    A summary of the comparison operators and their equivalent ufunc is shown here:    | Operator    | Equivalent ufunc    |-| Operator   | Equivalent ufunc    ||---------------|---------------------|---|---------------|---------------------||``==``         |``np.equal``         | -|``!=``         |``np.not_equal``     ||``<``          |``np.less``          | -|``<=``         |``np.less_equal``    ||``>``          |``np.greater``       | -|``>=``         |``np.greater_equal`` | Using comparison operators to subset an array:<jupyter_code>x[x > 3]<jupyter_output><empty_output><jupyter_text>Fast Sorting in NumPy: ``np.sort`` and ``np.argsort``Although Python has built-in ``sort`` and ``sorted`` functions to work with lists, we won't discuss them here because NumPy's ``np.sort`` function turns out to be much more efficient and useful for our purposes.By default ``np.sort`` uses an $\mathcal{O}[N\log N]$, *quicksort* algorithm, though *mergesort* and *heapsort* are also available. For most applications, the default quicksort is more than sufficient.To return a sorted version of the array without modifying the input, you can use ``np.sort``:<jupyter_code>x = np.array([2, 1, 4, 3, 5])
np.sort(x)<jupyter_output><empty_output><jupyter_text>If you prefer to sort the array in-place, you can instead use the ``sort`` method of arrays:<jupyter_code>x.sort()
print(x)<jupyter_output>[1 2 3 4 5]<jupyter_text>A related function is ``argsort``, which instead returns the *indices* of the sorted elements:<jupyter_code>x = np.array([2, 1, 4, 3, 5])
i = np.argsort(x)
print(i)<jupyter_output>[1 0 3 2 4]<jupyter_text>The first element of this result gives the index of the smallest element, the second value gives the index of the second smallest, and so on.These indices can then be used (via fancy indexing) to construct the sorted array if desired:<jupyter_code>x[i]<jupyter_output><empty_output><jupyter_text>Sorting along rows or columns A useful feature of NumPy's sorting algorithms is the ability to sort along specific rows or columns of a multidimensional array using the ``axis`` argument. For example:<jupyter_code>rand = np.random.RandomState(42)
X = rand.randint(0, 10, (4, 6))
print(X)
# sort each column of X
np.sort(X, axis=0)
# sort each row of X
np.sort(X, axis=1)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup