<jupyter_start><jupyter_text>**Chapter 1.2 Elements of Linear Algebra**---  **1.2.1 Linear Spaces****Linear Combination**: an expression constructed from a subset by multiplying each term by a constant and adding the results.**Linear Subspace**: a subset U⊆V that is closed under vector addition ($u_1$ + $u_2 \epsilon U$) and scalar multiplication ($\alpha u_1 \epsilon U$)**Span**: the span($w_1$,...,$w_m$) is the set of all linear combinations of $w_j$'s given that $w_1$,...,$w_m \epsilon V$<jupyter_code>a1 = 1
a2 = 1
u1 = 1
u2 = 1
x = a1 * u1 + a2 * u2
print(x)<jupyter_output>2<jupyter_text>In the code above, $a_1$ and $a_2$ are mutable constants while $u_1$ and $u_2$ are variables. Any combination of values for $a_1$ and $a_2$ will result in *x* being a linear combination. The set of all possible linear combinations of $u_1$ and $u_2$ is the span and is denoted as span($u_1$,$u_2$). span($u_1$,$u_2$) as well as all other spans are linear subspaces because they are comprised of linear combinations, which are inherently closed under vector addition and scalar multiplication. **Linear Independence**: A list of vector $u_1,...,u_m$ is said to be linearly independent if none of them can be written as a linear combination of the others<jupyter_code>def dependence_check(vector1, vector2):
  matrix = np.stack((vector1, vector2), axis = -1)
  det = np.linalg.det(matrix)
  if det == 0:
    print("the vectors are linearly dependent")
  else:
    print("the vectors are linearly independent")
a = np.array([1, 2])
b = np.array([2, 4])
c = np.array([2, 5])
dependence_check(a, b)
dependence_check(a, c)<jupyter_output>the vectors are linearly dependent
the vectors are linearly independent<jupyter_text>The above code checks if the sets of vectors *a* and *b* as well as *a* and *c* are linearly independent. *a* and *b* are not linearly independent because *b* = *a* $\cdot$ 2. *a* and *c* are linearly independent because neither can be written as a linear combination of another. **Column Space**: the column space of *A*, where *A* is an n x m matrix and *A* ∈ $R^{n \cdot m}$, is the span of columns of A**Basis**: A basis of *U*, where *U* is a linear subspace of *V*, is a list of vectors $u_1,...,u_m$ that span *U* and are linearly independent. A basis is a unique representation and will always have the same dimension as other bases of *U*.**Dimension**: the number of rows by the number of columns of a matrix.**Rank**: the dimension of the vector space spanned by a matrix's columns.<jupyter_code>import numpy as np
import pprint
def column_space(matrix):
  b = a.transpose()
  rank = 0
  for row in b:
    rank += 1
    print(row)
  print(f'rank: {rank}')
  print(f'dimension: {matrix.size}')
a = np.array([[1, 2], [3, 4]])
column_space(a)<jupyter_output>[1 3]
[2 4]
rank: 2
dimension: 4<jupyter_text>The column space of *a* would be $\begin{pmatrix}1\\ 3\end{pmatrix}c_1 + \begin{pmatrix}2\\4\end{pmatrix}c_2$ for the code above. The basis of the column space would also be {$\begin{pmatrix}1\\3\end{pmatrix}, \begin{pmatrix}2\\4\end{pmatrix}$} since these vectors are minimally spanning. The rank is equal to the number of vectors in the column space, while the dimension is equal to the total number of elements in the array.  1.2.2 Orthogonality**Inner Product**: $ = u \cdot v = \sum_i^n u_iv_i$**Norm**: $\|v\| = \sqrt {\sum_1^n u_i^2}$**Orthonormal**: A list of vectors $u_1,...,u_m$ is orthonormal if the $u_i$'s are pairwise orthogonal and each has norm 1.<jupyter_code>a = np.identity(3, dtype = int)
print(a)<jupyter_output>[[1 0 0]
 [0 1 0]
 [0 0 1]]<jupyter_text>In the code above, the basis of *a* is orthonormal because the $u_i$'s are pairwise orthogonal and each has norm 1. **Best Approximation Theorem**: we have a linear subspace $U \subseteq V$ and a vector $v \nsubseteq U$ and we want to find the vector $v^*$ in $U$ that is closest to $v$ in 2-norm. We want to solve $min_{v^* \epsilon U} \|v^* - v \|$. To confirm the optimality of $v^*$, we use the Pythagorean theorem and find that $\|v - \alpha u_1 \|^2 \geq \|v - v^* \|^2$ **Orthogonal Projection**: Let $U \subseteq V$ be a linear subspace with orthonormal basis $q_1,...,q_m$ and let $v \epsilon V$. For any $u \epsilon U$, $\| v - P_Uv \| \leq \| v - u \|$. Furthermore, if $u \epsilon U$ and the previous inequality is an equality, then $u = P_Uv$  1.2.3 Eigenvalues and Eigenvectors**Eigenvalue**: Let $A \epsilon R^{dxd}$ be a square matrix. Then $\lambda \epsilon R$ is an eigenvalue of A if there exists a nonzero vector $x \neq 0$ such that $Ax = \lambda x$**Eigenvector**: The vector x in the previous equation is referred to as the eigenvector<jupyter_code>a = np.array([[0, 1], [-2, -3]])
val, vect = np.linalg.eig(a)
print("eigenvalues: ", val)
print("eigenvectors: ", vect)<jupyter_output>eigenvalue:  [-1. -2.]
eigenvector:  [[ 0.70710678 -0.4472136 ]
 [-0.70710678  0.89442719]]<jupyter_text>The above code solves for the eigenvalues and eigenvectors of matrix $a$.<jupyter_code>a = np.array([[0, -1], [1, 0]])
val, vect = np.linalg.eig(a)
print("eigenvalues: ", val)
print("eigenvectors: ", vect)<jupyter_output>eigenvalue:  [0.+1.j 0.-1.j]
eigenvector:  [[0.70710678+0.j         0.70710678-0.j        ]
 [0.        -0.70710678j 0.        +0.70710678j]]<jupyter_text>Looking at the code above, the matrix $\begin{pmatrix}0 & -1\\1 & 0\end{pmatrix}$ has no real solution when solving for eigenvalues.<jupyter_code>a = np.array([[5, 2], [2, 5]])
val, vect = np.linalg.eig(a)
print("eigenvalues: ", val)
print("eigenvectors: ", vect)<jupyter_output>eigenvalue:  [7. 3.]
eigenvector:  [[ 0.70710678 -0.70710678]
 [ 0.70710678  0.70710678]]