<jupyter_start><jupyter_text>Regression Relationships- deterministic (such as Celius  Fahrenheit) - these are exact- statistical (height & weight) - displays scatter or trend in graph Notation> Greek letters express variables in the population> $Y_i = \alpha + \beta(x_i - \bar{x}) + \epsilon_i$ > Latin characters express variables in the sample> $y_i = a + b(x_i - \bar{x}) + e_i$ > bar expresses the calculated mean> $\bar{x}$> hat expresses the expected value (best fit, i.e. regression line)> $\hat{x}$ Linear Regression Equation*line of best fit*$Y=a+bX$ where Y is the *dependent* (or response, outcome) variable, and X is the *independent* (or predictor/explanatory) variable.$\displaystyle a = \frac{ (\sum y)(\sum x^2) - (\sum x)(\sum xy) }{ n(\sum x^2) - (\sum x)^2}$$\displaystyle b = \frac{ n(\sum xy) - (\sum x)(\sum y) }{ n(\sum x^2) - (\sum x)^2}$  Derivation- let $x_i$ be the ith predictor value (x axis)- let $y_i$ be the ith observed value (y axis)- let $\hat{y}_i$ be the ith predicted value using a regression line (y value of the line of best fit) Find the least error*The error between observation and prediction needs to be a low as possible*- $e_i = y_i - \hat{y}_i$Squaring the sum of errors allows us to express the optimal value - the least squares criterion$\displaystyle\sum_{i=1}^{n} e_i^2$*This is better than using summing the absolute value, as squaring not only creates positive values, it is supported better with calculus**So, we need to find the best fitted line, a.k.a the values a and b, where  $\hat{y}_i = mx_i + c$. We can express the sum of values in terms found in the aforementioned linear formula*$\displaystyle\begin{split}Q &= \sum_{i=1}^{n} e_i^2 \\              &= \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 \\              &= \sum_{i=1}^{n} \big(y_i - (a + bx_i) \big)^2 \\ \end{split}$*Instead of finding 'a' the intercept, which isn't useful for many statistical analysis (i.e. height and weight won't ever become 0), we use the arithmetic mean of x and y as the origin.*$\hat{y}_i = a + b(x_i - \bar{x})$ Find the minimum interceptTo find the minimum with respects to a, the intercept, solve the derivative at 0$\displaystyle\begin{split}Q &= \sum_{i=1}^{n} \Big(y_i - \big(a + b(x_i - \bar{x}) \big) \Big)^2  \\ & \text{     using chain rule: } f = g \circ h (x) \implies f^\prime = g^\prime \circ h(x) \cdot h^\prime(x) \\& \text{     where } g(x) = \sum_{i=1}^n ( x_i^2 )\text{, and  } h(x) = \Big(y_i - \big(a + b(x_i - \bar{x}) \big) \Big) \\\frac{dQ}{da} &= 2\sum_{i=1}^{n} \Big(y_i - \big(a + b(x_i - \bar{x}) \big) \Big) (-1) \equiv 0 \\&= - \sum_{i=1}^n y_i + \sum_{i=1}^n a + b \sum_{i=1}^n (x_i - \bar{x}) \equiv 0 \\& \text{as } \sum_{i=1}^n a = na \text{. and } \sum_{i=1}^n (x_i - \bar{x}) = 0 \\&= - \sum_{i=1}^n y_i + na \equiv 0 \\a &= \frac{1}{n} \sum\limits_{i=1}^n y_i \\  &= \bar{y} \\  & \text{ a is the mean of all y values at the minimum/best fit, when the mean of all x's is taken as the mid point} \\Q &= \sum_{i=1}^{n} \Big(y_i - \big( \bar{y} + b(x_i - \bar{x}) \big) \Big)^2  \\ \end{split}$ Find the minimum slopeTo find the minimum with respects to b, the slope, solve the derivative at 0$\displaystyle\begin{split}Q &= \sum_{i=1}^{n} \Big(y_i - \big( \bar{y} + b(x_i - \bar{x}) \big) \Big)^2  \\ & \text{using chain rule: } f = g \circ h (x) \implies f^\prime = g^\prime \circ h(x) \cdot h^\prime(x) \\& \text{where } g(x) = \sum_{i=1}^n ( x_i^2 )\text{, and  } h(x) = \Big(y_i - \big( \bar{y} + b(x_i - \bar{x}) \big) \\\frac{dQ}{db} &= 2\sum_{i=1}^{n} \Big(y_i - \big( \bar{y} + b(x_i - \bar{x}) \big) \Big) \cdot - (x_i - \bar{x}) \equiv 0 \\ &= - \sum_{i=1}^{n} (y_i -  \bar{y})(x_i - \bar{x}) + b \sum_{i=1}^{n} (x_i - \bar{x})(x_i - \bar{x}) \equiv 0 \\ & b \sum_{i=1}^{n} (x_i - \bar{x})^2 = \sum_{i=1}^{n} (y_i -  \bar{y})(x_i - \bar{x}) \\ b &= \frac{ \sum_{i=1}^{n} (x_i - \bar{x})(y_i -  \bar{y}) }{  \sum_{i=1}^{n} (x_i - \bar{x})^2 } \\\end{split}$<jupyter_code># Regression Line
SIZE = 1200
SAMPLE_SIZE = 120

# Generate random variables for our population
M = 0.3
C = 3.0
x = np.random.uniform(size=SIZE) * 20
y = 10.0 + np.random.normal(scale=0.8, size=SIZE)
y = y+(x*M-C)

# get midpoint
x_mean_pop = x.sum() / x.size
y_mean_pop = y.sum() / y.size

# Take a random sample for analysis
sample = np.random.choice(range(SAMPLE_SIZE), 120)
x_sample = x[sample]
y_sample = y[sample]
x_mean_sample = x_sample.sum() / x_sample.size
y_mean_sample = y_sample.sum() / y_sample.size

# Manual Way to get intercept and slope for our sample
nom = np.fromfunction(lambda i: (x_sample[i] - x_mean_sample) * (y_sample[i] - y_mean_sample), shape = (SAMPLE_SIZE,), dtype=np.int )
denom = np.fromfunction(lambda i: (x_sample[i] - x_mean_sample) ** 2, shape = (SAMPLE_SIZE,), dtype=np.int )
slope_sample = nom.sum() / denom.sum()
intercept_sample = y_mean - slope_sample*x_mean

# The Numpy way for our population
slope_pop, intercept_pop = np.polyfit(x, y, 1)

# build ab line
abline_x = np.linspace(0,20.0,20)
abline_values_pop = [slope_pop * i + intercept_pop for i in abline_x]
abline_values_sample = [slope_sample * i + intercept_sample for i in abline_x]

plt.figure(figsize=(14,6))
plt.margins(0,0)
plt.title('')

plt.plot(x_mean_pop, y_mean_pop, color='indigo', lw=1, alpha=0.5, marker='o', label='Arithmetic mean of population x={:0.2f}, y={:0.2f}'.format(x_mean_pop, y_mean_pop))
plt.scatter(x, y, color='slateblue', s=1, lw=1, alpha=0.3, label='Distribution of population')
plt.plot(abline_x, abline_values_pop, color='rebeccapurple', lw=1, alpha=1.0, dashes =(6,6), label=r'Regression Line (using np.polyfit) of population $\mu_Y = E(Y) = \alpha + \beta(x-\bar{x})$')

plt.plot(x_mean_sample, y_mean_sample, color='maroon', lw=1, alpha=0.5, marker='o', label='Arithmetic mean of sample x={:0.2f}, y={:0.2f}'.format(x_mean_sample, y_mean_sample))
plt.scatter(x_sample, y_sample, color='red', s=1, lw=1, alpha=0.7, label='Distribution of sample')
plt.plot(abline_x, abline_values_sample, color='orangered', lw=1, alpha=1.0, dashes =(6,6), label=r'Manual Regression Line of sample $\hat{y} = a + b(x-\bar{x})$')

plt.xlim((0,20.0))
plt.ylim((0,15.0))
plt.legend()
plt.show()<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup