Then we set s`,j to 1 if ` is the jth neighbor of the data point i and zero otherwise. We can then the rewrite the objective function for the data set as

$$E(\mathbf{W})=\frac{1}{2}\sum_{i=1}^{n}\mathbf{w}_{i}^{\top}\mathbf{S}_{i}^{\top}\mathbf{Y}\mathbf{Y}^{\top}\mathbf{S}_{i}\mathbf{w}_{i}-\frac{1}{2}\sum_{i=1}^{n}\log\mathbf{w}_{i}^{\top}\mathbf{11}^{\top}\mathbf{w}_{i}+\text{const},\tag{10}$$  which is a similar to $i$th constraint.  
A fixed point can be found by taking gradients with respect to wi,

$$\frac{\mathrm{d}E(\mathbf{W})}{\mathrm{d}\mathbf{w}_{i}}=\mathbf{S}_{i}^{\intercal}\mathbf{YY}^{\intercal}\mathbf{S}_{i}\mathbf{w}_{i}-\frac{1}{\mathbf{w}_{i}^{\intercal}}\mathbf{1}$$ is given by ...
which implies that the direction of wiis given by

$$\mathbf{w}_{i}\propto\mathbf{C}_{i}^{-1}\mathbf{1}$$

where Ci = S
>
i YY>Si has been called the "local covariance matrix" by Roweis and Saul [2000], removing the croupier matrix we can express the local covariance matrix in the same form given by Roweis and Saul [2000],

$$\mathbf{C}_{i}=\sum_{j\in{\mathcal{N}}(i)}(\mathbf{y}_{j,:}-\mathbf{y}_{i,:})(\mathbf{y}_{j,:}-\mathbf{y}_{i,:})^{\top}.$$

For standard LLE the magnitude of the vector wiis set by the fact that 1
>wi = 1. In our alternative formulation we can find the magnitude of the vector through differentiation of (8) with respect to m2 i,i leading to the following fixed point

 $\quad m_{i,i}^{-2}=\left\|\mathbf{y}_{i,:}-\sum_{j\in\mathcal{N}(i)}\mathbf{y}_{j,:}\hat{m}_{j,i}\right\|_2^2,$  shows explicitly that $m_{i,:}$ estimates the precision with wi. 
where ˆmj,i = −mj,i/mi,i. This update shows explicitly that mi,i estimates the precision with which each individual regression problem is solved.

## 2.7.1 Determining The Embedding In Lle

If the data is truly low dimensional, then we might expect that the local linear relationships between neighbors continue to hold for a data set X, of lower dimensionality, *q < p*, than the original data Y. The next step in the LLE procedure is to find this data set. We do this by minimizing the objective function in (9) with respect to this new, low dimensional data set. Writing the objective in terms of this reduced dimensional data set, X, we have

$$E(\mathbf{X})={\frac{1}{2}}\sum_{i=1}^{n}\mathbf{m}_{.,i}^{\top}\mathbf{X}\mathbf{X}^{\top}\mathbf{m}_{.,i}+\text{const}$$ $$={\frac{1}{2}}\text{tr}\left(\mathbf{M}\mathbf{M}^{\top}\mathbf{X}\mathbf{X}\right)+\text{const}$$ $$={\frac{1}{2}}\sum_{i=1}^{n}\mathbf{x}_{i.}^{\top}\mathbf{M}\mathbf{M}^{\top}\mathbf{x}_{i.}+\text{const}.$$

Clearly the objective function is trivially minimized by setting X = 0, so to avoid this solution a constraint is imposed that X>X = I. This leads to an eigenvalue problem of the form

## Mm>Ui = Λiui.

Here the smallest q + 1 eigenvalues are extracted. The smallest eigenvector is the constant eigenvector and is associated with an eigenvalue of zero. This is because, by construction, we have set MM>1 = 0. The next q eigenvectors are retained to make up the low dimensional representation so we have x:,i = ui+1 for i = 1*..q.*
Extracting the latent coordinates in LLE is extremely similar to the process suggested in Laplacian eigenmaps, despite different motivations. Though in the LLE case the constraint on the latent embeddings is not scaled by the degree matrix. The procedure is also identical to that used in classical multidimensional scaling, and therefore matches that used in MVU and MEU, although again the motivation is different. Rather than distance matching, as suggested for CMDS, in LLE we are looking for a 'representative', low dimensional, data set.