  We consider two popular Graph Representation Learning (GRL) methods: message
passing for node classification and network embedding for link prediction. For
each, we pick a popular model that we: (i) linearize and (ii) and switch its
training objective to Frobenius norm error minimization. These simplifications
can cast the training into finding the optimal parameters in closed-form. We
program in TensorFlow a functional form of Truncated Singular Value
Decomposition (SVD), such that, we could decompose a dense matrix $\mathbf{M}$,
without explicitly computing $\mathbf{M}$. We achieve competitive performance
on popular GRL tasks while providing orders of magnitude speedup. We
open-source our code at http://github.com/samihaija/tf-fsvd
