  Distributed bilevel optimization has gained increasing popularity due to its
wide applications in large-scale machine learning. This paper focuses on a
class of nonconvex-strongly-convex distributed stochastic bilevel optimization
(DSBO) problems with personalized inner-level objectives. Existing algorithms
require extra computation loops to compute inner-level solutions accurately and
Hessian inverse matrices for estimating the hypergradient, which incurs high
computational complexity of gradient evaluation. To address this issue, we
propose a loopless personalized distributed algorithm (termed LoPA) that
leverages iterative approximation for the inner-level solutions and Hessian
inverse matrices. Theoretical analysis shows that LoPA has a sublinear rate of
${{\mathcal{O}}}( {\frac{{1}}{{(1-\rho)^2K}} + \frac{{b^{\frac{2}{3}}
}}{{\left( {1 - \rho } \right)^{\frac{2}{3}}K^{\frac{2}{3}} }} +
\frac{{1}}{\sqrt{ K }} ( \sigma_{\rm{p}} + \frac{1}{\sqrt{m}}\sigma_{\rm{c}}) }
)$, where $K$ is the total number of iterations, $b$ quantifies the data
heterogeneity across nodes, and $\sigma_{\rm p}, \sigma_{\rm c}$ represent the
gradient sampling variances associated with the inner-level and out-level
variables, respectively. We provide an explicit characterization of the
heterogeneity, and develop a variant of LoPA based on gradient tracking to
eliminate the heterogeneity, yielding a rate of ${{\mathcal{O}}}(\frac{{{1}}}{{
(1-\rho)^4K }} + \frac{1}{{\sqrt{K}}}( \sigma_{\rm{p}} +
\frac{1}{\sqrt{m}}\sigma_{\rm{c}} ) )$. The computational complexity of LoPA is
shown to be of the order of ${{\mathcal{O}}}({\epsilon^{-2}})$ thanks to the
loopless structure, outperforming existing counterparts for DSBO by an order of
${{\mathcal{O}}}(\log{\epsilon^{-1}})$. Numerical experiments demonstrate the
effectiveness of the proposed algorithm.
