  Risk and sparsity requirements often need to be enforced simultaneously in
many applications, e.g., in portfolio optimization, assortment planning, and
treatment planning. Properly balancing these potentially conflicting
requirements entails the formulation of functional constrained optimization
with either convex or nonconvex objectives. In this paper, we focus on
projection-free methods that can generate a sparse trajectory for solving these
challenging functional constrained optimization problems. Specifically, for the
convex setting, we propose a Level Conditional Gradient (LCG) method, which
leverages a level-set framework to update the approximation of the optimal
value and an inner conditional gradient oracle (CGO) for solving mini-max
subproblems. We show that the method achieves
$\mathcal{O}\big(\frac{1}{\epsilon^2}\log\frac{1}{\epsilon}\big)$ iteration
complexity for solving both smooth and nonsmooth cases without dependency on a
possibly large size of optimal dual Lagrange multiplier. For the nonconvex
setting, we introduce the Level Inexact Proximal Point (IPP-LCG) method and the
Direct Nonconvex Conditional Gradient (DNCG) method. The first approach taps
into the advantage of LCG by transforming the problem into a series of convex
subproblems and exhibits an
$\mathcal{O}\big(\frac{1}{\epsilon^3}\log\frac{1}{\epsilon}\big)$ iteration
complexity for finding an ($\epsilon,\epsilon$)-KKT point. The DNCG is the
first single-loop projection-free method, with iteration complexity bounded by
$\mathcal{O}\big(1/\epsilon^4\big)$ for computing a so-called $\epsilon$-Wolfe
point. We demonstrate the effectiveness of LCG, IPP-LCG and DNCG by devising
formulations and conducting numerical experiments on two risk averse sparse
optimization applications: a portfolio selection problem with and without
cardinality requirement, and a radiation therapy planning problem in
healthcare.
