  Recent technological advancements in space, air and ground components have
made possible a new network paradigm called "space-air-ground integrated
network" (SAGIN). Unmanned aerial vehicles (UAVs) play a key role in SAGINs.
However, due to UAVs' high dynamics and complexity, the real-world deployment
of a SAGIN becomes a major barrier for realizing such SAGINs. Compared to the
space and terrestrial components, UAVs are expected to meet performance
requirements with high flexibility and dynamics using limited resources.
Therefore, employing UAVs in various usage scenarios requires well-designed
planning in algorithmic approaches. In this paper, we provide a comprehensive
review of recent learning-based algorithmic approaches. We consider possible
reward functions and discuss the state-of-the-art algorithms for optimizing the
reward functions, including Q-learning, deep Q-learning, multi-armed bandit
(MAB), particle swarm optimization (PSO) and satisfaction-based learning
algorithms. Unlike other survey papers, we focus on the methodological
perspective of the optimization problem, which can be applicable to various
UAV-assisted missions on a SAGIN using these algorithms. We simulate users and
environments according to real-world scenarios and compare the learning-based
and PSO-based methods in terms of throughput, load, fairness, computation time,
etc. We also implement and evaluate the 2-dimensional (2D) and 3-dimensional
(3D) variations of these algorithms to reflect different deployment cases. Our
simulation suggests that the $3$D satisfaction-based learning algorithm
outperforms the other approaches for various metrics in most cases. We discuss
some open challenges at the end and our findings aim to provide design
guidelines for algorithm selections while optimizing the deployment of
UAV-assisted SAGINs.
