  Unmanned Aerial Systems (UAS) are being increasingly deployed for commercial,
civilian, and military applications. The current UAS state-of-the-art still
depends on a remote human controller with robust wireless links to perform
several of these applications. The lack of autonomy restricts the domains of
application and tasks for which a UAS can be deployed. Enabling autonomy and
intelligence to the UAS will help overcome this hurdle and expand its use
improving safety and efficiency. The exponential increase in computing
resources and the availability of large amount of data in this digital era has
led to the resurgence of machine learning from its last winter. Therefore, in
this chapter, we discuss how some of the advances in machine learning,
specifically deep learning and reinforcement learning can be leveraged to
develop next-generation autonomous UAS. We first begin motivating this chapter
by discussing the application, challenges, and opportunities of the current UAS
in the introductory section. We then provide an overview of some of the key
deep learning and reinforcement learning techniques discussed throughout this
chapter. A key area of focus that will be essential to enable autonomy to UAS
is computer vision. Accordingly, we discuss how deep learning approaches have
been used to accomplish some of the basic tasks that contribute to providing
UAS autonomy. Then we discuss how reinforcement learning is explored for using
this information to provide autonomous control and navigation for UAS. Next, we
provide the reader with directions to choose appropriate simulation suites and
hardware platforms that will help to rapidly prototype novel machine learning
based solutions for UAS. We additionally discuss the open problems and
challenges pertaining to each aspect of developing autonomous UAS solutions to
shine light on potential research areas.
