  A hallmark of human intelligence is the ability to construct self-contained
chunks of knowledge and reuse them in novel combinations for solving different
problems. Learning such compositional structures has been a challenge for
artificial systems, due to the underlying combinatorial search. To date,
research into compositional learning has largely proceeded separately from work
on lifelong or continual learning. This dissertation integrated these two lines
of work to present a general-purpose framework for lifelong learning of
functionally compositional structures. The framework separates the learning
into two stages: learning how to combine existing components to assimilate a
novel problem, and learning how to adapt the existing components to accommodate
the new problem. This separation explicitly handles the trade-off between
stability and flexibility. This dissertation instantiated the framework into
various supervised and reinforcement learning (RL) algorithms. Supervised
learning evaluations found that 1) compositional models improve lifelong
learning of diverse tasks, 2) the multi-stage process permits lifelong learning
of compositional knowledge, and 3) the components learned by the framework
represent self-contained and reusable functions. Similar RL evaluations
demonstrated that 1) algorithms under the framework accelerate the discovery of
high-performing policies, and 2) these algorithms retain or improve performance
on previously learned tasks. The dissertation extended one lifelong
compositional RL algorithm to the nonstationary setting, where the task
distribution varies over time, and found that modularity permits individually
tracking changes to different elements in the environment. The final
contribution of this dissertation was a new benchmark for compositional RL,
which exposed that existing methods struggle to discover the compositional
properties of the environment.
