  Creating open-ended algorithms, which generate their own never-ending stream
of novel and appropriately challenging learning opportunities, could help to
automate and accelerate progress in machine learning. A recent step in this
direction is the Paired Open-Ended Trailblazer (POET), an algorithm that
generates and solves its own challenges, and allows solutions to goal-switch
between challenges to avoid local optima. However, the original POET was unable
to demonstrate its full creative potential because of limitations of the
algorithm itself and because of external issues including a limited problem
space and lack of a universal progress measure. Importantly, both limitations
pose impediments not only for POET, but for the pursuit of open-endedness in
general. Here we introduce and empirically validate two new innovations to the
original algorithm, as well as two external innovations designed to help
elucidate its full potential. Together, these four advances enable the most
open-ended algorithmic demonstration to date. The algorithmic innovations are
(1) a domain-general measure of how meaningfully novel new challenges are,
enabling the system to potentially create and solve interesting challenges
endlessly, and (2) an efficient heuristic for determining when agents should
goal-switch from one problem to another (helping open-ended search better
scale). Outside the algorithm itself, to enable a more definitive demonstration
of open-endedness, we introduce (3) a novel, more flexible way to encode
environmental challenges, and (4) a generic measure of the extent to which a
system continues to exhibit open-ended innovation. Enhanced POET produces a
diverse range of sophisticated behaviors that solve a wide range of
environmental challenges, many of which cannot be solved through other means.
