  We train embodied neural networks to plan and navigate unseen complex 3D
environments, emphasising real-world deployment. Rather than requiring prior
knowledge of the agent or environment, the planner learns to model the state
transitions and rewards. To avoid the potentially hazardous trial-and-error of
reinforcement learning, we focus on differentiable planners such as Value
Iteration Networks (VIN), which are trained offline from safe expert
demonstrations. Although they work well in small simulations, we address two
major limitations that hinder their deployment. First, we observed that current
differentiable planners struggle to plan long-term in environments with a high
branching complexity. While they should ideally learn to assign low rewards to
obstacles to avoid collisions, we posit that the constraints imposed on the
network are not strong enough to guarantee the network to learn sufficiently
large penalties for every possible collision. We thus impose a structural
constraint on the value iteration, which explicitly learns to model any
impossible actions. Secondly, we extend the model to work with a limited
perspective camera under translation and rotation, which is crucial for real
robot deployment. Many VIN-like planners assume a 360 degrees or overhead view
without rotation. In contrast, our method uses a memory-efficient lattice map
to aggregate CNN embeddings of partial observations, and models the rotational
dynamics explicitly using a 3D state-space grid (translation and rotation). Our
proposals significantly improve semantic navigation and exploration on several
2D and 3D environments, succeeding in settings that are otherwise challenging
for this class of methods. As far as we know, we are the first to successfully
perform differentiable planning on the difficult Active Vision Dataset,
consisting of real images captured from a robot.
