  Artificial intelligence systems increasingly involve continual learning to
enable flexibility in general situations that are not encountered during system
training. Human interaction with autonomous systems is broadly studied, but
research has hitherto under-explored interactions that occur while the system
is actively learning, and can noticeably change its behaviour in minutes. In
this pilot study, we investigate how the interaction between a human and a
continually learning prediction agent develops as the agent develops
competency. Additionally, we compare two different agent architectures to
assess how representational choices in agent design affect the human-agent
interaction. We develop a virtual reality environment and a time-based
prediction task wherein learned predictions from a reinforcement learning (RL)
algorithm augment human predictions. We assess how a participant's performance
and behaviour in this task differs across agent types, using both quantitative
and qualitative analyses. Our findings suggest that human trust of the system
may be influenced by early interactions with the agent, and that trust in turn
affects strategic behaviour, but limitations of the pilot study rule out any
conclusive statement. We identify trust as a key feature of interaction to
focus on when considering RL-based technologies, and make several
recommendations for modification to this study in preparation for a
larger-scale investigation. A video summary of this paper can be found at
https://youtu.be/oVYJdnBqTwQ .
