  Many keyhole interventions rely on bi-manual handling of surgical
instruments, forcing the main surgeon to rely on a second surgeon to act as a
camera assistant. In addition to the burden of excessively involving surgical
staff, this may lead to reduced image stability, increased task completion time
and sometimes errors due to the monotony of the task. Robotic endoscope
holders, controlled by a set of basic instructions, have been proposed as an
alternative, but their unnatural handling may increase the cognitive load of
the (solo) surgeon, which hinders their clinical acceptance. More seamless
integration in the surgical workflow would be achieved if robotic endoscope
holders collaborated with the operating surgeon via semantically rich
instructions that closely resemble instructions that would otherwise be issued
to a human camera assistant, such as "focus on my right-hand instrument". As a
proof of concept, this paper presents a novel system that paves the way towards
a synergistic interaction between surgeons and robotic endoscope holders. The
proposed platform allows the surgeon to perform a bimanual coordination and
navigation task, while a robotic arm autonomously performs the endoscope
positioning tasks. Within our system, we propose a novel tooltip localization
method based on surgical tool segmentation and a novel visual servoing approach
that ensures smooth and appropriate motion of the endoscope camera. We validate
our vision pipeline and run a user study of this system. The clinical relevance
of the study is ensured through the use of a laparoscopic exercise validated by
the European Academy of Gynaecological Surgery which involves bi-manual
coordination and navigation. Successful application of our proposed system
provides a promising starting point towards broader clinical adoption of
robotic endoscope holders.
