text: In principle, reinforcement learning and policy search methods can enable robots to learn highly complex and general skills that may allow them to function amid the complexity and diversity of the real world. However, training a policy that generalizes well across a wide range of real-world conditions requires far greater quantity and diversity of experience than is practical to collect with a single robot. Fortunately, it is possible for multiple robots to share their experience with one another, and thereby, learn a policy collectively. In this work, we explore distributed and asynchronous policy learning as a means to achieve generalization and improved training times on challenging, real-world manipulation tasks. We propose a distributed and asynchronous version of Guided Policy Search and use it to demonstrate collective policy learning on a vision-based door opening task using four robots. We show that it achieves better generalization, utilization, and training times than the single robot alternative.
  @cite_24: Rapidly expanding internet resources and wireless networking have potential to liberate robots and automation systems from limited onboard computation, memory, and software. “Cloud Robotics” describes an approach that recognizes the wide availability of networking and incorporates open-source elements to greatly extend earlier concepts of “Online Robots” and “Networked Robots”. In this paper we consider how cloud-based data and computation can facilitate 3D robot grasping. We present a system architecture, implemented prototype, and initial experimental data for a cloud-based robot grasping system that incorporates a Willow Garage PR2 robot with onboard color and depth cameras, Google's proprietary object recognition engine, the Point Cloud Library (PCL) for pose estimation, Columbia University's GraspIt! toolkit and OpenRAVE for 3D grasping and our prior approach to sampling-based grasp analysis to address uncertainty in pose. We report data from experiments in recognition (a recall rate of 80 for the objects in our test set), pose estimation (failure rate under 14 ), and grasping (failure rate under 23 ) and initial results on recall and false positives in larger data sets using confidence measures.
  @cite_5: 
  @cite_1: The Cloud infrastructure and its extensive set of Internet-accessible resources has potential to provide significant benefits to robots and automation systems. We consider robots and automation systems that rely on data or code from a network to support their operation, i.e., where not all sensing, computation, and memory is integrated into a standalone system. This survey is organized around four potential benefits of the Cloud: 1) Big Data: access to libraries of images, maps, trajectories, and descriptive data; 2) Cloud Computing: access to parallel grid computing on demand for statistical analysis, learning, and motion planning; 3) Collective Robot Learning: robots sharing trajectories, control policies, and outcomes; and 4) Human Computation: use of crowdsourcing to tap human skills for analyzing images and video, classification, learning, and error recovery. The Cloud can also improve robots and automation systems by providing access to: a) datasets, publications, models, benchmarks, and simulation tools; b) open competitions for designs and systems; and c) open-source software. This survey includes over 150 references on results and open challenges. A website with new developments and updates is available at: http: goldberg.berkeley.edu cloud-robotics
  @cite_8: Robots of the future will act in the real world. To realize robots with the common sense to do this will, it is generally believed, require massively parallel processing. This is a problem for those of us who want to do experiments with robots in the real world today—it is hard to build active, limber, situated robots when they have to carry along heavy brains. Our answer is “remote-brained robots.” A remote-brained robot is designed to have the brain and body separate, both conceptually and physically. It allows us to tie artificial intelligence (AI) with massive parallelism directly to the real world, enabling the verification of high-level AI techniques, which could previously only be used in simulation. Once the brain is placed remotely from the body, it encourages us to investigate many important research topics of embodied agents working in the real world. This paper describes such platform for future robotics and principal experiments done on the platform.
summary: A central motivation behind our work is the ability to apply deep learning to robotic manipulation by making it feasible to collect large amounts of on-policy experience with real physical platforms. While this may seem impractical for small-scale laboratory experiments, it becomes much more realistic when we consider a possible future where robots are deployed in the real-world to perform a wide variety of skills. The challenges of asynchrony, utilization, and parallelism, which we aim to address in this work, are central for such real-world deployments. The ability of robotic systems to learn more quickly and effectively by pooling their collective experience has long been recognized in the domain of cloud robotics, where it is typically referred to as collective robotic learning @cite_8 @cite_5 @cite_24 @cite_1 . Our work therefore represents a step toward more practical and powerful collective learning with distributed, asynchronous data collection.