text: This paper develops a path planner that minimizes risk (e.g. motion execution) while maximizing accumulated reward (e.g., quality of sensor viewpoint) motivated by visual assistance or tracking scenarios in unstructured or confined environments. In these scenarios, the robot should maintain the best viewpoint as it moves to the goal. However, in unstructured or confined environments, some paths may increase the risk of collision; therefore there is a tradeoff between risk and reward. Conventional state-dependent risk or probabilistic uncertainty modeling do not consider path-level risk or is difficult to acquire. This risk-reward planner explicitly represents risk as a function of motion plans, i.e., paths. Without manual assignment of the negative impact to the planner caused by risk, this planner takes in a pre-established viewpoint quality map and plans target location and path leading to it simultaneously, in order to maximize overall reward along the entire path while minimizing risk. Exact and approximate algorithms are presented, whose solution is further demonstrated on a physical tethered aerial vehicle. Other than the visual assistance problem, the proposed framework also provides a new planning paradigm to address minimum-risk planning under dynamical risk and absence of substructure optimality and to balance the trade-off between reward and risk.
  @cite_7: When controlling dynamic systems, such as mobile robots in uncertain environments, there is a trade off between risk and reward. For example, a race car can turn a corner faster by taking a more challenging path. This paper proposes a new approach to planning a control sequence with a guaranteed risk bound. Given a stochastic dynamic model, the problem is to find a control sequence that optimizes a performance metric, while satisfying chance constraints i.e. constraints on the upper bound of the probability of failure. We propose a two-stage optimization approach, with the upper stage optimizing the risk allocation and the lower stage calculating the optimal control sequence that maximizes reward. In general, the upper-stage is a non-convex optimization problem, which is hard to solve. We develop a new iterative algorithm for this stage that efficiently computes the risk allocation with a small penalty to optimality. The algorithm is implemented and tested on the autonomous underwater vehicle (AUV) depth planning problem, and demonstrates a substantial improvement in computation cost and suboptimality, compared to the prior arts.
  @cite_21: Recent advances in Autonomous Underwater Vehicle (AUV) technology have facilitated the collection of oceanographic data at a fraction of the cost of ship-based sampling methods. Unlike oceanographic data collection in the deep ocean, operation of AUVs in coastal regions exposes them to the risk of collision with ships and land. Such concerns are particularly prominent for slow-moving AUVs since ocean current magnitudes are often strong enough to alter the planned path significantly. Prior work using predictive ocean currents relies upon deterministic outcomes, which do not account for the uncertainty in the ocean current predictions themselves. To improve the safety and reliability of AUV operation in coastal regions, we introduce two stochastic planners: (a) a Minimum Expected Risk planner and (b) a risk-aware Markov Decision Process, both of which have the ability to utilize ocean current predictions probabilistically. We report results from extensive simulation studies in realistic ocean current fields obtained from widely used regional ocean models. Our simulations show that the proposed planners have lower collision risk than state-of-the-art methods. We present additional results from field experiments where ocean current predictions were used to plan the paths of two Slocum gliders. Field trials indicate the practical usefulness of our techniques over long-term deployments, showing them to be ideal for AUV operations.
  @cite_1: Autonomous agents operating in partially observable stochastic environments often face the problem of optimizing expected performance while bounding the risk of violating safety constraints. Such problems can be modeled as chance-constrained POMDP's (CC-POMDP's). Our first contribution is a systematic derivation of execution risk in POMDP domains, which improves upon how chance constraints are handled in the constrained POMDP literature. Second, we present RAO*, a heuristic forward search algorithm producing optimal, deterministic, finite-horizon policies for CC-POMDP's. In addition to the utility heuristic, RAO* leverages an admissible execution risk heuristic to quickly detect and prune overly-risky policy branches. Third, we demonstrate the usefulness of RAO* in two challenging domains of practical interest: power supply restoration and autonomous science agents.
  @cite_23: 
  @cite_2: 
  @cite_13: 
  @cite_25: 
  @cite_12: between planner conservatism and the risk of infeasibility. This paper presents a novel real-time planning algorithm, chance constrained rapidly-exploring random trees (CC-RRT), which uses chance constraints to guarantee probabilistic feasibility for linear systems subject to process noise and or uncertain, possibly dynamic obstacles. By using RRT, the algorithm enjoys the computational benets of sampling-based algorithms, such as trajectory-wise constraint checking and incorporation of heuristics, while explicitly incorporating uncertainty within the formulation. Under the assumption of Gaussian noise, probabilistic feasibility at each time step can be established through simple simulation of the state conditional mean and the evaluation of linear constraints. Alternatively, a small amount of additional computation can be used to explicitly compute a less conservative probability bound at each time step. Simulation results show that this algorithm can be used for ecient identication and execution of probabilistically safe paths in real time.
  @cite_11: This work seeks to address the problem of planning in the presence of uncertainty and constraints. Such problems arise in many situations, including the basis of this work, which involves planning for a team of first responders (both humans and robots) operating in an urban environment. The problem is framed as a Partially-Observable Markov Decision Process (POMDP) with constraints, and it is shown that even in a relatively simple planning problem, modeling constraints as large penalties does not lead to good solutions. The main contribution of the work is a new online algorithm that explicitly ensures constraint feasibility while remaining computationally tractable. Its performance is demonstrated on an example problem and it is demonstrated that our online algorithm generates policies comparable to an offline constrained POMDP algorithm.
summary: The trade-off between reward and risk was mostly addressed as chance or probability of success failure. @cite_12 @cite_2 proposed chance-constrained rapidly-exploring random tree (CC-RRT) approaches, which used chance constraints to guarantee probabilistic feasibility at each time step and over entire trajectory. Another popular approach to handle reward and risk is to use (PO)MDP. As standard MDP inherently contains reward but not risk, researchers have looked into representing risk as negative reward (penalty) @cite_21 or constraints (C-POMDP) with unit cost for constraint violation @cite_11 @cite_23 @cite_25 . Going beyond unit cost, CC-POMDP was proposed by @cite_1 , which was based on a bound on the probability (chance) of some event happening during policy execution. RMPC is another alternative, with an emphasis on risk allocation, i.e., to allocate more risk for more rewarding actions @cite_7 @cite_13 .