text: Pedestrians and vehicles often share the road in complex inner city traffic. This leads to interactions between the vehicle and pedestrians, with each affecting the other's motion. In order to create robust methods to reason about pedestrian behavior and to design interfaces of communication between self-driving cars and pedestrians we need to better understand such interactions. In this paper, we present a data-driven approach to implicitly model pedestrians' interactions with vehicles, to better predict pedestrian behavior. We propose a LSTM model that takes as input the past trajectories of the pedestrian and ego-vehicle, and pedestrian head orientation, and predicts the future positions of the pedestrian. Our experiments based on a real-world, inner city dataset captured with vehicle mounted cameras, show that the usage of such cues improve pedestrian prediction when compared to a baseline that purely uses the past trajectory of the pedestrian.
  @cite_18: Future vehicle systems for active pedestrian safety will not only require a high recognition performance but also an accurate analysis of the developing traffic situation. In this paper, we present a study on pedestrian path prediction and action classification at short subsecond time intervals. We consider four representative approaches: two novel approaches (based on Gaussian process dynamical models and probabilistic hierarchical trajectory matching) that use augmented features derived from dense optical flow and two approaches as baseline that use positional information only (a Kalman filter and its extension to interacting multiple models). In experiments using stereo vision data obtained from a vehicle, we investigate the accuracy of path prediction and action classification at various time horizons, the effect of various errors (image localization, vehicle egomotion estimation), and the benefit of the proposed approaches. The scenario of interest is that of a crossing pedestrian, who might stop or continue walking at the road curbside. Results indicate similar performance of the four approaches on walking motion, with near-linear dynamics. During stopping, however, the two newly proposed approaches, with nonlinear and or higher order models and augmented motion features, achieve a more accurate position prediction of 10-50 cm at a time horizon of 0-0.77 s around the stopping event.
  @cite_29: We present a novel approach combining pedestrian intention recognition and path prediction for advanced video-based driver assistance systems. The core algorithm uses an Interacting Multiple Model Filter in combination with a Latent-dynamic Conditional Random Field model. The model integrates pedestrian dynamics and situational awareness using observations from a stereo-video system for pedestrian detection and human head pose estimation. Evaluation of our method is performed on a public available dataset addressing scenarios of lateral approaching pedestrians that might cross the road, turn into the road or stop at the curbside. During experiments, we demonstrate that the proposed approach leads to better path prediction performance in terms of a smaller lateral position error compared to state-of-the-art pedestrian intention recognition and path prediction approaches. The computational costs of our approach is comparatively low and therefore can be ported easily onto a real-time system.
  @cite_32: We present a novel approach for pedestrian intention recognition for advanced video-based driver assistance systems using a Latent-dynamic Conditional Random Field model. The model integrates pedestrian dynamics and situational awareness using observations from a stereo-video system for pedestrian detection and human head pose estimation. The model is able to capture both intrinsic and extrinsic class dynamics. Evaluation of our method is performed on a public available dataset addressing scenarios of lateral approaching pedestrians that might cross the road, turn into the road or stop at the curbside. During experiments, we demonstrate that the proposed approach leads to better stability and class separation compared to state-of-the-art pedestrian intention recognition approaches.
  @cite_3: With the rapid development of the techniques for autonomous driving and ADAS in the last decade, more advanced methods to understand pedestrian behavior are required. Crosswalks at intersections are the one of most hazardous where many accidents between turning-vehicles and pedestrians occur. In this paper, we present a method for estimating the pedestrian’s intention to cross a signalized crosswalk or stop in front of it. The intention is crucial to not only the collision avoidance but also smooth traffic in the context of autonomous driving by reducing unnecessary risk margins. Regarding the behavioral flow of pedestrian: assessment, decision-making and physical movement, as a stochastic process, we construct a probabilistic model with the Dynamic Bayesian Network. It takes account of not only pedestrian physical states but also contextual information and integrates the relationship among them. By employing the particle filter as a Bayesian filtering framework, the model estimates the pedestrian state from signal information and pedestrian position measurements. Evaluation using experimental data collected in real traffic scene shows that the proposed model has an ability to detect the pedestrian intention to cross a crosswalk even when he she is far from it.
  @cite_0: Active safety systems which assess highly dynamic traffic situations including pedestrians are required with growing demands in autonomous driving and ADAS. In this paper, we focus on one of the most hazardous traffic situations: the possible collision between a pedestrian and a turning vehicle at intersections. This paper presents a probabilistic model of pedestrian behavior to signalized crosswalks. For this purpose, we take not only pedestrian physical states but also contextual information into account. We propose a model based on the Dynamic Bayesian Network (DBN) which integrates relations among the intersection context information and the pedestrian behavior in the same way as human. Afterwards, the model jointly estimates their states by the particle filter. Experimental evaluation using real traffic data shows that this model is able to recognize the pedestrian crossing decision in advance from the traffic signal and pedestrian position information.
  @cite_19: 
  @cite_2: 
  @cite_15: When given a single frame of the video, humans can not only interpret the content of the scene, but also they are able to forecast the near future. This ability is mostly driven by their rich prior knowledge about the visual world, both in terms of (i) the dynamics of moving agents, as well as (ii) the semantic of the scene. In this work we exploit the interplay between these two key elements to predict scene-specific motion patterns. First, we extract patch descriptors encoding the probability of moving to the adjacent patches, and the probability of being in that particular patch or changing behavior. Then, we introduce a Dynamic Bayesian Network which exploits this scene specific knowledge for trajectory prediction. Experimental results demonstrate that our method is able to accurately predict trajectories and transfer predictions to a novel scene characterized by similar elements.
summary: More closely related to our work are approaches that use vehicle mounted cameras and use information of pedestrian dynamics coupled with the pedestrian's awareness @cite_29 @cite_2 @cite_15 @cite_3 @cite_0 @cite_18 . However they are usually focused on specific scenarios as signalized crosswalks @cite_3 and intersections @cite_0 , or does not have a high diversity in number of scenarios and pedestrians @cite_15 @cite_19 @cite_29 @cite_32 . Our approach is not restricted to a specific scenario and we reason about pedestrian future positions by learning such cues from realistic data.