Papers

Collection of papers in trajectory forecasting categorised according to the high-level structure

The literature survey is categorized as:

1. Classical: Papers not utilizing neural networks for trajectory forecasting
2. Motion-Based: Papers utilizing neural networks for trajectory forecasting without modelling interactions with neighbouring agents or physical spaces.
3. Agent-Agent Interactions: Papers utilizing neural networks for trajectory forecasting modelling interactions with neighbouring agents but not physical spaces.
4. Agent-Space Interactions: Papers utilizing neural networks for trajectory forecasting modelling interactions with physical spaces but not neighbouring agents.
5. Agent-Agent-Space Interactions: Papers utilizing neural networks for trajectory forecasting modelling interactions with both physical spaces as well as neighbouring agents.
6. Miscellaneous: Papers related to related topics like activity forecasting, human body dynamics

Classical

1. Social Force Model for Pedestrian Dynamics, 1998 Paper
2. Simulation of pedestrian dynamics using a two-dimensional cellular automaton, 2001 Paper
3. Discrete Choice Models for Pedestrian Walking Behavior, 2006 Paper
4. Continuum crowds, 2006 Paper
5. Modelling Smooth Paths Using Gaussian Processes, 2007 Paper
6. Reciprocal n-body Collision Avoidance (ORCA), 2008 Paper
7. You’ll Never Walk Alone: Modeling Social Behavior for Multi-target Tracking, 2009 Paper
8. Socially-Aware Large-Scale Crowd Forecasting, 2014 Paper
9. Learning to Predict Trajectories of Cooperatively Navigating Agents, 2014 Paper
10. Understanding pedestrian behaviors from stationary crowd groups, 2015 Paper
11. Learning Social Etiquette: Human Trajectory Understanding In Crowded Scenes, 2016 Paper
12. Point-based Path Prediction from Polar Histograms, 2016 Paper

Motion-Based

1. Bi-Prediction: Pedestrian Trajectory Prediction Based on Bidirectional LSTM Classification, 2017 Paper
2. RED: A simple but effective Baseline Predictor for the TrajNet Benchmark, 2018 Paper
3. Convolutional Neural Network for Trajectory Prediction, 2018 Paper
4. Location-Velocity Attention for Pedestrian Trajectory Prediction, 2019 Paper
5. The Simpler the Better: Constant Velocity for Pedestrian Motion Prediction, 2019 Paper
6. Transformer Networks for Trajectory Forecasting, 2020 Paper

Agent-Agent Interaction

1. Social LSTM: Human Trajectory Prediction in Crowded Spaces, 2016 Paper
2. A Data-driven Model for Interaction-Aware Pedestrian Motion Prediction in Object Cluttered Environments, 2017 Paper
3. Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection, 2017 Paper
4. Social Attention: Modeling Attention in Human Crowds, 2017 Paper
5. 3DOF Pedestrian Trajectory Prediction Learned from Long-Term Autonomous Mobile Robot Deployment Data, 2017 Paper
6. Encoding Crowd Interaction with Deep Neural Network for Pedestrian Trajectory Prediction, 2018 Paper
7. Group LSTM: Group Trajectory Prediction in Crowded Scenarios, 2018 Paper
8. MX-LSTM: mixing tracklets and vislets to jointly forecast trajectories and head poses, 2018 Paper
9. StarNet: Pedestrian Trajectory Prediction using Deep Neural Network in Star Topology, 2019 Paper
10. SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Prediction, 2019 Paper
11. Recursive Social Behavior Graph for Trajectory Prediction, 2020 Paper
12. Collaborative Motion Prediction via Neural Motion Message Passing Paper

Multimodal

1. Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks, 2018 Paper
2. Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANs, 2019 Paper
3. Which Way Are You Going? Imitative Decision Learning for Path Forecasting in Dynamic Scenes, 2019 Paper
4. Analyzing the Variety Loss in the Context of Probabilistic Trajectory Prediction, 2019 Paper
5. The Trajectron: Probabilistic Multi-Agent Trajectory Modeling With Dynamic Spatiotemporal Graphs, 2019 Paper
6. STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction, 2019 Paper
7. Stochastic Trajectory Prediction with Social Graph Network, 2019 Paper
8. Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction, 2020 Paper
9. It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction, 2020 Paper
10. STAR: Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction, 2020 Paper

Agent-Agent-Space Interaction

1. Context-Aware Trajectory Prediction in Crowded Spaces, 2017 Paper
2. Human Trajectory Prediction using Spatially aware Deep Attention Models, 2017 Paper
3. SS-LSTM: A Hierarchical LSTM Model for Pedestrian Trajectory Prediction, 2018 Paper
4. A Data-driven Model for Interaction-aware Pedestrian Motion Prediction in Object Cluttered Environments, 2018 Paper
5. Multi-Agent Tensor Fusion for Contextual Trajectory Prediction, 2019 Paper

Multimodal

1. DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents, 2017 Paper
2. SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints, 2019 Paper
3. Peeking into the Future: Predicting Future Person Activities and Locations in Videos, 2019 Paper
4. Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks, 2019 Paper
5. Social-WaGDAT: Interaction-aware Trajectory Prediction via Wasserstein Graph Double-Attention Network, 2020 Paper
6. Trajectron++: Multi-Agent Generative Trajectory Forecasting With Heterogeneous Data for Control, 2020 Paper
7. Reciprocal Learning Networks for Human Trajectory Prediction, 2020 Paper
8. The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction, 2020 Paper
9. Dynamic and Static Context-aware LSTM for Multi-agent Motion Prediction, 2020 Paper

Agent-Space Interaction

Multimodal

1. Accurate and Diverse Sampling of Sequences based on a “Best of Many” Sample Objective, 2018 Paper
2. Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction, 2019 Paper
3. Scene Compliant Trajectory Forecast with Agent-Centric Spatio-Temporal Grids, 2020 Paper
4. Goal-GAN: Multimodal Trajectory Prediction Based on Goal Position Estimation, 2020 Paper

Miscellaneous

Activity Forecasting

1. Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach, 2011 Paper
2. Activity forecasting, 2012 Paper
3. Context-Based Pedestrian Path Prediction, 2014 Paper
4. Pedestrian’s Trajectory Forecast in Public Traffic with Artificial Neural Networks, 2014 Paper
5. Learning Intentions for Improved Human Motion Prediction, 2014 Paper
6. Pedestrian Path, Pose, and Intention Prediction Through Gaussian Process Dynamical Models and Pedestrian Activity Recognition, 2019 Paper

Human Body Dynamics

1. Gaussian Process Dynamical Models for Human Motion, 2008 Paper
2. Recurrent Network Models for Human Dynamics, 2015 Paper

Evaluation

Comparison of popular human trajectory forecasting papers based on the datasets on which the methods have been evaluated.

Method	ETH/UCY	SDD	TrajNet++	Multipath
S-LSTM	✓
DESIRE		✓
S-GAN	✓
Sophie	✓	✓
Trajectron	✓
Social-BiGAT	✓
Social-STGCNN	✓
Multiverse				✓
PECNet	✓	✓	✓
D-LSTM			✓
Social-NCE			✓

A Note on Evaluation benchmarks

Evaluation on TrajNet++ is preferred in comparison to ETH/UCY as the test set and the evaluation protocol for TrajNet++ is fixed (and extensive!). More details here. The variation in ADE/FDE greatly reduces among different methods when evaluated on equal grounds on TrajNet++ (leaderboard) in comparison to the numbers reported on ETH/UCY.

Trajectory Forecasting Framework

If you are new to trajectory forecasting, do check out the TrajNet++ framework! TrajNet++ is a code-base with specific focus on human trajectory forecasting, and having more than 10 trajectory forecastng baselines already implemented.