Learning-based motion planning in dynamic environments using GNNs and temporal encoding
Article No.: 2175, Pages 30003 - 30015
Abstract
Learning-based methods have shown promising performance for accelerating motion planning, but mostly in the setting of static environments. For the more challenging problem of planning in dynamic environments, such as multi-arm assembly tasks and human-robot interaction, motion planners need to consider the trajectories of the dynamic obstacles and reason about temporal-spatial interactions in very large state spaces. We propose a GNN-based approach that uses temporal encoding and imitation learning with data aggregation for learning both the embeddings and the edge prioritization policies. Experiments show that the proposed methods can significantly accelerate online planning over state-of-the-art complete dynamic planning algorithms. The learned models can often reduce costly collision checking operations by more than 1000x, and thus accelerating planning by up to 95%, while achieving high success rates on hard instances as well.
Supplementary Material
Supplemental material.
- Download
- 31.81 MB
References
[1]
M. J. Bency, A. H. Qureshi, and M. C. Yip. Neural path planning: Fixed time, near-optimal path generation via oracle imitation. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2019, Macau, SAR, China, November 3-8, 2019, pages 3965-3972. IEEE, 2019.
[2]
D. Berenson, S. S. Srinivasa, and J. J. Kuffner. Task space regions: A framework for pose-constrained manipulation planning. Int. J. Robotics Res., 30(12):1435-1460, 2011.
[3]
L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig. Safe learning in robotics: From learning-based control to safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems, 5:411-444, 2022.
[4]
J. Chase Kew, B. Ichter, M. Bandari, T.-W. E. Lee, and A. Faust. Neural collision clearance estimator for batched motion planning. In S. M. LaValle, M. Lin, T. Ojala, D. Shell, and J. Yu, editors, Algorithmic Foundations of Robotics XIV, pages 73-89, Cham, 2021. Springer International Publishing. ISBN 978-3-030-66723-8.
[5]
B. Chen, B. Dai, Q. Lin, G. Ye, H. Liu, and L. Song. Learning to plan in high dimensions via neural exploration-exploitation trees. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https://openreview.net/forum?id=rJgJDAVKvB.
[6]
N. Das and M. Yip. Learning-based proxy collision detection for robot motion planning applications. IEEE Transactions on Robotics, 36(4):1096-1114, 2020.
[7]
J. Dong, M. Mukadam, F. Dellaert, and B. Boots. Motion planning as probabilistic inference using gaussian processes and factor graphs. In Robotics: Science and Systems, volume 12, 2016.
[8]
F. Gama, Q. Li, E. I. Tolstaya, A. Prorok, and A. Ribeiro. Synthesizing decentralized controllers with graph neural networks and imitation learning. IEEE Trans. Signal Process., 70:19321946,2022.
[9]
V. Garg, S. Jegelka, and T. Jaakkola. Generalization and representational limits of graph neural networks. In International Conference on Machine Learning, pages 3419-3430. PMLR, 2020.
[10]
E. N. Gilbert. Random plane networks. Journal of the society for industrial and applied mathematics, 9(4):533-543, 1961.
[11]
J. P. Gonzalez, A. Dornbush, and M. Likhachev. Using state dominance for path planning in dynamic environments with moving obstacles. In 2012 IEEE International Conference on Robotics and Automation, pages 4009-4015. IEEE, 2012.
[12]
C. Guan, W. Vega-Brown, and N. Roy. Efficient planning for near-optimal compliant manipulation leveraging environmental contact. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, May 21-25, 2018, pages 215-222. IEEE, 2018.
[13]
H. Ha, J. Xu, and S. Song. Learning a decentralized multi-arm motion planner. In J. Kober, F. Ramos, and C. J. Tomlin, editors, 4th Conference on Robot Learning, CoRL 2020, 16-18 November 2020, Virtual Event / Cambridge, MA, USA, volume 155 of Proceedings of Machine Learning Research, pages 103-114. PMLR, 2020. URL https://proceedings.mlr.press/v155/ha21a.html.
[14]
K. Hauser. On responsiveness, safety, and completeness in real-time motion planning. Autonomous Robots, 32(1):35-48, 2012.
[15]
S. Hochreiter and J. Schmidhuber. Long Short-Term Memory. Neural Computation, 9(8): 1735-1780,11 1997. ISSN 0899-7667.
[16]
B. Ichter and M. Pavone. Robot motion planning in learned latent spaces. IEEE Robotics Autom. Lett., 4(3):2407-2414, 2019.
[17]
B. Ichter, J. Harrison, and M. Pavone. Learning sampling distributions for robot motion planning. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, May 21-25, 2018, pages 7087-7094. IEEE, 2018.
[18]
T. Jurgenson and A. Tamar. Harnessing reinforcement learning for neural motion planning. In A. Bicchi, H. Kress-Gazit, and S. Hutchinson, editors, Robotics: Science and Systems XV, University of Freiburg, Freiburg im Breisgau, Germany, June 22-26, 2019, 2019.
[19]
S. Karaman and E. Frazzoli. Sampling-based algorithms for optimal motion planning. Int. J. Robotics Res., 30(7):846-894, 2011.
[20]
L. E. Kavraki, P. Svestka, J. Latombe, and M. H. Overmars. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE Trans. Robotics Autom., 12(4): 566-580, 1996.
[21]
A. Khan, A. Ribeiro, V. Kumar, and A. G. Francis. Graph neural networks for motion planning. CoRR, abs/2006.06248, 2020. URL https://arxiv.org/abs/2006.06248.
[22]
A. Khan, E. Tolstaya, A. Ribeiro, and V. Kumar. Graph policy gradients for large scale robot control. In Conference on robot learning, pages 823-834. PMLR, 2020.
[23]
L. Lee, E. Parisotto, D. S. Chaplot, E. P. Xing, and R. Salakhutdinov. Gated path planning networks. In J. G. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 2953-2961. PMLR, 2018. URL http://proceedings.mlr.press/v80/lee18c.html.
[24]
Q. Li, F. Gama, A. Ribeiro, and A. Prorok. Graph neural networks for decentralized multi-robot path planning. CoRR, abs/1912.06095, 2019. URL http://arxiv.org/abs/1912.06095.
[25]
Q. Li, W. Lin, Z. Liu, and A. Prorok. Message-aware graph attention networks for large-scale multi-robot path planning. IEEE Robotics Autom. Lett., 6(3):5533-5540, 2021.
[26]
S. Li and J. A. Shah. Safe and efficient high dimensional motion planning in space-time with time parameterized prediction. In 2019 international conference on robotics and automation (ICRA), pages 5012-5018. IEEE, 2019.
[27]
Z. S. O. B. S. S. Madaan, R. Learning adaptive sampling distributions for motion planning by self-imitation. Workshop on Machine Learning in Robot Motion Planning, IEEE IROS, 2018.
[28]
B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In European conference on computer vision, pages 405-421. Springer, 2020.
[29]
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013.
[30]
V. Narayanan, M. Phillips, and M. Likhachev. Anytime safe interval path planning for dynamic environments. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 4708-4715. IEEE, 2012.
[31]
S. Niu, S. Chen, H. Guo, C. Targonski, M. C. Smith, and J. Kovacevic. Generalized value iteration networks: Life beyond lattices. In S. A. McIlraith and K. Q. Weinberger, editors, Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 6246-6253. AAAI Press, 2018. URL https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16552.
[32]
K. Okumura, R. Yonetani, M. Nishimura, and A. Kanezaki. Ctrms: Learning to construct cooperative timed roadmaps for multi-agent path planning in continuous spaces. In P. Faliszewski, V. Mascardi, C. Pelachaud, and M. E. Taylor, editors, 21st International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2022, Auckland, New Zealand, May 9-13, 2022, pages 972-981. International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2022. URL https://www.ifaamas.org/Proceedings/aamas2022/pdfs/p972.pdf.
[33]
C. Park, J. Pan, and D. Manocha. Itomp: Incremental trajectory optimization for real-time replanning in dynamic environments. In Twenty-Second International Conference on Automated Planning and Scheduling, 2012.
[34]
M. Phillips and M. Likhachev. Sipp: Safe interval path planning for dynamic environments. In 2011 IEEE International Conference on Robotics and Automation, pages 5628-5635. IEEE, 2011.
[35]
C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 77-85. IEEE Computer Society, 2017.
[36]
A. H. Qureshi, Y. Miao, A. Simeonov, and M. C. Yip. Motion planning networks: Bridging the gap between learning-based and classical motion planners. IEEE Transactions on Robotics, 37 (1):48-66, 2021.
[37]
J. Reif and M. Sharir. Motion planning in the presence of moving obstacles. Journal of the ACM (JACM), 41(4):764-790, 1994.
[38]
S. Ross, G. J. Gordon, and D. Bagnell. A reduction of imitation learning and structured prediction to no-regret online learning. In G. J. Gordon, D. B. Dunson, and M. Dudik, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2011, Fort Lauderdale, USA, April 11-13, 2011, volume 15 of JMLR Proceedings, pages 627-635. JMLR.org, 2011. URL http://proceedings.mlr.press/v15/ross11a/ross11a.pdf.
[39]
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
[40]
R. A. M. Strudel, R. Garcia, J. Carpentier, J. Laumond, I. Laptev, and C. Schmid. Learning obstacle representations for neural motion planning. CoRR, abs/2008.11174, 2020. URL https://arxiv.org/abs/2008.11174.
[41]
A. Tamar, S. Levine, P. Abbeel, Y. Wu, and G. Thomas. Value iteration networks. In D. D. Lee, M. Sugiyama, U. von Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 2146-2154, 2016. URL https://proceedings.neurips.cc/paper/2016/hash/c21002f464c5fc5bee3b98ced83963b8-Abstract.html.
[42]
E. I. Tolstaya, F. Gama, J. Paulos, G. J. Pappas, V. Kumar, and A. Ribeiro. Learning decentralized controllers for robot swarms with graph neural networks. In L. P. Kaelbling, D. Kragic, and K. Sugiura, editors, 3rd Annual Conference on Robot Learning, CoRL 2019, Osaka, Japan, October 30 - November 1, 2019, Proceedings, volume 100 of Proceedings of Machine Learning Research, pages 671-682. PMLR, 2019. URL http://proceedings.mlr.press/v100/tolstaya20a.html.
[43]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008, 2017. URL https://proceedings.neuripscc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html.
[44]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[45]
F. Xue and P. R. Kumar. The number of neighbors needed for connectivity of wireless networks. Wireless networks, 10(2):169-181, 2004.
[46]
C. Yu and S. Gao. Reducing collision checking for sampling-based motion planning using graph neural networks. In M. Ranzato, A. Beygelzimer, Y. N. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 4274-4289, 2021. URL https://proceedings.neurips.cc/paper/2021/hash/224e5e49814ca908e58c02e28a0462c1-Abstract.html.
[47]
C. Zhang, J. Huh, and D. D. Lee. Learning implicit sampling distributions for motion planning. In 2018IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018, Madrid, Spain, October 1-5, 2018, pages 3654-3661. IEEE, 2018. 8594028.
[48]
L. Zhou, V. D. Sharma, Q. Li, A. Prorok, A. Ribeiro, and V. Kumar. Graph neural networks for decentralized multi-robot submodular action selection. arXiv preprint arXiv:2105.08601, 2021.
Index Terms
- Learning-based motion planning in dynamic environments using GNNs and temporal encoding
Index terms have been assigned to the content through auto-classification.
Recommendations
Continual planning and acting in dynamic multiagent environments
In order to behave intelligently, artificial agents must be able to deliberatively plan their future actions. Unfortunately, realistic agent environments are usually highly dynamic and only partially observable, which makes planning computationally ...
Motion planning for group movement simulation in dynamic environments
Many methods have been proposed to simulate group movement in complex environments with static obstacles. A critical part of this kind of simulation is the path planning method, which is used to prevent the moving objects from colliding with obstacles, ...
Planning in highly dynamic environments: an anytime approach for planning under time constraints
In this paper, we present a novel and domain-independent planner aimed at working in highly dynamic environments with time constraints. The planner follows the anytime principles: a first solution can be quickly computed and the quality of the final ...
Comments
Information & Contributors
Information
Published In
November 2022
39114 pages
ISBN:9781713871088
- Editors:
- S. Koyejo,
- S. Mohamed,
- A. Agarwal,
- D. Belgrave,
- K. Cho,
- A. Oh
Copyright © 2022 Neural Information Processing Systems Foundation, Inc.
Publisher
Curran Associates Inc.
Red Hook, NY, United States
Publication History
Published: 28 November 2022
Qualifiers
- Research-article
- Research
- Refereed limited
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025