skip to main content
10.1145/3581791.3596836acmconferencesArticle/Chapter ViewAbstractPublication PagesmobisysConference Proceedingsconference-collections
research-article
Open access

Industrial Knee-jerk: In-Network Simultaneous Planning and Control on a TSN Switch

Published: 18 June 2023 Publication History

Abstract

Rapid advances in programmable network devices catalyzed the development of in-network computing, which is foreseen as a key enabler to empower the intelligence of production lines and mechanical arms in Industry 4.0. Various pioneering approaches have demonstrated the significant benefits of moving simple yet delay-sensitive industrial control tasks performed by servers to network switches. However, our detailed field study at a top-tier auto glass factory reveals that current practice fails to achieve a real-time and deterministic intelligent decision closure as leaving those complex yet essential planning tasks still on edge or cloud. In this paper, we design and implement a brand-new industrial switch, named Netopia, on a commercial Zynq platform through software and hardware co-design. Netopia enables planning and control to simultaneously perform on a network switch during communication. At the core of Netopia are three simple yet effective modules - a determinism guarantee mechanism, a computing acceleration scheme, and a packet deterministic forwarding framework that work hand-in-hand to ensure mechanical arms obtain intelligent control commands with low and deterministic latency. Comprehensive evaluations in industrial environments demonstrate that Netopia achieves an average end-to-end intelligent decision latency of 3.0ms with a jitter < 0.4ms, reduced by > 86% over existing works.

References

[1]
2015. Enhancements for Scheduled Traffic.
[2]
2020. Timing and Synchronization for Time-Sensitive Applications.
[3]
Mamta Agiwal, Abhishek Roy, and Navrati Saxena. 2016. Next generation 5G wireless networks: A comprehensive survey. IEEE Communications Surveys & Tutorials (2016).
[4]
Ali J Ben Ali, Marziye Kouroshli, Sofiya Semenova, Zakieh Sadat Hashemifar, Steven Y Ko, and Karthik Dantu. 2022. Edge-SLAM: edge-assisted visual simultaneous localization and mapping. ACM Transactions on Embedded Computing Systems (2022).
[5]
Sara Alonso, Jesus Lazaro, Jaime Jimenez, Leire Muguira, and Unai Bidarte. 2021. Evaluating the OpenAMP framework in real-time embedded SoC platforms. In 2021 XXXVI Conference on Design of Circuits and Integrated Systems (DCIS).
[6]
Andrea Bonci, Pangcheng David Cen Cheng, Marina Indri, Giacomo Nabissi, and Fiorella Sibona. 2021. Human-robot perception in industrial environments: A survey. Sensors (2021).
[7]
Ismail Butun, Yusuf Kursat Tuncel, and Kasim Oztoprak. 2021. Application Layer Packet Processing Using PISA Switches. Sensors (2021).
[8]
Tamás Czimmermann, Gastone Ciuti, Mario Milazzo, Marcello Chiurazzi, Stefano Roccella, Calogero Maria Oddo, and Paolo Dario. 2020. Visual-based defect detection and classification approaches for industrial applications---a survey. Sensors (2020).
[9]
Hongwen Dong, Kechen Song, Yu He, Jing Xu, Yunhui Yan, and Qinggang Meng. 2020. PGA-Net: Pyramid Feature Fusion and Global Context Attention Network for Automated Surface Defect Detection. IEEE Transactions on Industrial Informatics (2020).
[10]
Guoguang Du, Kai Wang, Shiguo Lian, and Kaiyong Zhao. 2021. Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review. Artificial Intelligence Review (2021).
[11]
Chelsea Finn, Ian Goodfellow, and Sergey Levine. 2016. Unsupervised learning for physical interaction through video prediction. Advances in neural information processing systems (2016).
[12]
Open Networking Foundation. 2022. Programming Protocol-independent Packet Processors. https://opennetworking.org/p4/
[13]
FRANKA. 2022. Minimum system and network requirements. https://frankaemika.github.io/docs/requirements.html
[14]
Nomios Group. 2022. What is white box switching? https://www.nomios.com/resources/white-box-switching/
[15]
Vivek Hajarnavis and Ken Young. 2008. An assessment of PLC software structure suitability for the support of flexible manufacturing processes. IEEE transactions on automation science and engineering (2008).
[16]
Xiaowu He, Xiangwen Zhuge, Fan Dang, Wang Xu, and Zheng Yang. 2023. Deep-Scheduler: Enabling Flow-Aware Scheduling in Time-Sensitive Networking. In IEEE INFOCOM 2023-IEEE Conference on Computer Communications. IEEE.
[17]
Yu He, Kechen Song, Qinggang Meng, and Yunhui Yan. 2019. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement (2019).
[18]
Ning Hu, Zhihong Tian, Xiaojiang Du, and Mohsen Guizani. 2021. An energy-efficient in-network computing paradigm for 6G. IEEE Transactions on Green Communications and Networking (2021).
[19]
NVIDIA Inc. 2022. Jetson TX2. https://www.nvidia.com/en-sg/autonomous-machines/embedded-systems/jetson-tx2/
[20]
Xilinx Inc. 2020. Petalinux - Xilinx wiki. https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842250/PetaLinux
[21]
Xilinx Inc. 2022. Flexible DSP Solutions. https://www.xilinx.com/products/technology/dsp.html
[22]
Xilinx Inc. 2022. Intellectual Property. https://www.xilinx.com/products/intellectual-property.html
[23]
Xilinx Inc. 2022. MPSoC PS and PL Ethernet Example Projects. https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/478937213/MPSoC+PS+and+PL+Ethernet+Example+Projects
[24]
Xilinx Inc. 2022. Tri-Mode Ethernet Media Access Controller (TEMAC). https://www.xilinx.com/products/intellectual-property/temac.html
[25]
Xilinx Inc. 2022. Zynq® UltraScale+™ MPSoC. https://www.xilinx.com/products/silicon-devices/soc/zynq-ultrascale-mpsoc.html
[26]
Somayeh Kianpisheh and Tarik Taleb. 2022. A Survey on In-network Computing: Programmable Data Plane And Technology Specific Applications. IEEE Communications Surveys & Tutorials (2022).
[27]
Changhoon Kim. 2016. Programming the network dataplane. ACM SIGCOMM: Florianopolis, Brazil (2016).
[28]
Ike Kunze, René Glebke, Jan Scheiper, Matthias Bodenbenner, Robert H Schmitt, and Klaus Wehrle. 2021. Investigating the applicability of in-network computing to industrial scenarios. In 2021 4th IEEE International Conference on Industrial Cyber-Physical Systems (ICPS).
[29]
Sándor Laki, Csaba Györgyi, József Pető, Péter Vörös, and Géza Szabó. 2022. In-Network Velocity Control of Industrial Robot Arms. In Proceedings of the USENIX NSDI.
[30]
Sander Lass and Norbert Gronau. 2020. A factory operating system for extending existing factories to Industry 4.0. Computers in industry (2020).
[31]
Sergey Levine, Peter Pastor, Alex Krizhevsky, Julian Ibarz, and Deirdre Quillen. 2018. Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. The International journal of robotics research (2018).
[32]
Haoran Li, Meng Xu, Chong Li, Chenyang Lu, Christopher Gill, Linh Phan, Insup Lee, and Oleg Sokolsky. 2021. Towards Virtualization-Agnostic Latency for Time-Sensitive Applications. In 29th International Conference on Real-Time Networks and Systems.
[33]
Luyang Liu, Hongyu Li, and Marco Gruteser. 2019. Edge assisted real-time object detection for mobile augmented reality. In The 25th annual international conference on mobile computing and networking.
[34]
Sicong Liu, Yingyan Lin, Zimu Zhou, Kaiming Nan, Hui Liu, and Junzhao Du. 2018. On-demand deep model compression for mobile devices: A usage-driven model selection framework. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services.
[35]
Zheng Liu, Hiroyuki Ukida, Pradeep Ramuhalli, and Kurt Niel. 2015. Integrated Imaging and Vision Techniques for Industrial Inspection. Advances in Computer Vision and Pattern Recognition (2015).
[36]
Ryosuke Okuda, Yuki Kajiwara, and Kazuaki Terashima. 2014. A survey of technical trend of ADAS and autonomous driving. In Technical Papers of 2014 International Symposium on VLSI Design, Automation and Test.
[37]
OpoenAMP. 2022. OpenAMP Project. https://www.openampproject.org/
[38]
Arthi Padmanabhan, Neil Agarwal, Anand Iyer, Ganesh Ananthanarayanan, Yuanchao Shu, Nikolaos Karianakis, Guoqing Harry Xu, and Ravi Netravali. 2023. GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge. In USENIX Symposium on Networked Systems Design and Implementation (NSDI).
[39]
Jorge Ribeiro, Rui Lima, Tiago Eckhardt, and Sara Paiva. 2021. Robotic process automation and artificial intelligence in industry 4.0--a literature review. Procedia Computer Science (2021).
[40]
Gerasimos G Rigatos. 2011. Modelling and control for intelligent industrial systems. adaptive algorithms in robotics and industrial engineering (2011).
[41]
Manasvini Sethuraman, Anirudh Sarma, Ashutosh Dhekne, and Umakishore Ramachandran. 2021. Foresight: planning for spatial and temporal variations in bandwidth for streaming services on mobile devices. In Proceedings of the 12th ACM Multimedia Systems Conference.
[42]
Alaa Sheta, Nazeeh Ghatasheh, Hossam Faris, and Ali Rodan. 2018. Robotics Evolution: from Remote Brain to Cloud. International Journal of Control and Automation (2018).
[43]
Arif Şirinterlikçi, Murat Tiryakioğlu, Adam Bird, Amie Harris, and Kevin Kweder. 2009. Repeatability and accuracy of an industrial robot: Laboratory experience for a design of experiments course. The Technology Interface Journal (2009).
[44]
Emiliano Sisinni, Abusayeed Saifullah, Song Han, Ulf Jennehag, and Mikael Gidlund. 2018. Industrial internet of things: Challenges, opportunities, and directions. IEEE transactions on industrial informatics (2018).
[45]
PLOS ONE Staff. 2016. Retraction: biomechanical characteristics of hand coordination in grasping activities of daily living.
[46]
Guangzhi Tang and Konstantinos P Michmizos. 2018. Gridbot: An autonomous robot controlled by a spiking neural network mimicking the brain's navigational system. In Proceedings of the International Conference on Neuromorphic Systems.
[47]
Yunchao Tang, Mingyou Chen, Chenglin Wang, Lufeng Luo, Jinhui Li, Guoping Lian, and Xiangjun Zou. 2020. Recognition and localization methods for vision-based fruit picking robots: A review. Frontiers in Plant Science (2020).
[48]
Akshay Thirugnanam, Jun Zeng, and Koushil Sreenath. 2022. Safety-Critical Control and Planning for Obstacle Avoidance between Polytopes with Control Barrier Functions. In IEEE International Conference on Robotics and Automation.
[49]
Axel Vick, Vojtěch Vonásek, Robert Pěnička, and Jörg Krüger. 2015. Robot control as a service---towards cloud-based motion planning and control for industrial robots. In 2015 10th International Workshop on Robot Motion and Control (RoMoCo). IEEE, 33--39.
[50]
Michal Vocetka, Róbert Huňady, Martin Hagara, Zdenko Bobovskỳ, Tomáš Kot, and Václav Krys. 2020. Influence of the Approach Direction on the Repeatability of an Industrial Robot. Applied Sciences (2020).
[51]
Daniel Weimer, Bernd Scholz-Reiter, and Moshe Shpitalni. 2016. Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. CIRP annals (2016).
[52]
Patrick Wenzel, Torsten Schön, Laura Leal-Taixé, and Daniel Cremers. 2021. Vision-based mobile robotics obstacle avoidance with deep reinforcement learning. In 2021 IEEE International Conference on Robotics and Automation (ICRA).
[53]
Wikipedia. 2022. CentOS. https://en.wikipedia.org/w/index.php?title=CentOS&oldid=1118193821
[54]
Wikipedia. 2022. Debian. https://en.wikipedia.org/w/index.php?title=Debian&oldid=1126272347
[55]
Wikipedia. 2022. Direct memory access. https://en.wikipedia.org/w/index.php?title=Direct_memory_access&oldid=1123461808
[56]
Wikipedia. 2022. Docker (software). https://en.wikipedia.org/w/index.php?title=Docker_(software)&oldid=1123196945
[57]
Wikipedia. 2022. EtherCAT. https://en.wikipedia.org/w/index.php?title=EtherCAT&oldid=1120354882.
[58]
Wikipedia. 2022. Inverse kinematics. https://en.wikipedia.org/w/index.php?title=Inverse_kinematics&oldid=1124543632.
[59]
Wikipedia. 2022. Modbus. https://en.wikipedia.org/w/index.php?title=Modbus&oldid=1125273307
[60]
Wikipedia. 2022. Network Time Protocol. https://en.wikipedia.org/w/index.php?title=Network_Time_Protocol&oldid=1125151433
[61]
Wikipedia. 2022. Patellar reflex. https://en.wikipedia.org/w/index.php?title=Patellar_reflex&oldid=1069824082
[62]
Wikipedia. 2022. PID controller. https://en.wikipedia.org/w/index.php?title=PID_controller&oldid=1121264197.
[63]
Wikipedia. 2022. Pipelining (DSP implementation). https://en.wikipedia.org/w/index.php?title=Pipelining_(DSP_implementation)&oldid=1121275138
[64]
Wikipedia. 2022. Profinet --- Wikipedia, The Free Encyclopedia. https://en.wikipedia.org/w/index.php?title=Profinet&oldid=1118142114
[65]
Wikipedia. 2022. Tegra. https://en.wikipedia.org/w/index.php?title=Tegra&oldid=1123369665
[66]
Wikipedia. 2022. Virtualization. https://en.wikipedia.org/w/index.php?title=Virtualization&oldid=1116535435
[67]
Wikipedia. 2022. Xenomai. https://en.wikipedia.org/w/index.php?title=Xenomai&oldid=1109190278
[68]
Jingao Xu, Hao Cao, Zheng Yang, Longfei Shangguan, Jialin Zhang, Xiaowu He, and Yunhao Liu. 2022. SwarmMap: Scaling Up Real-time Collaborative Visual SLAM at the Edge. In Proceedings of the USENIX NSDI. 977--993.
[69]
Mengwei Xu, Zhe Fu, Xiao Ma, Li Zhang, Yanan Li, Feng Qian, Shangguang Wang, Ke Li, Jingyu Yang, and Xuanzhe Liu. 2021. From cloud to edge: a first look at public edge platforms. In Proceedings of the 21st ACM Internet Measurement Conference.
[70]
Zheng Yang, Yi Zhao, Fan Dang, Xiaowu He, Jiahang Wu, Hao Cao, Zeyu Wang, and Yunhao Liu. 2023. CaaS: Enabling Control-as-a-Service for Time-Sensitive Networking. In IEEE INFOCOM 2023-IEEE Conference on Computer Communications. IEEE.
[71]
Han Zhang, Abhijith Anilkumar, Matt Fredrikson, and Yuvraj Agarwal. 2021. Capture: Centralized Library Management for Heterogeneous {IoT} Devices. In 30th USENIX Security Symposium (USENIX Security 21).
[72]
Jialin Zhang, Xiang Huang, Jingao Xu, Yue Wu, Qiang Ma, Xin Miao, Li Zhang, Pengpeng Chen, and Zheng Yang. 2022. Edge Assisted Real-time Instance Segmentation on Mobile Devices. In Proceedings of the IEEE ICDCS.
[73]
Yi Zhao, Zheng Yang, Xiaowu He, Jiahang Wu, Hao Cao, Liang Dong, Fan Dang, and Yunhao Liu. 2022. E-TSN: Enabling Event-triggered Critical Traffic in Time-Sensitive Networking for Industrial Applications. In 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS). 691--701.
[74]
Ruogu Zhou and Guoliang Xing. 2014. nshield: A noninvasive nfc security system for mobiledevices. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services.
[75]
Fengyuan Zhu, Mingwei Ouyang, Luwei Feng, Yaoyu Liu, Xiaohua Tian, Meng Jin, Dongyao Chen, and Xinbing Wang. 2022. Enabling software-defined PHY for backscatter networks. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services.

Cited By

View all

Index Terms

  1. Industrial Knee-jerk: In-Network Simultaneous Planning and Control on a TSN Switch

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Conferences
        MobiSys '23: Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services
        June 2023
        651 pages
        ISBN:9798400701108
        DOI:10.1145/3581791
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Sponsors

        In-Cooperation

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 18 June 2023

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. time-sensitive networking
        2. in-network computing
        3. industrial control

        Qualifiers

        • Research-article

        Conference

        MobiSys '23
        Sponsor:

        Acceptance Rates

        MobiSys '23 Paper Acceptance Rate 41 of 198 submissions, 21%;
        Overall Acceptance Rate 274 of 1,679 submissions, 16%

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)530
        • Downloads (Last 6 weeks)86
        Reflects downloads up to 23 Dec 2024

        Other Metrics

        Citations

        Cited By

        View all

        View Options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Login options

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media