Digital Twin and Meta RL Empowered Fast-Adaptation of Joint User Scheduling and Task Offloading for Mobile Industrial IoT
Pages 3254 - 3266
Abstract
The industrial Internet of Things (IoT) system is integrated with the emerging artificial intelligence (AI) paradigms to empower industrial automation and self-evolving capabilities. AI-driven resource allocation across cyber-physical domains for mobile industrial IoT must consider its fundamental requirements and key characteristics such as high reliability, low latency, and environmental dynamics. The challenge is twofold. Industrial systems are fault-sensitive, which makes them intolerable of trial-and-error-based learning and optimization approaches. In addition, learning models cannot adapt to changing industrial IoT environment with dynamic communication noise and machinery disturbances. In this paper, we propose joint optimization for the nonorthogonal multiple access (NOMA) and multi-tier hybrid cloud-edge computing empowered industrial IoT that results in improved utilization of communication and computing resources. Second, we establish the fine-grained digital twin for industrial IoT (DT-IIoT) to simulate the changing industrial environment to support trial-and-error-based safe learning. Third, we leverage meta reinforcement learning (meta RL) to improve the generalization and fast adaptation of the learning models for DT-IIoT. Finally, the feasibility and efficiency of these schemes are evaluated through extensive experiments.
References
[1]
O. San, “The digital twin revolution,” Nature Comput. Sci., vol. 1, no. 5, pp. 307–308, May 2021.
[2]
H. Xu, J. Wu, Q. Pan, X. Guan, and M. Guizani, “A survey on digital twin for industrial Internet of Things: Applications, technologies and tools,” IEEE Commun. Surveys Tuts., early access, Jul. 20, 2023. 10.1109/COMST.2023.3297395.
[3]
F. Tao, H. Zhang, A. Liu, and A. Y. C. Nee, “Digital twin in industry: State-of-the-art,” IEEE Trans. Ind. Informat., vol. 15, no. 4, pp. 2405–2415, Apr. 2019.
[4]
Z. Zhouet al., “Secure and latency-aware digital twin assisted resource scheduling for 5G edge computing-empowered distribution grids,” IEEE Trans. Ind. Informat., vol. 18, no. 7, pp. 4933–4943, Jul. 2022.
[5]
L. U. Khan, Z. Han, W. Saad, E. Hossain, M. Guizani, and C. S. Hong, “Digital twin of wireless systems: Overview, taxonomy, challenges, and opportunities,” 2022, arXiv:2202.02559.
[6]
H. Xu, X. Liu, W. Yu, D. Griffith, and N. Golmie, “Reinforcement learning-based control and networking co-design for industrial Internet of Things,” IEEE J. Sel. Areas Commun., vol. 38, no. 5, pp. 885–898, May 2020.
[7]
S. Kim, K.-J. Park, and C. Lu, “A survey on network security for cyber-physical systems: From threats to resilient design,” IEEE Commun. Surveys Tuts., vol. 24, no. 3, pp. 1534–1573, 3rd Quart., 2022.
[8]
Y. Ma, C. Lu, and Y. Wang, “Efficient holistic control: Self-awareness across controllers and wireless networks,” ACM Trans. Cyber-Phys. Syst., vol. 4, no. 4, pp. 1–27, Oct. 2020.
[9]
K. Wang, Y. Zhou, Z. Liu, Z. Shao, X. Luo, and Y. Yang, “Online task scheduling and resource allocation for intelligent NOMA-based industrial Internet of Things,” IEEE J. Sel. Areas Commun., vol. 38, no. 5, pp. 803–815, May 2020.
[10]
Y. Yang, “Multi-tier computing networks for intelligent IoT,” Nature Electron., vol. 2, no. 1, pp. 4–5, Jan. 2019.
[11]
H. Xu, J. Wu, J. Li, and X. Lin, “Deep-reinforcement-learning-based cybertwin architecture for 6G IIoT: An integrated design of control, communication, and computing,” IEEE Internet Things J., vol. 8, no. 22, pp. 16337–16348, Nov. 2021.
[12]
L. Liu, S. Liu, and W. Shi, “4C: A computation, communication, and control co-design framework for CAVs,” IEEE Wireless Commun., vol. 28, no. 4, pp. 42–48, Aug. 2021.
[13]
H. Xu, J. Wu, H. Pan, J. Gu, and X. Guan, “Delay safety-aware digital twin empowered industrial sensing-actuation systems using transferable and reinforced learning,” IEEE Trans. Ind. Informat., early access, Feb. 2, 2023. 10.1109/TII.2023.3241616.
[14]
Q. Pan, J. Wu, A. K. Bashir, J. Li, W. Yang, and Y. D. Al-Otaibi, “Joint protection of energy security and information privacy for energy harvesting: An incentive federated learning approach,” IEEE Trans. Ind. Informat., vol. 18, no. 5, pp. 3473–3483, May 2022.
[15]
K. Zhang, J. Cao, and Y. Zhang, “Adaptive digital twin and multiagent deep reinforcement learning for vehicular edge computing and networks,” IEEE Trans. Ind. Informat., vol. 18, no. 2, pp. 1405–1413, Feb. 2022.
[16]
Y. He, M. Yang, Z. He, and M. Guizani, “Resource allocation based on digital twin-enabled federated learning framework in heterogeneous cellular network,” IEEE Trans. Veh. Technol., vol. 72, no. 1, pp. 1149–1158, Jan. 2023. 10.1109/TVT.2022.3205778.
[17]
Y. Yuan, G. Zheng, K.-K. Wong, and K. B. Letaief, “Meta-reinforcement learning based resource allocation for dynamic V2X communications,” IEEE Trans. Veh. Technol., vol. 70, no. 9, pp. 8964–8977, Sep. 2021.
[18]
C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proc. Int. Conf. Mach. Learn., 2017, pp. 1126–1135.
[19]
X. Zang, H. Yao, G. Zheng, N. Xu, K. Xu, and Z. Li, “MetaLight: Value-based meta-reinforcement learning for traffic signal control,” in Proc. AAAI Conf. Artif. Intell., vol. 34, no. 1, 2020, pp. 1153–1160.
[20]
T. Z. Zhaoet al., “Offline meta-reinforcement learning for industrial insertion,” in Proc. Int. Conf. Robot. Autom. (ICRA), May 2022, pp. 6386–6393.
[21]
A. Gupta, R. Mendonca, Y. Liu, P. Abbeel, and S. Levine, “Meta-reinforcement learning of structured exploration strategies,” in Proc. Adv. Neural Inf. Process. Syst., vol. 31, 2018, pp. 1–10.
[22]
X. Liu, W. Yu, F. Liang, D. Griffith, and N. Golmie, “On deep reinforcement learning security for industrial Internet of Things,” Comput. Commun., vol. 168, pp. 20–32, Feb. 2021.
[23]
Y. Maet al., “Optimal dynamic transmission scheduling for wireless networked control systems,” IEEE Trans. Control Syst. Technol., vol. 30, no. 6, pp. 2360–2376, Nov. 2022.
[24]
F. Tao, J. Cheng, Q. Qi, M. Zhang, H. Zhang, and F. Sui, “Digital twin-driven product design, manufacturing and service with big data,” Int. J. Adv. Manuf. Technol., vol. 94, nos. 9–12, pp. 3563–3576, Feb. 2018.
[25]
J. Cheng, H. Zhang, F. Tao, and C.-F. Juang, “DT-II: Digital twin enhanced industrial Internet reference framework towards smart manufacturing,” Robot. Comput.-Integr. Manuf., vol. 62, Apr. 2020, Art. no.
[26]
Z. Müller-Zhang, P. O. Antonino, and T. Kuhn, “Dynamic process planning using digital twins and reinforcement learning,” in Proc. 25th IEEE Int. Conf. Emerg. Technol. Factory Autom. (ETFA), vol. 1, Sep. 2020, pp. 1757–1764.
[27]
M. D. Anis, S. Taghipour, and C.-G. Lee, “Optimal RUL estimation: A state-of-art digital twin application,” in Proc. Annu. Rel. Maintainability Symp. (RAMS), Jan. 2020, pp. 1–7.
[28]
V. H. Pong, A. V. Nair, L. Smith, C. Huang, and S. Levine, “Offline meta-reinforcement learning with online self-supervision,” in Proc. 39th Int. Conf. Mach. Learn., Jul. 2022, pp. 17811–17829.
[29]
J. Becket al., “A survey of meta-reinforcement learning,” 2023, arXiv:2301.08028.
[30]
K. Rakelly, A. Zhou, C. Finn, S. Levine, and D. Quillen, “Efficient off-policy meta-reinforcement learning via probabilistic context variables,” in Proc. 36th Int. Conf. Mach. Learn., 2019, pp. 5331–5340.
Index Terms
- Digital Twin and Meta RL Empowered Fast-Adaptation of Joint User Scheduling and Task Offloading for Mobile Industrial IoT
Index terms have been assigned to the content through auto-classification.
Recommendations
Energy-Efficient UAV Scheduling and Probabilistic Task Offloading for Digital Twin-Empowered Consumer Electronics Industry
The digital twin (DT)–powered consumer electronics industry has been proposed to create virtual representations of the physical products, manufacturing processes, and manufacturing systems, and to enable manufacturers to truly promote the ...
Blockchain-based trust mechanism for digital twin empowered Industrial Internet of Things
AbstractEmerging technologies such as blockchain and digital twins are essential for the rapid development and employment in Industry 5.0 revolution. With the growing number of industrial IoT nodes, optimizing the network and availing of ...
Highlights- Proposed blockchain-based trust mechanism to reduce energy issues.
- Explored the ...
A Study on Intelligent Manufacturing Industrial Internet for Injection Molding Industry Based on Digital Twin
As one of the basic manufacturing industries in China, injection molding industry is faced with the problems of low degree of informatization and intelligence, resulting in low production efficiency and high costs. It is urgent to integrate deeply with ...
Comments
Information & Contributors
Information
Published In
0733-8716 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 04 September 2023
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 06 Jan 2025