Downlink Scheduling in LTE with Deep Reinforcement Learning, LSTMs and Pointers
Pages 763 - 770
Abstract
Downlink scheduling in the LTE system is an open problem for which several heuristic solutions exist. Recently, there has been an increase in interest in applying machine learning to networking problems, including downlink scheduling. We propose a LSTM/Pointer Network-based downlink scheduler which flexibly handles changing numbers of UEs via the use of a recurrent neural network. We integrate the channel quality indicator and the buffer size of each UE as the observation and train the network using a Deep Reinforcement Learning algorithm. Our experiments demonstrate that our approach results in a scheduler which generalised across changing number of UEs and resource blocks and performed within the range of traditional schedulers.
References
[1]
F. Al-Tam, N. Correia, and J. Rodriguez, “Learn to schedule (LEASCH): A deep reinforcement learning approach for radio resource scheduling in the 5G MAC Layer,” IEEE Access, vol. 8, pp. 108088–108101, 2020.
[2]
Y. Abiko, D. Mochizuki, T. Saito, D. Ikeda, T. Mizuno, and H. Mineno, “Proposal of allocating radio resources to multiple slices in 5G using deep reinforcement learning,” 2019 IEEE 8th Global Conference on Consumer Electronics, GCCE 2019, no. 17, pp. 129–130, 2019.
[3]
J. Koo, V. B. Mendiratta, M. R. Rahman, and A. Walid, “Deep reinforcement learning for network slicing with heterogeneous resource requirements and time varying traffic dynamics,” 15th International Conference on Network and Service Management, CNSM 2019, 2019.
[4]
R. Li, Z. Zhao, Q. Sun, C.-L. I, C. Yang, X. Chen, M. Zhao, and H. Zhang, “Deep reinforcement learning for resource management in network slicing,” IEEE Journal on Selected Areas in Communications, vol. 38, no. 2, pp. 334–349, 2018. [Online]. Available: http://arxiv.org/abs/1805.06591.
[5]
D. Horgan, J. Quan, D. Budden, G. Barth-Maron, M. Hessel, H. Van Hasselt, and D. Silver, “Distributed prioritized experience replay,” 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, pp. 1–19, 2018.
[6]
R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, “Policy gradient methods for reinforcement learning with function approximation,” Advances in neural information processing systems, pp. 1057–1063, 2000.
[7]
H. Mao, M. Alizadeh, I. Menache, and S. Kandula, “Resource management with deep reinforcement learning,” HotNets 2016 - Proceedings of the 15th ACM Workshop on Hot Topics in Networks, pp. 50–56, 2016.
[8]
J. Wang, C. Xu, Y. Huangfu, R. Li, Y. Ge, and J. Wang, “Deep reinforcement learning for scheduling in cellular networks,” 2019 11th International Conference on Wireless Communications and Signal Processing, WCSP 2019, 2019.
[9]
N. Sharma, S. Zhang, S. R. Somayajula Venkata, F. Malandra, N. Mas-tronarde, and J. Chakareski, “Deep reinforcement learning for delay-sensitive LTE downlink scheduling,” in 2020 IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications, 2020, pp. 1–6.
[10]
R. Pascanu, T. Mikolov, and Y. Bengio, “On the difficulty of training recurrent neural networks,” in Proceedings of the 30th International Conference on International Conference on Machine Learning - Volume 28, ser. ICML'13. JMLR.org, 2013, p. III-1310–III-1318.
[11]
S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
[12]
O. Vinyals, M. Fortunato, and N. Jaitly, “Pointer networks,” in Advances in Neural Information Processing Systems, C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, Eds., vol. 28. Curran Associates, Inc., 2015. [Online]. Available: https://proceedings.neurips.cc/paper/2015/file/29921001f2f04bd3baee84a12e98098f-Paper.pdf.
[13]
M. Plappert, R. Houthooft, P. Dhariwal, S. Sidor, R. Y. Chen, X. Chen, T. Asfour, P. Abbeel, and M. Andrychowicz, “Parameter space noise for exploration,” in 6th International Conference on Learning Representation, ICLR 2018 - Conference Track Proceedings, 2018, pp. 1–18.
[14]
T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings, 2016.
[15]
P. Gawlowicz and A. Zubow, “Ns-3 meets OpenAI Gym: The playground for machine learning in networking research,” MSWiM 2019 - Proceedings of the 22nd International ACM Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, pp. 113–120, 2019.
[16]
G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “OpenAI Gym,” CoRR, vol. arXiv:, 2016. [Online]. Available: http://arxiv.org/abs/1606.01540.
[17]
O. Vinyals, S. Bengio, and M. Kudlur, “Order matters: Sequence to sequence for sets,” 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings, pp. 1–11, 2016.
[18]
S. Mehri and L. Sigal, “Middle-out decoding,” in Proceedings of the 32nd International Conference on Neural Information Processing Systems, ser. NIPS'18. Curran Associates Inc., 2018, p. 5523–5534.
Index Terms
- Downlink Scheduling in LTE with Deep Reinforcement Learning, LSTMs and Pointers
Index terms have been assigned to the content through auto-classification.
Recommendations
On the performance of LTE downlink scheduling algorithms: A case study on edge throughput
Highlights- The main objective of the proposed scheduling algorithm is to increase the edge throughput without sacrificing cell throughput in LTE networks.
AbstractRadio resource allocation is a crucial task in the LTE networks. To increase the overall user experience, an efficient radio resource allocation algorithm should be utilized. In this work, a new scheduling algorithm has been proposed ...
Multi Objective Resource Scheduling in LTE Networks Using Reinforcement Learning
The use of the intelligent packet scheduling process is absolutely necessary in order to make the radio resources usage more efficient in recent high-bit-rate demanding radio access technologies such as Long Term Evolution LTE. Packet scheduling ...
A Novel Packet Scheduling Scheme for Downlink LTE System
IIH-MSP '11: Proceedings of the 2011 Seventh International Conference on Intelligent Information Hiding and Multimedia Signal ProcessingLong term evolution (LTE) is the next generation wireless system. There are not many researches for LTE downlink scheduling. It uses orthogonal frequency division multiple access (OFDMA) in downlink. Until now, the goal for the LTE scheduler is ...
Comments
Information & Contributors
Information
Published In
Nov 2021
1016 pages
Copyright © 2021.
Publisher
IEEE Press
Publication History
Published: 29 November 2021
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 16 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in