research-article

Link-Adaptation for Improved Quality-of-Service in V2V Communication using Reinforcement Learning

Authors:

Serene Banerjee,

Sleeba Paul Puthepurakel,

Pratyush Kiran Uppuluri,

Subhadip Bandyopadhyay,

Y Sree Kanth Reddy,

Ranjani H. G.Authors Info & Claims

AIMLSystems '22: Proceedings of the Second International Conference on AI-ML Systems

Article No.: 1, Pages 1 - 7

https://doi.org/10.1145/3564121.3564122

Published: 16 May 2023 Publication History

Abstract

For autonomous driving, safer travel, and fleet management, Vehicle-to-Vehicle (V2V) communication protocols are an emerging area of research and development. State-of-the-art techniques include machine learning (ML) and reinforcement learning (RL) to adapt modulation and coding rates as the vehicle moves. However, channel state estimations are often incorrect and rapidly changing in a V2V scenario. We propose a combination of input features, including (a) sensor inputs from other parameters in the vehicle, such as speed and global positioning system (GPS), (b) estimation of interference and load for each of the vehicles, and (c) channel state estimation to find the optimal rate that would maximize Quality-of-Service. Our model uses an ensemble of RL-agents to predict trends in the input parameters and to find the inter-dependencies of these input parameters. An RL agent then utilizes these inputs to find the best modulation and coding rate as the vehicle moves. We demonstrate our results through prototype experiments using real data collected from customer networks.

References

[1]

R. C. Daniels, and R. W. Heath, “Link Adaptation with Position/Motion Information in Vehicle-to-Vehicle Networks”, in IEEE Trans. On Wireless Communications, vol. 11, no. 2, pp. 505—509, 2011.

[2]

K. M. Kord, A. Elbery, S. Sorour, H. S. Hassanein, A. B. Sediq, A. Afana, and H. Abou-Zeid, “Enhanced C-V2X Uplink Resource Allocation Using Vehicle Maneiuver Prediction”, in Proc. IEEE Int. Conf. On Communications, May 2022, Seoul, South Korea.

[3]

W. Xu, S, Guo, S. Ma, H. Zhou, M. Wi, and W. Zhuang, “Augmenting Drive-thru Internet via Reinforcement Learning-based Rate Adaptation”, in IEEE Journal on Internet of Things, vol. 7, no. 4, pp. 3114—3123, 2020.

[4]

L. Pellaco, V. Saxena, M. Bengtsson, and J. Jalden, “Wireless Link Adaptation with Outdated CSI – A Hybrid Data-driven and Model-based Approach”, in Proc. IEEE Int. Workshop on Signal Processing Advances in Wireless Communications, pp. 1—5, 2020.

[5]

R. F. Atallah, C. M. Assi, and M. J. Khabbaz, “Scheduling the Operation of a Connected Vehicular Network Using Deep Reinforcement Learning”, in IEEE Trans. On Intelligent Transportation Systems, vol. 20, no. 5, pp. 1669—1682, 2018.

[6]

H. Yang, X.-Y. Liu, S. Zhong, and A. Walid, “Deep Reinforcement Learning for Automated Stock Trading: An Ensemble Strategy”, in Proc. ACM Int. Conf. on AI in Finance, pp. 1—8, 2020.

Digital Library

[7]

K. Zhou, H. Wang, W. X. Zhao, Y. Zhu, S. Wang, F. Zhang, Z. Wang, and J.-R. Wen, “S3-rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization”, in Proc. ACM Int. Conf. on Information and Knowledge Management, pp. 1893—1902, 2020.

Digital Library

[8]

Z. Hu, K. Wan, X. Gao, and Y. Zhai, “A Dynamic Adjusting Reward Function Method for Deep Reinforcement Learning with Adjustable Parameters”, in Hindawi Mathematical Problems in Engineering, 2019.

[9]

M. O. Khan, and L. Qiu, “Accurate WiFi Packet Delivery Rate Estimation and Applications”, in Proc. IEEE Int. Conf. on Computer Communications (INFOCOM), pp. 1—9, 2016.

Digital Library

[10]

Z. MacHardy, A. Khan, K. Obana, and S. Iwashina, “V2X Access Technologies: Regulation, Research, and Remaining Challenges”, in IEEE Communications Surveys and Tutorials, vol. 20, no. 3, pp. 1858—1877, 2018.

[11]

J. B. Kenney, Toyota Info Technology Center, “Vehicle to Everything is paving the way for safer travel and cooperative autonomous driving”, IEEE Webinar Series: The Road to 6G, Sep. 23, 2021.

[12]

M. P. Mota, D. C. Araujo, F. H. C. Neto, A. LF de Almeida, and F. R. Cavalcanti, “Adaptive Modulation and Coding Based on Reinforcement Learning for 5G Networks”, in Proc. IEEE Int. Conf. on Globecom Workshops, pp. 1–6, Dec. 2019.

[13]

W. Jiang and H. D. Schotten, “Recurrent Neural Networks with Long Short-Term Memory for Fading Channel Prediction”, in Proc. IEEE Int. Conf. on Vehicular Technology (VTC-Spring), pp. 1—5, May 2020.

[14]

W. Jiang, H. D. Schotten, and J.-Y. Xiang, “Neural Network–Based Wireless Channel Prediction”, in Wiley Machine Learning for Future Wireless Communications, pp. 303—325, 2020.

[15]

W. Jiang, and H. D. Schotten, “Neural Network-Based Fading Channel Prediction: A Comprehensive Overview”, in IEEE Access, vol. 7, pp. 118112—118124, 2019.

[16]

R. Li, F. Li, X. Li, and Y. Wang, “QGrid: Q-learning Based Routing Protocol for Vehicular Ad Hoc Networks”, in Proc. IEEE Int. Conf. on Performance Computing and Communications (IPCCC), pp. 1—8, 2014.

[17]

B. Sliwa, R. Falkenberg, T. Liebig, J. Pillmann and C. Wietfeld, “Machine Learning Based Context-Predictive Car-to-Cloud Communication Using Multi-layer Connectivity Maps for Upcoming 5G Networks”, in Proc. IEEE Int. Conf. on Vehicular Technology (VTC-Fall), pp. 1—7, 2018.

[18]

L. Liang, H. Ye, and G. Y. Li, “Toward Intelligent Vehicular Networks: A Machine Learning Framework”, in IEEE Trans. On Internet of Things, vol. 6, no. 1, pp. 124—135, 2018.

[19]

T. Wang, A. Hussain, Y. Cao, and S. Gulomjon, “An Improved Channel Estimation Technique for IEEE 802.11 p Standard in Vehicular Communications”, in MDPI Sensors, vol. 19, no. 1, pp. 98, 2018.

[20]

J. Pillmann, B. Sliwa, C. Kastin, and C. Wietfeld, “Empirical Evaluation of Predictive Channel-Aware Transmission for Resource Efficient Car-to-Cloud Communication”, in Proc. IEEE Int. Conf. on Vehicular Networking (VNC), pp. 235—238, 2017.

[21]

D. Shukla, A. Prakash, and R. Tripathi, “Comparative Analysis of Channel Estimation Techniques in Vehicular Communication”, in Springer Advances in VLSI, Communication, and Signal Processing, pp. 679—688, 2021.

[22]

V2X Services Based on NR; User Equipment (UE) Radio Transmission and Reception (Release 16), Tech. Specification Group Radio Access Network (TSG RAN), 3GPP TR 38.886. 2019.

[23]

Study on Enhancement of 3GPP Support for 5G V2X Services (R16), Technical Specification Group Service & System Aspects, 3GPP TR 22.886. 2018.

[24]

S. Banerjee, and R. Martin, “Frequency-aware Time Series Forecasting, Anomaly Detection, Classification, and Granger-Causality”, in Proc. IEEE Int. Conf. on Communication Systems and Networks (COMSNETS), pp. 217—221, 2022.

[25]

M. Hussien, M. F. A. Ahmed, G. Dahman, K. K. Nguyen, M. Cheriet, and G. Poitau, “Towards More Reliable Deep Learning-Based Link Adaptation for WiFi 6”, in Proc. IEEE Int. Conf. on Communications (ICC), pp. 1—6, 2021.

[26]

Y. Guo, H. Shi, A. Kumar, K. Grauman, T. Rosing, and R. Feris, “Spottune: Transfer Learning Through Adaptive Fine-Tuning”, in Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 4805—4814, 2019.

[27]

Reinforcement Learning comparison of REINFORCE and Actor-Critic: https://github.com/nima-siboni/simplest-world-Actor-Critic.

Cited By

Feukeu EMbuyu S(2023)Machine Learning for Link Adaptation Problem Formulation in VANETs2023 17th International Conference on Telecommunication Systems, Services, and Applications (TSSA)10.1109/TSSA59948.2023.10366997(1-5)Online publication date: 12-Oct-2023
https://doi.org/10.1109/TSSA59948.2023.10366997
Feukeu EMbuyu S(2023)Application of Machine Learning for Link Adaptation in VANETs2023 IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI)10.1109/SOLI60636.2023.10425136(1-5)Online publication date: 11-Dec-2023
https://doi.org/10.1109/SOLI60636.2023.10425136

Index Terms

Link-Adaptation for Improved Quality-of-Service in V2V Communication using Reinforcement Learning
1. Software and its engineering
  1. Software creation and management
    1. Software development process management
      1. Software development methods
        Agile software development

Recommendations

An early collision warning algorithm for vehicles based on V2V communication

The WAVE (IEEE 1609) standard and vehicular technology create a new opportunity for telematics service. Safety is one of the top issues of telematics service. In this paper, we propose a proactive safety method that can alert drivers before the accident ...
Real-Time V2V Communication With a Machine Learning-Based System for Detecting Drowsiness of Drivers

This article applies machine learning to detect whether a driver is drowsy and alert the driver. The drowsiness of a driver can lead to accidents resulting in severe physical injuries, including deaths, and significant economic losses. Driver fatigue ...
V2V Communication-based AEB Validation in Traffic Accident Simulation Scenario
ICFEICT 2021: International Conference on Frontiers of Electronics, Information and Computation Technologies

Autonomous Emergency Braking (AEB) system can effectively avoid traffic accidents and reduce the degree of casualties. However, AEB systems based on traditional sensors will have blind spots and are greatly affected by environmental factors. This paper ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AIMLSystems '22: Proceedings of the Second International Conference on AI-ML Systems

October 2022

209 pages

ISBN:9781450398473

DOI:10.1145/3564121

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 May 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AIMLSystems 2022

AIMLSystems 2022: The Second International Conference on AI-ML Systems

October 12 - 15, 2022

Bangalore, India

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
66
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)3

Reflects downloads up to 12 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Feukeu EMbuyu S(2023)Machine Learning for Link Adaptation Problem Formulation in VANETs2023 17th International Conference on Telecommunication Systems, Services, and Applications (TSSA)10.1109/TSSA59948.2023.10366997(1-5)Online publication date: 12-Oct-2023
https://doi.org/10.1109/TSSA59948.2023.10366997
Feukeu EMbuyu S(2023)Application of Machine Learning for Link Adaptation in VANETs2023 IEEE International Conference on Service Operations and Logistics, and Informatics (SOLI)10.1109/SOLI60636.2023.10425136(1-5)Online publication date: 11-Dec-2023
https://doi.org/10.1109/SOLI60636.2023.10425136

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents