research-article

Live Migration of Operating System Containers in Encrypted Virtual Machines

Authors:

Sascha WesselAuthors Info & Claims

CCSW '21: Proceedings of the 2021 on Cloud Computing Security Workshop

Pages 125 - 137

https://doi.org/10.1145/3474123.3486761

Published: 15 November 2021 Publication History

Abstract

With the widespread use of Docker and Kubernetes, OS-level virtualization has become a key technology to deploy and run software. At the same time, data centers and cloud providers offer shared computing resources on demand. The use of these resources usually leads to a larger trusted computing base and less control over the data.

We present a confidential computing concept for the migration of operating system containers in secure encrypted virtual machines so that these are protected from the operator and administrator. In our approach, processes inside of the containers remain intact, i.e., they keep their state and do not have to be restarted. Network services inside of the containers remain unchanged and reachable. This is typically called live migration. Integrity and confidentiality of the data inside of the containers is enforced during migration as well as on the destination platform, namely in transit, in use and at rest. The authenticity and integrity of the destination platform is verified using remote attestation before any data is transferred.

While our core concept is not specific to a particular hardware, we present two different approaches corresponding to the first generation of AMD SEV as well as SEV-SNP. Our proof of concept implementation is based on the first generation of SEV.

References

[1]

2015. Intel®64 and IA-32 Architectures Software Developer's Manual. Technical Report 332831-056US. Intel.

[2]

2020. AMD SEV-SNP: Strengthening VM Isolation with Integrity Protection and More. Technical Report. AMD.

[3]

2020. Intel®Trust Domain Extensions. Technical Report 343961-002US. Intel.

[4]

2020. Secure Encrypted Virtualization API Version 0.24. Technical Report # 55766. AMD.

[5]

2021. Arm CCA Security Model 1.0. Technical Report DEN0096. Arm.

[6]

2021. SEV-ES Guest-Hypervisor Communication Block Standardization. Technical Report # 56421. AMD.

[7]

2021. SEV Secure Nested Paging Firmware ABI Specification. Technical Report # 56860. AMD

[8]

Fritz Alder, Arseny Kurnikov, Andrew Paverd, and N Asokan. 2018. Migrating SGX enclaves with persistent state. In 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, 195--206.

[9]

AMDESE. 2018. AMDESE/sev-tool. https://github.com/AMDESE/sev-tool/ Retrieved September 13, 2021 from

[10]

Arm. 2021. Arm Confidential Compute Architecture. https://www.arm.com/why-arm/architecture/security-features/arm-confidential-compute-architecture Retrieved June 27, 2021 from

[11]

Sergei Arnautov, Bohdan Trach, Franz Gregor, Thomas Knauth, Andre Martin, Christian Priebe, Joshua Lind, Divya Muthukumaran, Dan O'keeffe, Mark L Stillwell, et al. 2016. SCONE: Secure Linux Containers with Intel SGX. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). 689--703.

Digital Library

[12]

Robert Buhren, Hans-Niklas Jacob, Thilo Krachenfels, and Jean-Pierre Seifert. 2021. One Glitch to Rule Them All: Fault Injection Attacks Against AMD's Secure Encrypted Virtualization. arXiv preprint arXiv:2108.04575 (2021).

[13]

Robert Buhren, Christian Werling, and Jean-Pierre Seifert. 2019. Insecure Until Proven Updated: Analyzing AMD SEV's Remote Attestation. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 1087--1099.

Digital Library

[14]

D. Chandravathi and P.V. Lakshmi. 2018. Performance Analysis of Homomorphic Encryption algorithms for Cloud Data Security. International Journal for Research in Applied Science and Engineering Technology, Vol. 6 (03 2018). https://doi.org/10.22214/ijraset.2018.3243

[15]

Chia che Tsai, Donald E. Porter, and Mona Vij. 2017. Graphene-SGX: A Practical Library OS for Unmodified Applications on SGX. In 2017 USENIX Annual Technical Conference (USENIX ATC 17). USENIX Association, Santa Clara, CA, 645--658. https://www.usenix.org/conference/atc17/technical-sessions/presentation/tsai

Digital Library

[16]

Christopher Clark, Keir Fraser, Steven Hand, Jacob Gorm Hansen, Eric Jul, Christian Limpach, Ian Pratt, and Andrew Warfield. 2005. Live migration of virtual machines. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation-Volume 2. 273--286.

Digital Library

[17]

Jason A Donenfeld. 2017. WireGuard: Next Generation Kernel Network Tunnel. In NDSS.

[18]

F. B. Dubach and C. M. Shub. 1989. Process-Originated Migration in a Heterogeneous Environment. In Proceedings of the 17th Conference on ACM Annual Computer Science Conference (Louisville, Kentucky) (CSC '89). Association for Computing Machinery, New York, NY, USA, 98--102. https://doi.org/10.1145/75427.75437

Digital Library

[19]

Tobin Feldman-Fitzthum. 2020. RFC: Fast Migration for SEV and SEV-ES - blueprint and proof of concept. https://edk2.groups.io/g/devel/topic/77875297 Retrieved September 13, 2021 from

[20]

Simon Frost. 2021. arm CCA Attestation. https://connect.linaro.org/resources/armcca/attestation-architecture/ Retrieved September 14, 2021 from

[21]

Keerthana Govindaraj and Alexander Artemenko. 2018. Container live migration for latency critical industrial applications on edge computing. In 2018 IEEE 23rd International Conference on Emerging Technologies and Factory Automation (ETFA), Vol. 1. IEEE, 83--90.

Digital Library

[22]

Jinyu Gu, Zhichao Hua, Yubin Xia, Haibo Chen, Binyu Zang, Haibing Guan, and Jinming Li. 2017. Secure live migration of SGX enclaves on untrusted cloud. In 2017 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, 225--236.

[23]

Felicitas Hetzelt and Robert Buhren. 2017. Security analysis of encrypted virtual machines. ACM SIGPLAN Notices, Vol. 52, 7 (2017), 129--142.

Digital Library

[24]

Marcio de Lima E Silva Jim Cadden and Hubertus Franke. 2020. Leveraging AMD SEV in the IBM Hybrid Cloud. https://www.ibm.com/blogs/research/2020/11/amd-sev-ibm-hybrid-cloud/ Retrieved September 13, 2021 from

[25]

David Kaplan. 2017. PROTECTING VM REGISTER STATE WITH SEV-ES. Technical Report. AMD.

[26]

David Kaplan, Jeremy Powell, and Tom Woller. 2016. AMD Memory Encryption. Technical Report. AMD.

[27]

P Getzi Jeba Leelipushpam and J Sharmila. 2013. Live VM migration techniques in cloud environment-a survey. In 2013 IEEE Conference on Information & Communication Technologies. IEEE, 408--413.

[28]

Mengyuan Li, Yinqian Zhang, and Zhiqiang Lin. 2020. CROSSLINE: Breaking "Security-by-Crash" based Memory Isolation in AMD SEV. arXiv preprint arXiv:2008.00146 (2020).

[29]

Mengyuan Li, Yinqian Zhang, Huibo Wang, Kang Li, and Yueqiang Cheng. 2021. CIPHERLEAKS: Breaking Constant-time Cryptography on AMD SEV via the Ciphertext Side Channel. In 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 717--732. https://www.usenix.org/conference/usenixsecurity21/presentation/li-mengyuan

[30]

Lele Ma, Shanhe Yi, and Qun Li. 2017a. Efficient service handoff across edge servers via docker container migration. In Proceedings of the Second ACM/IEEE Symposium on Edge Computing. 1--13.

Digital Library

[31]

Lele Ma, Shanhe Yi, and Qun Li. 2017b. Efficient service handoff across edge servers via docker container migration. In Proceedings of the Second ACM/IEEE Symposium on Edge Computing. 1--13.

Digital Library

[32]

Paulo Martins, Leonel Sousa, and Artur Mariano. 2017. A survey on fully homomorphic encryption: An engineering perspective. ACM Computing Surveys (CSUR), Vol. 50, 6 (2017), 1--33.

Digital Library

[33]

Dejan S Milojivc ić, Fred Douglis, Yves Paindaveine, Richard Wheeler, and Songnian Zhou. 2000. Process migration. ACM Computing Surveys (CSUR), Vol. 32, 3 (2000), 241--299.

Digital Library

[34]

Saeid Mofrad, Fengwei Zhang, Shiyong Lu, and Weidong Shi. 2018. A Comparison Study of Intel SGX and AMD Memory Encryption Technology. In Proceedings of the 7th International Workshop on Hardware and Architectural Support for Security and Privacy. 1--8.

Digital Library

[35]

Mathias Morbitzer, Manuel Huber, Julian Horsch, and Sascha Wessel. 2018. Severed: Subverting AMD's Virtual Machine Encryption. In Proceedings of the 11th European Workshop on Systems Security. 1--6.

Digital Library

[36]

Cong Nie. 2007. Dynamic root of trust in trusted computing. In TKK T1105290 Seminar on Network Security. Citeseer.

[37]

Nelly Porter, Gilad Golan, and Sam Lugani. 2020. Introducing Google Cloud Confidential Computing with Confidential VMs. https://cloud.google.com/blog/products/identity-security/introducing-google-cloud-confidential-computing-with-confidential-vms Retrieved September 13, 2021 from

[38]

Alessandro Randazzo and Ilenia Tinnirello. 2019. Kata containers: An emerging architecture for enabling mec services in fast and secure way. In 2019 Sixth International Conference on Internet of Things: Systems, Management and Security (IOTSMS). IEEE, 209--214.

[39]

Adrian Reber. 2019. Container migration with Podman on RHEL.

[40]

Mark Russinovich. 2021. Azure and AMD announce landmark in confidential computing evolution. https://azure.microsoft.com/en-us/blog/azure-and-amd-enable-lift-and-shift-confidential-computing/ Retrieved September 13, 2021 from

[41]

Gulshan Soni and Mala Kalra. 2013. Comparative study of live virtual machine migration techniques in cloud. International Journal of Computer Applications, Vol. 84, 14 (2013).

Cited By

Altahat MDaradkeh TAgarwal A(2024)Optimized Encryption-Integrated Strategy for Containers Scheduling and Secure Migration in Multi-Cloud Data CentersIEEE Access10.1109/ACCESS.2024.338616912(51330-51345)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3386169
Pathak PNadeem MAnsar S(2024)Security assessment of operating system by using decision making algorithmsInternational Journal of Information Technology10.1007/s41870-023-01706-9Online publication date: 19-Jan-2024
https://doi.org/10.1007/s41870-023-01706-9
Gupta ANamasudra SKumar P(2024)A secure VM live migration technique in a cloud computing environment using blowfish and blockchain technologyThe Journal of Supercomputing10.1007/s11227-024-06461-780:19(27370-27393)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s11227-024-06461-7
Show More Cited By

Index Terms

Live Migration of Operating System Containers in Encrypted Virtual Machines
1. Security and privacy
  1. Systems security
    1. Operating systems security
      1. Virtualization and security

Recommendations

Performance Analysis for Pareto-Optimal Green Consolidation Based on Virtual Machines Live Migration

Huge energy requirement of cloud data centers is prime concern. Dynamic Virtual Machine VM consolidation based on VM live migration to switched-off or put some of the under-loaded host Physical Machines PMs into a low power consumption mode can ...
Traffic-sensitive live migration of virtual machines
CCGRID '15: Proceedings of the 15th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing

In this paper we address the problem of network contention between the migration traffic and the Virtual Machine (VM) application traffic for the live migration of co-located Virtual Machines. When VMs are migrated with pre-copy, they run at the source ...
OS-independent live migration scheme for bare-metal clouds
UCC '15: Proceedings of the 8th International Conference on Utility and Cloud Computing

Bare-metal clouds are an emerging and attractive platform for cloud users who demand extreme computer performance. Bare-metal clouds lease physical machines rather than virtual machines, eliminating a virtualization overhead and providing maximum ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CCSW '21: Proceedings of the 2021 on Cloud Computing Security Workshop

November 2021

161 pages

ISBN:9781450386531

DOI:10.1145/3474123

Program Chairs:
Yinqian Zhang
Southern University of Science and Technology
,
Marten van Dijk
Centrum Wiskunde & Informatica

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CCS '21

Sponsor:

SIGSAC

CCS '21: 2021 ACM SIGSAC Conference on Computer and Communications Security

November 15, 2021

Virtual Event, Republic of Korea

Acceptance Rates

Overall Acceptance Rate 37 of 108 submissions, 34%

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
443
Total Downloads

Downloads (Last 12 months)78
Downloads (Last 6 weeks)12

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Altahat MDaradkeh TAgarwal A(2024)Optimized Encryption-Integrated Strategy for Containers Scheduling and Secure Migration in Multi-Cloud Data CentersIEEE Access10.1109/ACCESS.2024.338616912(51330-51345)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3386169
Pathak PNadeem MAnsar S(2024)Security assessment of operating system by using decision making algorithmsInternational Journal of Information Technology10.1007/s41870-023-01706-9Online publication date: 19-Jan-2024
https://doi.org/10.1007/s41870-023-01706-9
Gupta ANamasudra SKumar P(2024)A secure VM live migration technique in a cloud computing environment using blowfish and blockchain technologyThe Journal of Supercomputing10.1007/s11227-024-06461-780:19(27370-27393)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s11227-024-06461-7
Boulougaris GKolomvatsos K(2023)An Inference Mechanism for Proactive Service Migration at the EdgeIEEE Transactions on Network and Service Management10.1109/TNSM.2023.327458120:4(4505-4516)Online publication date: 9-May-2023
https://dl.acm.org/doi/10.1109/TNSM.2023.3274581
Yan MGopalan K(2023)Performance Overheads of Confidential Virtual Machines2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)10.1109/MASCOTS59514.2023.10387607(1-8)Online publication date: 16-Oct-2023
https://doi.org/10.1109/MASCOTS59514.2023.10387607

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents