research-article

Public Access

Matched and Mismatched SOCs: A Qualitative Study on Security Operations Center Issues

Authors:

Faris Bugra Kokulu,

Yan Shoshitaishvili,

Gail-Joon AhnAuthors Info & Claims

CCS '19: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security

Pages 1955 - 1970

https://doi.org/10.1145/3319535.3354239

Published: 06 November 2019 Publication History

Abstract

Organizations, such as companies and governments, created Security Operations Centers (SOCs) to defend against computer security attacks. SOCs are central defense groups that focus on security incident management with capabilities such as monitoring, preventing, responding, and reporting. They are one of the most critical defense components of a modern organization's defense. Despite their critical importance to organizations, and the high frequency of reported security incidents, only a few research studies focus on problems specific to SOCs. In this study, to understand and identify the issues of SOCs, we conducted 18 semi-structured interviews with SOC analysts and managers who work for organizations from different industry sectors. Through our analysis of the interview data, we identified technical and non-technical issues that exist in SOC. Moreover, we found inherent disagreements between SOC managers and their analysts that, if not addressed, could entail a risk to SOC efficiency and effectiveness. We distill these issues into takeaways that apply both to future academic research and to SOC management. We believe that research should focus on improving the efficiency and effectiveness of SOCs.

Supplementary Material

WEBM File (p1955-kokulu.webm)

Download
127.76 MB

References

[1]

U.S Army. Persistent cyber training environment (pcte). https://www.peostri.army.mil/persistent-cyber-training-environment-pcte. Accessed: 2019-04--16.

[2]

Louise Axon, Bushra Alahmadi, Jason RC Nurse, Michael Goldsmith, and Sadie Creese. Sonification in security operations centres: what do security practitioners think? Internet Society, 2018.

[3]

Rob Barrett, Eser Kandogan, Paul P Maglio, Eben M Haber, Leila A Takayama, and Madhu Prabaker. Field studies of computer system administrators: analysis of system management tools and practices. In Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work, pages 388--395. ACM, 2004.

Digital Library

[4]

H Russell Bernard and Harvey Russell Bernard. Social research methods: Qualitative and quantitative approaches. Sage, 2012.

[5]

Sandeep Bhatt, Pratyusa K Manadhata, and Loai Zomlot. The operational role of security information and event management systems. IEEE security & Privacy, pages 35--41, 2014.

[6]

Patrick Biernacki and Dan Waldorf. Snowball sampling: Problems and techniques of chain referral sampling. Sociological methods & research, 10(2):141--163, 1981.

[7]

Chris Bing. Here's what the military's 'flight simulator' for cyberwarfare might look like. https://www.cyberscoop.com/cyber-command-training-raytheon-lockheed-martin-pcte, 2018.

[8]

David Botta, Rodrigo Werlinger, André Gagné, Konstantin Beznosov, Lee Iverson, Sidney Fels, and Brian Fisher. Towards understanding it security professionals and their tools. In Proceedings of the 3rd Symposium on Usable Privacy and Security, pages 100--111. ACM, 2007.

Digital Library

[9]

Douglas Clark, Brian Nelson, Pratim Sengupta, and Cynthia D'Angelo. Rethinking science learning through digital games and simulations: Genres, examples, and evidence. In Learning science: Computer games, simulations, and education workshop sponsored by the National Academy of Sciences, Washington, DC, 2009.

[10]

Anita D'Amico, Kirsten Whitley, Daniel Tesone, Brianne O'Brien, and Emilie Roth. Achieving cyber defense situational awareness: A cognitive task analysis of information assurance analysts. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 49, pages 229--233. SAGE Publications Sage CA: Los Angeles, CA, 2005.

[11]

Chris Dede, Brian Nelson, Diane Jass Ketelhut, Jody Clarke, and Cassie Bowman. Design-based research strategies for studying situated learning in a multi-user virtual environment. In Proceedings of the 6th international conference on Learning sciences, pages 158--165. International Society of the Learning Sciences, 2004.

Digital Library

[12]

Constanze Dietrich, Katharina Krombholz, Kevin Borgolte, and Tobias Fiebig. Investigating system operators' perspective on security misconfigurations. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pages 1272--1289. ACM, 2018.

Digital Library

[13]

Adam Doupé, Manuel Egele, Benjamin Caillat, Gianluca Stringhini, Gorkem Yakin, Ali Zand, Ludovico Cavedon, and Giovanni Vigna. Hit'em where it hurts: a live security exercise on cyber situational awareness. In Proceedings of the 27th Annual Computer Security Applications Conference, pages 51--61. ACM, 2011.

Digital Library

[14]

Margus Ernits and Kaido Kikkas. A live virtual simulator for teaching cybersecurity to information technology students. In International Conference on Learning and Collaboration Technologies, pages 474--486. Springer, 2016.

[15]

Inc. FireEye. Cyber security experts & solution providers. https://www.fireeye.com. Accessed: 2019-08--13.

[16]

Micro Focus. Intelligent security operations solutions: It secops software tools. https://www.microfocus.com/en-us/solutions/security-operations. Accessed: 2019-08--13.

[17]

Patricia I Fusch and Lawrence R Ness. Are we there yet? data saturation in qualitative research. The qualitative report, 20(9):1408--1416, 2015.

[18]

John R Goodall, Eric D Ragan, Chad A Steed, Joel W Reed, G David Richardson, Kelly MT Huffer, Robert A Bridges, and Jason A Laska. Situ: Identifying and explaining suspicious behavior in networks. IEEE Transactions on Visualization and Computer Graphics, 25(1):204--214, 2019.

Digital Library

[19]

Eben M Haber and John Bailey. Design guidelines for system administration tools developed through ethnographic field studies. In Proceedings of the 2007 Symposium on Computer Human Interaction for the Management of Information Technology, page 1. ACM, 2007.

Digital Library

[20]

Wajih Ul Hassan, Shengjian Guo, Ding Li, Zhengzhang Chen, Kangkook Jee, Zhichun Li, and Adam Bates. Nodoze: Combatting threat alert fatigue with automated provenance triage. In Network and Distributed Systems Security Symposium, 2019.

[21]

Cormac Herley and Paul C Van Oorschot. Sok: Science, security and the elusive goal of security as a scientific pursuit. In 2017 IEEE Symposium on Security and Privacy (SP), pages 99--120. IEEE, 2017.

[22]

C. Hill. Security operation center (soc). https://www.nascio.org/portals/0/awards/nominations2018/2018/NASCIO-IL-2018-Cybersecurity-SOC.pdf, 2017.

[23]

Gwo-Jen Hwang, Po-Han Wu, and Chi-Chang Chen. An online game approach for improving students' learning performance in web-based problem-solving activities. Computers & Education, 59(4):1246--1256, 2012.

Digital Library

[24]

McAfee Inc. Device-to-cloud cybersecurity. https://www.mcafee.com/en-us/index.html. Accessed: 2019-08--13.

[25]

McAfee Inc. What is an advanced persistent threat (apt)? https://www.mcafee.com/enterprise/en-us/products/network-security-platform.html. Accessed: 2019-08--13.

[26]

Cynthia E Irvine, Michael F Thompson, and Ken Allen. Cyberciege: gaming for information assurance. IEEE Security & Privacy, 3(3):61--64, 2005.

Digital Library

[27]

Fehér Dávid János and Nguyen Huu Phuoc Dai. Security concerns towards security operations centers. In 2018 IEEE 12th International Symposium on Applied Computational Intelligence and Informatics (SACI), pages 000273--000278. IEEE, 2018.

[28]

M. Kan. Boeing's wannacry run-in is a reminder to patch your systems. https://www.pcmag.com/news/360164/boeings-wannacry-run-in-is-a-reminder-to-patch-your-systems, 2018.

[29]

Bruce L Berg. Qualitative research methods for the social sciences. A Pearson Education Company, 2001.

[30]

J Richard Landis and Gary G Koch. The measurement of observer agreement for categorical data. biometrics, pages 159--174, 1977.

[31]

Xiaojing Liao, Kan Yuan, XiaoFeng Wang, Zhou Li, Luyi Xing, and Raheem Beyah. Acing the ioc game: Toward automatic discovery and analysis of open-source cyber threat intelligence. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 755--766. ACM, 2016.

Digital Library

[32]

Afsaneh Madani, Saed Rezayi, and Hossein Gharaee. Log management comprehensive architecture in security operation center (soc). In Computational Aspects of Social Networks (CASoN), 2011 International Conference On, pages 284--289. IEEE, 2011.

[33]

Natalia Miloslavskaya. Analysis of siem systems and their usage in security operations and security intelligence centers. In First International Early Research Career Enhancement School on Biologically Inspired Cognitive Architectures, pages 282--288. Springer, 2017.

[34]

Jelena Mirkovic and Peter Reiher. A taxonomy of ddos attack and ddos defense mechanisms. ACM SIGCOMM Computer Communication Review, 34(2):39--53, 2004.

Digital Library

[35]

Brian C Nelson and Diane Jass Ketelhut. Scientific inquiry in educational multi-user virtual environments. Educational Psychology Review, 19(3):265--283, 2007.

[36]

Cyril Onwubiko. Cyber security operations centre: Security monitoring for protecting business and supporting cyber defense strategy. In Cyber Situational Awareness, Data Analytics and Assessment (CyberSA), 2015 International Conference on, pages 1--10. IEEE, 2015.

[37]

Miloslava Plachkinova and Chris Maurer. Teaching case: Security breach at target. Journal of Information Systems Education, 29(1):11, 2018.

[38]

Inc. Proofpoint. Cyber security solutions from proofpoint. https://www.proofpoint.com/us. Accessed: 2019-08--13.

[39]

D. Ritchey. Creating the gsoc: 4 leading examples of successful security operations centers. https://www.securitymagazine.com/articles/87849-creating-the-gsoc-4-leading-examples-of-successful-security-operations-centers, 2017.

[40]

Sharma. Why do data breaches happen? https://www.marshall.usc.edu/blog/why-do-data-breaches-happen, 2017.

[41]

Rock Stevens, Daniel Votipka, Elissa M Redmiles, Colin Ahern, Patrick Sweeney, and Michelle L Mazurek. The battle for new york: a case study of applied digital threat modeling at the enterprise level. In 27th $$USENIX$$ Security Symposium ($$USENIX$$ Security 18), pages 621--637, 2018.

[42]

Anselm Strauss and Juliet Corbin. Basics of qualitative research: Procedures and techniques for developing grounded theory, 1998.

[43]

Sathya Chandran Sundaramurthy, Alexandru G Bardas, Jacob Case, Xinming Ou, Michael Wesch, John McHugh, S Raj Rajagopalan, and Lorrie Faith Cranor. A human capital model for mitigating security analyst burnout. In Eleventh Symposium On Usable Privacy and Security ($$SOUPS$$ 2015), pages 347--359, 2015.

[44]

Sathya Chandran Sundaramurthy, Jacob Case, Tony Truong, Loai Zomlot, and Marcel Hoffmann. A tale of three security operation centers. In Proceedings of the 2014 ACM Workshop on Security Information Workers, pages 43--50. ACM, 2014.

[45]

Sathya Chandran Sundaramurthy, John McHugh, Xinming Ou, Michael Wesch, Alexandru G Bardas, and S Raj Rajagopalan. Turning contradictions into innovations or: How we learned to stop whining and improve security operations. In Proc. 12th Symp. Usable Privacy and Security, 2016.

[46]

Sathya Chandran Sundaramurthy, John McHugh, Xinming Simon Ou, S Raj Rajagopalan, and Michael Wesch. An anthropological approach to studying csirts. IEEE Security & Privacy, 12(5):52--60, 2014.

[47]

Sathya Chandran Sundaramurthy, Michael Wesch, Xinming Ou, John McHugh, S Raj Rajagopalan, and Alexandru Bardas. Humans are dynamic. our tools should be too. innovations from the anthropological study of security operations centers. IEEE Internet Computing, 2017.

[48]

Splunk Technology. Siem, aiops, application management, log management, machine learning, and compliance. https://www.splunk.com/. Accessed: 2019-08--13.

[49]

Rodrigo Werlinger, Kirstie Hawkey, and Konstantin Beznosov. Security practitioners in context: their activities and interactions. In CHI'08 Extended Abstracts on Human Factors in Computing Systems, pages 3789--3794. ACM, 2008.

Digital Library

[50]

Rodrigo Werlinger, Kirstie Hawkey, and Konstantin Beznosov. An integrated view of human, organizational, and technological challenges of it security management. Information Management & Computer Security, 17(1):4--19, 2009.

[51]

Rodrigo Werlinger, Kirstie Hawkey, David Botta, and Konstantin Beznosov. Security practitioners in context: Their activities and interactions with other stakeholders within organizations. International Journal of Human-Computer Studies, 67(7):584--606, 2009.

Digital Library

[52]

Ting-Fang Yen, Alina Oprea, Kaan Onarlioglu, Todd Leetham, William Robertson, Ari Juels, and Engin Kirda. Beehive: Large-scale log analysis for detecting suspicious activity in enterprise networks. In Proceedings of the 29th Annual Computer Security Applications Conference, pages 199--208. ACM, 2013.

Digital Library

[53]

Shuhong Yuan and Chijia Zou. The security operations center based on correlation analysis. In Communication Software and Networks (ICCSN), 2011 IEEE 3rd International Conference on, pages 334--337. IEEE, 2011.

[54]

Chen Zhong, Tao Lin, Peng Liu, John Yen, and Kai Chen. A cyber security data triage operation retrieval system. Computers & Security, 76:12--31, 2018.

[55]

Carson Zimmerman. Ten strategies of a world-class cybersecurity operations center. MITRE Corporate Communications and Public Affairs. Appendices, 2014.

Cited By

Liu YLi TZhang RJin ZTong MLiu WWang YYang Z(2025)A Context-Aware Clustering Approach for Assisting Operators in Classifying Security AlertsIEEE Transactions on Software Engineering10.1109/TSE.2024.349758851:1(153-171)Online publication date: Jan-2025
https://doi.org/10.1109/TSE.2024.3497588
Althobaiti KAlsufyani N(2024)A review of organization-oriented phishing researchPeerJ Computer Science10.7717/peerj-cs.248710(e2487)Online publication date: 27-Nov-2024
https://doi.org/10.7717/peerj-cs.2487
Sun ZKokulu FZhang POest AStringhini GBao TWang RShoshitaishvili YDoupé AAhn G(2024)From Victims to Defenders: An Exploration of the Phishing Attack Reporting EcosystemProceedings of the 27th International Symposium on Research in Attacks, Intrusions and Defenses10.1145/3678890.3678926(49-64)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3678890.3678926
Show More Cited By

Index Terms

Matched and Mismatched SOCs: A Qualitative Study on Security Operations Center Issues

Recommendations

Information security management in SOCs and SICs

At present new sophisticated attacks make organizations’ IT infrastructure (ITI) break-in more professional and dangerously effective. All organizations must oppose this properly designed and centralized information security (IS) management systems. Learn ...
Flexible Reconfigurable On-chip Networks for Multi-core SoCs
HEART '18: Proceedings of the 9th International Symposium on Highly-Efficient Accelerators and Reconfigurable Technologies

Multi and many-core embedded SoCs (System-on-Chip) provide key solutions to meet the extraordinary demands of current and future applications. This fact becomes critical when the chip design dives to the limitation of sub-nanometer technologies that ...
The awareness of operators: a goal-directed task analysis in SOCs for critical infrastructure
Abstract
Security operation centers (SOCs) are increasingly established to meet the growing threat against cyber security. The operators of SOCs respond to complex incidents under time constraints. Within critical infrastructure, the consequences of human ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CCS '19: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security

November 2019

2755 pages

ISBN:9781450367479

DOI:10.1145/3319535

General Chairs:
Lorenzo Cavallaro
King's College London, UK
,
Johannes Kinder
Bundeswehr University Munich, Germany
,
Program Chairs:
XiaoFeng Wang
Indiana University, USA
,
Jonathan Katz
George Mason University, USA

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSAC: ACM Special Interest Group on Security, Audit, and Control

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

CCS '19

Sponsor:

SIGSAC

CCS '19: 2019 ACM SIGSAC Conference on Computer and Communications Security

November 11 - 15, 2019

London, United Kingdom

Acceptance Rates

CCS '19 Paper Acceptance Rate 149 of 934 submissions, 16%;

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '25

Sponsor:
sigsac

ACM SIGSAC Conference on Computer and Communications Security

October 13 - 17, 2025

Taipei , Taiwan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

65
Total Citations
View Citations
4,196
Total Downloads

Downloads (Last 12 months)1,136
Downloads (Last 6 weeks)158

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu YLi TZhang RJin ZTong MLiu WWang YYang Z(2025)A Context-Aware Clustering Approach for Assisting Operators in Classifying Security AlertsIEEE Transactions on Software Engineering10.1109/TSE.2024.349758851:1(153-171)Online publication date: Jan-2025
https://doi.org/10.1109/TSE.2024.3497588
Althobaiti KAlsufyani N(2024)A review of organization-oriented phishing researchPeerJ Computer Science10.7717/peerj-cs.248710(e2487)Online publication date: 27-Nov-2024
https://doi.org/10.7717/peerj-cs.2487
Sun ZKokulu FZhang POest AStringhini GBao TWang RShoshitaishvili YDoupé AAhn G(2024)From Victims to Defenders: An Exploration of the Phishing Attack Reporting EcosystemProceedings of the 27th International Symposium on Research in Attacks, Intrusions and Defenses10.1145/3678890.3678926(49-64)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3678890.3678926
Baruwal Chhetri MTariq SSingh RJalalvand FParis CNepal S(2024)Towards Human-AI Teaming to Mitigate Alert Fatigue in Security Operations CentresACM Transactions on Internet Technology10.1145/367000924:3(1-22)Online publication date: 15-Jul-2024
https://dl.acm.org/doi/10.1145/3670009
Nepal SHernandez JLewis RChaudhry AHouck BKnudsen ERojas RTankus BPrafullchandra HCzerwinski M(2024)Burnout in Cybersecurity Incident Responders: Exploring the Factors that Light the FireProceedings of the ACM on Human-Computer Interaction10.1145/36373048:CSCW1(1-35)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637304
Wang YDai YZhang Whan lzhou xmeng m(2024)Overview of research on network security operation system of industrial control systemFifth International Conference on Control, Robotics, and Intelligent System (CCRIS 2024)10.1117/12.3050656(49)Online publication date: 28-Oct-2024
https://doi.org/10.1117/12.3050656
Zakharov AShabalin AKruchkovsky K(2024)Features of Creating a Virtual Laboratory for Developing and Applying Correlation Rules in Modern SIEM-Systems when Training Security Operation Center Analysts (by the Example of Microsoft Windows)2024 International Russian Smart Industry Conference (SmartIndustryCon)10.1109/SmartIndustryCon61328.2024.10515421(367-372)Online publication date: 25-Mar-2024
https://doi.org/10.1109/SmartIndustryCon61328.2024.10515421
Han YSeo HYoon M(2024)Detecting Internet of Things Malware on Evidence GenerationIEEE Internet of Things Journal10.1109/JIOT.2024.343952811:22(36950-36964)Online publication date: 15-Nov-2024
https://doi.org/10.1109/JIOT.2024.3439528
Schiele NEgberts JGadyatskaya O(2024)Nobody Knows the Risks I Have Seen: Evaluating the Gap Between Risk Analysis and Security Operations2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW)10.1109/EuroSPW61312.2024.00059(473-483)Online publication date: 8-Jul-2024
https://doi.org/10.1109/EuroSPW61312.2024.00059
Li LHe YHuang FZhao ZSong ZZhou TLi ZZhang F(2024)An Automated Alert Cross-Verification System with Graph Neural Networks for IDS Events2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD61410.2024.10580010(2240-2245)Online publication date: 8-May-2024
https://doi.org/10.1109/CSCWD61410.2024.10580010
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents