"Frankly, what we're all experiencing today is a bit of the new normal," PagerDuty CEO Jennifer Tejada said on the impact of the ongoing worldwide cyber outage and its ripple effect on businesses. "When you think about the complexity associated with new technology... what you're ultimately seeing is tech fragility."
"It's important to remember that software is imperfect... the important thing to think about is how do you prepare, how do you anticipate, and how do you respond when it happens," she added.
Joining us now is Pager Duty's chair and CEO, Jennifer Tejada. Pager duty monitors the health of a company, software, systems, networks, alerts when there are irregularities that need to be fixed. John, I know you guys are partners with Crowd Strike and with Microsoft. Always good to see you. Just first give me a sense of the scale of this outage. What was the last time you at Crowd Strike saw something with this big an impact? Well, I think, you know, pager duty, we see a lot of these, in fact, most of these because we serve 70% of the Fortune 100. And frankly, I think we're all experiencing today is a bit of the new normal. When you think about the complexity associated with new technology, our ability to continuously deploy, the fact that lots of investments been made, made and modernizing technology. We're adding automation, we're adding generative AI. But you're ultimately seeing is tech fragility. And I think it's important to remember that software is imperfect. So the new normal is not whether or not you're going to experience a major incident as either a company or a brand or an end consumer relying on those services. It's when. And the important thing to think about is how do you prepare? How do you anticipate and how do you respond when it happens? Well, I've heard that when it comes to cyber attacks, but this was a self-inflicted wound. So it it feels a bit strange to suggest that, well, we should get used to companies pushing out updates that crash businesses and cause millions or perhaps even billions of dollars worth of lost productivity. What will AI save us? No, I think you're going to need, you're going to continue to need intelligent humans in the loop. I mean, when you just think about the impact of an event like this, we saw just during the peak of this, we saw over 150% increased volume of major incidents in 200% increase in higher agency above normal. We're still seeing 50% increases in event traffic right now. And I think what you're finding is that you need the thousands. Of IT professionals, software professionals and operations people that are working through restoring the products and services that that rely on, on the technology endpoints that saw a failure. I do think that the genesis of these incidents can vary. It can be cyber related, it can be human error. It can be again, imperfect software or it can be the the conflation of all these things coming together at the same time and one of the challenges with the technology. Ecosystem is is changing everyday. It's a lot like weather systems. You can study what's happened in the past, but you can't know how different events might come together in the future and storm and create, you know, a difficult situation, a disastrous situation. So it is about how you respond quickly, how you orchestrate the information that you have, how you separate the signal from the noise, get to recovery, and then very importantly, take the lessons that you learned. From that incident and deploy change so that it doesn't happen again. I mean, you just talked about tech fragility and I realize we're talking about a buggy content update that that caused this entire situation over the last less than 24 hours, but does it raise the threat profile? With hackers, with cyber security incidents, with maybe adversarial nation states who say, ohh, look, this is the situation here. This is the fragility. Now we plan our next attack. That's, I mean that, that is that threat exists all the time. And if you look back over the last several months, I mean we've seen major incidents increase on the platform over 40% just year on year. So that fragility does increase the threat vector. And I think it is really important message to leaders to think about how they're investing in both modernizing infrastructure through investments, but also modernizing the way they operate, right, if you just improve your technology. Back but you don't change the way people operate, you don't build really good safety mechanisms around test automation, change control, et cetera. You're going to continue to have these challenges and even with all those systems in place, accidents will happen you you will have problems that you you don't plan for, but you can't anticipate and be prepared for when they do So John, is this a case where some IT pros work over the weekend rebooting some machines and this will be mostly. In the rearview mirror by next week, you know, I think we'll learn more of the details as we go on. I have to say that I think the way the Crowd Strike team has responded to this incident has been adorable. They've been transparent, they've taken accountability, they've been updating constantly. They're working very hard to make sure they get their customers back online. And it also reinforces the need for contingencies and for redundancy. Single systems become single points of failure. And so you need to have multiple redundant systems. I mean, during during a time like this, we see an incredible increase of volume and data crossing our platform. We can't afford to have a bad day on a day like today. And we put ourselves in a strong position by having redundant communication platforms, redundant cloud environments, redundant services that ensure that we are resilient in a moment like this. We also. Take precautions like we don't ship new things into production on a day like this to protect our customers and sure, they've got visibility and can orchestrate the work they need to undertake.
Yes but this statement would be sensible if we were talking about first order software. But when the software which was specifically deployed to be antifragile turns out to be massively fragile, we have a serious problem. It would be like going to a KFC and finding out they’re out of chicken.
Are you joking! All it would have taken was for IT to add the agent as part of their life cycle management process for patch management. It would have been caught during dev testing! This was user error spurred by laziness, hubris, greed and incompetence.
In today's digital-first world, outages are a matter of "when," not "if." Ready to fortify your digital operations? Let PagerDuty boost your resilience. 💪 #DigitalResilience#TechSolutions
"Frankly, what we're all experiencing today is a bit of the new normal," PagerDuty CEO Jennifer Tejada said on the impact of the ongoing worldwide cyber outage and its ripple effect on businesses. "When you think about the complexity associated with new technology... what you're ultimately seeing is tech fragility."
"It's important to remember that software is imperfect... the important thing to think about is how do you prepare, how do you anticipate, and how do you respond when it happens," she added.
This week's tech roundup has a story that has dwarfed all others: the Crowdstrike incident.
The scale and impact of the Crowdstrike debacle have dominated the headlines this week.
As the tech industry grapples with the fallout, it's a stark reminder of the importance of cybersecurity and the potential widespread impact of software failures.
Read more information on the unfolding situation here: https://lnkd.in/eFguEYQR
⚠️IMPORTANT⚠️
This is a common misconception that gets a lot of businesses into big trouble!
No business is too small for a cyber-attack. They happen to businesses of all sizes and when you least expect it.
To avoid this, check out our FREE Technology Errors and Omissions Application.
DataStream is able to shop your tech E&O + cyber application to a network of 15+ carriers to find you the best possible policy coverages and pricing.
https://hubs.li/Q02yWX5N0#cyberattack#cyberinsurance#cyberworld#cybernews
Are you using outdated technology to run your business?
Technology is not like fashion. What supported your business in the 90s is not going to come back in style.
Relying on old tech has a name -
Tech Debt
Over time, settling for suboptimal solutions leads to inefficiencies, increased costs, and other problems.
The solution?
Find out estimated upgrade costs and a realistic timeline for your business with our no-cost IT cyber audit.
Interested in learning more?
Schedule a consultation to discuss next steps: https://rebrand.ly/Netcomp#Netcomp#ITSupport#ITSecurity#ITServices#IT#ITInfrastructure#BusinessGrowth#DigitalTransformation
Are you using outdated technology to run your business?
Technology is not like fashion. What supported your business in the 90s is not going to come back in style.
Relying on old tech has a name -
Tech Debt
Over time, settling for suboptimal solutions leads to inefficiencies, increased costs, and other problems.
The solution?
Find out estimated upgrade costs and a realistic timeline for your business with our no-cost IT cyber audit.
Interested in learning more?
Schedule a consultation to discuss next steps: https://rebrand.ly/Netcomp#Netcomp#ITSupport#ITSecurity#ITServices#IT#ITInfrastructure#BusinessGrowth#DigitalTransformation
🚨 One week on from the debacle that brought major companies to a standstill—banks, airlines, supermarkets—it's clear how vulnerable even the biggest players are. Surprisingly, the cause wasn't a cyber attack, but a shocking lapse by Crowd Strike, a lesser-known security company.
This incident has exposed the murky world of corporate security. Consider Microsoft, whose Windows OS has dominated corporate systems for decades. By keeping its software closed and tightly controlled, Microsoft inadvertently allowed vulnerabilities to persist—vulnerabilities that companies like Crowd Strike profit from by providing patches.
Crowd Strike, tasked with securing these vulnerabilities, inadvertently triggered last week's chaos due to a mistake in their update process. Their software, operating deep within Windows, bypassed Microsoft's oversight, exposing a flaw that spread across countless corporate networks.
This event underscores a critical flaw in closed-source technology: while it drives innovation and standardisation, it also leaves our corporate infrastructure vulnerable to both incompetence and malicious intent. It's a sobering reality for businesses reliant on these systems, trapped in a cycle of dependence on technologies they can't fully scrutinise or control.
As we reflect on the fallout from this incident, it's clear that a broader conversation about the future of corporate cybersecurity is overdue.
Rowan asks, "How do we balance innovation with transparency and security?"
Click here to read more what Rowan has to say on this topic:
https://lnkd.in/gF3we8q9
It's a challenge affecting everyone, from startups to multinational enterprises.
Let's ensure we learn from this and take steps to prevent such vulnerabilities from disrupting us again. 💡
#CyberSecurity#TechOutage#EnterpriseRisk#scienceandtechnology#squizz#totecs
"I should like to know about risks, out-of-pocket expenses, time required and remuneration, and so forth"—by which he meant: "What am I going to get out of it? and am I going to come back alive?" - Bilbo, The Hobbit
Bilbo would make a savvy business-hobbit (in fact, it seems he did after his return from the Lonely Mountain).
Too often, cybersecurity professionals just expect our customers to take our word for it: "Just go on this journey, I promise you'll like it."
But what we're really telling them is "just spend more money!"
One of the best things we can learn is to speak *their* language. What's the investment going to return? What makes spending $1M on MDR worth it when you're going through a breach?
How does this actually improve the *business?*
We can't talk about security for security's sake. We need to build security plans and procedures that help people succeed and thrive. Computers for peoples' sake, not the other way around.
---
I help nonprofits and businesses build cybersecurity programs and make safer, more sustainable impact. If that's something you need help with, please don't hesitate to reach out to me!
💻 Over 8 million computers impacted worldwide
🏢 25% of Fortune 500 companies experienced outages
💸 US$5.4 billion in direct costs, US$15 billion globally
🌐 Massive disruptions to businesses and government agencies
⚠️ Critical services crippled, bringing thousands of businesses to a halt
🚨 The full repercussions of the recent CrowdStrike outage are still unfolding, but what we do know for certain is that preparation and resilience are non-negotiable.
🔥 Dive into the vital lessons learned from this catastrophic event in our new article by Professor Tafline Ramos (née Murnane), PhD and learn how to de-risk your organisation 💥
Link to the article in comments. 🔗
#CyberSecurity#CrowdStrike#QualityEngineering#Planit#QualityInsights#Software#BusinessResilience
🚨 One week on from the debacle that brought major companies to a standstill—banks, airlines, supermarkets—it's clear how vulnerable even the biggest players are. Surprisingly, the cause wasn't a cyber-attack, but a shocking lapse by Crowd Strike, a lesser-known security company.
This incident has exposed the murky world of corporate security. Consider Microsoft, whose Windows OS has dominated corporate systems for decades. By keeping its software closed and tightly controlled, Microsoft inadvertently allowed vulnerabilities to persist—vulnerabilities that companies like Crowd Strike profit from by providing patches.
Crowd Strike, tasked with securing these vulnerabilities, inadvertently triggered last week's chaos due to a mistake in their update process. Their software, operating deep within Windows, bypassed Microsoft's oversight, exposing a flaw that spread across countless corporate networks.
This event underscores a critical flaw in closed-source technology: while it drives innovation and standardisation, it also leaves our corporate infrastructure vulnerable to both incompetence and malicious intent. It's a sobering reality for businesses reliant on these systems, trapped in a cycle of dependence on technologies they can't fully scrutinise or control.
As we reflect on the fallout from this incident, it's clear that a broader conversation about the future of corporate cybersecurity is overdue.
Rowan asks, "How do we balance innovation with transparency and security?"
Click here to read more what Rowan has to say on this topic:
https://lnkd.in/gF3we8q9
It's a challenge affecting everyone, from startups to multinational enterprises.
Let's ensure we learn from this and take steps to prevent such vulnerabilities from disrupting us again. 💡
#CyberSecurity#TechOutage#EnterpriseRisk#scienceandtechnology#squizz#totecs
Goldman Sachs IB Alum | CBS Value Investing | Hoya Lawya | Bearcat
2moYes but this statement would be sensible if we were talking about first order software. But when the software which was specifically deployed to be antifragile turns out to be massively fragile, we have a serious problem. It would be like going to a KFC and finding out they’re out of chicken.