Airlines, banks, hospitals and other risk-averse organizations around the world have chosen cybersecurity company CrowdStrike to protect their computer systems from hackers and data breaches.
But a single glitch in a CrowdStrike software update caused global chaos on Friday, halting flights, taking banks and media offline and disrupting services including hospitals and retail stores.
“This is a function of very homogenous technology that’s built into the backbone of all IT infrastructure,” said Gregory Falco, an assistant professor of engineering at Cornell University. “The real cause of this disruption is that we’re dependent on very few companies, and everyone uses the same people, so they all go down at the same time.”
CrowdStrike said the issues with updates it issued that affected computers running Microsoft’s Windows operating system were not the result of a hacking incident or cyberattack, and the company apologized and said a fix was on the way.
But it wasn’t a quick fix: Fixing it required “on-the-ground work,” said Gartner analyst Eric Grenier.
“The fixes are working, but it’s a very manual process and there’s no magic key that can unlock it,” Grenier said. “That’s probably where companies are having the most trouble here.”
While CrowdStrike and its platform, Falcon, aren’t everyone’s customers, the company is one of the leading cybersecurity providers, especially in sectors like transportation, healthcare and banking, which have a huge stake in keeping their computer systems up and running.
“They’re typically risk-averse organizations, and they’re not looking for something crazy innovative, but they want something that works and will hold them accountable if something goes wrong, and that’s CrowdStrike,” Falco says. “And they look around at their peers in other industries and say, ‘This company is using this, so I’m going to need it too.'”
Concerns about the fragility of our globally connected technology ecosystem are not new: in the 1990s, there were fears that technological glitches would wreak havoc at the turn of the millennium.
“This is basically what we all worried about during Y2K, but this time it actually happened,” Australian cybersecurity consultant Troy Hunt wrote on social platform X.
Affected computers around the world on Friday were displaying the “blue screen of death,” a sign that something is wrong with Microsoft’s Windows operating system.
But what’s different now is that “these companies are more entrenched,” Falco said. “We’d like to think there are a lot of players available, but at the end of the day, the biggest companies are all using the same thing.”
CrowdStrike, which was founded in 2011 and went public in 2019, describes itself in its annual filings with financial regulators as “reinventing cybersecurity for the cloud era, transforming how cybersecurity is delivered and experienced by customers.” The company stresses that it is using artificial intelligence to keep up with its adversaries. The company reported having 29,000 customers as of earlier this year.
The Austin, Texas-based company is one of the best-known cybersecurity companies in the world, spending heavily on marketing, including a Super Bowl ad, and is known at cybersecurity conferences for its large booths displaying giant action figures representing the various state-sponsored hacking groups that CrowdStrike’s technology promises to protect against.
CrowdStrike CEO George Kurtz is one of the highest paid people in the world, with total compensation of over $230 million over the past three years. Kurtz also drives for a CrowdStrike-sponsored race car team.
After his initial comments on the issue were criticized as lacking remorse, Kurtz apologized in social media posts and on NBC’s “Today Show” on Friday.
“We understand the seriousness of the situation and deeply apologize for any inconvenience and trouble caused,” he said on X.
Cybersecurity industry analyst Richard Stiennon said this was a historic mistake by CrowdStrike.
“This is without a doubt the worst blunder, technical failure or glitch that has been made by a security software provider,” said Stiennon, who has followed the cybersecurity industry for 24 years.
While the issue is technically easy to fix, the hands-on work required to repair each affected computer could have a long-term impact on some organizations, he said. “Touching millions of machines is really hard, and people are on vacation right now. You know, the CEO is coming back from a trip to the Bahamas in about two weeks, and he won’t be able to use his computer when he does.”
Stiennon said he didn’t think the outage exposed larger problems for the cybersecurity industry or CrowdStrike as a company.
“The market will forgive them, customers will forgive them and this problem will pass,” he said.
Forrester analyst Allie Melen praised CrowdStrike for clearly communicating to customers what they needed to do to fix the problem, but said that to regain trust, the company needs to look more closely at what happened and what changes it can make to prevent it from happening again.
“A lot of this issue likely stems from the testing and software development process and the work they put into testing these kinds of updates before they were deployed,” Mellen said, “but we won’t know for sure what went wrong until we see the full retrospective.”
___
Associated Press writer Alan Suderman in Richmond, Virginia, contributed to this report.