97% of Windows Systems Impacted by CrowdStrike Update Are Back Online
At the end of last week, CrowdStrike representatives announced that 97% of Windows systems affected by the problematic update are now operating normally. The company also released a report explaining the cause of the incident. Meanwhile, insurance experts at Parametrix estimated the direct financial losses for Fortune 500 companies in the U.S. at $5.4 billion.
Background of the Incident
In mid-July, an update to the enterprise solution CrowdStrike Falcon Sensor caused millions of Windows systems to display the “blue screen of death” (BSOD). This led to widespread disruptions at airports, banks, healthcare facilities, and many other organizations across the U.S., UK, several EU countries, India, New Zealand, Australia, and more. According to Microsoft, about 8.5 million Windows systems were affected by the outage.
Technical Details and Company Responses
CrowdStrike and Microsoft have now published detailed technical reports on the incident. The companies revealed that the global outage was caused by a bug in the test system called Content Validator. After passing through Content Validator, the update was not subjected to further checks due to trust in previous successful deployments of the Inter-Process Communication (IPC) Template Type. As a result, the problematic update went unnoticed until it was deployed on client hosts running Falcon version 7.11 and later.
CrowdStrike has promised to improve its testing processes in the future by adding fuzzing, stress tests, stability checks, and more. The company will also implement phased rollouts for updates to smaller pools of clients and will begin publishing release notes for EDR content updates, which it had not done previously.
Restoration Efforts and Future Plans
Last week, CrowdStrike CEO George Kurtz reported that 97% of Windows systems affected by the faulty update have been restored and are operating normally. Kurtz emphasized that the speed of recovery increased thanks to “the creation of automated recovery methods and the mobilization of all our resources to support our customers.”
Regarding Microsoft’s response, Vice President John Cable stated that the company “deployed more than 5,000 support engineers who worked around the clock” to address the aftermath of the incident. He also hinted at possible future changes to Windows that could help prevent such widespread outages. Specifically, Cable noted that VBS enclaves and Azure Attestation can provide Windows security without requiring kernel-level access, which most Windows security products (including the problematic CrowdStrike Falcon Sensor) currently have. While he did not specify what changes might be coming to Windows, he said Microsoft will continue to strengthen its platform and “do even more to improve the resilience of the Windows ecosystem by working openly with the broader security community.”
Financial Impact on Fortune 500 Companies
Analysts at insurance company Parametrix calculated that U.S. Fortune 500 companies alone (about a quarter of which were affected) suffered losses totaling $5.4 billion due to the CrowdStrike update issue. This estimate does not include Microsoft’s losses, as the company was considered a key player in the event.
Parametrix noted that some industries within the Fortune 500 were barely impacted. For example, manufacturing, transportation (excluding airlines), and finance sectors reportedly lost only tens of millions of dollars. In contrast, retail and IT lost about half a billion dollars, airlines lost $860 million, and the banking and healthcare sectors lost over $3 billion combined.