A devastating IT outage caused by a faulty software update from security firm CrowdStrike sent shockwaves around the world, crippling critical infrastructure and forcing IT professionals to scramble for solutions. The incident, which occurred on Friday, affected an estimated 8.5 million Windows computers globally, including those used by airlines, hospitals, and funeral homes.
For one system administrator at a West Coast company that handles funeral and mortuary services, the outage was a wake-up call – literally. He was awoken in the early hours of the morning by a flurry of messages from colleagues, alerting him to a network issue that threatened to disrupt funeral services and burials. With the company’s entire infrastructure down, the administrator sprang into action, working a nearly 20-hour day to reset dozens of computers in person and resolve the problem.
The situation was urgent, as the computers needed to be back online to prevent disruptions to funeral service scheduling and mortuary communication with hospitals. “We had to make sure our company was good to go so we could get these families in and allow them to go through the services and be with their family members,” the administrator explained. “People are grieving, and we couldn’t afford to have any more disruptions.”
The flawed CrowdStrike update caused widespread chaos, with some IT professionals facing the daunting task of coordinating with remote employees or multiple locations across borders to manually reset devices. One junior system administrator in Indonesia had to overcome language barriers to resolve the issue, while a team at a healthcare organization in Maryland spent several hours bringing servers back online and then manually fixing over 5,000 devices.
The outage highlighted the often thankless and unglamorous nature of IT work, with one system administrator noting that “we aren’t noticed unless something is wrong.” However, the incident also underscored the critical importance of IT professionals in resolving such crises. As Shawn Henry, CrowdStrike’s chief security officer, acknowledged, “the confidence we built in drips over the years was lost in buckets within hours, and it was a gut punch… we let down the very people we committed to protect.”
