IT admins around the world are scrambling to fix a major issue with Windows computers today, after a faulty update from cybersecurity provider CrowdStrike knocked thousands of PCs and servers offline with a Blue Screen of Death (BSOD) error. While CrowdStrike has fixed the update that originally caused the problems, many systems are still offline, with banks, airlines, supermarkets, and TV broadcasters struggling to cope without their machines.

The fix, for many, won’t be easy. IT admins are still trying to use an initial workaround provided by CrowdStrike, which involves booting Windows systems into Safe Mode and deleting a system file:

These steps force Windows to boot into a Safe Mode environment where third-party drivers like CrowdStrike’s kernel-level driver aren’t able to load. IT admins then have to locate the faulty driver on the disk and delete it. This workaround requires, in most cases, physical access to a machine and in some environments could be complicated by disk encryption like BitLocker, or even a lack of admin rights to be able to delete the faulty driver.

The other option is to wait for CrowdStrike’s fix to come through — but getting it has been a problem. Some IT admins are simply rebooting machines over and over, hoping that the CrowdStrike update will get pushed through the network stack before CrowdStrike’s protection engine initializes and then BSODs the machine. This seems to be working for some, with reports of machines coming back online after being rebooted multiple times.

CrowdStrike’s update server and content delivery networks are likely being hammered by the millions of machines reaching its servers for an update, so it may take some time for the reboot method to work.

Businesses running virtual desktops may be able to recover quicker than others, by simply restoring affected hosts back to a point before CrowdStrike’s faulty update wreaked havoc. In environments where rebooting isn’t working, the workaround of booting into Safe Mode looks like the best option right now.

Either way, this issue isn’t going to be resolved in a matter of hours like the typical internet outages we see from cloud providers. “It could be some time for some systems that won’t automatically recover, but it is our mission to make sure every customer is fully recovered,” says CrowdStrike CEO George Kurtz in an interview with NBC News.

In that same interview, Kurtz apologized for the damage caused by CrowdStrike’s update, but there will undoubtedly be questions around how a faulty update like this ever managed to hit thousands or millions of machines around the world.

IT admins around the world are scrambling to fix a major issue with Windows computers today, after a faulty update from cybersecurity provider CrowdStrike knocked thousands of PCs and servers offline with a Blue Screen of Death (BSOD) error. While CrowdStrike has fixed the update that originally caused the problems, many systems are still offline, with banks, airlines, supermarkets, and TV broadcasters struggling to cope without their machines.

The fix, for many, won’t be easy. IT admins are still trying to use an initial workaround provided by CrowdStrike, which involves booting Windows systems into Safe Mode and deleting a system file:

These steps force Windows to boot into a Safe Mode environment where third-party drivers like CrowdStrike’s kernel-level driver aren’t able to load. IT admins then have to locate the faulty driver on the disk and delete it. This workaround requires, in most cases, physical access to a machine and in some environments could be complicated by disk encryption like BitLocker, or even a lack of admin rights to be able to delete the faulty driver.

The other option is to wait for CrowdStrike’s fix to come through — but getting it has been a problem. Some IT admins are simply rebooting machines over and over, hoping that the CrowdStrike update will get pushed through the network stack before CrowdStrike’s protection engine initializes and then BSODs the machine. This seems to be working for some, with reports of machines coming back online after being rebooted multiple times.

CrowdStrike’s update server and content delivery networks are likely being hammered by the millions of machines reaching its servers for an update, so it may take some time for the reboot method to work.

Businesses running virtual desktops may be able to recover quicker than others, by simply restoring affected hosts back to a point before CrowdStrike’s faulty update wreaked havoc. In environments where rebooting isn’t working, the workaround of booting into Safe Mode looks like the best option right now.

Either way, this issue isn’t going to be resolved in a matter of hours like the typical internet outages we see from cloud providers. “It could be some time for some systems that won’t automatically recover, but it is our mission to make sure every customer is fully recovered,” says CrowdStrike CEO George Kurtz in an interview with NBC News.

In that same interview, Kurtz apologized for the damage caused by CrowdStrike’s update, but there will undoubtedly be questions around how a faulty update like this ever managed to hit thousands or millions of machines around the world.