VMware has admitted an update on some versions of its Carbon Black endpoint solution is responsible for BSODs and boot loops on Windows machines after multiple organizations were affected by the problem.
The issue – where PCs began booting into blue screens, some of which flashed the stop code PFN_LIST_CORRUPT – was apparently caused by a changed ruleset by the company, which agreed to be acquired by Broadcom in May in a deal expected to close next year.
Users of Carbon Black’s EDR (Endpoint Detection and Response) product install sensors on each endpoint in their org – laptops, desktops, servers etc – and the sensors then monitor processes, active network connections, files being modified, and any other unwanted changes, ideally in the hopes of stopping an attacker from poking around and also to halt ransomware and other threats before they spread across the network.
The BSOD problem surfaced yesterday, with threat hunter Tim Geschwindt stating on Twitter he knew of about 50 organizations struggling with the issue, and saying the Carbon Black endpoint solution was “causing blue screens of death for devices running sensor version 3.7.0.1253” (later expanded to a broader range of sensors). The BSODs apparently began at 1430 UTC yesterday.
Over on Reddit, one admin says the net effect was “servers and workstations bluescreening ‘PFN_LIST_CORRUPT’,” with another sysadmin saying they had been “told verbally by VMware that they are inundated.”
VMware says in its Knowledge Base article that the cause was some updated threat research rulesets rolled out to cloud regions in the US East, Asia Pacific, and the EU, which, it added, hadn’t caused any trouble in its internal testing.
The problem affects devices running sensor versions from 3.6.x.x to 3.7.x.x, VMware says.
The virtualization giant, which hosts its VMware Explore event in San Francisco next week, has rolled back the rulesets, and promises that as machines check in, they will “get the updated ruleset and auto-resolve.”
Admins have been told to place affected devices into bypass mode via the Carbon Black Cloud Console to allow them to boot successfully and have the ruleset removed, although a “small subset” may require an additional workaround and those looking after them should open a support ticket. There’s more information in the Knowledge Base, and Carbon Black users should check for updates.
Carbon Black, a suite of cloud-native endpoint protection tools aimed at the enterprise cybersecurity market, was acquired by VMware in 2019 to form the centerpiece of the company’s new Security Business Unit.
The vendor patched critical flaws in Carbon Black’s App Control security tool earlier this year.
Tech analysts at Gartner have predicted that as VMware’s proposed owner, Broadcom may rationalize some products after the acquisition completes, noting that both Symantec and the Carbon Black unit offer endpoint protection products. The deal is expected to close before the end of Broadcom’s fiscal 2023, so by October next year.
We have asked VMware for comment. ®