.png)
What is the ROI of your Security technology stack?
With the influx of remote workforce throughout 2020, the Emagined Security Operations division received questions from current and potential customers regarding Return On Investment for specific technology stacks. The ROI discussions were naturally designed to find a single product or a small collection of products that provided the best security for the COST OR PRICE.
We understand that there is no silver bullet technology. Security failures are never due to one product or one user making a mistake. Its usually a combination of mistakes and failures that cause an incident.
Organizations that traditionally relied on network controls such as VPN, antivirus enabled network firewalls, and other On-Prem security controls found that their machines were falling victim to attacks.
Compounding the issue, Traditional IT Toolkits required the users come on site or VPN in periodically for OS and antivirus updates. Without regular security updates and the lack of enterprise grade network firewalls, organizations have found major weaknesses in their operational capabilities. Furthermore, the use of BYOD has skyrocketed, especially among organizations who supplied their workforce with desktop machines instead of more portable units.
Testing the ROI of Antivirus
Emagined Security's Threat Research team decided to take a hard look at security controls to determine what technology would help provide security to remote workforces, especially for those organizations who had never experienced this situation before.
In the first installment of these blog posts we decided to focus on antivirus. This control is the most common control applied to personal and corporate machines. The relatively low cost and performance impact of modern antivirus has made it a staple in most environments.
Building the Ransomware
We tasked an engineer with writing a test tool that simulates ransomware. Our designers had specific requirements:
It must actually manipulate >500 files on the system.
Instead of using robust encryption, we opted for simple XOR encoding of the binaries. (We did this just in case the binary was accidentally released to the wild.)
The simulated ransomware should be launched by the user.
It should use normal windows API calls or OS level functions, no advanced trickery. (Our goal was not to develop the next cyber weapon, its simply to show what an experienced developer can accomplish with a few hours of time.)
Tests should include both unsigned and signed binaries. (Though we never actually signed the binary - read on.)
The test will include a "right click" scan from the windows file system, followed up with a process scan once the binary was executed.
Source code for the final binary was about 100 lines of Go code. It was compiled and the researchers went about infecting Windows 10 virtual machines with various antivirus solutions installed.
The Antivirus Solutions Tested
Our tests consisted of the following Antivirus vendors. All signatures were upgraded and any additional ransomware protections were enabled.
Windows Defender
Sentinel One
Crowdstrike
Sophos
ESET
Malware Bytes
AVG/Avast
Bit Defender
The Results of our Test
All solutions failed to catch the malware on the individual file scan. This is concerning because it is common practice for many organizations who have been affected by an incident to save important documents, scan them with an antivirus tool, then move them to a non-infected workstation.
Only two of the vendors above detected our binary on run. Both of them detected after dozens of files were encoded.
The early version of the code was designed to parse through files on the operating system, bypassing any that were owned by anyone who was not a normal user on the system. Administrators, system processes, installers all were ignored. We also ignored decoy files dropped by antivirus vendors. Decoy files are easy to identify based on the text in the document or the directory. In our testing, the decoy data was predictable.
Once files were enumerated, a function would read the file into memory, apply an XOR string to all of the contents, then write to a new file. The original file was then deleted. For the IR experts reading the post - we understand this means the original file could still be recovered using forensic tools, as long as the disk did not perform SDD Trim.
Both Antivirus vendors who detected the simulated ransomware caught it when the original (victim) file was deleted. This only happened to files in the users Documents or Desktop Directories. This is because the antivirus software watched for any modifications to Office related documents in those directories. Regular text files were still encoded.
We went back to the drawing board. Instead of adding a new extension, we simply added a value such as "encrypted_" at the beginning of the filename.
We also decided to leave the file's magic bytes alone. The magic bytes tell your operating system what kind of file it is looking at. Each file type has a unique string of bytes that identify the file and in some cases version of the program required to open the file Our theory was that by leaving the magic bytes alone, antivirus and the operating system would still see a valid file, though the contents would be scrambled and the underlying data would be inaccessible.
This technique was successful at knocking one more vendor off the list. Because of the success, we decided to include decoy files in our manipulation. Ultimately, the decoys were encoded without issue. This suggest that our theory about magic bytes was accurate. After about 12 working hours we concluded the project was a success. Only one of eight engines was able to detect the code. All of the EDR technologies failed.
So What... Is Antivirus worth the ROI?
Does this mean that Antivirus or EDR are dead? Absolutely not.
However, this underscores the need for backups. Alternatively, the Sentinel One product has the capability to roll back changes. While Sentinel One did not detect the new threat, we were able to mark it as a threat in the Sentinel One console and use the remediation action "Roll Back" to restore the machine to pre-infected state.
Antivirus always has been a game of cat and mouse. This solution, on its own, will not provide adequate protection against threats found in the wild today. Additionally, we did not attempt data exfiltration, credential theft or any number of other techniques.