We are all too familiar with the Credit Card breaches that hit Target, Neiman Marcus, Michaels, Marriott, Holiday Inn, Sheraton, Westin, Renaissance and Radisson, possibly Sears and other retailers yet to be named.
Recently some information on the details of the malware has been released by iSight, McAfee, X-Force and others. I took a look at the details around BlackPoS and Kaptoxa as I could not believe organizations the size of Target, Neiman Marcus, and Michaels and not mention several hotels were unable to detect such an attack. So what would it take to “Detect and Respond” to such an attack?
First off, this malware was neither sophisticated nor overly stealthy. It looks and acts like most malware and APT you would find, see or analyze. As a part of my “Malware Management Framework” program I review malware such as this in order to see if I could or would detect such an attack, similar attack and compare it to what I already see or know in order to tweak my detection capabilities.
First, let’s look at the diagram McAfee had in their report:
As we can see there are multiple Point-of-Sale (PoS) systems connected to the ‘Exfiltrator’ system. This system apparently sent the CC files via FTP to a compromised Internet server that the criminals then harvested the credit card data from. By the way, FTP is often used to send transaction reports off the PoS terminals to a central system, so FTP may be normal traffic to many.
It is not clear from the report(s); but one would also assume, for sake of having a starting point that this ‘Exfiltrator’ system, let’s call it ’Patient 0’ is where the attack against the PoS systems originated once the malwarians were in. We know from Neiman Marcus the malware was deleted every day and reinstalled after a PoS reboot. This would lend to the fact there is a ‘Patient 0’ system that re-infected the PoS systems as they came back up. This means that there would be some type of traffic from ‘Patient 0’ to each PoS system and a LOT of it if this occurred daily. According to reports, almost 60,000 events (the articles say alerts, but I don’t believe that).
With these assumptions and details we can build a pattern of noise that would have been made by such an attack and what we should have, or could do to detect and respond to such an event. So let’s take a look at how to detect and respond to this type of attack. I will use Windows 7 versus the Windows XP since most of you are running the latest Windows Operating System (hopefully), but everything discussed still applies to Windows XP, just with a different Event ID. I will be referencing the latest EventID’s, someone can convert them to XP/Legacy ID’s and I will gladly post them.
1. FTP – The FTP traffic that went from the PoS system to ‘Patient 0’, or visa-versa should have been detected. Even if we assume FTP is normal traffic from the PoS, or to the PoS from a central system and even if the proper central FTP system was also ‘Patient 0’ this should have been detected.
a. Transaction logs are scheduled and deterministic. It is highly unlikely that the malwarians used the exact same schedule and the exact same central system.
b. Net Flow should have been used to detect this type of traffic as FTP is in the clear, the data sent by the malware was not, it was encrypted and would have looked like gibberish.
c. The traffic outbound from ‘Patient 0’ should have been caught as well as it was going to an untrusted source. All your FTP servers should have known IP’s that it would connect with and any variance alerted to.
d. Any FTP server should have been configured to collect all logins and activity. ‘Patient 0’ should have been no different. This system received FTP transfers from many PoS systems and this behavior should have been detected as abnormal outside the normal transaction log upload time
2. Port 445 – Windows communicates over port 445 for shares that are mapped from one Windows system to another. The malware mapped drive “S:” to another Windows system was used by the malwarians. This traffic should have also been known in a static environment and thus any variance alerted on.
3. Logs – Windows logs if configured should have seen all kinds of noise from this malware. If auditing was enabled on the PoS systems or ‘Patient 0’ then noise would have been made, a LOT of it! Collection of known malicious behavior should have been detected. Windows 7 Advanced Auditing should be set to ‘success and failure’ for many items. See another future Blog article for what to set, or better yet come to BSides Austin for the Windows Logging Workshop. For Windows XP logs, all audit logging should have been set to ‘success and failure’ and collected locally or better yet sent off to a syslog server or SIEM solution. For the sake of argument and the sheer noise it produces, let’s assume the Windows Firewall is off, so no Packet Filtering logging is enabled.
a. Execution of the FTP client should have been detected. Event ID 4688 triggering on ‘ftp.exe’ should be part of malicious tools and activity monitoring.
b. Connection between systems should have also been detected. Event ID 4688 triggering on ‘net.exe’ should be part of malicious tools and activity monitoring.
c. The PoS system when connected to a share should have also been detected. Event ID 4624 triggering a ‘Logon Type 3’ network logon should be part of malicious network activity monitoring.
d. The system the PoS connected to should have had a share connection detected. Event ID 5140 would list the share ‘Test’ was connected and what IP address did the connection and should be part of malicious network activity monitoring.
e. The system the PoS connected to should have had a share connection that disconnected detected. Event ID 4634 would list the ‘Logon Type 3’ was ‘logged off’ and should be part of malicious network activity monitoring.
4. CMD Shell executed – A command shell was used on ‘Patient 0’ as well as each PoS system.
a. Event ID 4688 triggering on ‘cmd.exe’ should be part of malicious tools and activity monitoring.
5. PSExec executed – The use of PSExec is very suspicious and should not be used by anyone except an administrator.
a. Event ID 7045 triggering on ‘PSEXECSVC.EXE’ or ‘PSEXEC.EXE’ should be part of malicious tools and activity monitoring.
6. The Malware – The malware files that were used should also have been
detected in many ways. The logs at a minimum could/would detect that something malicious was occurring.
b. Event ID 4688 & 4689 should be triggering as the new unknown application started and stopped with the names ‘Bladelogic.exe’ or ‘POSWDS.EXE’ (dum.exe) should be part of malicious tools and activity monitoring.
c. File Auditing is not heavily used by people, but in static environments like a PoS system or file server, or every administrator system you have, it would be easy to setup File Auditing on a handful of directories, or just exclude the noisy ones. New files being added to \System32 would have triggered the files dropping and the credit card data file ‘winxml.dll’ if File Auditing had been enabled. Add File Auditing to ‘\System32’, ‘\Drivers’, ‘\WBEM’, ‘\Fonts’, and ‘\twain_32’ and set to audit ‘Create File’ and ‘Create Folder’ for success.
d. Event Id 4663 would have been triggered in the logs for ‘access an object – WriteData (or AddFile)’ the new files that were added by the malware and should be part of malicious tools and activity monitoring.
7. User Accounts – There is a fair amount of discussion around the accounts used to access the PoS and FTP system(s). Any good InfoSec program should know what accounts are doing, especially if they are administrative accounts. Service accounts and administrator accounts have a behavior that is fairly normal. It is unlikely that the malwarians used the credentials ‘Best1_user’ account in normal ways. A more likely scenario is that the monitoring of accounts was not being performed. Successful logins for accounts will tell you more than failed attempts as a compromised account will not generate failed attempts. The credentials may have been sniffed, cracked or key logged and then used successfully, maybe even ‘a user account was created’ which is ‘Event ID 4720’.
a. Event Id 4624 would have been triggered in the logs for a ‘New Logon’ and should be part of malicious tools and activity monitoring.
Logging won't catch everything, especially since it must be configured, but there are so many log events I was able to reproduce that it is almost laughable that Target, Neiman Marcus, Michael's and others were unable, or unwilling to detect all this nefarious noise. Being PCI compliant is not the same as practicing PCI compliance daily. Compliance is one of the reasons these large breaches occur (IMHO) as we are so busy chasing and filling in compliance paperwork that we fail to actually practice REAL SECURITY.
Well Target, Neiman Marcus, Michael's and others. I just showed you and many others "HOW", so step up and implement Malware Management and do what many of us already know.
If you are not logging it, you are NOT doing it right.Other well-known security tools – many well known security tools would have detected the files dropping onto a PoS or Windows system by default. Tripwire for example would see all the new files in \System32. Custom configuration management tools would also detect new files in key places. Carbon Black would have seen the files, processes, Registry keys and command line details from the malware. The Windows Logging Service (WLS) would have caught the new processes and command line syntax used to trigger the new processes as well as the service being added. I have only named a few security solutions... No Anti-Virus software would not catch anything like this until long after it was discovered.
If you collected the logs locally and ran scripts to gather log data or used a simple syslog server to grep through the logs, you could have seen this activity. Of course a robust Log management solution like Splunk or one of the many SIEM solutions would have been another great and recommended method to detect this behavior “IF” properly configured. Most larger organizations that have compliance requirements like PCI and are required to use an advanced logging solution.
Of course there are many more things we could do to adequately detect and respond to security incidents before they turn into a large “Event” that may have you preparing for “Breach Notification”. So prepare now and integrate the "MalwareManagement Framework" into your Information security program before it is too late.
Don’t be a Target!
Want to learn more? Come to BSides Austin 2014 and take the 4 hour “Windows Logging Workshop”. Of course watch my Blog for more on Blue Team Detection and Response techniques.
IBM X-Force Blog:
Brian Krebs – Krebs on Security Blog: