Nxlog can be found here:
Being a Logoholic and a HUGE fan of Splunk, I of course use Splunk when and where I can and recommend it to clients and the community as the de-facto standard as far as what to compare other logging solutions or a ’SIEM' too. With the discontinuation of Splunk Storm (Boooo), Splunk's free, low storage, low retention option, I needed another solution that was cost effective and useful. The need is to conduct malware archaeology to harvest artifacts, do some malware lab work and recommend to SMB's a simple solution. In addition, an easy to use and easy to configure logging option for use in Incident Response engagements when a log solution was not currently being used. Of course I tried, tested, even found bugs I reported to the vendors on a few cloud based logging solutions. Something I can have the potential client deploy on suspect systems and configure the logs settings manually before I get onsite so real data is waiting for me when I arrive, or remotely review.
Nxlog is a generic syslog agent for Windows, MAC and NIX systems. Nxlog can even format the logs into JSON providing some formatting for ease of reporting. Several solutions use it as their default or recommended log agent. Even Splunk, Sumologic and LogEntries who have their own logging agent can use nxlog.
Another logging agent to consider is the "Windows Logging Agent" or WLS, but that is another Blog post for another time... More info on WLS can be found here:
A few folks reached out and asked "Why nxlog"? Thus this Blog post. There are several considerations when selecting a logging solution or vendor. The short list is:
1. What logs can their agent read, not just the default logs, but all the other logs on the system that often get ignored and not collected. Windows has lots of them.
2. What configuration options does the logging agent have. You can send all your data, sometimes WAY more than you will ever need to your logging solution and filter from there, if they can filter at the server, most cloud solutions cannot. Logs equal disk space and appliances have limited disks, limited disk space equals short retention periods. Properly configured logs, which by default in Windows and databases are crap, properly configured logs equals more disk space. Unless you can filter your log data BEFORE sending it to your log server, you will need more disk space or license, thus filtering log data before it is stored on your log server is key unless your log solution allows server side filtering like Splunk does. This helps to reduce known good noise, making what you are looking for easier.
3. License or recurring costs for basic needs, think SMB's, Analysis Labs, Testing, etc., think Cloud based solutions
4. Scalable for medium and large needs, typical logging solutions and Cloud based solutions that scale
5. The ability to send alerts that are meaningful. Not the entire log entry, but just the columns of the data that says this IP at this time had this detail. Most important is the ability to read the alerts on a Smartphone so we know if we have an issue and need to act in a simple easily readable email.
6. The ability to exclude data easily from your queries to reduce 'known good' noise. I really don't need log entries that just don't tell me anything and are completely normal. Filter the good out to reduce what you are looking at to make log analysis easier.
7. Of course being able to send log files like web servers, network hardware and other odd devices you might have is also important. Most of these are syslog capable and most can handle raw syslog data, but sometime little no formatting once in your log solution, JSON formatting is best.
8. Ease of use to create queries, easy searching to create reports and alerts is key, selecting columns of only the data you want, not all of it.
9. The ability to normalize data or mark data so known fields (EventID, Username, CommandLine, etc.) are easy to select making a simple report with only the parts of the logs that are meaningful.
10. Does the solution have some intelligence to correlate events, or will you need to build the correlation events, which is OK as I know what I want to see, most vendor compliance log reports suck IMHO and don't help in alerting me to a real attack, they are just so you can check the we have it on a compliance report.
If a log solution can do all of the above, then you have a solution to evaluate to see if it fits what you need, or a guide to evaluate some more enterprise capable solutions.
If you use these ten items to evaluate a logging solution, you should come up with a short list pretty quick. Here are a few solutions worth trying.
- Loggly Cloud (cost effective and scalable) I use this for my lab work! $45/mo. for 1GB per day of data
- Splunk Cloud (much more costly and scalable)
- Splunk Server (enterprise)
- LogRhythm (enterprise)
- ELK Stack (OpenSource project, poor alerting)
Happy Logging and don't forget to use the "Windows Logging Cheat Sheet" to learn what to enable, configure, gather and harvest.
#InfoSec #MalwareArchaeology #Logging
