Compliance departments and security teams have spent years putting all of their risk mitigation eggs in the SIEM basket. They’ve spent tons of time and money centralizing their incident response workflows around SIEMs, and have built substantial libraries of rules that need their constant attention.

Shrewd SIEM vendors built an amazingly sticky security product, but they’re also acutely aware that their customers have been less than satisfied. SIEM hasn’t lived up to its promise. That might be why we’ve heard so much about the “Next Gen SIEM” for the last few years as vendors try to reinvent themselves in order to stay relevant.

Is “Next Gen SIEM” really even a “thing,” though? Before plunging into a PoC, now might be a good time to figure out whether it’s the future of security monitoring, or if there’s a better option out there altogether.

First of all, relying on SIEM is a bit like relying on flu shots every year. While it’s better to get the shot than to skip it, you know it’s just preventing the known variants of the flu that were discovered in the last year. SIEM works the same way. It uses rules to correlate on known indicators, then reports on the results. That means it can’t detect unknown attacks or analyze large data sets, or understand network and user behavior.

SIEM is great at aggregating log files, performing simple correlation, and alerting security teams about potential issues. The problem is when analysts go back to those log files, logs don’t tell analysts a whole lot about what happened. It’s like reading a phone bill to find out who you called and for how long you spoke, but it doesn’t say anything about what you talked about on the call.

What you said in a conversation is much more informative than when you said it and for how long, and illustrates another limitation of SIEM. It’s good at aggregating modest amounts of data from many systems, but collecting the actual content of logged events isn’t what they were designed to do in the first place. What’s more, reporting back to an analyst the content of those events in a log file would be an overwhelming about of data with which to work.

From a network security perspective, the traffic on your network is that conversation, and has a wealth of information security teams can use to focus on important matters. SIEM logs can provide some useful context, but they won’t give you the complete picture like a recording of your network traffic can tell you.

Just how much or how little information is contained in a SIEM log is configurable, which impacts that value of that data because it might not include crucial details, but if you don’t see something in a log, that doesn’t mean something didn’t happen. Take for example the Destover malware which wiped out data across Sony’s network of workstations, and also changed timestamps and erased log files. The only way to know that happened at all is to perform a full forensic analysis. SIEM can’t even do that on its own.

So if your network packets never lie, and it’s nearly impossible to capture them in high fidelity or to keep them for longer than a few days or weeks using legacy, on-premise solutions available today, what can you do? The appliance-based nature of legacy PCAP solutions is hampered severely by high cost and difficult deployment. Building out more hardware simply won’t do and the cost benefit isn’t there if you’re only capturing a few network segments, limiting any real visibility that might be helpful for analysts.

Instead of cumbersome hardware, the cloud makes it easy to capture network traffic and retain it for as long as you need. Instead of knowing Jane talked to John for an hour and 20 minutes, the cloud lets you know they discussed product development plans and requirements. In other words, the cloud takes the shackles off full-fidelity PCAP and allows you to retain as much of it as you want so you can understand more and make better decisions.

That’s not to say unlimited, full-fidelity PCAP products should displace what you have in your security stack today. Quite the contrary! A better approach would be to use that data to help deliver the promise of SIEM. Using the ease and cost-effectiveness of the cloud to analyze network traffic for workloads inside the organization and outside of it on cloud architectures helps make threat hunting much easier. You can correlate multiple streams of data to open up dozens of potential new detection techniques to find both known and unknown threats which can’t be detected using rules- and signature-based solutions like SIEM alone.

Don’t forget: SIEMs require rules to correlate data, leaving them unable to detect unknown threats. Even though there are vendors who can perform behavioral analytics on log files to detect threats (such as UBA or UEBA), it’s likely these will be acquired by SIEM vendors to bolster their offerings. Even still, the accuracy of these products is still limited by the log files, which applications offer just enough information to for diagnostic support. The alternative is to take more information into log files, but a big chunk of important contest - which can be found in PCAPs - won’t be there, leaving you still with poor visibility.

Pervasive visibility from the network to the endpoint ought to be one of the primary drivers for when evaluating your next security purchase. And the marketing wrapper of “Next Gen SIEM” doesn’t hide the fact that the technology is getting long in the tooth. While these products are still ideal for coordinating your SecOps workflows, what’s needed are solutions that provide clear visibility into today’s threat landscape -- something SIEM, “Next Gen” or not, will never be able to deliver on its own.

Next blog post