In my last post, I provided a look ‘under the hood' to explain our real-time analytics strategy. I described our streaming technology as covering a full spectrum of detection techniques that casts a wide net and links together needles in a very large haystack to uncover the presence of malicious activity.

I am going to devote this post to the other half of the analytics spectrum: slow analysis.

Inasmuch as it isn't possible to detect all attackers using any single model or technology, it is also not possible to detect every attacker in real time. We are often required to free ourselves from the time constraints imposed by a streaming analytics infrastructure, and take a deeper look. In essence, this is slow analysis.

Retrospective Analysis

The primary slow analysis technique we use at ProtectWise is retrospective analysis, which is made possible by our ability to ingest and store full fidelity packet capture for long periods of time. As a result, we are able to practice full lifecycle intelligence curation. As we receive new intelligence or discover it as a part of our daily workflow, we feed it into both our real-time and retrospective discovery engines. By doing this, we are able to detect attacks moving forward, as well as re-examine the entire history of our customers' networks and discover previously undetected attacks, significantly narrowing the detection window from the industry average of more than 200 days.

It is tempting to focus on the use case of retroactively discovering missed attacks when discussing retrospective analysis. To be sure, finding something in the network history from weeks or months ago is very exciting. However, there is another equally important use case for retrospective analysis that we discuss with our customers: the assurance model of retrospective analysis.

The assurance model is the use of retrospective analysis to definitively prove that an exploit is not present on the network. This is especially useful for vulnerabilities that have been present in software for significant lengths of time prior to public disclosure. Over the past year we have seen many high profile instances of this class of vulnerability, Heartbleed being one of the most well known. The question in the minds of most CISO's when such vulnerabilities are disclosed is "I wonder if we were hit by that before we could patch?" The assurance model quickly provides our customers with the peace of mind that comes with being able to answer that question with a definitive "No."

The Network Never Lies

In addition to the storage of full fidelity packet capture, we maintain a detailed index of metadata that describes the state of a network at any given point in time. By doing this, we can fully reconstruct network state, down to the second in terms of time-fidelity, and down to the packets in term of content fidelity, at any point in time. This capability is unique to the ProtectWise cloud platform. It allows us to perform not only full fidelity retrospection, but also enables an arsenal of other deep analytics techniques that allow us to find subtle threats that would otherwise go undiscovered. At a high level, these include:

Data Mining and Pattern Discovery: These processes comprise a back end analysis regimen that models the behavior of attack progressions. These new behaviors are then modeled as logic in our threat engine and used to detect attacks carried out using the most current tactics.

Structural Analysis: This is a variance on data mining and pattern discovery, but it uses a graph theoretic approach to identify subtle patterns across the customer base. As mentioned above, we have the ability to reconstruct any network graph at any point in time in full fidelity. We also have the ability to fuse individual networks as abstractions in a greater meta-network graph and identify malicious actors who appear in more than one place. This intelligence is immediately leveraged in both the real-time and retrospective analytics environments.

Collective Analysis: Using the deep analysis techniques described above and our multi-tenant architecture, we capture intelligence from an attack in one customer, and immediately deploy that new intelligence for every other customer. We never share data between our customers, but we are able to use genericized patterns discovered in one customer's networks to look for the presence of threats in another network. By doing this, we are able to rapidly leverage increased knowledge across the customer base. Some of this intelligence is discovered in real time using our streaming approach to analytics, but some of the information we discover is subtle enough that it requires taking a deeper look. By fusing the two approaches together into a full life-cycle analytics strategy, we are able to paint a very full picture of the types of exploits that are being actively used by attackers.

Tying it All Together

Individual analytics techniques have their strong and weak points. Each technique is good at catching certain classes of attacks, but falls short at detecting others. Fusing them together makes full spectrum analysis possible.

At ProtectWise we have built a hierarchical decision engine that processes multiple inputs from the entire detection spectrum and adds to that very potent combination the dimension of time.

What we find in the present tells us what to look for in the past, and what we find in the past informs how and what we look for in the future.