25 Sep How Zeek IDS can Help Security Capture Institutional Knowledge for Cyber Alert Enrichment and Better Network Traffic Analysis
The network security analyst has a vexing challenge: a prerequisite for identifying abnormal or suspicious behavior is an understanding of what normal looks like. This means identifying each device on sprawling networks – and knowing its purpose.
That knowledge provides analysts with a better sense for which machines should talk to each other, over what protocols, and what characteristics or attributes are typically associated with such connections. With that level of understanding the anomalies tend to stand out. As a result, the organization benefits from faster, and more accurate, triage of alerts.
While this sounds simple, the reality is much harder. In a mid-to-large market enterprise, the technology environment can easily consist of thousands of hosts, machines, routers and other parts that comprise the IT infrastructure.
A complicating factor is that most IT environments are dynamic. IT operations routinely adds, patches, updates, and decommission servers and other parts of the infrastructure. Even more challenging is that more and more businesses are using a hybrid approach, where part of the infrastructure is on-premise, while the rest is cloud-based.
Retaining and Transferring Institutional Knowledge
Analysts often learn their environments as a byproduct of fulfilling their duties. Unfortunately, for many organizations, it’s also the sort of institutional knowledge that walks out the door when an analyst takes a different job. This gets expensive because research shows it costs businesses anywhere from 1.5x to 3x the salary to replace an employee.
Exactly how to capture institutional knowledge and transfer it from person-to-person, is a key challenge for security leaders too. It’s especially important today because there is a cybersecurity talent shortage. Bricata has developed a technical solution – a module in its threat detection solution – to address this problem.
The module was built using the Zeek IDS (formerly known as Bro) which is an open source software framework for analyzing network traffic, and one of three key detection technologies embedded in the Bricata appliance. Since the module is open source, it was presented and made available at the annual Zeek conference (formerly BroCon 2018) – an annual gathering of the Zeek IDS community.
The idea is to put a labeling capability at the fingertips of an analyst and within the network analysis tool, they are already using. This provides a concise way for analysts to share their knowledge about an environment. In other words, it’s using asset inventory as a means to capture knowledge about that IT environment and more importantly, the purpose of each device, box or host.
Better Triage and Cyber Alert Analysis
On its own, this a sound way for network security to retain institutional knowledge about the very IT infrastructure it is charged with protecting. In practice, the value goes much further. Those labels are married with network data the Bro framework is already generating – and in a way that allows more sophisticated threat detection and network analysis.
For example, if you are an analyst examining an IP address, you can’t make assumptions about what types of behavior – connections and protocols – that machine should or should not be using. However, if another analyst previously labeled that machine as a Microsoft SQL database server, you now have the context to discern what is normal and what is suspicious or even a clear threat.
Simply stated, these labels are used to enrich and fuse data sets to provide analysis that wouldn’t be possible otherwise.
Data Set Labeling and Machine Learning
Data enrichment is the process of taking information from one data set and using it to supplement another and reveal new insights. We are taking contextual information about IT assets and using it to enrich alerts about threat indicators.
Some variations of this idea have taken root in the security information and event management (SIEM) space. For example, a SIEM might take disparate data sets and marry it with Active Directory logs. However, with a few exceptions, there’s been very little done that’s useful for the network security analyst.
One of those exceptions is the functionality Bricata released with capabilities for tagging network events. This brings structure and workflow to a traditionally arbitrary way of describing network traffic. If that traffic meets certain conditions the analyst has defined, the solution will automatically tag it in a way the entire team can use to sort and filter alerts – and reduce the deluge.
Finally, this notion of labeling assets takes this a step further because it lays the foundation for dataset labeling, which is leveraged by supervised machine learning algorithms need to perform an analysis. So, data labels in the hands of network security capture institutional knowledge, enable better analysis and threat detection today and help lay the groundwork necessary for machine learning applications in the future.
* * *
The Bricata team presented this topic at the annual Zeek conference held at the Hyatt Regency Crystal City in Arlington, Va. The session, “Network Enrichment for Analysis and Threat Hunting” was recorded and is embedded below.
If you enjoyed this post, you might also like:
Considerations for Planning, Structuring and Deploying a New Network Security Strategy