03 Oct Bro IDS: 7 Takeaways from BroCon 2017
For three days in September, a small but growing community of cybersecurity professionals dedicated to a specialized open source tool, gathered at the National Center for Supercomputing Applications in Urbana, Illinois.
The gathering was a conference – BroCon 2017 – which is dedicated to Bro, an open source software designed to support network monitoring and intrusion detection.
Bricata sent three members of our team to attend the conference – Adam Pumphrey, Andrew Beard, and Kent Wilson – and they returned with some observations we thought useful to share.
What is Bro?
Before we dive into those observations, let’s briefly answer the question – what is Bro?
For those that are not familiar, Bro is an open source intrusion detection system (IDS). It is different from traditional IDS tools in that it is focused on network analysis. In other words, it is used to monitor traffic traversing a network to identify anomalies.
When an organization has a good baseline for what normal traffic patterns on an internal network look like, it is in a far better position to identify network behavioral characteristics that are abnormal. For example, if a workstation in accounting suddenly starts drawing information from a different database server, this may be an anomaly worth closer investigation.
Adam Pumphrey, from our team, was invited to present a Bro Primer early in the conference, and while Bro can feel highly technical and intimidating to those new to the technology, the community works hard to make newcomers feel welcome. One of the macro observations from the conference was that the Bro community is going to great lengths in an effort to make this important open source technology more accessible to non-technical users.
7 Takeaways from BroCon
The observations from the conference below represent a mix of perspectives submitted by our attending team. We have tried to organize and present these in an order from less-to-more technical for the benefit of our readers.
1) The Bro user community is evolving.
There is a steady shift among the diversity of people leveraging the Bro system. What was once a community built from a constituency largely comprised from national labs and higher education is now attracting a much wider interest group.
While many of the presentations spoke to the technical details of implementing Bro and building environments for housing the data produced by Bro, there were also a large number of topics focused on how operations personnel and users without a programming background might better leverage the data produced by Bro.
This is a pretty clear sign that the community is attempting to break down the technical barriers and make Bro more accessible to analysts who might not have the skillset to develop it on their own.
2) Remember to think like an adversary.
A presentation on threat hunting – Using Bro to Hunt Persistent Threats – by Benjamin Klimkowski about an academic team from U.S. Military Academy (West Point) was a good reminder to consider dedicating some time to studying the tools attackers use and their capabilities.
For example, obtaining packet capture (PCAP) samples is very difficult, so in order to be more effective, security professionals should find, use, create and study the same sort of tools adversaries use.
This is important because even battled hardened soldiers forget to think like an adversary sometimes. This is why the cybersecurity industry has red teams and why risk management shops designate someone to credibly argue an opposing view.
3) What do I do with Bro data now?
Broadly speaking, a question many organizations that have developed a Bro system ask is “What do I do with this data now?” From our perspective, this year’s conference provided much more material – details and case studies on the topic.
In a presentation called, Data Analysis, Machine Learning, Bro, and You!, Brian Wylie of Kitware demonstrated the Bro Analysis Tools (BAT) and how these enable dynamic analysis via several common machine learning algorithms. Brian and his team have been able to transform Bro data into Pandas.Dataframes and then analyze it with scikit-learn.
The process suggests how accessible machine learning algorithms are to those with the technical skill set to install and manipulate Python.
4) Visualizing Bro data
People were drawing and interpreting pictures long before writing letters and numbers. This is why visualizing data is so important to creating understanding. For example, Brian’s talk cited above also demonstrated ways to rapidly visualize Bro data using matplotlib and Jupyter Notebooks. A visual rendering of data is often a better way to identify network behavioral anomalies – activity that could indicate the presence of a threat.
5) New advances in Bro data retention
New advances are being implemented for the storage of Bro data, in large-volumes, and for long-term retention. This supports extremely high-performance query interaction over massive amounts of historical Bro data. These capabilities have proven ground-breaking for National Cyber Security Alliance (NSCA) who is now able to perform historical searches and analysis over very large time ranges. Justin Azoff of the NSCA and an active member of the Bro project presented a staggeringly impressive project on Bro Clickhouse that illustrated this performance.
6) The Bro package manager matures.
Last year, the Bro Package Manager was still in its infancy, but it has matured significantly. The purpose of the package manager it to make it easy for Bro users to install and managed third-party scripts, including plugins. Most of the development, that is not part of the core Bro system, is being pushed into such packages and plugins.
7) Data filters heat up.
While not entirely new, probabilistic data structures like Bloom filters and Cuckoo filters seem to be catching on as a viable means for storing large amounts of data in memory with a minimal footprint. These data structures allow for very high-speed membership tests, or in other words, asking if a given value exists in a list. Hash tables are still very relevant but the use should probably be limited to smaller data sets.
The larger segment of IDS has not seen much innovation for close to two decades until recently. The timing couldn’t be better, given the staggering state of cybersecurity, the industry needs innovation of the category the Bro community is helping to bring.
To that end, there has never been a better time to be involved in the Bro community. The conversations around growing the community, including a consortium of vendors, is interesting. Indeed, that’s one of our biggest takeaways from the conference.
Now watch Adam in action at the conference with “A Bro Primer:”
If you enjoy this post, you might also like: Cybersecurity Tools in Financial Services have become Part of the Problem
Photo: Adam Pumphrey, screenshot, slide 4