Layers of Cybersecurity: Signature Detection vs. Network Behavioral Analysis

signature detection vs network behavior

Signature-based detection techniques have been used since the earliest days of security monitoring. Virus scanners used signatures to identify infected files, and the earliest intrusion detection systems (IDS) relied heavily upon signatures definitions.

In previous years, these provide adequate protection until adversaries became more advanced.  Bad actors discovered methods of evading signatures, leaving the first-generation of signature-based detection systems ill-equipped to protect organizations from threats.

In an effort to determine a longer-term solution for these threats, new techniques were created to look for the effects of attacks rather than identify unique characteristics of the attackers.  This provides the benefit of potentially discovering unknown threats, but this technique does not come without challenges of its own.

Limitations of Signature-Based Detection

Signature-based detection is a process where a unique identifier is established about a known threat so that the threat can be identified in the future.  In the case of a virus scanner, it may be a unique pattern of code that attaches to a file, or it may be as simple as the hash of a known bad file. If that specific pattern, or signature, is discovered again, the file can be flagged as being infected.

As malware became more sophisticated, malware authors began using new techniques, like polymorphism, to change the pattern each time the object spread from one system to the next. As such, a simple pattern match wouldn’t be useful beyond a small handful of discovered devices.

In network detection systems like IDS, signatures are defined to look for characteristics within network traffic.  One of the more common definition methods are “Snort rules”.  A Snort rule defines characteristics in one or a series of network packets to identify malicious behavior.

For example, a Snort rule can be written to identify command-and-control (C2) traffic between an infected device and the adversary, regardless of where the adversary’s servers are kept.  While it is more difficult for adversaries to obfuscate network packets to evade the signature, it is relatively easy to encrypt the traffic, complicating the detection process.

One of the biggest limiting factors behind signatures is that these are always reactive in nature:  You always have to start with an instance of a virus or an understanding of a network attack in order to write a signature to detect them.  This means signatures can’t identify unknown and emerging threats.  Signatures only identify threats that are already known.

>>> Related: 7 Security Trends Shaping Today’s Threat Detection Technology 

Essential Supplement of Behavior-Based Detection

Unlike signature-based detection, behavior analysis is not searching for unique characteristics of the specific threat, rather it is looking at the results.  In medical terms, think of signatures as being a blood test to see if you are infected with a specific bacterium, while behavior analysis is observing your symptoms.  If you have a sore throat, runny nose, fever, chest congestion, you are probably sick.

In endpoint detection, this means looking at what any individual process is trying to achieve.  Regardless of fingerprint, if an executable is trying to gain privilege escalation it is possibly up to no good.

When looking at network behavior, it can be even more complicated.  Some products create a baseline for normal traffic patterns and then trigger an alert when there are anomalies.  Others try and identify when specific connections behave in unexpected ways.

The benefit of behavior analysis is that it has the potential of discovering unknown threats. One side effect is that it is prone to false positives.  In the medical analogy, you may be hot, sweating, and present labored breathing due to a cold…or you may have just exercised.  Additional context helps to sort these results out, but when false positives outnumber genuine detections, the solution can be more trouble than it is worth.

Additionally, behavior analysis can be much more resource intensive, so relying upon it to identify known threats may be expensive and runs the risk of missing a threat that would be easily identified with a signature.

A Balanced and Layered Defense in Depth

Both detection techniques are useful in a balanced and layered cybersecurity defense.  The Pareto principle (aka the “80/20 rule”) is a useful contrast for understanding how.

Eighty percent (or potentially more) of the incidents in your environment will be easily identifiable by signature-based detection. In fact, signatures are the most effective method of detecting known threats, which means it remains a fundamentally important methodology.

On the other hand, twenty percent (or less) of the issues will not be identifiable by signatures, but will likely cause eighty percent of the problems.  If your organization is under targeted attack, chances are, it will not be by an easily identifiable or known threat. So, behavioral analysis is clearly essential.

Modern cybersecurity defenses are balanced and layered, which means including detection methods for both known and unknown threats. Effective organizations can easily identify, prevent and dispatch of known threats using a signature-based solution – and complement this technique with behavior-based solutions in order catch the unknown threats a signature-based solution may miss.

If you enjoyed this post, you might also like:
Snort, Suricata and Zeek: 3 Open Source Technologies for Securing Modern Networks 

Back to Blog

Bricata and Garland Technology Announce Partnership
Technology Partnership delivers total network visibility and threat hunting to accelerate detection and response
+ +