Next-Gen IDS/IPSs: Caught between a ROC and a hard place [feedly]

----
Next-Gen IDS/IPSs: Caught between a ROC and a hard place
// A Collection of Bromides on Infrastructure

The market appears to have revisited its irrational exuberance about next-gen network IDS/IPSs, perhaps because every major security vendor has one (truth be told, throwing traffic at a set of cloud- or appliance-hosted sacrificial VMs isn't rocket science).

But there's another challenge too: these devices are caught between a ROC and a hard place: They often overwhelm IT with false alerts and (provably) will fail to detect some genuine attacks. So it is important to understand their strengths and weaknesses and to carefully plan their use.

The tech:  Potentially threatening traffic entering the network is forwarded to a VM running on the appliance.  The idea is that if it contains malware, the attacker will compromise the VM and the appliance will detect this and  alert the security team.  Typically only a subset of traffic is forwarded to a VM because attempting to execute all traffic in a small number of honeypot VMs is typically not (practically or economically) feasible.

• In passive mode (IDS), the appliance reports information that can help security teams identify a compromised user device, whereas
• In in-line mode (IPS) the appliance must decide in real-time whether the traffic contains malware or not. It blocks the connection if an attack is detected.   If not, it passes the traffic to the client.

If the malware is on an existing black-list (eg: VirusTotal) detection is easy, but if not, detection depends on  the vendor's "advanced" detection capabilities. Here's the rub:

• If the user is off-net or mobile, the next-gen IDS/IPS will likely be blind to their activity.
• Sophisticated malware is often crypted to ensure that it will bypass existing black-list (signature based) detection methods. So, if the bad guy is determined to get in, the standard detection tools won't help. (The same is true for endpoint AV).   So, most vendors claim "advanced execution detection" that aims to identify tell-tale signs of unknown malware when it executes on the appliance.
• Sophisticated malware is often "sleepy" – and next-gen IDS-aware.  It can detect that it is running in a VM and simply waits (sleeps) until it reaches an actual endpoint before executing its attack. A next-gen IDS/IPS will therefore fail to detect an attack.
• An alert issued by the IDS/IPS for malware that executed on the device relies entirely on the malware actually executing in a honeypot VM.  Key questions to ask the vendor include how you can ensure that the software on the appliance is the same as the software on your endpoints.  If it isn't precisely the same, then the appliance is basically useless.  You may see floods of alerts for attacks that would never execute on your endpoints given their particular patch levels.
• Finally, several vendors ship their own versions of Windows VMs on their appliances.  As Richard Stiennon has pointed out, this likely conflicts with Microsoft's license terms.  You should ensure that your vendor indemnifies your company for any future licensing problems.

Detection's Limits

Ultimately, next-gen IDS/IPS platforms are detection centric, and detection has fundamental limits that are mathematically provable.  Stick with me – I'll try to make the theory simple to understand (Here's a primer, and some state-of-the-art research).

A detector must be evaluated for accuracy by evaluating the frequency of its {True Positive, True Negative, False Positive, False Negative} results:

• TP: The frequency of samples where an attack was correctly identified
• TN: The frequency where a non-attack was correctly identified
• FP: The frequency of false alarms, and
• FN: The frequency of a real attack bypassing the detector.

These can be plotted on a graph called the Receiver Operating Characteristic (ROC), and can be shown as the areas of intersection of two statistical distributions that plot the the detection result for both non-attack traffic and real attacks.

Every detector has a threshold at which it will trigger an alarm (the green line).  A better detector separates the two curves more cleanly, and careful choice of the threshold is critical for accurate separation of real attacks from normal traffic.  The goal is to accurately detect attacks, without increasing False Positives or False Negatives, but no detector is perfect:

1. The detector will fail (FN) at some point and the attacker will succeed. (Yep, it's a definite)
2. Building a good detector is a careful balance of trading off false positives (which leave security teams swamped) against false negatives (which are very bad news).
3. Unfortunately today's rapidly moving cyber-landscape it is impossible to build a reliable detector for polymorphic/crypted malware:

"The challenge of signature–based detection is to model a space on the order of 2^(8n) signatures to catch attacks hidden by polymorphism. To cover thirty-byte decoders requires O(2^240) potential signatures; for comparison there exist an estimated 2^80 atoms in the universe."

The Result: "Compromise-first Detection"

"Compromise-first detection" happens when a detector is unable to distinguish between attack and non-attack traffic, causing significant overlap of the two distributions , as shown below.  The ratio of the TPF to FPF is sometimes called the Signal to Noise Ratio (SNR).  A low SNR loses True Positives in a sea of False Positives, training IT to ignore warnings.

Compromise-first detection is a very big deal. Delays in signature distribution together with detector inaccuracy aid attackers, and the cost of remediation is high: all systems that might have been penetrated must be re-imaged – and if the alert is a false positive, the entire exercise is a waste of time.

The net-net for any network-based detection technology is that it likely:

• Costs a lot more to run (in terms of increased operational headcount and complexity) than the sticker price on the box.
• Doesn't stop attacks that it detects – because operating such appliances inline impacts performance substantially.
• Doesn't deliver alerts that are meaningful given the patch level of your endpoints
• Cannot stop the compromise

Wouldn't it be so much better if endpoints could simply defeat each attack, accurately inform IT without false alarms, and self remediate?  Well, they can!

----

Shared via my feedly reader

Sent from my iPad