How big data can show us the way!

Network security systems receive an alert every time a customer’s system is targeted by malware or other security intrusions.  With the amount of attacks that take place on the internet, an enormous quantity of information is generated.  But typically, that information isn’t used to actually fix the problem.

Recently, HP have been doing something new with the information that they record within their Tipping Point Intrusion Prevention System (IPS)—information from 35 billion alerts recorded over five years from over one thousand customers.  With that information, they build attack profiles using big data analysis techniques.  This has created a brilliant view of both internal and external cyber-attacks.

What they have discovered is really interesting.

  • Each of the thousand customers was targeted at least ten times by a repeat persistent attack from the internet.
  • Some of the customers were targeted over 100 times by similar repeat persistent attacks from the internet
  • In all of the attacks, there was a collaboration across the board of similar attacks, either from similar proxies or using similar weaponised exploit attacks.  That is, the same people or groups were using similar attacks multiple times.
  • The type of attack used by an individual or group did not change across numerous targets.

So we have the information.  What do we do with it?

PrintIn the right hands, that information could be used to target individual attack signatures.  Think of is this way:  For most of us, once we learn a way to do something effectively, we will always try to do it that way.  The hackers are no different.  They all have a signature—the type of tools they use, the profile of the targets, the vector they use to get in, or where they launch attacks from.

Normally, this “signature” is buried among 30 billion alerts.  But big data analysis will be able to recognise it.  Using this information, the legal system could gain the ability to keep up with the mercurial world of the digital bandits.  This creates the possibility that these individuals could finally be brought before the courts.  But if you’re like me, you’re probably thinking:  “Good luck with that!”

It could be a long time before the legal system is able to protect us from cyber criminals.  We also have systems run by governments departments, like CERT and, in Australia, ASD.  But since not many people know about them and what they are doing, their ability to help is limited.  We need to put our collective heads together and come up with a workable solution for all cyber security problems.

The best current approach to the cyber crime problem is for us—the people reading this message—to collaborate in our defence of our businesses.  We need some sort of communication system, similar to the ones attacking bad guys use, where we can keep abreast of an attack and mitigate the risk of it happening to us.  But this idea comes with its own set of questions.  How we do that without exposing our critical IP to each other is one of the biggest problems we face.

The major players in the security space—Cisco, Fortigate, Microsoft and Symantec, to name a few—have systems that inform their clients of what is happening.  If there’s a new security threat, like a Trojan, spyware or phishing email, their clients get a warning.  But access to this information is currently ad hoc and imperfect, since the information you get depends on the security provider you happen to use.  I believe we need some sort of centralised system that takes all of that information from all areas of the cyber security arena and compiles it into something that management and board members can use on a regular basis.

How this could work is like this.  The firewall, IPS or gateway systems report anything sinister back to the main system.  This information is compiled, and the attack vector is analysed, not unlike what they do at the moment.  The difference is, that information is now released to other organisations with the same area of expertise.  They collaborate to develop a tactical response, which is then sent out to all people involved in that area of expertise. 

Some organisations might be wary of a scheme like this.  There is a chance that one vendor’s system could catch more threats or pick up more information than others, and that could expose the under-performing company’s system to scrutiny.  But ultimately, centralised reporting would be win-win for security providers and customers.  All companies participating in the system would be able to offer their clients a much higher level of security.  And customers’ faith in their security providers would be increased.

Big data offers great possibilities—it would be short-sighted of us not to take advantage of it


Roger Smith, is an educator. Teaching students at ADFA (UNSW) and showing them how vulnerable they are to cybercrime.

He is also CEO at R & I ICT Consulting Services Pty Ltd, an Amazon #1 author on Cybercrime and founder of the SME Security Framework. He is a Consultant who specialises in inexpensive and highly effective security strategies for small and medium businesses and not for profit organisations.

He has developed and authored the SME Security Framework and the Security Policy Training Course which are considered to be the definitive guides to helping SME's protect their organisation using the principles of Technology, Management, Adaptability and Compliance.

Leave a Reply