Bit9's APT Hacker Break In

We often hear of security companies who provide protection for other companies get breached.  So what's going on here?  How is this possible?  Honestly, I work with a lot of these security vendors and a lot of them don't know security or implement security pretty poorly in their products. This week, let's take a look at Bit9.  From their website, they claim to "continuously monitor and record all activity on servers and endpoints to detect and stop cyber threats".  Yet their break in, according their admission in their blog released yesterday, claims it was a SQL Injection attack that was undetected for at least 7 months.

The attackers, it seems broke in through a SQL injection attack, installed a malicious rootkit, cracked two additional user accounts, and grabbed the signing certificates.  All this activity went on without being detected by their infamous Advanced Persistent Threat Detection security monitoring software and their 24/7 security operations team which monitors everything.  lol.

The only way they found out about this was 7 months later when their customer complained they had malware running in their trusted network.  The malware was digitally signed by Bit9 as "trusted software". 

There are a few questions one must ask regarding this:

1. How did all this activity go unnoticed?  Why did Bit9 not install their own security monitoring software in the DMZ where they hosted the vulnerable machine?  That's a violation of security policy, in itself.  Did they not trust their own software to do the job right? lol

2. Why was an application not security tested before being released to the Internet?  That's a violation of another security policy.  SQL injection is one of the easiest vulnerabilities to find.  A script kiddie could have found this.  A half-decent web scanning tool would have found this. 

3. Access was gained to the server containing the signing certificate.

1. Why are the signing keys available on their Internet facing server?  It should be on an isolated network, if on a network at all.  This is a gross violation of key management.
2. Is Bit9 also claiming now that they don't even run their security monitoring software on the network that contains the keys to their signing certificates?  This is their trade secret and the back bone of their entire business.  If there's one place to run it, it would be here.  It's a gross violation of security policy not to monitor this.  It's more likely that they did run it there but never found the incident.  Bottom line: a lot of security products out on the Internet are fraudulent.

4. Bit9 claimed it was not detected because the system was shut off.  That's the purpose of security monitoring software.  Even if the server had been turned off, all the malicious activity should have been recorded and reported to the Operations Center.  Hello.

5. Bit9 then claims that they discovered the break in shortly after it was turned back online in January 2013.  It wasn't detected before while it was online but now since it is back online, it is detected?  Either they are monitoring the server or they are not monitoring the server.  Which one is it?

6. If Bit9 monitored customer networks 24/7 with a live security operations team, then why was it the customer who had to inform Bit9 that they were infected?  Think about it.  It was running on customer networks for 7 months without detection by their Advanced Threat Indicators (ATI) and  24/7 monitoring Bit9 was supposed to provide.  In the end, it was the customer who had detected it and complained to Bit9.  Bit9 then did an investigation and found malware on additional customer sites.  It was the customer who leaked to the press what really happened because they didn't like the way Bit9 had presented it. You have to wonder what kind of monitoring Bit9 does and more importantly, what value it provides? ;)

All in all, Bit9's account of what happened is inconsistent and highly questionable.

Lessons Learned From This

1.  Companies that do not have security policies, standards, and controls in place are *not* to be trusted with security.  People should refuse to do business with a company that fails to protect itself and fail to detect the security incidents they claim to be an expert at. It seems like an employee could just pop up a Internet facing application on Bit9's DMZ without any kind of protocol or control being enforced by ANYONE.

2.  All applications should be security tested before being released to the Internet, even insignificant ones.  A chain is as strong as its weakest link.  If all the other servers are hardened, but they leave one highly vulnerable server running next to it, then the other servers could be compromised.  That's like locking all doors to the house but leaving the windows open thinking the house is well locked. lol

3. Use security monitoring software that can actually catch incidents.  Have a Pen Tester attempt some nefarious activity and see if it was caught.  And by this, I don't mean run a Nessus scan, or even a port scan. lol.  Anyone can catch those.  Do an APT (Advanced Persistent Threat) break in. 

4. Have security policies, standards, controls as to what is allowed on the DMZ.

5. Have a strong key management policy.