QualysGuard Web Application Scanning v2 Review

Having done numerous penetration tests using various manual and automated tools, today we are focusing on a new tool called QualysGuard Web Application Scanning v2.4.1.  In the process of doing a pentest, we often use a quality automated tool to check for standard issues while we focus on the much more difficult issues of the testing.  As this reduces the time it takes to do a full test, allows us to work more efficiently, and besides who wants to waste time doing monotonous simplistic checking.  In this regard, I have used AppScan quite extensively, and HP WebInspect as well, and both are very good tools for the most part.  They help out on the basic checks quite a bit.

Quite recently, I was introduced to QualysGuard Web Application Scanner (WAS) v2.4.1.  This tool was very simple to use which is true to Qualys name.  Point and click and you are done.  Unfortunately, I found out that it didn't help with the standard checks either.

Problem #1
1. It couldn't even authenticate to basic web forms.  I've used AppScan on hundreds of sites, and not once was there a problem in not being able to authenticate.  A web security tools isn't very useful if it can't get passed the logon screen because that's where most of the application resides.  How is it supposed to check anything if it doesn't get passed the logon screen?  The Qualys product support/product manager's response to this is to use Selenium Scripting.  Unfortunately, the current applications that are being tested only run on Internet Explorer (IE) and Selenium scripting automatic record and playback only works on FireFox.  So one must learn a new scripting language in order to make it work with IE.  This is hardly an easy point and click solution.  Learning a new scripting language is time consuming and error prone.  Other professional web scanners have this feature built in.

Problem #2
2. It does not have a built-in manual explore like other professional tools.  For instance, manual explore is needed to fill in certain forms properly in order to get to the critical screens for testing.  For example, you must fill in a proper social security number to look up the customer and get to the rest of the application.  Qualys WAS does not support this feature.  This web scanner doesn't allow the user to fill in the initial forms with proper data thereby never testing the whole application, which is critical.  The Qualys product support/product manager's response was this is a simple point and click tool, "we don't support nor do we plan to support complex features such as manual explore." 

Problem #3
3. Web service scanner has limited functionality in comparison to other professional tools.  In this day and age, many web applications use web services.  To not support this feature properly is ridiculous.  The Qualys product support/product manager's response, "we only support web service fuzzing at this point."  What about testing authenticated web service calls?  It also doesn't support pre-populated data on web pages not web services other than the logon screen.  This pretty much reduces their web service testing to a dummy tool.  To make this work, you have to use tools like SOAPUI or Burp Suite Pro with scripting/plugins to pre-populate data, manual explore, and sequence test steps.

Problem #4
4. Lack of details provided by Qualys.
a. Most professional tools have an audit log that shows exactly what tests were performed and how they were performed.  Qualys does not provide an audit log of what tests they did.  We are supposed to guess instead as to what might have actually transpired.  Real reason behind not providing an audit log is more likely along the lines of they don't all the check they are supposed to and even if they did,  it probably wasn't exhaustive testing of say XSS.  Either way, we have no idea whether they did the work they claimed to have or not.  A Big Mystery Here!

b. No details provided on the actual request/response when a vulnerability is found.  True to Qualys name of simplicity.  The vulnerability finding is so simplistic and lacking any details as to how it was tested, one wonders how to test whether this finding is a false positive or not.  Well, I guess one is supposed to take Qualys word for it. :)

Problem #5
5. Missed critical session management vulnerabilities.  Qualys missed a critical session management vulnerability that I had to find manually that AppScan would have found.  The Qualys product support/product manager's response, "we are putting in a fix for this soon."

All in all, QualyGuard Web Application Scanner (WAS) v2 is lacking quite a bit in terms of quality and details.  Do you want to risk the security of your enterprise by relying on a product like this?  Currently, the product is premature and should not considered to be a proper product to used for PCI approved Web Scanning.  In fact, it should not even be PCI approved until it matures quite a bit.  Qualys needs to understand how a true web application scanner works before releasing a premature product to cash in on a exploding market.  

Update 05/20/2013
A coworker has confirmed the below:
Problem #1. With the latest patch, Selenium Scripting has been successfully used to log into basic web forms.

Problem #2. Selenium Scripting now works for doing manual exploration.