A company asked us for help with a troubling issue: anonymous web site users would randomly become authenticated as other users in their financial services application. The client’s engineering team had no meaningful log data, and wasn’t able to reproduce the issue over many months. They only knew of the issue because of multiple good-samaritan users who reported it. The company hired a pentesting firm to help identify the issue, but after a week long engagement, the vulnerability was still not identified. Carve was hired for a second engagement to investigate the issue and assess the application for other security vulnerabilities.

Once engaged, Meador and the Carve team identified and fixed the issue for the client in one day (see his blog post), and spent the rest of the engagement completing a thorough security assessment of the app. While I’m proud of our results and the value we gave this particular client, the story causes me to reflect on the challenges associated with penetration testing and building/operating secure applications.

All too often, pentesters bring a tool-focused mindset to their work. They run off-the-shelf vulnerability scanning tools designed to quickly find low hanging fruit security issues, re-skin the report in their company’s document template, and call it a day. This light-weight methodology keeps project fees low, but doesn’t provide the depth/breadth of coverage required to accurately assess risk. While the price of the initial engagement was half of our fees, that methodology did not help the aforementioned customer and they burned that cash.

Meador and crew approached the client’s problem with an engineering mindset. They built a test environment using the same framework used by the customer application. They copied client configuration details into the test environment. They wrote custom scripts and tooling to simulate load and user-behavior. By understanding how the application was built, our team engineered a solution that allowed them to reproduce the issue and solve it quickly. And while they did this work, they identified other security issues in the app.

Companies need pentests to satisfy regulatory, compliance, and customer-driven check-box demands for security. The problem here is that what checks-the-box doesn’t always help increase security or reduce risk. Customers who want more than box-checking deserve an accurate description of their security posture, and they need engineering insight to help them improve the security of applications and systems. We call this Security Engineering. Remediation of impactful security bugs, leading business/product/engineering teams to make decisions that increase security, and implementing automation and technical controls that provide lasting risk reduction are some of the hallmarks of a good Security Engineering partner.

Some people are happy spending money on PCPs (Pretty Crappy Pentests, as one of our customers refers to them). But if you’re not happy with the value you’re getting from pentesting, and demand more out of your security assessment engagements, then email me and let’s talk!