Watch to learn what a real penetration test takes.
Having issues accessing the video above? Watch the video here.
“We want to help you maximize analysts’ time and save money,” says Chad Horton.
Time is money, and professional penetration testing is all about time: it takes time to research which firms are legitimate and thorough, time to understand what the test involves, time to scope your environment, and time to perform the test properly. Chad and Terrill explains to attendees the ways businesses commonly skimp on the time and due diligence involved, resulting in a weak and limited test.
They went on to list out the reasons why penetration tests need to happen in a very specific way–these reasons can feel like sticking points for businesses, and they often result in wasted time. Ultimately, there are no shortcuts in penetration testing. The scenery is only found on the scenic route, and SecurityMetrics Penetration Testing Team is well traveled. Chad and Terrill uses case studies, stories, and years of experience to help attendees understand more deeply what a real penetration test takes.
This webinar was hosted on September 24th, as part of SecurityMetrics Summit 2020.
Thank you for joining us today for our presentation. We're excited to present to you guys what to expect when you're expecting a penetration tester. My name is Chad Horton. I've been performing penetration tests for fourteen years and was the cosponsor and coauthor for the PCI information supplement on penetration testing.
My name is Terrell Thorn.
I have a master's degree in cybersecurity, and I've been managing the team of penetration testers here at Security Metrics for the past three years.
Next.
At Security Metrics, it's our mission to help organizations avoid the negative costs associated with data breaches. Penetration tests are one of the tools get that can be used to do so. So it is our hope that in this presentation, we'll help expand your ability to shop and prepare for your next penetration test. Next.
So today, we're gonna cover a few different topics, to help you know what to expect with your penetration test. First, we're gonna talk about, the importance of knowing what you're buying, then we'll move on to, setting yourself up for success, and then taking action with the plan that you've created. Next.
So it's important to know that not all penetration tests are created equal, and different organizations have different processes and different flows for performing penetration tests. So it's important that you identify what what exactly the organization is going to be performing for you so that you make sure that it meets what you're needing. Next.
So it's important to know that a security assessment is only as good as the scope that it targets. We've had many different organizations approach us for penetration tests over the years, and the success varies greatly based on what the organization chooses to target. For example, we had one organization approach us and ask us to perform a pen test against a single application that had one form on it, which was a payment form. The organization itself had dozens of websites, each of them with hundreds of forms, but all they wanted us to test was the payment form specifically.
You see, what they had done is they had isolated their scope for PCI such that that was the only target that needed to be test. Well, as you can imagine from a pen tester's perspective, with such little scope, there was little to explore. And as a result, there was very minimal in order to report on. Whereas a broader assessment, which may have encompassed more of their more of their exposures would have been able to provide them better results. So the value of a penetration test can be reduced even before it starts by simply putting unnecessary limitations on the scope.
Next. So what I recommend organizations focus on when defining scope is to start with, just a simple question of what do you hope to accomplish by getting the assessment? Is this, compliance objective for you? Do you need to meet PCI or some other, industry standard?
Are you looking for a best practice to mitigate risk, regardless of what type of data you have in there? Or are you looking for a full attack scenario? You're worried about a specific, aspect of the application, and you wanna know what would happen if an attacker were able to do x y z. So oftentimes, organizations need evidence to convince upper management to remove legacy code.
We contracted with one such organization recently that had a piece of software that could only be run on an unsupported version of Windows OS, and the security team knew about this and knew about the risks involved, and knew what was possible theoretically, but they needed additional evidence and additional ammunition to take to the upper management to help make that change happen.
Similarly, we contracted with another organization that allowed customers to write custom Python scripts to filter data from the database, and the Python code would execute on the web server. The development team had brought up the security implications of this of this design multiple times to upper management, but they were never able to get the approval necessary to actually redesign the system.
But now after the report and after the penetration test, they were able to actually get the time and the energy and the funding that they needed to take care of that, outdated software.
All of this is just to say, consider what you're hoping to accomplish with the setting of the assessment. If you don't have a well defined goal, then it's really hard for your penetration testing partner to meet the expectations that you have. Next.
After you've defined what you hope to accomplish and what the scope of the assessment is, the next question is to determine what type of assessment you want to have performed. There's a huge spectrum of how the analyst can perform a test against a given scope. From one side, they can target completely blindly with only the knowledge of the domains and the IP addresses.
To the other side, they can perform it with design documents, source code, and configurations in hand. This spectrum is commonly referred to in the terms of black box, gray box, and white box assessments respectively.
The only thing that is a one of the things that always surprised me is the allure that a fully black box assessment has for customers. We've had organizations request assessments where the analyst is only given the contract name, the company name, and nothing else, leaving it up to the analyst to identify the IPs and the domains associated with the organization before they can even get started testing. Why this surprises me is because as a tester, I am a consultant. And regardless of how much information you give me, I have limited time to work on your contract. What do you wanna pay me to do? Do you wanna pay me to research the Internet and to try and find out what your office IP address is, or do you want me to actually test the security of your environment?
For this reason, security metrics penetration tests have a minimum bar for the amount of information we need in order to perform an assessment.
This includes credentials to all the roles that you want tested, API documentation, and some other things we'll go over here in a few minutes. However, as the organization, you can choose to take it further beyond the minimum bar. By providing source code or configuration standards, you can improve what the analyst is able to do and what he's able to work on during the assessment.
Next.
So, again, after you've asked yourself that question of what do you want to accomplish with the penetration test, it's time to look for ways to prepare for the penetration test to set yourself up for success. Next.
So as I mentioned before, one of the way areas that you can improve the quality of the pen test is by starting to identify which levels of authentication or which threat actors are you most concerned about. Are you concerned about an insider threat from an employee that has a compromised laptop, or are you concerned about a third party who has access to your web application potentially being malicious?
If not, the most common one that we see is are you concerned about customers from the customer's perspective attacking your website?
Before the assessment begins, you get to choose which of these roles you're gonna provide the analyst. For PCI, you've got some prescription there where they're gonna recommend anything that's external, so third parties and customers.
However, as a as a organization, you can choose to take it to whatever level you feel is appropriate.
Similarly, as we talked about before with dot API documentation, APIs are like a maze, and it takes a little bit of give and take and a little bit of trial and error in order to find your way through that API to understand which API calls are prerequisites to other calls in order to get the foreign keys and identifiers necessary to invoke all the API calls.
And what you can do to improve the analyst trial and error period is if you can provide them SOAP SOAP requests or Postman requests or just sample API documentation, you can cut out that trial and error period so that you maximize the analyst time on actually testing the API itself.
And finally, with white listing, you get to choose how the analyst test their time.
You know, around eight years ago, seven, eight years ago, I had the opportunity to test a web application that was using this new CloudFlare WAF. It was in the Cloud WAF service that was being provided. And at the time, I was able to spend about three hours to bypass Cloudflare's protections, and I was able to actually enumerate the database through SQL injection.
Now, the result was about two days later, CloudFlare had noticed my malicious traffic, and they had updated their filters in order to prevent me from exfiltrating the database using the same techniques.
But it didn't necessarily make the organization more secure. That made Cloudflare's product better, but not necessarily the application better. So that's the question I always ask organizations is, what do you want me to test? Do Do you want me to test the effectiveness of your security appliances, or do you want me to test the security of your application and the environment behind those devices?
And if it's the environment is what you're after, then consider white listing the analyst's, testing traffic during the assessment so that he doesn't get bogged down trying to bypass WAFs, DDoS scrubbers, etcetera, during the assessment.
Next.
So now that you've, prepared for your test, you've had your test, you get the report delivered to you, it's time to take all that work and effort and actually put that into action.
Next.
So it's vitally important to remember that once the penetration test is over, your job isn't finished. You get a report delivered from your penetration testing company, and the results that you get from that test and that report are only as good as the action that you take as a result of that information.
Next.
Once you receive the report, it's important to take action on the findings in the report, but it's also important that you don't fall into the whack a mole mentality.
Inside the report, you're gonna see exploit strings of how the attacker was actually able to exploit a vulnerability, and it's very tempting for developers to go and take that exploit string and write a regex or write a pattern to filter out that individual exploit to prevent the vulnerability.
That's a shortcut method, and the more important method to take is to actually gain an understanding of what the vulnerability is, how that exploit string worked so that you can write a more comprehensive rule and fix all potential exploits with it. Oftentimes, what we experience is organizations get in the mindset of let me do this fast. They take that string. They write a rule.
And then me as the penetration tester, I've got loads of different, these station techniques that I can use to bypass those attempts to try and filter them. So it's important that you understand how to fix the root cause of the actual issue, not just the individual exploit stream that's provided in the report. Also, it's important to look for process improvements. How is that vulnerability getting into your code?
Was there a a lack of oversight from code reviews, or was there some other process that should have caught that vulnerability before it got into the application?
Next.
Yeah. And like Chad said, it's really important to look at past vulnerability reports and look for trends that might help you identify these root causes. When we're performing a penetration test, we look at the past reports that we have for a company and look for trends that might help us identify vulnerabilities. But as a company, you should be doing that same thing and comparing your past penetration test, for those with previous years and those with this year. You wanna look for trends that may be an indication of a larger problem and then work to fix those breakdowns in your process.
For example, does your development team have adequate time to take security into account when developing your applications? Is there a security audit process that you go through before launching a major version update? Or is there a group or individual that owns the security of the applications that your team is creating?
And comparing these penetration testing reports with previous years can help you identify areas in which you can improve your process year over year.
Yeah. I often think of a single organization where for four years straight, they received an identical failing penetration test, which was outlining medium to high risk vulnerabilities.
And when we attempted to reach out to the organization and figure out why was this same issue occurring over and over again, their response was they're required to have a penetration test, but at the time they didn't feel like they were required to make the changes requested in the penetration test. In this instance, that organization was no more secure by performing the penetration test than they would have by not performing it. So it's important that you not only look for year over year trends, but that you prepare to take that action on the given reports.
Next.
So hopefully, we have covered, what we have covered sets you up to be a better buyer of a penetration test, one who comes prepared to maximize their output. Next.
We were recently approached by a customer who emulated all the qualities we've discussed here today, and they came as a prepared customer. They clearly communicated the desire of their desire to secure their web application.
They offered to provide the analyst the raw source code in order to assist in that assessment, and then they actively reviewed how issues came to be and established the necessary processes to avoid them, from occurring again. And our hope is that through this presentation, you now have those skill sets, you have that mindset as you're approaching your next penetration test.
Next.
If you have any questions about your upcoming penetration test or, anything about, what we've presented here today, you can email us. Our email addresses are on the screen there, or you can reach out to one of our sales representatives at the number as well.
Thank you for your time.
Thanks.
.svg)
