Watch to learn about the application penetration testing basics and best practices.
Having issues accessing the video above? Watch the video here.
In this webinar, SecurityMetrics' Chad Horton, CISSP, QSA, Penetration Test Manager, covers:
Learn more about SecurityMetrics Penetration Testing.
This webinar was hosted on January 26th, 2017.
Alright. We're gonna go ahead and get started.
Thanks everyone for your attendance this morning. We're excited to talk penetration testing with you today. I think today we can provide some valuable insight and help people better understand web application penetration testing.
So a couple of housekeeping items before we get into the meat of the presentation.
The most common question we receive is whether or not we'll be sending out a recording in the slide deck.
So, yes, we will be sending out a recording of the webinar to anyone who registered for the webinar. It will come to the email address that you used to register. So just watch your inbox over the next few days, and we'll email out the recording as well as a PDF of the slide deck for your review. That way, you can share it internally as well if you have coworkers that are interested.
Because this is a one zero one type webinar, we're gonna be going over a lot of the basics. So throughout the webinar, we'll be doing a few mini q and a sessions to ensure that you feel comfortable each section. So we'll get to question slides throughout the webinar a few different times. Feel free to chat in your question as they come up, and we'll try to address as many as we can throughout the webinar. We're gonna try to just keep those to one to two minutes, and we'll address have a bigger q and a at the end of the webinar. So if we don't get to your question, we'll try to get to it at the end. If for whatever reason we get swamped with questions and aren't able to get to them, then we'll reach out to you on an individual basis after the webinar.
With that said, I'm gonna turn it over to Chad to get into the presentation.
Thank you very much. Let's get started.
So, first off, we wanna start off by making sure that everyone understood the intent of this webinar and to make sure that they understand the audience that we're targeting as well. This presentation is not intended to train individuals on actually performing an application pen test. Instead, it goes into what you would need to know as a consumer who wants to have an application pen test performed.
While this presentation may use jargon that's specific to a specific compliance standard such as PCI or HIPAA, The principles we'll be discussing are independent from any compliance standard and can be used in any situation.
So the objective today is to help the audience understand the basics of penetration testing and hopefully be able to speak intelligently with management and pen test providers as a result of what they've learned here today.
So the agenda, as you guys all saw, hopefully, in the emails which were sent out a few days ago, the agenda for today is to start off by talking about application pen testing basics.
We'll then go on to talk about how penetration tests are performed, again, an introduction to that. And then we'll talk about application pen testing best practices as it pertains to the consumer.
So let's dive right in. Application pen testing basics.
What is an application pen test?
It is an app it is an assessment of the security of the code and the use of the underlying software and libraries on which the application runs on. So what this means is that you're gonna be identifying vulnerabilities that are commonly associated with coding errors, such as injection vulnerabilities, broken authentication or authorization, improper error handling, things of that nature.
Most of the issues identified during a penetration test will require a developer to actually go in and recode either a page, a function, or a class, etcetera. But that's the type of issues that you'll be identifying.
Natural question to be asking at this point is what's the difference between an application pen test and a network pen test? PTI has sort of driven this into the industry that there's two different types of pen test, and so I wanna spend a few minutes discussing on the differences between those. As described on the previous slide, an application pen test focuses on the code and the underlying software used to run an application.
In contrast, a network pen test focuses on the entire network, all hosts, all services, everything that is there. During this type of an assessment, an issue an analyst would look for issues such as outdated software, obsolete software, insecure service configuration of flaws in the network design. So if we were to line up the two and compare them, what you've really got is application, which is targeting coding flaws and insecurities of libraries, where a network pen test is targeting everything on the network. It's sort of like the big picture of everything that's there targeting all services.
And the remediation on an application side, a developer is gonna have to go and develop and find a way to fix a vulnerability. Whereas in a network pentests, in most situations, you're installing a patch or a fix that a vendor has already provided for you. You're not necessarily going in and recoding your FTP server or software or anything to that degree. Hopefully, that paints a fairly accurate and detailed description of the difference between an application and a network test.
So moving on, why would why do people look for application penetration tests? The first reason is that a developer is a a developer's job is to build an application that performs a specific function.
They typically are not security experts. Vulnerabilities are often introduced either as a result of poor coding practice or or simply just mistakes, they didn't know better. All the things that an analyst would focus on during an application pen test. They're gonna be looking for those type of errors that occur. These vulnerabilities that are often present can lead to compromises and loss of sensitive data if left on patch.
Data breaches can have a detrimental effect on the company's reputation and bottom line. So for example, using one of the most common breaches in the past few years or well known, well publicized is Target's two thousand thirteen compromise that occurred in December.
In Target's two thousand fourteen annual report, they recorded a loss of a hundred and ninety one million dollars in data breach related expenses.
Now, obviously, target is pretty large. That, number is obviously associated with how large they are, but data breaches in general can have a huge impact to the bottom line of an organization.
And finally, pen test are often required to specific for specific security standards like PCI, HIPAA, many government or banking, compliance standards also have requirements to get penetration testing performed.
So which application pen test which application should be tested?
This is actually verbiage directly from the pen test supplement. We'll actually include some links when we send out the the slide shows to help you get there. But any application written by or specifically for your organization that stores, processes, or transmits data that you wanna keep safe. Ultimately, what you're looking at there is code application code that you have the responsibility for.
Although the stores processes and transmits, that terminology is typically associated with PCI, that is a principle that can be applied and we frequently do apply to scoping out HIPAA assessments or even elective engagements as well. Well, this definition is not necessarily foolproof. It is a great it's a best place to start for beginners.
So common targets of an application, application pen test. The most common target is a ecommerce website. This is probably driven by PCI.
These websites are ones that accept credit card transactions.
While ecommerce are the most common, any application is a valid candidate for an application pen test. For example, we perform pen test on applications for human resources, accounting, banks, hotel and travel reservations.
Many of those didn't actually even accept credit card data, but they had data of sensitive nature enough that they were interested in making sure it was secure.
These applications can be in different forms as well. They can be in the forms of web applications, web APIs, or mobile applications.
Or another common target that we will include in the scope is that of a desktop application that an individual installs on his computer. This includes software that is used to search medical records, scan checks or receipts, or just interface with any sensitive data in general.
So when should a penetration test be performed?
The first and most important principle is before going into production.
It is the best practice to test it while it's in the development cycle, but at a minimum, we recommend testing it before it goes to production.
And following that, we recommend annually. Annually is also a PCI requirement, but it's also just a good guideline for any elective test or other, certification as well.
So with that being said, hopefully, that's some, intro to some basics of penetration testing. Let's step into our first q and a session.
Perfect. Thanks, Chad.
The first question, and I I know we get this often at SecurityMetrics, is can you kinda talk about the difference between a vulnerability test and a pen test?
Yes. Definitely. We'll be talking about that more in-depth later on. But just as a a quick, foreshadowing of what we'll be talking about is a vulnerability test or vulnerability scan is typically an automated solution that is meant to run and identify a wide range of general vulnerabilities or common vulnerabilities.
A penetration test is a manual process that dives in and looks for specific vulnerabilities that automated scanners typically have a difficult time identifying.
Again, like I said, we'll have some more data coming up here shortly on that topic.
Great. So someone asked, with code security, are you talking about static source code analysis or more the functional compiled version in production?
That's a great question. So for our penetration test, and we have a block on this, is the standard web app or application pen test we perform is a gray box where we're actually not provided the source code to the application during the assessment. So we're doing the compiled version whatever is in production or whatever is in the development environment depending on which we're targeting. Now if you refer to our website, we have a great write up. We can send out that link as well to a blog post that talks about the advantages of the different colored box boxed tent pen test, gray, white, and black.
But at this point, we we mainly focus on compiled versions that are in production.
Great. And for PCI DSS, is both internal and external penetration tests by a third party required?
So it seems like there's two questions in there, whether it's by a third party and the internal and external.
So I'm gonna approach those as two separate questions.
It's not required that it's by a third party, for PCI. It can be an internal qualified resource that is organizationally separate. Ultimately, if they wrote the application or they maintain the the firewall, the web server, whatever it may be, then they are not organizationally separate. Ultimately, you have a bias to view into that. So they just need to be an internal resources that separate from the environment they're testing.
But internal and external pen testing is required, and that is a topic that we have some future blog posts coming up on, which will go into more detail, hopefully, to help you understand, when and where that is appropriate.
Great. So we have two more questions that are about this section, then we'll get moving on.
Can application penetration test be used for validating websites that use something like WordPress before they're released into production?
Well, you can test a WordPress application.
It's mainly the underlying module that you're installing in your WordPress. So if you've installed your own custom modules to WordPress, then I would definitely recommend those. Otherwise, we're mainly would be focusing on, just the configuration and the setup and the maintenance of that WordPress blog. So while it can be, it I wouldn't necessarily require it or strongly recommend it unless you're inserting your own custom code via modules in WordPress.
Great. And then one last question before we get moving on. Would you say programming knowledge is a big advantage to be able to perform a penetration test?
Web application pen testing specifically, I I would say it's a huge advantage.
As we'll walk through some samples, some very basic examples of a penetration test here in a moment. But, just understanding how the web application is probably coded and to be able to, in your mind, to do different ways developers could have done it that would have introduced vulnerabilities is a huge asset to have as an analyst. So me, personally, when I'm looking at pen testers or I'm interviewing them, that is one of the qualifications I look for.
Great. We'll continue with the next section.
We'll keep doing more mini q and a's throughout and address some of the other more general questions at the end of the presentation.
Go ahead, Chad. K. Let's jump into performing application pen test. So before we dive in, let's talk about the objective.
So the objective of a application pen test is to identify and exploit issues that could allow an attacker to access or edit data they should not be able to. In order to effectively accomplish this objective, an analyst will need to test all the content on the application. It is not possible to accurately validate the security by sampling bits and pieces or specific vulnerabilities. So if you're looking for a holistic, comprehensive view, it really needs to be a comprehensive test.
So let's talk about some examples of what we mean by that. Can I access other user accounts? Can I edit those accounts that don't belong to me? Can I delete data that doesn't belong to me?
For those of you who had the joyous experience of getting your CISSP, you'll be familiar with this principle within the security triad. It is the CIA, confidentiality, integrity, and availability.
Those are the concepts that we're looking for within a an application penetration test.
Note that the objective is not to test the effectiveness of the security appliances like a web application firewall or an IPS.
This is probably one of the most common misunderstanding in organizations seeking a pen test. Two experiences I've had actually fairly recently with security appliances demonstrate why this is so. The first experience dealt with a behavioral based IPS. And for those of you who are not familiar with those, I'll try and give you a a brief overview of what they are.
What it does is you take the appliance, you put it in line, and you set it into what's commonly referred to as a monitor mode. It's gonna watch all traffic that goes to the web application, all requests, and all variables that are being passed in. And it's gonna identify, like, attributes of each request. So for the messages endpoint, it sees that it's typically an integer that's being passed in, things of that nature.
After a period of time, typically a few days, you turn the monitor mode off and you turn it into active mode. Now it has a baseline behavior.
And while it's active, anytime it sees request that veer from what it saw during the first few days, it'll start raising flags and start identifying potential impact, potential attacks.
Well, on one specific customer I had the opportunity to work with, I went in, identified SQL injection, and was able to enumerate the entire database.
Moments after delivering the report to them, one of their security administrators contacted me and asked me how I bypassed their IPS. It was, brand new. They had just put it in. It was top of the line. They had paid x number of dollars for it and thought that for sure it should have been able to block me from compromising their website.
So I'm like, well, I can go I I'm like, I'm not sure what was going on, but I didn't see any behavior indicative of it blocking me. So you may wanna go review it.
He went over and found out that they actually hadn't turned it off of monitor mode and into act mode. So he went and flipped the switch, turned it on, and now it started blocking.
So he asked me to go and pen test again. So I jumped in, pen tested again. It was in moments I had the entire database again, because it was in monitor mode while the attack was going on.
My attack signatures were now part of the normal baseline and were allowed. So, ultimately, I was able to compromise it again. So it took multiple iterations of me and him going back and forth to identify that he wasn't fully prepped or equipped to use an IPS of that caliber, and that he needed to go back and get training from the vendor to make sure that he implemented it properly.
But this kind of validates or stresses the importance of if the application behind it is vulnerable, then the IPS you're you're only reliant on that that security appliance to protect you. But if you instead fix the application behind it behind it, then the IPS is added protection. It's not the only thing protecting it. It's added protection. And that's really what people need to understand about security appliances. It's not in place of secure coding practices.
It's an added benefit to secure coding practices.
So in a different situation, I identified a a serious vulnerability in a legacy application that the customer just didn't wanna fix. The resources or developers that had built it had since left the organization, and they didn't have the ability to go in and patch it. So their solution was to purchase a web application firewall. They they dubbed it as one of the top of the line and they paid a significant amount of money for it in an effort to patch or, in essence, hide the initial vulnerability.
Well, most web application firewalls work off of signatures.
Signatures of known attacks. They look for keywords or key phrases inside of variables that are indicative of an attacker trying to break in. And when they see those, they raise a flag, raise an alarm, and they block the request preventing it from going to the application.
Well, as I started my retest and I went into, to see if the initial vulnerability was protected via the WAF, sure enough, the WAF did block me. I wasn't able to bypass it. So me being a dedicated pen tester decided to do some research on the WAF and to start profiling its signatures to try and identify what type of request it would or signatures or patterns it was blocking off of. And sure enough, within about thirty minutes to an hour of me working with some of my teammates here, we were actually able to bypass the signatures, find a way to obfuscate our request sufficiently that the WAF didn't block it, and I was able to compromise the database.
So again, this just emphasizes the need for not relying solely on security appliances.
The objective is to identify the security of the application, not the effectiveness of the appliances.
So let's jump into how pen test is performed. One of the first steps that's done or first thing that's done in a pen test is to launch a series of automated scans.
There are different types of scanners that you can run. There are general, general purpose scanners that most people are familiar with. Nessus, Saint, Nexpo, Retina. I think those are some of the most common ones I hear people talk about. But there are also web application specific scanners.
Arachne, BurbnZap are both proxies, but they have a scanner installed on them. Skipfish, w three a f, there are a series of them. They come and go every few years.
But ultimately, those are more specialized scanners that can give you the most accurate results on the whole. So Ness' for example, I find, to be not quite as effective as most web application scanners. Most web application scanners will outperform Ness' pretty consistently.
But there are also specialized scanners that pen testers will use during the assessment such as Derby, SQLNAP, Hydra.
There are other ones as well. I don't use a lot of specialized scanners. Most of the exploitation I do is by hand.
So while each of these scanners will be able to contribute to the result of a pen test, as I kind of discussed during the q and a, none of these will be able to replace the thoroughness of a manual process. Having an analyst actually go through the application and look for security issues themselves.
So the stages of a manual pen test, there are four that you typically will hear when talking to analysts. You've got the first one, which is walk through. Then once they understand the content of the application, they'll move on to identifying issues. Once they've identified all the issues, they move on to exploitation.
And then finally, they'll step back and document and create a report, to give to the customer.
During the walk through, as I kind of talked about before, they're ultimately trying to do is identify unique functionality.
The objective of pen of a pen test is to identify security issues, but that's not in a subset of the code base. They're really trying to evaluate the entire code base and all the funk make sure that they test all the functionality.
Now while SecurityMetrics advertises that we do comprehend comprehensive pen test, we probably average, if I had to guess, around ninety to ninety five percent, code coverage. We've actually seen examples where we get, root access onto a box, and we identify that there's legacy applications or legacy pages, legacy functionality that the customer didn't inform us about. And we've seen our our code coverage actually drop as low as forty percent in situations like that. So it's important that during this walk through walk through stage that the analyst understand what functionality is there. And as you and from the customer's perspective, if you can assist them in identifying legacy code or perhaps it's pages that aren't linked to that you have to, have a direct link to be able to reach them, that type of stuff will equip the pen tester most effectively to perform the application pen test.
Let's walk through a quick example. So, here we've got a basic messaging application.
This is just something we threw together just messing around.
But here you can see that I've got an inbox. I've got a series of messages which I've received. The subject, I can click on each of those. I can log in and be able to view a message.
And it also appears that I like, I've got the ability to delete messages as well. But if you look up there in the URL, you'll notice that the account ID variable, has a number there. That number is probably associated with my account and is used to specify which inbox we're looking at. That's indicative that this page is dynamic in nature and will vary in behavior depending on which number is put there. If I put twelve fourteen, I'm probably gonna get returned someone else's inbox or someone that's someone else's inbox with different messages.
We call that dynamic dynamic behavior and that's what an app a pen tester is looking for because that's the attack factor. That's what he's gonna be assessing. How do they handle that user input? Anytime there's, a portion of the application that accepts input, that's a portion of the application that could be compromised if they don't handle it correctly. So at this point, looking at this page, I would consider the application thus far to be one page so far. But let's keep on going and look what happens when we actually click on a given message.
So here we are on a different page. It looks like slash messages. And we see there that we have a similar identifier in the URL that's used to identify the specific message, that we are looking at and we're reading.
So, again, we see that this is a dynamic page. It changes based on whatever the ID is entered in. So our account for dynamic pages or requests would now be up to two pages.
As we click on the delete functionality, we'll notice that we actually get redirected to what appears to be the same endpoint as the viewing slash messages.
But this time, there's a control variable there called action, which is specifying what action is to be taken on the given message.
So we would actually consider this to be a different dynamic page request because the code associated with returning a message for viewing is gonna be different than the code that's used to return a mess or to delete a message from the database. That's two different code two different trees inside of the code, and we need to make sure that we test both of them. So from a pen tester's perspective, my count would now be up to three instead of just two.
After we've completed the walk through, we would transition to identifying the identifying issues within the application. So these are some common questions that an analyst will use when they're walking through the application in order to determine, like, what type of attacks to attempt. But let's walk through a few of these in regards to the application we just talked about just momentarily to go.
So what does this request do? Well, as we look at this one, it allowed a user to retrieve the content of a given message.
So what shouldn't it do? That would be my next question. If that's what it's meant to do, what shouldn't it do? And a natural response would be view allow me to view other people messages. I should only be able to see messages in my inbox and not ones that were deafened to other people. So if we were to then switch the identifier from twenty one eighty five to perhaps two one eight seven, we can see that here we got a different message and that we were able to access someone else's inbox or messages in someone else's inbox.
This is an example of a broken authorization where the application didn't authorize me as the user to access that message before returning the the contents of it. So, the concept of what does the what does the request do and what doesn't the request do is a great start from a pen tester's perspective on how to attack an application.
Another common stat question that they'll be asking is, how are errors handled? And one common way that you can, detect how errors are handled is to give it something that's unexpected.
In this instance, the mess the application itself was expecting an identifier with an integer value. Well, here, I provided nothing. I gave a blank identifier.
And as we can see, it returned verbose error messages. They're actually called, stack trace error messages that are typically only to be viewed by a developer, not for remote viewing of customers and things of that nature. So in this instance, that is an issue and we'll talk about that more in just a second.
Another step that was is common to be taking taken during a pen test is is user input sanitized.
And one way that I use or one method that I use to identify that is to send characters that are commonly associated with causing SQL errors. So in this instance, I've set a single, a double, and it's a double single and a double quote and then an escape character. And the result is we can see is that it actually returned over both SQL error message, which tells me that they're using Microsoft SQL Server, that they're probably vulnerable to SQL injection, and that there's at least one Windows box in the environment. That's where the database is. But it also tells me that they're using ASP as we can see there slash messages dot ASP. So that error message now leaked a few things to me, but most importantly to me, initially, would be that they're probably vulnerable to SQL inject.
So let's transition over to exploiting issues. Now that we've identified, hopefully, all the vulnerabilities that exist within the application, we wanna transition to exploiting them. And there's a few points I would like to talk about exploiting issues. And the first one is the importance of determining the actual impact of an issue. When it comes to getting management buy in, it's very important that, as a as an employee of the organization, you can effectively communicate the impact of said vulnerabilities to management. If they don't understand that vulnerabilities in a pen test could directly resulted in loss of credit card data or social security numbers or could lead to the entire application being taken down, then they're probably less likely to either maintain the security budget or increase it when necessary. So proper exploitation that demonstrates actual risk is of extreme importance for management and also for just the rest of the company to buy into it.
Not only that, but exploiting issues allows us to identify security issues that are one step removed from public access. So for example, as we'll see here momentarily, through that SQL injection, I could identify that there was a SQL server back there. There are also ways or techniques that you can use in SQL injection to identify the version of the SQL server running there. And with that version, I can now determine if the database is outdated.
And if it is, are there any attack factors or exploits that I can launch to now get shell access onto that server as well. But similarly, you can pivot to other servers within the network if you want. Once you get shell access onto a web server, you can start to scan other web servers from an internal perspective or other servers in the secure zone that may have not even been exploited publicly or exposed publicly, but you can see them from the perspective of the the web server. So exploiting issues is a of, extreme importance.
Oh, and the last thing I'd actually put in there is that you cannot potentially identify evidence of prior compromise. And that's one of the things that we typically look for when we get shell accesses. Okay. I identified this unauthenticated code execution as someone else. Let me look in their logs real quick. Let me do a a quick overview of symptoms and look for symptoms associated with a prior compromise.
But all of that provides value to you, the consumer, in regards to allowing them to move forward with exploiting.
But let using the examples we just went through on the, identifying, let's step through the exploitation process. So here we have the broken authorization.
We identified that as a user a, I was able to view and read messages that belong to user b. Well, the next question I would ask is, are there any sensitive messages the users have sent to each other? It's alarming how frequently we see situations like this. And we identify that customers are sending social security numbers, credit card data, tax questions, passwords, The most random stuff that you wouldn't expect to be in a messaging app appears there.
They're sending very sensitive data. So if I were an attacker or if I were the analyst, the first thing I'd do is launch a burp intruder to allow me to quickly iterate through a large space of IDs, say, one to ten thousand, download all the messages, and then start parsing through it or searching through it as quickly as I could. But other things that we commonly look for is does the application send sensitive information via message? This is common for, like, billing or shipping questions, password reset links, things of that nature.
All of that can be used maliciously if individuals are able to access it when they shouldn't be able to access.
So let's look at verbose errors.
As I mentioned before, this is a stack trace error that's typically not meant to be exposed publicly. So let's look at what information does this leak. Well, it's leaked four lines of code roughly. None of that is severely sensitive.
It's released some of the data and and associated with the stack trace error, like, what where the operation of code was when the error was encountered. So in this direct situation, it didn't leak that much, but it's not uncommon for these type of errors to leak highly valuable information. We've seen common experiences that we'll see is, like, path information, code snippets to very sensitive stuff such as encrypting credit cards. We've even actually gotten database credentials out of error messages such as these. So it's important that these are disabled for remote viewing.
In regards to the SQL injection one, as we talked about before, this leaked quite a few air quite a few, bits of data. It told us that you're running Microsoft. It's coded in ASP.
You're probably vulnerable to SQL injection. There's quite a few. An analyst at this point would have two choices, exploit it manually or use an automated tool such as SQL maps to try and exploit it. I personally am partial to manual exploitation whenever possible because I, I believe that I can send fewer requests and can optimize the attack over an automated solution. So I typically my preference is to do it manually, but there are many tools that could be used specialized tools that could be used to exploit it the vulnerability and retrieve data from the database.
But by exploiting this particular issue, the analyst would be able to quickly enumerate the database and to be able to tell you which DB user, the requests are being sent as, what the version of the database is, if stored procedures are being used.
They'd be able to quickly evaluate or profile that database in general.
So the next step in a penetration test is the documentation phase.
And it's important to note that this is the only deliverable of a pen test. And so it it's very important that you look for a few things when you're evaluating a pen test. The first thing is, what is the scope of the assessment? Did they clearly identify that they tested what you expected them to test?
And did they define any limitations or obstacles that inhibited them from testing?
But it must but more importantly, it must be it is important that they've clearly identified for each issue, what the issue is, where the where the vulnerability is, all instances if possible. If it's if it's a pandemic and it affects everything, they need to identify that as well. What the impact is if exploited and how it can be fixed.
But for me, like, one of the things I I typically stress to individuals talking about reports is that it's important that the issues not be generic text. And we'll talk we'll see this in just a second. But when we say what is the issue, I'm not looking for the definition of SQL injection or examples of how, SQL injection can be used to retrieve data from a database. Instead, I'm I would ask the question, what is the issue that pertains to this application?
Where is the issue within this application? What is the impact if exploited to this application? Not in general, tell me specifics. And as we step forward here in a second, you'll kinda see what I'm talking about. The reason why I place such high emphasis on that is because as a security firm, we've been asked to evaluate a wide range of pen test reports anywhere from glorified vulnerability scan reports to well documented attacks. In some of the reports, even I, with ten years of experience, couldn't understand what the penetration tester was saying. And so from the customer's perspective, I'm sure that they were even more lost than I was when they read that report.
But let's jump into what we just talked about. So broken authorization. I'm not gonna read all the text here. I've mainly put this here for you, to be able to go through when the slides are sent out and also when we send out the the webinar recording.
So that you can, spend more time if you would like, but this gives some very direct examples as to what I was talking about. So, there's the broken authentication. What is the issue? Users can view messages that belong to different accounts.
It doesn't have to be long, but that's enough data to tell the developer, oh, in our application, we've got this flaw. What is the impact? They can now view the content of those messages and attempt social engineering fraud or other such activities.
Now that sort of places helps the developer place emphasis or priority to that vulnerability. And then how can it be fixed? Have the application authenticate and authorize the user to view a given message before returning the message associated with a given ID. Now that's given direction. I haven't told him go and change this line in the code because, again, I don't have the code, but I've at least given him the detail for his application on how he needs to fix the issue.
Similar on verbose error messages, you can see that we've given specific feedback. How can this issue be fixed? Well, catch errors and then return a generic error message if the user is removed.
And similarly on sequel injection, I haven't given a definition of what sequel injection is. Instead, I've just said, the application does not sanitize user input. This allows the analyst to inject their own SQL queries to be executed by the database.
I've now told them what the issue as it is as it pertains to their environment is and then impact and how can it be fixed as well.
So with that being said, we're running a little bit short on time. So I'm gonna jump over to the quick Q and A for this section, and I'll hand it off.
Perfect. We have a few questions to address at this point. I'll probably hold off on a lot of them until the end of the webinar just to ensure that we can get through the entire presentation as I know. That's why you guys came.
Is it necessary, Chad, to do an exploit if you're doing a penetration test for PCI compliance?
That's a great question. And even in when we did the when we were working on the penetration test information supplement on pen test sorry. PCI supplement on penetration testing. That was a common point of discussion that we had.
Exploitation is definitely rec required per se in the PCI DSS saying that exploitation has to be there.
But in some instances, it may not always be possible to exploit. So say for example, we've done pen test where, the environment is sending gigabytes worth of traffic every minute. And if we attempt exploits that potentially impacts the environment, it could take it down and cause thousands and potentially hundreds of thousands of dollars worth of damage to the environment. So either they'll put us into a hot site, which mirrors as closely as they can production and have us test there and try the exploit, or they'll just have us report the vulnerabilities directly to them.
And then it's between them and the QSA or them and the merchant bank in regards to whether that's sufficient. So, I definitely recommend it. I think that's one of the most valuable pieces you'll get out of a penetration test. However, there are certain circumstances where it cannot be avoided that exploitation must be circumvented.
Great.
Most of our other questions are a little more general in nature for penetration testing. So I'm gonna turn it back over to Chad so that we can address them at the end of the webinar. That way, again, we ensure there's enough time for Chad to continue the presentation.
Sounds great. So let's jump into application penetration testing best practices.
So as you're shopping for a penetration test, there's a handful of topics, and I've chosen five of the most common ones and the most important ones as I view it, that a consumer needs to be aware of. We've kind of indicated this twice previously, but the concept of automated versus manual, quote, unquote, penetration test.
Whether you're doing a comprehensive or a targeted penetration test, whether you should test in production or nonproduction, what are the ramifications of that choice, evaluating how to evaluate a pen test provider and how to evaluate a deliverable.
So let's try and jump in real quick. So automated verse manual.
Simply said, automated is cheaper, automated is faster.
However, manual results manual testing will always yield more accurate results.
Because there's no industry wide accepted standard on how to perform a pen test, this is the first topic I I felt that needed to be discussed.
I was once asked by an individual in the banking industry, what defines a manual pen test? Does pushing a button manually define a manual pen test? To which I quickly responded that it it didn't.
From my perspective, a manual penetration testing, consists of verifying and validating the findings from automated scanners, but then manually assessing the application at a minimum investing time looking for security issues that automated scanners can't find or have a difficult time identifying, then exploiting any valid findings manually most of the time. However, there's many times where we'll use a specialized scanner, but most automated, quote, unquote, pen test solutions don't invest time into exploiting. They just run automated tools to identify vulnerabilities.
And then lastly, manually translating report data from generic information.
This is a description of SQL injection. This is a generic description of how to fix it. To specific information that the developer needs to know and have access to. I mean, that pen test report is the only area that they're or only resource they have to get specific guidance on their environment.
If a customer wanted guidance on what is SQL injection, there's plenty of resources they can go to. They can go to OWASP. Even Wikipedia has great articles and great content associated to the definition. But a pen test report needs to translate that to specific for the environment.
You know, perhaps my view is simply, a purest view. However, I feel that I have repeatedly over time been able to demonstrate that manual tests are more effective than automated at identifying comprehensive, comprehensively assessing an application.
And in the next slide, I'll actually provide you some generic, data. Over time, we're actually, we're actually discussing publishing some of our stats and how we set up environments to evaluate this. But this gives you a quick idea of why I, I I feel the way I do. Included on the left hand side are the foremost common categories of high slash security sorry, high or critical security issues that we identify during our web application pen test. In the two right hand column columns, I've compared the ability of automated and manual testing to identify said vulnerabilities.
So of the four most high critical risk issues that we identify, automated is only effective fully effective at identifying one of them and is less severely less effective at identifying the others. To put some context to that statement of limited, I I use it in terms of seventy five percent of the issues or less were identified, which means that the automated scan missed a quarter of the potential vulnerabilities, at least a quarter of them. So, again, for me, if, until the day automated scans can at least match or come close to matching the results of a manual pen test, I do not recommend a fully automated penetration test. Again, I think you've heard me mention this a few times, but we have an internal term for them. We call them glorified vulnerability scans. That's ultimately what they are. And I'll if we have enough time, I'll talk to you guys a little bit more about that in the coming slides.
The next question I get is comprehensive versus targeted. The most accurate assessment of a secure of the security of an environment can only be obtained by performing a comprehensive test. Ten years ago, I did a forensics. I, was called into a forensics investigation, to help them identify how an attacker was actively exfiltrating data out of a website.
The customer didn't wanna take down that website during forensics even know that knowing they were being compromised. And so they called me in to try and see if I could identify where the vulnerability was. This particular website allowed you to purchase towels from it. And, after about twenty or thirty minutes of searching the website, identified a variable which allowed you to determine which color of towel you were gonna purchase.
Red, blue, black, white, etcetera. And that color code was actually vulnerable to SQL injection.
Within minutes, twenty minutes, thirty minutes, I was actually able to start exfiltrating data out of the database and mirror what the attacker was doing at that time. And so we were able to help them fix it, but this kind of helps demonstrate that something as insignificant as the color code, a variable that wasn't even on the portion of the application that accepted credit cards or dealt with sensitive data, all it did was affect what color of towel the picture of the sorry. What color of towel, the picture was loaded. So which which picture was loaded.
But that was vulnerable SQL injection and led to a complete compromise. And so it kinda capitalizes the need for a comprehensive test. But with that being said, there may be times and there often are where hourly assessments can be of value and help the organization meet its, own security objectives. So for example, when there's budget restrictions, comprehensive web application pen test are expensive and can cost quite a bit of money, and so not always can you afford a comprehensive one.
When new code or functionality is getting ready to be deployed, at SecurityMetrics, we do frequent pen test. And every time our development has gone through another sprint and develop new functionality that they're gonna deploy, they have us do a test against any code that was affected by that release. So that helps demonstrate how we're not always doing comprehensive, but we do small targeted ones, and we're still getting value out of that assessment.
Ultimately, it's up to you, the customer, to determine how you wanna test the application. But But hopefully, through this discussion, now you feel a little bit more prepared or equipped to make that decision.
Production versus nonproduction. I'm not gonna read through all of these, but ultimately, you can review these and see which what are the advantages and disadvantages to each. If you can create a identical instance of the application in a nonproduction environment, there's a lot of advantages. And the biggest ones that people are interested in are it won't impact normal business operations.
If the analyst inadvertently exploit or exploits a vulnerability that inadvertently brings down the application, it's not gonna affect your customers. It's not gonna be visible to customers, and they can send whatever strings they want. Dirty data doesn't matter. It's very easy to revert back.
However, in production, you're gonna be able to show exact impact, exploit potential, and so you'll be able to say, I was able to target from this web server to this one with more accuracy than you would in a nonproduction environment.
I'm sorry. I'm gonna be moving a little bit fast. We're running short on time here, but, let's move on to evaluating pen test providers.
So what type of questions do they ask you before quoting you? This isn't a perfect analogy, but it helps capitalize or or demonstrate what I'm what I'm talking about here. Imagine you're purchasing a new house. You wanna build one.
You go to a builder and say, hey. I wanna build a house. And they say, great. Where do you wanna build it?
Well, I wanna build it over there. Okay? When do you want it done? I want it done by then.
Excellent. Here's your quote.
A builder who would only ask those two questions obviously doesn't understand what type of house you want. He doesn't understand how big you want it, if you want an entrance and you're downstairs, what type of functionality you want in the house. So you're probably not gonna get what you want. Similarly, if a a pen test provider doesn't ask you questions about the size or functionality of the application, they probably don't understand what they're getting into, and that can be very indicative of a pen test provider that relies solely on the web on vulnerability scanners.
They're not doing much manual testing at that point. The second thing is what type of experience do their pen testers have that will help them in assessing your environment. If If someone were to approach me today and ask me to pen test an IBM mainframe, I would have to respond to them and say, well, I can research it. I can implement or check for best practices there.
However, I have no experience with mainframes, and my ability to effectively evaluate that is gonna be a little limited. I would have to turn to one of my other analysts who potentially do have some experience with mainframes, to rely on. But if you approach a pen test provider and they don't have any experience either coding, maintaining environments, or maintaining, yeah, applications, then perhaps they don't have the necessary skills to be able to evaluate them. Well, certifications can be used.
They're not fully reliable.
Additionally, just because Appendenters haven't been pen testing for a very long time doesn't mean they're not effective. But, again, that's just another question you can ask to start getting some idea of the quality of the provider you're getting. And last but not least, make sure that they're familiar with the organizational the standard your organization is trying to comply with because they may, be able to help you better if they have experience than if they don't.
So when when people ask me about using internal resources for, performing pen test, the number one thing I ask is are they organizationally separate from the environment? While this is a PCI requirement, it's a best practice in general. There are environments here at SecurityMetrics that my my department has the opportunity to put up. And when we put those up, it's our policy that no one involved with either building the infrastructure or developing the code can be involved with the assessment. Because we've seen time and time again how that can sort of provide a bias going into the assessment, and you take things for granted.
Similarly, the previous questions still apply when evaluating resource internal resources. So that's a great place to start.
Evaluating a deliverable. I have some more stories, but I'm not gonna be able to share share with you, but I'll summarize by saying this.
Imagine how frustrated or let down you'll be if you spend thousands on an assessment and then you get a report that resembles the vulnerability scan you paid hundreds of dollars for.
Request a sample report from the provider and evaluate from that report, could your organization take action on the information they provided, and will your auditor accept or approve a report of that quality?
We've actually had instances where pen customers have gotten a pen test from a different resource.
We got the report, and the QSA's had to turn it down because the quality was substandard.
They didn't actually perform a pen test. It was a vulnerability scan. And so now the customer's left in the situation where they just paid that provider, and now they're gonna have to pay a different provider to test the exact same thing. So, those are some great, guidance questions to keep in mind when evaluating PenTest. So let's jump on the takeaways. Hopefully, our objectives were effective. And as a result of this, webinar, you feel like you better understand the basics of application pen test, and you feel able to have effective conversations with the management and providers in regards to web application pen testing.
With that being said, I'm gonna jump over to questions.
So Great.
Thanks again, Chad. That was very informative.
We received a lot of great questions. Some of them we're probably gonna have to reach out to on an individual basis because they're more specific to your environment or applications.
However, we do have a few great questions that we wanna address here in our last five minutes. So, Chad,
That's a great question. So we're doing this series. We did a a webinar about four months ago, I believe it was, on segmentation checks that gave you the basics of a seg check and where it needs to be applied. Today, we're talking about web applications and where it should be applied. And in a few months, we'll be doing another one in regards to sorry. Network pen test and where they need to be applied. So, our goal is to be able to provide you, the audience, content to help you guide that.
But with that being said, the first thing that I look for is the perimeter of the environment I'm trying to protect.
Ultimately, I imagine it like a wall or a fortress. And I want to identify any avenues into that environment or open ports.
If there's any open ports that lead to custom code, like a web application, then you need an application pen test. If there are any ports that lead to a off the shelf software such as, like, FTP, SSH, RDP, then you need a network pen test. And if there's no ports exposed, then you need a segmentation check.
That's kind of quick and cut and dry. But if you have questions in regards to knowing, identifying that type of information, I would strongly recommend calling or contacting our enterprise sales group, and they can help you in more detail than what I'm able to hear.
Great.
Before I move on to a few more questions, just a reminder because it did come up multiple times again. We will be sending out a recording of the webinar as well as this slide deck, and we'll include in that email a few links that Chad has referenced to blog posts that we've provided that, in fact, might even help with determining the type of pen test as well as methods of penetration testing, and we'll also have a link to the supplemental guide that's been referenced a few times for your for your review.
So, Chad, you've talked about automated solutions versus manual. Do you use an automated solution first before you get into the manual to help decide where to attack?
So as as we walk through, there's sort of those stages.
I find that it's the it's of extreme importance for me to walk through those stages in order. So I first off do the walk through, complete that, and then I try and identify all the vulnerabilities.
Part of that identification step is I'm gonna validate the findings from the automated scan, and then I'm gonna go through the rest of the application and identify everything.
That way, I can assure the customer that I've tested everything, and to the best of my ability, I've identified all vulnerabilities in the application before I move in and dive in and start attacking and exploitation.
Because that also allows me to have sort of like a a buffet, if you will, of vulnerabilities that I can choose from. And I get to choose whichever ones I wanna dive in first. But, typically, I'm gonna dive in after the ones that have the higher highest likelihood of resulting in a serious, serious compromise. So for both error messages, typically take me about five to ten minutes to evaluate, but SQL injection is gonna take me thirty minutes or an hour. So I'm probably gonna dive into the SQL injection first, and verbose error messages has, like, a a lower return value.
It's it's not gonna provide me the same value as SQL injection would be. So I do SQL injection first and verbose error messages.
I don't know. Hopefully, that helps answer the question that you had.
Great. And you talked about someone needing to be organizationally separate if they're an internal resource performing a penetration test.
Yes. So, the definition, as I understand it, is they're not in any of the departments or over any of the departments that manage, maintain, or build the environments to be tested. So the develop the manager of development, who is in charge of building the web application, cannot be, put, in charge of performing the application pen test because he's organizationally, involved in the responsibility of building or maintaining it. But someone who is in sales or, maybe they're in IT support at a different location and they don't have anything to do with that web application, then they are organizationally separate. They may manage some infrastructure, but they don't have manage the infrastructure that is gonna be tested. So that's kinda what they're looking for in that organizational independence.
Great.
And we'll wrap up with this last question.
Yeah. There are tools that you can use. At SecurityMetrics, we're not very well versed in those tools simply because for PCI and even HIPAA assessments, you're not required to include denial of service attacks, and we actually, prefer not to because just the very nature of them. So I'm not sure I could provide you a lot of great guidance in in regards to there. I'll see if any of my teammates do, and if we do, we'll we'll send it your way.
Great.
But it looks like you also mentioned is there a WAF or an IPS that would be a solution for protecting against them. There are d sorry. DOS scrubbers out there that you can contract with. They're ultimately a third party that you put in line, and they can help protect against those type of attacks.
Our QSAs would have better information. If you reached out to one of our, enterprise sales group, they can hopefully put you get that question answered for you.
Great.
So as I mentioned, we had a lot of questions, some we weren't able to get to. We'll reach out to you on an individual basis within the next few days to help address those. So we wanna thank everyone for their attendance. Again, watch your inbox for the recording and slides. And if you have further questions, like Chad mentioned, contact us here at SecurityMetrics, and we'd love to point you in the right direction. Thank you.
.svg)
