Testing Times: the importance of the right mix

One of the most important requirements of the PCI DSS is that the organisation regularly tests its security systems and processes.

One of the most important requirements of the PCI DSS is that the organisation regularly tests its security systems and processes. These testing requirements are broken down into several key activities:

  • Quarterly internal and external vulnerability scanning
  • Quarterly wireless scanning
  • Annual internal and external penetration testing

This article describes what is required by these tests, how to get the most out of them and, ultimately, how to reduce your business risk whilst satisfying PCI requirements.

Vulnerability Scanning

Vulnerability scanning refers to the use of automated tools to scan systems for weaknesses. As the tests are automated in nature, exploitation or deeper penetration is normally not performed, but these tests will identify any ‘low hanging fruit’. For example, common configuration errors or missing patches should be detected by an automated vulnerability assessment.

External – ASV

External vulnerability scanning on a quarterly basis is perhaps the most stringent of PCIrequirements, as it must be performed by a PCI Approved Scanning Vendor (ASV). Unlike other tests, the results are of the scan are shared with the Merchant’s Acquiring Bank, (or with the Payment Card Brands for a Service Provider) and vulnerabilities identified in the scan could lead directly to non-compliance.

The scan must be conducted not just against systems storing or processing cardholder data, but against any systems that could provide a path to those within the Cardholder Data Environment (CDE). This can make the scope of an ASV scan difficult to determine without detailed knowledge of the internal network, and therefore something that a specialist internal resource or a QSAshould be able to assist with.

In order to maintain compliance, an organisation must obtain four quarterly passing ASV scans. A failed PCI scan does not necessarily mean a loss of compliance. As long as the issue is remedied within an appropriate time period, compliance can still be maintained. If a weakness is resolved and a re-scan is performed before the end of the quarter, the environment will be considered to have passed.


Internal scanning must also be performed on a quarterly basis and can be conducted by either a qualified external company (not necessarily an ASV) or a qualified internal resource. Internal vulnerability scanning should cover every single component within the CDE. It is used to identify systems which could have exposed vulnerabilities, either through misconfiguration or missing security patches.

There is no specific guidance on conducting internal vulnerability scans, but MWR recommends running authenticated scans (scans with administrative login credentials) which can interrogate the system further to identify missing patches or configuration issues.

Wireless Assessment

The wireless scan is intended to uncover any rogue access points that may have been connected to the CDE. These situations expose the internal network to direct outside communication and, following the infamous breach of TJ Maxx in which rogue wireless devices were a key component, they are now a hot topic for PCI. Having a method of detecting rogue wireless devices is required whether your organisation makes use of wireless networks or not.

Wireless devices may have been maliciously placed as part of a targeted attack or, more commonly, unregistered devices are naively installed by employees for their own purposes. An effective wireless assessment should therefore cover all areas that have connectivity to the CDE, including both corporate office locations and data centres.

Wireless access points must be identified on at least a quarterly basis. This can be performed either by an external company or by a qualified internal resource. Typically this will be in the form of a sweep of the premises using a wireless spectrum analyser. The scan should cover all common 802.11 WiFi frequencies (including those in bands a,b,g and n) and aim to locate any unknown access points.

An alternative approach, which is common for organisations with a modern WiFi infrastructure, is to rely on the IDS capabilities of those devices. A mesh of corporate access points with IDScapabilities can alert whenever an access point is detected and, based on signal strength, calculate an approximate location. Where this approach is taken in place of regular scanning, the process must be supported by a detailed response procedure to any alerts.

Penetration Testing

There is no such thing as a PCI penetration test. The standard does not provide clarification: it states that a penetration test must be performed, but not what this entails. A high quality penetration test should not only satisfy the requirement but provide valuable information into the effectiveness of your controls. Here are some of the things that should be considered when looking for a penetration test to satisfy PCI requirement 11.3.

Goal Driven

It should go without saying that the goal of a penetration test within a PCI assessment should be to simulate an attacker attempting to recover cardholder data. Too many penetration tests aim for complete network coverage and report a list of technical vulnerabilities discovered. This is what a vulnerability scan is for and not what you should be paying a skilled consultant to perform.

A goal driven penetration test should also seek to assess any of the methods that could be used to obtain cardholder data. It should include physical security, test the awareness of your staff, test your remote sites, and test any exposed surface. This approach is more realistic, more likely to succeed, and could result in your organisation needing to reassess the effectiveness of some controls.

So why take this approach when it is clearly more costly than a simple tick-in-the-box test? Because it’s also less costly than a breach, and a real attacker won’t say, “Sorry, I didn’t realise that system was out of scope”.

Identifies Attack Paths

A penetration test report that reads, “your card data was compromised” is also of little value to the organisation. The penetration test should first identify, with your organisation’s support, potential routes into and out of the CDE.

Once these routes have been identified, the test should seek to confirm or deny which of these attack paths are viable and, if any is, which vulnerabilities made this possible. In this way, your organisation can determine where stronger controls are required and apply them appropriately. It will also allow a defence-in-depth model to be applied, with multiple layers of controls, rather simply patch fixing individual issues.

Assesses Scope Boundaries

Understanding the scope of systems which fall into the CDE is a particular challenge for any organisation, and typically involves detailed reviews of network diagrams, and firewall, switch and router configurations. For complex networks with many lines of communication, it may not immediately be clear what is connected to systems in the CDE (and hence what is also in scope for an assessment).

The penetration test should seek to identify any route into the CDE from the outside, whether this is external to the company (such as from the Internet or MPLS connections) or from a trusted part of the network that is considered outside of the CDE. This approach will confirm that the scope of your PCI DSS assessment is correct by proving (or disproving) the presence of controls at the boundaries.

Maps Vulnerabilities to PCI Requirements

This is a severe issue in the majority of penetration tests. Where technical vulnerabilities are discovered, the root cause of these issues is not determined, but simply reported as a High risk or a 4.0 level risk or similar.

The penetration test should not just identify vulnerabilities but seek to determine which controls (put in place to meet PCI requirements) are not functioning effectively. This requires input from a strong technical perspective as well as that of experienced QSAs.

For example, a weakness in an unpatched server can easily be remedied by patching it. But how has this server come to be in this state, if patching is performed in line with PCI requirement 6.1? Similarly, weak user passwords can be remedied by changing them to a more secure alternative, but this is an indication that both the password policy in place (to meet requirements 8.5.x) and the security awareness training (requirement 12.6) are not effective.

Changes in Environment

A final part of the standard, and one that is common to all the testing requirements above, states that tests must be performed after any significant change in the environment. This is defined as any significant infrastructure or application upgrade or modification. This statement is open to interpretation by both the QSA and your organisation, but such changes could include:

  • Major software version/branch upgrades
  • Major version code releases
  • New servers added to the CDE
  • Operating system upgrades
  • Additional sub-networks or VLANs added to the network
  • Change of network architecture
  • Deployment of a new platform or solution


The number of testing activities you need to perform can seem significant, but is in reality no more significant than those undertaken by any organisation – PCI or otherwise – that takes a mature approach to security.

The quarterly ASV scans are most crucial in maintaining compliance, reflecting the importance of securing the perimeter of your network. Wireless scanning will identify any risks that bypass perimeter filtering and allow access into the heart of the network. Detailed penetration testing provides the final validation that the controls you have implemented are functioning effectively.

It is vital that any weaknesses that are found are mapped to underlying PCI requirements. ForASV scans, this is a mandatory part of the process; for other tests, you have to rely on your testing provider to do this. Ultimately, these tests will benefit your organisation, reducing your business risk as well as fulfilling another compliance check box.



Accreditations & Certificates

MWR is an accredited member of The Cyber Security Incident Response Scheme (CSIR) approved by CREST (Council of Registered Ethical Security Testers).
MWR is certified under the Cyber Incident Response (CIR) scheme to deal with sophisticated targeted attacks against networks of national significance.
We are certified to comply with ISO 9001 and 14001 in the UK, internationally accepted standards that outline how to put an effective quality and environmental management systems in place.
MWR is certified to comply with ISO 27001 to help ensure our client information is managed securely.
As an Approved Scanning Vendor MWR is approved by PCI SSC to conduct external vulnerability scanning services to PCI DSS Requirement 11.2.2.
We are members of the Council of Registered Ethical Security Testers (CREST), an organisation serving the needs of the information security sector.
MWR is a supplier to the Crown Commercial Service (CCS), which provides commercial and procurement services to the UK public sector.
MWR is a Qualified Security Assessor, meaning we have been qualified by PCI to validate other organisation's adherence to PCI DSS.
As members of CHECK we are measured against high standards set by NCSC for the services we provide to Her Majesty's Government.
MWR’s consultants hold Certified Simulated Attack Manager (CCSAM) and Certified Simulated Attack Specialist (CCSAS) qualifications and are authorized by CREST to perform STAR penetration testing services.