Buggy or vulnerable software can present significant risk to an organization and may even result in expensive cyber security events. Part of KPMG Cyber’s mission is to reduce the likelihood and impact of such incidents so our clients can focus on their own goals like growth and innovation. With that in mind, the Patch and Vulnerability Management team at KPMG has taken our collective observations from working with clients across industries and consolidated them into an assessment framework. By assessing six high-level capabilities and 21 lower-level processes that encompass people, process, and technology, this framework provides a lightweight method for quickly benchmarking client programs against their industry as well as our greater client portfolio to provide immediate insights. While kept anonymous, these data points enable KPMG to serve our clients throughout their development journey and tailor our services to meet those areas where clients need more support.
We wanted to share some background on this assessment approach to provide some idea on how we think about vulnerability management, and because it gives the chance to hear from others who are also tackling this common challenge in IT organizations. These are the six high-level vulnerability management capabilities we assess:
- Discover: finding vulnerabilities in the environment. This capability focuses on the most commonly understood element of vulnerability management: Discovery digs into the technical implementation of the vulnerability scanners, striving to understand how scan jobs are orchestrated around maintenance windows or other freezes and how they are able to ensure full coverage of dynamic client environments, all while accounting for both breadth and depth of scanning. A clients’ ability to define their ownership of the vulnerability scanning function is crucial to delineating roles & responsibilities for other downstream capabilities like reporting. It lastly seeks to understand how they may prioritize deployment of scanning agents in a way that balances performance benefits with the cost to the business. The discovery capability is observed to be the highest scoring area for almost all clients we have assessed. However it also exhibits the most variance of scores, possibly owing to the complexity of differing IT environments, the wide selection of vulnerability scanning tools available and their respective various configuration options.
- Evaluate: making sense of the detected vulnerabilities. Inevitably all clients produce more vulnerability findings than they have the bandwidth to remediate resulting in the need to prioritize. While scanning vendors have come a long way in their ability to leverage CVSS ratings in conjunction with proprietary threat intelligence data, gaps often remain to include data from internal threat intelligence teams and asset/network context to augment these out-of-the-box priorities appropriately. This difficulty is due in part to incomplete CMDB data that commonly plagues programs’ ability to join in crucial asset and owner data. We see that the capability to evaluate is one that many clients make a concerted effort to improve but often becomes overly complex and under supported. Clients also show varying levels of partnership with remediation owners when it comes to managing the risk management decision making process of the remediation lifecycle. Often times the decision to remediate, mitigate, or accept risk is one that requires input from many groups and proves a common struggle amongst most clients.
- Report: communicating vulnerability management data. There is certainly no shortage of metrics that can and have been produced to showcase the contributions of vulnerability management programs but one thing is for sure: database integrity and flexibility matters. The engineering required to de-duplicate findings, join in additional data, snapshot reports, and timestamp reports for time series trending is something that can sometimes be a heavy lift for clients despite the immense value that they bring to the organization. (Making things worse, the data science skill set required to wrangle vulnerability big data is not common in an already-stretched cyber security labor market.) From the executive perspective we typically see three themes in understanding vulnerability posture:
- “Are we seeing the vulnerabilities we should be?”
- “Of the vulnerabilities we do see, how bad is our risk?”
- “How good are we at fixing the vulnerabilities we see, and are we getting better at it?”
While there are many ways to portray these three in metrics, simply framing the vulnerability management in these terms makes it easier for executives to give appropriate direction to their teams for action, and to make decisions about investment.
- Fix: mitigating the risk presented by the detected vulnerabilities. Being able to remediate is where the rubber really hits the road, but you might be surprised to hear that this is an area that clients across the board struggle with. Coordinating with and motivating hundreds of involved business partners to make a deliberate effort to remediate is no small feat. On one end of the remediation spectrum we see patching predictably accounts for 50-75% of all vulnerability findings despite having many tools that enable mass deployment and configuration change which is often hamstrung by staffing limitations, testing, and other deployment windows. On the other end of the spectrum where remediation activities are less predictable, we see unscheduled, ad-hoc efforts to mitigating zero day vulnerabilities using remediation SWAT teams that run as projects. In either case, the presence of a ticketing tool that can ingest scanning data, map it to an owner, and track time-to-remediation is often tightly correlated with risk reduction success and is something in which we see heavy active investment.
- Integrate: collaborating across the organization to reduce vulnerability risk. Security is a team sport. While clients have different organizational structures that may bundle vulnerability management in with other functions like a SOC or pentesting, we do consider collaboration across these teams to be paramount to operational excellence however they are place. If nothing else comes from these partnerships, we hope to see clients educating each other on the capabilities and limitations of their tools and sharing data in good faith. At a more advanced level, we see clients beginning to combine data from asset databases, threat intelligence, security operations centers, application security and penetration testing to supplement each other’s abilities. For example, vulnerability data is valuable for the SOC to understand which alerts may be of more or less concern depending on different asset context and conversely to understand what attacks are being seen to augment vulnerability prioritization. Based on our data, we observe that this entire capability is one that is consistently low-scoring and provides the most room for improvement.
- Govern: setting organizational expectations for managing vulnerabilities. Lastly, we assess the program’s ability to manage their business and compliance expectations through governance. Whether it is PCI scanning, setting and adhering to remediation SLAs, or just educating and enforcing system owners to make remediation a priority this is where executive leadership can empower their organization through policy, funding, and resourcing. In more advanced programs we also see clients starting to look at managing provisioning of scanner access to other business groups, controlling access to repositories where vulnerability data is stored and even exercising control over which fields are provided to owners to subvert competing metric generation.
Assessing these six capabilities at a high level is a low level of effort and allows us to quickly share a confident, directional notion of where the biggest opportunities exist for improvement with the least investment.
Overall, what we see during these exercises is that clients actively invest in vulnerability management to improve these areas through technology (tool integrations, vendor selection, and scanning automation) but still have significant room for improvement in people and process domains. At the most foundational level, we recommend clients discover, evaluate, and remediate to the best of their ability before moving onto the more supporting capabilities however we often find that leadership is concerned with certain areas of their programs. In any case, our hope is to occasionally check in with our clients on these six capabilities to share what we see working in other organizations. Armed with the details of “what good looks like” clients can make better decisions about how to sustainably reduce more risk, soonest, which increases the chance that they meet or exceed their objectives and fulfill their own missions.
