After more than 25 years of mitigating risks, ensuring compliance, and building robust security programs for Fortune 500 companies, I’ve learned that looking busy isn’t the same as being secure.
It’s an easy trap for busy cybersecurity leaders to fall into. We rely on metrics that tell a story of the tremendous efforts we’re expending – how many vulnerabilities we patched, how fast we responded – but often vulnerability management metrics get associated with operational metrics because traditional approaches to measuring and implementing vulnerability management does not actually reduce risk. So, we resort to various ways of reporting on how many patches were applied under the traditional 30/60/90-day patching method.
I call these vanity metrics: numbers that look impressive in reports but lack real-world impact. They offer reassurance, but not insights. Meanwhile, threats continue to grow more sophisticated, and attackers exploit the blind spots we’re not measuring. I’ve seen firsthand how this disconnect between measurement and meaning can leave organizations exposed.
In this article, I’ll explain why vanity metrics are not enough to protect today’s complex environments and why it’s time to stop measuring activity and start measuring effectiveness.
Drill Down: What Are Vanity Metrics?
Vanity metrics are numbers that look good in a report but offer little strategic value. They’re easy to track, simple to present, and are often used to demonstrate activity – but they don’t usually reflect actual risk reduction. They typically fall into three main types:
- Volume metrics – These count things: patches applied, vulnerabilities discovered, scans completed. They create a sense of productivity but don’t speak to business impact or risk relevance.
- Time-based metrics without risk context – Metrics like Mean Time to Detect (MTTD) or Mean Time to Remediate (MTTR) can sound impressive. But without prioritization based on criticality, speed is just the “how,” not the “what.”
- Coverage metrics – Percentages like “95% of assets scanned” or “90% of vulnerabilities patched” give an illusion of control. But they ignore the question of which 5% were missed – and whether they’re the ones that matter most.
Vanity metrics aren’t inherently wrong – but they’re dangerously incomplete. They track motion, not meaning. And if they’re not tied to threat relevance or business-critical assets, they can quietly undermine your entire security strategy.
Vanity Metrics: More Harm than Good
When vanity metrics dominate security reporting, they may do more harm than good. I’ve seen organizations burn through time and budget chasing numbers that looked great in executive briefings – while critical exposures were left untouched.
What goes wrong when you rely on vanity metrics?
- Misallocated effort – Teams focus on what’s easy to fix or what moves a metric, not what truly reduces risk. This creates a dangerous gap between what’s done and what needs to be done.
- False confidence – Upward-trending charts can mislead leadership into believing the organization is secure. Without context – exploitability, attack paths – that belief is fragile and can be costly.
- Broken prioritization – Massive vulnerability lists without context cause fatigue. High-risk issues can easily get lost in the noise, and remediation can get delayed where it matters most.
- Strategic stagnation – When reporting rewards activity over impact, innovation slows. The program becomes reactive – always busy, but not always safer.
I’ve seen breaches occur in environments full of glowing KPIs. The reason? Those KPIs weren’t tied to reality. A metric that doesn’t reflect actual business risk isn’t just meaningless – it’s dangerous.
Moving to Meaningful Metrics
If vanity metrics tell us what’s been done, meaningful metrics tell us what matters. They shift the focus from activity to impact – giving security teams and business leaders a shared understanding of actual risk.
A meaningful metric starts with a clear formula: risk = likelihood × impact. It doesn’t just ask “What vulnerabilities exist?” – it asks “Which of these can be exploited to reach our most critical assets, and what would the consequences be?” To make the shift to meaningful metrics, consider anchoring your reporting around five key metrics:
- Risk score (tied to business impact) – A meaningful risk score weighs exploitability, asset criticality, and potential impact. It should evolve dynamically as exposures change or as threat intelligence shifts. This score helps leadership understand security in business terms – not how many vulnerabilities exist, but how close we are to a meaningful breach.
- Critical asset exposure (tracked over time) – Not all assets are equal. You need to know which of your business-critical systems are currently exposed – and how that exposure is trending. Are you reducing risk to your most important infrastructure, or just spinning cycles on low-impact fixes? Tracking this over time shows whether your security program is actually closing the right gaps.
- Attack path mapping – Vulnerabilities don’t exist in isolation. Attackers chain together exposures – misconfigurations, overprivileged identities, unpatched CVEs – to reach high-value targets. Mapping these paths shows you how an attacker could actually move through your environment. It helps prioritize not just individual issues, but how they work together to form a threat.
- Exposure class breakdown – You need to understand what types of exposures are most prevalent – and most dangerous. Whether it’s credential misuse, missing patches, open ports, or cloud misconfigurations, this breakdown informs both tactical response and strategic planning. If 60% of your risk stems from identity-based exposures, for example, that should shape your investment decisions.
- Mean Time to Remediate (MTTR) for critical exposures – Average MTTR is a flawed metric. It gets dragged down by easy fixes and ignores the tough problems. What matters is how fast you’re closing the exposures that actually put you at risk. MTTR for critical exposures – those tied to exploitable attack paths or crown-jewel assets – is what really defines operational effectiveness.
Taken together and continuously updated, meaningful metrics give you more than a snapshot – they provide a living, contextual view of your threat exposure. They elevate security reporting from task tracking to strategic insight. And most importantly, they give both security teams and business leaders a common language for making risk-informed decisions.
The Bottom Line
Vanity metrics offer comfort. They fill dashboards, impress in boardrooms, and suggest progress. But in the real world – where threat actors don’t care how many patches you applied last month – they offer little protection.
Real security demands a shift from tracking what’s easy to measure to focusing on what actually matters. That means embracing metrics grounded in business risk. And this is where frameworks like Continuous Threat Exposure Management (CTEM) come into play. CTEM gives organizations the structure to move from static vulnerability lists to dynamic, prioritized action. And the results are compelling – Gartner projects that by 2026, organizations implementing CTEM could reduce breaches by two-thirds.
The metrics you choose shape the conversations you have – and the ones you miss. Vanity metrics keep everyone comfortable. Meaningful metrics force harder questions, but they get you closer to the truth. Because you can’t reduce risk if you’re not measuring it properly.
Note: This article is expertly written by Jason Fruge, CISO in Residence at XM Cyber.
About The Author
Original post here
