Stop Using Vulnerability Counts to Measure Software Security. - Meneely, Andy; Keller, Brandon (Communications of the ACM, 2025)
This article argues that counting fixed vulnerabilities is a misleading measure of software security, as it ignores the context, effort, and process improvements behind each fix. Vulnerabilities are discovered under varying conditions, influenced by factors such as tool advances, human diligence, and changes in APIs, making raw counts insufficient to gauge security. Instead, the authors propose vulnerability recidivism metrics, which track repeated types, modules, or authors associated with vulnerabilities to better assess process effectiveness and ongoing risk. Emphasizing a culture where developers can safely admit mistakes, reflect on them, and improve practices is key to meaningful cybersecurity measurement and improvement.