The 5 baselines any CISO must use.

The BOOM framework

This blog is nothing more than a shameless plug for the book “The metrics manifesto” written by Richard Seiersen. In this awesome book, Richard describes the BOOM framework (Baseline Objectives and Optimization Measurements). He describes 5 baselines that are foundational for measuring security performance and that allow you to articulate clear and measurable objectives to measure whether your security programs are actually making your environment more secure or not. Next to defining these terms, it also contains tons of details of how to implement the BOOM framework, including source code.

Survival Analysis

The first baseline is about measuring the survival time of events. By understanding how long events (like vulnerabilities or other risk causes) survive in your environment, and how that depends on the type or severity of event, you are reducing the uncertainty about your risk exposure. An example of this baseline is: “50% of critical vulnerabilities live for 48 hours or longer”. You need to avoid using averages over clusters of types of events and gather as much fine-grained data as possible.

Burn-down Metrics

The second baseline is concerned with the ratio of removed risk (in a certain timeframe) over the total new risk that is out there. It measures whether you are mitigating risk faster or slower than the growth of risk. For example: if in the last month you had 100 new vulnerabilities, and your team was able to remove 60, the burn-down rate is 60%. Although this is lower than 100% and hence risk is increasing, if the burn-down rate was 50% the month before, you still know you are improving. This is a very useful performance metric that developer teams can use to measure their own performance, and for CISOs to compare teams to decide on which team to focus to make the biggest impact on security in the coming period.

Arrival Rates

Burn-down metrics meaure how fast you are removing risks, the third baseline is measuring the rate at which risks emerge. Predicting what will happen tomorrow is difficult, but by leveraging intel-feeds and historical data (for your environment), you can build probability-curves that show the chances of a vulnerability for one of your technology stacks being reported coming month. Arrival rates are useful to know because the arrival of a new vulnerability defines work for your team. This baseline therefore helps you make decisions on resource allocation.

Wait Times

The fourth baseline, wait times, is a well known measurement in operations management. It is measuring the time between arrivals of risk causes, like vulnerabilities. Knowing this metric, helps you optimise your security operations teams. But it should also be used as a leading indicator for risk: if wait times are decreasing, risk is increasing.

Escape Rates

The last baseline is measuring how risks migrate across your environment. In particular, it measures the rate at which risk-causes move from an environment with one state of control to an environment with a lesser state of control. For example, it measures the rate at which risks are moving from your development environment to the production environment. In other words: the rate at which those risks “escape”. Modern software development teams are increasing their release velocity. This would increase the escape rate too, unless your security program is able to reduce escape rates without reducing release velocity.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Martijn Dekker, PhD

Martijn Dekker, PhD

Martijn is a top-executive, researcher and CISO of more than 25 years of experience pushing the limits of information security.