Risk information is any information which can influence a decision.
Some organisations have a tendency to only accept certain types of information as legitimate risk information. Such limitations increase the chance of something important being missed.
Why variety matters
Imagine your organisation’s risk management approach can only deal with qualitative information (such as policy papers, incident reports, or assessments) that describe risk in terms of high, medium and low. Such an approach would miss the patterns and trends that could be spotted by including quantitative information (such as network flows, or numbers of security incidents). Drawing on a variety of information sources may reveal risks that would otherwise be missed.
It is rare for organisations to explicitly exclude certain types of information, but they often have an unspoken bias towards a given type. Security is sometimes claimed to be ‘unquantifiable’, or qualitative information is discounted because it’s one person’s (subjective) opinion. Again, these biases can cause organisations to overlook valuable information when conducting cyber risk assessments.
You’re more likely to fall into this trap if your organisation adopts a single, standardised approach for every kind of cyber risk assessment. This is more likely to occur when organisations focus on completing the process of risk management, rather than on the risk reduction activities which should flow from it. When organisations get into this ‘defensive’ pattern of risk management behaviour, this closing down of what counts as ‘legitimate’ risk information can be exacerbated.
Help with assessing information sources
How can you know if you’re considering enough information sources?
This is more of an art than a science, and the technique we’re suggesting below uses a matrix that classifies information as qualitative or quantitative, and objective or subjective:
- qualitative information is about describing something in human language, such as written information presented in documents.
- quantitative information is about things that can be measured in numbers.
- objective information is verifiable and not subject to opinion (such as the number of laptops that your organisation holds, or the amount of money it would cost you to purchase a particular antivirus solution).
- subjective information is a matter of opinion (such as the judgement that a particular organisation is more at risk of DDoS attacks than of ransomware attacks).
By assigning each information type to the corresponding location on the grid, you’ll quickly be able to identify if there are any potential blind spots, as any empty quadrants will be immediately apparent. The grid below provides some examples of each type of risk information.
The purpose of this grid is not to categorise individual pieces of information. Neither are we suggesting that information from any of the four quadrants is ‘better’ than any other type. It is about looking at the spread of information sources that you use in your risk analysis, and spotting any blind spots.
So how might you go about doing this?
- Start by going go through all the various information sources that feed into your organisation’s risk assessment process.
- Place them into the grid above. If you’re not sure where to start, go back to a decision that related to cyber security in your organisation. What information was used to inform that decision? If nothing was written down, go back and speak to the person who made the decision and ask them what they used to decide on that security issue.
- Examine the grid. What does it look like? Are you weighted towards one quadrant, or perhaps one half?
- What other information could you have gathered to fill the gaps? How might that have changed the decision? Why are the gaps where they are? What blind spots this might cause in your approach to risk analysis?
The goal of this exercise is to help you spot situations where your risk assessments might be missing some valuable information. It won’t tell you exactly what you’re missing, but it can shine a light on organisational biases towards a particular type of information.
This is by no means the only way of categorising risk information. There are other properties of risk information which may be just as useful. For example, it is also worth considering whether you are using a balance of information about the past, and information about how you anticipate the future will unfold, with some interpretation.
Common organisational bias
By using the qualitative/quantitative, objective/subjective technique, the NCSC have recognised a common bias in many organisations, where the grid is heavily populated in the top-left and bottom-right quadrants, and empty in the other two. In such organisations, when assessing cyber risk, the terms ‘objective’ and ‘quantitative’ were taken to mean the same thing.
Our findings demonstrate that in these organisations, information that was inconsistent with this flawed assumption was ignored. For example, experts’ subjective assessments of probability were discounted.
If you have tried to apply this rule of thumb and have any questions, or would like to give us feedback, please get in touch.
Source: NCSC
