Risk Analysis – Valuable Decision Tool or Paper Tiger? By Ed Bassett
Risk Analysis – Valuable Decision Tool or Paper Tiger?
By Ed Bassett
Risk analysis is the cornerstone of information security decision making, yet it is often difficult to produce one that is effective in influencing strategic security decisions being made in the executive suite. Done well, a risk analysis will illuminate safeguard choices so that businesses can make smart choices that optimize effectiveness and cost. Factors such as simplicity, relevance, focus, realism, and big picture perspective can mean the difference between a truly powerful risk analysis and one that falls apart like a paper tiger when presented to decision makers.
Risk Analysis – the identification and evaluation of risks – is a foundational cornerstone of the information security business. Nearly everyone agrees that security decisions should be risk-based. That is, you should understand your risks and use this knowledge to select appropriate safeguards to reduce your risk to an acceptable level. Nearly everyone agrees on the basic risk formula – that risk is a function of threats (how likely is some “bad thing”), vulnerability (how likely are safeguards to stop the “bad thing”), and impact (how much damage with the “bad thing” cause). This formula has many incarnations. Classic qualitative risk analysis methods us it to compute an “annual loss expectancy” for each risk – the amount the risk is expected to cost each year (generally probability of the event times impact of the event). Quantitative methods use broader intervals such as “high”, “medium”, and “low” to evaluate probability and impact. Each has its merits and all seek to rank the relative severity of risks so that appropriate safeguards can be chosen.
The foundation seems sound, yet there are significant challenges in performing security risk analysis that is truly useful in making business decisions as opposed to the “paper tiger” analyses that are “unable to withstand the wind and the rain” (to quote Mao) often encountered when the report reaches the executive suite. Practitioners struggle with quantitative versus qualitative and subjective versus objective. They struggle to get credible information about the probability of certain events. They struggle to present their analysis in a way that is understandable to the business. And, they struggle to tie the safeguard decisions to the actual risk analysis. Often, the “risk analysis” doesn’t seem to really influence the decision making process.
Assuming we all want our hard work to be a valuable and powerful decision tool, let’s step back and look at why we’re doing this in the first place. The practical utility of risk analysis is in two areas:
- First, risk analysis is useful in deciding how to optimize limited (security) resources. In selecting safeguards, there are many trade-offs between effectiveness and cost. The “promise” of risk analysis is to provide structure to those decisions so that they are accurate and defensible.
- Second, risk analysis is useful in determining whether a particular safeguard is necessary and reasonable for your environment. This second utility is especially useful in regulatory compliance scenarios because risk analysis can often be used to justify alternatives to the prescribed controls.
In a purely academic sense, you would start with no safeguards and add exactly those that make sense from a risk perspective, essentially using the risk analysis as the safeguard “selector.” When the cost of a risk is greater than a proposed safeguard, the safeguard is “cost effective” and therefore selected. When the safeguard costs more than the risk it prevents, it is not selected. While this theoretically produces optimal results, it is seldom the real-world scenario.
In practice, businesses perform security risk analysis, the formal organized kind, with a wide range of motives:
- To meet a compliance requirement. In this case, the analysis is simply an exercise to satisfy the regulation and does not actually inform or influence the direction chosen by the business. The cynic in me says this is by far the most common scenario – there seem to be a lot of “shelf-ware” risk analysis reports.
- To tease out the relative merits of alternatives in cases where the best path is not obvious or that are "too close to call" after doing a less formal intuitive analysis.
- To justify a non-compliant solution. In this case, the safeguard decision is often driven by cost or other high-level factors and the business wishes to clearly document it as a defensible position. This is usually "ammunition for auditors," but could also be considered good diligence by the client in to avoid making a bad decision.
- To justify increased spending on security by illustrating that the level of risk is higher than generally perceived. Given companies are generally already spending what they (meaning the managing executives) think is "reasonable and necessary" for security, this is the "enlightenment" reason: to replace ignorance with understanding of the true risks.
- To gain efficiency (lower costs) or effectiveness (better security) by shifting controls/spending to the area of greatest risk benefit. This is nominally the "raison d'etre" for risk analysis.
In order to maximize real-world utility of a security risk analysis as a decision making tool, it is important to understand which of these reasons is compelling the business to undertake the analysis. It is also important to tailor the risk analysis process based on those reason(s). Here are some factors to consider in deciding what sort of analysis to perform:
- Simplicity. Business executives are in the risk management business almost by definition. They use a variety of techniques ranging from intuitive “gut feel” to formal quantitative analysis. They are used to making decisions based on imperfect information. Good news – because security risks are notoriously hard to analyze with precision. By honing in on which of the above reasons above are in play and presenting a simple, focused analysis that answers the key business need, you can greatly increase the chances that the risk analysis will be considered “valuable.”
- Relevance. All security decisions are based on risk analysis – but sometimes the analysis and decision making process is quick and informal. For example, in deciding whether lock my house on a regular basis, I may conduct only a very cursory informal analysis. I may assess that the cost is so low (locks already exist, effort to lock/unlock daily is trivial) that I choose to implement the safeguard without bothering to formally assess the actual level of risk. Does not locking increase the chances of a burglary? The statistics are not readily available; it’s easier just to lock the door. Similarly, I may choose not to hire guards on the property without ever knowing how much one might reduce my exposure since the cost is above my means regardless if they pay for themselves in increased safety for my home and family. These are reasonable risk decisions. A great many information security decisions can be made the same way. Over-analyzing the obvious can distract from the really interesting risk decisions where a structured risk analysis is necessary and beneficial. An easy way to keep the risk analysis relevant is to adopt as “baseline” controls those safeguards where the choice is obvious, saving the more rigorous analysis for the safeguard choices that are material to the business.
- Focus. Clearly compliance is the number one driver behind much security spending. Businesses overwhelmingly view compliance requirements as burdensome, even if security practitioners consider them to be “good practices everybody should be doing”. Businesses in the midst of extraordinary spending on security to meet a new compliance requirement are generally not looking at it as an opportunity to run around fixing other things where there is no compelling business imperative. Security practitioners are understandably excited by attention and spending on security. While it is possible (perhaps even likely) that a good risk analysis would find a compelling business imperative that the business was previously overlooking, it’s still a tough sell and may seem like a “witch hunt.”
- Realism. Overwhelmingly the response of executives to risks described as "serious" and "critical" risks is that they see them as less serious than the risk analyst does. Absent a direct tie to business risk, they may not be compelled to fix them. Occasionally, very occasionally, their eyes shoot wide open and they immediately initiate spending on corrective actions. In the vast majority of cases, traditional risk analysis is incomplete and ultimately disregarded by the business because it fails to assess "cost" and "benefit" using the same yardstick as the business. Usually, risk assessors assess cost in terms of incremental spending and benefit in terms of decreased security risk. Business executives measure cost in terms of diversion of scarce resources from other more beneficial investments (e.g., new offerings, more sales people) and they measure benefit in terms of reduced cost or increased revenue (both compelling business imperatives). It helps to illustrate risk in business terms, for example the connection between compliance and reduced cost (no findings or fines) and increased revenue (able to win/keep customers). When the security spending is just about "cost of doing business" (as is often the case with security costs) and the benefits are less tangible, the business decisions may not be swayed much by risk analysis.
- Big picture perspective. The essence of security risk analysis is making well-informed decisions about safeguards and spending. Often, practitioners use risk analysis to justify more safeguards because the analysis is focused only on security and is biased towards more security. A big picture risk analysis might suggest a shift or even reduction of spending. Often, the level of risk that is “acceptable” to a business is higher than what people paid to live with those risks day-to-day would like. If the business has bigger business risks in other areas that should be getting some of the security dollars, a good risk analysis can help point that out. It is important to at least be open to the possibility that the risk analysis may suggest moving costs away from security safeguards (e.g., doing less than is being done today; doing something less than fully compliant). In practice, this is usually not what the security practitioner has in mind – but eliminating the natural “pro-security” bias in a security risk analysis greatly improves the credibility of the analysis to business executives.
Perhaps the most important element in analyzing risk is to ensure that the analysis process itself is paying off – showing value to the business. In order to be valuable, the analysis must actually improve the choice of safeguards. Anything less is just a documentation exercise.
As noted above, some safeguards are obvious (either obviously going to be chosen or obviously not appropriate). For example, when regulatory compliance is the business driver, if a business can readily come up with a fully compliant solution at a reasonable cost, what is the benefit of spending more time and money on formal risk analysis? When dealing with the typical security regulation, a fairly complete set of safeguards is prescribed. For those items where a reasonably priced safeguard is available that meets the regulatory baseline, further analysis is not likely to be valuable.
This is really about looking at the cost-benefit of the risk analysis effort itself. Since detailed risk analysis isn't cheap, can the business realistically expect to save enough via reduced risk or more efficient safeguard choices to recover the cost of the analysis? Perhaps the analysis might point to a previously unconsidered solution that is cheaper and better than the obvious (default) solution. Or, it might illuminate some previously unknown risk that shows the default to be inadequate. It would have to be a risk so serious that it is worth both the cost of the additional analysis AND the increased cost of safeguards to address the risk.
In a compliance exercise, there are usually a handful of safeguard decisions where the answer is not obvious – the “hard” choices. For those, risk analysis can be very valuable – well worth the time and effort. For the “easy” safeguard choices, it is rare for a truly astounding result to come from a risk analysis. For those items, my advice is don’t spend a lot of time painting tiger stripes on an oversized three ring binder.