It’s easy to proselytise the benefits of structured analytical techniques, yet much harder to put them into practice. A more honest and realistic conversation about their role in cyber threat intelligence (CTI) analysis is required. A shift from idealism to pragmatism is long overdue.

Structured analytic techniques are a toolbox of methods within intelligence analysis. They help analysts interrogate evidence, consider multiple viewpoints, and mitigate cognitive bias. Richard Heuer and Randolph Pherson outline many of these in the book, Structured Analytic Techniques for Intelligence Analysis.

The CTI community has identified numerous advantages of using analytic techniques, yet rarely addressed first-order questions on how they should be managed and prioritised. Huer and Pherson’s book describes 50 techniques divided into eight different categories. Large government agencies might be able to implement such a repertoire of methods, yet this will be out of reach for many CTI functions.

We need a more honest and pragmatic discussion on the resources required to implement analytic techniques and how they should be balanced against competing demands. This article advocates for a flexible approach based on a CTI team’s unique context. This means making more intentional and ultimately qualified decisions about the role of analytical techniques in a CTI function.

Implementing Analytical Techniques

CTI leaders regularly have to make tough decisions when allocating finite resources. This makes it essential to understand what resources are required before implementing structured analytical techniques:

• Analysts require training: A variety of analytic techniques exist and analysts need time to build the skills and experience to use a wide range of methods.
• Achieving consistency involves process: Analytical techniques should provide reassurance, but this only works if they are implemented consistently. Developing a uniform approach will often require additional reviews or benchmarking exercises across a CTI team.
• Putting analytic techniques into practice is time intensive: Increasing the rigour of intelligence analysis naturally increases the time it takes to produce reports. Some analytic techniques also require active participation across multiple team members. This can introduce scheduling challenges or even bottlenecks when colleagues are busy or unavailable.

Deep integration of structured analytical techniques comes with opportunity costs.  Resources invested in analytical techniques cannot be spent elsewhere. Training initiatives focused on analytical techniques mean forgoing training on other CTI or security topics. Analytic techniques can increase the lead time of developing intelligence products or lead to fewer reports.

These are not necessarily bad tradeoffs or mean we should abandon analytical techniques. Everything in life (or cyber security) has opportunity costs. In fact, neglecting analytical techniques comes with its own negative implications. For instance, increasing the risk of inaccurate, biased, or misguided reporting. This is why robust analytical tradecraft is a high priority for most CTI leaders. Ultimately, understanding the implications and tradeoffs associated with analytical techniques allows us to make more informed decisions.

Managing Analytical Techniques

CTI leaders should make highly intentional and qualified decisions when implementing analytical techniques, weighed up against competing demands. Rather than diminishing their contribution, this approach should make a positive case for analytical techniques by focusing on their benefits and contribution to a security function’s objectives.

Managing analytical techniques should never be seen as a binary decision (i.e. where a CTI team has to either integrate 50+ different analytical techniques or neglect them entirely). CTI leaders should instead view the implementation of analytic techniques as a sliding scale that can be ramped up or down accordingly.

This is similar to how a risk management function might approach cyber security. A risk assessment would rarely conclude that an organisation should store hard drives offline in underground nuclear bunkers to achieve maximum security. Establishing security policies based on available budgets, risk severity, and broader organisational objectives is a far more practical approach.

Similarly, CTI teams should make their own decisions and judgments on how much to prioritise analytical techniques. Advocating for the extensive use of structured analytical techniques in environments where it would be unrealistic is akin to an overzealous and draconian cyber security policy.

Multiple factors are outlined below, which should help CTI leaders decide how to prioritise analytic techniques.

Resources

CTI functions come in a variety of shapes and sizes. Larger teams with healthy budgets will naturally have more scope to implement analytical techniques. Smaller teams, by contrast, will have to be more selective.

Richard Bejtlich’s concept of the ‘security one per cent’ really resonates with me here. Bejtlich stresses that a great deal of current security guidance is suited to a small minority of security functions that possess the resources to build a robust cyber security posture. But, many of the security strategies for the 1% might be irrelevant at best and damaging at worst to the remaining 99%.

We can apply this same logic to CTI teams and their use of analytical techniques. Merely having a CTI function is probably already evidence of being in the ‘security one per cent’, yet the size and scope of CTI teams still vary enormously. This inevitably means there is significant divergence in how teams can realistically implement analytical techniques into their workflows.

Existing advice on structured analytical techniques often caters to ‘ideal state’ CTI functions (i.e. those that are highly mature and well-resourced). For analytical techniques to thrive, advice needs to resonate with CTI functions operating in resource-constrained and already-stretched environments.

Importance of analytical judgements

The rigour of analysis should also reflect the seriousness of an intelligence request and the decisions they inform.

For example, the quality and accuracy of intelligence analysis become extremely important for a government preparing to issue sanctions against a foreign state. A government’s credibility would be severely damaged by getting public attribution wrong. This means a relevant government intelligence agency will likely invest additional time and resources to ensure their analysis is as accurate as possible. This might even involve using different structured analytical techniques simultaneously to stress test a report’s findings.

Conversely, intelligence analysis wouldn’t require anywhere the same level of analytical rigour when tracking an unexploited vulnerability for situational awareness. The additional time and effort invested in implementing in-depth analytical techniques would have a relatively low payoff.

Collection quality

Analytic techniques should also be weighed up against collection data quality. CTI functions relying exclusively on unreliable or secondary sources are more likely to misinterpret reported findings. When there is an increased risk of ambiguity, structured analytical techniques become more important than ever in preventing analytical pitfalls.  

Conversely, CTI teams with extensive access to high-quality and primary source data will be in a stronger position. This is because regular, first-hand observations of threat actors help analysts to learn intimate details of an adversary’s modus operandi. Intelligence based on a direct understanding of adversaries removes ambiguity by reducing the risk of misinterpretation.

Of course, this is not a zero-sum game. High-quality primary source collection is never a substitute for robust analytical tradecraft. Ideally, we would always hope for both. But, analytical techniques take on particular importance when dealing with unreliable sources.

Conclusion

For structured analytical techniques to thrive in the CTI analysis, advice needs to speak to a ground truth that resonates with the vast majority of CTI teams. This means we need a more honest and pragmatic discussion on the resources required to implement these techniques and the criteria we should use to prioritise their implementation against competing demands.

If we ignore this reality, the conversation around analytical techniques will increasingly rest on hollow foundations. Without a more pragmatic tone, the benefits of analytical techniques will be well discussed in conference talks and public blogs, yet largely absent from many CTI teams’ workflows.

Here, we should embrace the opportunities that exist for smaller CTI functions to focus on analytical techniques with high leverage. As JD Work highlighted to me in a productive exchange on this topic, a lot of intel 101 can be taught in under a day by discussing alternative competing hypotheses. This highlights that a relatively small amount of effort on analytical tradecraft can have dramatic and disproportional benefits, provided we are smart in focusing on the right areas.

Rather than chase perfection, it is time we explore what’s realistic.