One way to predict is by using data, building “quant” models and scrutinizing the statistics of many variables. Alternatively, we can rely on qualitative judgment and human experience. Ideally, we can combine the benefits of both styles, but doing so requires that we reorient our approach to data.

Rather than focusing on variables, we propose focusing on experiences, and carefully assessing their relevance in order to extrapolate the future from the past. Relevance allows us to see exactly how much each prior outcome contributes to today’s prediction, and how that contribution changes over time. It also allows us to evaluate the confidence we should have in each specific prediction, which changes with the prediction circumstances. The future of prediction, we argue, must be ever more transparent and flexible in its use of data.

In this paper, we introduce a prediction system based on assessing the relevance of prior outcomes for future predictions, and describe the advantage it brings to both simple and complex quant models.

Key Highlights
We all need to predict what will happen in the future, because our decisions today depend on it. This is true in everyday life, and perhaps even more so for people who work in economics, policy, investing, medicine, sports analytics, and many other fields. One way to predict is with data, building “quant” models and applying sophisticated statistics. Another way is to rely on human judgment, expertise, and the wisdom that comes from prior experience. Both techniques are more valuable than ever; the problem is that they are locked in conflict with each other. Quants speak of variables and models, but the natural approach is to talk about lived experiences. For example: What happened during past recessions? How did other people react to a medication?

It turns out that this natural approach—learning from experience—can be used to form intuitive predictions from data. In short, prediction is about determining the relevance of prior experiences, something our brains do automatically all day long. If we can determine relevance, then our prediction is literally a relevance-weighted average of what actually happened in the past. Thankfully, it turns out there is a unique and rigorous way to compute relevance for any data set, mathematically. In fact, relevance-based predictions have an elegant equivalence to traditional statistics which offers insights in its own right. It means that we can translate the language of quants into more intuitive terms. But beyond getting better transparency into which observations contribute to each prediction, there is an even more important benefit: we gain the flexibility to adapt our prediction process in real-time to each specific prediction task.

This is only possible with transparency. Casting prediction in terms of the relevance of past experience lets us scrutinize how much we should truly rely on each episode. Every time we go to make a prediction, we reassess relevance, homing in on prior periods that are both similar to today, and different from average (thus informative, and memorable). This formula directs our attention incredibly well, and it can aligns with mathematical precision. We can then ask, is it better to discard irrelevant observations entirely, focusing on a narrow set of just a few things from the past, or should we cast a wider net? And, what attributes of prior experience are most important when we assess relevance? In other words, what variables should we focus on today that perhaps we did not need yesterday? This flexibility leads to what we call “situational learning.” It mirrors the natural thought process, and in doing so it offers more effective predictions, because it can account for real-world complexities, regime shifts, and more. With this perspective as a building block, we can improve prediction by making it more transparent, more flexible, and less arbitrary.