Why addiction to certainty is a blight on business & what to do about it

We hate uncertainty. In a series of studies in the 1980s, Daniel Kahneman and Amos Tversky demonstrated that most people would rather accept the certainty of a small payment than the possibility of a larger one. By the same token, if we are troubled by the possibility of a negative outcome, however unlikely it may be, we will pay a premium to eradicate that feeling of anxiety – which is why the insurance industry exists.  

The more visceral the negative outcome, the stronger our anxiety and the more desperate we are to remove it.  Indeed, the psychologists Pietro Badia, Bonnie McBane and Steve Suter have shown that most people would rather receive an electric shock now than face the prospect that they might get one later, without warning.

John Sills has just written an excellent piece about uncertainty in Management Today, focusing on the implications for customer experience design. As he points out, even the relatively innocuous situation in which your waitress doesn’t write down your order is enough to cause mild anxiety – and has the potential to spoil the experience, even if she gets the order exactly right. His advice is simple: identify the moments when consumers might begin to feel uncertain and make sure you intervene to reassure them. In other words, avoid uncertainty at all costs.

But, of course, it isn’t just when we’re consuming that uncertainty troubles us. It’s also when we’re at work. Employees hate it when their bosses appear uncertain about the future; investors hate it even more; and bosses put their employees under equal pressure to predict accurately how negotiations and pitches  will turn out, or how relationships will develop, or how others in their team will perform.  Anyone who is involved in revenue forecasting or project scoping will be under huge pressure to get the numbers right. And if you have the temerity to consider yourself an expert in your field, certainty is effectively what you sell.

The trouble is, such certainty can rarely be justified. How many times have you finished a job within budget and with time to spare? Compare this with the number of occasions when you have you had to bust a gut to meet the deadline, or had to push it back. Or when the budget you so painstakingly worked out proved woefully insufficient. So prevalent is this human tendency to make over-optimistic forecasts that Kahneman and Tversky gave it a name: the Planning Fallacy.

Why do we fall prey to it? Well, lots of reasons have been identified – from our failure to grasp the laws of statistics to evolutionary assertiveness (and, let’s face it, the most confident proposal still tends to win the most favour, even today). But it all still boils down to one thing: we hate uncertainty so much that we simply refuse to contemplate it.

This means that we don’t try hard enough to identify the things that might throw us off track. Something within us tells us it’ll all be fine, that we’ll find a way around any potential obstacles, that we are smart enough to succeed, whatever the odds. So we don’t even bother to work the odds out.

And, if we did, others – whether our colleagues, our clients, or both – would tell us to stop. Pessimism, as it would be interpreted, is almost as unacceptable in business as uncertainty.

Even when things don’t work out, we don’t learn our lesson. We look, instead, to blame the people doing the job for incompetence or inefficiency. We fire suppliers, file critical personnel reports and identify process issues which we believe to have caused the problem.

We rarely consider the idea that the plan might have been unrealistic in the first place.

This fear of uncertainty takes us to some pretty bad places. Major overspends on big construction projects like Sydney Harbour Bridge or Wembley Stadium; corporate scandals like Enron and Lehman Brothers; and a million other small business failures that you never read about can all be traced to our unwillingness to look reality in the face - because we're addicted to certainty.  However fallacious, we are pre-programmed to believe in confidently expressed project plans, plausibly explained growth patterns, and optimistic business forecasts that back us to succeed.

How can we avoid these pitfalls? Who would hire an expert who admitted to being unsure about the very subject she was claiming to be an expert in? Who would commit to a project that did not have a coherent budget and timing plan? Who would invest in anything without first seeing their projected returns?

It’s a tricky one. But the best experts in uncertain fields, like business planning, project planning and economic & political forecasting, will be certain about one thing: that they know how to navigate uncertainty, but not how to eradicate it. They will know that this latter goal is illusory. And they will confidently express this view to their clients, their employees, and their investors.

They will pursue a project by taking small steps, measuring the impact of each one, often comparing it with a different step in the same direction to see which one takes them further. They will use this feedback to figure out what the next step should be. At the same time, they will be alive to the changes around them and over which they have no control – such as the activity of competitors and legislators, the fortunes of the economy and the advent of new technologies.

They will make short-term forecasts, but be wary of long-term ones. They will express each forecast as a range and they will attach a probability to that range. From this, they might be able to extrapolate a single “average” figure to set as a target, but this will be better understood for the estimate it is and not be regarded as immutable.

In the same way, they will tackle projects in short sprints. They will apply the lessons learned from the last sprint to the next one, adjusting time and budget estimates accordingly.

They will know where they want to go. They will have a metaphorical compass. But, as Drew Hanson has said, they won't have a map.

They will be rewarded for moving things in the right direction, of course, but incentivised on the accuracy of their projections and forecasts, rather than on the scope of them. In many situations, not least when dealing with investors, it is better to project 5% growth and achieve it than 10% and fall short.

Such realism is at work only in pockets today – aided by the transparency and speed of feedback that can now be gathered digitally. Where it works, it is because data gives rigour to what might otherwise appear to be a wishy-washy experimentalism. It makes “test and learn” a smart strategy, not a weak cop-out.

There is no reason why such an approach cannot be adopted away from the digital domain – even if feedback is less clear, or less rapid. All that is required is for progress to be tracked, and the insight used to make adjustments to the plan. In slower moving domains, such as automotive or optical retail, it might not make sense to adjust too often. But that doesn’t mean it isn’t the right approach.

Finally, it’s worth pointing out that this isn’t a licence to avoid making big decisions. Perhaps one of the reasons we’re addicted to certainty is the need for leaders to demonstrate strength (and the desire of followers to perceive it). Butexperimental realism, to give this approach a name, makes decision making much less problematic. It allows many decisions to be made more quickly, without lengthy deliberation or apparent hesitancy.  This, in turn, frees up decision makers to spend more time thinking about the really big decisions that cannot be reversed, as well as creating a culture in which such decisions are naturally subject to empirical data analysis and realistic market assessments.

Like the customers in John Sills’ restaurant, the staff, investors and other interested parties would still need to have their uncertainty managed. But the regular provision of data would do precisely that. Horizons would be shortened and objectives reframed. The clarity of this framework would provide further reassurance, so that any anxiety induced by long-term uncertainty would be relieved by the knowledge that forensic attention to detail was being applied in the present and near future.

Much of this runs counter to the dominant culture of business – which pretty much states that you should know everything about what you're doing and, if you don’t, you should bluff convincingly. This really is the evolutionary law of the jungle. Surely it’s time we moved on.