I’ve been reading Tim Harford, (the ‘Undercover Economist’s) new book:‘Adapt: why Success always starts with Failure’, and I’ve relished the insights that behavioural economics offers about the real reasons why people make the choices they do. Seek out and try new ideas, expecting some will fail; start small; solicit feed-back and learn from your mistakes are the valuable mantras he offers for anyone with a new business and for any organisation aiming to innovate successfully in this disruptive world of to-day. But failure is not just difficult to manage; in some contexts it is unacceptable.
Harford’s premise is that the world is too complex to make it possible to see into the future. Identifying causes and effects is far from simple in complex situations, as the repeated failures of government support for emerging and failing industries testify. So how should one go about this innovation business?
Harford quotes a Russian-born engineer – Peter Pachinsky, whose principles he summarises thus:
1. Seek and try new ideas and new things, expecting some will fail.
2 When trying out something new, do it on a scale that is survivable; create safe spaces for failure and move forward in small steps.
3. Seek out feed-back, make sure you know when you have failed, and learn from your mistakes.
‘Fail early; fail often’ is the gist of this message. And its corrollary is: in unpredictable conditions, back a number of horses (or perhaps back the Bookmaker!). But the strategy more commonly followed in business in the face of uncertainty is to back only those projects with short odds, and to abandon anything else, thus closing down the range of possible innovations.
And secondly, it pays to back good reputations more than backing your own judgement. The Howard Hughes Medical Institute backs people rather than projects (the Medical Research Council used to close down his laboratory when its head retired or died.) Not only did HHMI-funded researchers produce the most important, unusual and influential research, more highly-cited articles, and win more awards; but they also produced more failures!
Harford argues that actual observation of what happens on the ground – trial-and-error – and ‘knowledge of the particular circumstance of time and place’ is the best way to advance. Create as many separate experiments as possible, even if they appear to embody contradictory views on what might work; and encourage some long-shot experiments, even though failure is likely, because the rewards are so great. Solutions to one problem often unlock another, and different strategies may combine to create entirely new possibilities.
There is no better way of proving a case than through the use of randomised trials, Harford avers, and few more effective methods than stratified trials – in which an approach is used in a number of similar but slightly different forms (with different methods of evaluation ) – because solutions effective in one situation often need adapting for others.
But it is never easy to get untainted feed-back from trials: observing outcomes requires care, open-ness and perceptiveness, and often persistence; and too many people in reporting chains depend upon the reporting of success. More like action-research than randomised trials, it often involves a series of small steps (from which learning from the latest experiment may get passed round on the grapevine.) Moreover, cause and effect are not necessarily easy to identify.
With a story about how the choreographer Twyla Tharp rescued a show that was about to bomb, Harford identifies important elements in the process of failure: being willing to fail; finding space in which to fail; acknowledging failure; and reacting to failure. Not acknowledging failure can be the result of simple denial – because it damages one’s self-belief; one can convince oneself that the mistake doesn’t matter; or simply re-interpret failure as success; and failure can influence one’s judgment (see it in ‘Deal, no deal’!). We need a validation squad, he suggests: friends who will tell you how it is, take out the venom, identify what needs fixing – ‘people with good judgement in other parts of their lives who care about you and will give you their honest opinion with no strings attached’ (Twyla Tharp).
There are indeed some roles in which to make a mistake is unforgivable, among them those of pilot, doctor and judge; and many where ‘success’ is what is expected, among them, minister of state, police officer and CEO. The ability to adapt takes real courage, concludes Harford, at other times happy self-delusion, but the process of experimenting and correcting mistakes can be more liberating than the mistakes themselves are crushing, even though at the time we so often feel that the reverse is true. And it should be added that it requires a Champion and Shielder to argue your case and to protect you.