In 2021 I wrote an article titled “Without data you are just another person with an opinion”. It was premised on the idea that you don’t need an advanced degree in mathematics to get started on the journey of data analytics, to be able to interpret, present and make data driven decisions in an organisation. I went on to argue that instead, we have a responsibility to ensure that we interrogate and utilise all available data to make better decisions, no matter how major or minor they might seem.
I came across a quote a few weeks ago that read, “Data will talk to you if you are willing to listen” and I immediately recalled the story of Abraham Wald and the ‘survival bias’. Abraham Wald was a mathematician who was a member of the Statistical Research Group (SRG) at Columbia University during World War II, where he applied his statistical skills to various wartime problems.
One of the problems that Wald and the SRG worked on, was to examine the distribution of damage to aircraft returning from missions and how best they could minimise losses.
Figure 1 shows the typical distribution of damage caused to the aircraft. The red dots represent bullet holes or damage caused in battle.
The obvious initial analysis suggested that the aircraft be reinforced, particularly in the areas where they had experienced the greatest density of damage, notably the tips of the wings and the central area around the cockpit.
Abraham pointed out that perhaps there was another way to look at the data. Perhaps the reason certain areas of the planes weren’t covered in bullet holes was that planes that were shot in those areas did not return at all – Data indeed talks, if we are willing to listen! This “survival bias” is a logical error. We tend to only concentrate on the things that made it past some selection process and often overlooking those that did not.
This radical review incidentally led to an opposite conclusion, and instead of reinforcing the areas initially suggested, they in turn re-enforced the parts of the plane where there were no bullet holes.
The story behind the data is arguably more important than the data itself. Or more precisely, the reason behind why we are missing certain pieces of data may be more meaningful than the data we have.
At Africa People Advisory Group, we regularly challenge our colleagues and clients’ thinking about metrics and analytics. The premise has always been that we start off by collecting data that will lead to analysis and ultimately assessing the impact of the finding. We advocate that instead, one should begin with determining the impact, understanding the problem and what it is that we are trying to solve for. This is turn will guide us on what data we then need to collect and analyse.
This is key in correctly informing the analysis step, as this is often where we get stuck. I think for most of us that spend the vast majority of our days solving problems, analysing or interrogating data, we regularly find ourselves deep down a rabbit hole analysing the data we have from a 1-dimensional viewpoint. There is immense value in taking a step back and rephrasing the problem statement holistically: what is the data trying to tell me?” More importantly what is it not telling me, what data am I missing and why?
An alternative that I have found immensely valuable, is talking through the data and problem statement with a colleague or even your partner at home, particularly if their background and experience differs from your own. You will be surprised how often we miss the most mundane things that often are staring right at us!
Nicolas Constantinides, Managing Partner | Africa People Advisory Group