Before I did this writing lark I worked in the City – temping, IT helpdesk, software training – and before that I worked in various cigarette kiosks in and around Cardiff (South Walian smoker in the 90s? Bet I’ll have sold you 20 B&H at one time or another), and before that I was a maths undergraduate. An average one, as it happens (Desmond), but a passion for the discipline remained. From simple foundations you can build up, using a series of logical, irrefutable steps, some staggering truths which are immune to the criticism of those who question scientific method. Maths rules. Numbers are never wrong: though our interpretation of them frequently is.
It’s rare that my present and former worlds clash, but here’s a video on the maths of coeliac testing. It’s simple, surprising and highly effective. Watch it – you must, it’s only five minutes, and not hard – and then come back.
If you’re a coeliac, you’d now be forgiven for wondering whether there’s a 2 in 3 chance your diagnosis is wrong.
The very amiable mathematician, Dan Abramson of KCL, shows that the anti-endomysial blood test for coeliac has:
* a specificity of 98% (it correctly predicts you do not have coeliac when you do not have coeliac 98 times out of 100)
* a sensitivity of 97% (it correctly predicts you do have coeliac when you do have coeliac 97 times out of 100)
He also shows what would happen if 10,000 people from the general population were to undergo the test. Around 300 would be identified as positive – but as we know, only 100 (1%) would really have coeliac, therefore 200 would be wrongly identified as coeliac.
This looks alarming. If a positive test is wrong 2 times out of 3, why should you trust the positive test you had?
You are not a number
Here’s the thing. The calculation above is based on a random sample of the population. Effectively it relates to population screening, and demonstrates what would happen if we tested a town of 10,000 people.
But you, a diagnosed coeliac, are not one of those 10,000 randomly selected people. You selected yourself for a blood test – by visiting your doctor and asking for it as you were unwell – or you were selected by your doctor – who recommended it because you were experiencing symptoms or had IBS, or because you were at increased risk due to a sibling with auto-immune, for example. Neither is ‘random’.
(Had you been ‘plucked’ at random along with 9,999 others whose results were taken into account along with yours, for every real-positive You there would have been two false-positive Others to balance out the numbers.)
Abramson fails to take into account the context in which coeliac serological test is normally given. Population screening does not occur, so his scenario does not reflect real-life. He warns you to get a second opinion at the end of his clip, and although slightly tongue-in-cheek, I thought this a shame because someone either with symptoms, or a family history of coeliac, or with IBS, who is selected for a blood test and tests positive, is far, far more likely to actually be a coeliac than 1 in 3.
But in a sense, the ‘second opinion’ point is valid in this case, because we can imagine the biopsy – the second stage of a coeliac diagnosis – as a second opinion of sorts. Bloods are not 100% accurate, and biopsy is required to confirm coeliac (as might genetic typing, in some particular cases).
Diagnosis is about a series of tests – not just one. And everything you went through to get diagnosed was very important in establishing as high a confidence in your diagnosis as possible. You are overwhelmingly likely to be correctly diagnosed if you went through it.
The corollaries
There are many important lessons from this, and I’m enthusiastic about them because they all stem from a maths lesson, thereby to my mind underlining its brilliance.
The first is it’s a useful argument against those in favour of population screening for coeliac. Those false positive folk (2% of the population) would be unnecessarily worried, and some may have to undergo a biopsy for no useful purpose other than to rule out a disease they probably never suspected in the first place.
The second is a useful reminder of why one test should never be relied upon, and why biopsy is important. No test is 100% accurate, but if you put together a number of tests and criteria, known to be excellent albeit not perfect, you can get very close to 100% overall accuracy.
The third: the statistics and mathematics of medical diagnosis is highly complex, and this is but a basic and minute glimpse of it. There is a lot of hardcore stuff out there, and interpreting it is a mammoth task, way above we mortals’ heads. That’s why we need medical statistitians and specialists to interpret it, to work out what is valid, to set aside what is not. It takes time – theories have to be tested, retested, critiqued and validated – and trials – a lot of trials, with large numbers of people – to demonstrate truths.
The fourth: be wary of alternative food sensitivity testing. These tests – essentially, those which are sold by nutritional therapists, high street health stores or private intolerance testing labs – have not undergone rigorous testing and analysis by specialists, and are typically used as sole tests. We’ve seen above why undergoing just one test is not a good idea…
The fifth: self-diagnosis is unreliable. There are many ways you can go wrong self-diagnosing, and while I can understand why people do it, it remains problematic – all that expert medical and statistical weight underpinning proper orthodox diagnosis is not with you when you self-diagnose, and the likelihood of your being wrong is high. Eat bread and feel ill afterwards? Could be a ‘false positive’. (I doubt the sensitivity of the ‘food consumption’ test is high…)
The sixth is to question. Question what someone selling you a test is telling you: not because they are being unscrupulous necessarily – they could well be sincere – but because the mathematics does not support the testing protocol they are advocating, and neither they, nor you, nor I, are equipped to judge it – only the experts are. Also, don’t be afraid to question what you see in the media about food sensitivity. How might this story have been sensationalised or covered in the media? “2 in 3 coeliacs are misdiagnosed!” And that would have been very wrong.
Question figures; find their provenance; establish their reliability. Numbers are never wrong: but our interpretation of them frequently is.