How Can We Help?
Site overview
Calibrated care
Single author exercises in medicine are always a bit worrying. None of us have enough experience to produce a balanced coverage of many fields, so the danger is that coverage will be either scrappy, superficial, or just plain wrong. It is easy for a blog to turn into a diatribe. This resource is intended as an interpreter to speed access to a field, rather than offering anything resembling definitive coverage.
What is it?
The web is teaming with information sources ranging from research articles to guidelines. The theme of this site is to look at value: how well does the test work/how good is the treatment/how bad are the adverse effects? This information should be easily sourced from high quality guidelines, but the problem is not just finding the guideline, it is determining its quality and then attempting to digest it. Most high quality guidelines are long and not always easy to read. Their emphasis on the value of an intervention is also inconsistent – the data is often available, but frequently requires moving to a supplementary document. The idea of calibrated care is for patients and practitioners to have easy access to information on how well things work, and how risky they are.
Providing this information raw is very daunting to readers, and can be challenging to interpret without the textual context. It is also disconcertingly difficult to remember. On this site is is hung on brief guidelines on diagnosis and treatment with a strong internal medicine focus, but with forays into other fields where value information might be helpful. The top-layer narrative, blog-like style was chosen to aid both accessibility and recall; deeper layers (from clickable links) veers closer to critical appraisal language but still with the aim of providing quickly readable gist rather than formal step-by-step article analysis, which while reassuring to EBM pundits, can be quite tedious or even bewildering to the general clinician.
Estimating benefits and harms
We often over-estimate benefits and underplay harms. Being optimistic is wonderful, except when we persuade patients to proceed on diagnostic and therapeutic adventures with only limited understanding of what is entailed. Informed consent makes the assumption that information is presented accurately and appropriately; the harsh truth is that it can be astoundingly difficult to explain a complex diagnostic and therapeutic pathway in sufficient detail for genuine informed decision making. Patients have a social contract of trust: their doctor has done the work needed to full understand the pros and cons of what is proposed, and so proffered advice on a diagnostic or therapeutic choice is accepted in good faith. Being able to provide choices in a manner appropriate to the needs and understanding of each patient is part of our job; to be able to calibrate that for each patient necessarily implies a full understanding of value.
Site purpose
This text has three purposes. Firstly it is aimed to be an exercise in calibration for young clinicians. Secondly, it will explore some of the issues around decision-making. It is not intended to be an anthology of test and treatment efficacy, and neither is it meant to be another exploration of evidence-based medicine. Thirdly, it explores alternative ways of conceptualising and memorising numbers in medicine. What is interesting about dipping into values based medicine in this way, is that after a while it seems common to develop an intuitive ‘right-sized’ grasp of absolute benefits and harms – there are always surprises, but being way off becomes less common.
How much information is enough
There is a lot to learn in medicine, and much information is left out for undergraduates. ‘Need to know’ versus ‘nice to know’ usually means that one is left with a bare minimum of information allowing progression from diagnosis to treatment. What is often left out is how well stuff works. Evidence based medicine has made some inroads into the ‘how well’ desert, but strangely has had quite limited real penetration into daily practice.
A key reason for this is that it is actually still quite difficult to access the information and apply it to individual patients at the bedside. While there may be some evidence, this is very patchy, is unappealing because it lacks certainty, and is often regarded as geeky or inappropriate examination material.
EBM
Evidence-based medicine (EBM) has been around for more than two decades now. In that time it has been moved from a startlingly new concept to something that is taught routinely in all undergraduate courses and whose language is familiar to most clinicians. There is however still a yawning chasm between a background understanding of the concepts and the use of them in daily practice. Indeed, some EBM practitioners advocate that the calculations and quantification around decision-making based on EBM should be done in the background, formulated into guidelines and then these guidelines used in daily practice. The trouble with this is that in order to cater for all the nuances of care of an individual patient a guideline has to be exhaustive and almost impossible to read. Some of these guidelines, extending for over 250 pages, are astoundingly comprehensive but simply indigestible. Added to this patients often have conditions or combinations of conditions spanning several guidelines meaning that in order to provide comprehensive guideline-based care, a clinician may need to access over 1000 pages of information for a single patient. Guidelines also age – one estimate found that only 50% of NICE guidelines were still up to date after 5 years1 .
Using evidence
A further problem with standard evidence-based medicine is that although there are answers to the big questions many of the finer details remain unexplored. For instance, we all know that it is a good idea to give aspirin to somebody who has had a recent stroke. Some of us may even know roughly how well it works in terms of being able to give a value for a number needed to treat (the number of patients needed to treat to prevent one further stroke, or the number of patients needed to treat to prevent one patient dying, or the number of patients needed to treat to prevent the combined unpleasant outcome of either death or a second stroke.) The NNT always needs to be accompanied by a time interval e.g. giving aspirin for one year after a stroke to 80 patients might prevent one further stroke. If given for two years it might prevent a stroke in 40 patients.
So that is the easy bit. What dose of aspirin do you give? When do you start giving it? Are there particular patients in whom you would be disinclined to use it? Is the benefit linear (In other words does the effect that is achieved early carry on for as long as the medication is used or does this benefit wane over time? (Short answer – most benefit in first 3 months.)
All of these questions look relatively straightforward and easy to answer, and are just the stuff of guidelines. The problem is that the guideline in your hand is of necessity slightly out of date, and revising high quality guidelines takes months to years.
But are you right?
Using ‘best available evidence’ is not the same as necessarily being correct. One of the invidious difficulties of evidence based practice is the ability we all have to lull ourselves into believing that simply because we have explored the data and acknowledged its weaknesses, we somehow have transformed a state of uncertainty into one of greater certainty. It is useful to acknowledge that sometimes we just don’t know. Errors can be of certainty (poor quality evidence, or evidence pointing in different directions) or of precision (wide confidence interval on good evidence, because still not enough of it) and quite often both.
Information overload
A further challenge for younger clinicians has to do with the sheer volume of information needed to practice with a clear understanding of efficacy. Insurmountable seems to mean exactly that, and it numbs us into a belief that it is impossible to gain control of this body of evidence. The reality is that while that is technically correct – none of us can know it all (and we probably shouldn’t pretend to try) but quite a lot of cross-calibration is feasible.
What this means is that the higher the number of stable points of value information you have under your control, the better you become at estimating where new information will sit in terms of efficacy. It is an interesting exercise to do this when reading a new article: once you have gone as far through the abstract to find the primary endpoint, try to quantify (in absolute terms) the effect size of the intervention. The more you do this, the easier it gets to be reasonably accurate (and sometimes spectacularly and surprisingly wrong!)
But will this value apply to my patient?
Generalisability to patients outside the context of large US and European randomised trials2 is sometimes difficult. Where there is uncertainty, an attempt has been made to highlight this.3 The growing body of evidence-based clinical practice guidelines (the National Guideline Clearinghouse has nearly 400 on diabetes alone4) does not always provide guidance on clinical worth for the individual patient, particularly the elderly with multiple co-morbidities, and there are emerging concerns that too rigorous implementation of all possible recommendations may even be harmful5
Apart from some pharmaco-genetic variability, humans are remarkably similar the world over; the differences mostly arise from their society and their health-care environments. However the carefully selected participants in a drug trial designed to demonstrate efficacy may differ considerably from the spectrum of individuals, often with multiple co-morbidities, encountered in the real world. As a general rule trial data over-estimates real world benefits and underestimates harms.6
Alderson LJ, \alderson P, Tan T. Median life span of a cohort of National Institute for Health and Care Excellence clinical guidelines was about 60 months. J Clin Epidemiol. 2013 http://dx.doi.org/10.1016/j.jclinepi.2013.07.012 ↩
Layde PM, Broste SK, Desbiens N, et al. Generalisability of clinical studies conducted at tertiary care medical centres: a population based analysis. J Clin Epidemiol. 1996;49:835-41. ↩
Burgers JS, van Everdingen JJ. Beyond the evidence in clinical guidelines. Lancet. 2004;364:392-3 ↩
O’Connor PJ. Adding value to evidence-based guidelines. JAMA. 2005;294:741-3. ↩
Boyd CM, Darer J, Boult C, et al. Clinical practice guidelines and quality of care for older patients with multiple comrbid diseases. JAMA. 2005;294:716-24.. ↩
Contopoulos-Ioannidis D, Tseretopoulou X, Anker M, et al. Comparative rates of harms in randomised trials from more developed versus less developed countries may be different. J Clin Epidemiol. 2016;78:10-21 ↩