Prognostic modelling: what is it and how can it be used?
by Sarah Seaton, RDS East Midlands
Within the RDS, different members of team have varying expertise and interests. Mine is largely in observational research, and prognostic modelling, and this is generally what I advise on. Recently, I gave some staff training on this topic within the East Midlands and the below is a transcribed version of that session.
What is prognosis?
Prognostic modelling aims to estimate the risk of a future outcome in an individual. The outcomes are generally very specific. Examples include: mortality; disease progression and disease recurrence. Different factors (variables) which help predict the risk of this event occurring are used. This is known as multivariable analysis. There are two kinds of studies: development and validation. Here I will focus on development, but the validation studies are as important, if not more so. So, what we are doing is taking factors about the patient (e.g. age, weight, smoking status) and using it to predict an outcome (e.g. development of lung cancer).
Selecting factors that predict the outcome
The factors which are used need to be objective (easy to measure). These factors need to have data available at the time the prediction is made. For example, think of a situation where a GP might want to predict a patient’s risk of developing diabetes in the next 12 months. They have a fifteen minute consultation period where they will have very basic information available to them, for example: blood pressure, weight, height and ethnic group. These simple, objective factors are appropriate for the setting (GP practice) and are then used to predict the risk of diabetes.
Statistical parsimony
Keeping in mind our example of a GP appointment brings us to the idea of statistical parsimony. This is basically the idea that everything should be as simple as it can be (for medical people this is very much like Occam’s Razor). Choosing which factors predict an outcome can be a statistical exercise, but (and bearing in mind I am a statistician!) it is probably far more sensible to consider this as a clinical question, which needs literature reviewing and discussion amongst colleagues. The key issue here is to have as few variables as you can, for statistical reasons and for simplicities sake.
Statistical analysis and sample size calculations
This is not the time or place to discuss the methods behind this, but I will recommend a few excellent articles and books for anyone interested in further reading. Suffice to say, the work is not always trivial, and definitely needs a statistician. Sample size calculations for these types of studies are perhaps a little “unorthodox” and won’t be what you are used to, so make sure the statistician involved has done one before, or has read about them.
Presentation of results
The main issue of risk scores is presentation and ease of their use. There are some very good examples of risk score presentation and use. For example, take a look at the CRASH-2 “calculator” for predicting mortality after head injury. Although not beautiful, it is very simple to get a result out.
However, this remains a difficult area, and important consideration needs to be given to the fact that the model needs to be possible to easily implement. I think, possibly, there is great i-phone app potential here, if a sensible model is developed.
Getting funding
It is very hard to get funding for prognostic studies, and they are often “bolt ons” to other studies. In my opinion there is a good reason for this. It is not enough to tell a patient they are “at risk” of a disease. Something has to change in light of this new found risk. If nothing changes, then it was perhaps unethical to identify the patient when nothing could be offered, and a feeling of “so what?” is left behind. This, I think, is particularly true of the NIHR funders. So, as you write that application, or develop that model, think to yourself: what is going to change with this knowledge?
This is a whistle stop article, and much more detail is needed for anyone wanting to do one of these studies. A good starting point is:
Moons et al. Prognosis and prognostic research: what, why and how? BMJ 2009; 338.
Steyerberg, E. Clinical prediction models: a practical approach to development, validation and updating.