Discovering I knew nothing - a mathematician's insights on building a website to help people interpret children’s heart surgery data

I am an applied mathematician working at University College London and I’ve worked in the sensitive area of survival after children’s heart surgery for several years.  I thought I knew what is difficult to understand, what needs explaining and how to talk about it. But, in developing a new NIHR-funded website on survival after surgery, I sat down and really listened to parents of children who had had heart surgery as well as science writers, policy and press officers and discovered I knew nothing.

Making NHS surgery statistics public

In a movement triggered by the Bristol Inquiry in the late 1990s into deaths of children after heart surgery, the NHS is increasingly publishing statistics about the surgeries it undertakes. But making data public can only drive accountability and improvement if it is also understandable.

Children’s heart surgery is one the most complex specialties. Because children and their heart conditions are so different, one hospital could easily treat many more particularly complex cases than another hospital. This means that it is not fair or valid to simply compare the raw survival rates between hospitals - we need to adjust for the risk of each child using a statistical formula.

I was part of the team that developed this statistical formula, which is used every year by the National Congenital Heart Disease Audit to publish the previous 3 years’ survival data. However, these reports are by no means accessible or understandable for everyone.

What I did about it

Before applying for this current NIHR grant to update the statistical formula, I decided that my responsibility extended beyond developing as good a formula as I could – I also have a responsibility to communicate how the formula works and what it can and can't do. So in my grant application, I also asked NIHR for funding to build a website to explain how my work is used to monitor survival rates. NIHR were very supportive and in fact further encouraged me to be much more ambitious about engaging with parents and other users.

Listening to parents, professionals and the media: making it work

I had never done anything like this before, so  I gathered some experts: Sir David Spiegelhalter, Professor of the Public Understanding of Risk at Cambridge and his brilliant web guru Mike Pearson; the charity Sense about Science which specialises in responsible reporting of scientific evidence in the media; and Dr Tim Rakow at King’s College London, an experimental psychologist researching how people interpret the language and presentation of risk and risk-adjusted data.

For the next 15 months we went back and forth between listening to users and developing the website, the site constantly evolving. Tim's team also tested different versions of the language and layout on large groups of volunteers.

What I have learned: the big picture

  • People have a right to NHS information, presented in a clear and balanced way. And they value this access.
  • The website would have not have been fit for purpose without the early and continuous involvement of parents and other users.
  • There is a huge amount of effort involved in accessibly presenting abstract and difficult concepts – do not underestimate it and do not shirk it.
  • Yes, you can do it. And yes you should do it. Plus you might be surprised by how much your funder will support you!

What I have learned: the details I’ll take with me for a long time

  • These statistics are not just numbers, but represent the fears and worries of parents whose children have had to undergo heart surgery. Language has to be simple and straightforward as many relatives may read the site in a state of anxiety.
  • People will interpret things in ways you had no idea were possible – but in retrospect were entirely obvious. Repeating these checks over and over with fresh eyes is priceless.
  • Experts from other areas will challenge you and what your work means in new ways. Learn from them.
  • Never think you know how much detail people “need to know”. Let them tell you – I consistently provided too much and too little information about different parts of the work.
  • Layer the information as much as you can – not everyone wants the same information or in the same way.
  • You can spend days worrying about a single word (e.g. “unforeseeable” vs “unforeseen” vs “chance”). For key concepts, you need to.
  • This mathematician is not naturally gifted at web design. Or web navigation. Delegate.

I started off building a website because I felt it was my responsibility, but I didn’t really know what I was doing. But by forming a partnership with Sense about Science who have experience of public user-testing and wide collaborations with academics, we managed to integrate the maths stuff with the psychology of how people interpret charts and explanations. But, the most important contributions were the thoughts, worries and feedback from parents, professionals and media experts. For instance the existence, location and content of the “key points” came entirely from workshop feedback. It’s been an enormous amount of work, much more and much more difficult than I anticipated, but it’s been so rewarding too. I will take this experience with me into my future work: it is my responsibility to communicate what I’ve done to people who  it might affect and it is my responsibility to do this as clearly as I can. 

If you’re a researcher embarking on a public engagement project and would like some help and advice, please don’t hesitate to contact Emily Jesper, Sense about Science: enquiries@senseaboutscience.org.

Explore the website for yourself at http://childrensheartsurgery.info

 

Christina Pagel

Christina Pagel
Reader in Operational Research, University College London

The views and opinions expressed in this blog are those of the authors and do not  necessarily reflect those of the NIHR, NHS or the Department of Health.