Face to Face

My tour of the RDSs – which commenced with my trip to Leicester to observe RDS EM’s NIHR Fellowship Day – continued recently with a visit to RDS London. I was invited by Lauren Bell to observe their Proposal Feedback Forum and to meet Lauren and her RDS London colleagues.

It has always been one of my intentions, with this blog, to reach out to other RDS staff and advisers across England. Although we all offer the same type of support and guidance to researchers applying for funding for health and social care research, our different areas are quite distinct – each with its own needs and requirements when it comes to how this support is delivered. RDS London serves, as you might expect, a tremendously research-active part of the country, and I was excited by the opportunity to observe them in action.

I’ve written before about what I view as the main ‘USPs’ of the RDS. Peer review is, in my mind, one at the very top of that list: the fact that researchers, regardless of stage or experience, can access free help and advice from methodological experts in a wide variety of areas. Offering advice on a one-to-one basis is a core part of the way all RDSs do business: a researcher who approaches us asking for advice will typically liaise with a single adviser, who will be their main point of contact with the wider RDS. However, in addition to this lead adviser, researchers can access a breadth of methodological expertise, as required. One of the ways that many of the RDSs choose to expand upon this is to hold meetings where advisers act as a funding panel, reading, assessing and giving feedback to researchers before they submit their applications to be faced with the real deal. This trial run can be tremendously valuable to researchers – allowing them to fine-tune their applications and, on occasion, make the sometimes difficult decision that more work is required before they’re ready to submit for real.

One of the main difficulties faced by researchers who have had applications rejected by funders is the paucity of information about how to improve their applications prior to resubmission. I’m a big believer in re-designing and re-submitting, as I’ve written about before. However, to many clinical researchers who are relative novices when it comes to research, this type of rejection can be devastating and mean that they turn away from research, leaving their important, clinically-driven questions unanswered. This is unfortunate as these are the very researchers that the NIHR is keen to engage and to encourage to conduct research.

In the South East, we hold a regular Pre-submission Panel, where we ask researchers to submit draft funding applications which are reviewed by a large group of our advisers. Our collective advice is then fed back to the researcher by the lead adviser. In essence, we mimic a funding panel but with the added advantage of offering detailed advice on how to improve applications. RDS London do something similar, as do many other RDSs.

In the past couple of years, in addition to this type of review, RDS London have offered the opportunity for researchers to come and present to their panel in person. They still submit their near-to-completion funding application for review, but they also prepare a 10-minute presentation, including any specific questions they want answered, to present to RDS panel prior to a larger discussion of the proposal. This allows for immediate feedback to the researcher about their plans and their questions, and also for a discussion to take place with them about any concerns identified by the panel.

The particular panel I observed had 2 researchers presenting – one an experienced academic researcher with a clinical background and the other a practicing clinician who was a more novice researcher. After each of the of their presentations, they would get comments and questions from a range of methodologists, covering statistics, health economics, Patient & Public Involvement (PPI), qualitative research and health psychology. A discussion would ensue during which various aspects of the question, application and design would be clarified or altered. It was a fascinating process to observe, not least because I could see the benefit of the process to both the experienced researcher and the less experienced. The entire process was positive and encouraging, with an emphasis on making the research application stronger and more competitive. There was no doubt that both researchers left having received valuable feedback which will hopefully translate into successful applications, better research and ultimately benefit to patients.

There is definitely something to be said for having the researcher present to face a panel and answer questions directly. Issues can often be resolved far more easily in this kind of environment when compared to via email or even face-to-face meetings with just a single adviser. It can also help with what is one of my biggest frustrations when working with a researcher: what to do when your advice is ignored. There is something about advisers coming together in this way that makes it harder to brush our advice aside.

It is also a wonderful opportunity of personal development, and RDS London recognise and take advantage of this. It is reassuring for more junior or newly-appointed advisers to observe more experienced colleagues and learn from the type of advice they give.

For my part, there are three take home messages. First, that RDS advice can make a real difference to the competitiveness of research applications regardless of how experienced the researcher may be. Secondly, that the format used by RDS London is something worth exploring in my own patch down in the South East. And, finally, that despite our geographical barriers, we are one RDS, offering the same high-quality advice and support to researchers across England.

If you’re a researcher in London working in health or social care, do get in touch with RDS London. If you can, do ask to present at their Proposal Feedback Forum.

If you’re elsewhere in England – there is an RDS for you! Find out where here.

If you’re in the South East – Surrey, Sussex or Kent – come talk to me and my colleagues!

Time to Write?

Finding time to focus on research can be challenging. Research is, for many of the clinicians with whom I work, something that is done in their ‘free’ time. They recognise the value of research and believe passionately about delivering evidence-based care to their patients, yet it is not something they can necessarily dedicate time to during their working week. They have heavy clinical responsibilities and having the time to think about their research plans can be difficult; more difficult still is finding the time to actually sit down and tackle one of the NIHR research programme’s detailed application forms.

And, yet, these are the very people the NIHR is trying to reach. As I talked about here, the NIHR is a funder of research for, and from, the NHS. It is these clinical researchers – those who know intimately the problems faced in clinical practice – who we want to encourage to put forward questions to address through research to find evidence-based solutions that they can use in their practice to the benefit of their patients.

There are a number of ways to tackle the problem of time. I believe that working with an RDS adviser, like myself, can help make the process of designing a research study and submitting a funding application easier. As I’ve said before, there are many advantages to making use of the services the NIHR Research Design Service offer to health and social care researchers free of charge. I won’t go into detail of these here, but our collective experience with funding applications and funders, our individual expertise as researchers, and our ability to offer peer and lay review are just a few ways we can support busy clinicians who wish to pursue research.

Another way of making time for research is to actually allocate a block of it specifically for writing. Bits and pieces, written and read here and there in evenings and weekends can make for a difficult to follow funding application. It is surprisingly easy to spot when applications have been written by different people and no one has had the time to go though the entire document and pull it all together. But, often, there are few alternatives.

Last March, I was fortunate enough to be invited to speak and advise at a Research Retreat run by the Research Design and Conduct Service (the Welsh equivalent of the RDS). I wrote about my experience of the event specifically, and about the usefulness of Research Retreats in general, on my return. I found the event positive, productive and rewarding. It was a real pleasure to have an extended amount of time to dedicate to the development of specific research proposals and to observe just how far the research teams had progressed in the development of these over the course of just 3 days. With dedicated time to write, and experienced advisers to offer support, real progress was evident.

RDS SW have run a similar type of Retreat annually since 2009. Like my experience in Wales, they have received very positive feedback. Participants, in particular, have welcomed the dedicated time to concentrate solely on their research with RDS staff available throughout to offer expert support and guidance. It is always hard to come up with reliable measures of success when it comes to funding applications (as I’ve commented on here). It is also difficult to know for sure what leads to success. That said, RDS SW are aware of 20 applications, worked on during their Retreat, which were successfully submitted to funders. Of these, 9 were funded. This, if you’re at all familiar with the typical success rates of the various NIHR funding programmes (RfPB, for example, has a success rate around 20%), is pretty impressive.

Over the past couple of months, I’ve been involved in a group made up of RDS staff from across the country with the aim of offering something similar to health and social care researchers in England. This is the first time such an event has been offered nationally, but it is our hope that it will give busy clinicians the time, space and support to act on their research aspirations and produce an almost submission-ready funding application.

For this first venture, teams of 2 or 3 researchers can apply to take part in a 5-day RDS Residential Grant Writing Retreat to be held from 13th – 17th June 2016 in Bath. More details of the entry requirements, application and selection process, cost, and venue are available here.

If you think this is something that might interest you and your fellow researchers, do have a look. I can only speak from my own experiences of a similar type of event, but I do believe there is much to be gained by participating. At the very least, you are guaranteed dedicated time to write, with the help and guidance of experienced RDS advisers. And working in a supportive environment with other committed researchers and RDS advisers is likely to bring about even more.

NIHR Fellowships & ‘One NIHR’

I was lucky enough recently to visit my colleagues at the Research Design Service in the East Midlands (RDS EM), in Leicester to be specific. This was a first for me – to actually meet in person other RDS colleagues in their own patch. As I’ve written about before, we have a so-called ‘National Training Day’ the is held roughly every 2 years, but attending that has been, until now, been my only experience of meeting other RDS colleagues.

Social media can be a great enabler of actual face-to-face contact. My very first ‘follower’ on twitter was Sarah Seaton, an RDS adviser and NIHR doctoral fellow. Through twitter we’ve had numerous conversations on a wide range of health-research and RDS-related topics. As I’ve commented on before, the health research community on twitter is lively and varied. This interaction then led up to a hugely enjoyable Google hangout via the NIHR Hub (where there may or may not have been virtual hats involved at various points in proceedings), and finally to us deciding that it would be a good idea to arrange to meet up in person. Although we are badged as ‘one RDS’ and ‘one NIHR’, the fact still remains that many of us on-the-ground advisers don’t really have all that much contact with the wider, national RDS or, indeed, NIHR.

One of the things that Sarah does in RDS EM is to arrange a 2-hour NIHR Fellowship event every year. As she is herself an NIHR fellow, as well as being a RDS adviser, she is in the perfect position to do so. As we in the RDS SE were due to hold our fellowship event, for the first time, in a couple of weeks, it seemed like the perfect opportunity for me to visit, observe the event, and meet Sarah and her RDS EM colleagues.

Now, I can’t really write this particular post without sharing what I learned from the event. NIHR Fellowships, to quote Dawn Biram from the Trainees Coordinating Centre (TCC) who spoke, fund research and training to develop the research leaders of the future. With a variety of pathways open – for medics, clinically-trained health professionals, and non-clinical health researchers – at a variety of levels – Masters right the way up to Senior Lectureships – they are certainly a something worth considering when planning your research. And, as with all health and social care research applications, they are something on which your local RDS can provide advice, support and guidance.

Sarah had arranged for a range of speakers, all of whom play a different role in the fellowships. There was Dawn from the TCC, the secretariat which manage the fellowship scheme for the NIHR, Matt Bown, a current panel member for the doctoral fellowship pathway, Rhiannon Owen, a current NIHR fellow, Emma Watson, a doctoral fellow with Kidney Research UK, and finally Clare Gilles, an RDS EM adviser.

I was tweeting pretty much continuously during the event as there were so many tips and hints from each of the speakers. I use #NIHRtips, if anyone is interested, as this is a hashtag that I, and others, use for all sorts of NIHR and funding-related advice.

The overwhelming message from all the speakers was just how long it takes to put together a competitive application. These are huge endeavours and require input not just from yourself, but from a range of people both within and outside your NHS Trust or HEI. It is not unexpected for a fellowship application to take 6 months to a year to get right. Make sure you look at the guidance from last year’s competition and use that as a starting place – don’t wait until the new competition opens to put pen to paper (or fingers to keyboard). Try to get your hands on a successful fellow’s application form, so you can see what a really good application looks like (and just how long and complex it is!).

Applications are judged on 3 main things: you as an applicant, your research project & training plan and your institution (both NHS and HEI). Each one of these – person, plans, place – will require careful thought and preparation. So, update your CV, get that publication in, and apply for that small pot of funding to kick-start things. Talk to your local RDS about your plans and carry out a PPI consultation on your research ideas and design. Think about what other organisations you should involve – a clinical trials unit, the clinical research network, a charity or patient stakeholder group. Look at your options for HEIs, supervisors, and mentors. Approach the top people in your field and get advice. Get an idea of what training is out there and plan to attend the best there is available. Get your research networks started now.

There were also lots of tips about filling in the actual form. Panel members are only given a couple of weeks to shortlist 15 or so applications, so do whatever you can to make yours stand out. Be neat and careful with your spelling and grammar. Be consistent in your use of numbering and make sure your references are correct. Use bold, italics and underline to make your application clear and easy to read – don’t just have unbroken lines of dense text.

If you’re invited to interview, this is your chance to really demonstrate that you live up to your application in person – candidates are only interviewed if they are potentially fundable on paper. Make sure your presentation is second-perfect and set up as many mock interviews as you possibly can – your local RDS can help you with this. Google your panel and make sure you’re familiar with them, their research interests and their institutions. You can probably work out fairly easily who will be leading your interview, so anticipate what you might be asked. Be ready to defend your research plan, but don’t ignore input from the panel either. These people are experts in their fields, so give in to their greater experience if they suggest things to you. While you’re speaking, be confident and come out and move around the room. Remember, the panel is looking for future research leaders, so show your passion for your topic. Finally, your last slide will be left up during the Q&A portion of you interview, so use it to leave the panel with the message you want to convey.

Such events are invaluable and getting to talk to people directly can be really helpful if you’re planning to apply for an NIHR fellowship. Contact your local RDS and see if they’re doing something. Even if they’re not, they’ll have the experience of working with many NIHR fellows at various levels. They can help you with your application and may also be able to put you in touch with others who’ve been on the same journey.

From my point of view, it’s this collective experience that is so very valuable. When we share our experiences across the NIHR, we become stronger as a research community. We can help and support each other, offer advice and critical evaluation, and provide a network and a support group for researchers at all levels of the NIHR research pathway. I gained a lot from my visit, both in terms of knowing more about the NIHR fellowships, but also about what it means to be part of the larger NIHR and how, as a research community, we all have our parts to play in ensuring that patients and the public benefit from the very best evidence-based health and social care.

Adaptive Trials

We recently had a speaker come talk about adaptive clinical trials. It was a good seminar with lots of clear, real-life examples and it was well-attended by RDS advisers and researchers based both in universities and in the NHS. I won’t go into detail about the content of the seminar – people far better qualified than me have already done so in abundance – but it did get me thinking about how such designs could be used by the researchers with whom I and my colleagues work.

A large proportion of the health research funded by the NIHR, through such programmes as HTA, EME and RfPB, are clinical trials. And there are many issues to consider when designing an RCT, some of which I’ve discussed before. Most researchers engaged in such projects have input from one of the many Clinical Trials Units (CTUs) and have experienced statisticians, data managers and trial managers on their research teams. And, of course, many RDS advisers who support such studies ourselves have this experience.

What is particularly relevant about adaptive trials, not least in the context of providing advice for health researchers, is the opportunity for flexibility they offer. The goal of an adaptive design is to enable researchers to learn from the accumulating data and make key design changes accordingly as things progress. Usually, while a trial is still in progress, we do not look at any of the data. However, the accumulating material may be more informative than that available before the trial had started. And it is this previously available data – things like effect sizes, recruitment strategies and randomisation, dosages – on which the design of the trial was based. The idea behind adaptive designs is that a trial can be improved by making use of interim data to refine certain aspects of it.

Improvement can mean a number of things, usually to do with making a clinical trial more efficient. It can, for example, lead to doing away with a treatment arm if a particular dosage or intervention is demonstrated to be inappropriate. It can also mean identifying with greater accuracy the number of patients needed or lead to refinements in the recruitment strategy, target population or treatment randomisation. It can even lead to decision-making about, amongst others, key objectives, end-points, test statistics, or subsequent phases of the research.

The trick to it all is in the pre-specification: making it very clear in the trial protocol what purpose will be served by carrying out an interim analysis, when it will be done, on what measures, by whom, and to what end. You need to put procedures in place to ensure that the blind holds and limit the people who see the data at this pre-specified interim stage.

As an RDS adviser, I can see merit in this approach. The ability to carry out a sample size re-estimation, for example, is something that I think could benefit many projects. So too is the ability to drop inferior treatment groups or use preferential randomisation. There are, of course, many other options for adaptive designs, and these are just two examples that I can think of which could be applied to many projects.

The drive for efficiency in health research is nothing new. I’ve written about it before and there are examples of it in the NIHR funding programmes themselves – a recent example being the call the HTA programme issued in the summer of 2014 for ‘efficient study designs’. And, as always, an integral part of any NIHR funding application is the demonstration of the value for money of the research itself. This is, of course, not to say that adaptive trials are the answer or even appropriate. Such designs come with their own risks – errors due to their greater complexity, more time needed in the planning stages, uncertainties around the ethical implications, and the need for greater regulatory review, to name just a few.

As always, the design of a study needs to fit its research question. Adaptive trials offer an intriguing option when uncertainties mean refinements are required during the trial itself in order to optimise its design.

RDS USPs?

I’ve done a few short talks recently about the NIHR Research Design Service and the services we offer. It has been an interesting exercise – to try to distill in a short space of time the hours of concentrated effort that RDS advisers put into the applications we support and the variety of guises our advice can take. In many ways, this has been a marketing exercise – detailing the ‘unique selling points’ of the RDSs in a way that would appeal to NHS clinicians who are either already involved in research or are interested in so being.

I’ve talked numerous times before about the support RDS advisers can give, but it has been interesting to really focus in on what is unique about the RDSs. I think it comes down to 3 things: (1) collective experience, (2) peer review, and (3) lay review.

First of all, collectively, we have experience with literally hundreds of funding applications from a wide range of applied health and social care research funders. We’ve seen what works as well as what doesn’t. We’ve worked on bids that have been funded first time around and ones that have been funded fifth time around. We’ve worked on a huge variety of research topics, we know our remit as advisers and our strengths as researchers and methodologists in our own right. And, perhaps most importantly of all, we occupy a unique position of being committed to an applications success and yet not being part of the research team and, as such, not too close to the research idea. Of course, this is not to say that we can therefore guarantee a particular application’s success – but making use of our expertise certainly can’t hurt.

Peer review is, in my opinion, another big USP. As far as I am aware, all of the 10 RDSs in England offer some form of formal peer review. Like many, the RDS SE holds a regular meeting, which we call a ‘pre-submission panel’. Advisers from across our region get together and review funding applications in detail in a way that mirrors as closely as possible the assessment process of the NIHR research programmes. This trial run gives researchers an invaluable opportunity to address any potential weaknesses identified in their application prior to submission. It also allows them to make the, sometimes vital, decision not to submit just yet.

Thirdly, lay review is something that many RDSs offer. At the RDS SE, we offer researchers the opportunity to have their applications reviewed by 2 lay reviewers. This is also linked to the peer review process – where 2 lay reviewers sit on our pre-submission panel. Again, this reflects to some extent the NIHR’s assessment procedure and also allows researchers to hear directly from lay representatives their views on the research question and the research team’s plans to address it.

Now, I don’t in any way believe that consultation with an NIHR RDS will necessarily mean a successful application for funding. Certainly this would be impossible given that, in the South East at any rate, our aim is to provide advice and methodological support for as many projects as possible.

However, this does bring us to a fourth USP: we provide our service to researchers free of charge.

So, my take home message is this: if you are preparing an application for health or social care research funding, come and talk to us.

From frustration to funding

I attended the R&D meeting of one of our local NHS Trusts a few months ago where the main agenda item was the organization of the Trust’s annual research conference. During the meeting, two of the members started a fascinating discussion about the research process, which stayed with me long after the meeting had finished.

They were talking about the very beginning of the research process – about that spark of inspiration that can start an entire programme of research and from where such a spark can come. Interestingly, rather than talking about a fascinating theory or a novel approach, the fantastic force of inspiration they were discussing was that of simple frustration. The sources of this frustration encompassed a wide range of issues – from frustrations with clinical practice, patient progress and lack thereof, right through to frustrations with service organisation and delivery.

I found this a fascinating idea and it is one that is mirrored by the provision of research funding by the National Institute for Health Research (NIHR). The NIHR is largely a funder of NHS-based research – the funds are held by NHS Trusts rather than HEIs, the principle investigators are usually NHS clinicians, and the research topics prioritised are those which will bring about direct benefit to patients. This focus makes the NIHR unique amongst the variety of other funders of research that exist, including the research councils and numerous charities. The NIHR is all about research for, and from, the NHS. Research topics are prioritised by various panels precisely to ensure that the topics believed to be of greatest import to patients and NHS staff are those that receive funding.

When discussing funding applications with researchers, my colleagues and I often talk about the ‘story’ behind the research. This story is a vital one to tell when making an application for funding. It straddles various parts of the application form – the background and rationale sections, the PPI sections, the outcomes sections. Identifying the source of your spark of inspiration is an important part of telling this story. And, often, this spark is one of frustration with how things currently stand and the desire to change things for the better.

I wrote a blog post a while ago about turning the frustrations associated with poor feedback following funding application rejection into fuel to redesign and reapply. It seems only fitting to now attend to the other end of the spectrum and say that the frustrations felt about any issues of clinical practice can equally be used to inspire research questions. And, as with any research question, your local RDS can help refine and direct these frustrations into a research question appropriate for NIHR funding.

Don’t let frustrations get you down; use them to fuel your research questions and apply for funding to identify solutions. By so doing, you may uncover solutions that will improve care for patients as well as easing the frustrations of staff.

***

ETA: As Nikki points out below, the NIHR does fund non-NHS research – the Public Health Research programme being a prime example of this.

What is success?

I was part of a lively twitter discussion some time ago where the topic of ‘success’ was raised. What is success? From a researchers perspective, that’s a simple question to answer – success is getting the funding to do your study – but from an RDS perspective, the answer is much more complicated.

The discussion was started by Sarah Seaton, a statistician & RDS adviser for RDS East Midlands, who tweeted that she’d had the good news that a study she’d been advising on had been funded by the HTA programme. A group of us offered our congrats and then the Research Development & Innovation at University Hospitals Coventry & Warwickshire NHS Trust tweeted to say they were also waiting to hear about some funding outcomes for projects on which they’d advised. This comments sparked a new conversation about how services which support researchers making funding applications often do not hear about the outcome of these applications.

This is something that we at the RDS SE have discussed time and time again and, fortunately, it is something that is slowly changing. We will now hear officially on a regular basis about which projects have been funding by NIHR. This information can only be a good thing, especially in terms of evaluating and improving the advice we offer.

Many of the researchers I work with are ‘regulars’. Over time, we’ve build up a good relationship and whenever they are working on a project they come to me for advice. They’re great about letting me know the outcomes, something about which I love to hear from both a professional and a personal point of view. When you’ve been involved with a project from the early stages, you’re invested in the outcome even if you’re not actually going to be involved in the project itself. I put a lot of work and great number of hours into the projects on which I advise and it gives me a great sense of satisfaction to hear that one of them has been funded. This is, I am certain, true of every RDS adviser.

That said, it’s equally important to me to hear about the projects that have been rejected, especially when a researcher sends me the reviewers’ reports and the comments from the panel. Going through these helps me learn how to improve the advice I give. It also enables the researcher and I together to work out where to go next. As I’ve said before, many projects are funded upon resubmission, undoubtedly stronger for the extra work the initial rejected prompted.

However, I do sometimes work on projects which come in at the last minute and/or with researchers with whom I have not worked before. In these cases, I often don’t hear about the outcome of their applications. Even though I email to ask about the outcome once I know that the funding decisions have been made, I sometimes don’t hear back. It’s tempting in these scenarios to assume that their application has not been successful, but you can’t know for certain.

Hearing officially about the funding outcomes will have a number of benefits. For a start, it would make evaluating ourselves and improving our services much easier and more effective. This would be especially true if we could see the details of the panels and/or reviewers comments on the applications we supported. It will also help with the cohesiveness of the NIHR as a whole and be a great motivator for advisers who put so much time and effort into proposals they may never be involved with again.

It would also help us with our definition of ‘success’. To link this post back to where I originally started, for an RDS ‘success’ can be tricky to define. It isn’t always a case of getting a researcher to submit an application – this is a crude metric that misses out on the subtleties of what we do.

Often, for us, success is the exact opposite. Rather than getting an application submitted, success is getting a researcher to realize that submitting the application is the wrong decision. For many projects it would be far better to keep working on improving the research plan and wait until the next deadline. With many NIHR programmes, that’s only a 4 month delay. If you rush and submit a less than perfect project, you’ve then got a wait of usually 6 months to hear the outcome. And, if it’s a rejection, with many NIHR programmes there’s a 12-month waiting period before you can resubmit.

So, to sum this all up, I’d like to leave you with a couple of points. First, when you hear about a funding outcome, please do let your RDS (and whoever else helped you with the application) know and, if possible, let them see the feedback the submission received. This way, they can celebrate with you if the application was successful. And, if it wasn’t, then they can learn from the feedback, just as you can, and help you with the decision of where to go from here.

Secondly, do keep in mind that rushing to submit for that funding deadline isn’t always the best thing to do. Sometimes success is waiting in the short-term to enable you to get to the longer-term success of actually running your project.

Frustrating Funding Feedback – is there any other kind?

If you’re a researcher with even the smallest amount of experience, then you’re no stranger to rejection. It goes with the territory and the ups and downs of the application cycle is something to which we all have to adjust.

When it comes to rejection, perhaps the most frustrating thing about it all is the paucity of feedback. Many hundreds of hours collectively go into a single funding application. They are, of course, the work of the PI and collaborators primarily, but also finance people, R&D staff, possibly input from a CTU and, often, research advisers like myself. It is disheartening to learn that, after all this work, a project has been rejected. And it is beyond frustrating to see that the rejection of all this effort boils down to a few short bullet points on the rejection letter.

Of course I understand the sheer volume of applications that funders get and the pressure of the time limits under which they work. And, of course, different funders do offer different levels of feedback. Many programmes, RfPB for example, will send the PI the outcome letter with the ‘bullet points of doom’ and also include the reviews the application received.

The reviewers’ comments can be a bit of a double-edged sword. On the one hand, it can be gratifying to read the positive comments made by the reviewers in the face of an overall rejection by the funder. However, it can also be incredibly frustrating to realize that these positive comments were seemingly over looked by the panel and that the discussion obviously took a very different path. And when the bullet points don’t seem to match up with any of the reviewers comments, then you can be left rather bemused by the whole process.

At this point, I can understand the temptation to throw in the towel. And, for some projects, this probably is the end of the road.

However, this isn’t always the right move. If this was the project’s first rejection, then it is worth looking through all the feedback and identifying where changes could and should be made. I have seen many projects funded on their second or even third attempts. Indeed, many research teams benefit from getting the feedback about their project, however sparse it may seem, and being forced to rethink their original design. Although RfPB do not implement a time limit on resubmissions, other NIHR programmes do. However, this too is to the benefit of the project – in reality, you will need those 12 months to properly re-design the project and re-draft the application into something that is fundable.

Consult with your co-applicants and do get in contact with your local RDS. Even if you didn’t use an RDS in your first application, we would be happy to help with another attempt to secure funding. It is likely that this consultation alone will allow you to address some of the issues raised by the funding panel.

Research that comes from the right place – that is patient-centred, appropriately designed and proposed by a strong research team – is rarely entirely un-fundable.

Don’t let rejection get you down; just regroup, redesign and resubmit.

Research topics – the benefits of an eternal outsider?

I meet with a lot of NHS clinicians to talk about research questions and study design. Our region covers many different Trusts and the level of clinical expertise is immense. These meetings are always interesting and challenging. I have to confess, however, that I don’t usually have the first clue about the specifics of the topic area we’re discussing.

When I first started as research adviser, this bothered me greatly. It was what made me most nervous about meeting a clinical researcher for the first time. However, I’ve now learned that there is actually benefit in not knowing all the ins and outs of the topic area. After all, it’s the clinician who is the expert and it’s not clinical expertise they need.

Far better to let the clinician explain the research topic to me. I can then ask the simple and obvious questions that help elucidate the research question. This discussion usually forms the basis for the argument for why the research needs to be done. And determining the priority of the research topic is the very first thing a funder will do. For NIHR commissioned calls, this prioritisation process has already been at least partially done. However, for the plethora of researcher–led funding streams – both NIHR and otherwise – the argument about the importance of the research topic is the first one you must make.

The fact that I’m at a distance from the research topic means that I can play devil’s advocate. I have a tendency to cover draft protocols in comments and track changes and send emails full of questions to researchers who send me initial project proposals. It’s much better for me to be the one to do it than a funding panel who will ultimately reject a proposal if there are too many unanswered questions.

It’s difficult to know exactly who will be present at a given panel meeting where the funding decision will be made. Even though lists of members are available – and they are definitely worth a look – actual attendance varies. And, when it comes to RfPB, which panel will assess your application can vary too. Relatively recently, a bunch of ‘South East’ applications where discussed at a panel meeting in a different area entirely.

When it comes to the day of the actual meeting of the funding panel, there may be someone there just as experienced, or even more experienced, than the PI. But, then again, there may not. You also can’t predict which direction the discussion will take. Will the panel be focusing more on the priority of the topic area or will it be the design or the plan of analysis that takes up the bulk of time during the discussion? The discussion time will be tiny relative to the time that has gone in to preparing the application and so there’s simply no way of knowing which aspect in particular will capture the panel’s attention.

It’s my job to try to cover every possible angle.

Many RDSs offer researchers the opportunity to put their application forward for a ‘dry run’. At the RDS SE, we hold a regular ‘pre-submission panel’, where we all get together and try to replicate an NIHR funding panel meeting. As a group, we cover a range of specialities, both methodologically and with regards to specific health topics. We have lay representatives who attend and even use the RfPB programme’s assessment criteria to rate each submission. It’s a useful exercise for both the adviser and the researcher. From my perspective, I find it fascinating to see how my comments fit in with those of my colleagues and it’s reassuring to have actual proof that we’re all pretty much on the same page when it comes time to assess the proposals. And, from a researcher’s perspective, they get a lot of feedback on their proposal, far more than the few bullet points of doom that accompany the formal NIHR letters informing PIs about funding decisions.

I’ve come to embrace my role as the eternal outsider when it comes to the majority of specific health research topics. I believe it allows me to offer researchers a far more honest assessment of their research proposal and gives me the tools to push researchers to improve the quality of their applications. It’s also a strength of the RDS as a whole, allowing us to attempt to replicate funding panels for researchers.

If you are preparing an application for NIHR funding, it is worth talking to your local RDS and benefit from our expertise… and our lack of it.

Top tips from the RDS National Training Day!

I attended the NIHR Research Design Service National Training Day recently. I believe it’s the second one that’s been held, but it’s the first one that I’ve been able to attend.

Even within the Research Design Service South East (RDS SE), it’s fairly rare for members from our 3 sites to get together in person. We do it perhaps a handful of times a year – mainly for our thrice annually Pre-Submission Panels and our annual Away Day. So, the prospect of meeting up, with not only the other RDS SE people, but with those from all the other 9 RDSs was an exciting one.

Attendance was good, even from RDSs for whom the trip to London was considerably more arduous than my own one and a half hour direct train trip from Brighton. At my level – that of a general research adviser with no overarching strategic role – there’s basically no formal opportunity to contact people from the other RDSs. It’s one of the reasons that I’ve embraced social media to the extent that I have – it’s a great way of talking to other advisers in RDSs whom I wouldn’t otherwise meet. So to have this opportunity present itself, and for it to be face-to-face, was great.

I have to say the experience didn’t disappoint. There must have been about 150 of us there – a good representation across the board. And, for the most part, we were able to share experiences and reflect on the fact that, despite the regional variations across the country, we face the same issues and challenges when it comes to supporting researchers design fundable projects. The take-home message was that we are doing a good job, but that there is always room for improvement.

I’m not going to do a formal report of the day – that would, I imagine, be fairly dull. But I did want to share some of the things I found most valuable in terms of research advice. Hopefully this will resonate with anyone reading who works for an RDS – and perhaps some of you were there too? – and also be useful to any researchers reading, regardless of whether your located in an NHS Trust or an HEI.

One of the opening plenary talks was by Prof. Tom Walley, Director of the HTA programme and of the other NETS programmes. In terms of things changing, he was able to tell us that the HTA programme are piloting a new Expression of Interest form, which will take the place of the longer and more detailed outline application form currently being used. The goal is to make this first step in the application process easier and faster, both to write and to be reviewed. You can find out more about this here.

I was also able to pick out four top tips from Tom’s talk.

The first was about making very clear to funders the importance the research question to the NHS and its users. Determining the priority of the question is the first thing a funding panel will do, before they even look at who is on the team or how they’re proposing to address the question. This explanation needs to be explained clearly and convincingly to a non-expert reader.

The second point was about clearly defining the evidence gap, as defined by systematic reviews. Tom quoted the figure that half of studies are designed without reference to a systematic review of the evidence that already exists. This raises the question of whether, if a systematic review doesn’t exist in the area in which you work, should this be the first thing you do? Indeed, the RfPB programme has recently issued new guidance around applications for systematic reviews with a suggested funding limit of £150K. You can find out more about that here.

The third point was the observation that there are too few NHS applicants, especially acting as principle investigators. We RDSs need to be proactive about engaging with clinical researchers. We have work to do in liaising with Trusts and demonstrating the benefits engaging with us will have for their clinicians interested in research.

The final point was around value for money. Tom observed that we have likely reached, or are very soon to reach, a funding ceiling. This fact makes clearly describing the importance and relevance of your research question all the more important. Funders will increasingly only be funding ‘priority’ questions. There is a drive for efficiency, in terms of both study design and in making use of data sets already in existence and routinely collected data. The recent HTA call for ‘efficient study designs’ is a nice demonstration of this. Such studies also need to be pragmatic, to reflect the realities of care in the NHS. This drive for value will also mean fewer extensions and a greater monitoring of milestones and targets. Studies are more likely to be closed down if it becomes apparent that they will require a big extension in time and money in order to succeed. There will be more importance placed of having feasibility and pilot data in order to demonstrate that patients are out there and are willing to be recruited into, and remain in, clinical trials.

I found these messages helpful and will definitely be referring to them in my advising in the future. I was gratified to see a very clear role for the RDS in facilitating high quality research and hope that this is a message we can get out to researchers.

That is, in essence, the aim of this blog.