Tag Archives: RDS SE

From frustration to funding

I attended the R&D meeting of one of our local NHS Trusts a few months ago where the main agenda item was the organization of the Trust’s annual research conference. During the meeting, two of the members started a fascinating discussion about the research process, which stayed with me long after the meeting had finished.

They were talking about the very beginning of the research process – about that spark of inspiration that can start an entire programme of research and from where such a spark can come. Interestingly, rather than talking about a fascinating theory or a novel approach, the fantastic force of inspiration they were discussing was that of simple frustration. The sources of this frustration encompassed a wide range of issues – from frustrations with clinical practice, patient progress and lack thereof, right through to frustrations with service organisation and delivery.

I found this a fascinating idea and it is one that is mirrored by the provision of research funding by the National Institute for Health Research (NIHR). The NIHR is largely a funder of NHS-based research – the funds are held by NHS Trusts rather than HEIs, the principle investigators are usually NHS clinicians, and the research topics prioritised are those which will bring about direct benefit to patients. This focus makes the NIHR unique amongst the variety of other funders of research that exist, including the research councils and numerous charities. The NIHR is all about research for, and from, the NHS. Research topics are prioritised by various panels precisely to ensure that the topics believed to be of greatest import to patients and NHS staff are those that receive funding.

When discussing funding applications with researchers, my colleagues and I often talk about the ‘story’ behind the research. This story is a vital one to tell when making an application for funding. It straddles various parts of the application form – the background and rationale sections, the PPI sections, the outcomes sections. Identifying the source of your spark of inspiration is an important part of telling this story. And, often, this spark is one of frustration with how things currently stand and the desire to change things for the better.

I wrote a blog post a while ago about turning the frustrations associated with poor feedback following funding application rejection into fuel to redesign and reapply. It seems only fitting to now attend to the other end of the spectrum and say that the frustrations felt about any issues of clinical practice can equally be used to inspire research questions. And, as with any research question, your local RDS can help refine and direct these frustrations into a research question appropriate for NIHR funding.

Don’t let frustrations get you down; use them to fuel your research questions and apply for funding to identify solutions. By so doing, you may uncover solutions that will improve care for patients as well as easing the frustrations of staff.

***

ETA: As Nikki points out below, the NIHR does fund non-NHS research – the Public Health Research programme being a prime example of this.

What is success?

I was part of a lively twitter discussion some time ago where the topic of ‘success’ was raised. What is success? From a researchers perspective, that’s a simple question to answer – success is getting the funding to do your study – but from an RDS perspective, the answer is much more complicated.

The discussion was started by Sarah Seaton, a statistician & RDS adviser for RDS East Midlands, who tweeted that she’d had the good news that a study she’d been advising on had been funded by the HTA programme. A group of us offered our congrats and then the Research Development & Innovation at University Hospitals Coventry & Warwickshire NHS Trust tweeted to say they were also waiting to hear about some funding outcomes for projects on which they’d advised. This comments sparked a new conversation about how services which support researchers making funding applications often do not hear about the outcome of these applications.

This is something that we at the RDS SE have discussed time and time again and, fortunately, it is something that is slowly changing. We will now hear officially on a regular basis about which projects have been funding by NIHR. This information can only be a good thing, especially in terms of evaluating and improving the advice we offer.

Many of the researchers I work with are ‘regulars’. Over time, we’ve build up a good relationship and whenever they are working on a project they come to me for advice. They’re great about letting me know the outcomes, something about which I love to hear from both a professional and a personal point of view. When you’ve been involved with a project from the early stages, you’re invested in the outcome even if you’re not actually going to be involved in the project itself. I put a lot of work and great number of hours into the projects on which I advise and it gives me a great sense of satisfaction to hear that one of them has been funded. This is, I am certain, true of every RDS adviser.

That said, it’s equally important to me to hear about the projects that have been rejected, especially when a researcher sends me the reviewers’ reports and the comments from the panel. Going through these helps me learn how to improve the advice I give. It also enables the researcher and I together to work out where to go next. As I’ve said before, many projects are funded upon resubmission, undoubtedly stronger for the extra work the initial rejected prompted.

However, I do sometimes work on projects which come in at the last minute and/or with researchers with whom I have not worked before. In these cases, I often don’t hear about the outcome of their applications. Even though I email to ask about the outcome once I know that the funding decisions have been made, I sometimes don’t hear back. It’s tempting in these scenarios to assume that their application has not been successful, but you can’t know for certain.

Hearing officially about the funding outcomes will have a number of benefits. For a start, it would make evaluating ourselves and improving our services much easier and more effective. This would be especially true if we could see the details of the panels and/or reviewers comments on the applications we supported. It will also help with the cohesiveness of the NIHR as a whole and be a great motivator for advisers who put so much time and effort into proposals they may never be involved with again.

It would also help us with our definition of ‘success’. To link this post back to where I originally started, for an RDS ‘success’ can be tricky to define. It isn’t always a case of getting a researcher to submit an application – this is a crude metric that misses out on the subtleties of what we do.

Often, for us, success is the exact opposite. Rather than getting an application submitted, success is getting a researcher to realize that submitting the application is the wrong decision. For many projects it would be far better to keep working on improving the research plan and wait until the next deadline. With many NIHR programmes, that’s only a 4 month delay. If you rush and submit a less than perfect project, you’ve then got a wait of usually 6 months to hear the outcome. And, if it’s a rejection, with many NIHR programmes there’s a 12-month waiting period before you can resubmit.

So, to sum this all up, I’d like to leave you with a couple of points. First, when you hear about a funding outcome, please do let your RDS (and whoever else helped you with the application) know and, if possible, let them see the feedback the submission received. This way, they can celebrate with you if the application was successful. And, if it wasn’t, then they can learn from the feedback, just as you can, and help you with the decision of where to go from here.

Secondly, do keep in mind that rushing to submit for that funding deadline isn’t always the best thing to do. Sometimes success is waiting in the short-term to enable you to get to the longer-term success of actually running your project.

Frustrating Funding Feedback – is there any other kind?

If you’re a researcher with even the smallest amount of experience, then you’re no stranger to rejection. It goes with the territory and the ups and downs of the application cycle is something to which we all have to adjust.

When it comes to rejection, perhaps the most frustrating thing about it all is the paucity of feedback. Many hundreds of hours collectively go into a single funding application. They are, of course, the work of the PI and collaborators primarily, but also finance people, R&D staff, possibly input from a CTU and, often, research advisers like myself. It is disheartening to learn that, after all this work, a project has been rejected. And it is beyond frustrating to see that the rejection of all this effort boils down to a few short bullet points on the rejection letter.

Of course I understand the sheer volume of applications that funders get and the pressure of the time limits under which they work. And, of course, different funders do offer different levels of feedback. Many programmes, RfPB for example, will send the PI the outcome letter with the ‘bullet points of doom’ and also include the reviews the application received.

The reviewers’ comments can be a bit of a double-edged sword. On the one hand, it can be gratifying to read the positive comments made by the reviewers in the face of an overall rejection by the funder. However, it can also be incredibly frustrating to realize that these positive comments were seemingly over looked by the panel and that the discussion obviously took a very different path. And when the bullet points don’t seem to match up with any of the reviewers comments, then you can be left rather bemused by the whole process.

At this point, I can understand the temptation to throw in the towel. And, for some projects, this probably is the end of the road.

However, this isn’t always the right move. If this was the project’s first rejection, then it is worth looking through all the feedback and identifying where changes could and should be made. I have seen many projects funded on their second or even third attempts. Indeed, many research teams benefit from getting the feedback about their project, however sparse it may seem, and being forced to rethink their original design. Although RfPB do not implement a time limit on resubmissions, other NIHR programmes do. However, this too is to the benefit of the project – in reality, you will need those 12 months to properly re-design the project and re-draft the application into something that is fundable.

Consult with your co-applicants and do get in contact with your local RDS. Even if you didn’t use an RDS in your first application, we would be happy to help with another attempt to secure funding. It is likely that this consultation alone will allow you to address some of the issues raised by the funding panel.

Research that comes from the right place – that is patient-centred, appropriately designed and proposed by a strong research team – is rarely entirely un-fundable.

Don’t let rejection get you down; just regroup, redesign and resubmit.

Research topics – the benefits of an eternal outsider?

I meet with a lot of NHS clinicians to talk about research questions and study design. Our region covers many different Trusts and the level of clinical expertise is immense. These meetings are always interesting and challenging. I have to confess, however, that I don’t usually have the first clue about the specifics of the topic area we’re discussing.

When I first started as research adviser, this bothered me greatly. It was what made me most nervous about meeting a clinical researcher for the first time. However, I’ve now learned that there is actually benefit in not knowing all the ins and outs of the topic area. After all, it’s the clinician who is the expert and it’s not clinical expertise they need.

Far better to let the clinician explain the research topic to me. I can then ask the simple and obvious questions that help elucidate the research question. This discussion usually forms the basis for the argument for why the research needs to be done. And determining the priority of the research topic is the very first thing a funder will do. For NIHR commissioned calls, this prioritisation process has already been at least partially done. However, for the plethora of researcher–led funding streams – both NIHR and otherwise – the argument about the importance of the research topic is the first one you must make.

The fact that I’m at a distance from the research topic means that I can play devil’s advocate. I have a tendency to cover draft protocols in comments and track changes and send emails full of questions to researchers who send me initial project proposals. It’s much better for me to be the one to do it than a funding panel who will ultimately reject a proposal if there are too many unanswered questions.

It’s difficult to know exactly who will be present at a given panel meeting where the funding decision will be made. Even though lists of members are available – and they are definitely worth a look – actual attendance varies. And, when it comes to RfPB, which panel will assess your application can vary too. Relatively recently, a bunch of ‘South East’ applications where discussed at a panel meeting in a different area entirely.

When it comes to the day of the actual meeting of the funding panel, there may be someone there just as experienced, or even more experienced, than the PI. But, then again, there may not. You also can’t predict which direction the discussion will take. Will the panel be focusing more on the priority of the topic area or will it be the design or the plan of analysis that takes up the bulk of time during the discussion? The discussion time will be tiny relative to the time that has gone in to preparing the application and so there’s simply no way of knowing which aspect in particular will capture the panel’s attention.

It’s my job to try to cover every possible angle.

Many RDSs offer researchers the opportunity to put their application forward for a ‘dry run’. At the RDS SE, we hold a regular ‘pre-submission panel’, where we all get together and try to replicate an NIHR funding panel meeting. As a group, we cover a range of specialities, both methodologically and with regards to specific health topics. We have lay representatives who attend and even use the RfPB programme’s assessment criteria to rate each submission. It’s a useful exercise for both the adviser and the researcher. From my perspective, I find it fascinating to see how my comments fit in with those of my colleagues and it’s reassuring to have actual proof that we’re all pretty much on the same page when it comes time to assess the proposals. And, from a researcher’s perspective, they get a lot of feedback on their proposal, far more than the few bullet points of doom that accompany the formal NIHR letters informing PIs about funding decisions.

I’ve come to embrace my role as the eternal outsider when it comes to the majority of specific health research topics. I believe it allows me to offer researchers a far more honest assessment of their research proposal and gives me the tools to push researchers to improve the quality of their applications. It’s also a strength of the RDS as a whole, allowing us to attempt to replicate funding panels for researchers.

If you are preparing an application for NIHR funding, it is worth talking to your local RDS and benefit from our expertise… and our lack of it.

Top tips from the RDS National Training Day!

I attended the NIHR Research Design Service National Training Day recently. I believe it’s the second one that’s been held, but it’s the first one that I’ve been able to attend.

Even within the Research Design Service South East (RDS SE), it’s fairly rare for members from our 3 sites to get together in person. We do it perhaps a handful of times a year – mainly for our thrice annually Pre-Submission Panels and our annual Away Day. So, the prospect of meeting up, with not only the other RDS SE people, but with those from all the other 9 RDSs was an exciting one.

Attendance was good, even from RDSs for whom the trip to London was considerably more arduous than my own one and a half hour direct train trip from Brighton. At my level – that of a general research adviser with no overarching strategic role – there’s basically no formal opportunity to contact people from the other RDSs. It’s one of the reasons that I’ve embraced social media to the extent that I have – it’s a great way of talking to other advisers in RDSs whom I wouldn’t otherwise meet. So to have this opportunity present itself, and for it to be face-to-face, was great.

I have to say the experience didn’t disappoint. There must have been about 150 of us there – a good representation across the board. And, for the most part, we were able to share experiences and reflect on the fact that, despite the regional variations across the country, we face the same issues and challenges when it comes to supporting researchers design fundable projects. The take-home message was that we are doing a good job, but that there is always room for improvement.

I’m not going to do a formal report of the day – that would, I imagine, be fairly dull. But I did want to share some of the things I found most valuable in terms of research advice. Hopefully this will resonate with anyone reading who works for an RDS – and perhaps some of you were there too? – and also be useful to any researchers reading, regardless of whether your located in an NHS Trust or an HEI.

One of the opening plenary talks was by Prof. Tom Walley, Director of the HTA programme and of the other NETS programmes. In terms of things changing, he was able to tell us that the HTA programme are piloting a new Expression of Interest form, which will take the place of the longer and more detailed outline application form currently being used. The goal is to make this first step in the application process easier and faster, both to write and to be reviewed. You can find out more about this here.

I was also able to pick out four top tips from Tom’s talk.

The first was about making very clear to funders the importance the research question to the NHS and its users. Determining the priority of the question is the first thing a funding panel will do, before they even look at who is on the team or how they’re proposing to address the question. This explanation needs to be explained clearly and convincingly to a non-expert reader.

The second point was about clearly defining the evidence gap, as defined by systematic reviews. Tom quoted the figure that half of studies are designed without reference to a systematic review of the evidence that already exists. This raises the question of whether, if a systematic review doesn’t exist in the area in which you work, should this be the first thing you do? Indeed, the RfPB programme has recently issued new guidance around applications for systematic reviews with a suggested funding limit of £150K. You can find out more about that here.

The third point was the observation that there are too few NHS applicants, especially acting as principle investigators. We RDSs need to be proactive about engaging with clinical researchers. We have work to do in liaising with Trusts and demonstrating the benefits engaging with us will have for their clinicians interested in research.

The final point was around value for money. Tom observed that we have likely reached, or are very soon to reach, a funding ceiling. This fact makes clearly describing the importance and relevance of your research question all the more important. Funders will increasingly only be funding ‘priority’ questions. There is a drive for efficiency, in terms of both study design and in making use of data sets already in existence and routinely collected data. The recent HTA call for ‘efficient study designs’ is a nice demonstration of this. Such studies also need to be pragmatic, to reflect the realities of care in the NHS. This drive for value will also mean fewer extensions and a greater monitoring of milestones and targets. Studies are more likely to be closed down if it becomes apparent that they will require a big extension in time and money in order to succeed. There will be more importance placed of having feasibility and pilot data in order to demonstrate that patients are out there and are willing to be recruited into, and remain in, clinical trials.

I found these messages helpful and will definitely be referring to them in my advising in the future. I was gratified to see a very clear role for the RDS in facilitating high quality research and hope that this is a message we can get out to researchers.

That is, in essence, the aim of this blog.

The NIHR – lost in the acronyms?

As I’ve commented before, the NIHR loves its acronyms. Work in the system for long enough and you can have entire conversations that consist largely of seemingly random strings of letters.

There are the research programmes – RfPB, HS&DR, HTA, EME, PHR, PGfAR, and PDG. Of course, then there’s i4i, which goes for a trendy look by using the now-ubiquitous lower case ‘i’ and adding a number into the mix.

Then there are the two centres that manage these research programmes – NETSCC and the CCF.

And let’s not forget the need to ensure your costings are in line with AcoRD guidance. And the CRN that provides the infrastructure and support for research in the NHS.

And then there are the RDSs that support researchers. There are 10 altogether. I’ll spare you the entire list, but let’s just say their acronyms are all a bit like the one for which I work – the RDS SE.

Now, I won’t dispute that these can be useful short-hand when talking to colleagues well habituated to this alphabet soup. But, they often present a real barrier to researchers on the ‘outside’.

And, to be honest, they can also be a barrier to those who work inside the system as well.

I used to be the Programme Manager for ID&P when I worked at NCCHTA, which is now NETSCC. (Translation: the manager for identification and priortisation for the National Co-ordinating Centre for Health Technology Assessment, now the NIHR’s Evaluations, Trials and Studies Coordinating Centre), where we would have monthly internal meetings, each one run by a different area. When it came time for my area to present, I put together a blockbuster game, complete with a hexagonal-celled board, for us all to play, to introduce everyone to the acronyms used by this one department alone.

The point of this is the importance of simplicity. From the start, the NIHR puts up a pretty big barrier to engaging with researchers, many of whom don’t even know what these letters stand for, let alone the acronyms for the myriad of research programmes, initiatives, documents and support organisations.

So, in an effort to cut through the minefield of letters, let me give a simple message:

I’m Claire, a research adviser. If you’d like to conduct health research into an issue you see in your clinical practice, then come talk to me. I can help you with your research question and design and also who to approach for funding. This is a free service and there are advisers located throughout England.

Find out more here.

Alternatively, comment on this post and I’ll help point you in the right direction.

ETA: There are a couple of good glossaries, of which I’ve just been reminded.
– The NIHR’s glossary is here.
– NETSCC’s glossary is here.
Many thanks to Nicola Tose for reminding me!

ETA2: Sarah Seaton has kindly added to the acronym list — see below for even more! I’m sure there are many more out there as well.

ETA3: Another one for the list: the lovely people at the Complex Reviews Support Unit (CRSU) who provide support for the delivery of complex reviews that are funded and/or supported by NIHR.

Winding paths

Research careers are often meandering. You move from one position to the next, at times by luck as much as judgment. Different universities, different countries, different projects. Great importance is given to this semi-nomadic existence. It certainly has its difficulties, especially as you grow older and add a partner and children into the equation. Yet, it also has its benefits and I recognize that I would not have the job I do today had I not had experiences of these different places, projects and roles.

I’m a methodologist; a job I would not have considered when I first got my PhD in psychopathology back in 2004. I work for the National Institute for Health Research (NIHR), specifically for the Research Design Service South East (RDS SE). The NIHR does love its acronyms. There are 10 RDSs nationally, each covering a different area of England, and the RDS SE covers the counties of Sussex, Surrey and Kent.

I don’t mean for this blog to turn into an advertisement for the NIHR or RDS, but the service we offer is, in my experience at least, unique. We help researchers turn their research ideas into projects capable of competing for NIHR funding. You come to us with an idea to improve patient care and we help you to formulate a research question, plan a study to address that question and write an application to get the work funded.

Many of the researchers I meet are surprised at just how much support we offer. It’s not just the mechanics of bid writing, although we do that too, but also advice in study design, statistics, health economics, patient and public involvement (PPI; another great NIHR acronym), amongst others. I have friends who work in different disciplines who would love to access the kind of help the RDS offers, but nothing that I know of like this service exists outside of the structure of the NIHR.

I love planning studies and considering the best design to address each research question. It’s a challenge to design something that satisfies as a robust, academic experiment and also as a practical, clinical piece of work applicable to patients in the NHS. I also relish being able to work with a wide range of people – each an expert in their field, but who share the commitment to improving the care they can give their patients. It’s a privilege to help them achieve this aim in some small way.

I’m fortunate that my particular winding path has led me here. Hopefully, from time to time, I can help others who are taking the next steps of their own.

Late to the party…?

I’m fairly new to using social media in a professional context. In my personal life, I’m young enough to have a Facebook, but too old to have a tumblr. And, even on Facebook, the friends I have are just that – people I’ve known for years and with whom I have regular contact. I only have 86 of them and I post a lot of pictures of my children. I imagine its pretty boring to the vast majority of people, especially those who don’t know me.

The thought of actually using such a platform for work was, until recently entirely foreign. When I came back from maternity leave in January, a colleague mentioned that the Research Design Service South East, for whom I work as a methodologist, has a Twitter account. It’s @NIHR_RDSSE for those of you interested. Hearing this, I was intrigued – what possible benefit could we get as a service from the occasional posting of 140 characters?

As I explored Twitter, I was surprised by not only the number of professional organizations that have accounts, but by the number of individuals who tweet in, at least a semi-, professional capacity. I came across links to papers that I immediately downloaded, highlighted events of which I hadn’t been aware, and came across funding deadlines that I hadn’t yet got around to flagging. It was a revelation.

However, even more than realising what a great resource Twitter was, I found myself intrigued by the conversations in which individuals were engaged. It soon became clear that there was a real community of people on Twitter who were dedicated to, and passionate about, health research and improving patient care. And, perhaps more importantly, they were using Twitter as a medium to pursue these interests and engage with others who shared them.

For a while, that’s as far as my involvement went. I bookmarked some accounts, lurked on a few blogs, but still didn’t feel that I had anything to add to the conversation. Some weeks later, however, I found myself buried under a pile of draft NIHR applications for various funding programs. I spent a long week going through and commenting on them all. This process is a regular part of my job, but having so many to review in the same few days is, thankfully, rare. As I worked my way through them, I found myself writing the same set of comments time and again. These weren’t long, specific comments about individual applications, but rather short, global comments that I was repeating verbatim. I should write a list and hand them out to researchers, I thought to myself and that’s when it hit me – I could do just that and I knew the perfect platform. By the end of the day, I had a Twitter account, @ClaireRosten, and a hashtag, #NIHRtips.

I still tweet #NIHRtips whenever new ones come to mind and, to my delight, others also use the hashtag. I get re-tweets and have interactions with others because of them. A blog seems to be the logical progression. I hope that it will lead to more interactions and conversations with others who share my interests.

Of course, it also gives me something else to tweet about.

I realise I’m late to the party. But hopefully there’s enough time left for me to pick up a drink and join in.