masterclass in writing rfpb grant applications 28 th january 2011 carole fogg sophie hyndman helen...

Masterclass in Writing RfPB Grant Applications

28th January 2011

Carole Fogg

Sophie Hyndman

Helen Lloyd

Presented January 2011; updated and adapted for web, January 2012

Welcome!

Introductions

Aim and Objectives of the Day

RDS South Central:Research Design Support for NHS

and Social Care Researchers

Who are we and what can we do for you?

Some of our A-Team:Multi-disciplinary team:

• Statisticians

• Clinical Trialists

• Epidemiologists

• Qualitative Researchers

• Health Economists

• Patient/Public Experts

• Health Psychologists

What can the RDS SC do to help you?For all aspects of grant preparation, advice on:

• Sources of funding• Literature search and review• Developing and refining research objectives• Study design, including health economics & statistics

• Planning teams and budgets• Patient and Public Involvement (PPI)

• Research Advisor, with access to specific experts as required• Research Design Advisory Panels• Potential for collaboration

Communications: e-mail, newsletter, website

Training: research master classes

Question: In what way is a research grant application like a general election campaign?

Answer:They both require the right balance between

Aim

To support researchers in developing successful research grant proposals, specifically, Research for

Patient Benefit applications

ObjectivesTo provide:• An overview of funding applications, and how they relate to research

governance and ethics applications• An overview of NIHR funding schemes and the purpose and scope

of RfPB• An understanding of what reviewers expect to see in an RfPB grant

application• Some guidance on identifying what makes ‘good’ and ‘bad’ grant

applications• An introduction to the structure and content of the RfPB form• An understanding of the importance of patient and public

involvement in RfPB research• An understanding of methodological issues important to reviewers• An understanding of the resources and time required to undertake a

research project• An introduction to other sources of research support

Part 1Research in the NHS: Funding

Applications in Context

Important initiatives

• Good Clinical Practice (GCP) and the UK Medicines for Human Use (Clinical Trials) Regulations 2004

Responsibilities and accountabilities for clinical trials of investigational medicinal products

• Research Governance Framework for Health and Social Care (2005)

Informed by above, but for research more generally

• Best Research for Best Health (2006)

Infrastructure and systems

Salvatore Vuono. http://www.freedigitalphotos.net/

images/view_photog.php?photogid=659

http://www.freedigitalphotos.net/images/view_photog.php?photogid=659


The purpose of these initiatives

• To protect the safety, dignity, rights and wellbeing of research participants • To promote useful, ethical, valid, safe and affordable research• To prevent futile, unethical, invalid, dangerous and extravagant research• To provide research standards• To define responsibilities and accountabilities• To set out delivery and monitoring systems to ensure standards

Research Governance

Best Research for Best Health

• To make the UK a recognized centre of healthcare research excellence by providing a good research infrastructure

• To build research capacity• To fund research to improve health and social care• To improve and manage knowledge and information systems to simplify

research processes and maximise usefulness of research

Among other things…

• A range of funding schemes for health research• A system of research approvals to ensure that research

is well regulated and ethical

The result of these initiatives

What are the main ‘hoops’ to jump through?

All research carried out in the NHS requires:

• A sponsor • Ethics approval• R&D approvals

Most research will require:

• Funding

Some research will require

• Other approvals (e.g. Clinical Trial Authorisation)• Honorary contracts or research passports

1 ) Hula hoop moment from Circus Smirkus performance 2006. http://smirkus.orgImage provided courtesy of Wikipedia. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2

Where do I start?

Sayan Samanahttp://www.freedigitalphotos.net/images/

view_photog.php?photogid=2067

Steps to take1....

3. Apply for

researchfunding

5. Apply fo

r R+D

approval*

1. Write

proposal/

protocol

7. If required, attend to research passports/ honorary contracts

6. If required, attend to other approvals*

2. Find a sponsor

4. Apply for ethics

approval*

* IRAS

1. May not always be in this order! Image from Office Online

Q. What is the difference between a research protocol and a research grant application?

A. A research protocol is the definitive document that will inform the process of the research. Some R&D offices will insist on a GCP approved protocol, and provide templates for this (regardless of whether the project is a Clinical Trial of an Investigational Medical Product (CTIMP) or not)

The research protocol can be used as the basis for applying for ethics and funding, and there will be an overlap, but these latter documents will have different emphases, so it is not simply a matter of cutting and pasting

See http://www.oxfordradcliffe.nhs.uk/research/downloads.asp

Downloads 16 & 17 for GCP approved templates

Steps (1) Write a research protocol and (3) Apply for funding

Image from Office Online

http://www.oxfordradcliffe.nhs.uk/research/downloads.asp

Step (2) Find a sponsor

i.e.:

• Responsibility for the initiation and overall management, including quality assurance (research protocol, team, research environment)

• Responsibility for the finances associated with the study• Ensure all approvals obtained, e.g. ethics, clinical trial authorisation• Arrangements for handling of any Investigational Medicinal Products

(IMP)• Ensure Good Clinical Practice (GCP), monitoring and reporting

(including safety reporting)

Q. Why do you need a sponsor?A. You need an organisation to be accountable for the

research


Q. Who can be a sponsor?

A.Usually:

• Chief Investigator’s employing institution, e.g. a university or NHS trust

• A funding organisation• Where research is for training purposes, the research supervisor (on

behalf of their employer)• Private sector e.g. pharmaceutical industry

Step (4) National Research Ethics Committee (NRES) approval

http://www.nres.npsa.nhs.uk/

Role:

• Protect rights, safety, dignity and well-being of research participants• Facilitate and promote ethical research of potential benefit to

participants, science and society.

Advice on: information sheets and consent forms, GCP, procedures, use of personal data etc.

Apply: online via Integrated Research Application System (IRAS) https://www.myresearchproject.org.uk/


Step (5) Local trust research and development office (R&D) approval

Role:

• To give permission to conduct research in the site• If sponsor, to undertake that role; if hosting research (sponsored by

others), to undertake audit of a proportion of studies• To ensure all relevant research-related contracts are in place

Advice available:

• Variable and dependent on personnel/ time: NRES applications, financial aspects of study from trust perspective, more general research advice


When do you meet them?

• Variable and dependent on personnel/ time: for high risk/ drug related/ device related studies, as soon as you have a protocol; some offices may be interested in these and other studies at the development stage.

Apply:

1. Via IRAS Coordinated System for gaining NHS Permission (CSP) for research qualifying for the NIHR Clinical Research Network Portfolio

2. Directly to R&D Office for all non portfolio research.

http://www.crncc.nihr.ac.uk/about_us/processes/portfolio

http://www.crncc.nihr.ac.uk/about_us/processes/portfolio

Step (6) Other approvals

IRAS captures the information needed by:• Administration of Radioactive Substances Advisory Committee

(ARSAC) • Gene Therapy Advisory Committee (GTAC) • Medicines and Healthcare products Regulatory Agency (MHRA) • Ministry of Justice (National Offender Management Service) • HSC research offices • NRES / NHS / HSC Research Ethics Committees • National Information Governance Board for Health and Social Care

(NIGB). • Social Care Research Ethics Committee


Step (7) Research Passports and Honorary Contracts

Q. What is a research passport?

A. A set of signatures and checks from your employing organisation required prior to issuing an honorary contract or letter of access.

Q. What is an honorary contract?

B. A contract with an NHS trust for a researcher whose employer is not that NHS trust.

Q. What is a letter of access?

R. A letter permitting a researcher whose employer is not that NHS trust into the trust


Q. When are honorary contracts needed?

A. When: (a) the research is hosted in the NHS and (b) the researcher has no contract with the NHS and (c) the researcher’s activities in the NHS site will involve interacting with individual patients in a way that has a direct bearing on the quality of their care.

Q. When are letters of access needed?

B. When (a) and (b) above are present, but not (c)

Always check with trusts what level of access they consider you need

http://www.nihr.ac.uk/systems/Pages/systems_research_passports.aspx

http://www.nihr.ac.uk/systems/Pages/systems_research_passports.aspx

Q. Why are honorary contracts needed?

1. Patients in receipt of research procedures come under the NHS duty of care and the NHS indemnity scheme applies to the researchers and their activities.

2. The researcher is accountable to the NHS organisation for their activities, explicit supervision arrangements are in place, and the researcher is made aware of the relevant NHS policies with which they must comply in their activities.

3. The researcher’s substantive employer understands the research activities and has undertaken all the necessary checks in relation to the researcher’s suitability to carry out those activities.

Q. Where do I apply to get an honorary contract?

A. Usually Human Resources Department of NHS Trust where you wish to do the work.

(1) Involving a Comprehensive Local Research Network (CLRN)

http://www.ukcrn.org.uk/index/networks/comprehensive/clrns.html

Can work with investigators to:

• Provide NHS Support Costs for research projects• Facilitate permissions • Help with recruitment of sites and patients • Help with data collection• General advice on the research process in their area of expertise.

Steps you MIGHT have to take:


(2) Involving a Clinical Trials Unit (CTU)

Can work with investigators to:

• Develop new trials (design, collaboration, funding, communication with Clinical Research Network, regulatory and governance issues)

• Manage funded trials (questionnaire design, set-up, recruitment, pharmacovigilance, data management, monitoring, analyses, report preparation)

http://www.ukcrc-ctu.org.uk Image from Office Online

Part 2 – Research for Patient Benefit (RfPB) & The Importance of Sc pe

Image courtesy of Wikimedia commons (Public Domain)

NIHR Research Programmes

• Research for Patient Benefit (RfPB)

• Programme Grants for Applied Research (PGfAR)

• Invention for Innovation (i4i)

• Health Technology Assessment (HTA)

• Efficacy and Mechanism Evaluation (EME)

• Public Health Research (PHR)

• Health Services and Delivery Research (NIHR HS&DR)

- formerly 2 programmes: Health Services Research

(HSR) & Service Delivery and Organisation (SDO)

http://www.ccf.nihr.ac.uk/RfPB/Pages/home.aspx/

http://www.ccf.nihr.ac.uk/RfPB/Pages/home.aspx/

Research for Patient Benefit offers…

The opportunity to conduct research that:

• arises from daily practice in the NHS• is developed between NHS and other partners• is open to topic and method• shows tangible health and health care benefits for patients• amplifies the patient voice• that encourages collaboration• closes the loop between research and practice – will be

implemented or drive change

Aims / sc pe

• To support projects in health services and public health research

• Qualitative, quantitative or mixed method research that will:– study the way NHS services are provided and used– evaluate whether interventions are effective and

provide value for money– examine whether alternative means for providing

healthcare would be more effective

RfPB will also fund research that will:

• pilot or assess the feasibility of projects requiring major applications to other funding bodies

• formally assess innovations and developments in healthcare

Proposals which have emerged from interaction with patients/service users and public are particularly welcome

Systematic reviews may be funded as part of a larger project or as stand alone studies. For more details, see the RfPB FAQ on

systematic reviews.

Aims / sc pe

http://www.ccf.nihr.ac.uk/RfPB/Pages/FAQ.aspx

http://www.ccf.nihr.ac.uk/RfPB/Pages/FAQ.aspx

OUT of sc pe:

• Lab-based or basic science research, including

research based on animals

• Setting up or maintaining research units

• Proposals which are solely service developments

• Proposals which are solely audit, surveys, needs

assessment, technology development (although may

be included as part of an integrated research study)

Images from Office Online

Read the guidance...

Each RfPB competition has associated Guidance for Applicants that covers:

• aims• scope• eligibility etc

For general enquires

call 0208 843 8057

Or email [email protected]

Or visit http://www.ccf.nihr.ac.uk


mailto:[email protected]

http://www.ccf.nihr.ac.uk/

Selection Criteria and Eligibility

Applications judged on:

• the quality of the research proposed

• the significance and potential benefit of the research to the NHS

All NHS organisations and other providers of NHS services in England can apply (For joint NHS/university applications funding is awarded to the NHS partner)

• Probably with academic partners

• Potentially with strong component of service user involvement.

• Applications with collaborations in social care and third sector providers of health and social care are also encouraged

If strong case, can include academic partner outside England.


The Application Process• Full application deadline

• Preliminary scrutiny and is it in scope?

• External peer and lay review

• Full committee assessment

• Final ratification

• Notification of outcome Images from Office Online

Overall, a wide range of RfPB projects funded

• Based in a diversity of settings and care groups

• Concerned with clinical and non-clinical topics

• Using a diversity of methods and research techniques

See the detailed report on the first three competitions –

Prioritising Patients

Part 3 RfPB APPLICATIONS

A grant application, like a picture, should be:

• Clear• Unambiguous• Easy on the eye ☺• Easy on the brain

Photo by Remy Steinegger; Copyright : World Economic Forum. Licensed under the Creative Commons Attribution-Share Alike 2.0 Generic license (http://creativecommons.org/licenses/by-sa/2.0/deed.en)Source: Wikipedia

http://creativecommons.org/licenses/by-sa/2.0/deed.en

It should not be :

• Jumbled up• Unclear• Difficult to decipher• Confusing

Or indeed something you didn’t intend it to be

Original Author: David Angell (adapted by Odder). Licensed under Creative Commons Attribution 2.0 Generic Licence (http://creativecommons.org/licenses/by-sa/2.0/deed.en). Source: Wikipedia



(a) What would you expect to see in an RfPB grant application?

Part 3 RfPB APPLICATIONS (b) Examples of ‘good’ and ‘bad’

grant application sections

Group/interactive work

Simon Howdenwww.freedigitalphotos.net


For examples of good & bad grant application sections,

see [weblink]

http://www.freedigitalphotos.net/


(c) What a reviewer expects to see in an RfPB application form

Overall style

Limit use of overly technical language, buzzwords and acronyms

• Tech Speak (defined by Edward Tenner)“Post colloquial discourse modulation protocol for user status enhancement. It's a referential system for functional-structural, microscopically specific macroscopic-object redesignation. It's a universal semantic transformation procedure. It's a holophrastic technocratic sociolect. It's a meta-semiotic mode for task specific nomenclature." Get the idea?

• BuzzwordAn important-sounding word or phrase connected with a specialized field or group that is used primarily to impress laypersons: “‘Sensitivity’ is the buzzword in the beauty industry this fall.”

• Alphabet soupLanguage laced with acronyms. "An RFP was issued by DOE; ORNL responded for the ORR."

Reference: http://www.designsensory.com/pws/lesson4/index.htm, ‘Tech Speak’ as defined by Edward Tenner (1986) Tech Speak, Random House, in above website.

http://www.designsensory.com/pws/lesson4/index.htm

1. Research details

2. Lead Applicant Details

3. Lead Applicant CV

4. Joint Lead Applicant(s)

5. Co-applicant(s) details

6. Research & Development Office

7. History of the Application

8. Patient and public involvement

9. Case For Support – Part 1

10. Case for Support – Part 2

11. Case For Support – Part 3

12. Management and Governance

13. Intellectual Property and Innovation

14. NIHR Infrastructure

15. Finances

16. Monitoring information

17. Suggested reviewers

18. Supporting Documentation

19. Declarations and signatures

(1) Research details

• Title• Host organisation• Region• Duration• Proposed start date• (Application type) – Pilot, Feasibility, or Main Study*• (Theme Call Name) – Only fill this in if you are applying for a

specific theme call, e.g. dementia

*For definitions, see: http://www.netscc.ac.uk/glossary/

http://www.netscc.ac.uk/glossary/

Research Title

• Descriptive, yet concise

• Contains keywords relevant to the research

• Study question and main outcome measure to be included

• If a trial, also include intervention and population

• State if pilot/ feasibility

• Clear from title that study is in scope – e.g. for RfPB, has short-medium term patient benefit

• Whilst you may think some titles are genuinely funny:

Parasites as weapons of mouse destruction.

Journal Animal Ecology. 2008 Mar;77(2):201-4.

• They may be viewed by some as inappropriate:

Targeting promiscuous signaling pathways in cancer: another notch in the bedpost

Trends in Molecular Medicine. 2004 Dec;10 (12):591-598 .

Title: What can go wrong

Title: What can go wrong

• And some can appear very strange indeed:

Who Moved My Cheese by Dr Spencer Morgan

Putnam Adult (1998)


A bestseller!

• Lead applicant• Current Research Commitments and how this application will fit within

current research activities• Lead applicant’s C.V.• Joint lead, Co-applicants. N.B. There is now a system for co-applicants to

formally confirm involvement, add C.V.s, and approve the final application. The latter must be done before submission.

Consider:• Kudos associated with being lead• Not necessarily a big-wig but need proper support if junior• Tends to be person initiating the research• Contract duration (especially universities), i.e. if your contract expires before

project, can you be lead?

(2-5) Applicants

Q. What is the difference between ‘co-applicants’ and ‘research team’?A. A research team may consist of more people than simply the applicants, but

N.B. the application will be judged primarily by those appearing on the form

• Host must be an NHS organisation

• Host R&D office contact details

(6) Research and Development Office


(7) History of This Application

• If previous RfPB, NIHR may well look to see what the initial feedback was and how the current application has changed, BUT always viewed as a NEW application (no resubmission policy)

• If previously submitted elsewhere, just for their information

• Details of related applications

• Details of concurrent applications are not asked for, but RfPB would obviously want to know if a submitted project gets funded from another source

(8) Patient and Public Involvement

Image courtesy of healthcaregovernancereview.wordpress.com

What is Patient and Public

Involvement?

• An active partnership between the patient/public and researchers, NOT the use of patients/public as ‘subjects’ to be studied (e.g. to be interviewed or assessed) by researchers

For example:

• Working together with patients to develop a good research proposal

• Commenting on the methods or processes of the research, being actively involved in the research process


Examples of PPI

• Being part of the research team

• Helping design the research

• Being a member of project reference group

• Writing lay summaries and patient information sheets

• Carrying out questionnaires with the participants

More power in the decision making process of the research


Nature of PPI in Research

Consultation: Asking people who use your services for their views on your research to help inform decision making

E.g. Commenting on the ability to understand lay summaries

Collaboration: Ongoing partnership with members of the public throughout the course of the research project.

E.g. A member of public on the project steering group.

User-Led/ User Controlled:

Members of the public lead the research and are in control of the research. This is often, through a community or voluntary organisation

E.g. Service users may decide on the research question; or manage the research itself

Proper PPI in Research IS NOT:

• Asking the patient or the public to answer research questions and provide data, i.e. asking them to be a ‘guinea pig’ in your research – enrolling them into the study

• Listening to patients to design your service better – to elicit service design suggestions

• Asking for feedback about a web based intervention – to evaluate an intervention


X

Why Involvement?

• To prevent researchers making mistakes that are obvious to patients

• To guide finances

• For greater transparency, making research with patients,

for patients, that is more relevant to patients.

• Obligation to the tax payer

• To meet the requirements of the NHS Constitution

• To meets the requirements of funders

Can you spot Involvement?

The following are all extracts taken from National

Institute of Health Research grant proposals

Which are Involvement and which are not?


"We plan a steering group for the project with four patient representatives as equal members of the research team."

1. Involvement

2. Participation

3. None of these

4. Unsure

"Individual service users who have been referred ... via this and other routes will be interviewed regarding their experiences."

1. Involvement

2. Participation

3. None of these

4. Unsure

"Our patients agreed with us that the study would deliver important information and that the research was timely and

important."

1. Involvement

2. Participation

3. None of these

4. Unsure

What can the RDS Offer?

• We provide a lay review service as part of our pre-submission review panel and as a separate activity

• We can advertise your project through our PPI network and help to identify patients to work with you

• We can provide financial support to patients or members of the public working on projects supported by the RDS up until the point of submission.

• We can provide PPI advice from our staff on the PPI team.

http://www.rds-sc.org/public-patient-involvement/

What can the RDS do?

• Give guidance on payment structures.

• Provide assistance in writing lay summaries usually by getting lay members to write them either with you or for you.

• We have useful guidance helping you consider what involvement to use to get the best out of involvement in your project.

David Castillo Dominiciwww.freedigitalphotos.net



How can we work together?

• Maybe you are part of, or know of, a group that wants to get involved in research, if so register the group via the RDS website

• Researchers can advertise their projects through our systems

• If you would like to get involved individually as a patient or a member of the public register with us via the RDS website


http://www.rds-sc.nihr.ac.uk

http://www.rds-sc.nihr.ac.uk/

What reviewers expect to see: Patient & Public Involvement (PPI)

• Have the needs of the service users been considered?

• Have you made any assumptions about service users or the co-operation of other involved parties which may not actually be met?

• Are there lay co-applicants?

• Are there lay people on the project steering group?

• Have the relevant national / local organisations been contacted for their input? Are there plans to involve them in the study design / study management / in dissemination of the results?

• Is there evidence of consulting the INVOLVE guidelines on PPI?

• State if PPI input at study design stage was not undertaken and say why this was the case (N.B. all efforts must be made...)

• Is the PPI element ‘tokenistic’, or has real thought and planning been put into it?

(9) Case for Support - Part 1

• Aims and objectives

• Plain English (Lay) and Scientific Summaries

• Background and rationale

• Why research is needed now

Aims and Objectives

• Need absolute clarity of objectives / primary outcome / secondary outcomes, and ensure that these relate to RfPB scope and are not contradicting the title

• Is this definitive / exploratory / preliminary work? What would the upshot be if it is successful in the case of the latter?

• If there is a null hypothesis, is it clear and appropriate?

• Do any qualitative aspects of the research have clearly stated objectives / questions / intentions?

• If a pilot or feasibility study, make this clear, and also how it would lead on to full trial

Q. What is the difference between a feasibility study and a pilot study?

A. Feasibility Studies “Can this study be done?” Used to estimate important parameters needed to design the main study, e.g. standard deviation of the outcome measure; willingness of participants to be randomised and clinicians to recruit them; number of eligible patients; characteristics / design of proposed outcome measure; follow-up rates, response rates, adherence/compliance rates

Pilot studies are a version of the main study that is run in miniature to test whether the components of the main study can all work together, e.g. to test recruitment, randomisation, treatment, and follow-up assessments

(http://www.netscc.ac.uk/glossary/#glos6)

Abstract in plain English(Lay Summary)

Concise, convincing summary of:

• Problem (e.g. disease burden)

• Research question and rationale

• Design and method

• Expected outcomes

• Expected impact

BUT absolutely understandable to the lay person:

• No clinical or research jargon

• No difficult words

Involve a PPI representative!

(Scientific) Summary

Concise, convincing summary of:

• Problem (e.g. disease burden)

• Research question and rationale

• Design and method

• Expected outcomes

• Expected impact

A ‘Mini-Me’ of the main protocol!

Background / Rationale

This section needs to

PACK a PUNCH

!!!

Using the literature:

Image from Wikimedia Commons (Public Domain).

• Define the research issue

• Justify WHY the research is necessary – clear case for clinical importance and

impact - E.g. Disease / condition burden (local / national figures)

• Back up logic of aims and objectives

• Provide evidence of best design approach

• Highlight if study fits into a desired but under-represented area in the funding

stream - E.g. social care, minority groups

Relevance of proposal to RfPB

Look at RfPB scope in Guidance

Think of the:

• Likely benefits to patients• Implications for health services• Potential impact on policy / service delivery

Make sure you are showing how your work fits in with

the RfPB remit


Why is this research needed now?

• What evidence is available to justify the study at this time: make a case

• Is the research addressing the right questions in the subject area at the right time in relation to, for example, the availability of knowledge, or the development of technology, Government priorities?

Are you presenting a convincing argument?

Don’t forget this is a competitive process!

(10) Case for Support Part 2: Research Plan & Methodology

Image: NASA

You should include:• Study design – appropriate to meet aims and objectives• Setting (location, population)• Sample – sampling frame, eligibility criteria• Sample size• Intervention (if relevant)• Details of randomisation if RCT• Appropriate data types and sources• Outcomes/ other measures – validated/appropriate for the population • Procedures • Data analysis methods

Consider:• How will potential sources of bias be avoided/ taken care of? Who is

blinded and how will this be achieved?• Feasibility• Bear in mind what will be reported (e.g. CONSORT, STARD etc)

• Clearly defined and consistent with study title/objectives• Eligibility criteria – how generalisable are the results to the NHS?

?GENDER

?disease severity

?comorbidities

?location

?new diagnosis

?previous treatment

?concomitant medications

?exposure

Study population

?age

Image by Xedos4, freedigital photos.nethttp://www.freedigitalphotos.net/images/


Intervention (i)

• Clear description of intervention

?product / technology/therapy ?timing

?dose ?length of exposure

?number of sessions/treatments

?staff administering treatment (training, quality assurance, experience, preferences)

• Allocation - randomisation method – blocks / stratification / matching• Blinding – who, how and why?• Balance between standardisation and tailoring to the patient• Will it be affected by patient/ carer adherence?

Intervention (ii)

Why was it chosen?

• Evidence of piloting / testing or feasibility/ effectiveness from the literature

• Clear description and evidence-base for control / comparison group(s)

For Example,

• Why placebo / alternative treatment chosen?

• Why 3 weeks treatment rather than 6?

Intervention (iii)

• Feasibility and other benefits

• Provision for gathering information on the patients / staff view of the research, whether it works or not

• Patient benefit of intervention – Recovery– Empowerment– Independence– Increased confidence in their outlook– Benefits added on existing standard of care

Outcomes• Measurements must accurately reflect the impact of the intervention • If surrogate / proxy measures are used, give evidence that these

can accurately predict long-term outcomes

• WHO?• HOW?• WHEN?• WHY?

Follow-up:• Appropriate length of follow-up to see the

effect on outcome measures• Motivation of patients to complete study!

Image in Public domain, courtesy of Wikimedia

Statistical Considerations

Statisticians are useful

for much more than just a

sample size calculation!!!

Prof. David Spiegelhalter playing with Arco Iris samba band, by Tamara K.This file is licensed under the

Creative Commons Attribution-Share Alike 3.0 Unported license

(http://creativecommons.org/licenses/by-sa/3.0/deed.en)

http://en.wikipedia.org/wiki/en:Creative_Commons



Type of outcomeo Continuouso Binary

Clinically relevant and relevant to patients Measurable Accurate

Cohort study Cross sectional study Case-control study Randomised controlled trial

Effect sizeo Clinically significanto Results in a change in practice

Based on pilot study/previous evidence Correct and appropriate for the study design chosen Account for loss to follow-up

Which patients to include in the analysiso E.g. principle of intention to treat in

randomised controlled trials Account for confounding Missing data Subgroup analysis Multiplicity Sensitivity analysis

Defining study objective

Defining a study design

Defining the primary outcome

Sample size calculation

Statistical analysis

Sample size

• State null hypothesis and choice of statistical test• Is the detectable effect size likely to be clinically significant and

result in a change in practice?• Is the sample size based on previous evidence / pilot results?

• Are the sample size calculations correct and appropriate for the study design chosen?

• Has loss to follow-up been accounted for in the sample size?

Common statistical errors in applications

• Statistical analysis does not meet objectives or type of data collected

• Section is too ‘general’ ‘continuous data will be analysed using t-tests’

• Copied from other/ previous grant application without tailoring

• No costing for statistical support

Outcomes / Analysis• Measurements accurately reflect the impact of the intervention or

are surrogate / proxy measures (and if so, evidence that these can accurately predict long-term outcomes)

• Appropriate length of follow-up to see the effect on outcome measures

• If there is a long period of follow-up, how will the patients be motivated to complete the study?

• Protection of outcome measurement from bias• Sub-analysis clear and documented – rationale and design• Listing of confounding factors – measured / factors for stratification

with rationale • Set out analysis plan logically – descriptive tabulations, approach to

analyzing associations among variables, qualitative analytic methods

What is Health Economics?

• The NHS operates in a world of scarce resources (e.g. MONEY, labour, medicines, hospital beds)

• Health economics informs how scarce resources may be allocated among treatment, prevention & diagnosis of different diseases. At the extreme:

Karl Marx: State allocates resources

Adam Smith (man in £20 note): Free marketImages in Public Domain. Courtesy of

Wikimedia.

How do we prioritise resources to treat & prevent disease?

• Need (how do you measure it / competing needs)

• Discrimination (e.g. by age, smokers …)

• Personal merit and social esteem

• Lottery

• Ability to pay (free market approach)

Do you think any of these methods would work in the NHS consistently?

In the NHS, new interventions prioritised if:

• they can be shown to be effective and fit for purpose

• AND GOOD VALUE FOR MONEY – i.e. the intervention is likely to be cost-effective (mainly through NICE)


Health economics can help us understand the costs and benefits of different interventions:

For example:• If the health benefit generated by a new intervention is worth the

additional cost

• If an intervention is more effective than its alternative, but its related costs are exorbitant, this intervention will represent poor value for money and not a good use of NHS resources

• If one intervention provides more health outcomes for the same level of cost this intervention should be adopted

UK research funders increasingly value applications that answer health economic questions

“Beagle 2 could have crashed into Mars because the atmosphere on the planet was less dense than expected”, explained Professor Pillinger. “The probe may have been going too fast for its parachute and airbags to bring about a soft landing.” But contrary to belief, the mission was not a failure. The technology used for the project could play a major part in managing the tuberculosis pandemic.

(Open University WebSite, March 2010)

Photograph: ESA

Planning a project

The people you needThe materials, equipment and space you need

The time you need

How you organise the above

Who?What?When?Where?

For how long?

Think about what will need to be done on a day to day basisThink about maximising benefits to all involved

The people you need

Photograph: NASA

Various roles, for example:

• Collaborators • Steering committee• Day to day project manager/ co-ordinator• Research assistants• Administrative staff• Support staff• Consultants• Data Safety Monitoring Board• Patients/ public

• Collaborators Bring the right skill mix to the research team Varying degrees of involvement

• Steering committee Overseeing the work, ensuring that the study happens as intended

• Day to day project manager/ co-ordinator Managing the study on a day to day basis

• Research assistants Data collection Collection and processing of samples

• Administrative staffAdministrative support

• Support staffPeople external to the study, but whose help may be needed.E.g. staff on wards, library, ethics committee, local data protection officer

• ConsultantsFor specialist help and adviceE.g. statisticians, economists, computing

• Data Safety Monitoring BoardIndependent ; periodically check on data being collected for patient safety purposes; broad-ranging monitoring of data quality and integrity

• Patients/ publicDifferent levels of involvementE.g. Consultation, collaboration, user control

Organising the personnel

• Who is in charge of what?

• Must be clearly designated at the start

• Everyone involved must be committed in terms of interest and time

Staff management: very important

Logistical issues in recruitment

Photograph: NASA

Staff

• Liaison with personnel • Development of job descriptions• Advertisements• Interviewing• Getting proper references

Sites

• Liaison with management• Liaison with health professionals• Information about the study, what it will entail for the study site, the information being collected etc.

Participants

• Look at throughput/ numbers to give you an idea about how long it will take • First contact, informed consent

Give yourself

time!

Recruiting Participants

• Clear recruitment plan and sampling strategy (note any key groups of patients likely to be excluded)

• Realistic timing - use local data and apply eligibility criteria, refusals, variation in caseload etc

• Make effort to ensure inclusion of minority / disadvantaged groups / range of patients across all socioeconomic

groups

– describe provision for overcoming

poor literacy and language barriers


Image: ddpavumbafreedigitalphotos.net


Photograph: NASA

The importance of communication

Before the start of the study

• Who should know about the study?• Who should be involved?

• Formal agreements (e.g. sponsorship, financial arrangements, data ownership and authorship)

• External communication (e.g. Websites, Press Office)

Examples of who needs to know:

• Trust R&D offices• Other senior managers• Ethics committee(s)• Clinicians caring for patients involved• Nursing staff if wards involved• Comprehensive Local Research Networks • Medical records officer• Clinic/ward clerks• GPs

On-going communication

•Clearly defined roles between researchers – ‘chain of command’, appropriate support

•To ensure data quality, reaching of project milestones•With ethics committee (any change in protocol, methods,

dilemmas etc.)•With participants: contact person for everyone, whenever required

(e.g. everyday, or ‘on-call’, as necessary)•Back-up information •With study sites and personnel: politics and facilitation•Clearly identified researchers on wards etc.•General information exchange •Query-answering•Trouble-shooting•Feedback and acknowledgement

Photograph: NASA

The other resources you need

Stationery

Printing

Postage

Travel

Tests

Equipment

Telephones

Miscellaneous, e.g. staff trainingPhotos: Wikimedia

Logistical considerations in study design

Photograph: NASA

• Can your protocol be operationalised as a guidance document for staff?

• Sample selection and monitoring recruitment

If applicable:

• Randomisation

• Arranging an intervention

Assigning responsibility and ensuring time for these

Logistical considerations in data collection & data management

Data collection on MarsPhotograph: NASA

Types of tools

• Measuring outcomes – tools need finding (N.B. permissions) or developing (N.B. validity, reliability)

• Lay and Non-English speaking issues• Collecting other information, e.g. process, administrative, contextual• Coding for the computer, development of database• Only collect what you need to answer the question

Logistics

• How will the data be collected: By whom? Where? When? • Holidays, bank holidays (postage)• Data checking and quality control• Confidentiality/ data protection issues• Procedures for reporting adverse events• Data management and storage (e.g. double blind entry, restricted access, back-

up, encryption etc)• Computing – coding and data entry• Data cleaning

Piloting

Photograph: NASA

The importance of piloting

Always pilot!

It checks that everything works:

• Management and personnel• Methods• Feasibility – technically, politically, logistically, methodologically• Training of fieldworkers• Sample• Timing and content of questionnaires/ interviews • Validity of tools and data sources (feedback from participants)• Overall time-scale• Data entry, cleaning and analysis

Logistical issues in data analysis

Cray 2 Supercomputer, NASAPhotograph: NASA

Planned before the study has started:

• Equipment (hardware, software)

• Statistical methods

• Statistics and computing support

• Other analysis issues (e.g. qualitative, health economic)

TIME

Pigeon postSource: Wikipedia (Public Domain)

(11) Case for Support – Part 3Dissemination, Outputs & Expertise

Routes for Outputs/ Dissemination

Via:

• Involved patients / lay members & local interest groups

• Research network meetings

• Targeted conferences for presentations / posters

• Targeted peer-reviewed journals - maximum readership within field

• Other media, as appropriate

• Use of workshops / training packages

• Guidelines which could be altered by the results

• Targeted groups of specialist / professional bodies

• Inclusion of results in existing meta-analyses / Cochrane reviews

• Members of the team involved in teaching

Not forgetting…

• Participant patients and staff• Report for funding body• Any reports for Ethics Committee

And logistically…

• Time• Personnel

And importantly…

• How will findings be translated into health care to produce patient benefits

Expertise: have you got your A-team?

1 ) Mad Scientist drawn by JJ, Image provided courtesy of Wikipedia. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.22) The School of Athens - fresco by Raffaello Sanzio. Image in the public domain supplied courtesy of Wikipedia.3) Marie Currie ca. 1898. http://www.mlahanas.de/Physics/Bios/MarieCurie.html Image in the public domain supplied courtesy of Wikipedia.4) This image is available from the United States Library of Congress's Prints and Photographs Division under the digital ID cph.3b46036. Image in the public domain supplied courtesy of Wikipedia.

1

3

2

4

Hypatia (Ὑπατία)

http://www.mlahanas.de/Physics/Bios/MarieCurie.html

Questions a funder will ask:

What are the specific roles of members of the research team?

Does the team:

• Have appropriate track records in research?

• Demonstrate appropriate skills mix - clinical and research experience?

• Include, where appropriate, a statistician, health economist?

• Include a PPI representative?

• Have genuine interest and time to commit to your research

Overall, why is this group of people best placed to do the work?

(12) Management and Governance

• Research timetable: milestones and deliverables

• Management arrangements

• Previous relevant work

• Measures of success, risks and how to manage them

• Ethical issues

Ethics

• Does the research involve specific areas with particular ethical implications?

• Ethics committee approval required?

• Has the proposal already been granted a favourable opinion from relevant regulatory authorities?


(13) Intellectual Property and Innovation

• Current IPR held by applicants and relevance to current application

• Has a search for existing IPR been undertaken?

• Related/ similar IPR which may affect freedom to operate?

• Will IPR be produced or improved through the research?

• Management of any IPR

• Competitive devices, practices and technologies and advantages over these of any likely to be produced through the research?

• Benefits of any likely IPR

• Regulatory considerations and risks to be considered for any new technology to reach the market?

(14) NIHR Infrastructure

• Links to NIHR networks - look at www.ukcrn.org.uk

• RDS input

• Involvement with other NIHR partners


http://www.ukcrn.org.uk/

Source: nuttakit, freedigitalphotos.nethttp://www.freedigitalphotos.net/images/


(15) Finances

Value for money

• Justification for major sources of expenditure, e.g. staff

• Justification for how research costs have been allocated

• Justification for how NHS Support and Treatment costs will be allocated.

The people you will need to involve

• The local NHS R&D Finance Officer of your employer

• If you are collaborating with other organisations, the appropriate Finance Officer in each one

• Your local Comprehensive Local Research Network (for any NHS Support Costs required)

Photo: renjith krishnan, www.freedigitalphotos.net, http://www.freedigitalphotos.net/images/view_photog.php?photogid=721


Always: Consult your Finance Office at an early stage

Be aware:

• That costs are broken down by project year• Of timings for university/ trust/ CLRN approval• Of which organisation is taking responsibility for the project – their

authorised signatory will have to sign off the form.• That if the study is a collaboration between organisations, a ‘system’ will

not necessarily exist to sort out respective costs• That the participation/ costs of other collaborators will also have to be

signed off by them• That, if successful, formal collaboration agreements will need to be in

place before funds are distributed

The RfPB Finance Form



Directly Incurred Costs

Q. What is a directly incurred cost?A. Costs of people/ other things that are being

employed/ used exclusively for the project

• Research, technical and other staff• Travel and subsistence• Equipment• Consumables• PPI• Other, e.g: Conference fees, room hire, publication, training



HEI Indirect Costs –Estates Charges

Q. What are HEI Indirect Costs – Estates Charges?For universities only, the estate costs for staff, i.e. physical environment required for general staff support.

• E.g. For Oxford University, for 2010/11, the estate charges, per full time member of staff per year, regardless of grade, are £3,367 (£12,020 for lab. (scientific) departments)



HEI Indirect Costs – Other Indirect Costs

Q. What are HEI indirect Costs –Other Indirect Costs?For universities only, (i) the overheads associated with employing someone, (ii) the costs of people already employed taking ‘time out’ for the purposes of the study (e.g. supervision), & (iii) university-based technical support (e.g. laboratory costs)

• E.g. For Oxford University, for 2010/11, the additional costs (or ‘overheads’) associated with employing a full time member of staff, regardless of grade, is £44,124.



• Seek advice from Finance Department

Commercial/ Other Partner Organisation Indirect Costs

Q. What are ‘Commercial/ Other partner Organisation Indirect Costs’?A. The costs of resources used by the research that are shared by other

activities.



NHS Overheads

E.g.

• Estates charges

• Telephone rental etc.

Q. What are ‘NHS Overheads’?A. Costs of resources used by a project, but shared

by other activities



NHS Support Costs

Q. What are ‘NHS support costs’?A. The additional patient care costs that result from the

research being run in an NHS organisation. These costs would end when the project ends.

• Extra patient tests• Extra in-patient days• Extra staff time, e.g. consultants or nurses taking consent, extra

attention due to the research

This funding would usually come via the Comprehensive Local Research Network, and is not included in the total of the grant

application figure



NHS Estimated Treatment Costs

Q. What are ‘estimated treatment costs’?A. The treatment costs that relate to the research

intervention.

• You are asked for information on the research treatment (intervention), and the normal standard treatment, so the additional costs for running the research intervention can be calculated.

• These costs would normally come from the NHS as part of normal patient care commissioning

Different treatment, or treatment administered in a new way, or a new place.



Costing people

• Researchers involved with the study (but already in post)• Researchers employed exclusively for the study, e.g.

experienced day to day study/ clinical trial manager/ co-ordinator/ primary researcher, research assistants

• Collaborators• Steering committee• Administrative staff• Support staff• Consultants• Data Safety Monitoring Board (DSMB) • Patients and public involved with research (PPI)



Salaries:

Regular staff (internal) Consultancy (external)

Basic salary (check scales)National insuranceSuperannuationIncrements/ pay risesIndirect costs (includingEstate costs)

Daily rateRate per sample etc

Have you considered dates for each major phase of the study, with staff responsible



Stationery E.g. How many questionnaires, how long, how many times; invitations; letters, information sheets, how to represent study to public/ patients

Printing E.g. How many questionnaires, how long, how many times; invitations; letters, information sheets

PostageE.g. How many mailings, follow-ups and reminders

Travel E.g. researcher to participant, participant to researcher, meetings to sites to set up and run project, dissemination, PPI, Steering Committee meetings

Costing other resources



TestsE.g. Clinical tests, laboratory tests

Equipment E.g. monitoring instruments, computing equipment (including software), desks, chairs

TelephonesE.g. Office and mobile; standard rates for institutions?

MiscellaneousE.g. Staff training, running costs of equipment, advertisements for staff, incentives (within ethics guidance)



• How the resources requested will be used

• How realistic and accurate the costings are

• Whether the ‘best price’ has been found for tests etc?

• Whether anything appears ‘over-budgeted’?

• Whether the requested resources are justified?

• Does the proposal represent the most efficient use of resources?

• Is the research good value for money?

Give yourself some leeway

Overall budget

Have you considered:



(16) Monitoring information

• Suggest three potential peer reviewers who have relevant expertise

• May or may not be used

(17) Suggested Reviewers

• Information used by DoH and NIHR to monitor and categorise applications

• RDS use and satisfaction with service

• Carbon guidelines

(18) Supporting Documentation

E.g.

• Flow diagram

• Gantt Chart

• Bibliography - ensure that all papers cited are listed clearly and logically

• Letters of support

Image in public domain courtesy of Wikimedia

The inclusion of a Gantt Chart is mandatory

31st FW Balkan Operations Gantt

chart. United States Air Force, 2000

(19) Declarations and Signatures

• Lead Applicant

• Host Organisation (finances)

• Host Organisation (Head of Department/ Senior Manager)

Give yourself TIME to get these!

renjith krishnan, freedigitalphotos.nethttp://www.freedigitalphotos.net/images/view_photog.php?photogid=721

Part 4 Top Tips:

Optimising your chances of success

(Inspired by RfPB feedback)

Make sure:

• Your study is in scope

• You submit your proposal on time

• It is complete!

• It is a ‘well-crafted’ proposal

• Your proposal is understandable to a diverse audience

• You have shown the relevance of the study to the NHS

• You have had adequate lay involvement

• You have the right skill mix

• You have made the necessary research partnerships

Make sure:

• You have demonstrated a need for the study, and your knowledge of the area

• Any necessary pilot work is undertaken or planned appropriately

• Interventions (e.g. for RCTs) are clearly described

• You have an appropriate study design

• Other aspects of the methods and procedures are clear

• You have consulted appropriate experts and incorporated their suggestions

• You have given yourself a realistic timetable

• Your study is sensibly funded

• Don’t be over-ambitious!

Any questions?

MANY THANKS!

Photo: Simon Howden, FreeDigitalPhotos.net http://www.freedigitalphotos.net/images/view_ph

otog.php?photogid=404 FreeDigitalPhotos.net staffFreeDigitalPhotos.net

Evaluation

masterclass in writing rfpb grant applications 28 th january 2011 carole fogg sophie hyndman helen...

Documents

health research

research funding

rfpb research

research protocol

research standards

research project

research capacity

affordable research