the family nurse partnership (fnp) programme developing the evaluation framework
Post on 02-Jan-2016
32 Views
Preview:
DESCRIPTION
TRANSCRIPT
FNP Programme
• It is an intensive nurse-led home visiting programme that enables the family nurse to visit the same client from early pregnancy until the child is two
• It is an intervention for young, first time mothers, who meet the broad eligibility criteria (19 and under at LMP, keeping their baby, living within geographical boundaries, <28 wks gestation at recruitment)
• Programme aims:Improve pregnancy outcomes Improve child developmentImprove the economic self-sufficiency of the family
FNP in UK• England since April 2007. Implemented in 55 LA/ PCT areas.• FNP is now being tested in Scotland for the first time in NHS
Lothian City of Edinburgh Community Health Partnership (Edinburgh CHP). First babies enrolled in March 2010.
• 2 teams in NHS Tayside, covering Dundee, Angus and Perth and Kinross CHP’s.
• England: Formative evaluation of the first ten sites reported 2011
• England: RCT in 18 sites which will report, initially, in 2013. The RCT will assess what the benefits and costs associated with FNP, looking closely at prenatal health behaviours, and early child health outcomes.
FNP EvaluationEvaluation Tem: Scottish Centre for Social Research
The overall aim: evaluate the implementation of the programme in Scotland (Lothian)
Specific questions:• Is the programme being implemented as intended? If not, why not?• How does the programme work in Scotland (Lothian)? • Wider implications for implementing the programme in Scotland.
Evaluation implementation: Monitoring and Evaluation Framework: Internal (FNP data) and External (stakeholder interviews, qualitative panel and focus groups) relevant to outcomes of interest in Scotland
How we developed M+E framework
• Series of focused meetings facilitated to achieve consensus on the what the programme was intending to achieve (outcomes) and how this was intended to operate (processes and assumptions)
• Produced two logic models: a Google Earth view and an implementation model
• These provided framework for the M+E
Assesstransferability
and effectivenessin Scotland
Dev't & maintenanceof a skilled FN team
Improvepregnancy and birthoutcomes
Home visits antentally
as per FNPschedule
Internal monitoringExternal evaluation
trainingsupervision
team meetingslearning sets
Improved nursing skills/practice e.g. in promoting attachment
self-efficacy& taking an ecological/PH approach
Enhanced understanding of FNP delivery
and effectiveness
Improvechild health
and dev't
Home visits (postnatally)
until child is 2 yrsas per FNPschedule
Improveparental life course
Improved child health & development
reduced A+E visits and hosp'n for injuries
redc'n in cases of neglectimproved HOME scores
responsivered'n in delays in language
+ cognitive devt
Better maternal healthReduced substance use
Improved maternal mental healthReduced PND
More health enhancing behaviours (diet, physical activity etc)
Parents engage in child health-enhancing behaviours
% of immunisations from 0-2 yearsbreastfeeding: initiation & maintenance
italics = outcomes listed in service level agreement
Enhanced infrastructure(in Edinburgh?)
to support vulnerable mothers
Reduced neonatal risk factorse.g. preterm deliveries, birthweight,
neurodevelopmental impairment,foetal alcohol syndrome
Improved adolescent outcomesred'n in child abuse/neglect (0-15)
fewer arrests + adjudication for bad conduct
Early childhoodfewer safety hazards in home
more stimulating home environreduced recorded unintentional injuries
improved school readiness/ pre-school language
fewer child beh probs within clinical range
Improved health behaviours in pregnancyReduced substance use (tobacco, alcohol and drug)
More health enhancing behaviours (diet, physical activity etc)
Improved maternal mental healthAppropriate uptake of preventative services
Parents demonstrate more competent care & improved parenting
e.g.good attachmentstimulating care and environs
safety in homeuse of other progs/community resources
involvement of Dads
Enhanced econ self sufficiencyno of months mums are working
Parental lifecourse (3-4 yrs after prog)fewer pregnancies
more space between 1st and 2nd pregnanciesno of months in workforce
less federal assistance/food stampshigher rates of living with father of child
higher rates of marriage
Enhanced parental life course (13 yrs after prog)
Less reliance on benefitsFewer arrests /
convictions/days in jailInc space between 1st & 2nd C'n
SHORT TERM OUTCOMES
LONG TERM OUTCOMES
[based on US trials]
INTERMEDIATE OUTCOMES[from US trials: 4-6 yr follow up]
PROGRAMME GOALS
ACTIVITIES
Preliminary logic model Preliminary logic model using EB and incorporating outcomes in service level agreementusing EB and incorporating outcomes in service level agreement
Improvepregnancy and birthoutcomes
Home visits antentally
as per FNPschedule
Improveparental
life course
Improvechild health
and dev't
Home visits (postnatally)
until child is 2 yrsas per FNPschedule
Reduced neonatal risk factorse.g. preterm deliveries, birthweight,
neurodevelopmental impairment,foetal alcohol syndrome
Dev't & maintenanceof a skilled FN team
Assesstransferability
and effectivenessin Scotland
trainingsupervision
team meetingslearning sets
Improved child health & development
reduced A+E visits and hosp'n for injuries
redc'n in cases of neglectimproved HOME scores
responsivered'n in delays in language
+ cognitive devt
Better maternal healthReduced substance use
Improved maternal mental healthMore health enhancing behaviours
(diet, physical activity etc)
Internal monitoringExternal evaluation
Improved nursing skills/practice e.g. in promoting attachment
self-efficacy& taking an ecological/PH approach
Enhanced understanding of FNP delivery
and effectiveness
Contribute toa Healthier,
Wealhier....Fairer Scotland
Contribute toRelevant National Outcomes ....
Cn have best start in lifeImproved life chances for children,
yp and familes at riskYp are successful learners
Better employment opportunitiesLonger, healthier lives
Tackled significant inequalitiesLive lives free from crime and danger
Public services are high quality, continually improving
& responsive to people's needs
....and most relevant HEAT TargetsH2: dental registrationsH3: healthy child weight
H4: Alcohol screeing and BIH6 Smoking cessation
H7: breastfeeding
Contribute to Edinburgh outcomes (as per SOA)
Edinburgh's children are healthyMental health and wellbeing is improved
Our children have the best startin life, are able to make + sustain relnships
and are ready to succeedChildren's early years' devt, learingand care experiences are improved
so that they are ready for school
black italics = outcomes listed in service level agreementBlue text = link to Scottish outcomes and/or programmes
Improved adolescent outcomesUS evidence indicates:
e.g. red'n in abuse/neglect reduced antisocial behaviour/crime
fewer arrests + adjudication for bad conduct
In additition, we anticipate:improved educational attainment
Cycle of deprivation interruptedOffspring themselves
have better parenting skills
Money saved by the state
Enhanced parental life course (13 yrs after prog)
Less reliance on benefitsFewer arrests /
convictions/days in jailBigger interval between
1st & 2nd C'n
Parental lifecourse US evidence indicates (3-4 years after prog):
fewer pregnanciesmore space between 1st and 2nd pregnancies
no of months in workforceless state assistance
higher rates of living with father of childhigher rates of marriage
In additition, we anticipate:better maternal /paternal mental health
Less domestic abuseBetter use of services
More accessing good quality child care
influence/inform thinking on the role/delivery of
community nursing
Enhanced infrastructurein Edinburghto support
vulnerable mothers
LONG TERM OUTCOMES and IMPACT 5+ years after end of intervention
INTERMEDIATE OUTCOMESup to 5 years after end of intervention
SHORT TERM OUTCOMESACTIVITIESPROGGOALS
Parents engage in child health-enhancing behaviours
% of immunisations from 0-2 yearse.g. breastfeeding: [H7]
Registration with dentist? [H2]Better weaning practices/diet [link to H3]
Uptake of Healthy StartImproved use of community services
Improved health behaviours in pregnancyReduced substance use (tobacco, alcohol and drug)
Better diet, more PAImproved maternal mental health
Appropriate uptake of preventative servicesUptake of screening services (CEL 31)
Use of Vit D supplements & folic acid (CEL 36)
Parents demonstrate more competent care & improved parenting
e.g.good attachmentstimulating care and environs
safety in homeuse of other progs/comm'ty resources
involvement of Dads
Enhanced econ self sufficiency
Early childhoodUS evidence indicates:
fewer safety hazards in homemore stimulating home environ
reduced recorded unintentional injuriesimproved school readiness/
pre-school languagefewer child beh probs
In addition, we antcipate:Less dental disease at P1
Inc fluoride varnish applicationsLower BMI
Better mental healthImproved relationship
between child and mother
FNP: High level strategic model
supportive local (HB, LA, CHP)and national (SG)
infrastructure
Office and ITsystems & staffto manage data/
monitor FNP
Recruitment & capacity building of team
eg. training,supervision, learning sets
awareness raising /liaison with agencies/services
incl health care providers
prod'n of reports of monitoring data(specify timing/frequency)
Skilled team
Better pregnancy outcomes
Improvedchild health
+ devt
Improved parental
life course
processing/maintaining
info andsystems
ACTIVITIESINPUTS SHORT TERM OUTCOMESOUTPUTS
Programme budgetcirca £1.6m over 3 years
all requisite trainingattended by FN team [1]
weekly supervisionof FNs [2]
team of 6 FNs,supervisor & support team
with FNP competencies [3]
145 women enrolledwho meet eligibility
criteria [8]
FNs have caseload of
max 25 families [9]clients receive at least 80% of weekly visits for 1st month
following enrolmentthen alternate weeks until birth [11]
clients receive at least 65%of weekly home vists
for first 6 weeks post partum [12]
Clients receiveat least 65% of home vistis on alternate weeks
from age 6 weeks to 21 months [13]
monthly home vists between 21months
and 2 years [14]Implementation records
and all assessment forms completed [R1]
all monitoring data entered onto ITsystems [R2]
Client understands how & demonstrates
sensitive/competentcare of child
Client understands how to keep child safe
and creates a safe & stimulating home environ
Client understandspotential role of others
in supporting her, and mobilies this support
Client becomes aware of community resources
and accesses these
Client has improved knowledge/ behaviours
in prenatal health
discuss personal health35-40% during pregnancy14-20% during infancy,
10-15% during toddlerhood [15]
discuss maternal role23-25% during pregnancy45-50% during infancy,
40-45% during toddlerhood [16]
discuss lifecourse devt10-15% during pregnancy10-15% during infancy,
18-20% during toddlerhood [17]
discuss environ health5-7% during pregnancy7-10% during infancy,
7-10% during toddlerhood [18]
discuss family + friends10-15% during pregnancy10-15% during infancy,
10-15% in toddlerhood [19]
discuss comm'ty resources% not specified
in licensing req'ts (2-8) [20]
Client plans for futureand achieves goals
re education, employmentand future pregnancies
FN visitsscheduled/structured
as per fidelity requirements [10]
Developing/agreeingreferral pathways
Eligible women recruited by 28th week(60% by 16th week) [6]
75% of those offered programme are enrolled [7]Prod'n of
clear eligibility criteria & referral pathways [4]
Provision of info on eligibility
criteria to key agenciesworking with TG [5]
FNP: Implementation model
FNP: Embedded implementation model
Improved parental
life course
FN visitsscheduled/structured
as per fidelity requirements
discuss lifecourse devt10-15% during pregnancy10-15% during infancy,
18-20% during toddlerhood
discuss comm'ty resources% not specified
in licensing req'ts (2-8)
discuss family + friends10-15% during pregnancy10-15% during infancy,
10-15% during toddlerhood
discuss maternal role23-25% during pregnancy45-50% during infancy,
40-45% during toddlerhood
discuss personal health35-40% during pregnancy14-20% during infancy,
10-15% during toddlerhood
discuss environ health5-7% during pregnancy7-10% during infancy,
7-10% during toddlerhood
Client understands how to
keep child safeand creates a safe
& stimulating home environ
Client becomes aware of community resources
and accesses these
Client plans for futureand achieves goals
re education, employmentand future pregnancies
145 women enrolled
who meet eligibilitycriteria
OUTPUTS SHORT TERM OUTCOMES
Intendeddomain outcomes
(general)
Client has improved health knowledge
and behaviours
Client understands & demonstrates
sensitive/competentcare of child
Uptake of screening services
(CEL 31) [21]
Attends antental appointmentsand classes [22] more natural births [25]
improved knowledgeof health behaviours
and impact on child [26]
Mother taking good care of selfe.g. reduced use alcohol,
tobacco,cannabisgood diet [27]
Better pregnancy outcomes
[28]
knows principles of 'good parenting' [30]
better infant mental health [34]
fewer hazards in homemore safe practices
e.g. use of safety equipment[38]
availability/use of books, toys etc [39]
client engages inhelp seeking behsto avert crisis [41]
involvementof Dads,
other family membersand friends [43]
reduced domestic abuse
[42]
Client understandspotential role of others from personal network
in supporting her, and mobilies this support fewer accidents [40]
More infant HI practices e.g:b-feeding initiat'n
& maintenance (H7); weaning practices (H3)
tooth brushing [29]
decreased child maltreatment [35]
confidence in parenting role [31]
+ve parentingpractices [32]
good bonding
/ attachment [33]
well supportedmother [44]
less anxiety,depression, PND
in mums [45]
referrals to agencies/sources of support [46] use of community
supports/ resources [47] greater interval between pregnanciesfewer unplanned pregnancies
mums know what they want to achievemums on path to meet their goals [48]
Improvedchild health
+ devte.g thriving babies [49]
better prepared for birth [24]
stimulated, alert and responsive
babiesgood language dev't [36]
posseses child safetyknowledge
knows age appopriate ways to stimulate child [37]
uptake of Healthy Start
and use of Vit D [23]
From Having to Using……
We used the logic models to frame monitoring and evaluation of the programme by: – Prioritising key outcomes and assumptions of interest– Prioritise key questions addressing above – Agreeing who would collect and analyse which data, when
and how• Decisions underpinned by considerations of feasibility,
acceptability and data robustness (including how to improve these)
Box code (from logic model) 3 and links to 1 and 2
Logic If the team attend training and are supervised, then they will possess requisite competencies
Question Does team receive the training & support intended & develop req’d knowledge/ skills?
Indicator(s) Who’ll collect data?
Is this a fidelity req’t?
Who’ll analyse?
Any additional considerations?
Proportion of team attending each mandatory course*
% of learning events run*
Self reports of satisfaction and perceived utility/effectiveness of training and learning events.
Frequency of supervision sessions*e.g. for each FN, no of weeks per quarter that timetabled supervision takes place, expressed as a proportion of working/available weeks ie excludes sickness absence
% of required accompanied visits that take place*e.g. no of accompanied visits per FN per 4 months and % of FNs who receive min quota of accompanied visits every 4 months.
Self-reports of feasibility of roles and competence to deliver it (based on job spec and on practice); also whether/how psychologist support worked in practice
Internal
Internal
External (interviews with all FNs and supervisor)
Internal
Internal
External (interviews with FNP)
FR: Attendance at 4 residential training courses
FR: supervisor runs pre and post learning events
FR for weekly supervision
FR for each FN to be accompanied at least once every 4 months
FNP
FNP
ScotCen
FNP
FNP
ScotCen
It will be imperative that the FNP builds in a process of regular review (every 3 months?)in order to address any shortfalls in delivery req’ts.
There would need to be some agreement about what qualifies as supervision e.g. a quick catch up in the corridor?All measures on this page will require good record keeping, submission of records to the administrator within a workable timescaleWe have assumed that the FN’s and supervisor meet person spec in job desc and so this does not need monitored
Box code (from logic model) n/a
Assumption Attrition will be low
Logic The project is only viable if most families participate ANDIf families find the support useful, they will stay engaged
Question Does project meet the fidelity targets for attrition?
Indicator(s) Who will collect data: internal or external evaluation?
Is thisa fidelity requirement/goal?
Who’ll analyse the data in the first instance?
Any additional considerations?
Percentage leaving/dropping out of programme*Calculated as total no having left the programme divided by no enrolled.Implement programme alerts at monthly intervals if feasible.
Internal via UK004B
FR/G:Cumulative prog attrition is 40% or less thro to the child’s 2nd birthdayand is10% or less during pregnancy…..
FNPBut included in ScotCen reports
Box code (from logic model) n/a
Assumption Attrition will be low
Logic The project is only viable if most families participate ANDIf families find the support useful, they will stay engaged
Question Does project meet the fidelity targets for attrition?
Indicator(s) Who will collect data: internal or external evaluation?
Is thisa fidelity requirement/goal?
Who’ll analyse the data in the first instance?
Any additional considerations?
Percentage leaving/dropping out of programme*Calculated as total no having left the programme divided by no enrolled.Implement programme alerts at monthly intervals if feasible.
Acceptability and perceived utility of FN support
Internal via UK004B
External viainterviews with clients/familiesIncl (if possible) a sample of those who drop out/leave)
FR/G:Cumulative prog attrition is 40% or less thro to the child’s 2nd birthdayand is10% or less during pregnancy…..
FNPBut included in ScotCen reports
Form UK004B makes no provision for client leaving because they did not like the programme. Suggest that some extra fields are added to cover broader range of possible reasons for leaving
The M+E framework in action…Outcome: Mother takes good care of self
Logic: If mother takes good care of self, the risk factors for the infant are reduced
Question: Is there evidence that the FNP results in improved knowledge /health behaviours in clients prior to/following birth of baby?
Indicator: Clients’ accounts of what they have learned about risk/protective factors
Topic guide: Have you and your family nurse talked about smoking? What about drinking alcohol? Taking drugs? The food you should eat or not eat during your pregnancy? Keeping the baby safe?
Interview responses
Well I, to be honest I already knew about like drinking alcohol and taking drugs but I never knew about the smoking thing because my gran smoked with all her three kids while she was pregnant and my gran keeps on saying that later on they were fine but..
…because during my past pregnancy I was actually getting ready to drink alcohol again and she convinced me not to because I make a good home for
the baby and the baby’s depending on me and stuff like that
Application
• Scottish context: ensures evaluation relevant to outcomes of interest in Scotland
• Evaluation tools: Identifies data collection relevant to outcomes of interest
• Wider work: informed NHS Lothian on wider maternity services work
Ist report: intake and early pregnancyhttp://www.scotland.gov.uk/Publications/2011/07/28142203/0
Contact: Vikki Milne, victoria.milne@scotland.gsi.gov.uk
top related