open access protocol understanding repeated non-attendance … · missed appointments. from their...

Understanding repeated non-attendancein health services: a pilot analysisof administrative data and full studyprotocol for a national retrospectivecohort

Andrea E Williamson,1 David A Ellis,2 Philip Wilson,3 Ross McQueenie,1

Alex McConnachie4

To cite: Williamson AE,Ellis DA, Wilson P, et al.Understanding repeated non-attendance in health services:a pilot analysisof administrative data and fullstudy protocol for a nationalretrospective cohort. BMJOpen 2017;7:e014120.doi:10.1136/bmjopen-2016-014120

▸ Prepublication history andadditional material isavailable. To view please visitthe journal (http://dx.doi.org/10.1136/bmjopen-2016-014120).

Received 1 September 2016Revised 14 December 2016Accepted 20 January 2017

For numbered affiliations seeend of article.

Correspondence toAndrea E Williamson;[email protected]

ABSTRACTIntroduction: Understanding the causes of lowengagement in healthcare is a pre-requisite forimproving health services’ contribution to tacklinghealth inequalities. Low engagement includes missinghealthcare appointments. Serially (having a pattern of)missing general practice (GP) appointments mayprovide a risk marker for vulnerability and poorerhealth outcomes.Methods and analysis: A proof of concept pilotusing GP appointment data and a focus group withGPs informed the development of missedappointment categories: patients can be classifiedbased on the number of appointments missed eachyear. The full study, using a retrospective cohortdesign, will link routine health service and educationdata to determine the relationship between GPappointment attendance, health outcomes, healthcareusage, preventive health activity and socialcircumstances taking a life course approach and usingdata from the whole journey in the National HealthService (NHS) healthcare. 172 practices will berecruited (∼900 000 patients) across Scotland. Thestatistical analysis will focus on 2 key areas: factorsthat predict patients who serially miss appointments,and serial missed appointments as a predictor offuture patient outcomes. Regression models will helpunderstand how missed appointment patterns areassociated with patient and practice characteristics.We shall identify key factors associated with serialmissed appointments and potential interactions thatmight predict them.Ethics and dissemination: The results of the projectwill inform debates concerning how best to reducenon-attendance and increase patient engagementwithin healthcare systems. Significant non-academicbeneficiaries include governments, policymakers andmedical practitioners. Results will be disseminated viaa combination of academic outputs (papers,conferences), social media and through collaborativepublic health/policy fora.

INTRODUCTIONTackling health inequalities is a global healthpriority1 and for health service provision tohave an effective role, we should understandbetter the reasons behind, risks associated withand needs of patients who do not engageeffectively with healthcare provision (even if itis free at the point of access); and tailor ser-vices better to meet those needs. Thereremains a lack of published work concerningrepeated missed appointments, but previousresearch typically focuses on the financial costsassociated with non-attendance. One estimatehas placed the cost of missed UK general prac-tice (GP; community-based family medicine)appointments at £150 million per year.2

Recent Scottish government data suggest thateach missed hospital outpatient appointmentcosts National Health Services (NHS) Scotland£120.3 International data on costs to health-care systems are sparse. In a complex adaptive

Strengths and limitations of this study

▪ This study will answer important questions relat-ing to the health service component of tacklinghealth inequalities.

▪ A large data set enables the researchers to followthe patients’ journey across the whole healthcaresystem.

▪ The study uses data security and linkagecapabilities in a sensitive and robust manner.

▪ The study has a clear yet flexible data analysisplan using the expertise of a multidisciplinaryresearch team.

▪ There are limitations of using administrative datafrom a range of data sources of variable dataquality.

Williamson AE, et al. BMJ Open 2017;7:e014120. doi:10.1136/bmjopen-2016-014120 1

Open Access Protocol

on May 29, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2016-014120 on 14 F

ebruary 2017. Dow

nloaded from

http://dx.doi.org/10.1136/bmjopen-2016-014120



http://crossmark.crossref.org/dialog/?doi=10.1136/bmjopen-2016-014120&domain=pdf&date_stamp=2017-02-13

http://bmjopen.bmj.com

http://bmjopen.bmj.com/

system such as healthcare, the financial costs are contest-able because clinicians will ‘catch up’ or get on with othercare or administrative tasks. What is important are thecosts of opportunities missed for improving patients’health and the potential for substantial long-term savingsto health systems.Until now, studies investigating missed appointments

have focused on single missed appointments or singledisease areas and have indicated that they are associatedwith poorer health outcomes.3–6 Studies of single missedappointments have produced conflicting results when itcomes to designing effective interventions that canincrease attendance.7–10 This may be due to a relianceon small samples in disparate settings11–15 and confla-tion of patients who occasionally miss appointments withpatients who have an established pattern of missingmany.The Health and Social Care Information Centre in

England has recently published data about repeatedmissed appointments. From their analysis of recordedmissed outpatient hospital appointments in England, 1in 50 patients (65 590 of 3.5 million) who missed anappointment failed to attend three or more furtherappointments within 3 months.16

We hypothesise that repeated missed appointmentsreflect a pattern of behaviour. We use the term ‘seri-ally’ missing appointments to reflect this pattern,which may be interrupted by attended appointments.Clinicians do report that patients who serially missappointments (SMA) are of particular concern becausethey may have very poor health, may be socially disad-vantaged and are high users of unscheduled care com-pared with patients who occasionally or never missappointments.17

There is accumulating evidence that negative experi-ences in early life have pervasive consequences forhealth over the life course including ‘extensive evidenceof a strong link between early adversity and a wide rangeof health-threatening behaviours’.18 This body of worktherefore provides a conceptual framework for betterunderstanding ‘chaotic lives’19 as an explanatory factorin health usage behaviours such as missed appoint-ments, and introduces the possibility that serial missedappointments contribute to the inverse care law, whichstates that healthcare provision is least likely to be pro-vided to those who need it most.20

In the UK, publicly funded GP provides almost univer-sal coverage for the population and generates around90% of health contacts. Appointment making is typicallyunder the control of each patient directly. GP appoint-ments therefore provide a sensible starting point for thisstudy of health and other outcomes across patients’ lifecourse. Subsequent results will also have relevance forglobal health systems where patients have direct accessto a wider range of healthcare specialties.Scotland has an established data linkage infrastructure

which is under continuous development. This path-finder study will, for the first time, link large GP data

sets (including appointment data) with data from acrosspatients’ whole journey through healthcare.The overarching study question is: is serially missing

GP appointments a risk marker for vulnerability andpoorer health outcomes and thus a useful target fordeveloping interventions to improve engagement inhealth promoting care across the health system?

Aim and research questionsThe overall aim of the study is to determine the relation-ship between GP appointment attendance, healthcareusage, preventive health activity, health outcomes andsocial circumstances taking a life course approach andusing extracted health service and other relevantadministrative data.A pilot study sought to answer the first research ques-

tion described below (figure 1). The subsequent ques-tions underpin the full research protocol whichcompares cohorts of Scottish patients (from birth toolder age) who never, occasionally and serially miss GPappointments.An introduction to the full study protocol is described,

followed by the methods and results from a mixed-methods pilot study that informed the protocol. Adescription of protocol participants, data sources, vari-ables and statistical analysis then follows.

METHODS AND ANALYSISThe full study protocol is for a retrospective cohort studyof GP patient records linked with secondary care andeducation administrative records in Scotland.The study started in July 2015 and will finish in

December 2017. A pilot study was conducted betweenJuly and September 2015 which is described next. The

Figure 1 Study research questions.

2 Williamson AE, et al. BMJ Open 2017;7:e014120. doi:10.1136/bmjopen-2016-014120

Open Access



jopen.bmj.com

/B

MJ O


ebruary 2017. Dow

nloaded from


cohort of 909 073 GP patient records for the full studywas available in the National Safehaven from September2016 and analysis of these data is underway. Permissionto access education data is secured, and the outcome oflinkage permissions for health data is not yet confirmed.

Pilot studyThe pilot study was separated into two subsections: afocus group to inform and refine definition develop-ment (research question 1) and a ‘proof of concept’quantitative data analysis.

METHODSFocus groupA focus group was conducted in September 2015 withfive GP participants. A focus group was judged the mostappropriate method to use because we aimed topromote discussion of the topic such that participantswould be able to compare and contrast their ownexperiences with others from a range of practice andprofessional experience settings.21 Linked to this wasthe aim of asking participants to make sense of andprovide feedback on the presented pilot data. The GPswere a convenience, purposive sample based on twomain principles. The first took into account the evi-dence surrounding single missed appointments. Thisdescribes missed appointments being more common indeprived urban practices. The sample thereforeincluded GPs who worked in deprived and affluenturban areas and a practice with a significant rural prac-tice population from Scotland. Second, the sampleincluded the views of frontline GPs and GPs who had arange of strategic roles in practice development and GPmanagement, locally and nationally. AEW and PW usedtheir professional knowledge of GP networks and prac-tice profiles to approach and recruit participants. Fiveout of 12 GPs contacted were able to attend the focusgroup. Each GP contacted reported that they felt thiswas an important topic worthy of attention. Barriers toattending were location of the focus group (conductedin Glasgow) and managing time away from other profes-sional work. Online supplementary additional file 1describes each participant’s characteristics. Detailedinformation about participants’ practice characteristicswas not collected. Three of the participants knew eachother from their professional roles outside of clinicalpractice. AEW conducted the focus group and theanalysis was conducted using Framework Analysis.

Framework Analysis is a useful thematic analysisapproach, especially when considering a focused topiclike this one. Also in the context of being part of alarger mixed-methods study, epistemologically, its usewas a good fit.22 DAE attended the focus group and pre-sented initial results from the ‘proof of concept’ pilot(described next) for discussion. Online supplementaryadditional file 2 describes the topics covered in thefocus group.

Proof of conceptResearch that uses GP appointment data has not previ-ously been conducted using the clinical recordingsystems in the Scottish NHS. A proof of concept pilotstudy was undertaken using the NHS Trusted ThirdParty (TTP) Albasoft with 67 705 patient records todetermine whether retrieving appointment data wasfeasible, to refine other data parameters and to informthe definition development as described in researchquestion 1. An additional confidentiality controlensured that the research team did not know the iden-tity of the recruited GP practices.Online supplementary additional file 3 describes the

definition and role of TTPs.Albasoft purposively recruited 10 Scottish practices on

our behalf with the practice characteristics illustrated infigure 2.Data were cleaned and appointment rules applied to

categorise appointments as attended or missed (Did NotAttend, DNA). Online supplementary additional file 4describes this process. This was primarily based on the‘in’ and ‘out’ time recorded for each appointment. Ifthis was recorded as ‘0’, then the appointment was classi-fied as DNA. For each patient, the total number ofappointments made during the 3-year period was calcu-lated, as well as the number and percentage of appoint-ments missed. Appointments that were recordedincorrectly in the system were removed at this stage.These included appointments where administrativerecords had remained open for over 24 hours, making itdifficult to confirm that these were genuine appoint-ments and not administrative errors. The pilot appoint-ment rules are set out in table 1 below.

RESULTSFocus groupFocus group participants reported making clear distinc-tions between patients who occasionally miss

Figure 2 Pilot practice

recruitment.


Open Access



jopen.bmj.com

/B

MJ O


ebruary 2017. Dow

nloaded from







appointments and those who miss many. Patients whooccasionally miss appointments do so because a crisis oranother understandable circumstance has arisen;patients who SMA, described as missing more than twoor three appointments, can be easily identified by GPs.They were described as tending to have mental health,addiction and/or social issues. They were described ashigh risk or vulnerable with concerns about their widerfamily. Patients who SMA were viewed as being differentfrom the general GP population and being morelikely to have lifestyles associated with housing instability,money problems and a panicked lifestyle (P2). Patientswho SMA were also described as being unable tomanage GPs’ expectation of them and fit into GPs’ pre-determined slots. “there’s the occasional DNA which arequite normal and often those are quite significant [intotal numbers for the practice] but the serial people Ithink that’s a reflection of the chaos in their lifewhether that’s you know- mental health or issues withthe social functioning- and inability to manage ourexpectation of them- to fit into our pre-defined slots”(P5).All participants agreed with that view. However, one

participant also considered that not all patients whoSMA can be viewed as high risk; that instead somepatients do not value free healthcare. It was reportedthat some patients who SMA go on to book anotherappointment the next day; “I don’t think it’s the valueof the GP- I think it’s the value of that appointment-I think the fact that it’s, if you don’t miss it, ifyou miss it is no big deal you just make anotherone” (P4).Missed appointments were viewed as being more

prevalent in practices in deprived settings, but occurredin affluent areas too. In the affluent setting, they wereimportant for a minority of marginalised, isolatedpatients with the same profile as described above whowere viewed as living ‘chaotic’ lives.Practices represented in the focus group do not have

protocols for managing patients who SMA becauseresponse is dependent on the patient’s context. GPsunderstood that SMAs usually mean patients withcomplex needs with workload implications for the prac-tice. Strategies described were varied, including allowingpatients only to book on the day; “my impression is thatdeprived practices have a much higher percentage of onthe day appointments because they skew it towards

people that don’t attend” (P3), seeing the patient whenthey walk in, or the GP booking the follow-up appoint-ment for the patient—a relationship building strategy.This could still lead to patients missing an appointment,even just a couple of hours after it was made. It wasreported that some practices do remove patients fromtheir list for SMA and this created tension with otherpractices.The focus group was also asked to comment on the

results from the proof of concept initial data and theymade recommendations about the full study designdescribed in figure 3.

Proof of conceptA pilot analysis of 67 705 patient records showed thatwhile just over 60% of our sample missed no appoint-ments, over 30% missed one or more appointmentduring the 3-year period with nearly 10% of patientsmissing three or more appointments.Assuming that our final sample provides a similar distri-bution, we will classify patients based on the number ofappointments missed as follows:▸ Never missed appointments: 0 per year average over a

3-year period.▸ Low missed appointments: <1 per year average over a

3-year period.▸ Medium missed appointments: 1–2 per year average

over a 3-year period.▸ High missed appointments: >2 per year average over a

3-year period.Our sampling both in the pilot data stage and the

final full study sample was conducted such that we werelikely to get a representative sample of Scottish patientsand practices. Since our pilot sample was large, it isappropriate to assume that this will scale up accordinglyfor the full study. The distribution of missed appoint-ments also suggested useful categories based on integernumbers of missed appointments per year. This will behelpful for policy and clinical stakeholders.

FULL STUDY PROTOCOLParticipants and study sizeOur target recruitment of GP practices seeks to ensurethat a spread of urban and rural practices, affluentand practices characterised by serving areas of blankethigh socioeconomic (Deep End) deprivation. Theinformation request made to practices can be viewedin figure 4.

Data sources and variablesGP dataThe TTP has recruited the practices on the study team’sbehalf and will undertake some specific data aggregationbefore transferring the data securely to the NationalSafehaven for analysis. ‘Safehavens are specialised,secure environments supported by trained, specialiststaff where data in electronic patient records can be

Table 1 Rules to identify genuine appointments

Data description Reason for removal

Total appointment time

<0 min

Record open for more than

24 hours

Total waiting time <0 min Record open for more than

24 hours

Appointment <2 min Not a medical appointment

Administrator slot Not a medical appointment


Open Access



jopen.bmj.com

/B

MJ O


ebruary 2017. Dow

nloaded from


processed and linked with other health data (and/ornon-health-related data) and made available for analysisto facilitate research while protecting patient identityand privacy’.23 These are: calculating urban rural classifi-cation, Scottish Index of Multiple Deprivation (SIMD)decile, categorising ethnicity into ‘non-black andminority ethnicity (BME)’, ‘visibly BME’, and ‘non-visible BME’ and rounding travel distance to practice/emergency department for each patient record to thenearest kilometre. Once in a Safehaven, additional stepswill be taken to ensure that acceptable anonymisationprinciples are being applied, especially in relation toreporting of sensitive social vulnerability data and report-ing of rare conditions.A new data file containing the appointment history for

each patient record will be generated, which will bemerged with individual patient information (onlinesupplementary additional file 4 describes this processbased on our pilot data set).

Appointment validation and categorisationEach appointment will be coded based on session typerecorded by the practice (eg, book on day appoint-ments, or immunisation clinic) and then further by pro-fessional type (eg, GP partner, practice nurse). Thesedescriptions are determined by individual practices, socategorisation will be conducted by the GPs in theresearch team. The appointment rules set out in thepilot study will be applied. A sensitivity analysis based onthe time the appointment takes will then also be con-ducted by comparing a random sample of patientappointments as described in figure 5.The appointment rules will be refined based on this.

The time interval cut-off for apparently attendedappointments will be determined by using the timeinterval that most accurately matches to actual attendedappointments. Slots designated non-face-to-faceappointments will then be removed leaving onlyattended and non-attended face-to-face appointments.

Figure 3 Focus group

recommendations for the full

study design.


Open Access



jopen.bmj.com

/B

MJ O


ebruary 2017. Dow

nloaded from



The appointment categories described from the pilotstudy regarding non-attendance for all patients willthen be applied to the yearly average number ofmissed appointments over the 3-year period to generatethe four categories of patients for further analysis.Using an average over 3 years takes account of what isrecognised in the frequent attenders (rather than non-attenders) literature that patients’ appointment behav-iour may vary over time in relation to illness episodesor social crises.24

Health and education data linkageLinkage will be conducted as access permissions anddata sets become available (figure 6). Each administra-tive data source is available for different time periods(eg, hospital inpatients since 1981 and education out-comes since 2002) and this will be made explicit wheninterpreting the results. The TTP will provide theSafehaven indexing team with a file containing the GPdata set Community Health Index (CHI) number and

other patient identifiers. Every patient in the ScottishNHS has a CHI number, a unique identifier that is usedas such across all NHS in Scotland. This forms thecohort for the study. All data providers will supply identi-fiers to be probability matched to the study cohort bythe Safehaven linkage team (based on the CHI numberand using other patient identifiers probabilistically forthe small number of records where it is anticipated thatCHI will be missing), who will return a set of uniqueindex numbers for those individuals successfullymatched to the study cohort; each data provider willreceive a different set of unique index numbers, and willuse these index numbers as the basis of their dataextract. Each data extract will be submitted to theSafehaven linkage team, which will replace the differentindex numbers with a common number across all files.

Figure 4 Information request

sent to target practices.

Figure 5 Random sample of GP appointments for validation

and sensitivity analysis.

Figure 6 Proposed data sets for linkage with GP data. A&E,

accident and emergency; GP, general practitioner; NHS,

National Health Service; SMR, Scottish Morbidity Record.


Open Access



jopen.bmj.com

/B

MJ O


ebruary 2017. Dow

nloaded from


This common number is the unique patient identifierthat the research team will work from during analysis.

BiasAccounting for patient turnoverThis study seeks to ensure the inclusion of patientswho are marginalised and who are often missing fromhealth service studies. There is evidence of overlapbetween patients who miss appointments and thosewho are removed from practice lists,25 a recognition ofthe impact that geographical boundary areas have onpatients who move around,26 notwithstanding the gapin the literature about registration interruptions forpatients who may go to prison or patients who remainunregistered once they are removed from GP practicelists. We will therefore summarise the numbers ofpatients joining and/or leaving their practice duringthe study period, with reasons where this informationis available. We will seek to provide a full analysis ofthe data available for these patients and comparethese with the patients who are registered for the3-year study period. Patients who are not registeredwith participating practices, and are being seen as‘temporary residents’ by these practices, are excludedfrom the study. This is because these patients’ full clin-ical record is held by their registered GP, so verylimited information is available. Temporary residentstend to be people on holiday in the practice area butwill include some people who would be consideredmarginalised.

Statistical methodsOur statistical analysis is based on the study being aretrospective cohort study. We will focus on two keyareas: predictors of high rates of serial missed appoint-ments, and serial missed appointments as a predictor offuture patient outcomes.Patient characteristics and practice characteristics may

be associated with high rates of serial missed appoint-ments. Analyses will initially be descriptive, summarisingthe rate of missed appointments in relation to the otherfactors recorded at the point of entry to the study.Associations with patient characteristics will be assessedas a whole, and in relation to different types of practices(eg, separately in rural and urban practices).Subsequently, we will build regression models (Poissonor negative binomial),27 to help understand how our cat-egories of missed appointments are associated withpatient and practice characteristics.When considering other outcomes in relation to serial

missed appointments, the missed appointment rate cat-egory (none, <1, 1–2 or >2 per year) will be the pre-dictor variable. Appropriate regression models,according to the outcome, will be used to assess whetherany associations with serial missed appointment rates areindependent of other patient-level or practice-levelfactors. Conflicting interactions will be controlled for byusing an ‘offset term’ in our negative binomial model

which accounts for number of appointments made orany other relevant factors.We also plan to measure relevant quantitative variables

(described next) recorded during the study interval asso-ciated with having a lot of missed appointments. We willexplore whether these differ from the predictive factorsalready recorded at entry to the study.

Quantitative variablesThe following potential predictors of frequent non-attendance will be analysed:

Demographics;Patients’ age, gender, minority ethnic group status(where available), deprivation decile, rural/urbansplit, number of address moves, distance lived fromGP practice and distance from nearest accident andemergency (A&E) will be explored;Health conditions.

Health conditions will be reported using separatecategories:1. The incidence of multimorbidity calculated from

extracted Read codes based on previous counts inScotland.28

2. Descriptions of health conditions based on the prior-ity 1 Read codes that GP practices in Scotland use topopulate patients’ key information summaries for GPout of hours services. This is novel work as a codingstructure has not previously been applied to theseRead codes. Read codes are the clinical codingsystem used in UK GP to record clinical and adminis-trative activity and diagnoses.

3. A count of psychotropic medicine prescriptionsbased on the British National Formulary. This is inorder to describe levels of psychological morbiditythat are not captured by diagnostic criteria.

4. These variables will then be compared with theInternational Classification of Diseases (ICD) 10coding data from patients’ secondary care linked datacompiled from hospital admissions and outpatientattendances. Diagnostic data from emergency depart-ment attendance was deemed not of sufficient qualityto use.

Social vulnerabilityOne aspect of this study which is particularly ground-breaking is our investigation of retrievable informationabout patients’ social vulnerability. The AdverseChildhood Experiences (ACE) questionnaire29 will beused as a template to match its nine descriptors of adver-sity to relevant Read codes in the patient’s GP record. Inaddition, coding that maps the consequences of ACE willbe analysed. A recent quantitative evaluation of Severeand Multiple Disadvantage will also be matched to GPRead codes. This examines the overlap of patients beinghomeless, in substance misuse services or in prison overthe preceding year.30 Further, an exploration of add-itional Read codes that describe social vulnerability willbe mapped. An anonymised text search linked to Read


Open Access



jopen.bmj.com

/B

MJ O


ebruary 2017. Dow

nloaded from


codes from the data set will provide additional informa-tion about social vulnerability as it is recorded in the free-text portion of GP records. Taken together, these willprovide the first research evidence about the breadthand depth of social vulnerability recording by GPs.

Healthcare usageRead coding in relation to cervical, breast and bowelscreening attendance will be retrieved in addition to theproportion of patients who have had their blood pressurechecked and have participated in child health surveil-lance and vaccination programmes across the life course.A subanalysis of usage of practice nurse and other health-care professionals’ appointments in the practice will alsobe conducted and include an exploration of the relation-ship between attending all primary care appointmentsand categories of non-attendance. This is because datafrom the GP focus group suggested that there is anoverlap between patients who are serial non-attendersand patients who are very frequent attenders. We willtherefore consider the rate of attending appointments asa potential predictor of the rate of non-attendance.Referrals that GPs make into other primary and second-ary care services will also be analysed. Outpatient atten-dances, hospital admissions and usage of emergencydepartments, NHS 24 triage, GP out of hours and ambu-lance services will also be analysed when linked databecome available with a specific focus on how this relatesto unmet need, for example, how might GP appointmentcategory relate to patterns of other healthcare usagebetween scheduled and unscheduled secondary care use?

Healthcare engagementAn analysis of GP Read codes and linked secondary caredata will be carried out in the following categories:1. Patients not attending primary and secondary care

appointments;2. Patients refusing screening;3. Patients being exception-reported (ie, excluded from

the denominator population) from the Quality andOutcomes Framework (QOF) system for perform-ance measurement in GP;

4. Practices’ measures of non-engagement with care forlong-term conditions;

5. Patients taking ‘irregular discharge’ from hospital(when patients leave against medical advice);

6. Patients not waiting to be seen in emergencydepartments.

Family linkageDiagnoses of children who are able to be linked throughfamily linkage will be analysed based on their mother’sappointment category. This is contingent on the childbeing included in the practice study population.

Education dataAttendance at school, exclusion from school and educa-tional attainment when leaving school will be made with

approximately a sixth of our patient cohort for whomlinked education data are available. This has the poten-tial to inform future planning around earlier interven-tions to reduce serial missed appointments.

Practice-level dataEach patient record will be allocated a unique practiceID enabling the research team to analyse each patientrecord output clustered by practice. This will be the pro-portion of patients aged over 75, by ethnicity (propor-tion BME), patient rurality, patient level of deprivationdecile, patient distance to practice, distance to A&Eappointments offered/engaged, days from whenappointment is made, multimorbidity count, ACE scoremore than 4, Severe and Multiple Disadvantage score,hospital referrals and proportion of each appointmentcategory by practice. These analyses and output will berefined as the study proceeds taking patient-level find-ings and multilevel modelling that characterise therespective contributions of practice-level and individual-level factors to missed appointment patterns.

Health outcomesMortality data regarding date and cause of death will beused from GP and linked data. This will sit alongsideadditional linked obstetric outcomes (from the ScottishBirth Record) for relevant women. Table 2 summarisesthe study quantitative variables.

ETHICS AND DISSEMINATIONThis pathfinder linkage retrospective cohort study isnecessarily complex in design and implementationbecause, although cross-sectional, it seeks to take a lifecourse approach and follow the patients’ journeythrough the healthcare system. Careful attention and sig-nificant resource has been devoted to the considerationof patient privacy and confidentiality. This has been inte-grated throughout the design of the study alongside thenecessary data access and handling permissions.Additionally, a study of this nature, which involves stake-holders across the NHS and other public services,requires a flexible time frame to allow access to raw dataand to share findings between members of the researchteam based in several institutions. The proof of conceptpilot did not require ethical approval because it was con-sidered service evaluation with the agreement that wewould not publish any results about the practices whichtook part. A letter of comfort was obtained from theWest of Scotland NHS Ethics Committee and the MVLSEthics Committee confirming that the full study did notneed health service ethics permissions.Owing to the sensitive nature of administrative data

from the NHS and public education system in Scotland,the data sets generated and/or analysed during thecurrent study will not be publicly available. They havebeen made available to the research team under con-trolled access and strictly for the purposes of this


Open Access



jopen.bmj.com

/B

MJ O


ebruary 2017. Dow

nloaded from


Table 2 Summary of quantitative categories and variables

Data categories Variables

Patient demographics Age

Sex

Ethnicity

Count of address moves

Distance to practice

Distance to A&E

SIMD decile

Rural8 score

Health conditions Multimorbidity count

Priority 1 read codes

Psychotropic medication prescribing (BNF chapter)

Secondary healthcare diagnoses (inpatient and outpatient)

Social vulnerability Adverse Childhood Experiences

Severe and Multiple Disadvantage

General social vulnerability coding frame

Healthcare usage Breast screening

Bowel screening

Cervical screening

BP checked

Child health surveillance

Vaccinations

Practice nurse appointments

Other healthcare professional appointments

Primary care attendance distribution

Hospital referrals

Outpatient attendances

Hospital admissions

Emergency departments attendance

NHS 24 triage

GP out of hours

Ambulance services callouts

Healthcare engagement DNA codes

Refused screening

QOF exempt

Inappropriate use codes

Self-discharge codes

Study exit Patient death

Patient moved practice

Family linkage Secondary healthcare linkage with mother and child

Education data School attendance

School exclusion

School attainment

Health outcomes Cause of death

GP practice characteristics Practice list size

Patient age distribution

Ethnicity category distribution

Patient rural8 score distribution

Patient SIMD score distribution

Patient distance to practice distribution

Patient distance to A&E distribution

Number of appointments offered/patients engaged past 3 years distribution

Number of days since appointments made distribution

Patient multimorbidity score distribution

Patient ACE score distribution

Patient SMD score distribution

Patient hospital referrals distribution

Primary care attendance pattern distribution

A&E, accident and emergency; ACE, Adverse Childhood Experiences; BNF, British National Formulary; BP, blood pressure; DNA, Did NotAttend; GP, general practitioner; NHS, National Health Service; QOF, Quality and Outcomes Framework; SMID, Scottish Index of MultipleDeprivation; SMR, Scottish Morbidity Record, SQA, Scottish Qualification Authority; SMD Severe and Multiple Disadvantage.


Open Access



jopen.bmj.com

/B

MJ O


ebruary 2017. Dow

nloaded from


research study only. Summary data, at the level of dis-closure checked output from the National Safehavenand statistical code, can be requested from the corre-sponding author on reasonable request.

Planned outputsAlongside peer-reviewed academic papers reporting thefindings described above, the following additionaloutputs are planned.

Data visualisationSeveral web pages will be built to sit alongside keyresults. This will allow for the rapid construction of inter-active data visualisations which will be created using‘Shiny’,31 a web application framework for R which isthe statistical software used for the study analysis. Asimple platform will allow researchers and collaboratorsto interact with the analyses in real time and generatecustom tables and graphs as required. It can alsoprovide non-experts with access to simple and complexstatistical analysis using a point-and-click interface. Thiswill not rely on raw data and will simply pull informationfrom the summary descriptive analyses.

Case studiesWe also intend to use case studies to develop and illus-trate our findings throughout the course of all our ana-lyses. For example, we will be able to identify typicalpatient profiles of those who appear to miss manyappointments in a very short period of time andcompare these events with short-term and long-termhealth outcomes.

CONCLUSIONWe shall identify key factors associated with serial missedappointments ranked in order of importance asdescribed above, but given the large sample size, weshall also be able to consider potential interactions thatmight predict serially missed appointments.Finally, this approach also explores the theory that low

engagement with healthcare should be viewed as ahealth harming behaviour, and will inform the debateabout tackling health inequalities at the health servicedelivery level. Moving from theory into application, theresults will allow us to better understand and developfuture interventions to reduce serial missedappointments.

Author affiliations1General Practice and Primary Care, School of Medicine, Dentistry andNursing, MVLS, University of Glasgow, Glasgow, UK2Department of Psychology, University of Lancaster, Lancaster, UK3Centre for Rural Health, Institute of Applied Health Sciences University ofAberdeen, Aberdeen, UK4Robertson Centre for Biostatistics, Institute of Health and Wellbeing, MVLSUniversity of Glasgow, Glasgow, UK

Twitter Follow Andrea Williamson @aewilliamsonl and David Ellis@davidaellis and Ross McQueenie @RossMcqueenie

Acknowledgements The authors would like to acknowledge all GP practicesand GPs who took part in the pilot study, and also colleagues at ScottishGovernment who are supportive of the study in a variety of ways especiallyEllen Lynch in the Health Analytics Division. Dave Kelly’s technical andprocedural expertise, wisdom and patience as director of the TTP Albasoft,underpins all of what has been achieved until now.

Contributors AEW is the principal investigator for the study. DAE, PW andAM are co-investigators on the study and RM is the research assistant. AEWconceived and developed the initial research proposal, reviewed the literature,conducted and analysed the pilot focus group, contributed to analysis andinterpretation of the quantitative pilot data, developed the predictors,outcomes and associations of interest and led on the writing of the paper.DAE supported the development of the initial research proposal, reviewed theliterature, conducted and analysed the quantitative pilot data, developed thestatistical and output plan, and contributed to the writing of the paper. PWsupported the development of the initial research proposal, reviewed thequalitative and quantitative pilot results, reviewed the statistical and outputplan and contributed to the writing of the paper. RM reviewed the statisticaland output plan and contributed to the writing of the paper. AM providedexpert statistical input to the study as it was developed, reviewed thestatistical and output plan and contributed to the writing of the paper. Allauthors read and approved the final manuscript.

Funding This study is supported by a Scottish Government Chief ScientistOffice research grant (CZH/4/1118) with Safehaven and data linkage costssupported in lieu by the DSLS at Scottish Government.

Competing interests None declared.

Ethics approval Ethical permission to conduct the GP focus group andpublish the results was obtained by the MVLS Ethics Committee, University ofGlasgow (ref 200140181). Multisite NHS R&D approval for the full study wasobtained for all Scottish Health Boards (NRS16/186358).

Provenance and peer review Not commissioned; externally peer reviewed.

Data sharing statement This study uses anonymised patient data only andhas obtained the permissions required to do so as described above.

Open Access This is an Open Access article distributed in accordance withthe terms of the Creative Commons Attribution (CC BY 4.0) license, whichpermits others to distribute, remix, adapt and build upon this work, forcommercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

REFERENCES1. Commission on Social Determinants of Health. Closing the gap in a

generation: health equity through action on the social determinantsof health. Final Report of the Commission on Social Determinants ofHealth. Geneva: World Health Organisation, 2008.

2. George A, Rubin G. Non-attendance in general practice: asystematic review and its implications for access to primary healthcare. Fam Pract 2003;20:178–84.

3. Campbell K, Millard A, McCartney G, et al. Who is least likely toattend? An analysis of outpatient appointment ‘did not attend’ (DNA)data in Scotland. Edinburgh: NHS Health Scotland, 2015.

4. Neal RD, Hussain-Gambles M, Allgar VL, et al. Reasons for andconsequences of missed appointments in general practice in the UK:questionnaire survey and prospective review of medical records.BMC Fam Pract 2005;6:47.

5. Mitchell AJ, Selmes T. Why don’t patients attend theirappointments? Maintaining engagement with psychiatric services.Adv Psychiatr Treat 2007;13:423–34.

6. Ciechanowski PS, Russo J, Katon W, et al. Where is the patient?The association of psychosocial factors and missed primary careappointments in patients with diabetes. Gen Hosp Psychiatry2006;28:9–16.

7. Lehmann TN, Aebi A, Lehmann D, Olivet M, et al. Missedappointments at a Swiss university outpatient clinic. Public Health2007;121:790–99.

8. Nielsen KM, Faergeman O, Foldspang A, et al. Cardiacrehabilitation: health characteristics and socio-economic statusamong those who do not attend. Eur J Public Health2008;18:479–83.


Open Access



jopen.bmj.com

/B

MJ O


ebruary 2017. Dow

nloaded from

http://twitter.com/aewilliamsonl

http://twitter.com/RossMcqueenie

http://creativecommons.org/licenses/by/4.0/



http://dx.doi.org/10.1093/fampra/20.2.178

http://dx.doi.org/10.1186/1471-2296-6-47

http://dx.doi.org/10.1192/apt.bp.106.003202

http://dx.doi.org/10.1093/eurpub/ckn060


9. Cashman SB, Savageau JA, Lemay CA, et al. Patient health statusand appointment keeping in an urban community health center.J Health Care Poor Underserved 2004;15:474–88.

10. Masuda Y, Kubo A, Kokaze A, et al. Personal features and dropoutfrom diabetic care. Environ Health Prev Med 2006;11:115–19.

11. Gucciardia E. A systemic review of attrition from diabetes educationservices: strategies to improve attrition and retention research. CanJ Diab Care 2008;32.

12. Murdock A, Rogers C, Lindsay H, et al. Why do patients not keeptheir appointments? Prospective study in gastroenterology outpatientclinic. J R Soc Med 2002;95:284–6.

13. Farley RL, Wade TD, Birchmore L. Factors influencing attendance atcardiac rehabilitation among coronary heart disease patients. EurJ Cardiovasc Nurs 2003;2:205–12.

14. Woodward V. Non attendance at a nurse-run asthma clinic. J ClinNurs 1998;7:380–1.

15. Killaspy H, Banerjee S, King M, et al. Prospective controlled study ofpsychiatric out-patient non- attendance, characteristics and outcome.Br J Psychiatry 2000;176:160–5.

16. Health and Social Care Information Centre England. Provisionalmonthly topic of interest: outpatient did not attend (DNA)appointment. London, 2014.

17. Watt G. Deep End Report 19. Access to specialists. The 19thmeeting of ‘General Practitioners at the Deep End’. University ofGlasgow, 2013.

18. Shonkoff JP, Garner AS, Committee on Psychosocial Aspects ofChild and Family Health; Committee on Early Childhood, Adoption,and Dependent Care; Section on Developmental and BehavioralPediatrics, et al. The lifelong effects of early childhood adversity andtoxic stress. Pediatrics 2012;129:e232–46.

19. Husain-Gambles M, Neal RD, Dempsey O, et al. Missedappointments in primary care: questionnaire and focus group studyof health professionals. Br J Gen Pract 2004;54:108–13.

20. Hart JT. The inverse care law. Lancet 1971;1:405–12.

21. Kitzinger J. Qualitative research. Introducing focus groups. BMJ1995;311:299–302.

22. Ritchie J, Lewis J. Qualitative research practice. A guide for socialscience students and researchers. Ist edn. London: SagePublications, 2003.

23. The Scottish Government. Charter for Safe Havens in Scotland:Handling Unconsented Data from National Health Service PatientRecords to Support Research and Statistics. Edinburgh, 2015. http://www.gov.scot/Publications/2015/11/4783/2

24. Smits FT, Brouwer HJ, van Weert HC, et al. Predictability ofpersistent frequent attendance: a historic 3-year cohort study.Br J Gen Pract 2009;59:e44–50.

25. Williamson AE, Mullen K, Wilson P. Understanding “revolving door”patients in general practice: a qualitative study. BMC Fam Pract2014;15:33.

26. O’Reilly D, Steele K. The removal of patients who live outside thepractice boundary: a study of outside-area removals in NorthernIreland in 2001-2002. Br J Gen Pract 2005;55:384–6.

27. Zuur AF, Ieno EN. A protocol for conducting and presenting resultsof regression-type analyses. Methods Ecol Evol 2016;7:636–45.

28. Barnett K, Mercer SW, Norbury G, et al. Epidemiology ofmultimorbidity and implications for health care, research andmedical education: a cross-sectional study. Lancet 2012;380:37–43.

29. Bellis MA, Ashton K, Hughes K, et al. Welsh Adverse ChildhoodExperiences (ACE) Study. Adverse childhood experiences and theirimpact on health-harming behaviours in the Welsh adult population.NHS Wales, 2015.

30. Bramley G, Fitzpatrick S, Edwards J, et al. Hard edges mappingsevere and multiple disadvantage England. England: LankellyChase Foundation, 2015.

31. Ellis DA, Merdian HL. Thinking outside the box: developing dynamicdata visualizations for psychology with Shiny. Front Psychol2015;6:1782.


Open Access



jopen.bmj.com

/B

MJ O


ebruary 2017. Dow

nloaded from

http://dx.doi.org/10.1353/hpu.2004.0037

http://dx.doi.org/10.1265/ehpm.11.115

http://dx.doi.org/10.1016/S1474-5151(03)00060-4

http://dx.doi.org/10.1016/S1474-5151(03)00060-4

http://dx.doi.org/10.1046/j.1365-2702.1998.00183.x

http://dx.doi.org/10.1046/j.1365-2702.1998.00183.x

http://dx.doi.org/10.1192/bjp.176.2.160

http://dx.doi.org/10.1542/peds.2011-2663

http://dx.doi.org/10.1016/S0140-6736(71)92410-X

http://dx.doi.org/10.1136/bmj.311.7000.299

http://www.gov.scot/Publications/2015/11/4783/2



http://dx.doi.org/10.3399/bjgp09X395120

http://dx.doi.org/10.1186/1471-2296-15-33

http://dx.doi.org/10.1111/2041-210X.12577

http://dx.doi.org/10.1016/S0140-6736(12)60240-2

http://dx.doi.org/10.3389/fpsyg.2015.01782


open access protocol understanding repeated non-attendance … · missed appointments. from their...

Documents