30 | December 2018 Medical Writing | Volume 27 Number 4

Rikke Havner Alrø1, Marie-Louise Krogh1,and Claire Gudex2

1. Centre for Innovative Medical Technology,Odense University Hospital, Odense,Denmark

2. Department of Clinical Research, Universityof Southern Denmark, Odense, Denmark

Correspondence to:Claire Gudexclaire.gudex@rsyd.dk

Patient-reported outcome (PRO) measureshave been used for some time in the Danishhealthcare sector, with patients completingquestionnaires about their health andtreatment experiences. This has mostlyoccurred through research projects, andquestionnaires have traditionally been inpaper form and sent through the post orcompleted at the clinic. The last few yearshave seen major changes, however, primarilyin the way the questionnaires are completed,the breadth of PRO use, and the ambition

level for using the data that are collected.This change in PRO use is partly a result of

developments in the Danish Healthcare QualityProgramme that was introduced in 2004.1 Thisprogramme aims to ensure continuousdevelopment of the quality of care to create betterpatient pathways and to prevent errors andunintended events in the healthcare system. Twofundamental objectives of the programme are toinvolve the users – patients and their relatives –in healthcare decisions and to ensure a patient-centred culture within hospital departments.

Systematic hospital collection ofpatient-reported outcome data via patient apps

AbstractMedical writers are increasingly likely to come across patient apps in their work, and we describe herethe background and use of two apps for assessing patient-reported outcome (PRO).

Systematic collection of PRO data via patient apps has been recently introduced in a large Danishuniversity hospital. Experiences so far show that the approach can help staff to focus on the individualpatient’s needs and can improve communication between patients and staff. Some patients have alsobeen able to avoid unnecessary hospital visits.

Effective clinical use of systematic PRO data requires the data to be easily accessed and visualised,and both patients and staff need to know that the data provided will make a meaningful contributionto health care.

www.emwa.org Volume 27 Number 4 | Medical Writing December 2018 | 31

A main strategy in the 2015 to 2018 nationalquality programme is to work towards thesystematic use of cross-sector PRO measureswith the aim of directly influencing treatmentdecisions and ensuring quality of care.2  SuchPRO data should include patient-reportedsymptoms, self-assessed health, and treatmentexperiences and should enable assessment of theeffect of a treatment or health intervention.

This means that it will be essential to collectvalid, reliable data to guide decision-making andthe development and evaluation of healthservices. The systematic use of PRO data isgenerally seen as a positive step, but it is still adeveloping concept, and as yet there is nonational consensus on which PRO measuresshould be used and in what format.

Systematic PRO data as apriority at OUH – SvendborgHospitalOdense University Hospital (OUH) andSvendborg Hospital together provide local andacute services for the Danish island of Funen.They represent the largest referral hospital inSouthern Denmark, with all medical specialties,approximately  1,000  hospital beds, and about

1 million outpatient visits per year (data fromOUH management). In addition, OUH takespatients from all over Denmark due to its highlyspecialised services.

OUH and Svendborg Hospital haveprioritised the systematic collection of PRO datain the expectation that this will benefit patients –primarily through faster and more appropriatediagnostic pathways and fewer, more targetedfollow-up visits. PRO data have been key aspectsof clinical research for some time, for example thecollection of EQ-5D data alongside measures offunctional independence and physical mobilityfor patients undergoing orthopaedic surgery,3 thedevelopment of a quality of life questionnaire forthyroid disease,4 and quality of life in toddlerswith middle ear disease.5 Systematic collection ofPRO data is not without its challenges, however,including lower completion rates for elderlypatients with impaired cognitive skills,3 and theneed to reassure healthcare staff that the PROdata make a meaningful contribution to theindividual patient’s care.6

OUH has recently established a networkinggroup for departments using PRO measures ona systematic basis. This mean that departmentscan help each other and can draw from eachother’s experiences using PRO measures. “The

departments are very positive. Most of them usePRO measures that have already been developed,which is of course the easiest solution. Otherswill have to develop new measures, and that is achallenge. But they can take advantage of theexperiences we already have and can use theexisting PRO measures as a starting point”, saysJon Sigurjónsson, PRO consultant at OUH.

The availability of electronic platforms hassignificantly changed PRO data collection. Insteadof developing paper-based PRO questionnairesfor a single purpose and then discarding themafter the requisite number of years, the data cannow be collected via platforms such as RedCapor apps on mobile devices that facilitate storage,analysis, and feedback.7 Many of the PROmeasures that will be used for systematic datacollection at OUH will be implemented throughthe regional app, “My Patient Journey”.

My Patient Journey – an appfor patients and medical staffThe “My Patient Journey” app was developedin  2014  at the Centre for Innovative MedicalTechnology at OUH for easier digitalcommunication between patients, medical staff,and hospital departments.8 The app helpspatients find and keep track of information from

Alrø et al. – Systematic hospital collection of patient-reported outcome data via patient apps

Figure 1: Comparison of the old and the new follow-upsystems for patients who have undergone a cardiologyintervention

A (lower picture): the ‘old’ - the patient visits thehospital and the municipal rehabilitation service inperson. B (upper picture): the ‘new’ - the patient sendshealth status data to the hospital and the municipalrehabilitation service, who then advise on the form andfrequency of follow-up.

Cre

dit:

Mar

ie-L

ouise

Kro

gh.

the hospital and aims to give them a betteroverview and experience in communicating withthe hospital. Today, it is in use all over the Regionof Southern Denmark by approximately 44,000patients and 2,000 clinicians (data extracted from“My Patient Journey”). The app is primarily forpatients, but it can be downloaded from GooglePlay or App Store (as “Mit forløb”).

The “My Patient Journey” app is now the userinterface for patients at OUH. Patients can sendtext messages to medical staff and can accessinformation about their own treatment in theform of text, videos, and images (thus replacing

32 | December 2018 Medical Writing | Volume 27 Number 4

the more general paper pamphlets). The app isalso a platform for the patients to enter data –such as weight or blood pressure  – and foranswering questionnaires uploaded by themedical staff.

The nursing staff are the main operators of theapp in the hospital. The app has been integratedinto the electronic medical journal that is alreadyused on a daily basis for recording patients’ visitsand information and for exchanging data withother hospital departments, general practitioners,and the municipalities.

The “My Patient Journey” app can be adaptedto the specific needs of the hospital departmentand for selected patient groups. Clinicaldepartments are thus free to choose which PROmeasures and other questions should beincluded. This is typically done with patientinvolvement to ensure relevant data collectionand minimal respondent burden. The emphasisfor PRO measures is on existing, validated healthand quality of life questionnaires.

When questionnaires are applied within the“My Patient Journey” app, an algorithm can becreated that calculates a score based on theindividual patient’s answers. The patient’s scorecan then determine how the medical staff shouldfollow up with the patient.

A PRO app for patients in aheart rehabilitationprogrammeThe cardiology clinic at Svendborg Hospital hasbeen running a pilot PRO project in apartnership with the municipal rehabilitationservices. This is a cross-sector collaborationwhere the responses that patients make to themedical staff ’s questions and to the PROmeasures are made available to the hospitaldepartment and the municipal rehabilitationservices at the same time (Figure 1).

The two PRO measures used in this projectare the Hospital Anxiety and Depression Scale(HADS) and the HeartQol. Both have 14 items;the HADS can be used to identify persons at riskof clinical anxiety or depression, while theHeartQol was developed by the EuropeanAssociation of Preventive Cardiology for patientswith ischaemic heart disease. The department hasbeen collecting patient data using these PROmeasures for some time, but in paper form.

In this project, the PRO measures are loaded

into the “My Patient Journey” app, and patientswho have undergone a cardiology interventionare asked to complete them at the first nurseconsultation (about two weeks after the inter -vention) and again after three and six months. Inthe meantime, the patient is referred to arehabilitation programme including exercisetraining and education on healthy living withheart disease.

The objective of the project is to see whetherthe PRO data can enrich the patients’ contactswith the hospital and municipal services throughmore relevant discussions and treatment. Thismay be in the form of a more individualisedrehabilitation programme or interventions foranxiety, depression, cessation of smoking, weightloss, etc.

The 30 patients who have partic ipated in theproject so far have given positive feedback. Theylike the easy interactive format where they canjust send an SMS to the nurse they know to askabout something in their daily life, and they knowthat the nurse’s answer is based on the patient’scurrent health and status. They also feel betterprepared to talk to the nurse at their nextconsultation and to discuss health and emotionalissues.

The nurses have found it easier toprepare for consultations withpatients and can focus the discussionon the issues that the individualpatient is currently facing. The PROdata enable the staff to stratifypatients earlier on, and patients whoare doing well and do not need closefollow-up can avoid unnecessaryhospital visits. The advantages forthe rehabilitation staff are that thepatient-reported data on problemsand challenges help them to betterplan the rehabilitation programmein advance, provide a way offollowing the patient’s progress, andcan identify areas that need morefocus.

An issue still requiring attentionis the response rate to the PROmeasures completed via an app, assome patients were less accepting of thisapproach. These were especially older patientswho were less familiar with smartphones andtablets. This will be one of the aspects to beevaluated during the next phase of the project,

which is introducing the PRO app into routineclinical use.

A PRO app for patientsundergoing prostate cancersurgeryFollowing a successful two-year project,9 a PROapp is now used routinely with patientsundergoing prostate cancer surgery at theurological ward at OUH. Using “My PatientJourney” as a platform, patients answer thequestions electronically from home both beforesurgery and again at 3, 6, and 12 months aftersurgery. The questionnaire has been developedby a national working group under the DanishHealth Authority and is aimed at all patients withprostate cancer regardless of the type oftreatment they get. At the same time-points, thepatient also has a blood test taken by the generalpractiti oner. The PRO app scores the patient’sanswers, and the resulting score and blood testresults give a colour-coding for whether tele -phone or outpatient follow-up is necessary(orange or red) or not necessary (green), see theexample in Figure 2 overleaf. The objective hereis to use the PRO data to reduce the number of

unnecessary hospital visits.This approach has been a great

success. The use of the PRO data haseliminated two-thirds of the follow-up visits in this patient group, thusgiving the staff more time forpatients with more complicatedproblems and reducing the waitinglists.

The patients report that the apphelps them to be more active andinvolved in their treatment anddiscussions with hospital staff. Theapp approach appears to beespecially beneficial for elderlypatients, who can now send a pictureor a video via “My Patient Journey”instead of having to make the trip tothe hospital. An important element,however, is that patients can see that

the medical staff have made active useof the PRO data, and that the data are not justcollected and then stored.

Next steps include broadening the use of thePRO app, for example to patients with prostatecancer who are being treated with medicines

Effective clinicaluse of systematic

PRO data requiresthe data to be

easily accessed andvisualised at theclinical contact,

and both patientsand staff need to

know that the dataprovided will make

a meaningfulcontribution to

improved healthcare.

Systematic hospital collection of patient-reported outcome data via patient apps – Alrø et al.

www.emwa.org Volume 27 Number 4 | Medical Writing December 2018 | 33

Alrø et al. – Systematic hospital collection of patient-reported outcome data via patient apps

rather than surgery. This may be morechallenging in terms of resources, as the PRO appis not expected to reduce visits and it takes timefor the medical staff to go through the patient’sanswers to the questionnaire and discuss them asappropriate.

PRO measures are the future“It is not a question of whether or not we shoulduse PRO measures. We should definitely use

them and much more than today  – in everydepartment in the hospital”, says Kim Brixen,medical director at OUH. “It providespossibilities for better and faster diagnostics inaddition to fewer and better follow-ups. PROmeasures are also useful for collecting researchdata, which is especially valuable for a universityhospital. Depending on the patient’s diagnosis, Ibelieve that many outpatient visits can bereplaced by PRO surveys.”

Another potential advantage of the appapproach to PRO data collection is in clinicalresearch projects, where patients with baselinePRO data can easily be block-randomised anddivided into intervention and control groups.

There are notes of caution when using PROapps, however. One is the tendency to focus onthe specific health issues that emerge from thepatient data, thus potentially missing importantinformation that is not asked about. A similar

Figure 2: An example from the cardiology project of how PRO app data can be used Here, questions from the Hospital Anxiety and Depression Scale (HADS) are combined with blood test results (here on cholesterol level) to stratify patientsat follow-up to RED (current problem, needs immediate attention), YELLOW (current problem, needs follow-up), or GREEN (no current problem). In thisexample, the patient has several “red” and “yellow” issues at the first visit, and these improve to green over time.

English translation:Baseline – Heart rehabilitation: HADS questions (Choose 1 answer): I feel tense or “wound up”: Most of the time / A lot of the time / From time to time,occasionally / Not at all. I get a sort of frightened feeling like ‘butterflies’ in the stomach: Not at all / Occasionally / Quite Often / Very OftenMy focus points [for] Initial – Midway – Final Consultation: Cholesterol – Blood pressure – Smoking – Diabetes – Diet – Exercise – Weight – Sleep –Alcohol – Anxiety and depression - Stress

KOLESTEROL

BLODTRYK

RYGNING

SUKKERSYGE

KOST

MOTION

VÆGET

SØVN

ALKOHOL

ANGST OG DEPRESSION

STRESS

Visitations-samtale

Midvejs-samtale

Afsluttendesamtal

34 | December 2018 Medical Writing | Volume 27 Number 4

problem can arise with telephone and e-mailconsultations and from telemedicine, as the fullpicture of a person’s health is only achievedthrough a (well-performed) traditionalconsultation. This is an issue that requires furtherresearch, as it could have negative effects onpatient treatment.

A further issue is the extent to which the PROmeasures should be piloted and validated beforethe hospital initiates large projects or implementsroutine data collection. In many countries, PROquestionnaires are typically put through a longtesting process before they are used. OUH hasdecided to test the use of PRO measures in “reallife”, however, by putting them into use andcorrecting any errors as they are identified.

While Denmark is known for its compre -hensive system of health registers that can belinked through the individual personal identifier,it is recognised that the sharing of PRO databetween patient and medical staff, and betweenhospital sectors, has to be done with care andattention to individual privacy. Written informedconsent to share the PRO data is typically donethrough the PRO app. The patient is presentedwith relevant information, and this needs to beregistered in the app as “read” before it is possibleto give consent. Consent can also be withdrawnthrough the app. The issues surround ing dataprivacy and informed consent may become moreprominent as the sharing of personal health databecomes more widespread.

ConclusionsThere is still much to learn about the systematiccollection of patient-reported outcome data viaapps as an aid to optimising healthcare treatmentand care. However, it appears to be a promisingapproach for focusing on the individual patient’sneeds and current status and for improvingcommunication between patients and healthcarestaff.

An important next step is the more formalevaluation of the PRO app projects. Thecardiology project described here is currentlybeing evaluated using the Model for Assessmentof Telemedicine approach. This involves assess -ment of several aspects such as the clinical effects,patient safety, the patients’ perspectives andexperiences, financial aspects, and organi sationaleffects.

Effective clinical use of systematic PRO datarequires the data to be easily accessed and

visualised at the clinical contact, and bothpatients and staff need to know that the dataprovided will make a meaningful contribution toimproved health care.

AcknowledgementsThe authors would like to thank the followingpeople for providing information for this article:l Kim Brixen, medical director, Odense

University Hospitall Ulla Geertsen, MD, Department of

Urology, Odense University Hospitall Jens Lauritsen, MD, PhD, Department of

Orthopaedics and Traumatology, OdenseUniversity Hospital

l Jon Sigurjónsson, Patient-reportedinformation consultant, OdenseUniversity Hospital

l Claus Tveskov, MD, PhD, Department ofInternal Medicine and Acute Medicine,Svendborg Hospital

We also thank Raquel Billiones for reviewingthe manuscript.

Conflicts of interestThe authors declare no conflicts of interest.

References1. The Danish Healthcare Quality Programme

(DDKM). Imaging Management.2010;12(5). Available from:https://healthmanagement.org/c/hospital/issuearticle/the-danish-healthcare-quality-programme-ddkm.

2. Danish Ministry for Health and Prevention[in Danish]. [Nationalt Kvalitetsprogram forSundhedsområdet 2015–2018]. 2015.Available from: https://www.sum.dk/~/media/Filer%20-%20Publikationer_i_pdf/2015/Nationalt-kvalitetsprogram-for-sundhedsomraadet/Nationalt%20kvalitetsprogram%20for%20sundhedsomr%C3%A5det%20-%20april%202015.ashx.

3. Pedersen TJ, Lauritsen JM. Routinefunctional assessment for hip fracturepatients. Acta Orthop. 2016;87(4):374–9.

4. Winther KH, Cramon P, Watt T, Bjorner JB,Ekholm O, Feldt-Rasmussen U, et al.Disease-specific as well as generic quality oflife is widely impacted in autoimmunehypothyroidism and improves during thefirst six months of levothyroxine therapy.PLoS One. 2016;11(6):e0156925.

5. Indius JH, Alqaderi SK, Kjeldsen AD,Heidemann CH. Middle ear disease inDanish toddlers attending nursery day-care:applicability of OM-6, disease specificquality of life and predictors for middle earsymptoms. Int J Pediatr Otorhinolaryngol.2018;110:130–4.

6. Pedersen TJ, Kristensen HK. A criticaldiscourse analysis of the attitudes ofoccupational therapists and physiotherapiststowards the systematic use of standardisedoutcome measurement. Disability andRehabilitation. 2016;38(16):1592–602.

7. Svendsen MT, Andersen F, Andersen KH,Andersen KE. Can an app supportingpsoriasis patients improve adherence totopical treatment? A single-blindrandomized controlled trial. BMCDermatol. 2018;18(1):2.

8. MedWare ApS. My Patient Journey.Accessed 28th August 2018. Available from:http://www.medware.dk/en/ project/my-patient-journey/.

9. The Danish Cancer Society [In Danish].[Klinisk anvendelse af Patient ReportedOutcome Measures (PROM) – enevalueringsrapport]. 2016. Available from:https://www.cancer.dk/dyn/resources/File/file/9/6659/1505726340/klinisk-anvendelse-af-patient-reported-outcome-measures-prom.pdf.

Author informationRikke Havner Alrø, MA, is a communi -cations consultant for Centre for InnovativeMedical Technology at Odense UniversityHospital.

Marie-Louise Krogh is an innovationconsultant at Odense University Hospital.

Claire Gudex, MD, PhD, is a health servicesresearcher with a joint appointment atOdense University Hospital and University ofSouthern Denmark.

Systematic hospital collection of patient-reported outcome data via patient apps – Alrø et al.