ORIGINAL ARTICLE

SMS versus voice messaging to deliver MNCHcommunication in rural Malawi: assessment of deliverysuccess and user experienceJessica Crawford,a Erin Larsen-Cooper,b Zachariah Jezman,a Stacey C Cunningham,c Emily Bancroftb

Mobile SMS health messages had higher successful delivery and led to higher intended or actual behaviorchange among subscribers than voice messages. Providing multiple delivery modalities led to greateroverall access.

ABSTRACTObjective: To determine the difference in delivery success of health messages delivered through pushed SMS, pushedvoice messages sent to personal phones, and voice messages retrieved from a community phone (‘‘retrieved voicemessaging’’), as well as the difference in quality of the user experience.Methods: We analyzed the project’s electronic monitoring data between September 2011 and June 2013, includingdemographics, enrollment data, and messages sent and successfully delivered. We also collected and analyzedinformation from quarterly phone-based surveys with users to assess quality of the user experience, includingacceptability, comprehension, new information learned, and reported behavior change.Results: More than half of subscribers enrolled in the retrieved voice messaging service while nearly one-third enrolledin the pushed SMS service and less than 10% in pushed voice messaging. Message delivery success was highest amongpushed SMS subscribers and lowest among retrieved voice subscribers. Overall, 99% of survey respondents reportedtrusting messages they received, and about 75% of respondents recalled the last message they received and learnedsomething new. Almost 75% of respondents reported that they had already changed or intended to change theirbehavior based on received messages. Intended or actual behavior change was significantly higher among pushed SMSenrollees than among pushed or retrieved voice messaging enrollees (P5.01).Conclusion: All message modalities led to high levels of satisfaction, comprehension, and new information learned.Due to lower cost, higher delivery success, and higher levels of intended or actual behavior change, SMS is the preferreddelivery modality. However, the majority of users included in this study did not have access to a personal phone, andretrieved voice messages provided an opportunity to access a population that otherwise could not be served. Providingmultiple methods by which users could access the service was crucial in extending reach beyond literate personal phoneowners.

BACKGROUND

D espite significant progress toward improvingmaternal and child health, Malawi still has high

maternal and infant mortality; the maternal mortalityratio is 675 maternal deaths per 100,000 live births and

the under-5 mortality rate is 112 deaths per 1,000births.1 Underlying causes of poor health for womenand children include limited availability of timely andreliable health information for decision-making andpoor access to and use of health facilities.2 Knowingwhen and where to go for care is integral to maxi-mizing health care access and use and to reducingmaternal and child mortality.

With increased availability and use of mobiletechnology, mHealth is becoming a widely used

a VillageReach, Balaka, Malawi.b VillageReach, Seattle, WA, USA.c VillageReach, Seattle, WA, USA. Now with Ipas, Chapel Hill, NC, USA.

Correspondence to Jessica Crawford (jessica.crawford@villagereach.org).

Global Health: Science and Practice 1

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

strategy to address barriers to accessing healthinformation and care.3–5 mHealth is the use ofmobile phones to promote healthy behavior,increase use of health services, improve adher-ence to health advice, and increase access tohealth information. One growing application ofmHealth is mobile messaging, whereby healthinformation and promotion messages are sentdirectly to clients. Previous studies have high-lighted ways in which mobile messaging can besuccessful in smoking cessation, weight loss, dietand physical activity, treatment adherence, anddisease management.6,7

Mobile messaging is increasingly beingapplied to improve reproductive, maternal, neo-natal, and child health (RMNCH), and there isgrowing evidence of its effectiveness.8–11 Forexample, a U.S.-based program, text4baby, senttext messages to traditionally underserved preg-nant women encouraging them to adopt healthyattitudes, beliefs, and behaviors during preg-nancy. The program was effective in increasingagreement with the statement, ‘‘I am prepared tobe a new mother.’’ However, it did not lead tosignificant behavior change among enrollees.10

The effects of the intervention were greateramong enrollees with more education, suggest-ing that literacy and comprehension of messagesmay play crucial roles in the effectiveness ofmHealth projects.

An mHealth project in Zanzibar called WiredMothers sent pregnant women text messageswith appointment reminders and health educa-tion information, and it also gave enrolleesphone vouchers to call health care providersdirectly with questions. The project significantlyincreased facility-based births among urbanwomen but had no significant effect on ruralwomen.11

In Thailand, the ‘‘Better Border and HealthcareProgram’’ found that sending users text messagesregarding antenatal care visits and immunizationsincreased the number of mothers and childrenattending scheduled appointments.12

Mobile messages are generally delivered byeither short message service (SMS) or voicemessaging; both offer distinct advantages anddisadvantages. SMS is available on an estimated98% of mobile phones, does not require technicalexpertise to use, and is adaptable to multiplemHealth purposes.13 SMS messages can beaccessed at user convenience and can be deliv-ered to phones that are turned off or have flatbatteries. In addition, telecom costs for SMS are

generally less expensive than costs for voicecommunication. Given these advantages, it is notsurprising that SMS is the most common deliverymethod for mobile development services.13

Voice messages, although used less often inmHealth projects, offer the advantages of acces-sibility to illiterate populations, ability to containmore information per message than SMS, andability to be recorded in any language (not alllanguages/characters are supported by SMS).14

In addition, recorded voice messages allowinformation to be conveyed by a clinical ‘‘char-acter,’’ as used in Bangladesh by the MobileAlliance for Maternal Action (MAMA), to buildrapport and trust over time.15

Research comparing the outcomes of voiceversus SMS messaging is limited in the mHealthliterature, although user preference has beenexamined in a small number of studies. A U.S.-based study among literate English speakersfound that 72% of participants in an mHealthprogram targeting exercise, diet, and smokingpreferred to enroll in automated voice messagesover SMS. Those who preferred SMS tended tobe younger and to have higher levels of comfortwith computers and higher levels of SMS use.16

The Mobile Technology for CommunityHealth (MOTECH) initiative in Ghana used a‘‘Mobile Midwife’’ application to provide preg-nant women with health education and remin-ders to access necessary medical services.Messages could be received in voice or SMSformat, depending on the enrollees’ preference;99% of enrollees chose voice.17

Questionnaires administered to potentialmHealth users in Argentina, however, found thatthe users were similarly open to receiving SMSand voice messages during pregnancy, with 96%of participants willing to receive SMS messagesand 87% willing to receive voice messages.18 Toour knowledge, no studies have been conducted todetermine the difference in acceptability, compre-hension, or behavior change between usersreceiving messages through voice compared withSMS formats.

This article describes the messaging compo-nent of an mHealth pilot project in Malawiaimed at improving knowledge and uptake ofhome- and facility-based MNCH care practicesusing both SMS and voice message deliverymodalities. We report findings from the pilotexamining differences between SMS and voicemessaging with regard to delivery success andquality of user experience, including acceptability,

Mobile healthmessaging hasbeen successful inpublic healthprograms, such asfor smokingcessation andweight loss.

Literacy may be acritical factor inthe effectivenessof mHealthprojects.

SMS has severaladvantages overvoicecommunication,but voicemessages mayprovideaccessibility toilliteratepopulations.

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 2

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

comprehension, new information learned, andreported behavior change.

PROJECT DESCRIPTION

Chipatala Cha Pa Foni (CCPF) (‘‘Health Center byPhone’’) is a mobile health project of ConcernWorldwide’s Innovations for Maternal, Newborn& Child Health initiative, and its partnersVillageReach and the Malawi Ministry of Health(MOH). A nationwide campaign generated morethan 6,000 ideas of how to improve quality ofhealth services from community members inMalawi; one of the winning ideas became CCPF.

CCPF consists of a toll-free hotline offeringprotocol-based health information, advice, andreferrals as well as a mobile messaging serviceoffering automated tips and reminders forpregnant women, guardians of young children,and women of childbearing age. Starting in 2011,CCPF was piloted in 4 rural health centercatchment areas in Balaka District in SouthernMalawi, representing a population of 32,000women of childbearing age, 24,000 childrenunder 5, and 7,000 expected pregnancies peryear. The pilot project aimed to improve thecoverage and quality of RMNCH in BalakaDistrict. Early success and interest has resultedin scale up to an additional 3 districts.

The hotline component of CCPF is staffed byindividuals trained in RMNCH using modulesfrom the MOH’s curriculum for communityhealth workers (CHWs). When answering calls,hotline workers are prompted by software ontouch screen devices to identify clients’ symp-toms and/or information needs. They answerquestions, offer health advice, and provideinformation on when and where to seek care ifclients have symptoms or danger signs thatcannot be treated safely at home.

Hotline workers also offer eligible clients theopportunity to enroll in the tips and remindersmobile messaging service where they can accessvoice or SMS messages on a weekly basis.Messages offer timely information and remin-ders about important health services, based onthe estimated date of delivery for pregnantwomen or the child’s age for caregivers ofchildren under 1. The tips and reminders areintended to elicit behavior change by helpingclients understand their susceptibility to andseriousness of common maternal and childhealth illness, and they provide cues to action,reminding pregnant women and caregivers of

children to seek timely preventive care. Messagesfor women of childbearing age were added laterin implementation.

During project design, we found that lowliteracy and phone ownership among the targetpopulation may limit access to the service.Although an estimated 75% of women over 15in Balaka District are literate,1 poor women andwomen living in rural areas are less likely to beliterate than their wealthier and urban counter-parts.19 Furthermore, a baseline assessmentfound that only approximately one-third ofwomen of childbearing age in the catchmentarea own a mobile phone.20 Thus, we developed 3

© V

illageReach 2012

A communication poster encourages caregivers of children to call theChipatala Cha Pa Foni hotline to get answers to their health-relatedquestions.

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 3

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

message delivery modalities in order to betteraccommodate individuals with low literacy orwithout access to personal phones:

1. Pushed SMS Messages: Clients with accessto a personal or household phone canautomatically receive SMS messages on theirmobile phones.

2. Pushed Voice Messages: Low-literacy cli-ents with access to a personal or householdphone can receive weekly recorded voicemessages that are sent to the phone at a specifictime each week.

3. Retrieved Voice Messages: Clients withoutaccess to a personal phone can access theirweekly messages by calling the toll-freehotline from any phone and entering anappropriate code to hear their message.

Voice messages are more expensive to deliverso clients with personal phones who identified asbeing literate were generally encouraged toregister for SMS. Low-literacy clients were offeredthe option of registering for voice messages. Priorto launch, each message delivery option was pilottested among a small group of users in Balaka toensure delivery and accessibility.

Due to low phone ownership in Balaka, weprovided an additional point of access to CCPF.We originally planned to distribute phones toMalawi’s paid cadre of CHWs. However, given thelarge number of potential users without access toa personal phone, community volunteers fromeach village were instead recruited in consultation

with traditional leaders in the area and providedwith a low-cost phone. The volunteers wereoriented to the program and provided with smallincentives throughout the pilot period to encour-age their continued participation. In order toprovide phones to all volunteers, we soughtpartnerships that would allow us to stretch thebudget to provide phones to a larger number ofpeople than originally anticipated. Our telecompartner agreed to provide a discount on their basiclow-cost promotional phones, which were ulti-mately purchased for the project.

Message Development and ContentMessages were developed in collaboration withMOH staff with different areas of expertise, withtechnical support from PATH, the GrameenFoundation, and BabyCenter. The content wasdesigned to complement national policies and toaddress local myths and traditions around preg-nancy and child rearing. The final content wasreviewed and revised by a group of women fromBalaka with experience working on RMNCHissues at the community level. Messages wereinitially developed in English and later translatedto Chichewa and Chiyao, the primary locallanguages spoken in Balaka.

The final content was the same for both voiceand SMS messages. However, SMS messageswere delivered more frequently than voicemessages because voice messages were longerthan SMS. Messages were delivered once ortwice per week depending on the stage ofpregnancy or age of the child. Table 1 providesexamples of SMS and voice messages.

Content was reviewed periodically andupdates were made to include new information.For example, new information was added inmid-2012 about the launch of a new pneumo-coccal vaccine in Malawi.

Originally, CCPF relied on volunteers toadvertise the service in the community. Later,CCPF staff started to attend antenatal care clinicsto inform pregnant women about CCPF and tohelp them sign up for the service. In addition,large community events were held to informcommunity members about CCPF and how toaccess the service.

DATA AND METHODS

To understand the difference in delivery successand quality between the 3 different messagingmodalities (pushed SMS, pushed voice messages,

© V

illageReach 2013

A hotline worker answers an incoming calling using a touch screen device thatprompts the worker to identify the client’s symptoms and information needs.

SMS messageswere deliveredmore frequentlythan voicemessages becausethey were shorterthan the voicemessages.

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 4

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

retrieved voice messages), we collected andanalyzed information from routine monthlymonitoring data and quarterly phone-basedsurveys with users.

Routine Monthly MonitoringData from the messaging service were monitoredmonthly from 3 technology sources built to deliverthe service:

N Hotline Software. Built by Baobab Health, aMalawian nongovernmental organization pro-viding eHealth solutions, the hotline softwareis protocol-based and guides hotline workersthrough each call according to informationcollected through interactions with the client.The software includes demographic informa-tion and tips and reminders enrollment.

N Notification Application. Built by Village-Reach, the application connects the hotlinesoftware to a communications server andpushes voice and SMS messages to enrolleeswith personal phones. This application re-cords data on messages sent and successfullydelivered.

N INTELLIVR (IVR) Software. Built byYo! Uganda, a software company in Uganda,IVR hosts incoming calls and also serves as theoutgoing-voice gateway for the tips and remin-ders service.

Monthly data for all enrolled clients fromSeptember 2011 through June 2013 were analyzedto describe the number and characteristics ofenrollees by delivery modality as well as deliverysuccess rates.

Quarterly Phone-Based SurveysStarting roughly 3 months following the launch ofCCPF, periodic phone-based surveys were con-ducted among a sample of enrollees. Clientsenrolled in pushed SMS and voice messagingservices were randomly selected from a list of allenrollees and were contacted directly using thephone number for message delivery. Because wedid not have phone numbers for clients enrolled inthe retrieved voice message service, random selec-tion was not feasible. Instead, we sent communityvolunteers a SMS asking them to identify a womanin their area who had used the service and to notify

TABLE 1. Sample SMS and Voice Messages

Subscriber

Message Type

SMS Voice

PregnantWomen

Message 1: When you and your familyknow that you are pregnant, a visit toANC will help you understand everythingyou need to do to keep the baby healthy.

Message 2: You and your growing babyneed food for energy, body building,and protection. Meat, eggs, greens,vegetables, rice, and fruits help yourbaby grow.

The ANC is your partner in the pregnancy. It is important to go to all 4of your visits and to use the tablets that they have given to you. Yourbaby is continuing to grow! Your baby’s fingernails and fuzzy hairare appearing and it’s able to do things like swallow and kick. In thenext 3 weeks, your baby will double in size. Body building foods likemeat, beans and eggs are important for you and your baby. Try totake some every day. And remember; only take medicines given toyou by a nurse. Traditional medicines can be dangerous for yourbaby. You might be feeling that you need to urinate a lot. This is anormal part of pregnancy. Try to drink less water before bed. If youfeel pain or burning when you urinate you may have infection.

Caregivers ofChildrenUnder 1

Message 1: Make sure your baby has itsvaccination. In the first week, your babyshould get polio vaccine by mouth and theBCG vaccine against TB by injection.

Message 2: Keep the cord stump cleanand dry. Do not force it to fall off. If thereis discharge or redness visit the clinic.Infections can be serious.

Congratulations on being a new mom! This week, make sure your babyhas all its vaccinations, including the polio vaccine and the BCGvaccine against tuberculosis. You should start breastfeedingimmediately and remember to only give your child breast milk for thefirst 6 months of his life. Carry your baby close to your breasts becauseit will help give your baby good physical and emotional conditions andhelps you to grow close. Keep your baby’s cord stump clean and dryand do not force it to fall off. If there is discharge or redness visit theclinic. Infections can be serious. Your baby should be placed on hisback for sleeping until he is strong enough to roll over himself.

Abbreviations: ANC, antenatal care; BCG, bacille Calmette-Guerin; SMS, short message service; TB, tuberculosis.

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 5

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

hotline staff once they gave the community phoneover to the client. At that time, the client wascontacted directly for the survey.

For all participants, a hotline worker orvolunteer administered the questionnaire, whichcollected information on frequency of messagesreceived and quality of the experience. Quality ofuser experience was measured as:

N Acceptability: proportion of respondentswho indicated they trust the messages

N Comprehension: proportion of respondentswho could describe the last message theyreceived; the description given by the respon-dent was then checked in the analysis phaseagainst the messages the respondent shouldhave received

N New information learned: proportion ofrespondents who indicated they learned newinformation about their pregnancy or child’shealth and could list topics they deemed to benew information

N Reported behavior change: proportion ofrespondents indicating a health behaviorchange or intent to change behavior based onthe messages received. Behavior change intentwas scored according to a series of questionsregarding specific behaviors targeted in the

messaging. For pregnant women, these beha-viors include number of antenatal care visits,food and medicine consumption, and place ofdelivery. For caregivers of children, thesebehaviors include medicine and vaccinationsfor the child, exclusive breastfeeding, andnumber of postnatal care visits.

The survey was implemented at 3, 6, 12, and18 months following the launch of the service.Data from 266 questionnaires were entered intoan Excel spreadsheet and analyzed using STATAversion 10.

RESULTS

Routine Monthly MonitoringDuring the first 2 years of implementation(through June 2013), more than 5,000 pregnantwomen and caregivers of children under 1registered for the tips and reminders service.Client registration data were collected starting inJuly 2011, when clients could begin registeringfor the service, while monitoring of deliverysuccess rates started in September 2011 when themessaging service was officially launched.

Table 2 displays the number of registeredclients and demographics by content type anddelivery method. More than half of clientsenrolled in the retrieved voice service, nearlyone-third enrolled in the pushed SMS service,and less than 10% enrolled in the pushed voicemessage service. The mean age of pregnantenrollees was 25, and they registered, on average,16 weeks before their estimated due date.Caregivers of children under 1 registered, onaverage, 36 weeks before their child’s first birthday.

Characteristics of clients were similar acrossservice types with the exception of occupation.Pregnant women who registered for pushed SMSmessages were more likely to be employed in theformal sector than those who registered for(pushed or retrieved) voice messages. Formalsector includes those who reported their occupa-tion as ‘‘business,’’ ‘‘health care worker,’’ or‘‘teacher.’’ Informal sector includes those whoreported their occupation as ‘‘farmer,’’ ‘‘house-wife,’’ ‘‘student,’’ or ‘‘other.’’

The proportion of messages successfullypushed and retrieved varies by month. Forpushed messages (SMS and voice), success ratesrepresent the number of messages successfullyreceived after 3 attempts divided by the numberof messages attempted. For retrieved voice

TABLE 2. Demographics of Registered Users, by SubscriberType and Delivery Method (July 2011–June 2013)

PushedSMS

PushedVoice

RetrievedVoice

Pregnant Women

No. of registered users 704 238 1,559

Age, mean, y 25 26 25

Employed in formal sector, % 13 9 8

No. of weeks eligible formessages, mean 16 16 16

Caregivers of Children Under 1

No. of registered users 733 224 1,654

Age of child, mean, mo 4 4 4

Female, % 54 53 55

No. of weeks eligible formessages, mean 36 35 36

The majority ofsubscribersenrolled in the‘‘retrieved voicemessaging’’service becausethey did not own apersonal phone.

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 6

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

messages, success rates represent the number ofmessages successfully retrieved from the IVRmenu divided by the number of messages weestimate should have been retrieved each month.Overall success rates for pushed SMS and voicemessages ranged from 54% to 64% and werehigher for pushed SMS than for pushed voicemessages (Table 3). Success rates for retrievedvoice messages were lower, at 27% to 38%.

Delivery success changed over time, with theproportion of pushed voice messages successfullydelivered to a personal phone increasing slightlyover time, and the proportion of retrieved voicemessages successfully delivered decreasing overtime (Figure 1). Pushed SMS delivery successrates remained relatively stable over time, with adecline in the beginning of 2013, as a result of anumber of technical issues. For example, config-uration changes by the telecom provider unin-tentionally reduced the number of times oursystem attempted to send messages, resulting inmessages being attempted less than 3 times. Theissue has since been rectified, but it affectedsuccess rates for January–May 2013.

To better understand voice message retrievalrates, we also calculated the proportion ofmessages successfully played based on thenumber of attempts (Figure 2). This proportionis calculated from the number of times themessage menu is reached (that is, a person callsCCPF’s toll-free number and enters ‘‘2’’ to hear amessage and then ‘‘1’’ to hear a pregnancymessage) and how many times messages areplayed. Using this definition of delivery successfor retrieved voice messages, messages weresuccessfully played in 56% of the instances whenthe message menu was reached, and success wasslightly higher for messages played from the

child message menu than messages played fromthe pregnancy message menu (60% versus 54%).

Phone-Based SurveysHotline workers and volunteers were able toreach 266 total clients for the phone survey. Ofthose, 96 were enrolled in the pushed SMSservice, 30 in the pushed voice service, and 140 inthe retrieved voice service (Table 4).

TABLE 3. Delivery Success of Messages, by Delivery Method and Subscriber Type (September 2011–June 2013)

Pushed SMS Pushed Voice Retrieved Voice

Messages Pregnancy Child Pregnancy Child Pregnancy Child

No. attempted or expecteda 19,356 20,363 3,022 2,815 32,054 52,829

No. successfully received or retrieved 11,825 13,053 1,820 1,515 12,257 14,455

Percent success 61% 64% 60% 54% 38% 27%a For retrieved voice messages, the expected number of messages retrieved is calculated weekly based on the number of current subscribers. Forpushed SMS and voice messages, the number of messages attempted was based on actual number of attempts.

FIGURE 1. Delivery Success Rates of Messages Over Time, byDelivery Method, September 2011–June 2013

Success rate for pushed SMS and voice messages is the percent of messagessuccessfully received within 3 attempts; for retrieved voice messages, the successrate is the percent of expected messages successfully retrieved.

The SMS servicehad higherdelivery successrates than thepushed orretrieved voicemessagingservices.

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 7

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

Slightly more clients enrolled to receivemessages about child health than about preg-nancy were included in the survey. At the time ofthe survey, SMS enrollees reported the highestnumber of messages received, at an average of6.9 messages. Pushed voice and retrieved voiceenrollees included in the survey had received anaverage of 3.1 and 3.7 messages, respectively, atthe time of survey. Overall, 19% (51) ofrespondents indicated that they had not yetreceived any messages, the majority of which(29) were among pushed voice enrollees.

Overall, 99% of respondents reported trustingmessages they received (acceptability), and 75%recalled the last message they received (compre-hension) (Table 5). Pushed SMS and retrievedvoice enrollees were more likely than pushedvoice enrollees to recall the last message received,but the observed differences were not statisticallysignificant.

Just over three-quarters of enrollees indi-cated that they had learned something new fromthe messaging service and were able to describethe new information. Pushed SMS enrollees weremore likely to respond that they had learnedsomething new.

Almost three-quarters of respondents reportedthat they had already changed or intended tochange their behavior based on the messages theyreceived. Pushed SMS enrollees were significantlymore likely to report intended or actual behaviorchange than pushed or retrieved voice messagingenrollees (P5.01). See box for examples of beha-viors that respondents said they intended to change.

DISCUSSION

Understanding the effects of message modality isimportant when deciding between SMS or voicemessage services in mHealth programs. AlthoughSMS is a common and effective method, voicemessages may be a desirable option based onliteracy levels and mobile phone access.

The data presented in this article reveal thatdelivery success rates for all message modalitiesare less than ideal. Phone network challenges inrural Balaka were commonly reported by com-munity volunteers and clients throughout thepilot period and likely contributed to the overalllow success rates.

Message delivery rates are far more success-ful among SMS than voice enrollees. Pushingvoice messages to clients with personal phones isa complex process, requiring the client to answerthe phone at the time of delivery, whereas SMSmessages can be delivered at any time, includingwhen the phone is turned off. We believe this isthe major reason that SMS messages pushed topersonal phones have a higher delivery successrate than voice messages.

During the first few months of implementa-tion, we varied the times that we pushed voicemessages to users to find times that clients weremost likely to be available to answer the phone.When tips and reminders were launched, voicemessages were attempted hourly between 12:00

FIGURE 2. Percentage of Successful Message RetrievalAttempts Among Retrieved Voice Message Subscribers, byType of Subscriber, September 2011–June 2013

TABLE 4. Phone-Based Survey Sample Characteristics, byDelivery Method

PushedSMS

(n596)

PushedVoice

(n530)

RetrievedVoice

(n5140)Total

(N5266)

No. of subscribers

Pregnancy messages 48 13 66 127

Child messages 48 17 74 139

No. of messagesreceived, mean 6.9 3.1 3.7 4.7

SMS subscriberswere significantlymore likely toreport intended oractual behaviorchange thanpushed orretrieved voicemessagingsubscribers.

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 8

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

pm and 6:00 pm local time (CAT) becauseanecdotal reports indicated that phones weremost likely to be charged and switched on duringdaylight hours. In November 2011, we pushedthe voice messages between 2:00 pm and 6:00pm because we noted more successful attemptsoccurred during that timeframe. In early 2012,we discovered a configuration problem betweenthe telephone card and the telecom provider thatwas also contributing to the low delivery successrate. After changing the timing that we pushedthe voice messages and fixing the configurationproblem, delivery success rates for pushed voicemessages increased, but the delivery success rateremained highest for pushed SMS messages.

Retrieved voice messages have the lowestdelivery success rate, and this rate decreased overtime. Retrieving voice messages from the IVRmenu using a community phone requires initia-tive on behalf of the client who must seek out avolunteer in order to access the communityphone. In addition, it requires the volunteer tobe available. There were some instances wherevolunteers moved away from their assignedvillage or became inactive over time. It is alsopossible that some community volunteers acted asgatekeepers to the service. We heard anecdotalreports of a small number of volunteers refusingto give out the CCPF telephone number or tellingclients that the service must be accessed through acommunity phone rather than a personal phone.

Furthermore, many community phones brokeduring the project period, limiting access to theservice for community phone users. The phonesgiven to community volunteers were low cost,and approximately 60% of the phones distributedto volunteers were not working 2 years after theoriginal implementation began. It is likely thatthe unreliability of community phones played arole in the low delivery success of retrieved voicemessages in 2 ways:

N Malfunctioning keypads made it difficult toenter an accurate estimated due date orchild’s birthday in order to hear a message.This is a major reason stated by bothvolunteers and users for why numerous callsthat reached the voice message menu neversuccessfully heard a message.

N Clients who signed up for voice messages in avillage where the phone eventually brokewere unlikely to find another way to accessthe service. Some users stated that after thecommunity phone in their village broke, they

no longer accessed the CCPF hotline or voicemessages.

The data also reveal that delivery success forretrieved voice messages is significantly higheramong users subscribed to pregnancy messagesthan users subscribed to child messages (basedon the definition of success used in Table 3).Pregnant users register an average of 16 weeksbefore their estimated due date while caregiversof children register an average of 36 weeks beforetheir child’s first birthday. It is possible that theaverage pregnancy-message enrollee was morelikely to take the initiative to visit a communityvolunteer because the eligibility period wasshorter than for the average child-messageenrollee. Similarly, it is less likely that acommunity volunteer would become inactive orthe community phone would break during ashorter eligibility period. For pushed messageenrollees, there is no clear relationship betweenmessage type and success rates because deliverysuccess of voice messages was significantlyhigher among pregnancy-message enrollees thanchild-message enrollees but the opposite wastrue for pushed SMS messages.

Although delivery success was higher amongpushed SMS messages compared with otherdelivery modalities, the data reveal that clientswere highly satisfied with and trusted the voiceand SMS messages equally and were likely tolearn new information and recall messagesregardless of message delivery modality. SMSenrollees, however, were significantly more likelyto report intended or actual behavior changethan pushed voice enrollees. We suspect that this

TABLE 5. Quality of User Experience, by Delivery Method

Outcome

PushedSMS

(n596)

PushedVoice

(n530)

RetrievedVoice

(n5140)Total

(N5266)

Acceptability 99% 94% 100% 99%

Comprehension 75% 63% 76% 75%

New information 79% 76% 71% 77%

Behavior change(intended or actual) 91% 56% 66% 74%

Missing information was excluded from the analysis. If a respondent answered thatthey had not received any messages, they were not asked questions about themessage service.

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 9

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

may be because voice messages can only be heardonce while SMS messages can be saved in theclient’s inbox and read multiple times, and evenshared with others. It is also possible that shorterand more frequent messages in the form of SMSare more effective at eliciting intended behaviorchange than longer voice messages that aredelivered once per week. Finally, the differingcharacteristics of individuals who sign up forSMS messages might make them more willing orable to change their behavior.

At baseline, this mHealth project discoveredthat most women in the project area did not own amobile phone. To address this problem, weprovided an additional access point by providingcommunity volunteers with mobile phones.Although the solution was not perfect, our studydemonstrates that it is feasible to provide low-literate clients without a personal phone withmobile health information and to stimulatebehavior change. Although delivery success wasthe lowest for retrieved voice message subscribers,approximately 1,200 voice messages were success-fully retrieved each month. A similar number ofpushed SMS and voice messages were success-fully delivered each month. This finding empha-sizes the importance of recognizing that mobilehealth messaging can potentially marginalizepopulations without access to mobile phones ifefforts are not made to adapt mHealth projects tomeet the needs of the population. Not surpris-ingly, phone ownership is inversely correlated

with poverty,21 and women and people living inrural areas are also less likely to own mobilephones than their male and urban counterparts.19

Thus, offering a retrieved voice message option forthose without personal phones is an importantstrategy to deliver health messages to an other-wise hard-to-reach population.

LimitationsThere are a number of important limitations toconsider when interpreting the data. First, datafrom the hotline software (such as demographicdata and tips and reminders enrollment status)were entered by hotline workers during callswith clients, and mistakes were common.Implausible data points, such as estimated duedates that were more than 9 months from thecall date, had to be excluded. In addition, thedata available from the IVR software is by calland cannot be grouped by unique caller, makingit difficult to draw conclusions about individualbehavior.

Second, our calculation for the number ofmessages retrieved by voice message subscribersprovides only an estimate and is not completelyanalogous to the success rate for pushed mes-sages. Pushed message success rates are based on3 attempts while the retrieved message successrate is based on an unknown number of attemptssince a subscriber can attempt to retrieve amessage an unlimited number of times.Furthermore, some subscribers likely do not try

Shorter and morefrequentmessages in theform of SMS maybe more effectiveat promptingbehavior changethan longer voicemessages.

BOX. Quality of User Experience: Selected Examples Reported by Survey Respondents

Recalled messages:

N I should breastfeed my child up to 6 months without giving supplementary foods.

N If the child has shown signs of drying in the mouth, I should run to the hospital.

New information:

N I have learnt how to solve small problems without going to hospital.

N The child is supposed to receive measles vaccine.

Intended or actual behavior change:

N The number of antenatal care visits I plan to attend.

N The types of foods I am eating.

N The age I will give my baby food other than breast milk.

N How often my child sleeps under a bed net.

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 10

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

at all. The proportion of message retrievalattempts where a message is successfully played(Figure 2), however, is based on only 1 callattempt and cannot tell us about individualbehavior. For example, if 50% of messageretrieval attempts are successfully played, wecannot determine whether this translates to 10unique callers attempting to retrieve a messageand only half are successful, or 5 unique callerseach trying to retrieve a message twice andfailing once and successful on their second try.

There were also a number of limitations tothe type of study design used for the phone-based surveys. First, the characteristics of enrol-lees that largely determined their enrollmenttype, such as literacy level and access to apersonal phone, could be associated with theirlevel of trust, comprehension, and knowledge,and/or behavior changes reported during thesurvey. Thus, any differences observed in surveyresponses between the different delivery methodsmay be attributable to characteristics of the sampleand not necessarily the delivery method. Inaddition, among retrieved voice message enrollees,the sample was not randomly selected. Althoughthe community volunteers did not know thepurpose of their assignment to identify a client, itis possible that they chose women according tocertain characteristics that may bias the sample.We did not collect enough demographic informa-tion during the phone surveys to be able to controlfor potential confounding characteristics. Also,information collected about the enrollees’ mater-nal and child health knowledge and/or behaviorchange intent relied on self-report, which maypresent an additional form of bias. Becausesurveys were conducted by CCPF staff or volun-teers, respondents may have wanted to provideanswers that they felt were acceptable to thesurveyor. Thus, satisfaction, trust, and behaviorchange indicators may be inflated. On the otherhand, respondents were asked to describe theirlast message and new information learned.Because the information provided was cross-checked against messages sent to the respondent,these indicators are less susceptible to bias. Finally,the sample size is small, making detection of smalldifferences in outcomes difficult.

CONCLUSION

Mobile technology, specifically SMS and voicesmessages, can be successfully used to extendhealth information services to pregnant women

and caregivers of young children in rural Malawi.All 3 message modalities led to high levels ofsatisfaction, comprehension, and new informa-tion learned. Due to lower cost, higher deliverysuccess, and higher levels of intent to changebehavior, SMS is the preferred delivery modalitywhen possible.

Implementing mHealth projects in areas withlow phone penetration creates unique challengesin ensuring access and generating demand. Byadapting the service to the context—allowingusers to retrieve voice messages from the IVRsystem without a personal phone and providingcommunity volunteers with phones to serve asaccess points to the service—this rural popula-tion gained access to the mHealth service thatwould not otherwise have been possible.Although these adaptations presented their ownchallenges, particularly apparent in the lowsuccess rate of retrieved messages, availabilityof community volunteers, and the difficultysustaining the volunteer model over time asphones began to break, providing multiplemethods by which users could access the servicewas crucial in extending reach beyond literatepersonal phone owners.

Further research to understand potentialmodels for increasing access to mobile phonesamong low-literate, rural populations is war-ranted. Prior to implementing a similar service,it is important to understand the cost-effective-ness of the intervention and cost implications at

© V

illageReach 2013

A woman uses a community volunteer’s mobile phone to access the healthmessage delivered by the Chipatala Cha Pa Foni project.

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 11

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct

scale. At the time of this study, an economicevaluation of CCPF was underway. Finally,further research examining the translation ofhealth information transmitted through mobilephones into healthy behaviors is needed tounderstand the potential impact of mHealthinterventions on their intended health outcomes.

Acknowledgments: This project was funded by Concern Worldwide’sInnovations for Maternal, Newborn & Child Health initiative.

Competing Interests: None declared.

REFERENCES1. National Statistical Office (NSO) [Malawi], ICF Macro. Malawi

demographic and health survey 2010. Zomba (Malawi): NSO;2011. Co-published by ICF Macro. Available from: http://www.measuredhs.com/publications/publication-FR247-DHS-Final-Reports.cfm

2. Combs Thorsen V, Sundby J, Malata A. Piecing together thematernal death puzzle through narratives: the three delays modelrevisited. PLoS One. 2012;7(12):e52090. CrossRef. Medline

3. Vital Wave Consulting. mHealth in the global south: landscapeanalysis. Washington (DC): Vodafone Group Foundation; 2008.Co-published by the United Nations Foundation Partnership.Available from: http://www.foresightfordevelopment.org/sobipro/download-file/46-448/54

4. Cole-Lewis H, Kershaw T. Text messaging as a tool for behaviorchange in disease prevention and management. Epidemiol Rev.2010;32(1):56–69. CrossRef. Medline

5. Sloninsky D, Mechael PN. Towards the development of anmHealth strategy: a literature review. New York: MillenniumVillages Project; 2008. Available from: http://www.who.int/goe/mobile_health/mHealthReview_Aug09.pdf

6. Free C, Phillips G, Felix L, Galli L, Patel V, Edwards P. Theeffectiveness of M-health technologies for improving health andhealth services: a systematic review protocol. BMC Res Notes.2010;3(1):250. CrossRef. Medline

7. Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. Theeffectiveness of mobile-health technology-based healthbehaviour change or disease management interventions forhealth care consumers: a systematic review. PLoS Med.2013;10(1):e1001362. CrossRef. Medline

8. Sharma R, Hebbal M, Ankola AV, Murugabupathy V. Mobile-phone text messaging (SMS) for providing oral health educationto mothers of preschool children in Belgaum City. J TelemedTelecare. 2011;17(8):432–436. CrossRef. Medline

9. Tamrat T, Kachnowski S. Special delivery: an analysis of mHealthin maternal and newborn health programs and their outcomesaround the world. Matern Child Health J. 2012;16(5):1092–1101. CrossRef. Medline

10. Evans WD, Wallace JL, Snider J. Pilot evaluation of the text4babymobile health program. BMC Public Health. 2012;12(1):1031.CrossRef. Medline

11. Lund S, Hemed M, Nielsen BB, Said A, Said K, Makungu MH, et al.Mobile phones as a health communication tool to improve skilledattendance at delivery in Zanzibar: a cluster-randomised controlledtrial. BJOG. 2012;119(10):1256–1264. CrossRef. Medline

12. Kaewkungwal J, Singhasivanon P, Khamsiriwatchara A, SawangS, Meankaew P, Wechsart A. Application of smart phone in‘‘Better Border Healthcare Program’’: a module for mother andchild care. BMC Med Inform Decis Mak. 2010;10:69. CrossRef.Medline

13. GSMA Mobile for Development Intelligence (MDI). Scalingmobile for development: a developing world opportunity.London: MDI; 2013. Available from: https://d2sjbwdjcc5vgl.cloudfront.net/Scaling_Mobile_-_interim_publish_MDI_template.pdf

14. GSMA Mobile for Development Intelligence (MDI). Value of voice– an unsung opportunity. London: MDI; [2013]. Available from:https://wiki.gsmaintelligence.com/gsma_kb/images/2/20/Value_of_Voice_-_an_unsung_opportunity.pdf

15. MAMA: Mobile Alliance for Maternal Action: the power ofhealth in every mama’s hand [Internet]. Washington (DC): UnitedNations Foundation; c2013. MAMA Bangladesh;[cited 2013 Jan 7]; [about 3 screens]. Available from: http://www.mobilemamaalliance.org/mama-bangladesh

16. Greaney ML, Puleo E, Sprunck-Harrild K, Bennett GG,Cunningham MA, Gillman MW, et al. Electronic reminders forcancer prevention: factors associated with preference forautomated voice reminders or text messages. Prev Med.2012;55(2):151–154. CrossRef. Medline

17. Grameen Foundation. Mobile technology for community healthin Ghana: what it is and what Grameen Foundation has learnedso far. 2nd edition. Washington (DC): Grameen Foundation;2012. Available from: http://www.grameenfoundation.org/sites/grameenfoundation.org/files/MOTECH-Lessons-Learned-Sept-2012.pdf

18. Cormick G, Kim NA, Rodgers A, Gibbons L, Buekens PM,Belizan JM, et al. Interest of pregnant women in the use of SMS(short message service) text messages for the improvement ofperinatal and postnatal care. Reprod Health. 2012;9:9.CrossRef. Medline

19. Wesolowski A, Eagle N, Noor AM, Snow RW, Buckee CO.Heterogeneous mobile phone ownership and usage patterns inKenya. PLoS One. 2012;7(4):e35319. CrossRef. Medline

20. Invest in Knowledge. Baseline report for an evaluation of theinformation and communications technology to improve healthservices for mothers and children project. Zomba (Malawi):Invest in Knowledge; 2012.

21. Noordam AC, Kuepper BM, Stekelenburg J, Milen A.Improvement of maternal health services through the use ofmobile phones. Trop Med Int Health. 2011;16(5):622–626.CrossRef. Medline

______________________________________________________________________________________________________________________________________________Peer Reviewed

Received: 2013 Oct 17; Accepted: 2013 Dec 17

� Crawford et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricteduse, distribution, and reproduction in any medium, provided the original author and source are properly cited. To view a copy of the license, visithttp://creativecommons.org/licenses/by/3.0/______________________________________________________________________________________________________________________________________________

SMS versus voice messaging to deliver health information in Malawi www.ghspjournal.org

Global Health: Science and Practice 12

10.9

745/

GH

SP-D

-13-

0015

5 A

dvan

ce A

cces

s A

rtic

le p

ublis

hed

on J

anua

ry 2

8, 2

014

as d

oi:

Glo

b H

ealth

Sci

Pra

ct