Technical Assistance Consultant’s Report

This consultant’s report does not necessarily reflect the views of ADB or the Government concerned, and ADB and the Government cannot be held liable for its contents. (For project preparatory technical assistance: All the views expressed herein may not be incorporated into the proposed project’s design.

Project Number: 41078 May 2010

Lao People's Democratic Republic: Piloting Community e-Centers for Better Health (Financed by the Republic of Korea e-Asia and Knowledge Partnership Fund)

Prepared by Alexis de Morsier, ECOS Laos

1

ADB TA-7058 LAO

Piloting Community e-center for Better Health

E-Health Centers M&E Final Report

Consultant: Alexis de Morsier alexis@ecoslaos.com Position: Evaluation Specialist Period: March to April 2010 Vientiane, 28 April 2010

2

ACRONYMS

ADB - Asian Development Bank DH - District hospital DHO - District Health Office Lao PDR - Lao People’s Democratic Republic HC - Health Center HMIS - Health Management Information System ICT - Information and communications technology IPD/OPD - Inpatient/Outpatient Data MoH - Ministry of Health NAST - National Authority Science Technology NSC - National statistic Center PH - Province hospital PHC - Primary Health Care PHO - Province Health Office PSTO - Provincial Science and Technology Office SVK - Savannakhet province TA - Technical assistance VHW - Village Health Worker VTE - Vientiane Capital XKG - Xieng Khuang province

Front page photo: Health staff using email software in Nakala HC

3

CONTENTS

Acronyms ................................................................................................................................................................ 2 Contents ................................................................................................................................................................. 3 Purpose ................................................................................................................................................................... 4

M&E Outputs .................................................................................................................................................. 4 M&E Reference Documents ........................................................................................................................... 4

Project Milestone ................................................................................................................................................... 5 Output 1: Development of e-health center network ............................................................................................. 6

The quantity of data transferred using Internet connection .......................................................................... 7 Location of e-centers and Internet communication quality ........................................................................... 9 Internet Service Provider (ISP). ..................................................................................................................... 10 Communication Costs ................................................................................................................................... 10 consumables ................................................................................................................................................. 12 Community Access to E-center ..................................................................................................................... 12 ICT Services ................................................................................................................................................... 12 Output 1: Indicators ...................................................................................................................................... 12

Output 2: Development of e-Health Content and Services .................................................................................. 13 Web Site: www.health-ecenter.gov.la .......................................................................................................... 13 WEB Site Management ................................................................................................................................. 14 Availability of e-health contents CD-ROMs .................................................................................................. 14 Users Feedback regarding use and quality of e-health contents ................................................................. 15 Output 2: Indicators ...................................................................................................................................... 15

Output 3: Capacity Building .................................................................................................................................. 16 ICT Training ................................................................................................................................................... 17 E-health education and computer exploring training ................................................................................... 17 Training Sessions .......................................................................................................................................... 18 Refresher Trainings Courses (RTC) ................................................................................................................ 19 Output 3: Indicators ...................................................................................................................................... 20

Output 4: Monitoring & Evaluation ...................................................................................................................... 21 E-Center reporting Framework ..................................................................................................................... 21 Electronic reporting forms ............................................................................................................................ 21 E-Center monitoring Forms .......................................................................................................................... 22 Provincial monitoring Team .......................................................................................................................... 22 Output 4: Indicators ...................................................................................................................................... 22

Outcome: .............................................................................................................................................................. 23 Outcome Indicators ...................................................................................................................................... 23

Impact: Improved health in rural communities .................................................................................................... 25 Conclusions ........................................................................................................................................................... 25

Output 1: Recommendations ....................................................................................................................... 26 Output 2: Recommendations ....................................................................................................................... 26 Output 3: recommendations ........................................................................................................................ 27 Output 4: recommendations ........................................................................................................................ 27

4

PURPOSE

This report presents the final evaluation of the TA 10780 – Piloting Community e-centers for better health project based on activities carried out by the M&E specialist from July 2008 to March 2010. M&E activities where carried in both Xieng Khuang and Savannakhet provinces during: (i) the baseline survey, (ii) provincial inception workshops, (iii) ICT training and e-centers installation (iv) the final survey and (v) final provincial and national workshops. This final evaluation is based on (i) original data collected during M&E activities, (ii) reports provided XMG, (iii) data issued by NAST, (iv) ADB mission reports and (v) e-centers users’ feedback collected during workshop.

M&E OUTPUTS This table list all documents produced by the M&E specialist during the course of the project. Activities M&E Outputs Date Monitoring design M&E Framework plan report September 2008 Monitoring design E-Center facilities monitoring Forms – (Lao version) September 2008 Monitoring design E-Center ICT evaluation Forms – (Lao version) September 2008 Baseline survey SVK Baseline survey preliminary report October 2008 Baseline survey XKG Baseline survey preliminary report October 2008 Baseline survey Final Baseline survey report November 2008 Costs evaluation E-center Equipment and Costs evaluation report May 2009 Monitoring design Monthly e-Health activities reporting forms October2009 Monitoring design Monthly e-Center activities reporting forms October2009 Implementation XKG E-center implementation report -(Annexes) – (HMIS) October2009 Implementation Final E-centers implementation report November 2009 Monitoring design Final Survey monitoring forms March 2010 Monitoring design M&E Final workshop draft for discussion March 2010 Final survey Final E-center final evaluation report April 2010

Table 1: Documents produced by M&E specialist

M&E REFERENCE DOCUMENTS This table lists all documents used during the M&E final evaluation. Sources Documents Date ADB TA Lao Piloting Community e-Centers for Better Health (PDF 64 Kb) January 2008 XMG Presentation to Committee Members Updated (07.23.08) (PDF 2Mb) July 2008 XMG Training Days Calculation-v1.0-DRAFT (XLS 20 kb) July 2008 XMG TA 7058 Lao Inception Report-v1.0-FINAL (Doc 879Kb) July 2008* XMG TA 7058 Detailed Implementation Plan XMG-v3 0 (Doc 109Kb) July 2009 XMG Report on Baseline survey in Xieng Khuang November 2008 ADB ADB Review Mission (Nov) (PDF 384 Kb) November 2008 XMG ICT Training Number of Participants v1 7(04032008) (Doc 541 Kb) April 2009 XMG Procurement Doc for Hardware e-Center Project (Doc 440 Kb) May 2009 XMG ICT Course Curriculum and Training Plan-V1.0-FINAL (PDF 983 Kb) July 2009 NAST Original Equipment deliveries documents to provinces (PDF 384 kb) September 2009 XKG PSTO Original XKG ICT training participant list (PDF 510 Kb) September 2009 SVK PSTO Original SVK ICT training participant list (PDF 391 Kb) October 2009 NAST ICT training participant list (PDF 77 Kb) 2009 XMG ICT Refresher Training Final report (Doc 824 Kb) March 2010 XMG Lao PCR outline v2 (LV input - 03 29 10) (Doc 225 Kb) March 2010 XMG Training Report Initial Training v1.0 FINAL (02.01.10) (Doc 977Kb) March 2010

5

ADB ADB Review Mission 29 March 2010 (Doc 127 Kb) March 2010 Table 2: Project’s documents used by M&E

PROJECT MILESTONE

Project period Activities Outputs Completed Inception Period Inception meeting Project presentation

Presentation by XMG, e-Health content production firm and Monitoring & evaluation specialist

July 2008

Pilot site selection Selection of pilots sites by provincial health office

August 2008

Coordination meetings between NAST, XMG and M&E specialist for base line survey preparation.

Official introduction document for provincial authorities

September 2008

Base line survey preparation

Assessment survey tools development by XMG and M&E specialist

September 2008

Base line survey SVK survey (6 days) XKG survey (5 days) Report redaction

Assessment of every pilot sites Baseline and assessment report Detailed list of required equipment

October 2008

Inception workshop

workshop in SVK and XKG (2x2 days)

Detailed Presentation of project, equipment, training course, M&E, and provincial implementation plan Final list of trainee for every course

December 2008

Equipment Procurement

Bidding process Preparation of bidding document for all equipment Evaluation and selection of best offers, equipment order

July 2009

Equipment delivery to province

Delivery of equipment to PSTO on each province

September 2009

Training Maintenance training (2x3days)

Training by XMG. Training on computer preparation and maintenance

September/October 2009

Application training (2x2 days)

Training by XMG. Maintenance trained staff will act as facilitator for application training

September/October 2009

Basic Training (2x5 days)

Training by XMG Maintenance trained staff will act as facilitator for basic training

September/October 2009

Delivery Delivery to e-center sites (2x10 days)

Delivery to district hospital and health center. XMG and Maintenance trained staff will install and test e-center functionality

September/October 2009

Implementation Monitoring

Visit of e-center sites (10 days)

Monitoring of delivered training and ICT equipment

September/October 2009

Refresher Training

Delivery to e-center sites (2x10 days)

Refresher training by XMG in each e-center

February/March 2010

Final Evaluation Survey

(10 days) Final assessments of e-centers March 2010

Final Provincial Workshop

(2x2 days) Presentation of project activities and findings including users feedback

March 2010

Final National Workshop

(2 days) Presentation of project activities and findings including users feedback

March 2010

Final Report (3 days) Meeting with XMG, reports redaction

April 2010

Table 3: Project milestone

6

OUTPUT 1: DEVELOPMENT OF E-HEALTH CENTER NETWORK

• Functionality of e-center ICT equipments

All 13 visited e-health centers (out of 16) where functional and operational and well maintained at the time of the final survey.

In HCs and DHs without photocopy machine (10 e-centers out of 13 visited e-centers) the All-In-One printer was found with a Low Toner Warning indicating that the toner cartridge was empty.

No problem regarding ICT equipment where reported by e-center staff or provincial monitoring team.

• The quality of internet connection

As result of the change of implemented architecture (replacement of modem due to unavailability in Lao PDR), the bandwidth of the Internet communication has been reduced by more 50% (from 156 Kb/sec to 56 Kb/sec) and signal quality by 75% (replacement modem without antenna)

Left: Winphone modem with external 20cm antenna Right: USB GPRS modem with integrated 5cm antenna

7

THE QUANTITY OF DATA TRANSFERRED USING INTERNET CONNECTION The quantification of internet usage by every e-health center can be evaluated by monitoring the communication traffic data. The software provided with the GPRS modem allow to monitor the following Internet connection indicators:

1. Daily connection duration 2. Daily amount of data

sent/receive 3. Total connection duration 4. Total amount of data

sent/received

Statistic screen of GPRS modem software

The number of email messages sent/received can also give a indication regarding the use of ICT services. For DH this number is only indicative because most of users prefer to use their own public email address (Gmail, MSN, Yahoo) rather than project provided email address.

e-center Start Date

End Date

Duration Data Sent

Data Received

Mail Messages

Phoukhout DH 24-Sep-09 10-mar-10 4h30 5.3 Mb 25Mb < 10 Napha HC No data * Phaxay DH 3-sep-09 9-mar-10 10 h. 4.9 Mb 66.8 Mb > 10 Ladsen HC 3-sep-09 9-mar-10 5.3 min. 1.3 Mb 3.3 Mb - Khoun DH 28-Feb-10 10-mar-10 1h30 1.3 Mb 4.6Mb < 20 Khet gnoun HC No usage data collected due to low connectivity** Kham DH No data * Khethai HC No data * Atsaphonthong DH 1-mar-10 15-mar-10 1h. 500 Kb 1 Mb < 10 Nachan HC 12-mar-10 16-mar-10 2h30 2Mb 14Mb >20 Champhone DH 1-dec-09 17-mar-10 30h25 23Mb 139Mb >30 Taleo HC 10 mar-10 17-mar-10 3h51 1.4Mb 7.4Mb >30 Sepone DH No data * Dong savanh HC No data * Songkhone DH 27-oct-09 18-mar-10 9h15 10Mb 55.5 Mb > 20 Nakala HC 26-Oct-09 18-mar-10 8h20 3.2Mb 46Mb > 30

Table 4: Quantity of ICT communication by e-health center * 5 e-health center where not visited during the final survey due distances and time constraint.

8

** No connectivity during pilot period and very low connectivity after replacement of the ISP during refresher training.

9

•

LOCATION OF E-CENTERS AND INTERNET COMMUNICATION QUALITY The followings maps show the quality of internet communication at the end of the project

Notes: Phoukhout DH is using a newly installed telephone line to access the internet

Notes: Sepone DH is using an existing Winphone to access the internet. Adtsaphonthong DH is still using telephone line for internet access

Bad Internet connexion

Good Internet connexion

Bad Internet connexion

Good Internet connexion

10

INTERNET SERVICE PROVIDER (ISP). There are now three ISPs operating in the pilot provinces—Lao Telecommunications (LTC), Enterprise Telecommunications Lao (ETL) and Starcom. The e-centers used the ISP that provided the best coverage in their area. Independent of the ISP, some sites have better connectivity than other e-centers due to distance to the actual antennae of the ISP. Source: (ADB Aide memoire – 29 March 2010)

COMMUNICATION COSTS

The following table show that the cost for all wireless internet communication has a standardized cost of 1’000 kips per Mb regardless of the speed of the connection. The coverage of 3G network is growing rapidly on both provinces.

ISP costs comparison for Internet communication (March 2010)

Protocol Speed Paiment method Cost of communication Telephone line Dial-Up 56 Kb/sec Prepaid card 5’000kips/hours Telephone line Dial-Up 56 Kb/sec Monthly 160'000 Kips/month Mobile phone GPRS 56 Kb/sec Prepaid card 1’000Kips/Mb Winphone CDMA 128 Kb/sec Prepaid card 1’000Kips/Mb Mobile phone 3G 384 Kb/sec Prepaid card 1'000 Kips/Mb ADSL ADSL 128 Kb/sec Monthly 300'000 Kips/month ADSL ADSL 256 Kb/sec Monthly 500'000 Kips/month ADSL ADSL 512 Kb/sec Monthly 750'000 Kips/month

Table 5: Internet communication costs

DH and HC received a monthly communication support of 100’000 kips from the e-Health Center project. PHD used existing ADSL connection supported by other project. No information has been collected regarding PSTO communication support.

Example of Free Medical videos available at http://www.3rbdr.com/video/category/4/Obstetrics

If the communication support of 100’000Kips seem sufficient for covering email communication costs, experiences has show that DH with good internet connection (Sepone-Phoukhout) can spent more than 100’000Kips in one day by downloading 60 minutes of medical videos from internet (YouTube, DailyMotion, Free Medical video…) or by browsing medical sites in Thai or English language.

11

12

CONSUMABLES

The final survey has show that the All-In-One printer was heavily used as printing and photocopy machine. (6 of 8 HCs and 4 of 8 DHs had a “Low Toner” warning on printer control panel.)

Phaxay DH has bought new toner cartridge with their budget - at the cost of 950’000 Kips (~100 USD). Market price for toner cartridge can range from 200’000 Kips (refilled cartridge) to 950’000 Kips (original brand cartridge).

After M&E feedback to NAST regarding the “Low Toner”, one new toner cartridge was distributed to each HCs and DHs e-center during final provincial workshop.

COMMUNITY ACCESS TO E-CENTER

The regulation approved on 1 December 2009 provided the operating procedures for the e-center. The regulation covers the operating hours, charge rates, and fund management

Link: Regulation in Lao language Link: Regulation in English language

Access to e-center is allowed during working hours but difference can be seen between HC and DH

o HCs e-centers are open to HC staff, VHWs and general population on request. o DHs e-centers are open to DH staff and VHWs during normal working time.

In some DHs, staff living in the hospital can also access the e-center after normal working time. In HCs, e-center where open outside of official hours since staff live nearby and were very happy to support the community demand.

SUSTAINABILITY OF ICT SERVICES

ICT services provided by HCs can eventually generate enough income for sustaining communication costs. However, provincials and districts authorities have proposed to support HC for other ICT related costs (Toner, repairs…).

Both NAST and MOH have indicated that they will seek further support from international donors to expand the e-center concept into other provinces

OUTPUT 1: INDICATORS 'Increased percentage of health facilities providing ICT services for at least 50% of the time' The quality of internet connection has been improved in 5 e-health center by replacing the IPS provider during refresher training. At the end of the pilot period, only 1 health center (Ketghnoun) still has internet connectivity problems. The monitoring has show that 15 out of 16 (94%)of DHs and HCs e-centers were providing basic ICT services during official working time at the end of the pilot period.

13

OUTPUT 2: DEVELOPMENT OF E-HEALTH CONTENT AND SERVICES

WEB SITE: WWW.HEALTH-ECENTER.GOV.LA The website design was approved after MOH and NAST ensured the website was consistent with Lao PDR Government health and ICT standards. The website has both TA specific content and health information. It is hosted on the TA server… It was decided that CIEH has overall responsibility for the management of the website supported by the website content MOH Committee. Source: (ADB Aide memoire – 29 March 2010)

Web site home page (11 November 2009)

Web site home page (19 March 2010)

At the time of the report, no detailed statistic data regarding web site usage as been published. However the hits counter in the web site home page display the number of visits and the numbers of articles of the web site (41 articles and 768 visits in November 2009 and 50 articles and 1028 visits in March 2010)

14

WEB SITE MANAGEMENT XMG Global with assistance from CIEH prepared appropriate health materials and loaded them onto the website. MOH held a workshop on 3 October 2009 for the MOH staff who are responsible for updating the website content. The engagement of this group is important for the sustainability of the website. It was decided that CIEH has overall responsibility for the management of the website supported by the website content MOH Committee. Draft guidelines for approval of material are yet to be approved. However, even without the guidelines, the website is being updated e.g. information on the launch, refresher training and final TA workshops have been up loaded onto the website. Source: (ADB Aide memoire – 29 March 2010)

AVAILABILITY OF E-HEALTH CONTENTS CD-ROMS

Two of three planned e-health content CD-ROMs were produced and distributed to e-center. Original CD-ROMs are kept in PHD for further replication (distribution to other DH)

Content of these interactive CD-ROMs can be classified as:

• Interactive template including self learning and auto-evaluation tools

• Thematic videos produced by Item Interactive

• Audios files in local language

a. Maternal and Newborn Health (MNH)

b. Village Health Volunteers (VHW) Basic Skills

These CD-ROMs were demonstrated and used during ICT training ensuring that every participant can used e-health content for self training or e-health education sessions. These CD-ROMs where also demonstrated on each e-center after equipment installation.

A copy of the immunization CDROM prepared in 2006 under the ADB supported Communicable Disease Control project, was distributed to PHDs. Each PHD is responsible for the duplication of the CD-ROM and its distribution to DHs and HCs e-health center.

15

USERS FEEDBACK REGARDING USE AND QUALITY OF E-HEALTH CONTENTS

Discussions held with DH and HC staff and some VHW during final survey and comment issued during final provincial workshops allow following comments regarding e-health content:

o Project WEB Site

Even if the project web site is in Lao language, the lack of updated content reduces the interest from e-center users to use it.

o Produced CD-ROMs

Interactive CD-ROM’s have been well evaluated by DHs and HCs staff as tools for self training and VHWs education. However, VHWs have a more mitigated evaluation: (i) video in local languages are much appreciated but (ii) the requirements of a computer reduce the dissemination of e-health education to targeted populations. A general request from VHW’s and community after e-health content training is to have CD/DVD-ROMs version of e-health content videos for a wide dissemination through the community. Nearly every household have TV and CD/DVD player set.

o Internet medical WEB sites

User’s feedback during final workshops has make obvious that the access to medical sites available, are limited to health staff with good English skill and e-centers with high speed internet connection. However all users has understand the value of e-health content freely available on the internet.

OUTPUT 2: INDICATORS 'E-health content, including three CDROMs, has been demonstrated as effective tools for skills training' MNH CD-ROM has been generally presented to DHs and HCs medical staff during E-health education and computer exploring courses. A total of 80 medical staff has followed training sessions. VHW CD-ROM has been presented to VHWs during E-health education and computer exploring courses. A total of 216 VHW has followed training sessions. Discussion with DH, HC and some VHWs during final survey has shown real interest in the produced CD-Roms. No feedback was received from the Health Specialist regarding the effectiveness of CD-ROMs. Demonstrated CD-ROMs

Written materials

Video Target staff Total Participant

MNH Lao - English Lao, English DH, HC 80 VHW Lao - English Lao, English, Hmong, Khmu and

Lao Theung VHW 216

Immunization No data Lao, English DH, HC, VHW No data Table 6: CD-ROMs used for skill training

16

OUTPUT 3: CAPACITY BUILDING

Different courses were designed for different groups depending on their role and function in the e-center project. The planned initial training program is detailed in ICT Course Curriculum and Training Plan, but at the request of provinces and with agreement of the participants, the maintenance and the application training were merged and delivered together to PSTOs, PHDs and DHs participants. Training courses Target staff Duration Target

Participant Basic network administrator NAST - CIEH 2 days 16 WEB administrator training, NAST - CIEH 2 days 8 WEB content update training NAST - CIEH - PHD 1 days 8 Computer maintenance and application training PHD - PSTO - DH 5 days 24 Basic training HC 5 days 16 E-health education and computer exploring courses DH-HC-VHW 1 days - Total planned participant 72

Table 7: Training courses names, targeted staff and duration

Basic training for HCs staff in Xieng Khuang PSTO

training rooms

Mr. Ole (NAST) and Miss Linda (CIEH) acting as facilitator during ICT training

E-health education and computer exploring courses in

Phoukhout DH VHWs following E-health education and computer

exploring courses in Khetthai HC

17

ICT TRAINING

The baseline survey as showed the lack of ICT knowledge for DHs and HCs health staff and PSTOs technical staff. The following table shows the total number of training participant by training courses and type of e-centers. PSTOs, DH, and HCs staff have mostly of beneficiated of ICT training regarding the number of training duration (5 days) and quantity of trained staff (>15).

E-center Training Duration

in day Planned

Trained staff Actual

Trained staff Trained

female staff NAST Network Administration training 2 4 4 3

WEB content update training 1 4 4 3 CIEH

Network Administration training 2 4 4 3 WEB content update training 1 4 4 3

PSTO Computer maintenance and application training *

3+2 16 18 7

PHD Computer maintenance and application training *

3+2 8 8 0

DH Computer maintenance and application training *

3+2 16 17 5

HC Basic training 5 16 16 7 Total 72 75 31

Table 8: ICT Trained staff by trainings courses and e-center type

* Trainings sessions in Xieng Khuang has showed that the same participants were following the Basic computer maintenance (3 days) training and the Basic application training (2 days). To simplify procedures, these two trainings were merged into a single training session in Savannakhet province. Please note that these numbers varies from the NAST statistics. In the NAST table, DHs and HCs staff has followed all 4 trainings (Basic computer troubleshooting, Application training, Health Education and Basic computer training) but in reality, HCs staff have only followed Health Education and Basic computer training (5days+1days) and DHs staff only followed the Basic computer troubleshooting training and the Application training.

E-HEALTH EDUCATION AND COMPUTER EXPLORING TRAINING

E-health education and computer exploring training were provided on-site, after the installation of e-center equipment. In DHs, these training sessions have mostly involved all DHs staff identified as having basic ICT knowledge (see baseline survey). In HCs, training sessions have involved all HCs staff and present VHWs.

Facility Staff

Training Duration in day Trained staff

VHW

DH E-health education and computer exploring courses

8 x 1 day 55 -

HC E-health education and computer exploring courses

8 x 1 day 25 216

Total 80 216 Table 9: Trained staff by e-center type

18

ICT TRAINING SESSIONS

The following tables resume trainings date and participants.

Duration Location Datec Trained

staff Females

XKG Training sessions Basic computer Maintenance and Application

5 days PSTO 7-11 Sep 2009 16 7

Basic computer 5 days PSTO 14-17 Sep 2009 17 7 E-health education and computer exploring courses

1 day Phaxay DH 21 Sep-2 Oct 2009 1 day Latsene HC 21 Sep-2 Oct 2009 27 1 day Khetthai HC 21 Sep-2 Oct 2009 11 1 day Phoukout DH 21 Sep-2 Oct 2009 1 day Napa HC 21 Sep-2 Oct 2009 10 1 day Kham DH 21 Sep-2 Oct 2009 1 day Khoun DH 21 Sep-2 Oct 2009 67 1 day Khetnghoun HC 21 Sep-2 Oct 2009

SVK Training sessions Basic computer Maintenance and Application

5 days PSTO 12-16 Oct 2009 19 5

Basic computer 5 days PSTO 19-23 Oct 2009 17 7 E-health education and computer exploring courses

1 day Sepol DH 26 Oct 2009 1 day Dongsavanh HC 27 Oct 2009 24 1 day Atsaphangthong DH 28 Oct 2009 1 day Nachan HC 29 Oct 2009 21 1 day Champhone DH 02 Nov 2009 1 day Taleo HC 03 Nov 2009 24 1 day Songkhone DH 04 Nov 2009 1 day Nakala HC 05 Nov 2009 57

Vientiane Training sessions Web administrator NAST 5-6 March 2010 8 3 Web content update NAST 7 March 2010 8 3 Basic administrator NAST 8-9 March 2010 8 3

Table 10: Training dates and number of participants Source: ICT training participant list (NAST Sep-Oct 09).

19

REFRESHER TRAININGS COURSES (RTC)

Objective of refresher training are quoted by XMG:

For participants who attended the courses [], the RTC desired outcome is: 1. To focus in providing a refresher of skills and knowledge to continually ensure skills development. 2. To provide the opportunity for a ‘one-on-one’ with participants to address any specific concerns or

issues they have with the use of technology or with health education content. 3. To select proficient and highly competent users that can be developed as a Super User to help in future

training and continuing education at the CEC.

Source: XMG Training report for RTC and Web Admin (page 5)

E-center Training Duration in

day Trained

staff Trained

female staff PSTO Computer maintenance and application training 1 14 12 PHD Computer maintenance and application training 1 12 8 DH Computer maintenance and application training 1 59 27 HC Basic training 1 36 16 Total 121 63

Table 11: Refresher training participants by e-center type Province Duration Training location Training Date Trained

staff Trained

female staff Xieng Khuang

1 day PSTO 4-Mar 2010 7 5 1 day PHD 5-Mar 2010 5 1 1 day Phaxay DH 22-Feb 2010 9 5 1 day Latsene HC 23-Feb 2010 6 5 1 day Khetthai HC 24-Feb 2010 7 2 1 day Phoukout DH 25-Feb 2010 7 2 1 day Napa HC 26-Feb 2010 6 1 1 day Kham DH 1-Mar 2010 10 5 1 day Khoun DH 2-Mar 2010 3 0 1 day Khetnghoun HC 3-Mar 2010 3 1

Savannakhet

1 day PSTO 8-Mar 2010 7 7 1 day PHD 19-Mar 2010 7 7 1 day Sepol DH 16-Mar 2010 3 0

1 day Dongsavanh HC 10-Mar 2010 4 4

1 day Atsaphangthong DH 11-Mar 2010 8 5

1 day Nachan HC 12-Mar 2010 5 0

1 day Champhone DH 15-Mar 2010 14 8

1 day Taleo HC 18-Mar 2010 3 1

1 day Songkhone DH 17-Mar 2010 5 2

1 day Nakala HC 9-Mar 2010 2 2

Total 20 days 121 63 Table 12: Refresher training sessions dates and number of participants

20

OUTPUT 3: INDICATORS

Number of computer literate health staff increased by 50% in targeted health facilities

The following tables show the number of staff using computer and email in each e-center before and after the project implementation. The computer capacity of DHs health staff has increased by 63% during the course of the project. Because HCs staff have no computer capacity at the beginning of the project, the computer capacity has increased by 200%. District Hospitals Before implementation

Oct - 2008 After implementation

Mar - 2010 Computer literate

increase Total

Staff Using

computer Using Email

Total Staff

Using computer

Using Email

Using computer

Using Email

Phaxay 34 4 1 36 7 2 75% 100%

Phoukout 34 2 0 36 8 2 300% 200%

Kham 30 5 1 30 8 2 80% 100%

Khoun 30 4 0 33 8 2 100% 200%

Champhone 104 12 0 104 15 4 25% 400%

Songkhone 68 12 0 68 20 2 66% 200%

Atsaphangthong 49 8 1 49 12 2 50% 100%

Sepon 58 8 1 58 12 4 50% 300%

Total Staff 408 55 4 449 90 19 63% 375%

Table 13: Number of computer literate DH health staff increase Health Centers Before implementation

Oct - 2008 After implementation

Mar - 2010 Computer literate

increase Total

Staff Using

computer Using Email

Total Staff

Using computer

Using Email

Using computer

Using Email

Latsene 3 0 0 3 2 2 200% 200%

Khetthai 2 0 0 2 2 2 200% 200%

Napa 3 0 0 3 2 2 200% 200%

Khetnghoun 3 0 0 3 2 2 200% 200%

Nachan 3 0 0 3 2 2 200% 200%

Taleo 4 0 0 4 2 2 200% 200%

Nakala 3 0 0 3 2 2 200% 200%

Dong Savanh 4 0 0 4 2 2 200% 200%

Total Staff 28 0 0 28 16 16 200% 200%

Table 14: Number of computer literate HC health staff increase

21

OUTPUT 4: MONITORING & EVALUATION

E-CENTER REPORTING FRAMEWORK

Proposed reporting levels and information flow (M&E Framework plan – July 2008)

ELECTRONIC REPORTING FORMS

Specific electronic reporting form has been developed for monitoring monthly e-center’s activities and copied on every computer.

Link: E-Health reporting form (Excel 2007 format with cells protection) Link: E-Center activities Reporting form (Excel 2007)

The E-Health reporting form follow the Lao HMIS standard has been validated by the MoH. These forms have not been used as described in the monitoring framework (due to communication problems and limited reporting training). However, existing HMIS paper reports were used for data collection.

Health activities indicators on the walls of the Nachan

HC (baseline survey) DH staff compiling monthly health activities reports

received from HC

22

E-CENTER MONITORING FORMS Several monitoring forms were designed and used by the M&E specialist to capture ICT and health related data during baseline and final surveys. Monitoring forms: Link: Baseline survey tools in English (PDF) Link: Baseline survey tools in Lao (PDF) Link: ICT Survey tools in English (PDF) Link: ICT Survey tools in Lao (PDF) Link: Final survey tools (Word 2007)

PROVINCIAL MONITORING TEAM Provincial monitoring teams were organized by PHDs and PSTOs and conducted monthly monitoring trips to support e-centers. Due to TA time constraint and a lack of coordination with the M&E specialist regarding the implementation and support of provincial monitoring team, they did not receive any training or information regarding monitoring procedures and were not able to report efficiently e-center activities.

OUTPUT 4: INDICATORS ‘The M&E specialist report will be disseminated by end of April 2010 to NAST, MOH, PSTO, PHD’ After approval by ADB, this M&E final report was sent by e-mail to ADB, NAST, CIEH, PSTOs and PHDs by the 10-May-2010. ‘It will also be loaded onto the ADB website.’ After revision and approval by ADB, M&E related documents have been loaded into the ECOS Laos website and related documents are available through hyperlinks. The CIEH will be responsible for uploading the present report onto the e-center project web site.

23

OUTCOME:

The TA had two related outcome: 1. ' Number of computer-literate health staff increased by 50% in targeted health facilities’

This indicator was exceeded as presented in the Output 3: Indicators section of this report. 2. 'Utilization of health facilities and VHWs improved by 25%'

The short time for the monitoring period (4 months) and Internet access limitation for e-reporting has limited the possibility to calculate accurately the increase of utilization of health facilities. The construction of new building, foreign support or pandemic events can drastically affect the utilization of Health Facilities.

However, the evaluation of health facilities utilization can be reached by analyzing the number of deliveries by month in health facility. These data were collected during the final evaluation survey and final provincial workshops.

OUTCOME INDICATORS

60% of health activities data collected can be used for evaluating the improvement of utilization of health facilities. (3 HCs has no delivery service, XKG DHs data not complete).

Analyze show that the utilization of Districts hospitals regarding deliveries service has increased between 14 to 62 percent. In the 5 Health Centers with delivery service, the increase varies from 27 to 120 percent. It can be assumed that the implementation have more impact on HCs than DHs regarding deliveries.

District hospitals Period Nov-2008 to Feb 2009

Period Nov-2009 to Feb-2010

Activities increase

OPD IPD Birth Alive

OPD IPD Birth Alive

OPD IPD Birth Alive

Phaxay No data 1643 106 - Phoukout 828 221 - - - 16 Kham 1566 - 370 No data

Khoun 882 34 34 No data Champhone 2240 756 116 2772 1049 136 24 % 38 % 17 % Songkhone 557 337 69 706 373 112 27 % 11 % 62 % Atsaphangthong 2157 588 57 2038 738 68 -5 % 25 % 19 % Sepon 1897 1094 110 2594 1525 125 36 % 39 % 14 %

Table 15: Improvement in utilization of District Hospitals Health Centers Period Nov-2008 to Feb

2009 Period Nov-2009 to Feb-

2010 Activity increase

OPD IPD Birth Alive

OPD IPD Birth Alive

OPD IPD Birth Alive

Latsene 68 4 - 108 2 - 59% -50% - Khetthai No data Napa 184 38 4 380 101 6 106% 165% 50% Khetnghoun 41 3 - 147 10 - 258% 233% - Nachan 170 45 - 233 41 - 37% -9% - Taleo 806 19 18 575 11 23 -28% -42% 27% Nakala 421 - 5 464 - 11 10% - 120% Dong Savanh 563 56 12 665 47 16 18% - 16% 33%

24

Table 16: Improvement in utilization of Health Centers

25

IMPACT: IMPROVED HEALTH IN RURAL COMMUNITIES

‘Child mortality numbers reduced by 10% in surrounding communities (<5 kilometers) in 1 year’ Monitoring child mortality in Lao rural area is difficult for the following reasons:

• Only 56% of births are registered in rural area (WHO: Child Survival Profile – Laos (2007) • Children deaths are not reported to health center but to village authorities and no way to collate. • A general low usage by rural population of Health Center for attended deliveries • Reference Statistics too old (2005) or only available for provincial levels data • No data is readily available at village level and no 2009 data is available at provincial level from MOH.

The impact of this project cannot be evaluated with indicators data available at the time of the project.

The project selected one HV and one DH in each of the eight target district in two provinces. This type of implementation made it difficult to evaluate the effect of the e-centers on the quality of service provided. The next pilot phase should focus on the computerization of one district (all e-centers in one district) in order to enable measuring the evolution at the district level. The reason is that it is easier to monitor indicators at district level than at village level because village data are hardly available.

Experience show that monitoring the number of deliveries in HC and DH can be a relevant indicator for measuring the impact of improved health and the utility of the Maternal and Newborn Health (MNH) CD-ROMs.

CONCLUSION

Health education material

The internet connection allowed the MoH to easily access health education material on the web. This aspect (for the MoH to download material from U-Tube for instance) was not initially planned. The project has showed that the material locally produced and in local language (including Lao, Khamu and Hmong) is really appreciated by the target population. This interest for locally adapted material should be taken into consideration by the MoH. With minimum investment, the MoH can access through the net to almost unlimited health and medical knowledge and disseminate at provincial and district level. CIEH should be put in charge of building resources since this department can easily access, download and has the skills to translate the material into local languages.

Monitoring & Evaluation

The MoH has already been using an efficient and well structured monitoring system. The project offered the possibility to use electronic format of the existing monitoring tools and this was also really appreciated by the e-centers’ staff. The MoH Planning and Statistic Department should be put in charge of monitoring because this actor is already in charge of data collection using electronic format of the form currently used; this would save both financial and human resources.

Capacity building

In terms of capacity building we can consider three main achievements:

1. 16 health staff without prior experience in English language and computer use have been made able to produce report in word and excel and send it by mail at the end of their training.

2. District hospital staff have been trained in maintenance of e-centers and use of equipment (digital camera, scanner and printer) ad how to integrate video ad digital pictures in PowerPoint.

26

3. The final provincial workshop illustrated that the training had improved the presentation skills of the staff.

RECCOMENDATIONS

OUTPUT 1: RECOMMENDATIONS The following recommendations are proposed to resolve issues regarding: • the quality of internet connection

Each PSTO must: (i) evaluate ICT equipment available on the market, (ii) evaluate offers and coverage provided by local ISP, (iii) provide recommendations regarding technologies (3G, CDMA, GPRS), equipment (Winphone, mobile phone, modems) and ISP for improving internet access quality of e-health center. This study must also be disseminated through provincial and district governments for supporting their own ICT implementation plans.

• the internet communication cost for DH and HC

The allowed communication support to DHs and HCs is sufficient for email transmission but limit the capacity to browse and download documents and video from internet web sites. To reduce the communication cost of HCs and DHs, CIEH and PHDs must define procedures to allow users to propose/request - through the project Web site - to download medical videos. These videos can be downloaded either by CIEH or PHDs, then transferred to CD/DVD-ROMs and distributed to DHs and HCs at very low cost.

• the cost of ICT consumables

NAST and PSTO must evaluate the possibility to buy, store and deliver standard toners cartridges at fair price to provincial government agencies and offices with the following impacts: (i) provide NAST and PSTOs some budget income, (ii) reduce overall ICT costs for provincial and district governments

OUTPUT 2: RECOMMENDATIONS The following recommendations are proposed to promote the use of e-health content by the MoH:

• Project WEB Site

MoH and CIEH must understand that the project web site is a new technological tool at their disposal for: • Improving health staff professional education by downloading, translating and disseminating existing

e-health contents freely available from the Internet. • Promoting the use of the project web site for hosting other websites developed by international

supported health projects for generating some CIEH budget income by leasing server space. • Promoting the use of the project web site to all health facilities with an internet access

• Produced CD-ROMs

CIEH and PHDs must use multimedia skill learned during training sessions for duplicating e-health content videos on CD/DVD-ROMs to ensure the dissemination to e-health centers and other health facilities.

• Internet medical WEB sites

27

CIEH and PHDs must select, download and compile medical videos downloaded from the internet into CD/DVD-ROMs for wide spread dissemination to e-health centers and other health facilities.

OUTPUT 3: RECOMMENDATIONS The following recommendations are proposed to promote ICT capacity building: Given the general lack of ICT training programs in Lao PDR, NAST and CIEH must develop training plans and customized training programs to help their technical staff be more IT competent and proficient. The training rooms at PSTO set up under the TA provide equipment for training. PSTOs must promote the usage of their training and technical capacities by providing ICT training courses and support to provincial and district government staff. Province governments must understand that ICT services provided by PSTOs can reduce their overall ICT costs at provinces and districts levels. PHDs must allow and promote the use of ICT technologies (e-mail and e-health education) to DHs medical staff. PHDs e-health center must be open to medical doctors from any DHs when visiting PHDs. DHs must promote to use ICT technologies (emails and electronic reporting) for communication with PHDs and other e-health centers. HCs must try to use ICT technologies including emails, electronic reporting and e-health education on a daily basis.

OUTPUT 4: RECOMMENDATIONS The following recommendations are proposed to promote the use of electronic health reporting form: • The MoH must understand the value of loading their standardized electronic forms into the project

website and making them available to every health facilities with an internet connection. • The MoH must also think about developing database systems for handling and aggregating health

reporting forms at DH, PH, PHD and national levels.