The External Evaluators Are Coming:Turning the Process Around

So It Works for You

Laura RoperRoper LYV Consulting

You wake up at 3 a.m. from a nightmare

The external evaluators are coming!

You’ve been DREADING it

The program is going great, but• the donor kept pushing you to add on indicators in your log frame and you’ve only been collecting data on a third of them;• you couldn’t adapt the data base to include all the data you are collecting, so even much of that isn’t entered;• it hasn’t been a priority, because you have the basic data you need and you know everything is on course• but you know it will look bad to the external evaluators

That email you ignored is waiting for you asking....

...for documents, about logistics, about the schedule

You have a choice.

You can go into the defensive avoidance or

You can engage and co-opt

What’s defensive avoidance?

• You send documents at the last minute or not at all• The evaluators are kept waiting at the airport, because you didn’t remind the driver to get them• When they arrive at the office, you keep them waiting for 20 minutes, suddenly caught up with an email• You apologize, explaining the demands on your time, how busy the staff is and how bad the timing of the evaluation is• You’ve only arranged a few of the meetings, but promise you’ll set up the rest this afternoon (along with getting the rest of the documentation), and offer them some desk space.

How do the evaluators perceive this?

• At best, you’re disorganized• You’re might be incompetent• You might have something to hide• In any event, you’re going to make this hard, which will waste a lot of time• It’s going to be one of these exhausting adversarial processes

But remember, external consultants are humans too

Question your assumptions

You may think.... But maybe...

they’re just in it for the money they’re really committed to your issues

they’re here to find fault they really want to see how you can build on your successes

they were behind that ridiculous log frame M&E section

they actually don’t think log frames are the best planning tool for your type of organization

they don’t know enough about the context and the challenges you face to understand your work

they have life and professional experience that gives them good insight

a quick fly-in won’t do justice to the process

they argued with the donor about the tight timeframe, but couldn’t make any headway

5 things you can do to engage and co-opt

• Kill them with kindness• Set the terms of engagement• Tap into their expertise• Treat them like allies• Ask for and weigh in on the preliminary recommendations

Kill them with kindness

• Send them a welcome email as soon as you hear they’ve been contracted; anticipate their request for documents with attachments of key documents.• Make arrangements promptly and send a ‘welcome pack’ with basic information about arrival, currency, contacts, and the hotel, send more documents;• Have a welcome folder at the hotel, with the schedule for the stay, a hard copy of the welcome pack, and a note (perhaps written on a card produced by local artisans), and recommendations for local restaurants.

Set the terms of engagement, consider the following

• Meet with them first thing, set aside plenty of time, be calm and in control, and set out your expectations (this might include an orientation meeting with staff, quick daily debriefs to be sure things are going smoothly, and a one-on-one or team meeting at end of the process)• Next have a team meeting and set a tone of openness and transparency (e.g. you might ask staff to introduce themselves and share a thing they are most proud about their work of and one of the biggest challenges they face).

Set the terms of engagement, consider the following

• Ask the evaluators to do the same about their own practice, while sharing their own background and experience relevant to this assignment (you don’t want them overwhelming you with their credentials);• Ask them to explain the purpose of the evaluation, their approach and what they hope to learn; ask questions.• Then (having thought this out beforehand with staff), tell them what would make this evaluation most useful for you, while sharing with them your own monitoring, evaluation and learning processes (formal and informal).

Tap into their expertise (it will be flattering and could be useful). Ask them...

• To tell you about the most successful program like yours that they’ve evaluated/worked with;• What are the biggest challenges your type of program faces and what, in their knowledge, works to overcome those challenges;• What resources, networks, experts they know of that might be useful to your program;

All the while you (and your staff) should be sharing your expertise, experience and insights

Treat them like allies

This can be delicate, but• When the issue of the database comes up, ask them their advice on what you should upgrade to, how much it would cost, and do they think the donor would finance it;• When you are going over the indicators, share that you felt pressured to add more and more and you’d appreciate their help on identifying the key indicators and negotiating with the donor.

Ask for preliminary recommendations

• Plan a debrief at the end of their visit;• Set the tone by saying you’d love their thoughts on findings and actionable recommendations that you can be thinking about; • Whatever they say, don’t be defensive, ask for more information on their thinking before you object; • Explain (for the document) that it is useful to get a limited number of recommendations categorized by time frame, ease of implementation, and or urgency. • Thank them for their work; express interest in staying in touch.

For the future, your best chance for exerting control over the MEL process with donors is to develop your own MEL strategy that is best suited to the needs of your program and the culture of the organization to improve your negotiating position.

Laura Roper, Ph.D. runs Roper LYV Consulting

She is a general non-profit management consultant with extensive experience in conducting evaluations; design of monitoring, evaluation and learning systems; and training on a range of MEL topics and methods.

She can be reached at l.roper@rcn.com