saleema evaluation annual report year 1 …saleema.net/upload1/saleema evaluation year 1...

1 | P a g e

SALEEMA EVALUATION ANNUAL REPORT

YEAR 1 (PHASES 1-2)

SUBMITTED BY:

W. Douglas Evans, Ph.D.

SUBMITTED TO:

UNICEF SUDAN

UPDATED:

16 March 2016

2 | P a g e

The purpose of this report is to summarize year 1 (phases 1-2) Saleema campaign evaluation activities, preliminary data, interpretation of findings, and next steps for year 2.

1. GOALS AND OBJECTIVES OF THESALEEMA INITIATIVE

The Saleema Initiative provides positive communication tools that support the protection of girls from genital cutting, particularly in the context of efforts to promote collective abandonment of the practice at community level. The broad objective is to change the way that people talk about female genital cutting by promoting wide usage of new positive terminology to describe the natural bodies of girls and women. Saleema also aims to stimulate new discussions about FGM/C at family and community levels -- new both with regard to who talks to who (’talk pathways’) and the specific issues communicated about (’talk content’). The overall goal of the Saleema Communication Initiative is to promote long-term abandonment of Female Genital Mutilation and Cutting (FGMC) through changing social norms, attitudes, and intentions related to the practice. The immediate measureable objective of the program is to increase social acceptance of uncut girls, as reflected in increased use of the positive term Saleema and in normative beliefs about the acceptability of being uncut. More information on the initiative is available at: http://www.unicef.org/sudan/protection_6092.html. In Sudan, FGMC is a highly accepted social practice with deep cultural roots. It is based on social normative beliefs about how individuals in the community should behave (subjective norms, beliefs about what should be) what most people in the community actually do (descriptive norms, empirical beliefs) (Kahneman & Miller, 1986; Cialdini, 2007; Bicchieri, 2010). When a social norm such as FGMC is in place, families and individuals engage in the practice because they believe that it is expected of them and is prevalent. Without these perceptions, the social norm would be weakened and practice would become less widespread and may eventually cease to exist. Changing both subjective and descriptive social norms is thus a key step in social norm change (Cialdini, 2007). When subjective and descriptive norms are in conflict, many people follow descriptive norms, what they perceive the prevalent behavior to be (Fekadu & Kraft, 2002). Even if a norm is considered subjectively important (what should be), if individuals observe non-conformity they will be less likely to follow the norm. 2. BACKGROUND ON THE SALEEMA EVALUATION

In May 2014, UNICEF engaged an expert independent evaluator, Dr. W. Douglas Evans, to design a monitoring and evaluation framework for Saleema. In September 2014, after a site visit to Sudan and meetings with UNICEF, National Council for Child Welfare (NCCW), and other key Sudan government and non-governmental organizations, the framework was completed eloped and now serves as guidance for the evaluation implementation phase, including the data collection activities carried out by Ahfad University. In November 2014, Dr. Evans again visited Sudan and conducted an initial orientation to the evaluation framework and an in-depth, two-day training with the data collection team, including managers and field interviewers. The framework consists of a theoretical model for the evaluation, methodology for 1) monitoring of Saleema activities and 2) evaluation of Saleema outcomes, sampling plan, measures and instrumentation, and a data analysis plan. The evaluation framework was provided to the initial selected data collection organization, University of Khartoum, Development Studies and Research Institute (DSRI). After a series of negotiation between UNICEF, NCCW, and DSRI, a mutual decision was made t move the project to a new organization, Ahfad University, in May

http://www.unicef.org/sudan/protection_6092.html

3 | P a g e

2015. Ahfad University staff had participated in the November 2014 training, and moved forward with plans for the evaluation framework implementation. The Saleema evaluation will answer four major research questions. The first research question (RQ) of the Saleema Evaluation is RQ1) Is Saleema effective in improving normative beliefs about FGMC abandonment? The second question is RQ2) Is there a dose-response relationship between higher exposure to Saleema implementation and improved normative beliefs about FGMC abandonment? The third question is RQ3) Does brand equity in Saleema mediate the relationship between Saleema exposure and normative beliefs about FGMC abandonment? Finally, the fourth question is RQ4) How do different populations and regions within Sudan respond to Saleema and what relationship is there between these variations and Saleema outcomes? The study design and data collection plan has been developed specifically to answer these questions. Additional details on the evaluation plans are available in the final frame work document, dated 6 September 2014. 3. ROLE OF AHFAD UNIVERSITY IN EVALUATION IMPLEMENTATION AND DATA

COLLECTION

Overall, Ahfad University responsible for implementing the sampling and data collection plans laid out in the evaluation framework. The following major tasks will be undertaken in this project, with the initial steps having been taken in year 1.

4. MAJOR TASKS

The data collection organization will propose a specific technical approach to each of the following major tasks. Table 1 summarizes the tasks, with detailed descriptions to follow.

Table 1 Summary of Tasks

Task Comments Status

1. Sign contract Completed in year 1

2. Training 1 Conducted in Khartoum, led by independent evaluator

Completed initial training for year 1 (November 2014)

3. Develop Saleema Evaluation and Monitoring System (SEAMS)

Will include user interface; programmed in Access or other relational database

Initial design completed in year 1; final design and implementation planned for 2016

4. Collect T1 data Assume 6 weeks for baseline, 4 weeks for follow ups; NOTE: data file will be delivered to evaluator within 15 days of completion of each round of data collection (T1-T5)

Completed baseline (T1) data collection in year 1

5. Collect T2 data Scheduled for year 2

6. Training 2 Scheduled for year 2





4 | P a g e


12. Deliver final data/produce methodology report

Scheduled for year 4

a. Annual Training with Independent Evaluator

The independent evaluator, Dr. Evans held an initial training on the evaluation framework and upcoming evaluation activities in November 2014.Subsequent trainings are currently planned for February or March of2016-2018 (years 2-4). Each training follows a specific agenda:

Saleema program goals, objectives, conceptual framework, and major activities

Saleema evaluation framework

Saleema online monitoring system (SEAMS)

Sampling plan

Data collection instruments – survey instrument and qualitative instruments (focus group and IDI guides)

Data management procedures

Data file delivery to Independent Evaluator

Follow up data collection and related activities from 2015-2017

The trainings are each two days and will be held at UNICEF offices, Ahfad University, or nearby location in Khartoum.

b. Draw nationally representative sample, stratified by state

The evaluation framework calls for a nationally representative sample of Sudanese households, stratified by the 18 states across Sudan. To draw the baseline sample, Dr. Evans worked with Ahfad and an independent statistician from University of Khartoum, to identify all administrative units within the 18 states. Following a cluster randomized sample plan (see appendix to this report), we will randomly sampled 2 administrative units within each state in which to collect the required sample for that state (see Table 3 for sample size by state).

We calculated power based on descriptive normative beliefs about FGMC (i.e., that it is a widespread practice in the individual’s community) given two time points of interest: (1) at baseline; and (2) 1 year post-baseline (at T3, after the heavy up phase X2) to compare differences in that outcome between individuals with high versus low exposure to Saleema intervention and control conditions. Studies have shown that reductions in perceived norms of health behavior (descriptive norms) are associated with reductions in intentions and ultimately in behavior (Evans, Necheles, Longjohn, et al., 2007; Hecht, et al., 1997). We assumed 25% descriptive social norms for FGMC abandonment at baseline (belief that abandonment is common place in my community) at baseline, given the target population (Friedrich, et al., 1975).We assumed a 50% recruitment rate (i.e., success in enrolling participants) based on previous research (Alba, et al., 1987), 80% power (1-β) to detect a difference of 15% in descriptive norms between high and low Saleema exposure conditions, the standard deviation (SD) of descriptive norms is 1.5, and a two-sided significance level of 0.05, assuming 10% annual loss to follow up (attrition). Based on these assumptions, we needed a baseline sample of 199 baseline participants per group. As we plan to make comparisons between higher and lower exposure locations within Sudan, we planned to draw a baseline sample of this size from

5 | P a g e

each state. In order to evaluate differences in Saleema outcomes based on the planned SDI measure of campaign dosage, we planned to collect this sample for each of the 18 states. This results in 199 x 18 states = 3,582 participants drawn from across Sudan at baseline. The end line sample after attrition will be 2,900. Table 2 illustrates a potential sampling plan based on this analysis.

Table 2 Saleema Evaluation Sampling Plan^

Baseline (T1) T2 (mid-year 2) T3 (end of year 2)

T4 (end of year 3)

T5 (end of year 4)

199 (x 18 = 3,582) 179 (x18 = 3,222)

179 (x 18 = 3,222)

161 (x 18 = 2,898)

145 (x 18 = 2,610)

^Note that the sampling plan represents HOUSEHOLDS. We will collect either a male and female head of household interview where available for each sampled household.

Sampling Methodology The sampling methodology adopted for the implementation of Saleema is probabilistic. The evaluation of data quality of Saleema will not only be based on the reduction of sampling errors, but also on reducing non-sampling errors.

The target population:

Under the present federal system, the sample survey covers all 18 States of the country, i.e. the survey is on the national level.

Mainly one unit of analyses is concerned by this study: the head of the household.

The frame:

Probability sampling depends basically on an up-to-date population frame. In this study, a frame of the number of all HHs and population in each state, locality, is obtained and prepared from the CBS according to the 2008 National Population and Households Census.

Stratification:

Stratification is usually introduced to reduce the sampling error though the inclusion of all different characteristics of the population into the sample. For the purposes of this survey, stratification was done by:

i) State.

ii) Gender, males and female heads of households.

Sample Design:

A stratified two-stage sampling design has been adopted:

1- First stage: selecting a sample of Clusters (localities, administrative units, popular administrative units PAU) using PPS selection without replacement to get a sample of clusters from each stratum. 2- Second stage: systematic selection, with equal probability, of 26 households (HHs) from each selected Cluster.

6 | P a g e

The Sample Size: The formula used for the calculation of the sample size, according to the main objective of the survey, was:

z2 * r * (1-r) * deff

n = ---------------------------------------------

(RME * r)2 * RR

Where:

n = the required sample size, (number of HHs)

z = the value in the normal distribution that gives level of confidence 95%, (z = 2)

r=predicted value of indicator (in target/base population), (r=0.5)

deff = the design effect, ( deff = 2)

RME=relative margin of error at 95% confidence (RME=0.048).

RR = response rate (0.95).

By substitution:

n = 22*(0.5) (1-0.5) *2

(0.048*0.5)2 *0.95

n = 3655=3672 HHs, i.e. 208 HHs from each state.

The sample size calculated estimates 50% of the females are Saleema.

The total sample size was divided equally between males and females.

When female heads maybe fewer, in that case data collectors were told to use any old female in the HH.

Sample Selection:

i) Selection of Clusters: A systematic random sample of Clusters was selected within each stratum with probability

proportional to size in terms of households.

ii) Sample selection of households: After the selection of Clusters, household listing operation was carried out within

each selected Cluster (Updating-2014 the cluster list of households). The household listing of each cluster was served as a sampling frame for the selection of households within each selected Cluster.

A systematic random sample of 26 households was selected with equal probability within each selected cluster. To reduce field bias of selection, a table of the randomly selected HHs in each cluster according to the size of the cluster was prepared.

7 | P a g e

Listing Process:

The selection of the clusters was based on the PAUs as given in the frame by the 2008 Census. Listing was done according to the selected PAUs. Segmentation was done, first, for PAUs exceeding 150 HH. Then one segment was selected randomly, and listing was done to the selected segment. Then 26 HHs were selected with equal probability from the selected segment.

c. Saleema Online Monitoring System

Dr. Evans and Ahfad developed the online monitoring system, which is a relational database that is accessible online, with a user-friendly interface, and designed to track implementation of Saleema campaign activities. The Saleema Evaluation and Monitoring System (SEAMS) has a user-friendly data entry interface to allow the local State Council for Child Welfare (SCCW) to capture major activities under each of the three basic Saleema initiatives, Sufara, Colors, and Born Saleema. SEAMS are programmed in Microsoft Access and data will be uploaded on a continuing basis to a Central data registry to be maintained by Ahfad.

The final set of data elements to be collected over the subsequent phases 3-5 of the

Saleema evaluation will be specified in collaboration between UNICEF, NCCW, each State Council on Child Welfare (SCCW), the evaluator, and Ahfad. Note that Saleema TV broadcasts include both 1) National, and 2) State level. The types of broadcasts include Saleema advertisements, drama series, and moderated roundtable discussions. Of these, the ads will run most often, followed by drama series and then roundtables. SEAMS will record all information on National and State level broadcasts including quantity, date/time, location, and type. The combined TV exposure data is expected to vary by location and by time as a result of the heavy-up design and media buy.

Table 3 provides a minimal set of required data elements for community-level activities. The

final set of variables will expand on this list. Table 3 Minimum SEAMS Data Elements^*

Type of Event/Activity

Event/ Activity ID

Activity content

Location Date/ time

Duration Number of partici-pants

Local partners partici-pating

List of Events/ Activities below

Event 1

Event 2

Event 3, etc. *SEAMS also includes National and State level media data, as noted above.

d. Program measures &instruments

The independent evaluator, Dr. Evans developed and provided Ahfad with completed instruments. The instruments (see appendix) consist of 1) focus group guide, 2) in-depth interview (IDI) guide, and 3) questionnaire. Ahfad provided input on final specifications of these instruments, and revisions were made before baseline data collection.

8 | P a g e

The qualitative instruments consist of open-ended questions exploring topics related to the Saleema initiative and responses to messages and campaign activities. The focus group guide took approximately 90 minutes to administer. The IDI guide took approximately 45 minutes to administer. Ahfad recorded all sessions and provided detailed summaries and transcripts of the results to the independent evaluator (see Appendix).

The questionnaire is a 48-item, interviewer administered questionnaire consisting solely of close ended questions. It takes approximately 15 minutes to complete.

e. Implement data collection plan

Ahfad implemented the final data collection plan developed as deliverable 1 under the awarded

contract. The qualitative data collection consists of the following on an annual basis:

Four (4) focus groups with female heads of household, and 4 with male heads of household – two of each in Khartoum and two of each in one region outside of Khartoum. Focus groups will consist of approximately 9 individuals each. The participants will be from different households.

Approximately 10 IDIs with key informants in Khartoum and 10 in one region outside of Khartoum. The total number of IDIs will depend on an analysis of key stakeholders by community, development of a final version of the Figure 1 conceptual framework and definition of stakeholders, and the number of these types of stakeholders available by community.

The questionnaire implementation followed the sampling plan noted earlier. The instrument was administered following the methodology/modality described earlier. A pre test was conducted to test the four methods adopted in Saleema M&E project. The pretest was conducted in Khartoum state; Ombadaa locality which is considered as a semi rural area. A total of ten questionnaires were filled, two FGDs and two interviews were conducted, in addition to the photo voice. The average time for filling the questionnaire was 17-25 minute per the questionnaire and the time for the FGD was 60-90 minute. The testing of the methods revealed that the respondents were distracted by the laptops and were not comfortable with the recorder, however; the team managed to deal with this matter properly.

The instrument was pilot tested in Khartoum prior to full implementation with 25 individuals (10 for the questionnaire, two for the interviews and 10 for the FGD) and any issues (e.g., lack of comprehension, language issues) with questionnaire content reported back to the independent evaluator. Revisions were made in consultation with the independent evaluator as needed. A final technical test of the questionnaire in computer-enabled format was done to ensure that survey logic and programming functioned properly prior to final implementation.

Both the qualitative and quantitative baseline data collection were implemented across Sudan in October and November 2015. Detailed steps in the implementation were as follows:

Prior to the implementation process two days training on the use of data collection were conducted. The first day was led by two facilitators who developed the data base. They focused on the practical side of how to use the system to fill the questionnaire. Actually this training was a practical training and demonstrated the use of the data base. The second day of the training focused on the qualitative technique for the data collection (the interviews, FGD and the photo voice).

9 | P a g e

Then in coordination with NCCW, contacts with SCCW were done and criteria for selection for data collectors were made and sent to the SCCWs who selected the data collectors. Data collection across the state took place between Oct-Nov 2015. The data collection process faced so many challenges, some of these challenges were related to the organizational aspects of the travel, others were related to the use of laptops, and the reluctance of some respondents to provide information while the laptops were used. The same was for the use of recorders and cameras for the photo voice. The data collectors, under the guidance of the facilitators and the team in AUW, decided to use paper questionnaires then convert all answers and type them at night in the laptops, the thing which was time consuming for them, but they were obliged to do it.

f. Data management

AUW conducted logic checks and cleaned the data thoroughly after collection. Issues encountered in terms of data quality were addressed in consultation with the independent evaluator. AUW took the following specific steps in data management:

The process of data management was including the following steps:

1. Received data files (in Microsoft Access format) of each state revised by state

supervisor.

2. Each state data files compiled in one data file

3. The data files were checked by re-entering 5% of the data by data manager and

assistant data manger.

4. The data file of each state cleaned and sent to evaluator in 2 formats (Excel and SPSS

format).

5. During the data management process a progress reports were sent to project manager.

g. Delivery of data file and code book

AUW delivered a complete, clean data file both in a flat file format (Excel) and in SPSS format to

the independent evaluator upon completion of the final interview in each project year. The files

were accompanied by a code book along with data definitions sufficient for an analyst to fully

understand all variables contained in the data set. AUW was available to answer any questions

about the data files after delivery and provided the independent evaluator with update on status

of files, data cleaning, and data issues encountered as documented above.

h. Administrative

Throughout the year 1 implementation, Dr. Evans, UNICEF, NCCW, AUW, and other

collaborators participated in periodic Skype calls and other project management meetings.

Additionally, as needed, the Project Management staff communicated with UNICEF staff and

the independent evaluator, about problems or issues encountered and recommend solutions.

Provided periodic reports on data collection during the data collection period in October and

November 2015, noting interview attempts, completed interviews, and other basic statistics

regarding study progress.

AUW encountered a number of administrative and related issues and implemented the following

solutions:

10 | P a g e

There were some coordination problems that hindered communication between AUW, NCCW

and the SCCWs. This was dealt with through joint meetings with the SCCWs in Khartoum and

orientation session about the project. As for the financial issues they have been amended in the

project finance through NCCW ECT. All administrative issues and complains from data

collectors as well as facilitators were taken in to consideration and will be dealt with in the

second round.

5. RESULTS

a. Quantitative Results

The following tables reflect baseline data collection. The total sample at baseline was 3,346

across all 18 Sudan states. Table 1 depicts the sample descriptive statistics with mean scores,

percentage in category, and standard error (SE).

Table 1. Sample Descriptive Statistics

Descriptive (N = 3,346) Proportion, SE

Gender

Male 54.2% (3.55%)

Female 45.8% (3.55%)

Age Mean=37.41 (0.876)

Education:

No School 22.4% (3.39%)

Less than HS 36.4% (2.54%)

HS Completed 16.8% (1.74%)

More than HS 18.2% (2.72%)

Religious School 6.12% (2.56%)

Marital Status

Single 25.6% (3.28%)

Married 74.4% (3.28%)

Age

18-24 17.2% (2.3%)

25-34 28.9% (1.59%)

35-44 25.7% (1.42%)

45-older 28.1% (2.24%)

Table 2 depicts the results of a confirmatory factor analysis (CFA) run on the targeted beliefs

about the word Saleema (language regarding FGMC), which are targeted short-term outcomes

of the campaign. The CFA results show that the 4 measured Saleema beliefs form 1 factor,

11 | P a g e

which can be used as a scale variable in future analyses of follow up data in phase 3 and

beyond.

Table 2. Saleema Beliefs Confirmatory Factor Analysis

Beliefs % Agreement , SE Factor Loading Alpha

Saleema is good.

92.5%

0.7515

0.8383

(1.31%)

Saleema is God’s intention. 92.3%

0.7482 (1.83%)

More and more people in my community believe in Saleema.

83.1%

0.7342 (3.06%)

I would like to help tell people in my community about the importance of Saleema.

89.9%

0.7913 (0.0196)

Table 3 depicts the results of a confirmatory factor analysis (CFA) run on the message

receptivity construct (rational and affective reactions to ad content) in response to seeing the

Saleema advertisements shown in the questionnaire (which were drawn from the ads used

across Sudan during recent campaign implementation). The CFA results show that the

message receptivity items measured for Saleema ads form 1 factor, which can be used as a

scale variable in future analyses of follow up data in phase 3 and beyond.

Table 3. Message Receptivity CFA

Saleema Ads

Receptivity Mean Score (1-4), SE % Agreement , SE

Factor Loading

Alpha

Ad 1

This ad is convincing.

3.302 88.8% 0.5654

0.7856

(0.0572) (2.49%) Would you say the ad grabbed your attention?

92.7% 0.7027

(1.51%)

Would you say the ad gave you good reasons to talk to your adult family members about keeping girls whole, keeping them Saleema?

87.3%

0.7649

(1.99%)

12 | P a g e

Did you talk to your friends or other adult family members (not your child [ren]) about this ad?

78.9%

0.6584 (4.05%)

Would you say this ad said something important to you?

89.3% 0.7591

(2.11%)

Ad 2


3.264 90.3% 0.5617

0.7717


92.6% 0.7806

(1.78%)


87.5%

0.7493

(1.61%)


76.5%

0.5774

(7.00%)


89.2% 0.7663

(2.26%)

Ad 3


3.268 89.3% 0.5699

0.826


92.5% 0.7686

(2.25%)


87.4%

0.7821

(2.15%)

13 | P a g e


82.3%

0.6999

(2.93%)


89.7% 0.7971

(2.48%)

Table 4, on the following page, depicts the results of a confirmatory factor analysis (CFA) run on

the Saleema brand equity scale. Brand equity is a construct that measures identification with the

Saleema brand, what the campaign and its messages stand for. The CFA results show that the

brand equity items measured for Saleema which can be used as a scale variable (both for the

four categories of brand equity measured, and as a single scale) in future analyses of follow up

data in phase 3 and beyond.

14 | P a g e

Table 4. Brand Equity CFA

BE FACTOR

Saleema

Mean Score (1-4),

SE

% Agree (3 or

4), SE

Factor Loading

Alpha

Higher BE Factor Loading

(alpha=0.9304)

Brand

Loyalty

I would like to help spread the word about Saleema

3.147 86.1% 0.8649

0.9155

0.8733

(0.0662)

(0.0256)

I would defend Saleema if someone was putting it down (yes/no)

3.14 86.4% 0.8647 (0.0613

) (2.39%

)

I would wear a Saleema T-shirt.

3.11 84.3% 0.8209 (0.0671

) (2.77%

)

If I had the chance, I would promote Saleema to my friends.

3.064 83.2% 0.8448 (0.0775

) (3.55%

)

Leader-

ship/

Populari

ty

Saleema program or materials is there in my community when I need it

2.754 61.8% 0.5983

0.9314

0.9178

(0.124) (8.27%

)

I pay more attention to Saleema when I see it in my community

3.077 85.7%

0.8667 (0.0729

) (3.06%

)

Saleema is the best program for youth like me.

3.083 84.2% 0.8619 (0.0769

) (3.01%

)

Saleema is becoming more popular with people like me.

2.981 77.7% 0.8152 (0.0703

) (3.35%

)

Saleema is meant for people like me.

3.014 82.5% 0.8347 (0.0725

) (3.15%

)

I get a lot out of being involved with Saleema

3.037 84.0% 0.8487

(0.067) (2.97%

)

Being involved with Saleema is really worth it.

3.056 84.5% 0.8628 (0.0632

) (2.89%

)

3.046 83.4% 0.8192

15 | P a g e

The idea of keeping girls Saleema is growing and spreading in my community

(0.0725)

(2.94%)

Brand

Personality

Would you say that people in the Saleema advertising are :

0.9496

0.8255

Good/credible

3.056 86.9% 0.8791 (0.0657

) (3.02%

)

Clean and pure/ honest/genuine

3.053 87.1% 0.9228 (0.0622

) (2.88%

)

Healthy

3.054 87.7% 0.9172 (0.0621

) (2.8%)

Successful

3.067 87.7% 0.9351 (0.0625

) (2.8%)

Happy

3.076 88.0% 0.9235 (0.0649

) (2.82%

)

Brand

Awarene

ss

When you think of Saleema you think:

0.9249

0.8964

Girls should stay Saleema

3.072 84.7% 0.8546 (0.0787

) (3.18%

)

Girls who are Saleema are marriageable

3.084 85.3% 0.98764 (0.0814

) (2.98%

)

People should abandon cutting

3.059 83.0% 0.85818 (0.0784

) (3.6%)

It is God’s intention that girls stay Saleema

3.12 87.3% 0.8702 (0.0752

) (2.86%

)

People who abandon CUTTING are making the right choice

3.093 85.2% 0.8485 (0.0852

) (3.3%)

16 | P a g e

Table 5 depicts means, percent agreement, and CFA results for the Saleema intentions,

attitudes, and social norms scales. These scales show the baseline levels of these Saleema

campaign outcomes. Future analysis in phases 3 and beyond will examine effects of campaign

exposure on these outcomes. The CFA results show that the intentions, attitudes, and norms

items measured for Saleema which can be used as 3 scale variables in future analyses.

Table 5. FGMC Intentions, Attitudes, and Social Norms

Intentions, Attitudes, and Norms Mean Score (1-

4), SE

% Agreement ,

SE

Factor Loading

Alpha

Intentions

I will not practice CUTTING in the future.

3.28 85.4% 0.7225

0.8842

(0.0752) (2.48%) I will advise other people that I know in my community who have daughters, NOT to practice CUTTING in the future.

3.207 86.0% 0.8515

(0.0712) (2.51%)

I will advise my friends to sign a pledge not to practice CUTTING in the future.

3.172 85.6% 0.8625

(0.0701) (2.46%)

If I have the chance, I will bring a friend or family member to a future Saleema event in my community.

3.227 88.1% 0.8391

(0.0708) (2.28%)

Attitudes

I am confident that my family will abandon CUTTING in the future.

3.24 85.0% 0.8207

0.9183

(0.0814) (0.0276)

Abandoning the practice of CUTTING will benefit girls in my community

3.232 86.8% 0.8952 (0.0777) (0.0231)

Abandoning the practice of CUTTING will benefit my family

3.204 85.8% 0.892

(0.0805) (0.0268)

It would be acceptable for a member of my family to marry a girl who is uncut.

3.224 87.0% 0.8614

(0.0758) (0.0239)

Norms

Most people in your community practice CUTTING

2.829 67.7% 0.7699

0.8205

(0.114) (0.0524)

Most of my friends practice CUTTING 2.672 60.1% 0.800

4 (0.0995) (0.052)

It is appropriate for families in my community to practice CUTTING

2.299 39.4% 0.5823 (0.105) (0.052)

Sudanese society in general considers it appropriate to practice CUTTING

2.688 63.5% 0.729

(0.102) (0.0514)

17 | P a g e

18 | P a g e

b. Qualitative Results

Review of qualitative data from all 18 states revealed several major themes in terms of respondent

interpretation of what Saleema meant to them. These included the themes of inclusivity, preservation,

engagement, achievement, and supervision. There is a great deal of cross-referencing between the sub-

headings (ie, more than one theme emerges from many individual quotes and comments). For example,

the sub-heading that deals with resistance to stopping FGM is closely related to the subheading that

addresses the desire to maintain culture/tradition. Notably absent from these documents are remarks

about the strength of women and their ability to enact change. There was more attention paid to the

role that men must play in stopping FGM.

There were a number of challenges coding this document. In particular, some interviews are not written

as direct quotations. Other interviews and individual comments were very brief, such as a “yes” or “no”

response to a complex question requiring an elaborated answer. Overall, the transcription of interviews

was not as detailed as the research team will need in future data collection.

THEMATIC CODES WITH EXAMPLES

Note: Each major theme is described and then illustrative quotes from the interviews provided below.

Inclusivity: Including all people in messages and works is an admirable/effective means of communicating.

o “His opinion does not change the society unless it was a collective opinion.” (Northern State, 3).

o “Ali Alkhairy. Many of the inhabitants admire him because he loves the people.” (Northern State, 4)

o “Her opinion is that it is very good for the society unless the majority and the religious opinions are different” (Northern State, 4-5)

o “Imam Mohamed Haj Altahir, many of the people here admire him, he is religious man and loves the people” (Northern State, 5)

o “I heard about SALEEMA campaign. It is excellent and wide-spread; but it is obscure to the public and is understood only to a limited section and the society.” (Northern State, 6)

o “She heard on the T.V. about SALEEMA campaign, but it was difficult for her to understand it.” (Northern State, 6-7)

o “They admire the personalities, the language of Mohamed Ahmed Hassan. Simple and easily understood by all.” (Northern State, 7)

o “The values they cherish are: the morning glorifications prayer on time, the voluntary work, collective work, help the needy.” (Northern State, 9)

o On what is most important, “Security. Education. Health. Good social relations.” (Abu Karanka Men, 2)

o “Intensify campaigns [by] the different means of communication” (Abu Karanka, 3) o On people admired, “Because they have good and wide social relations” (Aldiain Men, 1) o “Sheikh Alkarory many people here admire him for his love to the people” (Saleema

survey personal meetings, 1)

19 | P a g e

o “His opinion does not change the society unless the opinion is collective.” (Saleema survey personal meetings, 1)

Inclusivity: It is important to reach all geographic regions o “The publicity is obscure for the people of the village because they are simple and

uneducated. I am one of those who found it difficult to understand.” (Northern State, 5) o “The positive messages concern the educated sector only and the people in general.

This is why the country people did not understand it.” (Northern State, 9)

Inclusivity: Dissemination of SALEEMA needs integration of young children in the campaign and the campaign in early education.


o “Yes, most of us heard of SALEEMA campaign, we contributed in the schools” (Abu Karanka, 3)

o “She suggest for Saleema campaign to adopt a system in the elementary schools- third and fourth classes the teach the students girls make complaints about the crime of circumcision.” (Saleema survey personal meeting, 3).

Inclusivity: Aiding the coming generations and caring for elders are cultural values. o On what is most important, “gratification of parents” o “Circumcision is cleanliness and guard. Do not reject the customs of your

grandmothers.” (Northern State, 5) o “Circumcision is chastity for the girl and proof of her purity. The decision is by the

mother and grandmother. Therefore have no option to talk to them about the harms caused by circumcision.” (Northern State, 8)

o “The grandmothers rebuked my daughters, therefore I circumcised them Sunnah.” (Northern State, 9)

Inclusivity: Dissemination of SALEEMA requires the enlightenment of men. o “We must talk to men first, because the uncircumcised will not be married. Please

introduce to us men of religion to speak about circumcision and the harms it causes.” (Northern State, 4)

o “Our society needs to talk to men in the first place; because they have the authority.” (Northern State, 8)

o “Men has to agree to marry the uncircumcised girl to encourage on leaving the girls safe and sound.” (Northern State, 8)

o “This decision is at the hands of mothers, we [men] have no opinion.” (Northern State, 8)

o “Circumcision in harmful and sickness. Therefore men must be enlightened and cultured so as not to send back our daughters after married.” (Northern State, 9)

o “The man is a basic supporter to the woman” (Aldiain Men, 5) o “Father happy, proud of his daughter” (Saleema survey personal meetings, 2)

20 | P a g e

o “Another said: our society needs to men first because if our girls were net circumcised will be returned to us from the honey moon to her mother's house.” (Saleema state Kalasa, 2)

o “We the men to stand by the safe girls.” (Saleema state Kalasa, 2)

Preservation: Maintaining society/customs is an important cultural value o “Her daughters were circumcised, because these are the customs of the village. What

can we do? She asked God predestined. She does not speak about circumcision in the society meetings because it is shameful to talk about these matters.” (Northern State, 4)

o “It our custom, it is cleanliness and guard to the girl. Her opinion is that it is very good for the society unless the majority and the religious opinions are different.” (Northern State, 4-5)

o “Circumcision is cleanliness and guard. Do not reject the customs of your grandmothers.” (Northern State, 5)

o “Circumcision according to his studies there is sunna and must do it according to the sunna. He will change his opinion about circumcision if it is abandoned by the Islamic sharia.” (Northern State, 5)

o “We must follow the religious side because many of the Sudanese people are religious. The publication needs further explanation. I respect there must be religious rooting for the SALEEMA campaign.” (Northern State, 5)

o “He spoke about the disadvantages of circumcision and it is an old custom, and that his opinion will not change anything in the society.” (Northern State, 6)

o On improving the campaign, “clarification of the harms done by the circumcision and the Islamic law (sharia) that support it.” (Northern State, 6)

o “There is the sunnah and people must adhere to it. She will not change her mind because it the sunnah of prophet Mohamed, God's blessing and peace be upon him” (Northern State, 6)

o “We change our perception of circumcision if the society abandoned it. Or if there is a proof of prohibition.” (Northern State, 8)

o “Circumcision is an old custom which was inherited; but it is deadly” (Northern State, 8) o “Circumcision is chastity for the girl and proof of her purity. The decision is by the

mother and grandmother. Therefore have no option to talk to them about the harms caused by circumcision.” (Northern State, 8)

o “The grandmothers rebuked my daughters, therefore I circumcised them Sunnah.” (Northern State, 9)

o I do not circumcise, “Because it is against the Islamic sharia” (Abu Karanka, 2). o “Circumcision is against Islam and has clear effects” (Abu Karanka, 3) o “His opinion does not change the society unless the opinion is collective.” (Saleema

survey personal meetings, 1) o “Circumcision must be performed on girls because it is an inherited customs and

tradition. It is forbidden to talk about this subject, circumcision protect the girl and her chastity.” (Saleema state Kasala, 1).

o “But I am afraid of the regard of the society to describe my daughter as the galfa or the uncircumcised.” (Northern State, 8)

Preservation: Maintaining the safety of families and society is an important cultural value. o “He wishes his family and all the Sudanese people are safe” (Northern State, 3).

21 | P a g e

o “The most important thing for him is to see his family safe.” (Northern State, 3) o “The most important thing in her life is good health and helping the others. She wishes

the well-being to all people of the region.” (Northern State, 4). o “The most important thing is good health and safety.” (Northern State, 5) o “The most important values on the observance of work, appointments and the religious

rituals.” (Northern State, 6) o “What makes us happy is to leave our daughters sound.” (Northern State, 8) o On people who admired, “Their wisdom in managing the society. Their success at the

local level, their ability to absorb-anger, their strong support to the society” (Abu Karanka Men, 1)


o On what is most important, “The good social relation. Loyalty to family” (Aldiain Men, 2).

o On what is most important, “To see the society around me in good condition [and] family stability.” (Aldiain Men, 2).

o “Most important thing to see his family safe.” (Saleema survey personal meeting, 2) o “We must talk about such subjects, especially when it contain problems for the mother

and the child and also the society, we must talk in the pulpit of Friday, specialize one Friday each month to talk about this subject.” (Saleema state Kasala, 1-2)

Preservation: SALEEMA suggests the correctness of the natural state/incorrectness of altered circumcised state.

o "Every girl is born unimpaired. Let every girl to grow up unharmed.” (Northern State, 3) o “She heard about SALEEMA campaign on the T.V. but she did not understand. For her

SALEEMA means not sick.” (Northern State, 4) o On photo 2, “Every girl is born sound. Let every girl grow-up sound.” (Northern State, 6) o “Every girl is born free, let her be free. But I am afraid of the regard of the society to

describe my daughter as the galfa or the uncircumcised.” (Northern State, 8) o On photo 2, “A girl wearing SALEEMA colors and black color. There is discrimination.

Sound and circumcised.” (Northern State, 8) o “What makes us happy is to leave our daughters sound.” (Northern State, 8) o “They heard of SALEEMA campaign on the T.V. Let her be safe or as God created her.”

(Northern State, 9) o On photo, 1 “The feeling of happiness from the baby child, But the look of the mother

was different, as if she wanted to say: we must leave her with her joy.” (Aldiain Men, 4) o On photos 2&3, “Every girl is born SALEEMA sound. Let her grow SALEEMA. Let her grow

sound, enlightenment must continue in order to abandon the custom.” (Saleema survey personal meetings, 1)

o On photo 3, “leaving the girl Saleema.” (Saleema survey personal meetings, 2)

Preservation: Circumcision is dangerous, leading to challenges/problems/illness. o “He spoke about the harms done by circumcision. He said his elder daughters were

circumcised without his knowledge. He refuses this custom. His opinion does not change the society unless it was a collective opinion.” (Northern State, 3).

o “The pharaoh circumcision is deadly to the girl. Adopt the sunna.” (Northern State, 6) o On photo 1, “Unhappy. Weeping. Unsafe. Sad” (Abu Karaka, 3).

22 | P a g e

o “Circumcision is the cause of the girls suffering at birth” (Aldiain Men, 3). o “It is a bad habit, and the damage is larger than benefit” (Aldiain Men, 3). o On how to improve the campaign, “They must speak a lot about the harms from

circumcision.” (Saleema survey personal meeting, 1) o “Circumcision not accepted because it is a harmful custom. It involves violation of the

genitals” (Abu Karanka, 2)

Preservation: There are benefits/less challenges to leaving females uncircumcised. o “What makes us happy is to leave our daughters sound.” (Northern State, 8) o “A number of girls in the quarter were married and were not circumcised. It is

decreasing of late. A social enlightenment happened.” (Northern State, 9) o “In order to preserve her health and safety [I do not circumcise]” (Aldiain Men, 3). o On photos, “[The woman is] happy because she understood the situation and became

safe Saleema” (Aldiain Men, 5)

Preservation: Combating circumcision is difficult and meets great resistance o “His daughters were circumcision, but this custom of the village, what can we do about

it?” (Northern State, 3). o “He lived for 20 years in Saudi Arabia. There is no circumcision there. He feels shame to

speak to the society about not circumcision his daughters.” (Northern State, 3). o “Her daughters were circumcised, because these are the customs of the village. What

can we do?” (Northern State, 4). o “It our custom, it is cleanliness and guard to the girl. Her opinion is that it is very good

for the society unless the majority and the religious opinions are different.” (Northern State, 4-5)

o “Circumcision is cleanliness and guard. Do not reject the customs of your grandmothers.” (Northern State, 5)

o “Circumcision according to his studies there is sunna and must do it according to the sunna. He will change his opinion about circumcision if it is abandoned by the Islamic sharia. The health opinion is not as important as the sharia. He circumcised his daughter according to Sunna.” (Northern State, 5).

o “There is sunna circumcision and there are prophetic Alhadeeth to support that. I don’t change my mind only by correct evidence from the religion and health. If there is a correct prophetic Hadeeth that support uncircumcision, then I will not circumcise my daughters.” (Northern State, 6)

o On improving the campaign, “clarification of the harms done by the circumcision and the Islamic law (sharia) that support it.” (Northern State, 6)

o “Very important for girls and it must be done, it is practiced heavily in the society. I have a girl who is not circumcised because her father hates it.” (Northern State, 8)

o “Circumcision is an old custom which was inherited; but it is deadly” (Northern State, 8) o “The practice is present in the society and in every house. It cannot be abandoned

easily, in spite of that the practice reduced because of the dangers. Circumcision became a social occasion.” (Abu Karanka, 2)

o “there are practices of circumcision which take place secretly” (Abu Karanka, 3). o “Circumcision is a harmful custom despite many families practice it” (Aldiain Men, 3) o “Circumcision some people adopt it as a means of profit” (Aldiain Men, 3) o “Yes, the practice is present, and will not be abandoned easily” (Aldiain Men, 3)

23 | P a g e

o “Yes, because circumcision is not present in the religion” (Aldiain Men, 3) o “This subject concerns women and they can cure it in their own way.” (Saleema state

Kasala, 1) o “She does not talk to the society about circumcision because it is shame.” (Personal

survey personal meeting, 2) o “Circumcision must be performed on girls because it is an inherited customs and

tradition. It is forbidden to talk about this subject, circumcision protect the girl and her chastity.” (Saleema state Kasala, 1).

o “[Circumcision] is very important to the girl and her cleanliness.” (Saleema state Kasala, 2).

o “I am afraid of the society to insult my girl and this would cause complications to her [if she is uncircumcised].” (Saleema state Kasala, 2).

Engagement: SALEEMA needs/has “joined voices” o On improving the campaign, “Social Dialogue. Collective abandonment.” (Northern

States, 3). o On photo 1, “People who agree on SALEEMA are signing papers they represent people

who rejected circumcision. A clear publicity to abandon circumcision.” (Northern State, 8)

o “Saleema campaign means the collective abandonment of female circumcision” (Aldiain Men, 4)

o On photo 1, “The signing. All the society must sign a treaty that forbids female circumcision.” (Personal survey personal meeting, 1)

o On photo 1, “The signing…we must do this everywhere.” (Personal survey personal meeting, 2)

o “She heard about SALEEMA campaign on radio and T.V she heard the children singing, it was beautiful.” (Personal survey personal meeting, 2)

o “Yes, I can declare [against circumcision] with bravery” (Abu Karanka, 2)

Engagement: Women play a unique and promising role in the society. o “This subject concern women and they can cure it in their own way.” (Saleema state

Kasala, 1)

Engagement: Dissemination of SALEEMA needs enlightenment enacted between everyday interactions and people as well as formal educational opportunities.

o “Enlightenment helps to abandon FGM.” (Northern State, 3). o On how to improve the campaign, “Undertake the responsibility of enlightening the

people of the village. (Northern State, 4).

o On what would reduce circumcision, “Through enlightenment and making clear the damages,” (Abu Karanka, 2)

o “The seminars. Training. Intensive information campaign through the local broadcast” (Abu Karanka, 2).

o “Yes, I can declare [against circumcision] with bravery” (Abu Karanka, 2) o On improving the campaign, “The necessity of direct seminars. The formation of

enlightenment workshops” (Abu Karanka, 3).

24 | P a g e

o On combating circumcision, “Through enlightenment and religious preaching” (Aldiain Men, 3).

o On combating circumcision, “Through workshops for the society and correcting their concepts” (Aldiain Men, 3)

o “The sort of information I wish to be introduced is the formation of enlightenment workshops about the targets the campaign. Field workshops. Discussion groups” (Aldiain Men, 3)

o On photo 2, “This picture makes me feel the importance of enlightenment” (Aldiain Men, 5)

o “He spoke about the harms of circumcision and said his old daughters were circumcised. But after the seminars and lectures he became refusing this custom because it is wrong, and harmful to the mother and the child.” (Saleema survey personal meetings, 1)

o “In the village there are many activities about SALEEMA, he said his elder daughters were circumcised, but the young ones are not because of the enlightenment.” (Saleema survey personal meetings, 2)

o “We must talk about such subjects, especially when it contain problems for the mother and the child and also the society, we must talk in the pulpit of Friday, specialize one Friday each month to talk about this subject.” (Saleema state Kasala, 1-2)

o “Enlightenment discussions for men and women” (Saleema state Kasala, 2)

Achievement: Achieving goals and societal progress is a cultural value. o “The most important values on the observance of work, appointments and the religious

rituals.” (Northern State, 6) o “They love perfect work, the good treatment,” (Northern State, 7) o On people admired, “Because they employed the capabilities of the local society to

create a new reality… All the time enlightenment helps to abandon circumcision” (Aldiai Men, 1)

o On people admired, “Because they were able to employ their capabilities” (Aldiain Men, 1)

o On what is important, “education; good performance of work” (Aliain Men, 2) o “Perfect work is one of the basic values” (Saleema State Kasala, 1)

Supervision: Dissemination of SALEEMA needs penalty of the law/supervision o “Issue strict laws on the midwives” (Abu Karanka, 2)

25 | P a g e

6. NEXT STEPS

This report concludes year 1 (phases 1-2) activities. Year 2 (phase 3) is scheduled to begin 1

December 2015. The following steps are outlined Table 2 above, and described in detail in the

final Saleema Evaluation Framework.

Multivariate regression modeling of baseline data to include:

Associations with Saleema exposure

Associations with brand equity

Other associations – media use, socio-demographics, geographic location (state)

Analysis of baseline qualitative data (not yet available)

Completion of 1st follow up (Spring 2016) and 2nd follow up (Fall 2016)

Implementation of heavy-up experiment

Increase Saleema exposure over course of 2016

Evaluate short-term effects of exposure on outcomes

26 | P a g e

LIST OF APPENDICES

1. Quantitative sampling plan

2. Data collection Instruments

3. Qualitative Transcripts

saleema evaluation annual report year 1 …saleema.net/upload1/saleema evaluation year 1...

Documents