1

Survey Solutions Supervisor Manual

Computational Tools team, Development Research Group

THE WORLD BANK GROUP

2

Table of Contents Overview of the survey workflow ..................................................................................................... 3

Why the survey workflow matters for your work ........................................................................................ 5

Changing the Interface Language ..................................................................................................... 6

Mozilla Firefox ........................................................................................................................................... 7

Microsoft Internet Explorer ....................................................................................................................... 9

The components of the supervisor software .................................................................................. 10

Reports ........................................................................................................................................................ 10

Interviews.................................................................................................................................................... 11

Interviewers ................................................................................................................................................ 13

How to perform common supervisor actions ................................................................................. 14

Find a survey case ....................................................................................................................................... 14

By questionnaire ..................................................................................................................................... 14

By assigned team member ...................................................................................................................... 15

By status .................................................................................................................................................. 17

Assign a survey case to an interviewer ....................................................................................................... 19

Case not yet assigned to an interviewer ................................................................................................. 19

Case to be reassigned to another interviewer ........................................................................................ 21

Review a survey case .................................................................................................................................. 23

Navigate .................................................................................................................................................. 23

Add a flag ................................................................................................................................................ 24

Add comments ........................................................................................................................................ 25

Approve, Reject ....................................................................................................................................... 26

See the overall progress of data collection ................................................................................................. 26

Surveys and statuses ............................................................................................................................... 26

Team members and statuses .................................................................................................................. 27

Quantity .................................................................................................................................................. 29

Speed ....................................................................................................................................................... 31

Change the password for an interviewer account ...................................................................................... 34

Lock an interviewer account ....................................................................................................................... 35

3

Manual for Field Supervisors

Overview of the survey workflow

To understand your role as a supervisor and how the use the supervisor software, you must first

understand the survey workflow. The survey workflow can be summarized by Figure 1 below.

Figure 1: Overview of the survey workflow

Headquarters, designated by the building at the top of the figure, determines the households that need

to be interviewed and their assignments across team supervisors.

Team supervisors, denoted by the laptops in the middle of Figure 1, first receive these survey

assignments (as depicted in Figure 2) and then allocate them to the members of their team (as depicted

in Figure 3).

Figure 2: Supervisors receive assignments Figure 3: Supervisors allocate assignments

4

Interviewers, represented by hands holding a tablet, receive the assignments from the team

supervisors, collect data for those assignments, and send completed assignments back to supervisors for

review (as depicted in Figure 4).

Figure 4: Interviewers send completed questionnaires to supervisors

Team supervisors, having received completed questionnaires, review these questionnaires to confirm

that all questions are answered and that answers are accurate, coherent, and plausible. After reviewing

these completed assignments, team supervisors either approve or reject them (as depicted in Figure 5).

If a team supervisor approves a completed assignment received from an interviewer, the assignment is

sent to headquarters, as represented by the paper with a checkmark in Figure 5. If a team supervisor

rejects a completed assignment received from an interviewer, the assignment is returned to the

interviewer initially responsible for completing it, as represented by the paper with an X in Figure 5.

Receiving the rejected assignment, the interviewer must either correct it or provide explanatory notes

on strange or implausible answers. When the assignments are corrected, the interviewer sends them

back to the team supervisor for approval or rejection, a process that continues until the assignments are

completed with the highest level of quality according to the team supervisor.

Figure 5: Supervisors reject or approve questionnaires

5

Why the survey workflow matters for your work

Your understanding the survey workflow is critical for at least two reasons.

FIRST, team supervisors are first and strongest line of defense for survey quality. You MUST ensure that

all survey assignments for your team are completed. You MUST review each survey assignment to

ensure its quality.

You can either approve or reject an assignment. This decision must be taken after a thorough review of

each assignment. Headquarters will hold YOU accountable for how well you ensure the quality of data

collected by your team.

SECOND, team supervisors need to understand the status of EACH assignment in order to manage the

work of your team and ensure that all assignments are completed, reviewed, and approved.

As an assignment moves through the survey workflow, it takes on a different status, as illustrated in

Figure 6.

When an assignment sent from headquarters to the supervisor, that assignment is assigned to

the supervisor (in the terminology of the software, SupervisorAssigned).

When an assignment is allocated by the supervisor to a member of their team, that assignment

is considered as assigned to an interviewer (in the terminology of the software,

InterviewerAssigned).

When an assignment is completed by an interviewer and sent to the supervisor, that

assignment is marked as completed by the interviewer (in the terminology of the software,

Completed).

If the supervisor approves a completed assignment, that assignment is approved by the

supervisor (in the terminology of the software, ApprovedBySupervisor).

If the supervisor rejects the completed assignment, that assignment takes on the status of

rejected by the supervisor (in the terminology of the software, RejectedBySupervisor)

When headquarters rejects an assignment, that assignment is considered as rejected by

headquarters (in the terminology of the software, RejectedByHeadquarters)

When headquarters approves an assignment, that assignment is considered as approved by

headquarters (in the terminology of the software, ApproveByHeadquarters)

6

Figure 6: Status of an assignment in each stage of the survey workflow

Changing the Interface Language

Survey Solutions interface for Headquarter and Supervisor users can be presented in different

languages—Russian, Chinese, or French. This article describes how to switch the interface to be

presented in an alternative language.

Recall that both HQ and Supervisor users work via a web browser. Switching the interface is done by

telling the browser to request the content in a particular language. Note that this affects both Survey

Solutions and other sites that are visited with the same browser.

The remainder describes step-by-step instructions for the particular browsers you may be using. Other

browsers likely to have similar functionality.

Google Chrome

The following instructions are based on Google Chrome version 50.0.2661.102.

1. Start Google Chrome browser;

2. Click on the drop down menu and select Settings item;

3. Click the Show advanced settings... link at the bottom;

4. Click Language and input settings button;

7

5. Click the Add button in the Languages list to add a language you need.

6. Drag and drop that language to the top of the list.

7. Click Done.

8. Close and restart the browser.

Mozilla Firefox

The following instructions are based on version 45.0.1 of Mozilla Firefox.

8

1. Start the Mozilla Firefox browser.

2. Click on the drop down menu and select Options item;

3. Select Content tab.

4. In the Languages section, click the Choose... button.

5. Select a new language you want to add from the drop-down menu and click the Add button.

6. Adjust the order of the languages to have the desired language at the top.

7. Click OK.

8. Close and restart the browser.

9

Microsoft Internet Explorer

The following instructions are based on the version 10.0.9200.16750 of Microsoft Internet Explorer.

1. Start Internet Explorer.

2. Click the gear icon to access the drop-down menu.

3. Select Internet Options.

4. In the General tab click Languages button:

5. Click the Add... button to add the desired language.

10

6. Use the Move up and Move down buttons to change the order of the languages to have the desired

language at the top of the list.

7. Click OK.

8. Close and restart the browser.

If you are using a different browser the steps to take are likely very similar. See also the W3C

Internationalization page for the additional information.

The components of the supervisor software

The supervisor software is a suite of connected tools for the supervisor to track the progress of their

team (Reports), manage the quality of their work (Interviews), and manage their members of the team

(Interviewers). The following section will provide an overview of each tool—Reports, Interviews, and

Teams and Roles—and of how each tool works.

Reports

Reports tab provides four different types of reports: Survey and Statuses, Team members and Statuses,

Quantity and Speed. For example, the “Surveys and Statuses” report (shown below), presents a

summary of assignments by questionnaire. Details on each report type and ways to tailor each report

are explained in the “See the overall progress of data collection” section.

11

Interviews

The Interview panel constitutes the core of where supervisor does their work. It is a detailed list of

survey assignments. To find this list, click on the Interviews tab at the top of the screen. The Interview

interface consists of several pieces.

FIRST (marked 1 in the image below) is the list of survey assignments, which captures location

information (Featured Questions), which member is responsible (Responsible), when the assignment

was last updated (Last Updated), whether it contains errors (Has Errors), and what status it has (Status).

SECOND (2) is a set of filters for narrowing the list down to the questionnaire (Template), team member

responsible (Responsible), and/or assignment status (Status).

THIRD (3) is a tool for allocating one or several survey assignments to a member of the team, selected

by clicking the tick box associated with the assignment.

FOURTH (4) is a button that can be clicked in order to review and approve or reject a completed survey

assignment.

12

The Interview panel allows the supervisor to (i) see the assignments allocated from Headquarters, (ii)

allocate the assignments to his/her team members, (iii) review completed assignments, and (iv) reject

or approve the assignments.

For any completed assignment, the supervisor clicks on the green button associated with that

assignment (depicted as 4 in the figure above). The supervisor software then opens an interface for

reviewing, commenting, and approving or rejecting the assignment.

The Interview review panel has several components (as displayed below).

“Filters” (marked 1 in the image below) are in the left-hand of the screen. They allow the supervisor to

navigate from one section of the questionnaire to another by clicking on a heading. Bolded headings

represent modules. Indented headings represent groups of questions or rows of rosters.

“Additional filters” (2) are on the top right-hand segment of the screen. They facilitate the display of

questions that may be of interest to the supervisor. The “All” filter presents all questions. The

“commented” and “flag” filters display those questions with comments or flags, respectively. The

“answered” questions, as the name suggests, are all those that have an answer. The “invalid” filter

displays those questions that failed a validation check (e.g., range, consistency with other responses,

etc.). The “supervisor’s” filter yields those special questions that only the supervisor sees and can

answer. The “enabled” filter returns questions that are available to be answered based on the skip logic

of the questionnaire and answers provided to key questions.

13

The center part of the screen (3) lists the questions and the answers of the content being reviewed. The

panel immediately to the right (4) is where the supervisor reads comments from the interviewer and

provides comments to the interviewer.

The final components are the green approve button and red reject button. These important buttons

determine whether the assignment is sent to headquarters for inclusion in the survey database

(Approve) or returned to the interviewer for correction (Reject).

Interviewers

The final major component of the supervisor software is the Interviewer dashboard, opened through

clicking on the Teams and Roles tab at the top of the screen.

This component is responsible for the small parts of human resource management that the supervisor

handles. Headquarters determines which human resources are in which team. Supervisor is relegated to

the role of managing the user accounts for their team. To do so, the supervisor clicks in the blue “Edit”

button associated with the team member (as depicted below).

14

How to perform common supervisor actions

Find a survey case

Use filters to narrow down your search—by questionnaire, by team member, and by survey status.

FIRST, click on the Interviews tab at the top of the screen (between the Reports and Interviewers tabs):

By questionnaire To filter by questionnaire, move your cursor to the Filter panel at the left of the screen. Then, click on

the drop-down menu below Templates. Select the questionnaire whose survey assignments you want

to see.

15

The results will be a list of only those assignments for that particular questionnaire, regardless of who is

responsible for them or what their status is.

By assigned team member

To filter by the team member responsible, click on the drop-down menu bellow Responsible and select

the team member whose assignments you want to see.

16

This will yield a list of all the assignments for that team member. The supervisor is also one team

member that can be selected. Selecting the supervisor’s questions so will provide a list of all those

assignments that have been sent to the supervisor by headquarters but not allocated to any other team

members (as depicted below).

17

By status To filter by status, click on the drop-down menu below Status and select the status of interest.

This will return the survey assignments that are in a given status. Of particular interest may be

assignments whose status is SupervisorAssigned or Completed. The SupervisorAssigned cases still need

be shared distributed to the team members. The Completed cases need to be reviewed and approved

or rejected by the supervisor.

An explanation of each survey status

The possible statuses of an assignment are be explained by this diagram.

18

Any – This will select assignments with any status—that is the full list of assignments for the

supervisor’s team, including those that have been completed.

SupervisorAssigned. Because all assignments come to the supervisor from headquarters, each new

interview assignment starts its life assigned to the supervisor—that is with a status of

SupervisorAssigned. This filter may be useful, then, for supervisors to identify those assignments

from headquarters that need to be assigned on to members of the supervisor’s data collection

team.

InterviewerAssigned. Once a supervisor reassigns a new interview from the supervisor to an

interviewer, that interview assignment’s status changes from SupervisorAssigned to

InterviewerAssigned. This filter provides the supervisor with a list of all of the interview assignment

have been assigned to individual interviewers but have not yet been completed.

RejectedBySupervisor. Supervisors play a strong role in quality control. They must either approve or

reject each questionnaire that an interviewer completes. If the supervisor rejects an interview, that

interview will have the status RejectedBySupervisor. If after review of the interview the supervisor

rejects the interviewer’s work then that case take on the status RejectedBySupervisor interviews are

by default returned to the interviewer that completed them, but the supervisor could filter

according by the RejectBySupervisor status in order to have the opportunity of reassigning a

rejected questionnaire to another interviewer within the team.

Completed. When an interviewer marks an interview as complete and synchronizes with the

supervisor, the completed interview assignment will be marked with the status Completed. The

19

supervisor may find this filter particularly useful in efficiently identifying interviews that need to be

reviewed and either approved or rejected.

ApprovedBySupervisor. If after careful review the supervisor approves an interview assignment, the

status becomes ApprovedBySupervisor. When the supervisor next synchronizes with the

headquarters this assignment will be sent to headquarters.

RejectedByHeadquarters. Headquarters plays a strong role in quality control. They must either

approve or reject each questionnaire that a supervisor approves. If the headquarters rejects an

interview, that interview will have the status RejectedByHeadquarters.

ApprovedbyHeadquarters. When headquarters approves an assignment, that assignment is

considered as ApprovedByHeadquarters.

Assign a survey case to an interviewer To begin, click on the Interviews tab at the top of the screen (between the Reports and Interviewers

tabs). The supervisor application will then show the full list of all assignments to the team.

Next, use filters to find the cases to be assigned. Consider the following common scenario.

Case not yet assigned to an interviewer Select the name of the survey from the Templates menu. Then select SupervisorAssigned from the

status menu (as displayed below).

20

This will display all of the survey cases that have not yet been assigned to an interviewer for a particular

survey or version of a survey (as displayed below).

Next, click in the tick box to the left of the survey case(s) to be assigned to an interviewer. Then, click

on the Assign interviewer and select an interviewer from the drop-down menu. This will assign the

selected survey case(s) to the selected interviewer. The interviewer will see these new assignments on

the dashboard of their tablet upon the next synchronization.

21

Case to be reassigned to another interviewer Use similar steps to reassign a survey case from one interviewer to another. This can be done in a few

quick steps. First, use filters to find the case or cases that need to be reassigned. The most common is to

reassign Rejected questionnaires or Complete questionnaires that are marked as complete but are not

(e.g., one interviewer does as much of the interview as they can but another interviewer needs to

continue the interview).

Second, select the case(s) by clicking in their tick box associated with the case(s). Then, select a different

interviewer from the drop-down Assign interviewer menu to be responsible for the interview.

22

While the steps for reassignment are simple, the supervisor needs to exercise extreme care in changing

the assignment of an interview. There is no problem in reassigning assignments that were not picked up

by their respective interviewers. However, if incomplete interviews are reassigned, there is a risk that

two interviewers may submit different versions of the same interview that will need to be reconciled

manually (e.g., one interviewer completes the interview, the other interviewer does not complete the

interview but marks it as complete).

The supervisor will know if any of the interviews being reassigned have been already received by the

assignees, and will have to confirm reassignment of such interviews:

This helps prevent the situations of unintentional duplication of assignments. Supervisors should

reassign interviews already received only in cases where there is reasonable confidence that the

interviewer has dropped out of the program and his device is unavailable.

Reassigning the interview does not magically carry over unsynchronized data from 1st interviewer to the

second one. Rather it redirects the assignment as seen by the supervisor (empty in most cases until

completed). If the first interviewer has opened a number of assignments and needs the second

23

interviewer to finish them, she should mark them as completed (even they are only partially filled out)

and let the supervisor know to reject them to the second interviewer. In that case the data already

collected to date will flow over to the second interviewer.

Review a survey case First, click on the Interviews tab at the top of the screen.

Then, click on the green icon next to the interview that you would like to review.

To review the questionnaire, the supervisor will need to use several tools.

Navigate To navigate through the questionnaire content, use two sets of filters. Those in the left-hand pane will

allow you to navigate to various groups of questions.

Clicking on bold-faced headings will bring you to the beginning of a module.

24

Clicking on non-bolded, indented headings will bring you to the beginning of groups of questions or

rows of a roster. The rows of roster will follow the following format: “Roster name: Roster row name”.

The supervisor can further navigate through the questionnaire by using the filters arrayed along the top

of the screen: all, flagged, commented, answered, invalid, supervisor’s, and enabled.

The “All” filter presents all questions. The “commented” and “flagged” filters display those questions

with comments or flags, respectively. The “answered” questions, as the name suggests, are all those

that have an answer. The “unanswered” questions are those that do not have an answer. The “invalid”

filter displays those questions that failed a validation check (e.g., range, consistency with other

responses, etc.). The “supervisor’s” filter yields those special questions that only the supervisor sees and

can answer. The “enabled” filter returns questions that are available to be answered based on the skip

logic of the questionnaire and answers provided to key questions.

Add a flag The supervisor will have review large amounts of information in each questionnaire. Flags enable the

supervisor to mark a particular question as a place to return later, perhaps when comparing the

consistency of an answer in an early module of the questionnaire with an answer in a much later

module of the questionnaire. The supervisor can see these flagged fields quickly, as explained above, by

clicking on the “flagged” filter along the top of the questionnaire review screen.

25

How does the supervisor add a flag? Follow these simple steps. First, hover over the question where you

want to leave a flag, moving your cursor to the left-hand side of the screen. Then, click on the flag icon.

Once a flag has been added, the supervisor can click on the “flagged” filter to see the field or fields

bearing a flag.

Add comments The space on the right-hand side of screen is also where the supervisor can both read comments from

the interviewer and provide comments to the interviewer.

To leave a comment, hover over the question to comment and move your cursor to the right-hand side

of the screen. Then, click on Comment text box.

Next, type your comment into the field provided and press on the Add comment button.

26

Approve, Reject The culmination of reviewing the questionnaire is the supervisor’s decision to approve or reject the

questionnaire.

To approve, click on the Approve button; to reject, the Reject button.

Whatever button is pressed, a confirmation screen will appear with the opportunity to provide

comments on the questionnaire as a whole. At this confirmation screen, click on the Approve or Reject

button. But be very sure of your decision before proceeding. There is no way to “un-approve” or “un-

reject” a questionnaire unless it is rejected by Headquarters.

See the overall progress of data collection

Surveys and statuses First, click on the Reports tab at the top of the screen and select “Surveys and Statuses” from the drop-

down menu.

This will yield team-level summary of how many assignments are in each status, by questionnaire.

27

Using the filter in the left-hand portion of the screen, you can select this same summary for individual

team members. To do this, click on the drop-down menu below “Team members” and select the

member whose summary you want to see.

Team members and statuses First, click on the Reports tab at the top of the screen and select “Team members and Statuses” from

the drop-down menu.

This report provides a summary of how many assignments are in each status, by team member

(including the supervisor).

28

To triage the counts in the table further, select the questionnaire for which you want to see summary

statistics using the drop-down menu under the Template rubric of the Filters pane on the left of the

screen. This will yield a summary of progress for the particular survey chosen.

29

Quantity The “Quantity” report provides the following reports for each questionnaire template:

Number of completed interviews: Provides the number of completed interviews for the supervisor’s data collection team. Interviews that have been rejected and then completed are counted as an additional completed interview.

Number of interview transactions by HQ: Provides the number of headquarters approvals and rejections for the supervisor’s data collection team.

30

Number of interviews approved by HQ: Provides the number of headquarters approved

interviews for the supervisor’s data collection team.

Number of interview transactions by supervisor: Provides the number of supervisor approvals

and rejections on interviews collected for the supervisor’s data collection team.

31

To use these report, first select “Quantity” from the Reports menu. Then, select

the “Type” of report and the questionnaire template.

Select a start date for the report (From). Next, define the time interval-- day,

week, or month (Period), and the number of time intervals that you would like

to display in the report (Period count). The maximum period count that can be

defined is 12.

Speed The “Speed” report provides the following reports for each questionnaire template:

Average interview duration: Provides the average time difference between the first recorded

answer on a tablet and when the complete button is selected.

32

Average supervisor processing time: Provides the average time difference between the moment

the complete button is pressed and the supervisor’s approval or rejection of that questionnaire.

Average HQ processing time: Provides the average time difference between a supervisor

approval of a questionnaire and a HQ approval or rejection of that questionnaire.

33

Average case assignment duration: Provides the average time difference between survey case

assignment to an interviewer and the moment an interviewer sets the survey case as complete

on a tablet.

Average overall case processing time: Provides the average time difference between survey case

assignment and survey case approval by HQ.

34

To use these reports, first select “Speed” from the Reports menu.

Then, select the “Type” of report and the questionnaire template. Select a start

date for the report (From). Next, define the time interval-- day, week, or month

(Period), and the number of time intervals that you would like to display in the

report (Period count). The maximum period count that can be defined is 12.

Change the password for an interviewer account During the survey, an interviewer may forget his or her password. To address problem, the supervisor

can change the interviewer’s password.

To start, click on the Teams and Roles tab at the top of the screen.

Then, click on the “Edit” button associated with the interviewer whose password needs to be changed.

35

Next, compose a new password and confirm it. Then click on the “Save changes” button at the bottom

of the screen.

Lock an interviewer account An interviewer may leave the survey for a variety of reasons, from illness to poor job performance. If

such a situation arises, you may need to lock the interviewer’s account. But before locking the

interviewer account, make sure that you have recovered all completely and partially complete

interviews from the interviewer’s tablet.

When you are ready to clock the account, first click on the Teams and Roles tab at the top of the screen

.

Then click on “Details” button associated with the team member whose account you would like to lock.

36

Next, click in the box next to “Is Locked”. Finally, click on the “Save changes” button to lock the account.