Training Questionnaire design :

March 2009

Objetives of the training

Objectives of the session is to understand:

Objectives of questionnaires

Advantages and disadvantages

Design of questionnaires

Type of questions used

Common problems and pitfalls

Training plan:

Impact

Intervention

Presentation plan:

Types of questionnaires

Stages for design

Presentation and layout

Type of questions

Closed questions

Problems and pitfalls

Additional information

Training questionnaire design I: Types of questionnaires

1.0 Stages of a survey design

Establish the goals of the project

Determine your sample - Whom to interview

Choose interviewing methodology - How to interview

Create your questionnaire

1.1 What is a questionnaire

A series of written questions/items in a fixed, rational order

A questionnaire is an instrument (form) to: Collect answers to questions Collect factual data Gathers information or measures

A well designed questionnaire: Gives accurate and relevant information to your

research question Minimises potential sources of bias Will more likely be completed

1.2 Advantages /disadvantages

Can reach a large number of people relatively easily and economically

Provide quantifiable

answers

Relatively easy to

analyse

Provides only limited insight into problem:Limited response

allowed by questionsMaybe not the right

questions are asked Varying response:

Misunderstanding/misinterpretation

Need to get it right first time:Hard to chase after

missing data

1.3 Types of questionnaires

Face to Face:

Telephone:

By post:

Email/internet:

Interviewer-administrated Vs Self-administrated

1.4 Advantages /disadvantages

Advantages: Cheap and easy to administer Preserves confidentiality Completed at respondent's

convenience No influence by interviewer

Disadvantages: Low response rate Questions can be

misunderstood No control by interviewer Time and resouces loss

Self-administered questionnaire:

Interview-administered questionnaire:

Advantages: Participation by illiterate people Clarification of ambiguity Quick answers

Disadvantages: Interviewer bias Needs more resources Only short questionnaires possible Especially on telephone Difficult for sensitive issues

1.5 Choice of the type of questionnaire

Choice of the questionnaire type will depend on several factors such as:

Speed

Cost

Internet Usage

Literacy Levels

Sensitive Questions

Training questionnaire design II: Stages for design

2.1 Stages in designing the questionnaire I

Design your survey plan: Decide on goals:

Identify risk factors for population affected by the earthquake

Know the subject: Lessons learnt, secondary data review

Formulate a hypothesis: People affected by the earthquake will face food security and

water access issue in the coming weeks

Define information needed to test hypothesis: Current access to water, changes in food consumption, etc..

2.2 Stages in designing the questionnaire II

Determine study population: sampling

Know the respondents: Occupation Special sensitivities Education Ethnicity / Language

Questionnaire needs to be adapted to your population, not the opposite!

Design variables and questions: Content of the questions

Format of the questions

Presentation and layout

Coding schedule (if appropriate)

Pilot and refine questionnaire

Training questionnaire design III: Presentation and layout

3.1 Presentation and layout

Clear consistent layout:Adequate space to answer / Large font sizeAppropriate page breaksAvoid

experimental layouts fancy logos printed on recycled paper/is an equal opportunity employer

etc

Using colour or printing questionnaire on coloured paper may help

Use filter questions, if necessary

Give clear instructions about how to answer the questions (design guideline if necessary)

3.2 Good practice for layout

Good appearance / easy on the eye

Short and simple

Relevant and logical

High response rate Easy data

summarisation and analysis

Vom Befrager einzutragen: Datum der Befragung: __.__._____ Initialen des Befragers: ____ Name des Arztes: _____________________ Befragung vollständig, keine weiteren Anrufe oder Nachfragen notwendig! Guten Tag, mein Name ist _______________: Ich bin Mitarbeiter im Kreisgesundheitsamt in Husum / Als Mitarbeiter des Robert Koch-Institutes unterstütze ich das Kreisgesundheitsamt Husum. Wir haben derzeit im Landkreis Nordfriesland eine ungewöhnliche Häufung von Masernerkrankungen. Wir möchten uns zunächst herzlich dafür bedanken, dass Sie uns Ihren Patienten ____________________________(Name des Falles) geboren am ______ als Masernfall gemeldet haben. Wir führen derzeit eine epidemiologische Untersuchung durch um diese Masernhäufung eindämmen zu können. Das Erfassen von Masernkomplikationen ist hier besonders wichtig. Vor kurzem müssten Sie von uns ein Fax mit weiteren Erläuterungen zu dieser Häufung erhalten haben, in dem wir auch auf diese Nachrecherchen hingewiesen haben. Um den von Ihnen gemeldeten Fall korrekt einordnen zu können, möchten wir noch kurz einzelne Informationen abklären. Fragenkomplex Symptome: Hatte Ihr Patient eines der mehrere der folgenden Symptome? Generalisierter Hautausschlag > 3 Tage: Ja Nein Unbekannt Wenn Beginn des Hautauschlags exakt bekannt ist, Datum genau angeben: __.__.____ Wenn Beginn des Ausschlages NICHT exakt bekannt ist, dann Zeitpunkt eingrenzen, nämlich: Beginn Ausschlag irgendwann zwischen dem __.__.____ und dem __.__.____ Fragenkomplex Labor:

Liegt Ihnen zu diesem Patienten einer oder mehrere der folgenden Labornachweise vor? (1) Virusisolierung (2) Nukleinsäurenachweis (PCR) (3) IgM (4) IgG mit 4-fachem Titeranstieg (5) Antikörpernachweis mit 4-fachem Titeranstieg

War Ihr Patient gegen Masern geimpft? Erste Masernimpfung: Ja (Impfausweis) Ja (aber nicht dokumentiert) wenn ja, wann (z.B. laut Impfausweis): __.__._____ Nein Unbekannt Zweite Masernimpfung: Ja (Impfausweis) Ja (aber nicht dokumentiert) wenn ja, wann (z.B. laut Impfausweis): __.__._____ Nein Unbekannt Dritte Masernimpfung: Ja (Impfausweis) Ja (aber nicht dokumentiert) wenn ja, wann (z.B. laut Impfausweis): __.__._____ Nein Unbekannt Arzt hat in der Praxis noch andere Masernfälle gehabt, die noch nicht gemeldet wurden: Ja Nein Wenn ja, dürfen wir Sie bitte uns diese Fälle schnellst möglich alle nachzumelden, denn davon hängt ab, wie realistisch wir diese Häufung einschätzen können. Haben Sie bei Ihrem Patientengut subjektiv den Eindruck, dass im Vergleich zum entsprechenden Zeiträumen der Vorjahre dieses Jahr deutlich mehr Masernfälle aufgetreten sind? Ja Nein unbekannt Sie werden in wenigen Tagen auf dem Postweg weitere Informationen zu der Masernhäufung von uns erhalten, Haben Sie hierzu Fragen? Vielen Dank für ihre Mithilfe!

Masern-Ausbruch Nordfriesland Juni 2001

Dr. Gerard Krause Abt. Infektionsepidemiologie Robert Koch-Institut Seestrasse 10 13353 Berlin

Vom Befrager einzutragen: - Datum der Befragung: ___.___._____ - Initialen des Befragers (Vor-+ Nachname): ___ ___ - Name des Arztes: ________________________________ - Befragung vollständig? (1) Ja (2) Nein, Nachfrage notwendig.

Guten Tag, mein Name ist ___ (bitte Namen nennen)___.

Ich bin Mitarbeiter im Kreisgesundheitsamt in Husum

oder Als Mitarbeiter des Robert Koch-Institutes unterstütze ich das Kreisgesundheitsamt Husum.

Wir verzeichnen derzeit im Landkreis Nordfriesland seit Anfang September 2005 eine ungewöhnliche Häufung von Masernerkrankungen. Wir möchten uns zunächst herzlich dafür bedanken, dass Sie uns

Ihren Patienten ___(Name des Falles wie in Kopfzeile)___, geboren am __(Datum wie in Kopfzeile)__,

als Masernfall gemeldet haben. Wir führen derzeit eine epidemiologische Untersuchung durch um diese Masernhäufung eindämmen zu können. Das Erfassen von Masernkomplikationen ist hier besonders wichtig. Um den von Ihnen gemeldeten Fall korrekt einordnen zu können, möchten wir noch kurz einzelne Informationen abklären. 1. Hatte Ihr Patient eines der mehrere der folgenden Symptome? Generalisierter Hautausschlag > 3 Tage: (1) Ja* (2) Nein (3) Unbekannt * Wenn ja: Wann begann der Hautauschlag?

(4) Datum genau bekannt: ___.___.2005 (5) Datum nicht genau bekannt: Auschlag begann zwischen ___.___.2005 und ___.___.2005. 2. Welche der folgenden Labornachweise liegen Ihnen zu diesem Patienten vor?

(Mehrfachnennungen möglich)

(1) Virusisolierung (2) Nukleinsäurenachweis (PCR) (3) IgM (4) IgG mit 4-fachem Titeranstieg (5) Antikörpernachweis mit 4-fachem Titeranstieg

3. War Ihr Patient gegen Masern geimpft? (bitte eine Angabe pro empfohlene Impfung)

Erste Masernimpfung:

Zweite Masernimpfung: Dritte Masernimpfung:

(1) Nein (2) Ja* (nicht dokumentiert) (3) Ja* (Impfausweis)

(1) Nein (2) Ja* (nicht dokumentiert) (3) Ja* (Impfausweis)

(1) Nein (2) Ja* (nicht dokumentiert) (3) Ja* (Impfausweis)

3.3 Organizing questions

Decide on order of items/questions : Easy difficult General particular Factual abstract

Where to place sensitive questions?Be aware of ordering effects!

Group questions by topic/ response options

Starting questions: Simple With closed format Relevant to main subject Non-offending Neither demographic nor personal questions

Don’t put most important item last!

Training questionnaire design IV: Type of questions

4.1 Contents of questions

Clear focus on research question

Avoid sidetracking

Avoid unnecessary information

Demographic information

Contact information (if non-anonymised)

4.2 Format of questions

Adjust to responding audience: Professionals vs. public Middle class vs. prisoners

Keep sentences simple and short

Define key words (“fully vaccinated”)

Remember option “don’t know”

Ask for one information at a time:

Use mutually exclusive and exhaustive answer options

Vertical order of answer options

Do you own cattle or have frequent contacts with cattle?

Yes No

4.3 Be accurate

Do you often touch cattle?

Yes No

How often did you touch a cattle during the past 3 months?

Once Twice Three times or more Not at all Don´t know

The range of error due to use of imprecise words may be as high as 20 to 30 percent.

4.4 Be objective

Did you drink the strange brownish drink in Rumania?

Yes No

Which beverage did you consume?

Water

Beer

Wine

Karkadé

None of them Don´t know

4.5 Be simple

Did you smoke not less than a mean amountof 7 cigarettes/2 days from 1999 onwards?

Yes No

Did you smoke an average of 2 pack of cigarettes/week for the last 5 years?

Yes No Don´t know

4.6 Bias

Bias = systematic differences in the measurement of a response

Recall bias: Cases more likely to remember than controls

Observer bias: Different interviewer – different interpretations Different interpretation of similar questions Reduce by structured questionnaire

Non response bias: Those who respond are different from those who do not

Telephone interviews: more females, elderly

Reduce Ensure high response rate Random choice of interview partners Correct during analysis (eg age, sex)

4.7 Format of questions

Two main question formats:Closed format forced

choice:

Yes No Don’t know

Always Sometimes Never

Open format free text:

Why do you feel concern about food accessibility during the next month? Please describe: __________________________________________________________________________________________________________________________________________________________

Multiple choice

Numeric open end

Rating scale

Agreement scale

Text open end

4.8 Closed or Open questions?

Advantages: Simple and quick Reduces discrimination against

less literate Easy to code, record, analyse Easy to compare

Easy to report results

Disadvantages: Restricted number of

possible answers Loss of information

Possible compromise: Insert field “others“

Advantages: Not directive Allows exploration of issues to generate hypothesis, qualitative research, focus groups, trawling questionnaires Used even if no comprehensive range of alternative choices Good for exploring knowledge and attitudes Detailed and unexpected answers possible

Disadvantages: Interviewer bias Time-consuming Coding problems Difficult to analyse! Difficult to compare groups

Training questionnaire design V: Closed questions

5.1 Closed questions

Straightforward response:

What is your age in years? ___ years

How long have you owned a cheap? ___ years

What is your sex (gender)? Male Female

Did you stay in your house when the earthquake occurred?

Yes No Don’t know

5.2 Closed questions

Checklist:

Which of the following agricultural activities did you do last week?

Buying seeds Preparing your field Planting Harvesting

5.3 Closed questions

Rating scale - Nominal:

Did you do use soap during the following domestic activities during the past six months?

Always Sometimes Seldomly Never

Cooking After toilets After fieldwork

Indicate your sex: ___Male ___Female

5.4 Closed questions

Rating scale – Numerical:

How useful would you think that information on the risk of being strike by earthquake? (please circle)

1 2 3 4 5 6 7Not at all useful Very useful

Analogue:

How much is your food insecurity severe (put the tick on the line)

0 10

5.6 Closed questions

Scales for measuring attitude (Lickert) Stray dogs carry a higher risk of rabies?:

No, I strongly disagree No, I disagree quite a lot No, I disagree just a little I’m not sure about this Yes, I agree just a little Yes, I agree quite a lot Yes, I strongly agree

5.7 Coding schedule

Questionnaire can be pre-coded

Quicker and easier data entry

Examples:Male 1 ill 1 Female 2 Not ill 0 Don’t know 3 Don’t know 9

Single 1 Separated 3 Married 2 Divorced 4 Widowed 5 Don’t know 9

Training questionnaire design VI: Problems and pitfalls

6.1 Problems and pitfalls

Avoid questions that ask two things at once - you won’t know which ‘bit’ people are answering:

Have you ever had stomach ache and diarrhoea?

Avoid ambiguity.....

Do you go to the woods a lot?

Avoid jargon/abbreviations/slang:

How often do you get up at night to PU? (pass urine) Should IVDUs be treated in the community?

6.2 Problems and pitfalls

Avoid not mutually exclusive options:

What age are you? 16-20 20-25 25-30 35-40

Avoid leading questionsDo you think that the food in the hotel made you sick?Did the hotel staff seem unhygenic to you? Do you agree that the hospital staff were close to exhaustion?

Typographical / spelling errors

6.3 Length of a questionnaire

Sufficient to capture needed data

Short enough to hold participants’ attention

Type of survey affects length

Types of questions affect length

Quantitative/Qualitative/Mixed approach affects length

Participant characteristics affect length

6.3 Questionnaire validation

Use or adapt existing questionnaires Validated (and possibly harmonised)

New questionnaires Not validated Needs to be tested (pilot)

Pilot with a similar group of people to your intended subjects

Highlights problems before starting Effects of alternative wording Overall impression on respondents and interviewers Final polishing after several amendments

6.4 Questionnaire introduction

Covering letter/ interview introduction:

Who you are/ you work for Why you are investigating (purpose of the survey) Why it is important to hear from the respondent

What may be done with the results and what possible impacts may occur with the results.

Where you obtained the respondent’s name How and where you can be contacted Guarantee of confidentiality Length of interview (be honest) Due date for response

Usefulness of study should be clear to all respondents

Training questionnaire design VII: Summary

7.1 Designing a questionnaire

It’s all about the questions…

Effect of a given word

Balance in question wording

Don’t know answers – offered or volunteered?

Using scales

Question order

Pre-testing

7.2: Designing questions

When devising your questions consider the following:

What is essential to know?

What would be useful to know?

What would be unnecessary?

Retain the former, keep the useful to a minimum and discard the rest

7.3 Best practices

A well designed questionnaire:

Will give appropriate data which allow to answer your research question

Will minimise potential sources of bias, thus increasing the validity of the questionnaire

Will much more likely be tested and completed

FINALLY, keep your questionnaire short and the questions simple, focused and appropriate

Training questionnaire design VIII:

Additional information

7.1 Documentación Questionnaire design

Survey method: