Research Methods Activity > Questionnaires“It is harder to produce a really good questionnaire than might be imagined.”

Key Points Reasons for Using

If carefully planned, it is a useful method for gathering a large number of general data and opinions from a large number of people relatively quickly.

Often useful at the start of a project – can provide baseline data and shape the nature of questions you wish to ask in interviews or consider when observing.

Before proceeding with a questionnaire the following needs to be considered: what type of information is needed to answer the research questions and who are the respondents?

Framing questions How responses will be analysed needs to considered at the design stage. Go back to hypothesis/objectives and decide on the questions you need to ask to

achieve these objectives - then work on ordering & wording. Each question should have a purpose. There are a number of issues regarding the wording of questions:

Type of question needs to considered. There are seven types of question, each with different benefits: verbal/open, list, category, ranking, quantity, grid & scale.

Start with questions about the factual information required. Do a pilot run and make adjustments as necessary. Questions do not always have to use words.

Collecting Reponses Anonymity: questionnaires are often anonymous. Need to consider that anonymity

means that responses cannot be followed up with individuals. Preservation of anonymity of respondents needs to carefully considered if promised.

How are you going to collect? Adults: In person or pigeonhole drop? Collecting in person leads to a much

higher response rate. Otherwise, distribute twice as many as are needed! Pupils: Completing collectively in class normally works well. An appropriate

atmosphere can be created to ensure respondents answer questions in good faith. Need to minimise the potential for influence by research and peer pressure. Questions can be read out – especially useful for mixed ability groups.

Plan how you will deal with non-response.

Analysing Collate data as you get it – often by tally. Open-ended responses are more difficult – can be categorised by researcher,

however there is an issue of the researcher attributing meaning. Otherwise, read through and note the significant points and the number of times they were cited.

Describe & explain the results – use charts & graphs to express results visually. Also need to find out if the results are statistically significant.

Interpretation – making sense of your results – what is the impact on original hypothesis/existing research?

Fearghal Kelly

Issues to consider when wording

questions

avoid ambiguity/imprecision

avoid making assumptions

avoid respondents using their memory

consider reading ability of respondents

be careful with sensitive issue questions

avoid hypothetical questions

avoid presuming questions

avoid leading questions

avoid double questions

avoid questions requiring knowledge

Strengths & Weaknesses of Questionnaires

Strengths Weaknesses Gather large amount of data, quickly. Allows comparison of responses to

particular questions. Data can be expressed statistically,

allowing comparisons with other studies. May enable overall statements to be made

concerning the sampled population. Enable the collection of background &

baseline information. Provide information in a format which is

easy to represent.

Questions about complex issues are difficult to compose & responses can be difficult to categorise.

Short responses often fail to reflect varying depth or complexity of feelings.

Researcher sets the agenda, not the respondent.

Questions asked can introduce subjectivity & bias.

Answers to open-ended questions are often codified leading to subjectivity.

Returns and response rate may be too low to ensure a valid research outcome.

Designing and Administering Questionnaires Checklist

1. Make sure you have approval to proceed from ethics/research committees, if appropriate, before you move too far with your preparation. Remember this can take some time.

2. Decide what you need to know and list all items about which information is required.3. Is a questionnaire the best way of obtaining information?4. If you decide a questionnaire will be best, begin to word questions. Write them on separate

cards to help ordering them later on.5. Check the wording of each question.6. Decide on a question type.7. When satisfied that all questions are well worded and of the right type, sort them into order.8. Write out instructions to be included on the questionnaire.9. Consider layout & appearance. Instructions must be clearly presented. Do you need a right-

hand margin for coding?10. Word-process your questionnaire.11. Decide on your sample. 12. Always pilot your questionnaire, no matter how pushed for time you are.13. Try out your methods of analysis on your pilot data.14. Make any adjustments to the questionnaire in light of pilot respondents’ comments and your

preliminary analysis.15. Decide how the questionnaire is to be distributed but before you distribute, check that you

know what you mean by anonymity and confidentiality – and that you make your definitions clear to your respondents.

16. Don’t forget to say when you would like questionnaires to be returned, if possible. Give the preferred day & date.

17. Decide what you are going to do about non-respondents before you distribute the questionnaires.

18. Begin to record responses as soon as completed questionnaires are received.19. Do not get involved with complicated statistics unless you know what you are doing.

Reading

Koshy, V. (2005) Action Research for Improving Practice. London: Paul Chapman Publishing.Burton, D. and Bartlett, S. (2005) Practitioner Research for Teachers. London: Paul Chapman Publishing.Bell, J. (2005) Doing Your Research Project, 4th edn. Maidenhead: Open University Press.

Fearghal Kelly