Surveys: Guidance on question wording
This resource offers some general advice on how to ask survey questions for self-completion questionnaires (e.g. web forms or paper questionnaires). This is not meant as a comprehensive guide to question design, instead it aims to provide brief guidance on asking a range of survey questions and how to avoid some common errors. Alongside the information given here, you should also consider the nature of your target audience when writing survey questions; a question aimed at children will be worded very differently to one designed for, say, company chief executives.
When writing survey questions, you should ask yourself what you are attempting to measure or gauge and then come up with an appropriate way of asking (or framing) the question and, where closed questions are used, an appropriate set of response categories. The list below details some of the more common opinions and occurrences you might wish to measure via a survey question, click on the appropriate link to see guidance on asking the question:
Agreement
A commonly used question format is to provide a list of statements and ask the respondent to indicate the extent to which they agree or disagree with each statement.
Suggested ways of framing the question: |
Please indicate whether you agree or disagree with the following statements |
Please indicate the extent to which you agree or disagree with the following statements |
To what extent do you agree or disagree with the following statements? |
Suggested response categories: |
Strongly agree |
Agree |
Neither agree nor disagree |
Disagree |
Strongly disagree |
|
Agree |
Disagree |
Don't know / unsure |
|
Notes:
- Depending on the nature of the statements, it may be necessary to have an 'opt out' such as 'not applicable' / 'don't know' / 'not experienced' etc.
- You may wish to consider whether to use the 'middle option' of 'neither agree nor disagree' - but bear in mind that excluding such an option may force respondents to express an opinion which they do not actually hold.
- Think carefully about how you express the statements respondents are being asked to consider, for example, try not to make them double-barrelled as the respondent may agree with the first part but not the second part of the statement, for example:
The University should encourage participation in sports by offering free sports membership and by improving the facilities available on each campus
Respondents may agree with one part of this statement but not necessarily with the other. In instances such as this it would be better to offer two or more separate statements so that respondents can indicate exactly which element they agree or disagree with.
Amount / quantity
You might wish to gauge respondents' perceptions of the amount or quantity of something they have been offered or received - for example, the amount of information provided, opportunities to do something, the amount of contact with a course tutor etc.
Suggested ways of framing the question: |
Overall was there too little or too much ...? |
How appropriate was the number of …? |
Is the proportion of ….[e.g. time] allocated to …[e.g. teaching x]...? |
Did you receive sufficient ... [e.g. information] from the following sources? |
Do you consider the ... [e.g. workload] to be generally appropriate? |
Were there adequate opportunities to…? |
Suggested response categories: |
Too few |
About right |
Too many |
|
Too high |
About right |
Too low |
|
Too much |
Sufficient |
Not enough |
|
More ... needed |
Amount was about right |
Less ... needed |
|
There is too much ... |
The amount of ... is about right |
There is not enough ... |
|
|
Too much |
About right |
Not enough |
|
As much as I wanted |
A little less than I wanted |
A lot less than I wanted |
(depending on what you are measuring it may make sense to also have categories for 'a lot more' and a 'little more' than I wanted) |
Notes:
- Remember that you can include a follow-up open-ended question, asking respondents to elaborate if they indicate there was too much or not enough of something (see separate notes on asking open-ended questions).
- Consider whether to include a 'don't know', 'unsure' or 'unable to comment' category alongside each of the above.
Degree / extent
Questions which aim to establish the degree or extent of something cover a broad range of topics, for example, how far respondents feel a certain way (e.g. enthusiasm, anxiety, agreement), or the extent to which they have made use of something, or the extent to which x has caused or contributed to y and so on. The following offers some general suggestions for constructing questions of this nature:
Suggested ways of framing the question: |
|
Please indicate the extent to which you… |
How far would you say you…? |
|
To what extent did the following ….. contribute to? |
To what degree have you…? |
Suggested response categories: |
Greatly |
To some extent |
Not at all |
|
Fully |
Partially |
Not at all |
|
More ... [e.g. enthusiastic] |
Less ... [e.g. enthusiastic] |
Made no difference |
|
Major contribution |
Minor contribution |
No contribution |
|
Severely |
Moderately |
Slightly |
|
|
A main cause |
A minor cause |
Not caused … |
|
Definitely yes |
Probably yes |
Uncertain |
Probably no |
Definitely no |
|
A main ... [e.g. source of support] |
An occasional ... [e.g. source of support] |
Not used / not relevant |
|
Notes:
- Consider whether to include opt out categories such as 'don't know', 'no experience', 'not sure' etc.
Frequency / length of time
You may wish to ask respondents how often they do something or for how long they have done something. The following suggestions should assist with such questions:
Suggested ways of framing the question: |
How frequently do you...? |
How often do the following things happen? |
|
|
Approximately how long have you [done x]…? |
|
What proportion of your time is devoted to…? |
|
As far as you can tell, how much longer do you expect to …? |
|
|
Suggested response categories: |
Often |
Sometimes |
Rarely |
Never |
|
|
Most / all
of the time |
Sometimes |
Rarely / never |
|
Most of the time |
Some of the time |
Rarely |
Never |
|
Every time |
More than once a week |
More than once a month |
More than once a year |
Once a year or less |
|
Daily |
Frequently |
Rarely |
Never |
|
Never |
Occasionally |
Often |
Always |
|
You may also wish to use more specific categories to describe the length or proportion of time, e.g: |
Less than 6 months |
6-11 months |
1-5 years |
More than 5 years |
|
Less than a year |
One year but less than two years |
Two years but less than three years |
Three years but less than four years |
Four years or more |
|
Over 75% of my
time |
51%-75% of my time |
25%-50% of my
time |
Under 25% of my time |
|
Notes:
- Consider whether to include opt out categories such as 'don't know', 'not sure', 'can't say' etc.
- If using a specific category to describe length of time (e.g. number of years, percentage of time etc), ensure your categories do not overlap, e.g:
0-1 year, 1-3 years, 3 years or more - anyone wishing to select 1 year or 3 years has two possible categories to tick, instead use:
Less than 1 year, 1-3 years, More than 3 years
Satisfaction
Survey questions commonly seek to establish how satisfied a respondent is with something, such as a service, an event, the performance of an organisation etc.
Suggested ways of framing the question: |
How satisfied are/were you with…? |
|
If you [did x], how satisfied were you with [y]? |
|
|
Please rate the following aspects of …? |
|
|
|
Suggested response categories: |
Very satisfied |
Fairly satisfied |
Not satisfied |
|
|
Very good |
Good |
Satisfactory |
Poor |
Very poor |
|
Generally very good |
Generally fairly good |
Generally poor |
|
|
|
Very satisfied |
Fairly satisfied |
Fairly dissatisfied |
Very dissatisfied |
|
Excellent |
Good |
Satisfactory |
Poor |
Very poor |
|
Highly satisfied |
Mainly satisfied |
Satisfied |
Not very satisfied |
Not at all satisfied |
|
Well above average |
Above average |
Average |
Below average |
Well below average |
|
Notes:
- Consider whether to include opt out categories such as 'don't know', 'no experience', 'not sure' etc.
Effectiveness
The suggestions below may assist in wording a question to evaluate the effectiveness of, say, a service or new initiative, or how well something is going:
Suggested ways of framing the question: |
How effective were the following … [e.g. in helping you to achieve the learning objectives]? |
|
|
How well did this arrangement work? |
|
|
|
How well are you coping with ... [e.g. your work in each of the following subjects]? |
|
|
|
How well did x achieve y? |
|
|
Suggested response categories: |
Very well |
Quite well |
Hardly / not at all |
|
Very effective |
Fairly effective |
Not effective |
|
Mostly effective |
Sometimes effective |
Rarely effective |
|
It worked very well |
It worked fairly well |
It did not work well |
|
Very well |
Well |
Not too well |
Poorly |
|
Well above average |
Above average |
Average |
Below average |
Well below average |
|
Notes:
- Consider whether to include opt out categories such as 'don't know', 'no experience', 'not sure' etc.
- Some of the response categories listed under 'satisfaction' may also fit with effectiveness questions.
Expectations / comparisons
You may wish to ask respondents how an event / service etc. lived up to their expectations, or you may wish to gauge respondents' opinions of a new service or system in comparison with the old service or system. The following suggestions should provide some ideas as to how to ask these questions:
Suggested ways of framing the question: |
How does this compare to what you expected … [e.g. before you started university]? |
|
To what extent do you consider that these things have changed over the past ... [e.g. three years]? |
|
How would ….. affect your feelings about ….? |
|
How do you think ... compares to ... in relation to the following areas? |
|
Do you think the new ... is/are better or worse than the existing arrangements…? |
|
Suggested response categories: |
Much worse now |
Worse now |
No change |
Better now |
Much better now |
|
Much more ... |
More ... |
No change |
Less ... |
Much less ... |
|
More ... than 2 years ago |
No different |
Less ... than two years ago |
|
Definitely more appealing |
Slightly more appealing |
Made no difference |
Not appealing |
|
More than I expected |
About what I expected |
Less than I expected |
|
New system is better |
No difference |
Previous system was better |
|
A lot more |
A bit more |
About the same |
A bit less |
A lot less |
|
Much better |
A bit better |
About the same |
A bit worse |
A lot worse |
|
Much better |
A little better |
No change |
A little worse |
Much worse |
|
Notes:
- Consider whether to include opt out categories such as 'don't know', 'no experience', 'not sure' etc.
Open-ended questions
You may wish to ask respondents open-ended questions which require them to answer in their own words. Often these will ask for elaboration on answers given to a previous closed question, or for further comments on a specific issue . The following list provides suggested ways of asking these questions:
Suggested ways of framing the question: |
Please use the space below to … [e.g. comment on x] |
|
|
What do you most like / dislike about…? |
|
|
Do you have any suggestions about ways in which ... could be improved / further developed? |
|
|
Are there any specific ways in which ... could be improved? |
|
|
Please use the space below to amplify any of the answers given above |
|
Please indicate / describe your main reasons for ... in the space below |
|
If you have any further comments on the issues raised in this questionnaire, please use the space below |
|
What do you think are the good features of … ? |
|
How do you think ... could be improved? |
|
What would be the advantages / disadvantages of ...? |
|
Please comment on any areas / aspects you have rated as … [e.g. 'good' / 'poor' / 'unsatisfactory' etc.] |
|
- Use open-ended questions sparingly (particularly if you have a large number of potential respondents) as responses to these questions take longer to process and analyse than closed responses.
- Keep your open-ended questions specific and focused to ensure that answers relate to topics you are interested in; a broad open-ended question will elicit a broad range of responses, some of which may not be relevant to the subject of your research.
Contact details
Survey Unit
University of Nottingham
University Park
Nottingham NG7 2RD
Email: survey-unit@nottingham.ac.uk
Tel: +44 115 84 66091
Fax: +44 115 84 66090
|