When you ask a community for an honest review, the way you ask matters. A question can sound polite and still push someone towards the answer an organisation would prefer. I find that troubling because we may later call those responses evidence of what people think. Before counting the answers, we should look carefully at the questions that produced them. Did we leave people room to disagree?
Anyone collecting community feedback should think about what a respondent is actually being asked to do. Can they recall the information? Do they understand the terms? Does an available response fit their experience? These are practical questions about whether the answers will mean what the researcher later claims they mean.
Ask one question at a time
A question about whether an activity was useful and well organised combines two judgements. A participant may agree with one and disagree with the other. A single response cannot show that distinction. Separating the questions gives the resulting information a clearer interpretation and a more useful role in improvement.
Wording should also avoid suggesting the desired answer. Asking how helpful an excellent programme was already includes praise. A more neutral question allows a participant to describe a poor experience as easily as a positive one. The purpose of feedback should be learning, rather than collecting agreement with the organiser's description.
Give the respondent a reasonable task
A question about recent behaviour needs a clear period. Last week, the previous month, and usually are different prompts. The respondent should not be expected to remember details with a precision that the situation makes unrealistic. The researcher should consider what information is both useful and reasonably available to the person answering.
Imagine a hypothetical community workshop survey. Instead of asking whether the entire programme changed someone's life, it might ask which session they attended and which explanation remained unclear. These questions are narrower, but they may produce more actionable information. A grand outcome question immediately after an event can invite speculation rather than evidence.
Response options need attention too. Some people may not know, may not have used the service, or may prefer not to answer a sensitive question. A form should not force them into a misleading category simply to avoid empty cells. The treatment of these responses should remain visible in later analysis.
The order of questions can influence how respondents understand later ones. A pilot should therefore consider the whole sequence as well as individual wording. Questions that introduce an assumption early may shape answers even when the later items appear neutral on their own.
Test the wording before relying on it
Trying the questions with a small number of appropriate people can reveal unexpected interpretations. Ask how they understood the wording and what made an answer difficult. The aim is to improve the instrument before collecting a larger set of responses, when the same ambiguity could affect many records.
Translation requires similar care. A translated question should preserve the intended meaning and response task. It may need review by people familiar with both the language and the subject. Literal similarity alone does not establish that respondents are answering the same question across versions.
A survey is a conversation shaped in advance. Its structure can help people describe their experience or quietly distort that description. Before worrying about how impressive the final chart will look, examine the questions that will produce it. Better wording gives the analysis a more dependable beginning.
Further reading: AAPOR best practices for survey research.
0 Comments