Purpose of Research and Relevance of Questions:
As you design the questionnaire always ask yourself:
WHAT AM I RESEARCHING?
Simply put this means you should only include questions on the survey which are relevant to the research question at hand. For example, if you're researching the views of a community on a new shopping mall being constructed in their town a question on their favorite flavor of ice-cream may not be relevant to the point at hand (unless of course you plan to open an ice-cream shop in the mall then ask away).
Irrelevant questions can have several disadvantageous effects on your survey such as increasing the length of time it takes to complete a survey (which may annoy participants) and giving you the researcher extra work to do because you'd have to analyse and sort, "junk data," which are irrelevant to your research question.
Complexity and Length of Survey:
You should keep this acronym in mind when designing your survey:
KISS: KEEP IT SHORT AND SIMPLE
First and foremost, remember to keep it SHORT. Long questionnaires can seem tedious to participants and they may become annoyed and quit before finishing it or simply, "fill in the blanks," autonomously without any thought going into their answers in an attempt to expedite what they view as an excruciating process. The general rule of thumb I use is no more than 20 questions in total. Open ended questions also tend to require more time and effort than close ended ones so you may want to limit their application to say no more than 25% of the questions on the survey.
When conducting community research, quite often your participants will not be academics and may not have the same appreciation for your expansive vocabulary and literary finesse that your colleagues do. Furthermore, utilizing complicated wording can make your questionnaire more difficult to interpret, which may affect the integrity of your data. Additionally, you may annoy or appear aloof to your participants if your questionnaire is hard to read.
So how does one word a question? Well you should use SIMPLE language that the layperson will easily comprehend. At the risk of sounding elitist, assume that your participants only possess rudimentary literary skills and let your wording reflect this assumption.
For Example: Instead of wording a question like this:
Is one of the opinion that the current endeavor to erect a commercial centre within the community will have positive implications for those residing in the immediate vicinity of the proposed development?
Try wording it like this:
Do you believe that the mall will be beneficial to your community?
Notice the differences between the two wordings. The first question is long and drawn out, using more complicated language than the context demands, whereas the second question gets straight to the point. Additionally, the more personal nature of the second question (using terms like, "you," and, "your," instead of, "one.") makes you seem less like an impersonal robot out to experiment on people of the community and more like a friendly-neighborhood researcher who views the people of the community as equals and respects them as such. This simple change in wording can have a tremendous effect on the public's perception of you as a researcher.
Use the Appropriate Question Format:
The format of your question should be given consideration. Appropriate question format can make data collection simpler for you.
For Example: lets look at a question asking participants their age. You might be tempted to format it like this:
Age:___
This will require participants to state their exact age on the questionnaire. But do you really need their exact age? It might be more practical, particularly if dealing with a large sample population, to instead format the question as a categorical one confining ages to various ranges as stated below:
Age
Under 10 [] 10-19 [] 20-29 [] 30-39 [] 40-49 [] 50-59 [] 60 and Over []
Sorting the data in ranges reduces variability and makes it easier to chart the data (see the section on categorization below).
You should also consider whether or not to use open or close-ended question formats
Open-ended questions are great for qualitative research with data that are highly variable in nature and can give greater insight into a participant's viewpoints in comparison to close-ended questions but they also take longer to fill out and are more difficult to interpret and analyse. These are best used if you are trying to gain someone's views on a particular topic as it allows for them to freely express their views and allows for the wide range of answers you are certain to get with such a question.
For Example: lets say we want to gauge people's views on the shopping mall and we format the question as a close-ended, multiple choice one:
What are Your Views on the Shopping Mall?
Will bring jobs to the community []
Will be convenient for residents []
Will improve access to cheaper goods for the community []
Will provide local consumers with more choice []
Will serve as a social centre for the community []
Will destroy the environment []
Will bring negative cultural influences into the community []
Will encourage juvenile delinquency []
Will drive local stores out of business []
Notice that the above question has ten possible responses, which might be impractical. Additionally, you have forced your participants into a limited range of answers which they may feel compelled to adhere to. It may be more appropriate to format the question as an open ended one (leaving ample room for your participants to provide a comprehensive answer):
What are your views on the mall?
____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Close-ended questions are excellent for quantitative research with limited data variability and are easier to fill out and analyse but by confining your participants to a set range of responses, you may miss parts of their full view on a topic. Close-ended questions are best used when collecting quantitative data which can be confined to set categories such as age and sex.
Another aspect of question design is categorization. Categorization comes into play when the question at hand requires participants to select an answer from a range.
For Example: In this question about ages are categorized into 10 year categories
Age
Under 10 [] 10-19 [] 20-29 [] 30-39 [] 40-49 [] 50-59 [] 60 and Over []
For Example: Consider the following scenarios.
Using a sample size of 100 people between the ages of 18-40, it might be appropriate to break the respondents into single age categories (Fig. 1):
Fig. 1:-Bar Graph of Sample Data for Scenario One
We could easily analyse the chart at this scale. At a brief glance, we can see that the survey population appears to be more numerous below the age of 25.
But what about applying the same technique to a sample size of 400 people between the ages of 10-60 (Fig. 2)?
Fig. 2:-Bar Graph of Sample Data for Scenario Two
Obviously these data are much more difficult to understand and analyse. In cases like these you may want to consider aggregating the data into categories of about 10 years or so as shown below (Fig. 3):
Fig. 3:-Aggregated Bar Graph of Sample Data for Scenario 2
Notice how much less cluttered the bar graph appears and how much easier it is to visually analyse the data when compared to the previous
Avoid Redundant Questions:
Adding redundant questions to a questionnaire will needlessly increase its length, give you more data than you need to sift through and again may annoy your participants (people tend to get easily annoyed by questionnaires for some reason).
While redundancy has similar effects to irrelevancy as mentioned before, it is important to note the differences between both concepts. Irrelevant questions are those which are not related to the research question whereas redundant questions are related to the research question but are repetitive in nature.
For example, a redundant pair of questions may sound like this:
Do you believe that the shopping mall will be beneficial to the community?
Yes [] No[]
and
Do you believe that the shopping mall will harm the community?
Yes [] No []
Those two questions could be easily combined into a single one as shown below:
What impact do you believe the shopping mall will mostly have on the community?
Positive [] Negative [] No Impact []
Attempt to Avoid Potentially Offensive Questions:
It should go without saying that one should not offend the participants of a survey should they wish for their co-operation. It may be difficult however to avoid offending everyone owing to the fact that people may be offended by apparently mundane questions (I once had a participant who was offended by a question asking about their age). While there is no definitive list as to what topics should be avoided in order to not offend your participants GENERALLY speaking one should avoid questions on controversial issues such as Race, Religion and Politics and avoid contentious issues such as Abortion and Same-sex Marriage.
However your research might pertain to these topics at hand and if so you should keep in mind that your questionnaire might offend some people. It is also entirely possible that you might inadvertently offend people as well. In short just prepare for the possibility for a participant becoming offended and angry with you for asking a certain question.
Try to Word Questions Objectively:
When wording questions try to remain as objective as possible in order to give the impression of neutrality. If questions are worded in a certain way they may give the impression of bias which may affect people's willingness to answer truthfully.
For Example: Using the aforementioned shopping mall scenario, lets say we want to find out the resident's views on the mall. A wording like the one below may convey some sort of bias:
Do you believe that the mall will have a positive impact on the residents of the area?
Yes [] No []
Notice that the question mentions a positive impact which might give the impression that we as a researcher are in support of the mall. Additionally, confining them to two set boxes in a close ended question may not tell the whole story as some people might have mixed views on the project. A more appropriate and neutral question format is:
What are your views on the mall?
_________________________________________________________________________________
_________________________________________________________________________________
Notice that this question is open-ended and worded more neutrally than the first as it does not specify a particular viewpoint. Additionally it is now more appropriate for people to convey mixed viewpoints on the situation.
Testing:
Finally, after you have designed your survey, it is important that you pilot test it in a controlled setting before letting it loose in the field. Much like a commercial airliner needs to pass certification trials before being allowed to carry passengers, so to must your survey. Handout samples of your survey to testers, trying to include a wide sample base. Lecturers, friends, family can all be used as Guinea Pigs for your survey (don't give it to an actual Guinea Pig though as they tend to be jerks that simply chew up the paper and not give you a straight answer). Note their opinions and modify your survey based on their feedback. Were they able to interpret it easily? Did it answer the research question? Was it tedious for them? These are all questions you should asking your testers. Do not be discouraged if their feedback is negative, look upon it as an opportunity to improve your questionnaire and be grateful that potential problems were caught in testing before application in the field.
With all that said, I'd just like to conclude this post by saying that there is no one correct method to designing a questionnaire and it is usually heavily related to the research question at hand. Feel free to modify or even ignore some of the suggestions that I made here and experiment on your own as I am by no means an expert on the matter. It is my sincere hope that you found this post helpful and informative.
Thanks for Reading,
Barindra
No comments:
Post a Comment