Questionnaires

Survey questions can be divided into two broad types: structured and unstructured. From an instrument design point of view, the structured questions pose the greater difficulties . From a content perspective, it may actually be more difficult to write good unstructured questions. Here, I'll discuss the variety of structured questions you can consider for your survey. 

Dichotomous Questions

When a question has two possible responses, we consider it dichotomous. Surveys often use dichotomous questions that ask for a Yes/No, True/False or Agree/Disagree response. There are a variety of ways to lay these questions out on a questionnaire:

Questions Based on Level Of Measurement

We can also classify questions in terms of their level of measurement. For instance, we might measure occupation using a nominal question. Here, the number next to each response has no meaning except as a placeholder for that response. The choice of a "2" for a lawyer and a "1" for a truck driver is arbitrary -- from the numbering system used we can't infer that a lawyer is "twice" something that a truck driver is.

We might ask respondents to rank order their preferences for presidential candidates using an ordinal question:

We want the respondent to put a 1, 2, 3 or 4 next to the candidate, where 1 is the respondent's first choice. Note that this could get confusing. We might want to state the prompt more explicitly so the respondent knows we want a number from one to 4 (the respondent might check their favorite candidate, or assign higher numbers to candidates they prefer more instead of understanding that we want rank ordering).

We can also construct survey questions that attempt to measure on an interval level. One of the most common of these types is the traditional 1-to-5 rating (or 1-to-7, or 1-to-9, etc.). This is sometimes referred to as a Likert response scale . Here, we see how we might ask an opinion question on a 1-to-5 bipolar scale (it's called bipolar because there is a neutral point and the two ends of the scale are at opposite positions of the opinion):

Another interval question uses an approach called the semantic differential. Here, an object is assessed by the respondent on a set of bipolar adjective pairs (using 5-point rating scale):

Finally, we can also get at interval measures by using what is called a cumulative or Guttman scale (see Guttman Scaling). Here, the respondent checks each item with which they agree. The items themselves are constructed so that they are cumulative -- if you agree to one, you probably agree to all of the ones above it in the list:

Filter or Contingency Questions

Sometimes you have to ask the respondent one question in order to determine if they are qualified or experienced enough to answer a subsequent one. This requires using a filter or contingency question. For instance, you may want to ask one question if the respondent has ever smoked marijuana and a different question if they have not. in this case, you would have to construct a filter question to determine whether they've ever smoked marijuana:

Filter questions can get very complex. Sometimes, you have to have multiple filter questions in order to direct your respondents to the correct subsequent questions. There are a few conventions you should keep in mind when using filters:

  • try to avoid having more than three levels (two jumps) for any question

Too many jumps will confuse the respondent and may discourage them from continuing with the survey.

  • if only two levels, use graphic to jump (e.g., arrow and box)

The example above shows how you can make effective use of an arrow and box to help direct the respondent to the correct subsequent question.

  • if possible, jump to a new page

If you can't fit the response to a filter on a single page, it's probably best to be able to say something like "If YES, please turn to page 4" rather that "If YES, please go to Question 38" because the respondent will generally have an easier time finding a page than a specific question.

 

 

Adapted from Trochim, 2002
http://imem01.tripod.com/rmguide/survey03.htm
imem01.tripod.com
crawl=CC-MAIN-2018-05
7366
application/http; msgtype=response
sha1:OLLT64L6NPXPKOZFC3GDK3MBO5ACNBUU
<urn:uuid:87828146-c3ad-4aa1-8932-febe26bd04bb>
2018-01-22T16:28:03Z
209.202.252.95
text/html
sha1:NIYI77BCJ2QI7UYKUNC3NAFGKWVS3NA6
<urn:uuid:c11da780-5601-44c1-acd5-7d40805424cb>
http://imem01.tripod.com/rmguide/survey03.htm
response
<urn:uuid:a702c636-ff61-416e-963e-81167d36934b>
robots: classic
hostname: ip-10-111-199-146.ec2.internal
software: Nutch 1.6 (CC)
isPartOf: CC-MAIN-2018-05
operator: Common Crawl Admin
description: Wide crawl of the web for January 2018
publisher: Common Crawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf