How multi-response questions in online surveys can be answered in record time

Here at MRQual we’re always looking for ways to improve the quality of the results from the online surveys that we run.  Recently we came across a trick that a number of respondents are using to answer multi-response questions in an unfeasibly short amount of time.

We’ve got several examples, but a typical one would be someone who gave 18 answers to a multi-response question in 2 seconds. Now it’s possible that they genuinely should have given 18 answers, but not in that time period.

The way they did it, we think, is by using a browser extension. This would allow them to submit multiple answers in one go, rather than selecting each individually. Why would they want to select 18 answers? Because they might think that if they do not they are more likely to be screened out.

What can be done about this? The easiest solution is to ask respondents to give only a certain number of answers to the question, and to use a plausibility check to show them a message if they fail to comply.  But if you really want them to be able to give as many answers as they wish, then you can ask the data cleaning team to check the number of answers against the time on that question.

Either way, this is a good example of how a little bit of planning ahead of fieldwork can yield cleaner survey results.

 

 
https://www.mrqual.com/2017/01/18/multi-response-questions-online-surveys-can-answered-record-time/
www.mrqual.com
crawl=CC-MAIN-2019-04
49670
application/http; msgtype=response
sha1:U2ABWWJF7PXXM3VMIYG3F3AQAHRDAV7U
<urn:uuid:9c23ba0b-2057-4ee6-90b1-4cfdea4240c4>
2019-01-21T06:05:04Z
52.18.45.104
text/html
sha1:FNJGDEK34U6OAHP3YVUR3PVJGOZHHUSD
<urn:uuid:92ec3753-f785-426d-9e54-6aa682485d77>
https://www.mrqual.com/2017/01/18/multi-response-questions-online-surveys-can-answered-record-time/
response
<urn:uuid:ea43f9b1-f2fd-4ab9-9fab-974fca26e192>
isPartOf: CC-MAIN-2019-04
publisher: Common Crawl
description: Wide crawl of the web for January 2019
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-182-67-179.ec2.internal
software: Apache Nutch 1.15 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 0.11-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/