Automatically generating large Qualtrics surveys in R

Qualtrics is a company used by the majority of researchers in the social sciences to generate online surveys for data collection purposes. While they offer a great deal of depth in survey customization and have a lot of powerful tools, one thing they’re really lacking in is scalability. If you need to generate (or delete, or modify) large numbers of similar question blocks, you’re faced with a tedious process of manually copying, clicking, and dragging.

I do a lot of research that broadly falls under ‘person perception’, which means I generally collect responses from a large number of participants in the form of video or text, and need to have each of them rated, usually by Mturk workers. This leaves me faced with manually creating hundreds of near-identical Qualtrics blocks each time I run a study.

Thankfully, there is an alternative to the tedium, and that’s to use Qualtrics’ advanced text importer (as seen here). It’s not widely used (at least, according to a call to Qualtrics support about the possibility of extending the features) but it can save a lot of time.

The way the tool works is that you generate a .txt file with the content you want in your survey, annotated with tags that tell Qualtrics how to format that content into questions. For example, when the text below is imported into Qualtrics, a new survey will be created with a multiple choice block that displays an image (hosted on Imgur) and allows participants a same/different response choice.
Mental rotation block

 

 

 

 

 

Since formatting such a file by hand would be just as tedious as creating the survey block-by-block, this is where some automation comes in handy. There are a few steps you’ll need to take to get this running:

  1. First, you need the set of stimuli you’re using organized in a single column of a spreadsheet. Save the spreadsheet as Stimuli.csv, and take note of the file location.
  2. Next, you’ll need to download and install R and Rstudio. R is an open source programming language that will be doing the heavy lifting here.
  3. Once installed, open Rstudio and you’re ready to go.

We’ll be using the cat function in R to generate our text files. Cat, short for concatenate and print, allows you to flexibly set up commands in R that will take your stimuli and write them to a text file in the order you specify. For example, imagine that your Stimuli.csv file contains a list of image links that you need embedded into Same/Different questions, each with their own block. If you replace “/Users/Yourname/Directory” with the directory you saved your file in, the code below will compile all of your links and save them in the correct format to Choiceblocks.txt, in the same directory. Windows users note that you’ll need to change the backslashes in your directory name to double backslashes to make this work.

Generating lots of blocks

Once you’ve run the code and have the Choiceblocks.txt file, simply create a new survey in Qualtrics, click “Advanced Options” in the upper-right section of the screen, and then click “Import Survey.” It may take some time if you’ve generate a lot of blocks, but you should soon see your survey appear. If the page hangs for a while, click the red cancel buttonĀ and then refresh – this often reveals a completed import.

This technique is highly modifiable. With some slight adjustments, for instance, you can generate blocks where the potential responses vary across questions, not just the stimuli. For example, you might want to generate a block of general knowledge questions, each with four possible answers. If you create a Stimuli.csv file with your questions in columnĀ one, and possible answers in columns two through five, then the following code will generate the desired survey blocks:

Generating knowledge

And if you wanted all your questions in a single block, just move the Block identifier into the first cat command (outside the for loop) as shown below. Here, I’ve added in ID notation for each question within the block, so it will be named according to its row in your spreadsheet.

generatingknowledgeoneblock

There are a few things this method sadly can’t do – it can’t add randomization, alter survey flow, or add advanced questions (like timers). But I still find it quite a time saver. The Qualtrics page has tags and formatting for a variety of different questions that you can substitute into the code presented here, and the help files in RStudio should point you in the right direction if you find yourself needing to change things in a more advanced way.

All of the code in this post can be found at my Github account here.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>