Yesterday I watched a friend suffer the consequences of not knowing the best way to get Qualtrics to spit out data in the easiest form. Instead of having a single outcome variable (what he was measuring to gauge the effects of a manipulation) and a single between-groups variable (one that reflected the random assignment), he had many variables, some of which were coded differently, that he had to combine.

That’s a headache, in any statistical software. So here I will walk you through how to set up a simple experiment on Qualtrics in a way that will give you a single outcome variable and a single predictor variable (factor or between-groups variable). My blog post on using logic can help with more complex designs. At a later date I might cover how to make sure all similar variables are coded the same way in Qualtrics.

Our example research question:

What kind of media makes people more anxious, the news or reality TV? We will have people self-report anxiety for the outcome variable.
Participants will be randomly assigned to one of two conditions, News or Reality TV.
Participants will watch a 3 minute clip (news or reality TV) and then complete the anxiety scale.
In the real world, we’d also be collecting demographics and possibly some other information. These will be included in our first block.

How to set up random assignment in Qualtrics

Note: I am updating this blog post in September 2025. Qualtrics has changed its layout, so I have changed one image, using a different project. The new won’t reflect the example I originally used, but you will still be able to see where things are! (Qualtrics will probably change everything again at some point).

Step 1: Prepare your blocks on Survey view

For our example, we will need four blocks on Qualtrics:

1. Consent and demographics.
2. News stimulus block
3. Reality TV stimulus block.
4. Outcome variable block.

Set up your blocks in the initial screen you get after creating a new survey:

Your first step is to set up the blocks for your survey. Make sure to name them something that reflects their content.

Above: The four blocks (collapsed) on the Survey view tab. The Qualtrics layout has changed (you can see the new one below), but this image shows the blocks needed for this example.

You will need to add all the relevant content to each block.

Step 2: Use Survey Flow to set up randomization

Once you have your four blocks, select Survey Flow from the top tabs:

Above: Location of Survey Flow tab as of September 2025. It is now on the left.

The Survey flow screen

You will then be able to set up your random assignment on this screen:

Above: The Survey flow screen. By default, it will show all the blocks in the order you set them up on the Survey tab.

By default, blocks are presented to all participants in order. When they finish one, they will jump straight to the following unless you set up a different flow.

Adding a Randomizer block

Above: Add a Randomizer block below your Consent by selecting “Add Below”.

Above: You have many options with Add Below. Select Randomizer.

Using the Randomizer

Qualtrics will randomly assign participants to one or more of anything you put below the Randomizer. You will need to put all of the blocks with the conditions to which people will be assigned below it (+Add new Element Here).

Above: The Randomizer when you add it. Next, click and drag your conditions to where it says “Add new element here.

The randomizer with our example blocks:

Above: The Randomizer with blocks of all conditions added.

If you leave it like this, all participants will see both clips.

Above: The default Randomizer setting: Participants will see all blocks.

Select the quantity of blocks you wish each participant to view.

You will need to set the field after “Randomly present” to “1” to present only one clip.

Above: The Randomizer after it has been set to show one block to each participant, with “Evenly present elements” selected.

If you select “Evenly present elements” Qualtrics will ensure that the blocks are presented in equal quantities. This is minor violation of random assignment, but will prevent randomicity from happening in a disagreeable fashion (e.g., getting 75 people in group 1 and 25 in group 2). The larger the sample, the less you need to worry about this.

Adding a control condition

It is easy to set up a control group (in this case, it would be a condition where participants view nothing). In Survey view, create a control block, with an innocuous message (matched in experimental blocks). In the survey flow, add it below the randomizing block. Participants will then be randomly assigned to view news reality TV, or nothing.

Remaining survey flow

After participants view the block they are randomized to, they will be shown the block(s) that are beneath it in the flow. In our example, they will view the Anxiety block, which contains the outcome variable. If you had any other blocks (e.g., questionnaires for covariates), you could put them below the Anxiety block. They would be shown to all participants.

Above: The complete survey flow. As it is set up, participants will view (and complete) block one (consent and demographics), and then be randomly assigned to one of the two conditions.
They will then all be presented with the Anxiety block.

I have added an End of Survey block, but if you do not, participants will be shown a default end of survey page.

Customizing the End of Survey Block

If you select “Customize” on the red End of Survey block, you can then write your own message that will be presented to participants. I sometimes use custom messages to direct participants to an external survey where I might collect their contact for different reasons (e.g., a raffle or if they agree to complete a post-test).

If you put an “End of Survey” block below a randomizer or any other sub-block, participants will not be presented with any further blocks (higher up in tree, but below the End block). You would use this when participants complete different blocks depending on their randomly assigned condition (or on different criteria determined by logic statements).

Dealing with data

When you download the data, you will get separate variable for the people assigned to News and TV (e.g., the variable that indicates the stimulus has been presented, and anything else, such as a timing variable that is on the separate blocks).

You will get single variable for the Anxiety Scale (e.g., everyone will have A1, A2, A3, A4… completed in the same columns)

To get a variable that specifies what condition each participant was assigned to, you need to select “export viewing order data for randomized surveys” when you download the data (export function on the Data and Analyses tab). You can then either use this variable (it will state the block each participant was assigned to) or recode it into a numerical variable, depending on the needs of your analyses and statistical software.

Above: To get a single variable that specifies which condition each participant was assigned to, when you download the data, click on “More options” and select “Export viewing order data….”

Related posts

For questions concerning logic (conditional display of questions, etc) see my new post: Qualtrics tips: Using Logic.

To learn how to efficiently export data from randomization, see my blog post on Downloading Data from Qualtrics.

Feeling grateful for all the time I’ve just saved you? Please buy me a coffee via Ko-Fi by clicking on the link below!

FAQ

How do I randomize participants between two stimulus blocks in Qualtrics?

In Survey Flow, add a Randomizer, drag your two stimulus blocks under it, set “Randomly present” to 1, and (optionally) check Evenly present elements to balance group sizes.

What does “Evenly present elements” actually do?

It keeps group counts roughly equal by tracking allocations. It’s a slight constraint on pure randomness, but it prevents lopsided splits (e.g., 75/25) in small samples.

Where should the outcome block go?

Place the outcome block after the Randomizer in the flow so it’s shown to everyone, regardless of which stimulus block they saw.

Will I get one set of anxiety items in the data file?

Yes. Because all participants answer the same outcome block, you’ll get a single set of columns (e.g., A1, A2, A3…) for all respondents.

How do I know which condition each participant saw?

When exporting data, click More options and select Export viewing order data. Use that field (or recode it) as your between-groups variable.

Participants are seeing both clips—what did I do wrong?

You likely left the Randomizer at its default. Change “Randomly present” from “All” to 1 so each participant sees only one stimulus block.

Can I add a control condition?

Yes. Create a control block in Survey view and drag it under the Randomizer alongside the other conditions. Set “Randomly present” to 1.

How do I stop the flow after certain branches?

Add a red End of Survey element inside the branch you want to terminate. Anything below it in the flow won’t be shown.

Why do I sometimes get multiple variables for similar things?

If similar items live in different blocks, Qualtrics may create separate columns. Keep items you want combined in the same block shown to all participants.

Qualtrics moved things again—does this change the setup?

Layouts shift over time, but the workflow is the same: build blocks in Survey view, wire them in Survey Flow, configure the Randomizer, then export with viewing order data.