Block Randomization
Block Randomization allows you to group pages of your survey into blocks, and then add logic to rotate, flip, or randomize the blocks. You can also add logic to the pages within a single block. This minimizes order bias in your survey.
To use Block Randomization:
- Go to the Design Survey section of the survey.
- In the left sidebar, click LOGIC.
- Click Block Randomization.
- Create blocks.
- Add logic to the blocks.
Creating Blocks
A block is a group of consecutive pages in your survey. You can create up to 300 blocks in your survey.
To create blocks:
- Under the LOGIC menu in the left sidebar, click Block Randomization.
- Click + New Block.
- Select the consecutive pages you want to include in the block. Some pages may be disabled if you already added logic to them that isn't compatible with Block Randomization.
- Click Add Block.
- Create more blocks, as needed.
- When you're finished creating all your blocks, click Done in the bottom-left corner of the menu.
Block names indicate the range of pages included in the block. For example, a block containing Page 4, Page 5, Page 6, and Page 7 of your survey would be named Block: P4-P7.
Editing Blocks
To edit a block:
- Under the LOGIC menu in the left sidebar, click Block Randomization.
- Click Edit Blocks.
- Click . next to the right of the block.
- Click Show block details or Break block apart.
Adding Logic to Blocks
You can add Block Logic to control the order in which the blocks, or pages within a block, are presented to the respondent. There are two ways to apply Block Logic:
|
Block Logic
|
Description
|
|---|---|
| Across multiple blocks | Apply randomization logic to the blocks themselves, or a subset of blocks. You can also set a limit on the number of blocks that will be shown to a respondent. |
| Within a single block | Apply randomization logic to the pages within a single block. |
You can apply both types of Block Logic to a survey.
To apply logic across multiple blocks:
- Under the LOGIC menu in the left sidebar, click Block Randomization.
- Under Add Block Logic, choose Across multiple blocks.
- Choose a Logic Type (see the table below for details).
- Under Blocks Affected, choose to apply the logic to All blocks in this survey or Selected blocks, where you can choose a subset of blocks to apply the logic to.
- Optional: Select Limit number of blocks to display and enter a limit.
- Click Apply.
|
Logic Type
|
Description
|
|---|---|
| No block randomization | No logic is applied across the blocks. They are shown to respondents in order. |
| Randomize blocks | Blocks are shown in a random order for each new respondent. |
| Flip blocks | On average, half of respondents will see the block order flipped. Example: 3, 2, 1 instead of 1, 2, 3. |
| Rotate blocks | Each respondent will see a random first block, with the following blocks arranged in successive order based on the first. Example: The first respondent might see 3, 4, 5, 1, 2; the second respondent might see 2, 3, 4, 5, 1; and the third respondent might see 5, 1, 2, 3, 4. |
If you apply logic across multiple blocks, you can also select the Limit number of blocks to display setting and enter the number of blocks you want each respondent to see. This setting can help reduce survey fatigue for any given respondent by showing them a smaller subset of blocks to answer, while still allowing you to collect responses for a larger survey.
The Limit number of blocks to display setting applies to all blocks in the survey, even if the Blocks Affected setting is set to only apply logic to Selected blocks.
For example, if you set a limit of 2 blocks, the respondent is shown the first 2 blocks they would have seen based on the Logic Type you applied. See the example below:
|
Total Blocks in Survey
|
Number of Blocks Limit
|
Logic Type
|
Blocks Affected
|
What Respondents See
|
|---|---|---|---|---|
| 4 (with no extra pages not included in a block) | 2 | Rotate blocks | All blocks in this survey |
Each respondent sees the first 2 blocks they would have seen based on their position in the rotation.
|
To apply logic within a single block:
- Under the LOGIC menu in the left sidebar, click Block Randomization.
- Under Add Block Logic, choose Within a single block.
- Click the block you want to apply logic to.
- Choose a Logic Type (see the table below for details).
- Under Pages Affected, choose to apply the logic to All pages in this block or Selected pages, where you can choose a subset of pages to apply the logic to.
- Click Apply.
|
Logic Type
|
Description
|
|---|---|
| No page randomization | No logic is applied to the pages within the block. They are shown to respondents in order. |
| Randomization | Pages within the block are shown in a random order for each new respondent. |
| Flip pages | On average, half of respondents will see the pages within the block flipped. Example: 3, 2, 1 instead of 1, 2, 3. |
| Rotate pages | Each respondent will see a random first page, with the following pages of the block arranged in successive order based on the first. Example: The first respondent might see 3, 4, 5, 1, 2 within the block; the second respondent might see 2, 3, 4, 5, 1; and the third respondent might see 5, 1, 2, 3, 4. |
Example
Block Randomization is commonly used for concept testing. Let's say you're testing three different advertisements. You'll need to create three blocks of pages, one for each ad.
| Page 1 | Page 2 | Page 3 | Page 4 | Page 5 | Page 6 | Page 7 | Page 8 |
|---|---|---|---|---|---|---|---|
| Block 1 | Block 2 | Block 3 | |||||
| F | F + F | F + F | F + F | F | |||
To show each respondent all three blocks in a random order, add logic to randomize all the blocks in the survey. All respondents will see Page 1 first and Page 8 last, and they'll see the blocks in a random order.
To show each respondent only one random block, first add logic to randomize all the blocks in the survey. Then, limit the number of blocks to display to 1. Respondents will see Page 1, then one random block selected from Block 1 OR Block 2 OR Block 3, then Page 8.
Using Block Randomization With Other Logic Options
|
Logic Option
|
How It Works with Block Randomization
|
|---|---|
| Skip Logic | You can use Question Skip Logic or Page Skip Logic to skip respondents out of a block and to the end of the survey, disqualification page, or a future page in your survey—as long as that page doesn't belong to a block, or have Page Randomization applied to it. |
| Advanced Branching | You can use Advanced Branching to skip respondents out of a block and to the next available block in the survey. |
| Page Randomization |
You can't apply Page Randomization to any pages contained within a block—that logic option will be disabled for pages within a block. Instead, you can randomize the pages within a single block in the Block Randomization feature to achieve the same result. |
| Question Randomization | You can apply Question Randomization to pages within blocks. |
Analyzing Results
When you send a survey with Block Randomization, you can see the order in which each respondent saw the pages in your survey by exporting the results in XLS format.
To export the page order per respondent:
- Go to the Analyze Results section of the survey.
- Click Save As in the upper-right corner of the page.
- Select Export file.
- Choose All responses data.
- Choose XLS.
- Under EXPORTS in the left sidebar, click the export to download it.
- Within the Excel folder, open PageOrder.xls.
Column A in the PageOrder.xls spreadsheet contains the Respondent ID, which you can match to the Respondent ID in the spreadsheet containing your survey results.
Column B contains the page number of the first page of the survey the respondent saw, Column C contains the page number of the second page of the survey the respondent saw, and so on.