Ranking Question

  • Classic Design
  • New Design
  • ?
    Classic and New Design
    D

    Classic Design is our original survey design tool. New Design is our new survey design tool that is currently in beta.

    Which version am I using? »

You can use our Ranking question type whenever you have a scenario where respondents should be ranking answers. This question type enables respondents to assign only a unique weight for each answer choice-- for example, they can't select two answers as their #1 choice.

 

How are the Answers Chosen?

Respondents can reorder the list for a ranking question in a couple of different ways:

  • Respondents can drag and drop the answer choices to re-order them as they wish.
  • Respondents can type the number corresponding to the ranking they're assigning

lightbulb Tip! If respondents are reporting that the choices aren't reordering, often this is reported when answer choices look very similar.  When the answer choices reorder the change is immediate.  If the answer choices are about the same length, respondents can be confused if the change has already taken place or not.  Let your respondent know they can drag and drop the choices and this should reduce confusion.

 

Creating a Ranking Question

For the following example, we're using 5 items that would need to be ranked from best to worst. We highly recommend clarifying in your instructions that respondents can drag and drop the answer choices or type the ranking into the drop-down box so that it's clear to your respondents how to answer this question type.

  1. Click the Design icon design to start creating the survey.
  2. Click Add Question.
  3. Choose the Ranking question type from the drop-down menu.
  4. Enter the text of the question into the Question Text box. (Example: Please rank the following items from best to worst by dragging and dropping the choices)
  5. Enter the five items to be ranked into the Answer Choices box.
  6. Click Save.

You'll want to clarify whether respondents should rank from their top choice down (or vice versa) in the question text like the example above.

An additional aside is that question and answer piping is not enabled for this question type, and piped answers can't be piped to or from ranking questions.

 

Add N/A:

You can add a N/A option or equivalent. When respondents tick this checkbox in their response, it will gray out the answer choice so they can rank the remaining lines. Just tick the checkbox, and if applicable, edit the text displayed. Note that editing the text in the N/A box to be longer or shorter also changes the size of the drop-down menu for picking the numerical rank for respondents.

If you chose to include an N/A option on the Ranking question, any N/A responses will not factor into the average ranking.

 

lightbulb Tip! You can also edit the question size and label space. For complete instructions and details, check out our article here.

The Ranking question asks respondents to compare items to each other by placing them in order of preference. In the Analyze Results section, an average ranking is calculated for each answer choice, allowing you to quickly evaluate the most preferred answer choice.

Creating a Ranking Question

To add this question type:

  1. Drag and drop Ranking into your survey from the BUILDER section on the Design Survey page.
  2. In the question text field, write instructions that ask respondents to rank the answer choices in order of preference, 1 being the highest. It may also help to include what all the choices have in common. For instance, "Rank the following ice cream flavors in order of preference, 1 being your most preferred flavor."
  3. In the Ranking Choices fields, enter the items you want respondents to rank.
  4. Configure any additional options.
  5. Click Save.
TIP! Try to limit the number of ranking choices to about 5. Asking respondents to rank too many items in relation to each other can be overwhelming.

Add an N/A Option

Under the Edit tab, you can choose to add an N/A column, and customize the label. When you add an N/A option to a Ranking question, it will add an N/A checkbox at the end of each row. If a respondent selects an N/A checkbox, that row will be greyed out and will not be included in that respondent's ranking order.

It's important to make sure that even the lowest ranked options still apply to that respondent. If a respondent doesn't have the option to declare that a ranking item does not apply to them, they will be forced to include that choice in their rankings anyway, which might produce bad data.

Additional Options

Under the Options tab, you can further customize the question in the following ways:

Learn more: Editing Questions

 

Survey Behavior

Before you send out your survey, preview & test your survey design to see what your survey will look like to respondents.

There are two ways for respondents to answer the Ranking question:

  • Drag and drop answer choices in order of preference.
  • Use the dropdown menus to select a numerical rank for each answer choice.
TIP! The answers may reorder themselves too quickly for respondents to notice, so it may help to include instructions in the question text to let respondents know what to expect. Learn more: Ranking Question is Automatically Ordering Answers

Analyzing Results

See example results (English only) »

Average Ranking

Ranking questions calculate the average ranking for each answer choice so you can determine which answer choice was most preferred overall. The answer choice with the largest average ranking is the most preferred choice.

The average ranking is calculated as follows, where:

w = weight of ranked position
x = response count for answer choice

x1w1 + x2w2 + x3w3 ... xnwn

Total

Weights are applied in reverse. In other words, the respondent's most preferred choice (which they rank as #1) has the largest weight, and their least preferred choice (which they rank in the last position) has a weight of 1. You can't change the default weights.

For example, if a Ranking question has 5 answer choices, weights are assigned as follows:

  • The #1 choice has a weight of 5
  • The #2 choice has a weight of 4
  • The #3 choice has a weight of 3
  • The #4 choice has a weight of 2
  • The #5 choice has a weight of 1

We apply weights in this way to ensure that when the data is presented on a chart, it's clear which answer choice is most preferred.

If you chose to include an N/A option on the Ranking question, any N/A responses will not factor into the average ranking.

Charts

The Depth display option allows you to choose between the following options:

  • Weighted Average: Charts the average ranking for each answer choice
  • Distribution: Charts the absolute number or percentage of respondents that selected each answer choice

The chart types available depend on the question type, and the display options you configure. Learn more: Custom Charts & Data Tables

The Ranking question asks respondents to compare items to each other by placing them in order of preference. An average ranking is calculated for each answer choice, allowing you to quickly evaluate the most preferred answer choice.