4 Tips for Improving Survey Response Rates
More and more organizations are using evaluations and surveys to find out what their employees, students, faculty, and alumni think and want. Evaluations and surveys can provide them with accurate, measurable data that they can’t get any other way. Not only are surveys an excellent way to keep abreast of needs, problems and the current state of affairs, but the results are essential when making plans for the future.
Low response rates are a continuing problem for surveys. Some people simply refuse to participate in surveys, while others, for a wide range of reasons, cannot participate. Still, a well-designed survey, coupled with incentives and techniques to elicit response, can help guarantee a healthy response rate.
Should you be concerned about low response rates? Absolutely. Low response rates are usually not random and may bias survey results. Conclusions drawn from unrepresentative data may erroneous and cause serious problems when used in the planning process.
Customers frequently ask us what kind of response rate they can expect for their survey. Given the vast number of variables with the potential to affect response rates, it can be difficult to estimate. We can, however, provide a set of guidelines or tools we believe will enable you to maximize response rates.
Why Do People Participate in Surveys?
Briefly, researchers divide approaches to survey participation into two distinct groups: the reasoned action approach and the psychological approach.
Reasoned Action Approach
The reasoned action approach relies on the theory of social exchange to explain why someone fills out a survey. Basically, the participant is more likely to participate if the rewards of participation outweigh the costs. Examples of this approach would be strategies such as including monetary incentives for participation or reducing survey length so that the time cost is perceived to be low and non-intrusive.
The use of incentives is a heavily researched area in response rate literature. Although several meta-analyses came to different conclusions, published reviews paint a very clear picture with respect to two issues: first, incentives are effective in increasing the response rates for mail surveys; and second, promised incentives are not as effective as enclosed incentives. Numerous studies demonstrate that postpaid incentives have no impact on response rates. While these two findings are almost universal, the effects of incentive size are less clear.
Unfortunately, some of the evidence on the role of incentive value is conflicting. The bulk of the data suggest that there is some merit in increasing the value of the incentive; however, this issue is far from settled.
Incentives use social exchange theory by causing participants to feel obligated to respond. However, some researchers suggest a model of diminishing returns. A small token may cause potential respondents to feel obligated to respond. But an incentive cannot be so large as to suggest payment for services rendered.
The psychological approach relies on heuristic factors such as reciprocity, helping, compliance, and selectivity as a way to reduce non-response. Reciprocity is the behavioral norm that people should treat others as they have been treated. This is the approach used when a dollar bill is included in a survey. If a benefit has been provided to the participant, some sort of reciprocal benefit should be in turn provided. The helping approach uses a strategy of specifically requesting help, as in a cover letter, as a way to compel participation. The compliance approach focuses on increasing participation by using an authoritative organization like the government as the sponsor rather than an anonymous source. Finally, using selectivity, or emphasizing the participant as being part of an exclusive group has also been used to increase participation.
Salience is simply how important or relevant a survey topic is to the survey recipient. Unfortunately, survey salience may be out of the researcher’s hands and dictated by project needs; nevertheless, it is useful to understand the effects of salience when designing both surveys and cover messages. If viewed as important to the respondent response has been shown to rise between 12 – 14 percent.
Salience is an important factor in respondent behavior; unfortunately, it is also one aspect of a survey that is difficult to alter. At a minimum, salience should be emphasized in the messages accompanying a survey.
Requests for Help
If people tend to follow a norm of social responsibility, they may be more likely to comply with a survey request couched in terms of asking for help. Some evidence indicates that this is indeed the case. A recent study found an 18 percentage point increase by including the phrase “it would really help us out” in their communications.
Putting It into Action: Four Tactics to Increase Participation
Now that we understand why people are willing to take surveys, let’s explore how to get as many of your target audience to participate as possible.
1: Choose an Appropriate Survey Length for Your Audience
Many researchers view a survey that is too long as an inhibitor to response, because longer surveys take more time to complete and thus increase the costs to the respondent. Respondents may fill out only part of the survey, or they may reject very long surveys outright.
In general, the experimental research on mail surveys indicates that shorter surveys do elicit higher response rates, but many of the differences are small. How long is too long? Generally, surveys that take longer than 15 minutes to complete are considered too long in most instances.
2: Make Sure the Survey Is Easy to Take and Return
One of the easiest ways to increase response rates in both electronic, phone and paper formats is thoughtful design. A well-designed, attractive survey that is easy to complete will improve response rates as well as data accuracy. In general, by making surveys easy to complete, you increase the likelihood that respondents will participate increases.
Many institutions that conduct frequent or extensive surveys get caught in the trap of making survey administration easy for the organization and overlooking the needs of the respondent.
||…instead of this…
- Choose the most effective and reliable delivery method to reach participants (see Choose the Right Survey Format later in this article for recommendations).
- Include at least a personalized greeting on the survey form.
- Reduce the number of questions asked.
- Vary the delivery formats to capture interest.
- Keep telephone survey questions short and to the point.
- Sending only printed and mailed questionnaires by bulk rate to save money.
- Neglecting to personalize mailings.
- Squeezing more questions onto a page to save printing costs.
- Using identical formatting in every mailing to save mailer costs.
- Designing long-winded questions for telephone surveys.
To increase participation, avoid falling into the traps listed above. It may make it easier for you to process the responses, but making it harder for your participants defeats the purpose of your survey—to gather enough appropriate information to make effective decisions. Design every survey to make the respondents’ job of completing it easier.
3: Contact Participants Multiple Times
One of the most successful techniques to increase response rates is the use of multiple contacts with members of the sample. This technique is now considered standard methodology for any survey. Studies suggest that to get full benefit from multiple contacts, do the following:
- Use a pre-survey notification message.
- Follow the pre-notification with a copy of the survey including a cover message.
- Contact non-respondents using combination of messages and surveys.
Studies using samples of the general population found that pre-notification letters increased response rates by 4–29 percentage points. Reminder postcards are also effective and have been shown to increase response rates from 3–8 percent.
Research also shows that increasing the number of surveys sent to respondents increases response rates. Researchers found that mailing a fourth survey increased the response rate by more than 30 percent. One study showed that two survey mailings versus one survey mailing increased response rates 12 to 20 percentage points.
An example of multiple contacts with a web survey was administered at a major university; regarding student housing. After the first e-mail notification, the response rate was leveling off at around 44 percent. After an e-mail reminder was sent to non-respondents, the response rate increased to 67 percent, and a final reminder to non-respondents notifying them of the deadline for the survey resulted in a final response rate of almost 72 percent, substantially higher than the rate after the first e-mail notification.
While multiple contacts can increase costs, re-contacting respondents is one of the best ways to ensure a good response rate. This is one reason that Web surveys are growing in popularity: three or four contacts with respondents can be costless, while three or four paper mailings can be quite expensive, especially if postage is required.
4: Choose the Right Survey Format
The right survey format can make or break a survey’s response rate. Knowing your audience and their capabilities and preferences has a strong effect on response rates. The following sections discuss several common survey formats.
But First: A Note about Using Multiple Survey Methods
Survey professionals have long recognized that some respondents prefer being surveyed by one mode, while others prefer another. A recent survey reported that among respondents to a telephone survey, 39.4% indicated that they would have preferred being surveyed by telephone, 22.7 percent preferred face-to-face interviews and 28.1 percent preferred mail.
Many organizations are not only offering multiple-modes at the start of a survey, but are choosing an alternative mode to surveys for non-responders. Both approaches make sense when you consider that 70 percent of the US population has access to the Internet (Pew Research Center, 2014), more than 96 percent of homes have a telephone, and many respondents are difficult to locate by mail due to moving or socio-economic status.
In a study of female veterinarians the surveyors employed a mixed-mode survey design in targeting women graduating from all US veterinary colleges during an 11-year period 1970-80. The questionnaire elicited information on a variety of health and occupational factors and required 35 minutes on average to complete.
In the first stage a mailed, self-administered questionnaires was employed, yielding a response rate of 82.9 percent. In the second stage, a telephone interview of all mail non-respondents was attempted, yielding a response rate here of only 30.1 percent, but increasing the overall response rate among those contacted to 90.2 percent. While this is an older study, the broader point remains: you can increase response rates by providing more response options.
Online surveys can offer several advantages, such as shorter administration time, lower costs, and fewer data-entry errors.
- Fast. Many people who will respond to an email invitation to take a web survey will do so the first day, and most will do so within a few days.
- Logos and other specialized graphics, fonts and multimedia options are available.
- You can adjust which questions participants see based on their responses to certain questions.
- Reduced data entry costs.
- Evidence suggests you will get more accurate answers to sensitive questions.
- Many people dislike unsolicited email even more than unsolicited regular mail. Be sure to adhere to anti-spam regulations.
- As with email, online surveys may not reflect the population as a whole.
- People can easily abandon in the middle of a questionnaire.
- As with mail surveys, online surveys may have serious response rate problems in populations of lower educational and general or computer literacy levels.
The most important factor to keep in mind with respect to online surveys is that an online survey will be successful only if the population has easy access to and is comfortable with using the internet, and if the survey administrator has a set of accurate email addresses. Further, ensure that your survey supports mobile devices; more and more respondents rely on tablets and smartphones for their online access.
Increasing Online Survey Response Rates
- Make sure your survey delivery supports mobile devices.
- Keep it simple and user friendly. Make the first page simple in order to get the survey started easily.
- Make participation voluntary, anonymous and confidential.
- Be relevant. Ensure that the survey topics and questions are of interest to the participants.
- Develop a value proposition for participants.
- Communicate in advance. Alert participants that a survey is coming.
- Communicate aggressively to track responses and send reminders to stimulate participation.
- Use graphics sparingly and strategically. Surveys with extensive graphical treatments have lower response rates than plain surveys.
- When possible, publish your results online to participants.
Another important aspect to consider when administering online surveys is access. Surveys that require the respondent to key an identification number and password into the survey have lower response rates than surveys that automatically log in the respondent.
Paper surveys are a tried and true format, and can reach a larger sample than the electronic format.
- Mail surveys are among the least expensive, compared to phone surveys.
- The questionnaire can include diagrams, graphics, etc.
- Mail surveys allow the respondent to answer at their leisure and are not considered as intrusive as other kinds of interviews.
- High accuracy available when forms are scanned instead of manually entered.
- Response time is usually longer than other methods.
- Response rates are often low or are unpredictable and may result in biased results.
Critical Activities and Inserts for Paper Surveys
For the best response rates, consider the following activities while planning a paper survey:
- Pre-Notification and Commitment Cards
Pre-notification usually involves a postcard or letter explaining the impending survey delivery and requests participation. Commitment cards ask the potential respondent to return a postcard indicating they will participate in the survey. Results of a recent study indicate that pre-notification cards provided a response rate of 32.3% while the commitment card resulted in a response rate of 20.4%.
- Reminder Postcards
Reminder postcards are sent to survey recipients roughly one week after the initial survey mailing. This card serves as a thank-you for those who responded and a reminder for those who haven’t. Our experience with these cards suggests they are very helpful. Findings range from a 10% increase to a response nearly equaling that of the initial mailing.
- Return Postage
Rather than requiring the respondent to provide their own postage, Return Postage has been shown to increase response rates significantly. In addition, using stamps on return envelopes has been shown to be better than business reply mail. However, depending on volume, the incremental return gained by stamping the return envelopes may be more than outweighed by the cost.
- Cover Letter
A strong Cover Letter is also a key motivator for respondents. It should explain the importance of the survey and the potential benefits to the respondent for the most direct and influential approach.
- Re-Mail Surveys
Re-mailing surveys entails distributing a second survey to either the entire respondent base or to non-responders only. Two studies provide considerable evidence of the benefits of using this approach. In one, a re-mail improved response rates from 39.5% to 50.0%, and the other showed an increase from 38.8% to 52.9%.
Types of Mail
Survey professionals have found that what type of mail you use to distribute and receive the survey has an impact on response rates. Research has examined both different mail types and postage types with results indicating they do have an impact on response rates (although there are frequently other variables involved).
- Certified Mail™ seems to be helpful for a lengthy survey. In a recent study of the impact of certified mail, varying questionnaire lengths were tested in combination with a certified mail process. The results indicated certified mail had little effect with short surveys. However, the response rate for certified mail was nearly double that of regular mail for long surveys.
- Express Mail® has also been found to positively affect response rates, especially for executives and business respondents. A recent study demonstrated that when express mail was used for both the mailing and return of the survey a response rate of 52% was achieved, compared to a response rate of 26% for the group receiving and returning their survey via regular US Mail. Express mail may also convey the importance of the survey to the potential respondent.
A survey must have a good response rate in order to produce accurate, useful results. We have outlined several ways you can do to improve your response rate. The response rates for a survey is critical and you shouldn’t t just leave it to chance. If you care about the data, you need to care about getting the best response rate possible.
How Can Scantron Help?
Scantron is a longtime leader in evaluation and survey solutions and services. We’ve served the education market from kindergarten to professional career development for more than 40 years. Our course evaluation solution, Class Climate, provides the best in course and instructor evaluations, and delivers broad-based survey functionality, as well.
Class Climate provides world-class support for course evaluations. It’s built-in structure mimics department and course structure for most institutions—and can be customized to fit as necessary. Libraries make it easy to design and re-use questions year-over-year for longitudinal tracking. Core questionnaires that allow for instructor customization ensure that the needs of both administration and individual faculty or departments are met. Automated reminders and connectors to many common learning management systems make it easy to increase and track responses. Quality dashboards provide immediate, at-a-glance quality tracking and comparison.
Class Climate can deliver surveys via mobile devices, online, or on paper and collect all responses, regardless of delivery, into a single database for analysis. Connections to common learning management systems ensure survey invitations are readily available to respondents.
Class Climate provides the greatest possible degree of automation for comprehensive course evaluation.
Need more? Scantron’s Professional Services team is here to help!
- Professional development enables you and your staff to navigate the software and interpret the results more effectively. Scantron Corporation has a fully staffed corporate development department through which we offer an array of product training and staff development classes and seminars for Class Climate. The team is dedicated to building capacity for change within organizations and institutions through a series of training and consulting activities.
- Get your first evaluation or survey project up and running faster with Scantron Implementation Services. We provide dedicated experts with extensive experience in course evaluation management to help you implement your system and begin processing your surveys quickly and efficiently for a more rapid return on investment. The Scantron implementation team works with you to create a manageable schedule customized to ensure your success in the evaluation process. Depending on your needs, consultations can be performed onsite or via the phone or web.
- Scantron offers a wide array of Professional Services to customize, enhance, and support your Class Climate capabilities. Save yourself time by letting Scantron do the work for you. We listen to your needs and then work with you to create the applications you require.
- Scantron’s world-class print services are available to print your custom forms and fulfill your large-volume needs with mailing and data processing services.
Whatever the evaluation or survey assistance you need, Scantron has the products, tools, services, and expertise to help you ensure that you have the right program for your students. Our web-based software, combined with our length and breadth of experience, help you get the most out of your evaluations and results. Contact us today and see how Scantron can meet you where you are and help you get to where you want to be.
Back to Class Climate
© 2018 Scantron Corporation. All rights reserved.