Improving Survey Response Rates:
Four Tactics to Increase Participation
More and more organizations are using evaluations and surveys to find out what their employees, students, faculty, and alumni think and want. Evaluations and surveys can provide them with accurate, measurable data that they can’t get any other way. Not only are surveys an excellent way to keep abreast of needs, problems and the current state of affairs, but the results are essential when making plans for the future.
Low response rates are a continuing problem for surveys. Some people simply refuse to participate in surveys, while others, for a wide range of reasons, cannot participate. Still, a well-designed survey, coupled with incentives and techniques to elicit response, can help guarantee a healthy response rate.
Should you be concerned about low response rates? Absolutely. Low response rates are usually not random and may bias survey results. Conclusions drawn from unrepresentative data may erroneous and cause serious problems when used in the planning process.
Customers frequently ask us what kind of response rate they can expect for their survey. Given the vast number of variables with the potential to affect response rates, it can be difficult to estimate. We can, however, provide a set of guidelines or tools we believe will enable you to maximize response rates.
Why Do People Participate in Surveys?
Briefly, researchers divide approaches to survey participation into two distinct groups: the reasoned action approach and the psychological approach.
Reasoned Action Approach
The reasoned action approach relies on the theory of social exchange to explain why someone fills out a survey. Basically, the participant is more likely to participate if the rewards of participation outweigh the costs. Examples of this approach would be strategies such as including monetary incentives for participation or reducing survey length so that the time cost is perceived to be low and non-intrusive.
The use of incentives is a heavily researched area in response rate literature. Although several meta-analyses came to different conclusions, published reviews paint a very clear picture with respect to two issues: first, incentives are effective in increasing the response rates for mail surveys; and second, promised incentives are not as effective as enclosed incentives. Numerous studies demonstrate that postpaid incentives have no impact on response rates. While these two findings are almost universal, the effects of incentive size are less clear.
Unfortunately, some of the evidence on the role of incentive value is conflicting. The bulk of the data suggest that there is some merit in increasing the value of the incentive; however, this issue is far from settled.
Incentives use social exchange theory by causing participants to feel obligated to respond. However, some researchers suggest a model of diminishing returns. A small token may cause potential respondents to feel obligated to respond. But an incentive cannot be so large as to suggest payment for services rendered.
The psychological approach relies on heuristic factors such as reciprocity, helping, compliance, and selectivity as a way to reduce non-response. Reciprocity is the behavioral norm that people should treat others as they have been treated. This is the approach used when a dollar bill is included in a survey. If a benefit has been provided to the participant, some sort of reciprocal benefit should be in turn provided. The helping approach uses a strategy of specifically requesting help, as in a cover letter, as a way to compel participation. The compliance approach focuses on increasing participation by using an authoritative organization like the government as the sponsor rather than an anonymous source. Finally, using selectivity, or emphasizing the participant as being part of an exclusive group has also been used to increase participation.
Salience is simply how important or relevant a survey topic is to the survey recipient. Unfortunately, survey salience may be out of the researcher’s hands and dictated by project needs; nevertheless, it is useful to understand the effects of salience when designing both surveys and cover messages. If viewed as important to the respondent response has been shown to rise between 12 – 14 percent.
Salience is an important factor in respondent behavior; unfortunately, it is also one aspect of a survey that is difficult to alter. At a minimum, salience should be emphasized in the messages accompanying a survey.
Requests for Help
If people tend to follow a norm of social responsibility, they may be more likely to comply with a survey request couched in terms of asking for help. Some evidence indicates that this is indeed the case. A recent study found an 18 percentage point increase by including the phrase “it would really help us out” in their communications.
Putting It into Action: Four Tactics to Increase Participation
Now that we understand why people are willing to take surveys, let’s explore how to get as many of your target audience to participate as possible.
1: Choose an Appropriate Survey Length for Your Audience
Many researchers view a survey that is too long as an inhibitor to response, because longer surveys take more time to complete and thus increase the costs to the respondent. Respondents may fill out only part of the survey, or they may reject very long surveys outright.
In general, the experimental research on mail surveys indicates that shorter surveys do elicit higher response rates, but many of the differences are small. How long is too long? Generally, surveys that take longer than 15 minutes to complete are considered too long in most instances.
2: Make Sure the Survey Is Easy to Take and Return
One of the easiest ways to increase response rates in both electronic, phone and paper formats is thoughtful design. A well-designed, attractive survey that is easy to complete will improve response rates as well as data accuracy. In general, by making surveys easy to complete, you increase the likelihood that respondents will participate increases.
Many institutions that conduct frequent or extensive surveys get caught in the trap of making survey administration easy for the organization and overlooking the needs of the respondent.
||…instead of this…
- Choose the most effective and reliable delivery method to reach participants (see Choose the Right Survey Format later in this article for recommendations).
- Include at least a personalized greeting on the survey form.
- Reduce the number of questions asked.
- Vary the delivery formats to capture interest.
- Keep telephone survey questions short and to the point.
- Sending only printed and mailed questionnaires by bulk rate to save money.
- Neglecting to personalize mailings.
- Squeezing more questions onto a page to save printing costs.
- Using identical formatting in every mailing to save mailer costs.
- Designing long-winded questions for telephone surveys.
To increase participation, avoid falling into the traps listed above. It may make it easier for you to process the responses, but making it harder for your participants defeats the purpose of your survey—to gather enough appropriate information to make effective decisions. Design every survey to make the respondents’ job of completing it easier.
3: Contact Participants Multiple Times
One of the most successful techniques to increase response rates is the use of multiple contacts with members of the sample. This technique is now considered standard methodology for any survey.
Studies suggest that to get full benefit from multiple contacts, do the following:
- Use a pre-survey notification message.
- Follow the pre-notification with a copy of the survey including a cover message.
- Contact non-respondents using combination of messages and surveys.
Studies using samples of the general population found that pre-notification letters increased response rates by 4–29 percentage points. Reminder postcards are also effective and have been shown to increase response rates from 3–8 percent.
Research also shows that increasing the number of surveys sent to respondents increases response rates. Researchers found that mailing a fourth survey increased the response rate by more than 30 percent. One study showed that two survey mailings versus one survey mailing increased response rates 12 to 20 percentage points.
A recent example of multiple contacts with a web survey was administered at a major university; regarding student housing. After the first e-mail notification, the response rate was leveling off at around 44 percent. After an e-mail reminder was sent to non-respondents, the response rate increased to 67 percent, and a final reminder to non-respondents notifying them of the deadline for the survey resulted in a final response rate of almost 72 percent, substantially higher than the rate after the first e-mail notification.
While multiple contacts can increase costs, re-contacting respondents is one of the best ways to ensure a good response rate. This is one reason that Web surveys are growing in popularity: three or four contacts with respondents can be costless, while three or four paper mailings can be quite expensive, especially if postage is required.
4: Choose the Right Survey Format
The right survey format can make or break a survey’s response rate. Knowing your audience and their capabilities and preferences has a strong effect on response rates. The following sections discuss several common survey formats.
But First: A Note about Using Multiple Survey Methods
Survey professionals have long recognized that some respondents prefer being surveyed by one mode, while others prefer another. A recent survey reported that among respondents to a telephone survey, 39.4% indicated that they would have preferred being surveyed by telephone, 22.7% preferred face-to-face interviews and 28.1% preferred mail.
Many organizations are not only offering multiple-modes at the start of a survey, but are choosing an alternative mode to surveys for non-responders. Both approaches make sense when you consider that slightly less than half of the US population has access to the Internet (US Department of Commerce, 2000), over 96% of homes have a telephone, and many respondents are difficult to locate by mail due to moving or socio-economic status.
In a recent study of female veterinarians the surveyors employed a mixed-mode survey design in targeting women graduating from all US veterinary colleges during an 11-year period 1970-80. The questionnaire elicited information on a variety of health and occupational factors and required 35 minutes on average to complete.
In the first stage a mailed, self-administered questionnaires was employed, yielding a response rate of 82.9%. In the second stage, a telephone interview of all mail non-respondents was attempted, yielding a response rate here of only 30.1%, but increasing the overall response rate among those contacted to 90.2%.
Web surveys can offer several advantages, such as shorter administration time, lower costs, and fewer data-entry errors. Yet some researchers question the validity of data obtained from Web surveys; a common finding, for example, is that responses from Web surveys tend to show more positive outcomes for computer- and technology-related items.
- Fast. Many people who will respond to an email invitation to take a web survey will do so the first day, and most will do so within a few days.
- Logos and other specialized graphics, fonts and multimedia options are available.
- You can adjust which questions participants see based on their responses to certain questions.
- Reduced data entry costs.
- Evidence suggests you will get more accurate answers to sensitive questions.
- Many people dislike unsolicited email even more than unsolicited regular mail. Be sure to adhere to anti-spam regulations.
- As with email, web surveys do not reflect the population as a whole.
- People can easily abandon in the middle of a questionnaire.
- As with mail surveys, electronic interviews may have serious response rate problems in populations of lower educational and general or computer literacy levels.
The most important factor to keep in mind with respect to Web surveys is that a Web survey will be successful only if the population has easy access to the Internet and is comfortable with using the Web, and if the survey administrator has an accurate e-mail address datafile.
Increasing Online Survey Response Rates
- Keep it simple and user friendly. Make the first page simple in order to get the survey started easily.
- Make participation voluntary, anonymous and confidential.
- Be relevant. Ensure that the survey topics and questions are of interest to the participants.
- Develop a value proposition for participants.
- Communicate in advance. Alert participants that a survey is coming.
- Communicate aggressively to track responses and send reminders to stimulate participation.
- Use graphics sparingly and strategically. Surveys with extensive graphical treatments have lower response rates than plain surveys.
- When possible, publish your results online to participants.
Another important aspect to consider when administering online surveys is access. Surveys that require the respondent to key an identification number and password into the survey have lower response rates than surveys that automatically log in the respondent.
Paper surveys are a tried and true format, and can reach a larger sample than the electronic format.
- Mail surveys are among the least expensive, compared to phone surveys.
- The questionnaire can include diagrams, graphics, etc.
- Mail surveys allow the respondent to answer at their leisure and are not considered as intrusive as other kinds of interviews.
- High accuracy available when forms are scanned instead of manually entered.
- Response time is usually longer than other methods.
- Response rates are often low or are unpredictable and may result in biased results.
Critical Activities and Inserts for Paper Surveys
For the best response rates, consider the following activities while planning a paper survey:
- Pre-Notification and Commitment Cards
Pre-notification usually involves a postcard or letter explaining the impending survey delivery and requests participation. Commitment cards ask the potential respondent to return a postcard indicating they will participate in the survey. Results of a recent study indicate that pre-notification cards provided a response rate of 32.3% while the commitment card resulted in a response rate of 20.4%.
- Reminder Postcards
Reminder postcards are sent to survey recipients roughly one week after the initial survey mailing. This card serves as a thank-you for those who responded and a reminder for those who haven’t. Our experience with these cards suggests they are very helpful. Findings range from a 10% increase to a response nearly equaling that of the initial mailing.
- Return Postage
Rather than requiring the respondent to provide their own postage, Return Postage has been shown to increase response rates significantly. In addition, using stamps on return envelopes has been shown to be better than business reply mail. However, depending on volume, the incremental return gained by stamping the return envelopes may be more than outweighed by the cost.
- Cover Letter
A strong Cover Letter is also a key motivator for respondents. It should explain the importance of the survey and the potential benefits to the respondent for the most direct and influential approach.
- Re-Mail Surveys
Re-mailing surveys entails distributing a second survey to either the entire respondent base or to non-responders only. Two studies provide considerable evidence of the benefits of using this approach. In one, a re-mail improved response rates from 39.5% to 50.0%, and the other showed an increase from 38.8% to 52.9%.
Types of Mail
Survey professionals have found that what type of mail you use to distribute and receive the survey has an impact on response rates. Research has examined both different mail types and postage types with results indicating they do have an impact on response rates (although there are frequently other variables involved).
- Certified Mail™ seems to be helpful for a lengthy survey. In a recent study of the impact of certified mail, varying questionnaire lengths were tested in combination with a certified mail process. The results indicated certified mail had little effect with short surveys. However, the response rate for certified mail was nearly double that of regular mail for long surveys.
- Express Mail® has also been found to positively affect response rates, especially for executives and business respondents. A recent study demonstrated that when express mail was used for both the mailing and return of the survey a response rate of 52% was achieved, compared to a response rate of 26% for the group receiving and returning their survey via regular US Mail. Express mail may also convey the importance of the survey to the potential respondent.
A survey must have a good response rate in order to produce accurate, useful results. We have outlined several ways you can do to improve your response rate. The response rates for a survey is critical and you shouldn’t t just leave it to chance. If you care about the data, you need to care about getting the best response rate possible.
How Can Scantron Help?
Scantron has 40 years of experience in building solutions that allow companies to measure various areas of their institutions—instructor and course evaluations, student and staff satisfaction, campus services satisfaction, learning outcomes surveys, and more. We can provide all of the pieces of your program, using our flexible course evaluation and survey application: Class Climate.
To be successful, you must consider and manage the distinct aspects and phases of a survey or evaluation:
Scantron offers a range of customized Implementation Services to enhance and support your Class Climate implementation process, including dedicated experts with extensive experience in course evaluation management. These services will enable you to implement your system and begin processing your surveys quickly and efficiently for a more rapid return on investment. The Scantron implementation team works with you to create a manageable schedule customized to ensure your success in the evaluation process. Depending on your needs, consultations can be done onsite or via the phone or web.
- Initiate Surveys
Save time and money by dispatching online surveys via email or batch-printing the paper questionnaires for an entire department. Create and distribute personalized questionnaires automatically. For online surveys, invite participants via email, paper, or LMS integration (e.g., Desire2Learn®, Blackboard®, Moodle™, etc.).
- Capture and Feedback
Data collection is one of the most complex aspects of survey administration. Gathering responses from survey participants can appear daunting to even the most seasoned professionals. Participants may be responding by paper or web. All of this data has to be collected—quickly and accurately—into a common database for reporting and analysis. Scantron recognizes this challenge, and offers the tools necessary to make the data collection phase as straightforward and reliable as possible, while saving time and money otherwise spent on data entry personnel.
- Further Analysis
Evaluations are conducted for the response data. If you’re like most administrators, you inevitably want to look at data from several angles. Being able to filter the data, then display it in a variety of clear and informative reports allows for more thorough analysis. Scantron understands that the more quickly you can put reports in the hands of an institution’s influencers and decision makers, the more quickly change can be effected.
- Quality Management
A Quality Management dashboard shows quality indicators as graphs or indexes as well as target values on scaled questions. Browse results and identify quality problems by comparing received responses to outside, validated standards. Set quality thresholds to automatically receive alerts about poor performance via an emailed report. Class Climate also allows for benchmarking to measure against future evaluations.
With Scantron, you can choose exactly the package of software, hardware, services, and forms to give you a total survey solution. We provide you with powerful functionality, depth, and breadth—notable attributes in the survey industry.
Contact us today to find out how the power of Scantron Class Climate can help!
Back to Class Climate
© 2014 Scantron Corporation. All rights reserved.