Survey response in the time of pandemic

As the COVID-19 pandemic persists, we know that many continue to be hesitant to field surveys and conduct research. We're still monitoring response rates and data impact and have provided an update on what we've uncovered since this originally posted.

It is a common refrain in the survey research industry: any change impacting large populations will have an impact on response rates and possibly change how people respond. During natural disasters such as hurricanes or typhoons, many halt research in the impacted areas. There is too much disruption to daily life to get an accurate read, respondents lose connectivity, and often response rates are low. Who wants to complete a survey when they are dealing with flooding or needing to evacuate their home?

That’s why many of our clients, across all industries, have been apprehensive about conducting research as usual during this uncertain time. Specifically, since the announcement of the European travel ban, we’ve been asked about whether or not we should continue conducting research.

By now, this unprecedented crisis has had a global impact and, in just seven short days, upended the way Americans live. Families are having to adjust to long-term school closings, putting the onus on parents and distance learning technologies to continue the education process. Those who are fortunate enough to still be working are doing so from make-shift home offices, while many are finding themselves unemployed or with severely reduced hours. We are told to remain as isolated as possible, and in that we are learning just how much we yearn for human connection.

So, with all of the rapid change, companies are right to be concerned about the impact of conducting research during these times.

• Are people too distracted or anxious to deal with completing online surveys?
• Have priorities changed as we await a turning point in the spread of coronavirus?
• Perhaps, but aren’t those important topics to understand as well? 

Fortunately, we can examine survey response rates over the last several weeks, including leading up to and through the European travel ban and the subsequent recommendations and additional precautionary measures. This insight helps put us at ease regarding the impact of the crisis on survey response.

Response rates are up

Data from Bellomy tracking studies and our sample partners all point to solid evidence that response rates are up, not down, over the past week.

While it is still early, most of our partners report response rate increases as much as 4 percentage points and none are reporting decreases, particularly when fielding B2C research. When looking at data from one of Bellomy’s utility trackers, we see similar gains in both click-through and response rates, around 2 percentage points in the week after the travel ban was announced.

The new work-from-home culture

With many commuters now grounded at home and not waiting in carpool lines at the local middle school, Americans are finding time on their hands. This extra time is most likely spent in front of a screen, so those who typically pass on a survey invitation may now be taking the time to open and complete it. It helps them pass the time, not to mention put a couple of extra dollars in their pockets, at a time when many of us will be seeking ways to earn extra income. 

One of our sample partners reported that they have received the largest number of completes in a single day over the past week — another indicator that the new way of life, however temporary, has people clicking research invitations and completing surveys more often. 

The bear market

While many commuters and nine-to-fivers are completing surveys while social distancing, other industries are feeling the impact differently. Take financial advisors, for example, who saw an 11-year bull market and record-setting highs in the stock market evaporate in a matter of days. Any available time is now spent consoling panicked clients unsure of what to do as they watch their savings take a sharp tumble into a bear market and near recession over the last week. How did this drastic, rapid change impact financial advisors’ willingness to participate in survey research? Surprisingly, our click-through and response rates for a study sent to financial advisors was the same both before and after the plunge into a bear market on March 11th and 12th.

Advisors, despite mounting pressure and dealing with anxious clients, still participated at the same rate during this research! 

What about other industries?

Are there other industries in which survey response is negatively impacted? While there are some unfortunate recent impacts to B2B research in the areas of small businesses and hospitality, another industry that has seen the biggest negative impact is healthcare. This shouldn’t come as a surprise given the strain the pandemic is causing on healthcare workers and systems and is validated by some of our sample partners who have seen hits to response rates among this group. Therefore, we recommend caution if planning research in this industry, perhaps waiting until hospitals have been able to control COVID-19 cases.

Quality of data

Our next step will be to monitor the quality of responses by comparing data before and after the coronavirus pandemic and travel ban. Since response rates have increased, it’s possible that trends will change as well. We can assume that non-response bias will decrease and perhaps the data will become more representative of the population being studied, all of which are positive outcomes. However, care will need to be given when analyzing data from both longitudinal tracking studies and ad hocs fielded before, during, and after these disruptions to understand the true causes for any shifts or changes in trends during this time.

Overall, despite these anxious times for many Americans, we are confident that survey response rates have not been negatively impacted for most types of research. While much of our day-to-day has been heavily impacted in order to mitigate the spread of coronavirus, we are glad to know that some aspects of our routines can continue as normal.

In addition, stay tuned to the Bellomy blog as we combine our research savvy and technology capabilities to listen to social media and connect with our proprietary SmartLab panel to bring you more specific impacts of the coronavirus pandemic. We’ll keep you informed because we know knowledge is power and we are all in this together. Stay safe and healthy!