Surveys are a vital tool to measure and understand the perceived developer experience within a team or organization. People who run surveys often face the challenge of decreasing responses. Survey fatigue can be a huge problem when trying to get meaningful data about the state of the developer experience within an organization.
Also, in my last workshop on developer experience (Developer Experience Demystified for Engineering Leaders), the topic of surveys and survey fatigue was brought up.
In this article, I will share eleven tips on how to avoid survey fatigue. These tips will help you to increase participation and the quality of the responses. Therefore you will be able to obtain better insights into the perceived experience of your engineers and derive better measures to improve it.
Clarify the Purpose
One of the biggest mistakes I have seen and made myself in the past was to assume that people knew the intention of the surveys we were running. This often isn’t the case. — and, even if it is, it doesn’t harm to remember people.
1. Explain the Purpose
Hopefully, you are not running surveys for the sake of collecting data but with the intent of learning about the developer experience and how people feel about their work. The ultimate goal is to use those insights to improve the lives of the people who are participating in the survey.
This is a purpose worth explaining. Therefore, when sharing a survey, it is worth remembering the audience and “their why” behind this survey.
2. Show the Consequences of Previous Surveys
Explaining the purpose isn’t enough if people are skeptical about that purpose. If you are running surveys regularly, you hopefully achieve something based on the insights you got in the past.
Share what you learned and ultimately improved based on the results from previous surveys. This helps the audience to connect participation in surveys with positive changes in their life and work.
Simplify
Many surveys I see ask for too much. The intent seems to be to do longer surveys less often. However, I found that this doesn’t really work well as the quality of answers and also the number of finishers drops significantly for surveys over 10 minutes.
3. Keep it Short
I would aim for surveys that can be answered within 5 minutes. I definitely would not exceed a length that takes the usual participant 10 minutes.
Usually, the shorter, the better. That is if the results can still lead to meaningful improvements.
It is also beneficial to show the duration upfront. When you share a link to a survey, participants are much more likely to click it when they can expect a 45-second survey rather than a 15-minute survey.
4. Use Clear Language
Nobody likes surveys where you must read questions (or answers) two times before understanding them.
The faster and easier to understand your survey is, the better the experience for participants. This leads to less time needed to participate and less frustration, too.
Make it Engaging
Nobody likes monotonous, boring, repetitive, or irrelevant tasks. — And yet, this is how many standard surveys can feel to the audience. Surveys should be engaging and motivate the audience to follow through. Furthermore, they should feel relevant to the people answering them. The following three tips can help with that.
5. Personalize
Ideally, participants see only those questions that are relevant to them. It can be frustrating to fill out a survey where most questions must be answered with “does not apply to me”.
This is why contextual surveys (later more) are a great tool to get feedback while participants are still in the experience you want feedback about. Other options are to tailor the survey closely to the audience you are targeting and to use conditional survey flows. This allows you to hide all questions where a participant could likely not give valuable feedback.
6. Mix it up
Be aware that open questions are only one possibility for surveys. Open questions are great for getting very deep insights and answers you would not have expected. However, they also take longer to answer and are harder to measure afterward. The breadth of responses can make it difficult to derive a single measure beneficial for many respondents.
Consider other question forms, such as single- or multiple-choice answers or scales. Those are often easier to answer and give you concrete data to analyze.
7. Show Progress
Not knowing how far from a goal you are, feels pretty uncomfortable. Therefore, let people know how close they are to completing the survey. Given that your survey has a decent length, this motivates respondents to follow through.
Why this works? First, knowing that you are close to a goal motivates you to push through. Second, seeing how much of the way they already have behind reminds respondents that they have already invested time in the survey that would be lost if they abort.
8. Optimize User Experience
Consider in which context and on which devices people are supposed to answer your survey and optimize it for this purpose. Many engineers are used to control their computers using keyboard shortcuts. If your survey allows them to do this, this can be a huge benefit.
If people, on the other hand, tend to answer your surveys on a mobile, this factor is less important, and the survey should be optimized for smartphones instead.
Be chill
Asking for feedback is great. But asking for feedback too often or at the wrong time can be annoying.
9. Reduce Frequency
When you experience reduced engagement with your surveys, it can be worthwhile to reduce the frequency of your surveys. The right frequency depends greatly on the length, quality, and relevance of your survey for the audience. Therefore it is important to be aware of a potential starting survey fatigue and respond by taking the measures here.
Bonus-Tip: Join Forces
You might not be the only one surveying your audience. Maybe another team or department is also asking for feedback via surveys. Respondents often don’t distinguish this, and therefore, all the surveys contribute to survey fatigue.
The solution: Join forces. If the HR department is interested in cultural feedback, some of it can be highly valuable to gauge developer experience (once filtered for the engineering department).
Maybe there is the option to add additional questions for engineers only and avoid asking the same questions twice in different surveys by different parties.
10. Timing
Consider when your audience most likely has time to answer your survey. Is this when they start their work day or, more likely, toward the end? The problem with surveys is that they are often just forgotten. People see your Slack message or email and forget about it in the next second if the timing isn’t convenient for them.
Furthermore, engagement usually is higher when people feel the need to give feedback. This is often directly when they experience something feedback-worthy, be it good or bad.
Bonus-Tip: Contextual Surveys
Surveys sent out via a communication channel regularly are not the only way to get meaningful feedback. When people go through processes or use a tool, they usually have an immediate opinion about their experience. This can be captured with a simple star rating (and an optional free text) at the end.
Such contextual surveys take seconds to answer and give immediate and direct feedback.
11. Ensure Anonymity
It is crucial for people to feel safe to answer openly. Especially when there might be an issue with psychological safety in the organization, but also otherwise, it is beneficial to allow anonymous participation.
You might want to be able to get back to people and ask for clarification about certain answers. But by all means, do not add a mandatory name field or use other means of identifying individuals.
When people trust in the anonymity of their responses, they are likely to share more openly and give you the important answers.
Be Careful with Incentives
I often read and hear advice to incentivize survey participation with bonuses, coupons, or other measures. This can work to get more people to finish the survey. However, it might lead to a couple of problems:
- When people do not really want to answer the survey but do so just because of an incentive, the quality of their answers might be reduced.
- Incentives can lead people to answer surveys that they actually don’t have anything meaningful to contribute to.
- People will start to take incentives for guaranteed and only respond to surveys where participation is incentivized.
In general, I would be very careful with incentives to motivate people (Read also: Bonuses are bad – change my mind)
Summary
Preventing survey fatigue necessitates a thoughtful approach prioritizing clarity, simplicity, engagement, and timing. Here is a summary of the 11 tips:
- Explain the Purpose: Clearly state the intention and the benefits of the survey.
- Show the Consequences of Previous Surveys: Showcase the impact of past surveys to encourage participation.
- Keep it Short: Keep surveys concise and within 5-10 minute completion timeframe.
- Use Clear Language: Make the survey easy to understand to minimize frustration.
- Personalize: Personalize surveys based on respondents’ roles and experiences.
- Mix it Up: Use various question types to retain interest and gather diverse insights.
- Show Progress: Show respondents how close they are to completing the survey.
- Optimize User Experience: Tailor the survey to fit the device or preference of the respondents.
- Reduce Frequency: Be mindful of over-surveying, which can lead to survey fatigue.
- Timing: Determine when respondents will most likely have time to engage with the survey.
- Ensure Anonymity: Give respondents the choice to remain anonymous to foster openness.
By employing these strategies, you can effectively decrease survey fatigue, increase engagement, and gather more meaningful data.
Are you experiencing survey fatigue in your organization? Schedule a free 30-minute call where we will discuss your situation and how to get more meaningful feedback from engineers.
One Response