There are often competing demands in market research that involve making trade-offs. The survey duration is a key example of this. When we have the opportunity to engage with a customer, we naturally want to find out as much as we can. However, longer surveys increase the respondent fatigue, reduce the response rate potentially skewing the data and perhaps worst of all, lead to unreliable data.
Another dilemma relates to the type of data we get from structured surveys. Rating scale questions (e.g. agreement scales, satisfaction scales etc.) are great for quickly analysing respondent feedback, but they also assume to know what is important. On the other hand, open ended feedback allows people to tell us what they want to say, but they produce unstructured data that can be difficult and time consuming to categorise and analyse.
At Evolve, we believe that the answer to this dilemma can be found in conversational engagement and text analytics. We believe this is the future of insights and have developed new technology to support a conversational approach to customer insights. Known as EVE (Evolved Voice Engine). This technology enables open ended dialogue with respondents to (a) giving them license to express themselves in their own words whilst simultaneously (b) understanding what they are saying and categorising it. We have been beta testing EVE on various tracking projects we run across categories and can now report back some compelling evidence as to the effectiveness of this new approach.
Thinking back to when we started this journey, we had discussed what success looked like in terms of using conversational surveys. The following are three areas we identified as important to success:
- We wanted to encourage respondents to tell us more by probing them and reflecting particular key words of the conversation back to them.
- We wanted to show that by increasing the amount of information provided by respondents, particularly into areas important to them, that we could better understand and explain their experience.
- And we wanted respondents to have a shorter, better surveying experience.
Based on the first generation of EVE test base, we analysed n=1,506 surveys and n=2,434 individual conversations using EVE. We look at the effect of the first answer respondents provide Eve as opposed to further interactions where respondents provide additional information. This allows us to compare the effect of EVE compared to a traditional open-ended feedback question.
Respondents give us significantly more information. We looked at how much information respondents were sharing when first probed by Eve, compared to the rest of the conversation. We found that 81% of respondents increased the volume of feedback by 50% or when EVE started dialogue with them after their first response. One in five respondents have more than twice as much to say when probed.
The increase in information allows us to better understand respondent and their mindset. A common methodology we use in market research to understand the drivers of respondent experience is ‘driver analysis’. Using this methodology on the ‘first’ and subsequent text data, we found that our ability to model respondent experience improved 24%. This shows that asking respondents more about their experience can help us to better understand their behaviour and overall feelings. This translates to valuable insights that can our clients can use to improve products, experiences and customer and employee satisfaction.
Respondents enjoy the surveying experience. When asked about their survey experience directly following the EVE interactions, 93% agreed that ‘was easy to complete’. Four in five (81%) agreed that ‘The conversational approach to capturing my feedback was a positive experience’. Almost all the remainder were at least neutral about this and only x5 disagreed. This is strong evidence that the experience for the respondent can be improved which means less change of dropping out of the survey and more thoughtful responses.
<img class=""alignnone wp-image-8291 size-full"" src=""https://www.humanlistening.com/wp-content/uploads/2019/03/Eve_Conversation_NPS.jpg"" alt=""conversation iphone"" width=""452"" height=""340"" />
These results are an encouraging sign that our approach offers a step change in how our clients can engage with customers. As a replacement or adjunct to existing surveys, the conversational approach is objectively better. Speak to an Evolve Research consultant today to see how we can improve your customer experience program by deploying EVE conversational approach.
