Remember the “Why” When Measuring Site Performance: Voice of Customer Methods & Tools
With all that can be learned from clickstream web analytics tools such as Google Analytics, Yahoo! Web Analytics, and Omniture Site Catalyst, it’s important to remember those tools reveal the “what” of your site visitor sessions, but not much about the “why.” Did your visitors see the pages they wanted to see? Were they able to complete their tasks? Is a high page-per-visit rate a result of an engaged visit, or visitors unsuccessfully browsing through pages not finding what they’re looking for? Voice of customer tools are an important component to any website and web analytics initiative and can provide valuable “Aha!” moments.
In this post we’ll take a look at some of the most popular tools and the best ways to utilize them.
SURVEYS
We’re all familiar with surveys, however they come in all shapes and sizes and selecting the right one can be challenging. Some do a great job of collecting and segmenting results, others are annoying and drive your visitors crazy! It’s very important to evaluate not only the technical capabilities of your survey tool, but its user experience as well.
When redesigning a site, ensure survey integration is part of the overall technology set. The benefits of making surveys readily available include:
- They can be always on and serve as a continuous listening methodology. Surveys don’t always need to be sent out. They can be prominently featured on the site so visitors know they can leave feedback if they’re so inclined.
- Both qualitative and quantitative data can be collected
- Feedback can come in real-time so you can evaluate the positive or negative impact of newly-added site features or content
- A small sample size of responses can still yield valuable insights
- Surveys have evolved beyond the simple pop up to use cookies and conditional logic as well as integrate with clickstream data
Site-Level Surveys
You’ve likely come across these types of surveys that often fall under the “annoying” category. They are presented either as a popup, or pop under window and ask general questions about your overall experience using the site. Another option is the permission based survey. Upon entering a site, you’re presented with an invitation to take a short survey at the end of your visit.
If you use this approach, be sure to select a vendor that utilizes cookies so that repeat visitors are not invited each time they come to the site. For instance, visitors should only be invited to take the survey once every three months.
iPerceptions offers a great free site-level survey tool called 4Q which asks only the following four crucial questions:
- What is the purpose of your visit to our website today?
- Were you able to complete your task?
- If you were not able to complete your task today, why not?
- Based on today’s visit how would you rate your site experience overall?
With answers to these questions, you get a great macro-level view of your site’s performance. Also, you can calculate a valuable metric: Task Completion Rate by Purpose of Visit. There’s almost no clearer indication of whether your site’s doing what it’s supposed to do.
Also, consider prompting users to take the survey from locations other than just the homepage. Be contextual and get segmented data by creating surveys for specific sections of the site. You’ll be more likely to get responses from visitors who have a vested interest in the quality of content and/or features that they use regularly. Used this way, site-level surveys can function closer to (often preferable) page-level surveys.
Other great site-level survey tools include:
Page-Level Surveys
Page-level surveys courteously sit in not-too-obtrusive places waiting for a user to take advantage of them. They often sit at the end of a browser, either on the right or left side or in a bottom corner. They are less likely to annoy visitors and provide the excellent benefit of highly contextual feedback which is the key to actionable data. You’ll hear from your most engaged or upset visitors and will be able to directly attribute their comments to specific elements of your website. This is a great compliment to A/B and Multivariate testing (another form of “what”).
An important thing to remember whether you’re using site-level or page-level surveys is to keep them short! The longer your survey, the less likely your visitors will complete it. Value your users’ time and only ask the questions they need in order to submit feedback.
Great page-level tools include UserVoice and Kampyle.
POLLS
Polls are essentially mini page-level surveys. These are great for quickly taking the temperature of your customers and visitors. They can be easily embedded in a sidebar or even within an email. The best polling tools allow someone to see the results of the poll after taking it. For example, a poll can be displayed in a user account area to solicit votes on which potential features would be most appreciated.
Polling tools:
RATING SYSTEMS
While most commonly featured on ecommerce sites in the form of product ratings, these tools can be used on non-transactional sites as well. A simple YES/NO answer to the question, “Did this [article/FAQ/video] help you?” can quickly let you know if certain content isn’t pulling its weight. More sophisticated sites use this information to dynamically sort content so the best rises to the top of lists.
A nice and inexpensive rating tool is RatingSystem.
Whatever tool or approach you select, make sure it’s appropriate for your site visitors and takes users, content, and context into account. Balance the “what” of your data with the “why” for truly actionable insights.
Let your voice be heard here! What tools, methods, and approaches have brought you success or challenges?
Internet Marketing, Social Media, User Interface Design, Web Analytics, Web Development
Speaking of VOC tools, I just heard of a very timely release from Google.
Sidewiki is now a feature in the Google Toolbar! http://bit.ly/vz2Cb
This allows any one to start a comment thread about any web page. This effectively enables the entire web with a page-level survey/poll/comments area. Ready or not, your page may start receiving feedback soon!
Great post. I completely agree that clickstream analytics, as valuable as they are, only give us half the picture. I think when we try to use behavioral metrics to infer what customers are thinking, it’s a little like paleontology. We’re essentially looking at fossils trying to determine what the dinosaurs were eating and what color their skin was. But we’re now dealing with dinosaurs. We’re dealing with real, live human beings, and we can talk to them through surveys.
I will caution, though, that not all surveys are created equally. We’re dealing with the human psyche, which is a complicated thing. It’s important we ask questions very carefully to ensure we are getting accurate results. For example, the Myers-Briggs test is a scientifically proven method for assessing personality. If you’ve ever taken it, you know how thorough and accurate it is. The results you get are very different than you would get if you simply asked someone to describe his or her personality.
I also worry that we sometimes ask about task completion in too simple a manner for us to gain actionable insight regarding why the task was or was not completed. Because our sites and customers are complicated, figuring out how to solve for the gap between intention and action requires the analysis of millions of variables. We have to ask carefully selected follow up questions and perform detailed analysis of actual site usage in order to better understand the underlying reasons for task success or failure. Such analysis—especially when it includes understanding customer mindset—requires complicated methodologies and can’t be made actionable by simply asking if customers completed their tasks when they visited the site.
Full disclosure: I work for ForeSee Results, so I am clearly biased towards the ForeSee methodology. However, I was a client for six years at Borders.com and TowerRecords.com before joining ForeSee six months ago, so I can attest to the value of measuring customer experience the ForeSee way.
I hate ForeSee Results popups and those like it. For surveys that are supposed to be “scientific”. they totally lack common sense. These annoying things pop up in the worst moments possible- like when people first enter a website, or they’re reading a product description, or entering their credit card information. It ruins the user experience and leaves a bad impression. And of course, the surveys are long and intrusive; but unlike others that give discount codes or raffle entries, there’s nothing worthwhile for participants at the end. I mean there’s no incentive to take one of these ForeSee Results surveys besides the “help us improve your user experience” shtick. What a bunch of baloney. What happened to the days when I can give feedback directly to the store I’m shopping from, not some third party annoyance that’s profiting more my time and effort?
I’ve seen enough Foresee Popups up till this point that I no longer ignore them. I actually take them and give as much false/ skewered information as possible. Seriously, retailers should put a little more common sense in online user experiences- this means as little distractions and annoyances as possible!
Apple prompted users for a survey in their support section in a way I thought was very refreshing.
Rather than take the overlay/popup up approach which forced the user to deal with it before viewing any content, it was formatted in more of an alert in a thin horizontal bar across the top of the page. It didn’t cover any content, and if the user closed it, they wouldn’t see it on any other pages as they navigated the section.
To me, it seemed an effective way to make the survey callout noticeable yet not annoying. Something I think other major survey vendors should emulate or at least offer a similar option.