Creating a Product Feedback Cycle – 352 Noodles & Doodles

No one launches a product or website and expects it to just work straight out of the box. Well, some people do – and some people even get remarkably lucky with a product that works perfectly at launch. The rest of us  rely on hard data, in-depth user testing and ongoing product testing. It’s called the feedback cycle, and it’s useful in just about every industry and in any user-facing product or service. And it can mean the difference between a product that meets your users’ needs for years and one that is doomed to fail.

The beautiful thing about a strong user feedback cycle is that it can – and should –  be used at any stage of development, from the ideation phase to post-launch bug fixes. In this week’s episode of 352 Noodles & Doodles, 352 director of design and UX Pete Bernardo will show you just how easy it is to implement an effective user feedback cycle, and the tools you’ll need to conduct each step of the cycle.

Transcript below.

Image credit: General Motors

[Pete Bernardo:] Hey, I’m Pete. Im the director of design for 352, and today I’m gonna talk to you about the 4 keys to a feedback cycle. 
 
As users engage with your website, they’re gonna be using desktop applications, tablets and smartphones, and it’s really important that you stay in tune with how they’re engaging with you. There’s a really great case study by Google called “The New Multi-Device World.” If you’ve read that case study, you’ll know that users are always evolving with how they interact with you online. They’ll start on a desktop and finish a conversion on a smartphone, or they’ll be engaging with you while they’re on the couch if they had a point of inspiration from something going on on TV.
 
The key with this, since they’re always involving, you need to have a good feedback cycle to understand how they want to engage with you and your product. So, I’m gonna talk to you about 4 Keys to a Feedback Cycle so you can stay in tune with your customers.
 
The first part of the feedback cycle is gonna be Analytics. With Analytics, the key thing is that, while it’s foundational and you may have been using it for a few years, that you pay attention to whether it’s evolved with your product. What I mean by that, as you’ve added landing pages or microsites, or maybe you’ve added a mobile application or an online support system – does you Analytics span to all of those systems? Does it reach everywhere you’re engaging with customers? If it doesn’t, it needs to.
 
And once you have healthy data and once you’re reaching all your users, or historically understanding them, you’re going to ask questions like: “Why are they coming to my site? Where are they coming from? (Geographically, that may be important to you.) What devices are they using to reach me? Does the behavior of people on smartphone differ from someone on smartphone from someone on a tablet or a desktop?”
 
So you use Analytics to understand everything about your customers historically, but another component of Analytics, especially in a feedback cycle, is to inform other parts of the cycle. So you use Analytics to make better survey questions. If you see abandonment or other issues in Analytics, you can ask it in a question in a survey. So with surveys, they can be a link on a website, or it can be an email you send out to customer base to request a quick survey. With surveys, we want to keep them conversational and quick. Don’t bombard your users, so they’re more likely to complete it and give you really good feedback.
 
One of the neat things about surveys, one of the critical parts, is that we want to measure attitudes and not behaviors. The reason we say that is because people are really bad about telling you how they did something, but they’re pretty good about telling what they like or don’t like. So, with surveys, when you think about your questions and try to keep the questions streamlined, ask them questions about what features they’d like to see or what issues they’re having with your product. Don’t ask them how they’d go about completing something.
 
A newer part of surveys, and something that’s actually very interesting, is that there are tools now – from SurveyMonkey and uSamp – they’ll actually allow you to build new audience. They’ll allow you to actually reach customers, people that aren’t your customers yet. You can set up demographic information or geographic information, and you’ll pay a price to these services, but you’ll get really valuable feedback if you’re trying to reach a new audience.
 
When you have your surveys and you understand what new features or what issues you’re having with your product – you’re gonna make changes right? But the problem with that is that you’re still sort of guessing a little bit. You can’t always just nail it the first time. So, a key thing to help with that is actually remote testing. With remote testing, it will allow you – in development – to make your changes, whether it’s content or functionality. And you can put it in front of real people.
 
And the same way we did with surveys, where we could pick who we’re putting it in front of if you’re using someone other than your audience, you can do that with remote testing as well. If you’ve never seen a remote test, they’re actually really, really interesting. You’ll see the end-user, you’ll see the device. And you’ll actually visually understand their frustrations or when they had a Eureka moment.
 
So I want to show you an example of a user test, we’ll pause right now and watch a quick video.
 
[Amazon User Test video]
 
So I think you can see how powerful those videos are – we get to see the end user using the Amazon application, you get to hear any positive or negative sentiment to how something went, and they’re very powerful. In this instance, he was using a mobile phone, but this works just as well on a tablet and will work just as well on a desktop application.
 
With remote testing, it’s not about pass or fail. With surveys or with analytics, it’s very pass/fail. Did this work for you or did this not work for you? With remote testing, you actually get to understand where their stumbling blocks might have been, and what prevented them from succeeded, or maybe they misunderstand what you wrote and they don’t know how to articulate that. Because you’re watching them, you get to see it and you can say, “Oh they missed it, it’s right there! I can’t understand why they missed it.”
 
There are a lot of studies and statistics that if you just do remote testing with 5 users before you launch, you can eliminate up to 80% of post-launch issues. So remote testing is a very valuable resource to getting your product right before you push it public.
 
Once you’ve pushed it public, the next component is actually understanding what works best for your end users. So while we’ve asked them questions about what functionality they like with surveys and we’ve watched them use your site with remote testing, it’s not until we get a lot of mass-market understanding that we can really optimize how your site works.
 
What split testing does is allow us to run two tests behind the scenes and judge whether copy or content or emphasis really impact the end user. So what that means is that you can go into a system, and say: I want to change the wording on this plugin. I want to go from “Join Now” or “Start Now,” or I want to change the color from blue to orange. And you get to see behind the scenes which one actually performs better. Your users will tell you, just by walking through your process. So whichever works best, you’ll find out.
 
And so, split testing and remote testing, they’re not rocket science. You don’t have
to be an expert software developer to use them. We have a blog post online that we’ll link to that goes itno more detail about the tools that can help you at each stage of this.
 
The key to a feedback cycle is that it is truly a cycle. Whether your product is just in the idea phase or whether you’re currently in development, or maybe your product has been around for a few years: the feedback cycle applies to whatever stage you’re at. So just hop on wherever you feel most comfortable, and keep iterating and going through your product and making it better.
 
I’m Pete Bernardo, I hope this helped you: Feedback Cycles.