Listen to Episode 3.
Transcription:
John: Welcome folks. We are here to do our second podcast, I am here to introduce Piper Lofrano.
Piper: Hello.
John: Hey Piper. So your job this time is to ask questions on behalf of publishers. Take it away.
Piper: So today we’re going to be talking about how to evaluate looks and usability when it comes to testing layouts. One of the biggest hurdles our publishers face when testing is the shock of seeing a new look to their site. How would you respond to someone who is nervous about changing their layout?
John: About changing their layout? Yeah, this is something we found over the last few years to be, probably the number one thing that people are concerned about, and it is a counter-intuitive point, it’s something to get over. What you’ve got to look at is your site hasn’t always looked the way it is looking today. It hasn’t looked like that forever, it has changed over time. What you’ve done as a publisher is, you’ve changed your content according to what you think the users will like- so effectively you’ve been doing testing, you’ve been doing layout changes, it’s just you haven’t been doing it in a formalized way, and you’ve probably not been doing it in a very drastic way. I mean, people in WordPress do it quite often. I think the idea of changing your theme is relatively new for most publishers, but it’s certainly not unknown. People are nervous, but you’ve got to remember that everything is reversible, particularly if you’re using the Ezoic system, and let’s be honest: unless you test something, you’re going to end up with the same site and looking exactly the same as it is now for the next… however long! So embracing change does take some guts, but you need to do that. Running a website is testing, that’s what people say. You’re trying different things. If you go to Amazon for example, you’re not going to it for its beautiful looks. You’re going there because the utility is better than other eCommerce sites. So do I only shop on beautiful looking eCommerce sites? No. I go to sites that work the best and have the best prices and help me find things and have got good feedback or good features. It’s the same with content; if you produce content you need to make sure that it’s usable, people can find all of the content that they need.
Piper: Mobile friendly.
John: Exactly. Mobile friendly. So I would say: use the controls in Ezoic, go through the previews, see a site, exclude the ones you don’t like, but bear in mind excluding ones you personally don’t like does restrict your testing to your own personal preference… So that does mean, you know, certain people say: “Oh a left menu site isn’t as good looking as a top menu site.” Is that the case? It’s hard to say because your particular set of key words are going to be producing different user experience results than another site that has a different set of key words.
Piper: So it’s all about looking at the data and that’s the most important thing for testing?
John: Yes, I would say concentrate on bounce rate, time on site, page views per visitor. If the bounce rate is going down, page views per visitor is going up, time on site is going up, that’s a good user experience and no matter which way you kind of cut it and say “well I personally don’t like it” you’ve got to go on the user data rather than the subjective data. Everybody who has owned a website for a long time is used to viewing their own site in the way that it is- and how can you instantly make yourself a new user? I think you were saying that anyway. When you go browse online for a recipe let’s say, you don’t necessarily remember the layout.
Piper: No. No. I remember the content and the recipe I found.
John: Is it displayed well? Can I see it on my phone.
Piper: Yeah, much more important.
John: Does it load quickly? Can I find other recipes easily?
Piper: Definitely. So what about return visitors who are used to seeing the site as it is? Will they be shocked when they see it? Do you guys do something to protect them from that shock?
John: Well we ‘cookie’ all the visitors so when you’re using Ezoic, Ezoic is an automated page layout testing platform, tests ad positions and page layouts. New users are treated differently to existing users. So people who are coming back to the site and that’s Google Analytics, people who have been to the site within the last 30 days get cookied and they get a different, we bring them into our system, or the system brings them in much more slowly. So anybody who is a brand new visitor who hasn’t been to the site within 30 days will be ‘cookied’ into a user group, which becomes an experimental set and then the return visitors are cookied into the old site for longer. So, particularly in the early days, a lot of your older visitors won’t be seeing the new layouts.
Piper: Alright. So what if a publisher starts testing and they get an email or a comment from a visitor saying that they don’t like the new layout? This is probably a return visitor saying they don’t like the new layout. What would you say to them?
John: I would say if you make any changes to your layout you’re going to get reaction. Quite often it’s skewed towards negative reaction because human nature is that people don’t like change. They genuinely don’t like change. You look at any update, whether it’s IOS update of your iPhone, Facebook has got a new layout that they rolled out, whatever it is. People love to say that they don’t like anything that’s changing, but again it’s going back to the numbers. If usability metrics are improving, you’re doing the right thing by the users. The majority of the users who will not write to you and say: “Hey thanks for that great user experience on the cooking site that I visited today. I really enjoyed navigating the new menus!” People don’t do that, people very rarely get in contact for positive reasons so again I would say try to be brave. If you were doing this yourself you would be making changes. It’s just you’re accelerating the number of complaints that you’re getting because you’re accelerating the changes that you’re making. You’re sort of compounding what would be (for you on your own doing your own tests) months and months and months of work into a few days so that means that you’re compounding all of your complaints into a few days. Of course, if there’s something wrong you should go and check out what they’ve got to say, but on the whole, it’s to do with subjective preference rather than a genuine usability complaint.
Piper: That’s what I think the biggest thing is is separating this subjective opinion from the actual data. For example, when someone goes into WordPress and chooses a theme, how are they choosing that theme? It’s because it’s pretty, but you don’t know how well it works and so I think that’s why what we’re doing here testing is the best way to find a new layout because you know it works based off the data.
John: Exactly and it is a funny thing to get your head around isn’t it, that different people are going to be seeing different layouts on your site and it’s not the same as you are looking at. I think that is an odd thing.
Piper: Definitely and when you’re working on it everyday or years at a time and you’ve seen it in this one layout and all of a sudden it changes. It can be shocking, but we know the methodology works.
John: Yes we do, and before we start ego stroking ourselves too much, we’ve got to try to be objective. We’ve got to be able to see it for what it is, that layout testing is a really important thing to do. If you’re serious about improving your usability, the only way of doing it is not doing it by eye or using a designer usability expert. It’s really by testing the theories with your real users and not having group things, which to be honest, a lot of sites that I have seen that have been built by usability experts or whatever have the same flaws as people who have built them themselves. I think the thing to remember about Ezoic is this isn’t us saying that we can produce a better looking website or a better site. It’s actually the users. The user data is what drives the testing and that is what everybody should be going for when they’re trying to change their site.
Piper: Alright.
John: Good enough?
Piper: Yeah, I think so.
John: Thanks Piper.