Time on Site is the Denominator

A major media company recently did a major redesign of their web site. When users complained about all the problems, and there were many, the sites (re)designers said the problems were the users' fault for not being familiar with the changes, and they pointed to an increase in the time users were spending on the site as proof that the redesign was successful.

It didn't occur to them that there was another very likely reason for the increased time on the site, namely that users were spending more time on the site because the site was harder to use and harder to navigate. In fact, parts of the redesign increased the number of steps needed to perform a number of common actions, sometimes significantly.

Underlying all this is that this major media company seems to not understand what time on site represents. It's not really their fault. Time on site is a phrase so prevalent these days that it gets 138 million hits on Google.

But here's the pivotal question: does the company get value from users spending more time on the site? In fact, they do not. Think about it this way. Which of the following two customers is more valuable to a clothing store?

• Alex goes to the store, spends an hour trying on pants to find one that fits and looks good, buys the pants and leaves.
• Pat goes to the store, tries on one pair of pants and discovers that they fit and look good, so buys them and leaves five minutes later.

The answer is obviously Pat, who left a happier customer. Because they hadn't spent an extra 55 minutes trying on pants, they are far more likely to buy other things, even before they leave, and they are more likely to tell their friends about their good experience.

If the store had measured time in store, they would have thought that Alex was a happier and better customer!

Time on site is still useful to measure — but in a different way. Here are two straightforward equations:

value received by users  time on site

value received by site (e.g., ad revenue)  time on site

Notice that time on site is the denominator in both equations. The first roughly measures how well you're satisfying your users while they're on your site, while the second roughly measures income over time.

Having visitors to your brick and mortar store costs you money, in terms of employee costs, utility costs, etc., even if the visitor buys nothing. The more visitors you have, the more employees you need, even if the visitors buy nothing. The same is true of your web site. Every minute somebody spends on your web site costs you for server time. Although the absolute cost may be smaller, it is not zero. The formulas above are obvious when you think about that plus the old standard ROI, or return on investment. If you measure and worry about about the return on investment for time on site, you'll have a much better understanding of how your site is doing.

What if you have an entertainment property? My company, Puzzazz, is a puzzle technology company that has built a popular free puzzle app for iOS. You might think we should measure time in app because more time in the app must indicate happier users. Nope. Take the people who solve just the New York Times crossword in Puzzazz. Some people can solve a Sunday NYT puzzle in a few minutes; others take an hour or more. Can we draw any conclusions about a user's happiness from the time it takes them to solve the puzzle? No. Different users are not comparable to each other for many reasons and doing things like averaging their times across different puzzles to determine value is not mathematically sound. Instead, we can look at things like puzzles per session and puzzles finished divided by puzzles started. Whatever your site or product is, you should spend the time necessary to figure out what the relevant metrics are.

It's certainly true that measuring your site's value and effectiveness is far more complicated than I lay out here. But it is even more complicated than measuring a number like time on site and expecting it to tell you much by itself. Especially when you use it as the numerator instead of the denominator.

Yes, Companies Do Listen

In October, I blogged about a major flaw in the Tully's web site, tullys.com -- basically, there was no way to find a Tully's location unless you knew about their secondary site, tullyscoffeeshops.com. Yesterday, Tully's fixed it, not only providing the link but making it prominent. Yes, it could be better still, but now it's a lot better than plenty of other sites.

The surprising thing was the post on their Facebook page (The Power of User-Generated Feedback for Companies) where they credited me with inspiring them to action. That's great to hear. Helping people improve the experience for their users and customers is why I write it. When I help people in my UX consulting practice or for free in my monthly UX office hours, I get to see the benefit firsthand. But, with my blog, it's hard to see the direct impact.

Thanks, Tully's.

Does Mint Want Unhappy Customers?

Imagine you go into a cheese shop trying to buy some cheese. You just want some Cheddar. You ask if they have it, but they won't tell you. Instead, they want you to sign up -- give them your name, contact information, bank and credit card accounts, and only then will they answer your question. OK. You do it, you give them all that information and then the answer is, no, they don't have any Cheddar, they've never carried it, and have no idea if they'll ever carry it. But they'll make a note of it as a suggestion.

Sounds preposterous, doesn't it? Well, it's Mint.com, a site which touts themselves as "the best free way to manage your money." But, if Mint doesn't support your bank, they can't very well help you manage your money, can they? They can't support you as a customer -- they shouldn't want you as a customer. But, they'd rather you go through the bother of signing up to discover you wasted your time because there's no way to find out if your bank is supported before you sign up.

But, wait, it gets worse. Mint seems to think this is a good idea. Their support staff argues that they can still help you with your other financial institutions and that tracking some of your financial institutions is better than tracking none. What?! The number one financial institution that most people have is their bank (or credit union). If they don't support your bank, Mint is probably useless to you. They should realize that. But, even if that wasn't the case, Microsoft Money or Quicken probably does support your bank, so tracking just some of your financial institutions is certainly not better than tracking all of them in Money or Quicken.

Does Mint really want unhappy customers? Apparently so, but it's a bad business practice.

If there are customers you can't make happy, you should send them away and you should send them away as early as you can. Don't spend your time and resources or waste their time if you know you can't make them happy. Send them to a competitor who can make them happy.

The fact is, customers who you cannot service are going to go away sooner or later -- your goal should be to make sure they don't go away mad.

Avoid Fictitious Use Cases

I have a stereo hidden in my kitchen cabinet, with wires that run to speakers in the wall. It's nice that the stereo's hidden away, but it's a pain to use. I wanted to connect it to the iPod on which we have all our music, and I recently discovered that Creative makes a wireless device that seemed perfect. I could put the iPod in a wireless dock on the counter and attach a wireless receiver to the stereo. Or so I thought. It turns out that Creative didn't really think things through.

The system consists of two parts, the Xdock and the X-Fi Receiver. There are only two ways you can possibly use this system, but the system doesn't really support either way.

The first way is to use the Xdock by itself and connect it to your stereo. But, there are no standard RCA audio connections to do so. The only outputs are S-Video, video, and audio optical. And, oh yeah, there's a headphone jack on the front. So, they've substituted an audio optical connection (and a mini one at that) for the universally standard RCA jacks. Just to make sure that I wasn't out of touch with the current state of the art in receivers, I checked on Amazon. Sure enough, the majority of stereo receivers that I looked at did not have audio optical connectors. RCA jacks are still the standard. So, even if you want to connect the Xdock to your stereo, the vast majority of people will be unable to do so. And, for me, there's nothing in my house that I can connect the Xdock to.

The second way is to use the Xdock as a transmitter and the X-Fi Receiver as a receiver. This is the usage that I bought them for. If you read Creative's web site, it sounds like you can control the whole thing from the Receiver. And that would be great. You tuck the X-Fi Receiver away in a cabinet or behind your stereo and just use the Xdock. Unfortunately, the very first thing that I wanted to control -- the volume, duh! -- can't be controlled by the Xdock. Only the volume control on the Receiver itself controls the volume of its audio output. It's as if the folks at Creative hadn't even thought of the obvious use of the transmitter/receiver pair.

Interestingly, the volume controls rotate continuously -- they're soft controls with the true volume level being set in firmware. So, of course, I expected both volume controls to work, and my initial assumption was that I had missed something in setup (the documentation is rather scanty). Sadly, there was nothing to miss.

So, how do the people at Creative think their devices are used? According to the support staff at Creative, "the sole purpose of the [X-Fi] Receiver is to be used by a user who would like to listen to the same music being played on the Xdock but not on the same volume level." What? The sole purpose for the X-Fi Receiver to exist is so that two separate people can listen to the same music, in different rooms, at different volume levels?! Yeah, there's a product need crying out for a solution!

I wouldn't have faulted Creative if they hadn't put volume controls on the product at all -- after all, the output has to go to an amplifier and most amplifiers have volume controls. If that had been the case, or if the sentence above had been on Creative's web site, I simply wouldn't have bought them and I wouldn't be a disgruntled customer. But, to provide controls that only work to support a fictitious use case, in a product that doesn't support multiple, obvious use cases -- that makes no sense.

Netflix Listened

This just in:

Wow. This is, unfortunately, a rare event. Netflix was adamant that the Profiles feature was going away, but they heard their users and reversed course.

"Because of an ongoing desire to make our website easier to use, we believed taking a feature away that is only used by a very small minority would help us improve the site for everyone. Listening to our members, we realized that users of this feature often describe it as an essential part of their Netflix experience. Simplicity is only one virtue and it can certainly be outweighed by utility." (from the Netflix blog: Profiles feature NOT going away)

My kids, who told me that the Netflix explanation made "absolutely no sense," will be quite happy. I still don't buy their explanation, but I guess it doesn't matter anymore. Now I can start thinking about that Roku box again. Bravo!

NetFlix is a Service, Not a Web Site

Just got this email from NetFlix:


The "helpful" FAQ page says nothing that isn't said or implied in the email. In short: Profiles are going away, yes, they're going away, yes, all the features of Profiles will be gone, yes, you're screwed if you rely on Profiles. Of course, they took a little longer to say that, but you get the idea.

I don't know about the average NetFlix customer, but Profiles (multiple queues) are an essential part of the way I use NetFlix. I have four queues -- the "family" queue, which has movies for my wife and I and the family, my "workout" queue which contains movies that I think are good for workouts (and that nobody else in my family wants to see -- alternated with MacGyver), plus each of my kids have their own queue. They're responsible for their queues, including returning movies after they've watched them.

But, NetFlix has decided to kill the feature in order to "improve the NetFlix website for all our customers." How exactly does that work? You take away a feature that I rely upon and somehow that makes the website better? And I love that amorphous "better" -- trust us, it will be "better" in some unspecified way. What feature will they give me to replace it?

Will they provide a quick way to reorder my movies, because it sure seems to me that, without queues, I'm going to be constantly reordering movies in order to make sure I get an appropriate replacement movie whenever one gets returned.

Are they going to provide a feature that allows me to give my kids access to the queue, without letting them remove movies I've added or view R-rated movies? Or am I just screwed?

Or have they just decided that I'm not their target customer and I don't matter enough? Is it a tiny percentage of customers who were using queues? That won't make me happy, but, if that's the case, at least they could be honest about it.

Underneath all this, I'm really annoyed by the statement that they're improving the "NetFlix website." It makes me think that, after all this time, they don't really understand their own product. I could care less about the NetFlix web site and I bet that's true of 99% of their customers. I use the NetFlix service. The web site is just the way to get to the service and that's true whether the service is DVDs by mail or movies delivered over the Internet. To me, an important part of the service is a slightly better version of what we had at the video store -- the person who returns a video gets to pick a new one. For people like me, the removal of multiple queues makes the NetFlix service worse than the video store. I don't think that's what they want.

I've really been quite happy with NetFlix. I was just about to buy a NetFlix box for my TV, despite the limited number of available movies. Now I'm going to rethink that.

Final thought: It's possible that NetFlix has some amazing new design that will give me all of the benefits of Profiles in a different way. And that's what I care about -- the benefits, not the feature. But, given their messaging, I doubt that's the case.

NetFlix, care to enlighten us?

Update: Here's a post about this on the NetFlix blog, with almost a 1,000 responses already:
http://blog.netflix.com/2008/06/profiles-feature-going-away.html

Don't Do Me Any Favors

Netflix did me a favor. Or at least they said they did.


On Friday, I sent a movie back to Netflix. The replacement is arriving on Saturday, more than a week later -- because they did me a favor and sent me the first movie in my queue.

Because of the Memorial Day holiday, Netflix didn't receive my return until Tuesday. Then, they sent me the note above telling me that the top movie in my queue isn't available locally, so they're going to do me the favor of sending it from San Jose instead. But, not only does this movie still show as available "now", but every other movie in my queue also shows as available "now". I'd be perfectly happy with the second movie. Or the third. Or the eighth. But, instead, they did me a favor, so instead of receiving a movie on Wednesday, I'm supposed to get it on Saturday. (But, don't cry for me -- I don't have time to watch it anyway.)

I'm not saying that software designers shouldn't try and figure out what users want and help them out. Of course, they should. But, ask yourself a few questions:

• Are you helping your user? or yourself?
• If you could ask the user what they want, what would they say? And would all users, or even almost all users, want the same thing?
  

For another example, my friend Mike Koss recently pointed out another annoying "favor". It's that wonderful Windows Desktop Clean-up Wizard that helps me out by nagging me that I have unused icons on my desktop. I never want to see it again! When they added the option to turn off the reminder (and buried it), did they consider whether they should just turn it off for everybody? Mike's post also adds his personal taxonomy of software annoyances and delights.

Misunderstanding the User

A while ago, I wrote about some good technical support by NetGear. Now here's some absolutely abysmal support by PayPal.

Like most PayPal customers, I use them because I have to, not because I want to. PayPal has a monopoly in online payment systems and there are things that I can't buy (or are much more difficult to buy) if I don't use PayPal. So, I use them a few times a year.

Recently, I bought something from a seller in Hong Kong who would only take PayPal and required that I become "verified" in order to sell to me. Now, I've been a PayPal user since 2000, but I never bother to become "verified" because the advantages don't matter to me. But, I figured, what's the disadvantage? I would add my checking account, then remove it after I'm verified. Unfortunately, that doesn't work.

When I removed my checking account, I once again became unverified. Apparently, nobody at PayPal bothered to look up the definition of "verify" and proper use of the past tense. Once I am verified, by definition, I cannot become unverified.

To make things more confusing, PayPal and the merchants who use PayPal have two different meanings for verified. In the one coherent response that I got from PayPal support, they even acknowledge this.

Becoming Verified will show others you have confirmed your identity in the PayPal system and the Verified status will increase confidence within the PayPal community.

Verification is also necessary to lift the sending and withdrawal limit on an account. Once you've reached these limits, the only way to continue to use your PayPal account to send and withdraw money is to become Verified.

Never mind that there's really no such thing as a "PayPal community" -- I think that's wishful thinking on their part. But it is true that someone I might transact with has a greater confidence in me if I'm verified. But I could care less about the limits. This support person also wrote:

The key to this is not that you suddenly become "not real"; we understand that you remain the same person who verified with us.

Interesting -- PayPal understands that I don't become "not real," yet they fail to communicate this to people that I transact with. Is there any logical reason for that?

By the way, if I were to give up and leave my checking account attached to my PayPal account (and I considered that), PayPal defaults to sending money from my checking account, not the credit card that I've used for 8 years. Every time, I do a transaction, I have to manually change it back to my credit card. Why?! And there's no way to change the default back to my credit card. I know that PayPal claims I'm 100% protected with my bank account, but I'd still rather use my credit card where the protection is mandated by Federal law.

In the process of figuring things out, I tried to get PayPal support to pass on a message to the people actually responsible for this, but they were completely incapable of doing that. They misunderstood what I was saying and sent me the same canned responses repeatedly. For example, in my last email to PayPal, I sent the following (this is an excerpt):

Personally, I could care less about sending and withdrawal limits or any of the other benefits of being "verified". If you insist on "unverifying" people who remove their bank accounts, you really should provide an indicator that the person has been verified as real. Hey, how about calling that "verified" and using some other word to indicate "currently has a bank account attached"? Please pass this information on to someone who can do something about it.

The canned response gave me the same instructions on how to remove my bank account that I'd already been sent before.

PayPal does two things wrong here:

• They're setting things up for their convenience, not their customers'. I would guess that there is an advantage to having my checking account be the default funding source and I would guess it has to do with revenue. But why should that inconvenience me?
  
• Their technical support staff lacks information they need and it looks like they respond to support queries without actually reading the questions they are responding to. When I send a message that says, in its entirety, "Will you pass on my feedback to someone who can actually change it instead of just sending me back canned responses? Pretty please?" and it gets a canned response repeating something I've already been told, it gets pretty comical.
  

I'm hoping that maybe somebody from PayPal reads blogs.