Thursday, October 29, 2009

Why You Need Anecdotal Evidence

Have you ever heard the phrase "That's just anecdotal"? Usually, it's used to discount somebody's opinion about the way something should be. In UX terms, if you've observed a user, saw them have a problem, and want to respond to what you saw, you may have someone tell you it's just anecdotal -- in other words, you don't have hard, objective, statistical evidence, so forget it.

But there's a contradiction here. There's a belief that, if you gather enough anecdotal evidence, it magically becomes hard evidence. You can even see this in the latest ad campaign for Windows. Millions of people sent in their ideas, or were surveyed or studied, and their opinions -- anecdotal evidence -- is now real evidence. How does this magic work? Well, it can't. Two people who said different but similar things, or even the same things under different conditions, can't be lumped together in some statistical box.

But that doesn't mean that anecdotal evidence is worthless. Quite the contrary. It's invaluable (and Microsoft should be commended for actually listening to users). Anecdotal evidence provides you with something that hard data can't -- feelings. In a typical usability test or, these days, a web site A/B test, success is measured by whether or not somebody succeeds in a task. How about whether they frowned or smiled, tapped their fingers on their desk impatiently, hummed to themselves, or cursed at their computer?

I recommend that you gather as much anecdotal evidence as you can and let it infect your world view. Try to feel what your users feel, think like they think, and use that to design your products. And, after that, gather hard evidence on whether you were right or wrong and move forward from there. But, if you don't start with a feeling about what's the right thing to do, no amount of hard evidence will help you.

Sunday, October 25, 2009

Can UX Swing an Election?

In 2000, the infamous "Butterfly Ballot" probably threw the election to George W. Bush, costing Americans more than a trillion dollars and who knows what else. Can it happen again? Unfortunately, the answer is yes. Right now, in King County, Washington, an abysmally designed ballot runs the risk of driving the state into the ground. The problem has two parts.

First, the State of Washington is unfortunate enough to have an idiot named Tim Eyman who makes his living on proposing initiatives. He doesn't do anything else. Initially, he denied that he was making money off the initiatives that he proposed, but enough information was made public that he had to admit it. When he proposed his first initiative, he apparently sold watches, but I think that business is long gone. His first initiative was a "$30 car tab initiative." Eyman picked that number out of thin air (literally, it came from Colorado, with no relationship to the cost of anything in Washington). Some of the people who voted for it who promptly lost their jobs after the election because of all the programs that were slashed, but they didn't see the connection. And Eyman, who said repeatedly that the initiative wasn't about him, went out and bought an expensive SUV right after the election. Amazingly, Eyman managed to garner a following of people who'll vote for anything that will supposedly cut their taxes without regard to what it actually means. This year, Eyman's initiative, I-1033, is one more poorly thought-out proposal. Pick the worst state budget in ten years, in the midst of a recession, and force the state to stick with that budget, essentially, forever.

So what's this have to do with UX? Well, Eyman is only thinking about himself. What provocative initiative can he propose to guarantee that he's still paid a salary? He doesn't care about the experience of the average Washingtonian. I don't even think he cares if the initiative passes. If it fails, he's got another one to propose to pay him through the next election. This is no way to design a government that works for all the people. With good design, you need to think about the whole range of people that will be affected -- and, funny thing about it, that's what our whole legislative process is designed for. Gadflies and devil's advocates can be extremely useful and have a long, rich history in this country. If Tim Eyman actually cared about the state, he would work with our government rather than against it, but, alas, there's no profit in that.

On to the second issue.

King County royally screwed up the ballot, literally marginalizing I-1033. Take a look:

The first column of the ballot contains the instructions that nobody reads, and the top of the second column looks like the start of the ballot, not the middle. This means that some King County voters, perhaps many, will start in the second column and miss voting in I-1033.

The naysayers argue that it's trivial and that, even it's not, it doesn't matter because both people voting for and against the initiative will miss it. On the trivial argument, go back to the top and think about the Butterfly Ballot and what an effect it had. On the second argument, that would be valid if every ballot in the state had the exact same problem, but only King County has the problem. And it is not the case that voters in King County and the rest of the state vote for and against initiatives in approximately equal percentages. In fact, King County voters are more likely to vote against Eyman's initiatives. Combine that with the fact that King County is the most populous county in the state and you definitely have the possibility of a fraudulent election. Despite a large push by the No On 1033 campaign to alert voters to the problem, I am sure that we will see significantly fewer people voting on I-1033 than on other ballot items.

In summary, design really matters and bad design hurts. The only question here is whether the bad design will be big enough to swing the election.

Tuesday, October 13, 2009

Does Tully's Want Customers?

Suppose you want to go to Tully's. You know how this works. You just go to and type in your zip code, perhaps your address, and you'll get a list of nearby Tully's. Or at least that's how it's supposed to work. Give it a try.

Can't figure it out? Neither could I. It turns out that there is an entirely separate web site, for that purpose. And does not contain a link anywhere to that site. Do they want customers?

This is a surprisingly common problem. Usually, it's just sites trying to be too subtle, favoring conforming to some style sheet over actually being usable. Look at, for example, where Store Locator is mixed in with the non-parallel Weekly Ad, Outlet Center, Services, and Gifts. or, where it's even smaller and in an even bigger list.

But Tully's has taken it to a new level by removing the link completely!

Update: Yes, Tully's does want customers. Read what happened.

Sunday, October 11, 2009

Does T-Mobile Want To Steal My Identity?

It can be hard to tell real companies from scam artists sometimes. I got a call the other day from T-Mobile about my bill. They had overcharged me by $24 and I was late paying the bill because I wanted it fixed (and, in this economy, they call the day after it's due!). The discussion of why they would possibly think I wanted text messaging turned off on my account when I switched from a BlackBerry to a MyTouch is a topic for another post in the future

The T-Mobile agent who called me asked for part of my social security number to verify that I was who I said I was. I refused. Hey, you called me! How do I know you're not a scam artist? He told me that he was from T-Mobile and I should believe him, that, if I didn't give him my social security number, he couldn't help me. All things a scam artist would say, of course. The fact remains that I had no proof he was who he said he was.

I tried to explain to the guy that T-Mobile should never, ever ask a question like that because, to the extent that people answer it, you're training them that it's OK to give your confidential information to somebody who calls you on the phone. You're enabling scam artists. Unfortunately, he just didn't get it.

The rules are simple. In the world of client-server architecture, it's known as "never trust the client". In the real world, it's "never trust somebody who calls you."
  • Never, ever give confidential information to somebody who calls you, even an innocuous thing like an account number. You don't know that they are who they say they are.
  • If you call somebody, never, ever ask for confidential information when you call somebody. If you need confidential information, ask them to call you back at a number which is posted prominently on your web site or which is well known (like 1-800-T-MOBILE).

Saturday, October 10, 2009

Is Comcast Helping Scammers?

Comcast wants to fight scammers, but they're inadvertently going to help them.

Comcast, like all Internet service providers, is directly impacted by so-called botnets, machines that have been hijacked by viruses and other malware to serve as robots in the service of scammers. The botnets are useful to the scammers because it allows them to send spam and launch attacks from many locations instead of a single location, which makes them much harder to catch and shut down.

Comcast's idea is to inflict popup ads on their customers that appear to be compromised. which provide them with information. According the the AP article, the ad says "Comcast has detected that there may be a virus on your computer(s). For information on how to clean your computer(s), please visit the Comcast Anti-Virus Center."

There are a couple of problems with this:

  • To the extent that it works, it trains people that popup ads that claim to be helping you clean your computer are legitimate. The problem is that, with this sole exception, none of them are.
  • It trains people that clicking on a link in an unexpected popup ad is an ok thing to do, when it almost never is.
  • It trains people that something like this can be trusted, when it's very easy to fake it.
I don't like the popup in any event, but, if they're going to do it, I think there are a couple of things they must do:
  • The popup shouldn't look at all like an ad and it certainly shouldn't mimic any OS feature.
  • The popup should contain no (that's zero) links in it. Just to be clear: None. Instead, the ad should say "... please visit in your browser and click on the xyz link ..." Train people not to click on links like that and train people that the only way to know for sure that they're actually on the comcast site is to go to comcast themselves, not to trust a link.
  • The popup should not have any button in it. No close button. Nothing to click on at all. Just "Close this window after you've read it." Don't train people to click buttons in unexpected popups.
And how about thinking if there's a better way to attack the whole problem, like doing something in concert with Microsoft and Apple (OS vendors), or Microsoft, Mozilla, and Google (browser vendors).