Usability testing

Why UserTesting.com?

1. $29 introductory price
2. Instantly tap into our 24x7 panel of users
3. Users match your target demographics
4. Observe users in their natural environment
5. Watch screen activity
6. Listen to the users' voices
7. Ask users follow-up questions
8. Annotate and share results
9. Export to QuickTime and .wmv
10. 1-year money-back guarantee

via usertesting.com

Well, after being recently featured in the TC, UserTesting.com is coming up with a panel of users, who'd be willing to user-test your site, record the video and talk-talk-talk about it. And this is all for a silly introductory price of... $29 per head.

Let me paraphrase it. You are ask some weird guy (or gal - you can choose the gender), within some age bracket (you can choose two or three age brackets), from some country (it can be as wide as "United States"), with some household income (definitely, there is some income - he needs to pay for his Internet connection, unless he's stealing his neighbor's wireless) to open your site, do *something* (if you're good enough in delivering your thoughts to the person in writing, you have fair chances to have it right) and than tell you about it. And this is all for... $29. Oh, don't forget that the guy will have "roughly 15 minutes" to complete the task (don't be silly to say "tasks"). It's... like... $116/hour. *I'm not paid that much.* Even close.

And the normal price is $39 per head. They do have some ideas or plans about revenue sharing (or they probably do this already), and it looks like $10 goes to a user, $29 stays in the pocket. It's sorta $30/hour for a user (decent paycheck for somebody sitting at home and just browsing sites, don't ya think?), and the company keeps the cream.

I mean, I don't want to rant or trash the idea. It's good, there is a need for it and it's probably useful. But come on. That's a decent cost. For such a premium I can bring a perspective client to the office, ask him to do the same shit, run a show for him, do the freaking sales pitch and he will still be smiling!

# Posted via web from opportunity__cost

Email subject A/B test

Email Subject Line Test: Which Otherwise Identical Message Got More Clicks (Not Opens) – Dash vs. Colon?

VERSION A	VS.	VERSION B

via whichtestwon.com

Very nice experiment.

My gut told me version A is a better winner, as dash works as "call for action". And I was sorta right.

Version A made it by almost 15% over the version B in clicks, however, version B received more open by more than 12%. I don't have a good explanation for that, but that's an interesting thing to come by.

# Posted via web from opportunity__cost

Web Usability Testing Tools

In the past few years, there has been massive growth in new and exciting cheap or free web site usability testing tools, so here’s my list of 24 tools you may need to use from time to time.

Gone are the days of using expensive recruitment firms, labs and massive amounts of time to create, deploy and report on usability tests.

By using these usability testing tools and others like them, you have for the first time a complete set of tools designed to tackle almost any usability research job.

From recruiting real users (with tools such as Ethnio) to conducting live one on one remote moderated tests (UserVue) to analyzing results of usability changes using A/B testing (Google Website Optimizer), there is a plethora of useful and usable tools to conduct usability testing.

via usefulusability.com

Great and the most detailed review of usability testing tools. Craig saves you a plenty of time instead of evaluating these tools yourself.

On the other note, I've run an experiment on using Amazon Mechanical Turk for conducting UX tests. As I thought, it didn't real work out. About 80% of "robots" went the easy way and just clicked on the first selection available. And although they claimed to spend from 4 to 15 minutes on test (each), the control response points averaged less than 3 minutes for a test.

Few interesting facts. They did click on the first available option while running through the navigation-path test, however, they provided a very mixed responses for a final survey screen (one question with five answers pre-set). It looks like they felt like it'll be a good indicator of them bringing some value back.

The other thing, those ones whom I rejected from the final run because of too little of time spent or abandoning the survey (but claiming a completion), the good portion of them actually reached out to me by email fighting back. I guess they spent at least as much effort as was required for completing a survey just to bitch on me. :)

Take care.

# Posted via web from opportunity__cost

Trying out post.ly... I still not sure what does it mean, but will see is a moment.

# Posted via web from opportunity__cost

A/B to increase user engagement

The question becomes 'do you want a smaller, highly engaged community, or a larger, less engaged community?' There's no easy answer here; it's more of a think-long-and-hard sort of situation that greatly depends on individual aspects of your startup.

via blog.mixpanel.com

Great blog post on using A/B testing to increase user engagement and some conclusions made.

# Posted via web from opportunity__cost

Some Ruby stuff

This morning I've got a questions from a friend and really couldn't give him a good answer. It's about a rather simple Ruby structure.

module Test
 def self.included(base)
  class << base
   def tester
    puts 'some message' + "\n"
   end
  end
 end
end

class Test2
 include Test

 tester
end

Speaking about the following block:

 def self.included(base)
  class << base
   def tester
    puts 'some message' + "\n"
   end
  end
 end

So what does really happen here? We're overriding the self.included which is called whenever Test module is included by other class or module (Test2) and Test2 is passed as the single argument to it.

Moving forward, whenever we're getting a callback to self.included, we're extending the class (Test2) with tester method which does something.

What I was wondering about is why tester is called there. Because if we miss including Test there we'll get NameError: undefined local variable or method `tester' for Test2:Class exception.

So basically, my real question is what make sense of using such a structure of the code? I bet the example I was given is not complete enough to understand this.

What do you think?

GMAT CR

Okay, so I missed a day of study (was too sick for that, and I didn't think going over the questions in that condition can make any good for me) and next day (instead of problem solving) I had a few hours preview class on Manhattan GMAT. They're telling people about what GMAT is and were pitching their courses. I didn't really solve a lot of questions that day, but got a pretty good intro to the actual exam process, and what computer adaptive tests are. I never had CAT before so it's going to be a pretty new experience for me.

Well, as for now, I just turned over to GMAT part which is called Critical Reasoning (CR), and (from the first sight) it felt to me pretty easy. I was able to score 80% without even paying enough attention and being constantly distracted by my little son, whom I've got to babysit for whole weekend. (And you know, he knows how to distract you from studying.) The most of my mistakes were pretty careless. (Not an excuse though.) And I completed only 40 questions today. I'll try to make at least 50 or 60 tomorrow.

My PMP exam

I don't know why I didn't blog about it before, so here is the story:

I passed the exam with success about a year ago (March 2008). The preparation process took me about 2 to 3 months. I can't say that it was very intensive, but I had a pretty solid plan and schedule and invested not less then an hour every day (with some very rare exceptions).

The whole exam-time starting from entering testing facilities and finishing with printed and stamped results took me around 2.15 hrs. PMI doesn't disclose the scores now (at least they did not that time), but gives you rating (below proficient, moderate proficient and proficient) for each of knowledge areas (plus professional responsibility). I've screwed up on one, everything else was rated as proficient.

Speaking about the preparation process. I've used three books, Andy Crowe's one, PMP Head First, and Rita's. I also have used a bunch of notes and spent a lot of time on solving exams. I believe I've solved more then 1,000 questions, plus the ones which come with every book. I didn't like Rita's book at all. This is no way one can use it to learn something. She is bringing complexities to the things that should be easy and intuitive to any manager. I didn't like her practice questions as well. She make them extremely weird and they are far from what you'll get.

PMP Head First book is very nice to read and understand. I believe that this is a best starters book, but you'll need something to add to it for details. This one book might be not enough. The questions which are in the book are simple, the "last exam" questions which come as download are very good.

Crowe's book is the best one to use with PMP Head First. The questions are good. The material is good. The way of depicting information is good. Overall, this is the best book. The questions are good for concept, but not complex enough to rely for passing an exam.

As for the sample questions, the best are PM Studies and Oliver Lehmann's. The downloadable PMP Head First are also very good. Andy Crowe book questions are good as well.

No surprises, although I had a few questions with English words I wasn't aware of. They were tough even to guess. English is not my native language and I speak three languages actually (Russian, Ukrainian and English). I didn't placed a request for translation (they gave me this option during the exam scheduling) because the translations will use different terminology and they are more confusing then helpful.

As far as I remember I didn't have any network diagram question at all. But I had a bunch of EV calculations. A lot of people are receiving NPV v. EPV question nowadays. I did, too.

No surprises at the test site. I was passing in Glendale, CA (LA county). They have a lock box for you, but they are very strict and force to take off even watches. They gave a simple calculator, although the program has a Calculator button on screen and you can use it as well. The class is not that noisy, but it really interrupts you when somebody new is coming into the class to take a seat. They have a video and audio recording of that quality that they hear even when you start writing on the scratch paper.

Basically, that's all I can remember. ;) Feel free to ask me more questions, but I can't disclose the actual questions (and no PMP should).