opportunity__cost

At Yelp, we push new code live every day. Pushing daily allows us to quickly prototype new features and squash bugs in a proactive manner. Because we aim to deploy new code so often, we're always looking for ways to make the process efficient and painless.

There are four main stages to the Yelp push process: code review, integration, testing, and finally, live deployment. Each step is important, and there are ways to maximize the efficiency of all of them.

An interesting and detailed guide on how does Yelp do releases.

Why UserTesting.com?

1. $29 introductory price
2. Instantly tap into our 24x7 panel of users
3. Users match your target demographics
4. Observe users in their natural environment
5. Watch screen activity
6. Listen to the users' voices
7. Ask users follow-up questions
8. Annotate and share results
9. Export to QuickTime and .wmv
10. 1-year money-back guarantee

Well, after being recently featured in the TC, UserTesting.com is coming up with a panel of users, who'd be willing to user-test your site, record the video and talk-talk-talk about it. And this is all for a silly introductory price of... $29 per head.

Let me paraphrase it. You are ask some weird guy (or gal - you can choose the gender), within some age bracket (you can choose two or three age brackets), from some country (it can be as wide as "United States"), with some household income (definitely, there is some income - he needs to pay for his Internet connection, unless he's stealing his neighbor's wireless) to open your site, do *something* (if you're good enough in delivering your thoughts to the person in writing, you have fair chances to have it right) and than tell you about it. And this is all for... $29. Oh, don't forget that the guy will have "roughly 15 minutes" to complete the task (don't be silly to say "tasks"). It's... like... $116/hour. *I'm not paid that much.* Even close.

And the normal price is $39 per head. They do have some ideas or plans about revenue sharing (or they probably do this already), and it looks like $10 goes to a user, $29 stays in the pocket. It's sorta $30/hour for a user (decent paycheck for somebody sitting at home and just browsing sites, don't ya think?), and the company keeps the cream.

I mean, I don't want to rant or trash the idea. It's good, there is a need for it and it's probably useful. But come on. That's a decent cost. For such a premium I can bring a perspective client to the office, ask him to do the same shit, run a show for him, do the freaking sales pitch and he will still be smiling!

The more urgent, ‘take control’ tone of Version B’s copy lifted add-to-cart button clicks a whopping 93% over Version A’s comfort-focused tone.

One more interesting A/B test that only with a minor copy change (featuring a major tone difference though) brought up actions by 93% over the regular marketing shit.

Good to remember.

In the past few years, there has been massive growth in new and exciting cheap or free web site usability testing tools, so here’s my list of 24 tools you may need to use from time to time.

Gone are the days of using expensive recruitment firms, labs and massive amounts of time to create, deploy and report on usability tests.

By using these usability testing tools and others like them, you have for the first time a complete set of tools designed to tackle almost any usability research job.

From recruiting real users (with tools such as Ethnio) to conducting live one on one remote moderated tests (UserVue) to analyzing results of usability changes using A/B testing (Google Website Optimizer), there is a plethora of useful and usable tools to conduct usability testing.

Great and the most detailed review of usability testing tools. Craig saves you a plenty of time instead of evaluating these tools yourself.

On the other note, I've run an experiment on using Amazon Mechanical Turk for conducting UX tests. As I thought, it didn't real work out. About 80% of "robots" went the easy way and just clicked on the first selection available. And although they claimed to spend from 4 to 15 minutes on test (each), the control response points averaged less than 3 minutes for a test.

Few interesting facts. They did click on the first available option while running through the navigation-path test, however, they provided a very mixed responses for a final survey screen (one question with five answers pre-set). It looks like they felt like it'll be a good indicator of them bringing some value back.

The other thing, those ones whom I rejected from the final run because of too little of time spent or abandoning the survey (but claiming a completion), the good portion of them actually reached out to me by email fighting back. I guess they spent at least as much effort as was required for completing a survey just to bitch on me. :)

Take care.

“Online Training” wording increased clickthroughs from the support page to the webinar page 10.4%. More importantly, webinar attendees doubled the first week the winning navigation link went live.

MailChimp run a simple test by changing Webinars to Online Training. My first idea was that Online Training would convert better. And, as you can see, I was right.

Why? I can't tell you exactly, but I have a strong feeling that webinars are often attributed to negative or neutral experience, while training (either online or offline) is good.

Figuring out your funnels is one of the most important things you can do to increase your quantitative understanding of your website. It's critical to get the starting measurements - the dropoff and conversion rates - before you change anything.  That's the only way you can know the effect of the changes you make. By constantly tweaking and measuring, you should be able to really improve your number of conversions.

Mixpanel has a set of functions that can be really useful for a company in the Web. While I still had no chance to compare their functionality to what Google Analytics provides, other than being real-time, I'm planning to spend some more time playing with it.

Mixpanel has a free 10K "data points" account (http://mixpanel.com/pricing) which is sorta confusing for a person like me. I'm more like thinking in visits, page views or things like that. But thanks to Tim Trefren from Mixpanel, who explained this to me:

"A data point is counted every time you track a visitor action. So, in terms of pageviews/month - if you track one event per page, you will use exactly the same number of data points as pageviews."

And, well, Mixpanel wouldn't be something you'll use to put on every page (like you can do with Google Analytics). You'd want to put it only on subset of them, linking to specific actions. Or you will probably use Mixpanel's API to assign "data points" to some triggers in your Web application' actions. This is, by the way, could be also a pretty useful feature.

Google Website Optimizer, a powerful tool that allows website owners to split traffic and test the effectiveness and conversion rates for an array of variables, has traditionally required a lot of back-and-forth between any given site and the Website Optimizer interface.

Google releases an API and is allowing site owners to conduct multivariate and A/B testing from their own platform. I'm not sure how well does it compare to Mixpanel or similar products, but I'm planning to find some time for it.