Creating experiments to test hypotheses – the Lean Startup

Lean startup course, lecture 3 notes (see 1st post)

This 3rd lecture is with the CEO of KISSmetrics, Hiten Shah (@hnshah)

Through our course of trying to develop products we hit on one that worked, that was luck, it was one of 20

What is an experiment: it’s an educated guess. You have an idea of what you’d like the outcome to be and instead of forcing that to be you run a series of experiments to see if you right or not

Before creating a new feature, display a mockup of that content and ask what users feel about it. You’ll learn what people need and you’ll save time not developing what people don’t want or won’t use

A/B testing: you’re going to sell dog collars online. You create 2 landing pages with 2 different value propositions (A: your dog will itch less; B: your dog will choke less). Everything else is the same except the value proposition. Then, measure which one works better

Why use A/B testing instead of just asking: people will usually lie to you, even without knowing: they’ll tell you what you want to hear, or what they think is true

It’s not about being statistical significant. You’re not Amazon yet. It’s just about getting more grounds to make your decision

When you start doing (testing) more radical things is when you get a winner. The wider the difference between your different scenarios, the more likely you have a winner

A winner is something disproportionally better than all the rest of the variations

About developing a new product “mobile local social photo” product: go talk to college students (demographics), see what similar apps they’re using, actually seeing them using those apps, the way they describe it, they say it. Understand how people use those apps, their phone. Get into the head of the user to be able to build something they are likely to use. All of this even before building the product

You want to know all the similar apps they’re using, how they’re using and find something they wish, or they need, across 30 people. That’s when you struck gold

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.